Home

Disclaimer: This is usable but still in active development and not yet in the ideal form I'd like it to be in. All sentences that begin with "TODO" are comments to myself.

I made this roadmap to help guide AI self-learning at Hack Club Colts Neck. It's a truncated and streamlined version of my journey self-studying in introductory deep learning material over the past few months. I've broken everything that I've learned down into a list of crucial topics with resources to learn from, problems to work on, and questions that probe at the knowledge I think you should take away.

I'm still very inexperienced myself. A good heuristic for thinking about this resource quality-wise: on par with what you would get if you asked a friend who had just finished a Psychology 101 course for a list of materials and tips on studying Psychology yourself.

If (when) you find questions and tips that may be misleading; you can start issues here or contact me directly using whatever method works here. Read through my guide before getting started, and use that to create a system that works for you. I can guarantee that spending a couple hours on improving your learning process in advance will have a massive return on investment that will carry over into anything you decide to self-learn in the future.

TODO

• Things to do personally before adding to the roadmap:

  • Lin alg., calculus, probability practice

    • Be able to answer all questions on these from the prereqs page from week 0 of Arena (I felt on top of the world as a teenager who "self-taught" very surface-level calc, linear algebra, and probability. Then I tried doing the questions and went asdfasdfasdfasf.)

    • Add practice materials to here

  • RL Review

    • https://spinningup.openai.com/en/latest/spinningup/rl_intro.html

    • https://lilianweng.github.io/posts/2018-02-19-rl-overview/

    • https://youtu.be/JgvyzIkgxF0 -> if good, add to the roadmap

    • https://karpathy.github.io/2016/05/31/rl/ -> policy gradient

  • Karpathy Tokenization video

  • Cool Projects

    • RL Bonus Exercises

      • RLHF

        • learning a human preference reward model

        • All the suggested paper replications (3, 1, 2 in order of interestingness to me)

        • Differential Learning Rates

      • PPO

        • Wordle

        • Other discrete environments

    • Finish doing the Karpathy exercises

    • Autodiff in C++ -> as much functionality as this

    • Numpy-NN, inspired by Elijah

• Complete all TODOs

  • Linear Algebra

  • Calculus

  • Probability

  • Grokking the Fundamentals (the links below are only visible on my end)

    • Neural Networks Basics

    • Grokking the Fundamentals

    • Grokking the Fundamentals

• Sanity check all questions through ChatGPT/Claude/Phind