Coding

Customize this roadmap based on your prior knowledge, but make sure you have all these skills down well before moving on. Some of the emphasis I place on coding practice might seem unnecessarily extensive, but trust me when I say that you will struggle if you don't do the appropriate practice.

Get very familiar w/ Python

• Roadmap I would recommend

  • Learn Python w/ Kaggle

    • This if you're a complete beginner to programming

      • Then, watch all lectures and solve all questions up to week 6 of CS50

    • This if you already know how to program

    • This for a good refresher after (video refreshers should never be a substitute for actual practice)

  • Practice:

    • from CS50

      • be able to do these and these with relative ease

    • if you need more practice, use Exercism

  • Go through this book and make sure you understand everything up to section 21

• Other resources you can use if you don't like my roadmap

  • MIT's intro to Python

  • Harvard's intro to Python

  • Corey Schafer -> if there's anything in particular that you're confused by, it'll probably be explained really well in here

Stop here for now. I'll tell you to revisit this page once you're more familiar with the basics of neural nets.

Get familiar w/ Numpy

• https://cs231n.github.io/python-numpy-tutorial/

Get familiar w/ Pytorch

• https://pytorch.org/tutorials/beginner/basics/intro.html

• https://pytorch.org/tutorials/beginner/nn_tutorial.html

Practice Numpy and Pytorch

• Implement as many of these exercises as you feel like doing in both Numpy and Pytorch

• Answer these questions (borrowed from https://arena3-chapter0-fundamentals.streamlit.app/[0.0]_Prerequisites):

  • At a high level, what is a torch.Tensor?

  • What is a nn.Parameter, and nn.Module?

  • When you call .backward(), where are your gradients stored?

  • What is a loss function? In general, what does it take for arguments, and what does it return?

  • What does an optimization algorithm do?

  • What is a hyperparameter, and how does it differ from a regular parameter?

  • What are some examples of hyperparameters?

Stop here for now and revisit once more familiar with building a basic neural net.

Getting Better with Tensors

Do a lot of this, and do it well. When you feel like you're done, spend a few more days on it. I still have not become completely comfortable with tensor manipulation and that's been a chink in my foundation that has truly bottlenecked my progress.

• Learn einops and einsum with this: https://arena3-chapter0-fundamentals.streamlit.app/[0.0]_Prerequisites

• https://arena3-chapter0-fundamentals.streamlit.app/[0.1]_Ray_Tracing

• Please do this well. I never built up a strong foundation with tensor manipulations and I still struggle whenever I embark on larger projects because of it.