Rabbithole

Become an AI engineer

Foundations

Foundations is a unit of 48 lessons in Rabbithole's Become an AI engineer course, at levels 3 to 5.

Lessons

  1. Machine learning: how it works and how to build itLevel 3
  2. Neural networks: nodes, layers and the cost of building themLevel 3
  3. Deep learning: stacked layers that learn their own featuresLevel 3
  4. Embeddings: vectors, skip-gram and joint spacesLevel 3
  5. GPUs: how they run, how they fail, how to read themLevel 3
  6. Machine translation: from encoder-decoders to the translation pipelineLevel 3
  7. Overfitting and generalisationLevel 3
  8. Supervised learning: training on labeled examplesLevel 3
  9. Unsupervised learning: patterns without labelsLevel 3
  10. Activation functions: why networks need them and why ReLU wonLevel 3
  11. Chat Templates: Formatting Conversations for a ModelLevel 3
  12. Gradient descent: steps, mini-batches and keeping training healthyLevel 3
  13. Learning Curves: Reading a Training RunLevel 3
  14. Loss function: one number to make smallLevel 3
  15. Explaining a Black-Box Model: From Pairwise Slices to LIMELevel 3
  16. Object Detection: From Random Boxes to Learned PipelinesLevel 3
  17. Reinforcement learning: policies, exploration and GRPOLevel 3
  18. Softmax: from raw scores to probabilitiesLevel 3
  19. Active Learning: Choosing What Gets LabeledLevel 4
  20. Adam: an adaptive learning rate for every parameterLevel 3
  21. Autoencoders: learning by rebuilding the inputLevel 3
  22. Backpropagation: how gradients flow backwardLevel 3
  23. Convolutional neural networks: sliding filters with shared weightsLevel 3
  24. Curriculum Learning: Easy Examples FirstLevel 4
  25. Data Augmentation: More Training Data From the Data You HaveLevel 3
  26. Domain randomization: how the randomization gets tunedLevel 4
  27. Evolution Strategies: searching by sampling, selecting and refittingLevel 4
  28. Generative Adversarial Networks: Two Networks, One GameLevel 3
  29. Hyperparameters: learning rate, batch size, dropout and searchLevel 3
  30. Meta-Learning: Learning to Adapt FastLevel 4
  31. Normalisation layersLevel 3
  32. Policy Gradient MethodsLevel 4
  33. Prediction-Based ExplorationLevel 4
  34. Recurrent neural networks: memory, cost and the vanishing gradientLevel 3
  35. Regularisation through dropoutLevel 4
  36. Residual connections: adding the input backLevel 4
  37. Semi-Supervised Learning: Mean Teacher and UDALevel 4
  38. Temporal Difference LearningLevel 4
  39. Q-Learning: a table of action valuesLevel 4
  40. Deep Q-Learning: Stabilizing Q-Learning with a Neural NetworkLevel 4
  41. Information Bottleneck: what a network's layers keep and forgetLevel 5
  42. Memory-Augmented Neural Networks: a Controller with a Memory to Read and WriteLevel 4
  43. Muon: orthogonalized updates for the 2-D weightsLevel 4
  44. The Neural Tangent Kernel at Infinite WidthLevel 5
  45. Normalizing Flows: Invertible Building BlocksLevel 4
  46. The Reparameterization TrickLevel 4
  47. Variational AutoencoderLevel 4
  48. Wasserstein Distance: The Cost of Moving Probability MassLevel 4

Sources

These lessons draw on 48 sources. The most used:

All units in Become an AI engineer