Become an AI engineer

# Foundations

Foundations is a unit of 48 lessons in Rabbithole's Become an AI engineer course, at levels 3 to 5.

## Lessons

1. Machine learning: how it works and how to build it · Level 3
2. Neural networks: nodes, layers and the cost of building them · Level 3
3. Deep learning: stacked layers that learn their own features · Level 3
4. Embeddings: vectors, skip-gram and joint spaces · Level 3
5. GPUs: how they run, how they fail, how to read them · Level 3
6. Machine translation: from encoder-decoders to the translation pipeline · Level 3
7. Overfitting and generalisation · Level 3
8. Supervised learning: training on labeled examples · Level 3
9. Unsupervised learning: patterns without labels · Level 3
10. Activation functions: why networks need them and why ReLU won · Level 3
11. Chat Templates: Formatting Conversations for a Model · Level 3
12. Gradient descent: steps, mini-batches and keeping training healthy · Level 3
13. Learning Curves: Reading a Training Run · Level 3
14. Loss function: one number to make small · Level 3
15. Explaining a Black-Box Model: From Pairwise Slices to LIME · Level 3
16. Object Detection: From Random Boxes to Learned Pipelines · Level 3
17. Reinforcement learning: policies, exploration and GRPO · Level 3
18. Softmax: from raw scores to probabilities · Level 3
19. Active Learning: Choosing What Gets Labeled · Level 4
20. Adam: an adaptive learning rate for every parameter · Level 3
21. Autoencoders: learning by rebuilding the input · Level 3
22. Backpropagation: how gradients flow backward · Level 3
23. Convolutional neural networks: sliding filters with shared weights · Level 3
24. Curriculum Learning: Easy Examples First · Level 4
25. Data Augmentation: More Training Data From the Data You Have · Level 3
26. Domain randomization: how the randomization gets tuned · Level 4
27. Evolution Strategies: searching by sampling, selecting and refitting · Level 4
28. Generative Adversarial Networks: Two Networks, One Game · Level 3
29. Hyperparameters: learning rate, batch size, dropout and search · Level 3
30. Meta-Learning: Learning to Adapt Fast · Level 4
31. Normalisation layers · Level 3
32. Policy Gradient Methods · Level 4
33. Prediction-Based Exploration · Level 4
34. Recurrent neural networks: memory, cost and the vanishing gradient · Level 3
35. Regularisation through dropout · Level 4
36. Residual connections: adding the input back · Level 4
37. Semi-Supervised Learning: Mean Teacher and UDA · Level 4
38. Temporal Difference Learning · Level 4
39. Q-Learning: a table of action values · Level 4
40. Deep Q-Learning: Stabilizing Q-Learning with a Neural Network · Level 4
41. Information Bottleneck: what a network's layers keep and forget · Level 5
42. Memory-Augmented Neural Networks: a Controller with a Memory to Read and Write · Level 4
43. Muon: orthogonalized updates for the 2-D weights · Level 4
44. The Neural Tangent Kernel at Infinite Width · Level 5
45. Normalizing Flows: Invertible Building Blocks · Level 4
46. The Reparameterization Trick · Level 4
47. Variational Autoencoder · Level 4
48. Wasserstein Distance: The Cost of Moving Probability Mass · Level 4

## Sources

These lessons draw on 48 sources. The most used:

- [Lil'Log](https://lilianweng.github.io/)
- [Hugging Face LLM Course](https://github.com/huggingface/course)
- [MIT 6.S191: Introduction to Deep Learning (2026)](https://www.youtube.com/playlist?list=PLkkuNyzb8LmxFutYuPA7B4oiMn6cjD6Rs)
- [MIT 6.S191: Introduction to Deep Learning](https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)
- [Understanding Deep Learning](https://github.com/udlbook/udlbook)
- [Deep Learning](https://www.deeplearningbook.org/)
- [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course)
- [Stanford CS224N: Natural Language Processing with Deep Learning, Spring 2024](https://www.youtube.com/playlist?list=PLoROMvodv4rOaMFbaqxPDoLWjDaRAdP9D)

[All units in Become an AI engineer](https://rabbithole.pro/courses/ai-engineer)

Source page: https://rabbithole.pro/courses/ai-engineer/foundations