Become an AI engineer

GPU programming

GPU programming is a unit of 16 lessons in Rabbithole's Become an AI engineer course, at levels 4 to 5.

Lessons

1. CUDA cluster launch control: cancelling thread blocks safelyLevel 4
2. Compute Fabric Transport: Reaching a Peer by Endpoint ID and OffsetLevel 4
3. How a GPU runs your threads: warps, independent scheduling, clusters and ABILevel 4
4. Matrix multiplication on acceleratorsLevel 4
5. GPU memory hierarchy: what sits where, and what it limitsLevel 4
6. The roofline model: math-limited or memory-limitedLevel 4
7. TPUs: the chip, its vector unit, and the numbers you can compute from itLevel 4
8. Unified memory: virtual addresses and access-counter migrationLevel 4
9. CUDA: architecture targets, memory ordering and what the device runtime does not promiseLevel 5
10. CUDA Tile: programming the GPU a tile at a timeLevel 5
11. CuTe layouts: hierarchical coordinates and generalized stridesLevel 5
12. FpSan: checking kernels with payload algebraLevel 5
13. Formal Kernel Verification: Proving a Kernel CorrectLevel 5
14. Kernel fusion: one kernel, fewer trips to memoryLevel 5
15. TMA: Bulk Copies Between Global and Shared MemoryLevel 5
16. Triton: Block-Based Compilation for GPU KernelsLevel 5

Sources

These lessons draw on 10 sources. The most used:

• CUDA Programming Guide
• Triton documentation
• Making Deep Learning Go Brrrr From First Principles
• GPU MODE lectures
• How to Scale Your Model
• NVIDIA Deep Learning Performance
• vLLM documentation
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

All units in Become an AI engineer