Become an AI engineer

# GPU programming

GPU programming is a unit of 16 lessons in Rabbithole's Become an AI engineer course, at levels 4 to 5.

## Lessons

1. CUDA cluster launch control: cancelling thread blocks safely · Level 4
2. Compute Fabric Transport: Reaching a Peer by Endpoint ID and Offset · Level 4
3. How a GPU runs your threads: warps, independent scheduling, clusters and ABI · Level 4
4. Matrix multiplication on accelerators · Level 4
5. GPU memory hierarchy: what sits where, and what it limits · Level 4
6. The roofline model: math-limited or memory-limited · Level 4
7. TPUs: the chip, its vector unit, and the numbers you can compute from it · Level 4
8. Unified memory: virtual addresses and access-counter migration · Level 4
9. CUDA: architecture targets, memory ordering and what the device runtime does not promise · Level 5
10. CUDA Tile: programming the GPU a tile at a time · Level 5
11. CuTe layouts: hierarchical coordinates and generalized strides · Level 5
12. FpSan: checking kernels with payload algebra · Level 5
13. Formal Kernel Verification: Proving a Kernel Correct · Level 5
14. Kernel fusion: one kernel, fewer trips to memory · Level 5
15. TMA: Bulk Copies Between Global and Shared Memory · Level 5
16. Triton: Block-Based Compilation for GPU Kernels · Level 5

## Sources

These lessons draw on 10 sources. The most used:

- [CUDA Programming Guide](https://docs.nvidia.com/cuda/cuda-programming-guide/)
- [Triton documentation](https://triton-lang.org/main/index.html)
- [Making Deep Learning Go Brrrr From First Principles](https://horace.io/brrr_intro.html)
- [GPU MODE lectures](https://www.youtube.com/@GPUMODE/videos)
- [How to Scale Your Model](https://jax-ml.github.io/scaling-book/)
- [NVIDIA Deep Learning Performance](https://docs.nvidia.com/deeplearning/performance/index.html)
- [vLLM documentation](https://docs.vllm.ai/en/latest/)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://arxiv.org/abs/2306.00978)

[All units in Become an AI engineer](https://rabbithole.pro/courses/ai-engineer)

Source page: https://rabbithole.pro/courses/ai-engineer/gpu-programming