Become an AI engineer

# LLM security

LLM security is a unit of 11 lessons in Rabbithole's Become an AI engineer course, at levels 3 to 5.

## Lessons

1. Excessive Agency: when an LLM can do too much · Level 3
2. Least privilege for agents · Level 3
3. LLM Supply Chain Security · Level 3
4. OWASP Top 10 for LLM Applications: Output Handling, Unbounded Consumption, and Tool-Calling Risk · Level 3
5. Prompt injection: where the instruction comes from, and how CaMeL blocks the damage · Level 3
6. Sensitive Information Disclosure · Level 3
7. Data exfiltration: how it works and where the SSRF defences go · Level 4
8. Indirect prompt injection: hijacking a model through what it reads · Level 4
9. The Instruction Hierarchy · Level 4
10. Safety classifiers: scoring a response while it streams · Level 4
11. Adversarial attacks on aligned models: GCG and its defenses · Level 5

## Sources

These lessons draw on 19 sources. The most used:

- [OWASP Top 10 for LLM Applications 2025](https://genai.owasp.org/llm-top-10/)
- [Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025)](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2025.pdf)
- [Lessons from Defending Gemini Against Indirect Prompt Injections](https://arxiv.org/abs/2505.14534)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://arxiv.org/abs/2404.13208)
- [Defeating Prompt Injections by Design](https://arxiv.org/abs/2503.18813)
- [OpenAI Codex and ChatGPT desktop documentation (ChatGPT Learn)](https://learn.chatgpt.com/docs/quickstart)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://arxiv.org/abs/2501.18837)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://arxiv.org/abs/2302.12173)

[All units in Become an AI engineer](https://rabbithole.pro/courses/ai-engineer)

Source page: https://rabbithole.pro/courses/ai-engineer/llm-security