Learn to use AI

# Does it work?

Does it work? is a unit of 8 lessons in Rabbithole's Learn to use AI course, at level 2.

## Lessons

1. Benchmarks: how AI models get tested · Level 2
2. Evaluation: how you tell whether an AI is any good · Level 2
3. Testing an AI agent: what evals are and why to start early · Level 2
4. NLP tasks: the jobs we give language models · Level 2
5. How AI reasoning is tested · Level 2
6. Robustness evaluation: does the AI hold up when things change? · Level 2
7. Selection Bias in Benchmarks · Level 2
8. Contamination: when a model has already seen the test · Level 2

## Sources

These lessons draw on 21 sources. The most used:

- [Holistic Evaluation of Language Models](https://arxiv.org/abs/2211.09110)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://arxiv.org/abs/2206.04615)
- [System Card: Claude Opus 4.6](https://www-cdn.anthropic.com/6a5fa276ac68b9aeb0c8b6af5fa36326e0e166dd/Claude%20Opus%204.6%20System%20Card.pdf)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://arxiv.org/abs/2403.04132)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://arxiv.org/abs/2311.12022)
- [Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)
- [ARC-AGI](https://arcprize.org/arc-agi)
- [Claude Docs — Test and Evaluate](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests)

[All units in Learn to use AI](https://rabbithole.pro/courses/learn-ai)

Source page: https://rabbithole.pro/courses/learn-ai/does-it-work