What are AI evals, and what are they for?
A practical introduction to testing AI systems, comparing changes, and learning from failures.
Published
An eval is a test of an AI system's behavior. You give the system an input, examine its output, and judge whether it did the job well enough. A single test might ask a model to answer a question from a document. An evaluation usually runs many such tests so you can see patterns instead of relying on one convincing example.
The system under test can be more than a model. It might include a prompt, retrieved documents, tools, and application code. An eval measures the behavior of the setup you actually run.
What goes into an eval?
Most evals need three things:
- Examples: Inputs that represent the work the system should do. These can be real cases, carefully written cases, or questions from a public benchmark.
- A way to judge the outputs: For a multiple-choice question, that may be an exact answer check. For a support reply, it may be a human reviewer or a rubric that checks accuracy, completeness, and tone.
- A result you can inspect: A summary score helps you compare runs, while the individual outputs show why a run improved or failed.
Suppose you are building an assistant that answers questions about your product documentation. You could collect 100 representative questions, record the facts a good answer must include, and run the assistant on all 100. When you change its prompt or retrieval settings, run the same questions again. If the score rises but answers to billing questions get worse, the individual results reveal a problem that the average hides.
What are evals for?
Evals help you make decisions about changes. They can show whether a new prompt fixes a known failure, whether a cheaper model performs well enough for your task, or whether a product update has broken behavior that used to work. Running them repeatedly makes regressions easier to catch before users do.
They also help you learn what to improve. Grouping failures by topic, difficulty, or user need can tell you whether the next step is better source material, a different prompt, a tool change, or a model change. A score alone rarely tells you which one.
What can a score tell you?
A score describes performance on a particular set of examples under a particular setup. It is useful when the examples resemble your real work and the grading method checks what you care about. A public benchmark can help compare broad capabilities, but a high benchmark score does not guarantee that your application will handle its own users well.
Be especially careful when comparing results from different prompts, subsets, or graders. Two runs may have the same score while making different mistakes. Review sample outputs, keep the test conditions consistent, and track the kinds of failures that matter most.
Where should you start?
Start with a small set of real tasks and failures you already know about. Write down what success looks like for each one, run your current system, and inspect the misses. Add examples as you discover new failure modes. When you want broader coverage, explore the benchmark library and read each benchmark's scoring rules and caveats before using its results.