Eval / Evaluation

In depth

The systematic process of testing an AI model's performance against a defined set of inputs and expected outputs. Evals measure whether a model is actually good at the task you care about, not just benchmarks. They can be automated (comparing outputs to ground truth) or human-judged (rating quality on a rubric). Running evals before and after changes is how teams catch regressions and validate improvements.