Evals & Safety
The American Invitational Mathematics Examination, a 15-question high school math competition run by the Mathematical Association of America that AI labs use as a math reasoning benchmark.
The American Invitational Mathematics Examination, a 15-question high school math competition run by the Mathematical Association of America that AI labs use as a math reasoning benchmark. Every answer is an integer from 0 to 999, so grading is exact match and needs no judge model. Labs report results on a specific year's exam, such as AIME 2025, because a newer exam is less likely to be in training data. With only 15 questions, one problem moves the score by almost 7 points, so small gaps between models are noise.
Primary source: maa.org
Example
Labs report results on a specific year's exam, such as AIME 2025, because a newer exam is less likely to be in training data.
Hands-on guides, comparisons, and tutorials that cover Evals & Safety.
FAQ
The American Invitational Mathematics Examination, a 15-question high school math competition run by the Mathematical Association of America that AI labs use as a math reasoning benchmark.
AIME (Benchmark) sits in the Evals & Safety part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Developers Digest publishes tutorials and videos that cover Evals & Safety topics including AIME (Benchmark). Check the blog and YouTube channel for hands-on walkthroughs.
Related
A factuality benchmark from OpenAI that tests whether a model can answer short, fact-seeking questions that each have one indisputable answer.
Benchmark contamination happens when a model's training data contains the questions or answers from an evaluation benchmark.
A multimodal benchmark of 2,500 expert-written questions across dozens of subjects, including mathematics, the humanities, and the natural sciences.

New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.