Benchmark Contamination

In depth

Benchmark contamination happens when a model's training data contains the questions or answers from an evaluation benchmark. The score then partly measures memorization instead of real capability. It is usually accidental, because benchmarks published online end up in web-scale pretraining data. Common defenses are private held-out test sets, benchmarks built from problems published after a model's training cutoff, and checks for overlap between training data and test items.

Primary source: arxiv.org