Verifier

In depth

A model or program that checks whether a candidate output is correct, used to score or filter what a generator produces. OpenAI's 2021 GSM8K paper trained a verifier to rank many sampled math solutions and pick the top one, and found this scaled better with more data than fine-tuning alone. Today verifiers include unit tests, proof checkers, and learned reward models, and they supply the reward signal for reinforcement learning on code and math, so a flawed verifier teaches the model to game it.

Primary source: arxiv.org