Humanity's Last Exam (HLE)

In depth

A multimodal benchmark of 2,500 expert-written questions across dozens of subjects, including mathematics, the humanities, and the natural sciences. It was built because models had passed 90% on older tests like MMLU. Every question has one unambiguous, auto-gradable answer (multiple choice or short answer) that cannot be found with a quick lookup. Labs often report two scores, with and without tool use, so check which one a launch post is quoting.

Primary source: arxiv.org