LMArena

In depth

A public leaderboard that ranks models by crowdsourced human preference: a user submits a prompt, two anonymous models answer, and the user votes for the better response before the model names are revealed. It started in 2023 as Chatbot Arena, built by LMSYS and UC Berkeley researchers, later became LMArena, and now runs as Arena at arena.ai. Votes are combined into ratings with a Bradley-Terry model, so scores reflect preference on real user prompts rather than correctness on a fixed test set. Separate arenas such as WebDev Arena rank models on specific task types.

Primary source: arxiv.org