Mixture of Agents

In depth

An ensemble technique where several LLMs answer the same prompt in layers. Each agent in a layer gets every output from the previous layer as extra context before writing its own response. Wang et al. introduced the approach in 2024 (arXiv 2406.04692), and their version built only from open-source models scored 65.1% on AlpacaEval 2.0, against 57.5% for GPT-4o. Each request costs many model calls, so it suits outputs where quality matters most, not high-volume traffic.

Primary source: arxiv.org