Benchmark leaderboard - 25 entries
MLE-bench leaderboard
The MLE-bench dataset is a collection of 75 Kaggle competitions which we use to evaluate the ML engineering capabilities of AI systems. As of October 17, 2026, Famou-Agent 2.0 (Gemini-3-Pro-Preview) ranks first on the MLE-bench board with 64.44 ± 1.18.
Last verified from github.com. Read the caveats
MLE-bench
25 entries, in the board's order
| 1 | Famou-Agent 2.0 (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 80.3 ± 1.52, Medium 64.04 ± 2.32, High 42.22 ± 2.22. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 64.44 ± 1.18 |
| 2 | AIBuildAI (Claude-Opus-4.6) Any Medal (%), All split (mean ± SEM). Low 77.27 ± 0.00, Medium 61.40 ± 0.88, High 46.67 ± 0.00. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 63.11 ± 0.44 |
| 3 | CAIR MARS+ (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 78.79 ± 1.52, Medium 60.53 ± 1.52, High 44.44 ± 2.22. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 62.67 ± 0.77 |
| 4 | MLEvolve (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 80.30 ± 1.52, Medium 57.89 ± 1.52, High 42.22 ± 2.22. Running time 12 hours. Source code available; grading reports available. Official leaderboard | 61.33 ± 1.33 |
| 5 | PiEvolve, Fractal AI Research (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 80.30 ± 1.52, Medium 58.77 ± 0.88, High 40.0 ± 0.00. Running time 24 hours. Footnote 3: computed by padding incomplete seeds with failing scores. Footnote 4: architecture primarily driven by Gemini-3-Pro-Preview, with a subset of modules utilizing GPT-5 and GPT-5-mini. Source code not available; grading reports available. Official leaderboard | 61.33 ± 0.77 |
| 6 | Famou-Agent 2.0 (Gemini-2.5-Pro) Any Medal (%), All split (mean ± SEM). Low 75.76 ± 1.52, Medium 57.89 ± 1.52, High 40.00 ± 0.00. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 59.56 ± 0.89 |
| 7 | ML-Master 2.0 (Deepseek-V3.2-Speciale) Any Medal (%), All split (mean ± SEM). Low 75.76 ± 1.51, Medium 50.88 ± 3.51, High 42.22 ± 2.22. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 56.44 ± 2.47 |
| 8 | CAIR MARS (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 74.24 ± 1.52, Medium 52.63 ± 3.04, High 37.78 ± 2.22. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 56.0 ± 1.54 |
| 9 | PiEvolve, Fractal AI Research (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 74.24 ± 3.03, Medium 45.61 ± 0.88, High 35.55 ± 2.22. Running time 12 hours. Footnote 3: computed by padding incomplete seeds with failing scores. Footnote 4: architecture primarily driven by Gemini-3-Pro-Preview, with a subset of modules utilizing GPT-5 and GPT-5-mini. Source code not available; grading reports available. Official leaderboard | 52.0 ± 0.77 |
| 10 | Leeroo (Gemini-3-Pro-Preview) Any Medal (%), All split (mean ± SEM). Low 68.18 ± 2.62, Medium 44.74 ± 1.52, High 40.00 ± 0.00. Running time 24 hours. Footnote 3: computed by padding incomplete seeds with failing scores. Footnote 4: architecture primarily driven by Gemini-3-Pro-Preview, with a subset of modules utilizing GPT-5 and GPT-5-mini. Source code available; grading reports available. Official leaderboard | 50.67 ± 1.33 |
| 11 | Thesis (gpt-5-codex) Any Medal (%), All split (mean ± SEM). Low 65.15 ± 1.52, Medium 45.61 ± 7.18, High 31.11 ± 2.22. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 48.44 ± 3.64 |
| 12 | CAIR MLE-STAR-Pro-1.5 (Gemini-2.5-Pro) Any Medal (%), All split (mean ± SEM). Low 68.18 ± 2.62, Medium 34.21 ± 1.52, High 33.33 ± 0.00. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 44.00 ± 1.33 |
| 13 | Famou-Agent (Gemini-2.5-Pro) Any Medal (%), All split (mean ± SEM). Low 62.12 ± 1.52, Medium 36.84 ± 1.52, High 33.33 ± 0.00. Running time 24 hours. Source code not available; grading reports available. Official leaderboard | 43.56 ± 0.89 |
| 14 | Operand ensemble (gpt-5, low verbosity/effort) Any Medal (%), All split (mean ± SEM). Low 63.64 ± 0.00, Medium 33.33 ± 0.88, High 20.00 ± 0.00. Running time 24 hours. Footnote 2: with some light assistance from an ensemble of models including Gemini-2.5-Pro, Grok-4, and Claude 4.1 Opus, distilled by Gemini-2.5-Pro. Footnote 3 (Medium, High, All): computed by padding incomplete seeds with failing scores. Source code not available; grading reports available. Official leaderboard | 39.56 ± 0.44 |
| 15 | CAIR MLE-STAR-Pro-1.0 (Gemini-2.5-Pro) Any Medal (%), All split (mean ± SEM). Low 66.67 ± 1.52, Medium 25.44 ± 0.88, High 31.11 ± 2.22. Running time 12 hours. Source code not available; grading reports available. Official leaderboard | 38.67 ± 0.77 |
| 16 | InternAgent (deepseek-r1) Any Medal (%), All split (mean ± SEM). Low 62.12 ± 3.03, Medium 26.32 ± 2.63, High 24.44 ± 2.22. Running time 12 hours. Source code not available; grading reports available. Official leaderboard | 36.44 ± 1.18 |
| 17 | R&D-Agent (gpt-5) Any Medal (%), All split (mean ± SEM). Low 68.18 ± 2.62, Medium 21.05 ± 1.52, High 22.22 ± 2.22. Running time 12 hours. Source code available; grading reports available. Official leaderboard | 35.11 ± 0.44 |
| 18 | Neo multi-agent (undisclosed) Any Medal (%), All split (mean ± SEM). Low 48.48 ± 1.52, Medium 29.82 ± 2.32, High 24.44 ± 2.22. Running time 36 hours. Source code not available; grading reports available. Official leaderboard | 34.22 ± 0.89 |
| 19 | AIRA-dojo (o3) Any Medal (%), All split (mean ± SEM). Low 55.00 ± 1.47, Medium 21.97 ± 1.17, High 21.67 ± 1.07. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 31.60 ± 0.82 |
| 20 | R&D-Agent (o3 + GPT-4.1) Any Medal (%), All split (mean ± SEM). Low 51.52 ± 4.01, Medium 19.30 ± 3.16, High 26.67 ± 0.00. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 30.22 ± 0.89 |
| 21 | ML-Master (deepseek-r1) Any Medal (%), All split (mean ± SEM). Low 48.48 ± 1.52, Medium 20.18 ± 2.32, High 24.44 ± 2.22. Running time 12 hours. Source code available; grading reports available. Official leaderboard | 29.33 ± 0.77 |
| 22 | R&D-Agent (o1-preview) Any Medal (%), All split (mean ± SEM). Low 48.18 ± 1.11, Medium 8.95 ± 1.05, High 18.67 ± 1.33. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 22.40 ± 0.50 |
| 23 | AIDE (o1-preview) Any Medal (%), All split (mean ± SEM). Low 35.91 ± 1.86, Medium 8.45 ± 0.43, High 11.67 ± 1.27. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 17.12 ± 0.61 |
| 24 | AIDE (gpt-4o-2024-08-06) Any Medal (%), All split (mean ± SEM). Low 18.55 ± 1.26, Medium 3.06 ± 0.33, High 8.15 ± 0.84. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 8.63 ± 0.54 |
| 25 | AIDE (claude-3-5-sonnet-20240620) Any Medal (%), All split (mean ± SEM). Low 19.70 ± 1.52, Medium 2.63 ± 1.52, High 2.22 ± 2.22. Running time 24 hours. Source code available; grading reports available. Official leaderboard | 7.56 ± 1.60 |
About this board
- On 2026-10-17 the official leaderboard (openai/mle-bench README) says: 'Update (04-24-2026): We are currently not taking any new submissions to the leaderboard while we develop an improved process for ensuring submissions are fair and comparable.' The newest main-table entry is dated 2026-03-06.
- The main table has 28 rows. Ranks 26-28 were left out (OpenHands gpt-4o-2024-08-06 4.89 ± 0.44, AIDE llama-3.1-405b-instruct 3.33 ± 0.38, MLAB gpt-4o-2024-08-06 1.60 ± 0.27).
- Two entries in 'Additional Leaderboard Submissions' were left out. The leaderboard says they are not directly comparable because they used test-set feedback: Disarray ensemble 77.78 ± 0.44 (2026-02-03, PR #118) and LoongFlow Gemini-3-Flash-Preview 62.66 ± 0.76 (2026-02-09, PR #119).
- Scores are the leaderboard's All (%) column, which is the Any Medal percentage on split75 (all 75 competitions). Some cells carry footnote markers [^2], [^3] or [^4]. We took those markers out of the score text and wrote what they mean into the setting column.
FAQ
What does MLE-bench measure?
The MLE-bench dataset is a collection of 75 Kaggle competitions which we use to evaluate the ML engineering capabilities of AI systems.
Which model leads MLE-bench?
As of October 17, 2026, Famou-Agent 2.0 (Gemini-3-Pro-Preview) ranks first on the MLE-bench board with 64.44 ± 1.18. That is the board's own order, not a Developers Digest ranking.
Where do these MLE-bench scores come from?
All 25 entries come from the official leaderboard at github.com. Every entry links to its source, which was read on October 17, 2026, and lists the setting it was run under.
Related reading
- Gemini 2.5 Pro and Gemini 3 Flash Deprecated in GitHub Copilot: What to Switch ToGemini 2.5 Pro and Gemini 3 Flash are deprecated in GitHub Copilot. What GitHub suggests instead, the October follow-up deprecations, and how to migrate.
- Harness Engineering and the Path to Self-Improving AIHarness engineering is where self-improving AI starts, Lilian Weng argues, not models rewriting their weights. What that means for developers building agents.
Sources
Last verified . Each row links to the page its score was copied from.
Newsletter
Get new benchmark results as they land
One email per week. New model releases, the leaderboard moves that matter and what they mean for what you build.