Benchmark leaderboard - 49 entries on 2 boards
SWE-bench (Verified and Pro) leaderboard
SWE-bench Verified is a human-filtered subset of 500 real GitHub issues scored by % Resolved, and SWE-Bench Pro evaluates AI agents on realistic software engineering tasks scored by Resolve Rate, the percentage of tasks an agent resolves. As of October 17, 2026, Claude 4.5 Opus and Claude 4.5 Opus (medium) are tied for first on the SWE-bench Verified board with 79.20 and Muse Spark 1.1 ranks first on the SWE-Bench Pro (Public Dataset) board with 61.50±3.10.
Last verified from swebench.com, labs.scale.com. Read the caveats
This page covers 2 separate boards: SWE-bench Verified and SWE-Bench Pro (Public Dataset). Each is scored on its own terms, so compare rows only within the same board.
SWE-bench Verified
24 entries, in the board's order
| 1 | Claude 4.5 Opus Agent: Sonar Foundation Agent (full Verified leaderboard) Official leaderboard | 79.20 |
| 2 | Claude 4.5 Opus (medium) Agent: live-SWE-agent Official leaderboard | 79.20 |
| 3 | Gemini 3 Pro Preview Agent: live-SWE-agent Official leaderboard | 77.40 |
| 4 | Claude 4 Sonnet Agent: EPAM AI/Run Developer Agent Official leaderboard | 76.80 |
| 5 | Multiple Agent: Atlassian Rovo Dev Official leaderboard | 76.80 |
| 6 | Claude 4.5 Opus (high) Bash Only: mini-SWE-agent 2.0.0, Avg. $0.75 per instance (bash-only rank 1) Official leaderboard | 76.80 |
| 7 | Multiple Agent: ACoder Official leaderboard | 76.40 |
| 8 | Gemini 3 Flash (high) Bash Only: mini-SWE-agent 2.0.0, Avg. $0.36 per instance Official leaderboard | 75.80 |
| 9 | MiniMax M2.5 (high) Bash Only: mini-SWE-agent 2.0.0, Avg. $0.07 per instance Official leaderboard | 75.80 |
| 10 | Multiple Agent: Warp Official leaderboard | 75.60 |
| 11 | Claude 4.6 Opus Bash Only: mini-SWE-agent 2.0.0, Avg. $0.55 per instance Official leaderboard | 75.60 |
| 12 | Multiple Agent: TRAE Official leaderboard | 75.20 |
| 13 | Claude Sonnet 4 Agent: Harness AI Official leaderboard | 74.80 |
| 14 | Claude 4.5 Sonnet Agent: Sonar Foundation Agent Official leaderboard | 74.80 |
| 15 | Claude 4 Sonnet Agent: Lingxi-v1.5 Official leaderboard | 74.60 |
| 16 | Multiple Agent: JoyCode Official leaderboard | 74.60 |
| 17 | Multiple Agent: Refact.ai Agent Official leaderboard | 74.40 |
| 18 | GPT-5 Agent: Prometheus-v1.2.1 Official leaderboard | 74.40 |
| 19 | Claude 4.5 Opus (medium) Bash Only: mini-SWE-agent 1.16.0, Avg. $0.72 per instance Official leaderboard | 74.40 |
| 20 | Gemini 3 Pro Preview Bash Only: mini-SWE-agent 1.15.0, Avg. $0.46 per instance Official leaderboard | 74.20 |
| 21 | Multiple Agent: Salesforce AI Research SAGE (OpenHands variant per logo filename) Official leaderboard | 73.80 |
| 22 | Claude 4 Opus Agent: Tools Official leaderboard | 73.20 |
| 23 | Multiple Agent: Salesforce AI Research SAGE (bash-only variant per logo filename) Official leaderboard | 73.00 |
| 24 | GLM 5 (high) Bash Only: mini-SWE-agent 2.0.0, Avg. $0.53 per instance Official leaderboard | 72.80 |
SWE-Bench Pro (Public Dataset)
25 entries, in the board's order
| 1 | Muse Spark 1.1 mini-swe-agent harness; uncapped cost, turn limit 250; public set (731 instances) Official leaderboard | 61.50±3.10 |
| 2 | gpt-5.4 (xHigh) mini-swe-agent harness; uncapped cost, turn limit 250; public set Official leaderboard | 59.10±3.56 |
| 3 | Muse Spark mini-swe-agent harness; uncapped cost, turn limit 250; public set Official leaderboard | 55.00±3.60 |
| 4 | claude-opus-4-6 (thinking) mini-swe-agent harness; uncapped cost, turn limit 250; public set Official leaderboard | 51.90±3.61 |
| 5 | gemini-3.1-pro (thinking) mini-swe-agent harness; uncapped cost, turn limit 250; public set Official leaderboard | 46.10±3.60 |
| 6 | claude-opus-4-5-20251101 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 45.89±3.60 |
| 7 | claude-4-5-Sonnet Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 43.60±3.60 |
| 8 | gemini-3-pro-preview Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 43.30±3.60 |
| 9 | claude-4-Sonnet Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 42.70±3.59 |
| 10 | gpt-5-2025-08-07 (High) Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 41.78±3.49 |
| 11 | gpt-5.2-codex Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 41.04±3.57 |
| 12 | claude-4-5-haiku Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 39.45±3.55 |
| 13 | qwen3-coder-480b-a35b Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 38.70±3.55 |
| 14 | minimax-2.1 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 36.81±3.55 |
| 15 | gemini-3-flash Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 34.63±3.55 |
| 16 | gpt-5.2 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 29.94±2.15 |
| 17 | kimi-k2-instruct Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 27.67±3.25 |
| 18 | qwen3-235b-a22b Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 21.41±2.25 |
| 19 | gpt-oss-120b Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 16.20±2.67 |
| 20 | deepseek-v3p2 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 15.56±2.63 |
| 21 | gemma-3-27b-it Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 11.38±2.15 |
| 22 | llama3-1-405b-instruct Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 11.18±2.15 |
| 23 | glm-4.6 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 9.67±2.15 |
| 24 | llama4-maverick-17b-instruct Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 5.24±1.24 |
| 25 | codestral-2405 Not marked as mini-swe-agent; uncapped cost, turn limit 250; public set Official leaderboard | 1.51±1.51 |
About this board
- SWE-Bench Pro (Scale Labs) leaderboard shows no per-entry submission dates, so date is blank on all 25 Pro entries; page read 2026-10-17 (scale.com/leaderboard/swe_bench_pro_public redirects to labs.scale.com/leaderboard/swe_bench_pro).
- SWE-bench Verified official leaderboard (swebench.com, read 2026-10-17) has no entry dated after 2026-02-26; newer frontier models (for example gpt-5.4, gemini-3.1-pro, Muse Spark) appear on SWE-Bench Pro but not on the Verified board.
- swebench.com Verified page interleaves full-leaderboard rows with Bash Only (mini-SWE-agent) rows that carry their own rank numbering; the 25 Verified entries are the first 25 rows in displayed order, with Bash Only rows noted in setting.
- SWE-Bench Pro Private Dataset leaderboard not recorded (public set only).
FAQ
What does SWE-bench (Verified and Pro) measure?
SWE-bench Verified is a human-filtered subset of 500 real GitHub issues scored by % Resolved, and SWE-Bench Pro evaluates AI agents on realistic software engineering tasks scored by Resolve Rate, the percentage of tasks an agent resolves. This page covers 2 separate boards: SWE-bench Verified and SWE-Bench Pro (Public Dataset). Each is scored on its own terms, so compare rows only within the same board.
Which model leads SWE-bench (Verified and Pro)?
As of October 17, 2026, Claude 4.5 Opus and Claude 4.5 Opus (medium) are tied for first on the SWE-bench Verified board with 79.20 and Muse Spark 1.1 ranks first on the SWE-Bench Pro (Public Dataset) board with 61.50±3.10. That is the board's own order, not a Developers Digest ranking.
Where do these SWE-bench (Verified and Pro) scores come from?
All 49 entries come from the official leaderboards at swebench.com and labs.scale.com. Every entry links to its source, which was read on October 17, 2026, and lists the setting it was run under.
Related reading
- LFM2.5-24B-A2B vs Qwen3.6-35B-A3B: Fast Local Agent or Better Coder?LFM2.5-24B-A2B vs Qwen3.6-35B-A3B for local agents: Liquid wins tool-use benchmarks, Qwen wins SWE-Bench Verified. Which one to run for which job.
- Qwen3.8-27B vs Opus 4.6 Max: Local Agentic Coding BenchmarksQwen3.8-27B is a 27B dense Apache-2.0 model that beats Opus 4.6 Max on SWE-bench Pro (61.7 vs 53.4). Benchmarks, hardware, cost and when to run it.
- LivePlan: Monitoring and Corrective Steering for Coding Agents, Without the LLM TaxLivePlan monitors SWE-agent runs and calls an advisor LLM only on drift, raising resolution rates up to 15.2 points for an extra $0.08 per instance.
- DCAS: Why Fine-Tuned Coding Agents Fall Apart When You Switch ScaffoldsFine-tuned coding agents collapse when you switch scaffolds, from 52.6% to 8.4% on SWE-bench Verified. DCAS shows how to train planning into the model.
- OwlPath: Ontology-Based Code Retrieval Cuts Agent Tokens 29%OwlPath encodes code into an OWL2 ontology so coding agents can answer multi-hop structural queries with SPARQL, cutting tokens 28.8% on SWE-bench Pro.
- PAIChecker: 13.6% of SWE-bench Verified Instances Have Misaligned PR-Issue PairsPAIChecker finds 13.6% of SWE-bench Verified instances pair a PR with an issue it does not resolve, penalizing agents that solve the stated problem.
Sources
Last verified . Each row links to the page its score was copied from.
Newsletter
Get new benchmark results as they land
One email per week. New model releases, the leaderboard moves that matter and what they mean for what you build.