Benchmarks › SWE-bench
SWE-bench
Solve long software engineering tasks in a repository
99.4top score · Claude Opus 5
16models ranked
4open-weights models
Resolve rate (%)higher is better
Oct 21, 2026last updated
How the test works
1
Task
642 tasks
- What the model doesSolve long software engineering tasks in a repository
- Kinds of tasksLong-horizon software engineering
- In → outText, repository → Code patch
2
Run
Locked Harbor protocol; fresh-sandbox patch replay
- Time limit50 min per task
- InternetNo
- Tools allowedLocal repository tools; model endpoint only
3
Score
Resolve rate (%)
- Graded byAll fail-to-pass and pass-to-pass tests must pass in a fresh evaluation environment
- Better isHigher
Sources: github.com, labs.scale.com
SWE-bench Pro v2 full leaderboardtop 10 of 16
Full leaderboard with filters →| Model | SWE-bench Pro v2 full | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
99.4xhigh | $4.88 | $5 / $25 | 53 | 1M | ||
99.2high | $1.99 | $2 / $10 | 84 | 1M | ||
99.1high | $3.91 | $10 / $50 | 58 | 1M | ||
97.7max | $2.00 | $3 / $15 | 112 | 1M | ||
96.9high | $1.73 | $10 / $50 | 59 | 1.05M | ||
96.7xhigh | — | $4 / $20 | — | 1M | ||
96.1xhigh | $0.39 | $2 / $10 | 53 | 1.05M | ||
95.8xhigh | $2.01 | $2 / $10 | 112 | 1M | ||
95.6max | $2.01 | $1.40 / $4.40 | 186 | 1M | ||
95.5xhigh | $1.18 | $4 / $20 | 85 | 1.05M |
Versions and boards on Model Menu
| Board | Models |
|---|---|
SWE-bench Pro v2 fullShown above | 16 |
| 16 | |
| 9 |
SWE-bench questions
What is SWE-bench?›
Solve long software engineering tasks in a repository. It has 642 tasks. Created by Scale AI and Reflection.
Which model is best on SWE-bench?›
Claude Opus 5 leads SWE-bench Pro v2 full with 99.4, ahead of Gemini 4 Argon (99.2) and Claude Fable 5.1 (99.1).
What is the best open-weights model on SWE-bench?›
Kimi K3 with 97.7, then GLM 5.3 (95.6).
How is SWE-bench scored?›
Resolve rate (%); all fail-to-pass and pass-to-pass tests must pass in a fresh evaluation environment.
Which versions of SWE-bench are there?›
SWE-bench Pro v2 full (16 models), SWE-bench Pro v2 hard (16 models), SWE-bench Pro private (9 models).