Highest SWE-bench Verified Scores in Epoch AI's Independent Runs: Claude Opus 4.7 Leads at 83.5%
Epoch AI runs the SWE-bench Verified coding benchmark itself, on 484 real GitHub issues, instead of relying on the scores AI labs report. Here are the 25 highest results in its data, with their error margins, and why Epoch rates the benchmark "Flawed".
SWE-bench Verified, share of tasks solved in Epoch AI's independent runs (10 highest)
Source: Epoch AI benchmarking data (CC BY 4.0), retrieved October 20, 2026; 484 tasks; standard errors about ±2 points.
claude-opus-4-7_max83.5%
gpt-5.5-pre-release_xhigh80.6%
gemini-3.5-flash_high79.3%
claude-opus-4-678.7%
glm-5.2_max78.7%
deepseek-v4-pro_max77.6%
qwen3.7-max77.3%
gpt-5.4-2026-03-05_high76.9%
qwen3.6-max-preview76.7%
kimi-k2.676.7%
As of · Source: Epoch AI, independent runs on 484 of 500 SWE-bench Verified tasks (runs started February to June 2026)12
| Rank | Model version (as labelled by Epoch) | Organization | Country | Released | Tasks solved (%) | Relative to #1 | Std. error (points) | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | claude-opus-4-7_max | Anthropic | United States | 2026-04-16 | 83.5 | ±1.7 | 1 | |
| 2 | gpt-5.5-pre-release_xhigh | OpenAI | United States | 2026-04-23 | 80.6 | ±1.8 | 1 | |
| 3 | gemini-3.5-flash_high | Google DeepMind | United States | 2026-05-19 | 79.3 | ±1.8 | 1 | |
| 4= | claude-opus-4-6 Second run in the data: lower score | Anthropic | United States | 2026-02-05 | 78.7 | ±1.9 | 1 | |
| 4= | glm-5.2_max | Z.ai (Zhipu AI) | China | 2026-06-16 | 78.7 | ±1.9 | 1 | |
| 6 | deepseek-v4-pro_max | DeepSeek | China | 2026-04-24 | 77.6 | ±1.9 | 1 | |
| 7 | qwen3.7-max | Alibaba | China | 2026-05-19 | 77.3 | ±1.9 | 1 | |
| 8 | gpt-5.4-2026-03-05_high | OpenAI | United States | 2026-03-05 | 76.9 | ±1.9 | 1 | |
| 9= | qwen3.6-max-preview | Alibaba | China | 2026-04-20 | 76.7 | ±1.9 | 1 | |
| 9= | kimi-k2.6 | Moonshot | China | 2026-04-20 | 76.7 | ±1.9 | 1 | |
| 9= | claude-opus-4-5-20251101 | Anthropic | United States | 2025-11-24 | 76.7 | ±1.9 | 1 | |
| 12 | gemini-3.1-pro-preview-customtools | Google DeepMind | United States | 2026-02-19 | 75.6 | ±2.0 | 1 | |
| 13 | gemini-3-flash-preview | Google DeepMind | United States | 2025-12-17 | 75.4 | ±2.0 | 1 | |
| 14 | claude-sonnet-4-6 | Anthropic | United States | 2026-02-17 | 75.2 | ±2.0 | 1 | |
| 15 | gpt-5.3-codex_high | OpenAI | United States | 2026-02-05 | 74.8 | ±2.0 | 1 | |
| 16 | glm-5.1 | Z.ai (Zhipu AI) | China | 2026-04-07 | 74.2 | ±2.0 | 1 | |
| 17= | kimi-k2.5 | Moonshot | China | 2026-01-27 | 73.8 | ±2.0 | 1 | |
| 17= | gpt-5.2-2025-12-11_high | OpenAI | United States | 2025-12-11 | 73.8 | ±2.0 | 1 | |
| 19 | gpt-5-2025-08-07_high | OpenAI | United States | 2025-08-07 | 73.6 | ±2.0 | 1 | |
| 20 | claude-opus-4-1-20250805 | Anthropic | United States | 2025-08-05 | 73.3 | ±2.0 | 1 | |
| 21 | gemini-3-pro-preview | Google DeepMind | United States | 2025-11-18 | 72.9 | ±2.0 | 1 | |
| 22 | glm-5 | Z.ai (Zhipu AI) | China | 2026-02-11 | 72.1 | ±2.1 | 1 | |
| 23 | gpt-5-2025-08-07_medium | OpenAI | United States | 2025-08-07 | 71.5 | ±2.1 | 1 | |
| 24 | claude-sonnet-4-5-20250929 | Anthropic | United States | 2025-09-29 | 71.3 | ±2.1 | 1 | |
| 25 | claude-opus-4-20250514 | Anthropic | United States | 2025-05-22 | 70.7 | ±2.1 | 1 |
Metric: the share of the 484 SWE-bench Verified tasks that validate on Epoch's infrastructure which a model resolves in Epoch's own runs, as its mean score; the standard error is Epoch's.12 These are independent measurements by Epoch AI. Scores that AI labs publish about their own models are not included, and mixing them with these would compare different prompts, scaffolds and token budgets. Epoch's file lists 35 runs; each model version is shown once at its highest run, and the two versions with a second, lower run in the file (claude-opus-4-6 at 75.6% and gpt-5.1-2025-11-13_high at 65.9%) are flagged. The file does not say which scaffold each run used (Epoch's simple loop, or Claude Code or Codex through Inspect-SWE), and Epoch warns results depend heavily on the scaffold, so a gap of a few points between rows is within the noise of both the standard error and the scaffolds.23 With rounding to one decimal, tied scores share a rank ("="). Epoch's verdict on the benchmark itself is "Flawed" (see below).3
The top 10
gpt-5.5-pre-release_xhigh
80.6%gemini-3.5-flash_high
79.3%Google DeepMind released it on 2026-05-19; Epoch's run started on 2026-06-01 and scored 79.3% with a standard error of 1.8 points.1
claude-opus-4-6
78.7%glm-5.2_max
78.7%Z.ai (Zhipu AI) released it on 2026-06-16; Epoch's run started on 2026-06-25 and scored 78.7% with a standard error of 1.9 points.1
deepseek-v4-pro_max
77.6%DeepSeek released it on 2026-04-24; Epoch's run started on 2026-06-18 and scored 77.6% with a standard error of 1.9 points.1
qwen3.7-max
77.3%Alibaba released it on 2026-05-19; Epoch's run started on 2026-06-18 and scored 77.3% with a standard error of 1.9 points.1
gpt-5.4-2026-03-05_high
76.9%OpenAI released it on 2026-03-05; Epoch's run started on 2026-03-06 and scored 76.9% with a standard error of 1.9 points.1
qwen3.6-max-preview
76.7%Alibaba released it on 2026-04-20; Epoch's run started on 2026-05-28 and scored 76.7% with a standard error of 1.9 points.1
kimi-k2.6
76.7%Moonshot released it on 2026-04-20; Epoch's run started on 2026-05-08 and scored 76.7% with a standard error of 1.9 points.1
What the benchmark measures
SWE-bench Verified is a human-validated subset of 500 tasks from the original SWE-bench set. Each task comes from a GitHub issue in one of 12 open-source Python repositories: the model gets the repository and the issue description, must change the code to fix it, and its patch is graded by running unit tests.2 Epoch evaluates 484 of the 500 and excludes 16 that do not reliably run on its infrastructure.2 Its default scaffold is a simple loop in which the model takes an action, such as a tool call, then sees the output, with a bash tool, a text editor and a patch tool, and it includes results from other scaffolds such as Claude Code or Codex; it noted that its scaffold was significantly upgraded in February 2026.2 The benchmark scores a model on isolated bug fixes, so it says nothing directly about how fast a software developer works: for the trials that measure that, see Do coding agents make developers faster?
Why Epoch calls it "Flawed"
Epoch's September 3, 2026 review gives SWE-bench Verified the verdict "Flawed". It cites an audit by OpenAI published February 23, 2026, which examined 27.6% of the tasks and found that 59.4% of those had flawed test cases rejecting functionally correct submissions, which Epoch reads as at least 16.4% of all tasks broken, and says it has "reasonable confidence that over 20% of questions have scoring-based issues".3 Because the benchmark is static, Epoch says every current and future model can be assumed to have trained on the codebases, so contamination is likely, and OpenAI's analysis concluded that improvements on it "no longer reflect meaningful improvements in models' real-world software development abilities".3 A separate group, RDI, reported a reward hack that achieves a perfect score.3 Epoch had earlier estimated an error rate of 5 to 10% for the benchmark.2 So the table ranks models on what this test measures, and the top scores, which sit close together, are not evidence of which model would help a particular team most.
For why a benchmark score is not a result for a role, see how we measure how much of a role AI does. Occupation tasks come from O*NET; this benchmark's tasks do not, they are GitHub issues.
Sources
- Epoch AI, Capabilities & benchmarking data, swe_bench_verified.csv (CC BY 4.0; 35 runs), retrieved 2026-10-20.
- Epoch AI, SWE-bench Verified (benchmark page and methodology), retrieved 2026-10-20.
- Epoch AI, SWE-bench Verified: Benchmark review, September 3, 2026.