Epoch AI

Epoch AI is an independent research nonprofit, described on its own site as a data-first research institute investigating the future of artificial intelligence; it says it was founded to help people understand what is happening in AI from a neutral perspective grounded in evidence.¹ It began in 2021 as a group of volunteers collecting data on hundreds of machine learning models, and formed Epoch AI after the success of its 2022 paper on training-compute trends.¹ Its work is free to use under a Creative Commons Attribution license, and, as an independent nonprofit, it says it relies on the ongoing support of donors to continue its research, and asks that its work be credited to its source and authors.¹

Epoch matters to MetaRoles for its independent benchmarking. Rather than repeating the scores AI labs report, it runs benchmarks itself on its own infrastructure and publishes the data, including its SWE-bench Verified results, in which it evaluates 484 of the benchmark’s 500 tasks and reports a standard error for each run.² It also reviews the benchmarks it runs: in September 2026 it rated SWE-bench Verified “Flawed”, citing an OpenAI audit that found broken test cases and likely contamination.³ We use its numbers in our ranking of SWE-bench Verified results and in the coding-agents feature, and keep lab-reported scores out of ranked columns.