Frontier open-weight models lag behind the most capable models by an average of 3 months in the Epoch Capabilities Index (ECI), our holistic measure of model capability. That corresponds to an average ECI gap of around 7 points, similar to the gap between o3 and GPT-5.
However, the gap varies considerably over time, sometimes even closing completely. Until the release of o1-mini, Llama 3.1-405B was rated on par with the closed-source state-of-the-art model, Claude 3.5 Sonnet.
You can see more detailed analysis about the gap in our earlier article.
Epoch's work is free to use, distribute, and reproduce provided the source and authors are credited under the Creative Commons BY license.
Learn more about this graph
We calculate the average gap between closed-weight and open-weight state-of-the-art performance according to our internal capability metric, the Epoch Capability Index (ECI). ECI is a composite measure which captures performance across many benchmarks.
Analysis
Explore this data
Benchmark results featuring the performance of leading AI models on challenging tasks.