Claude 3.5 Sonnet (October 2024) is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 133 and ranks 150th (tied with Gemini 2.5 Flash-Lite (Jun 2025) and Mistral Medium 3) of 241 tracked models. At the time of its release, it ranked 3rd of 80 tracked models.
| Domain | Benchmark | ||||
|---|---|---|---|---|---|
ECI scoreBest score 174 | 133 | 130 | 135 | 130 | |
| Ranking | #150 | #157 | #140 | #157 | |
| Conceptual reasoning | SimpleBenchBest score 95% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | Humanity's Last ExamBest score 55% | ||||
GPQA DiamondBest score 97% | |||||
| Software engineering | METR Time HorizonsBest score 17 hr 25 min | 20 min 31 sec | 11 min 24 sec | — | — |
Aider PolyglotBest score 88% | |||||
WeirdML (v2)Best score 94% | |||||
| Other | LiveBenchBest score 82% |