Claude 3.5 Sonnet (October 2024)
ECI#144
134Claude 3.5 Sonnet (October 2024) is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 134 and ranks 144th of 253 tracked models. At the time of its release, it ranked 3rd of 92 tracked models.
Claude 3.5 Sonnet (October 2024) vs select alternatives
| Domain | Benchmark | Claude 3.5 Sonnet (October 2024) | |||
|---|---|---|---|---|---|
ECI scoreBest score 167 | 134 | 130 | 136 | 132 | |
| Ranking | #144 | #155 | #134 | #150 | |
| Conceptual reasoning | SimpleBenchBest score 82% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | Humanity's Last ExamBest score 55% | ||||
GPQA DiamondBest score 96% | |||||
| Software engineering | METR Time HorizonsBest score 17 hr 25 min | 20 min 31 sec | 11 min 24 sec | — | — |
Aider PolyglotBest score 88% | |||||
WeirdML (v2)Best score 94% | |||||
| Other | LiveBenchBest score 82% |
