Claude 3.5 Sonnet (October 2024)
ECI#151
133Claude 3.5 Sonnet (October 2024) is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 133 and ranks 151st of 253 tracked models. At the time of its release, it ranked 3rd of 89 tracked models.
Claude 3.5 Sonnet (October 2024) vs select alternatives
| Domain | Benchmark | Claude 3.5 Sonnet (October 2024) | |||
|---|---|---|---|---|---|
ECI scoreBest score 169 | 133 | 130 | 136 | 132 | |
| Ranking | #151 | #159 | #142 | #156 | |
| Conceptual reasoning | SimpleBenchBest score 82% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | Humanity's Last ExamBest score 55% | ||||
GPQA DiamondBest score 96% | |||||
| Software engineering | METR Time HorizonsBest score 17 hr 25 min | 20 min 31 sec | 11 min 24 sec | — | — |
Aider PolyglotBest score 88% | |||||
WeirdML (v2)Best score 94% | |||||
| Other | LiveBenchBest score 82% |
