Claude 3.5 Sonnet (October 2024)
ECI#140
134Claude 3.5 Sonnet (October 2024) is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 134 and ranks 140th of 249 tracked models. At the time of its release, it ranked 3rd of 92 tracked models. At $6/Mtok for input tokens and $30/Mtok for output tokens, it is more expensive per token than similarly-capable models.
Input price$6 / 1M tokens
Output price$30 / 1M tokens
ParametersUnknown
Training computeUnknown
Claude 3.5 Sonnet (October 2024) vs select alternatives
Reasoning
| Domain | Benchmark | Claude 3.5 Sonnet (October 2024) | |||
|---|---|---|---|---|---|
ECI scoreBest score 166 | 134 | 130 | 136 | 132 | |
| Ranking | #140 | #151 | #130 | #146 | |
| Conceptual reasoning | SimpleBenchBest score 81.9% | ||||
| Mathematics | MATH Level 5Best score 98.1% | ||||
OTIS Mock AIME 2024-2025Best score 100.0% | |||||
| Science | Humanity's Last ExamBest score 46.5% | ||||
GPQA DiamondBest score 95.8% | |||||
| Software engineering | METR Time HorizonsBest score 1044.8 | 20.5 | 11.4 | — | — |
Aider PolyglotBest score 88.0% | |||||
WeirdML (v2)Best score 92.9% | |||||
| Other | LiveBenchBest score 82.3% |
