Back to models

Claude 3.5 Sonnet (October 2024)

ECI#150
133
Anthropic
Oct. 22, 2024
Closed weights

Claude 3.5 Sonnet (October 2024) is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 133 and ranks 150th (tied with Gemini 2.5 Flash-Lite (Jun 2025) and Mistral Medium 3) of 241 tracked models. At the time of its release, it ranked 3rd of 80 tracked models.

Claude 3.5 Sonnet (October 2024) vs select alternatives

DomainBenchmark
Claude 3.5 Sonnet (October 2024)
Anthropic
Oct. 22, 2024
Closed weights
Anthropic
Jun. 20, 2024
Closed weights
OpenAI
Sep. 12, 2024
Closed weights
Google DeepMind
Sep. 24, 2024
Closed weights
ECI scoreBest score 174
133130135130
Ranking#150#157#140#157
Conceptual reasoning
SimpleBenchBest score 95%
41%
28%
18%
27%
Mathematics
MATH Level 5Best score 98%
57%
52%
89%
70%
OTIS Mock AIME 2024-2025Best score 100%
8%
7%
47%
23%
Science
Humanity's Last ExamBest score 55%
4%
n/a
n/a
5%
GPQA DiamondBest score 97%
55%
54%
62%
57%
Software engineering
METR Time HorizonsBest score 17 hr 25 min
20 min 31 sec11 min 24 sec——
Aider PolyglotBest score 88%
52%
n/a
33%
n/a
WeirdML (v2)Best score 94%
40%
31%
36%
22%
Other
LiveBenchBest score 82%
59%
n/a
58%
n/a

Model metadata