Back to models

Claude 3 Sonnet

ECI#186
119
Anthropic
Feb. 29, 2024
Closed weights

Claude 3 Sonnet is a closed-weights model from Anthropic, released February 29, 2024. It has an ECI score of 119 and ranks 186th of 240 tracked models. At the time of its release, it ranked 6th of 45 tracked models.

Claude 3 Sonnet vs select alternatives

DomainBenchmark
Claude 3 Sonnet
Anthropic
Feb. 29, 2024
Closed weights
Claude 2
Anthropic
Jul. 11, 2023
Closed weights
Mistral Large
Mistral AI
Feb. 26, 2024
Closed weights
Llama 3-70B
Meta AI
Apr. 18, 2024
Open weights
ECI scoreBest score 173
119117120121
Ranking#186#191#185#181
Conceptual reasoning
DTBenchBest score 99%
54%
52%
56%
54%
Language understanding
WinoGrandeBest score 89%
75%
n/a
n/a
84%
Mathematics
MATH Level 5Best score 98%
18%
12%
24%
23%
OTIS Mock AIME 2024-2025Best score 100%
3%
3%
2%
4%
Science
GPQA DiamondBest score 97%
41%
35%
39%
41%
World knowledge
MMLUBest score 88%
76%
79%
69%
79%

Model metadata

General
OrganizationAnthropic
Release dateFeb. 29, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2023-08-01
Provider notesThe Claude 3 Model Family: Opus, Sonnet, Haiku
Pricing
Input price$3 / 1M tokens
Output price$15 / 1M tokens
Training
ParametersUnknown
Training computeUnknown
Training hardwareUnknown
Training timeUnknown
Training dataset sizeUnknown