Back to models

Claude 3 Sonnet

ECI#191
121
Anthropic
Feb. 29, 2024
Closed weights

Claude 3 Sonnet is a closed-weights model from Anthropic, released February 29, 2024. It has an ECI score of 121 and ranks 191st of 253 tracked models. At the time of its release, it ranked 6th of 51 tracked models.

Claude 3 Sonnet vs select alternatives

DomainBenchmark
Claude 3 Sonnet
Anthropic
Feb. 29, 2024
Closed weights
Claude Instant
Anthropic
Aug. 9, 2023
Closed weights
phi-3-medium 14B
Microsoft
Apr. 23, 2024
Open weights
Mistral Large
Mistral AI
Feb. 26, 2024
Closed weights
ECI scoreBest score 167
121120121122
Ranking#191#192#190#188
Conceptual reasoning
DTBenchBest score 99%
54%
46%
n/a
56%
Language understanding
WinoGrandeBest score 89%
75%
n/a
82%
n/a
Mathematics
MATH Level 5Best score 98%
18%
n/a
18%
24%
OTIS Mock AIME 2024-2025Best score 100%
3%
n/a
n/a
2%
Science
GPQA DiamondBest score 96%
41%
n/a
28%
39%
World knowledge
MMLUBest score 88%
76%
73%
78%
69%

Model metadata

General
OrganizationAnthropic
Release dateFeb. 29, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2023-08-01
Provider notesThe Claude 3 Model Family: Opus, Sonnet, Haiku
Pricing
Input price$3 / 1M tokens
Output price$15 / 1M tokens
Training
ParametersUnknown
Training computeUnknown
Training hardwareUnknown
Training timeUnknown
Training dataset sizeUnknown