Claude 3.5 Haiku
ECI#166
127Claude 3.5 Haiku is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 127 and ranks 166th of 249 tracked models. At the time of its release, it ranked 14th of 92 tracked models.
Claude 3.5 Haiku vs select alternatives
| Domain | Benchmark | Claude 3.5 Haiku | |||
|---|---|---|---|---|---|
ECI scoreBest score 166 | 127 | 127 | 130 | 127 | |
| Ranking | #166 | #168 | #151 | #164 | |
| Conceptual reasoning | DTBenchBest score 98% | ||||
| Games | BALROGBest score 58% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | CritPtBest score 32% | ||||
SciCodeBest score 62% | |||||
GPQA DiamondBest score 96% | |||||
| Software engineering | WeirdML (v2)Best score 93% | ||||
| Other | LiveBenchBest score 82% |
