Claude 3.5 Haiku
ECI#174
127Claude 3.5 Haiku is a closed-weights model from Anthropic, released October 22, 2024. It has an ECI score of 127 and ranks 174th of 257 tracked models. At the time of its release, it ranked 14th of 92 tracked models.
Claude 3.5 Haiku vs select alternatives
| Domain | Benchmark | Claude 3.5 Haiku | |||
|---|---|---|---|---|---|
ECI scoreBest score 167 | 127 | 127 | 130 | 127 | |
| Ranking | #174 | #176 | #159 | #172 | |
| Conceptual reasoning | DTBenchBest score 99% | ||||
| Games | BALROGBest score 68% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | CritPtBest score 32% | ||||
SciCodeBest score 67% | |||||
GPQA DiamondBest score 96% | |||||
| Software engineering | WeirdML (v2)Best score 94% | ||||
| Other | LiveBenchBest score 82% |
