Claude 3 Haiku
ECI#200
119Claude 3 Haiku is a closed-weights model from Anthropic, released March 7, 2024. It has an ECI score of 119 and ranks 200th of 253 tracked models. At the time of its release, it ranked 10th of 51 tracked models.
Claude 3 Haiku vs select alternatives
| Domain | Benchmark | Claude 3 Haiku | |||
|---|---|---|---|---|---|
ECI scoreBest score 169 | 119 | 119 | 116 | 116 | |
| Ranking | #200 | #198 | #211 | #213 | |
| Conceptual reasoning | LMCABest score 68.3 | 8.8 | — | — | 9.7 |
DTBenchBest score 99% | |||||
| Language understanding | WinoGrandeBest score 89% | ||||
| Long context | ForecastBenchBest score 67.0 | 54.8 | 58.7 | 59.4 | 46.4 |
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Science | GPQA DiamondBest score 96% | ||||
| Software engineering | WeirdML (v2)Best score 94% | ||||
| World knowledge | MMLUBest score 88% |
