Llama 3.1-405B
ECI#159
129Llama 3.1-405B is an open-weights model from Meta AI, released July 23, 2024. It has an ECI score of 129 and ranks 159th of 249 tracked models. At the time of its release, it ranked 3rd of 74 tracked models. At $4/Mtok for input tokens and $4/Mtok for output tokens, it is more expensive per token than similarly-capable models.
Input price$4 / 1M tokens
Output price$4 / 1M tokens
Parameters4.1e11
Training compute3.8e25 FLOP
Llama 3.1-405B vs select alternatives
Reasoning
| Domain | Benchmark | Llama 3.1-405B | |||
|---|---|---|---|---|---|
ECI scoreBest score 166 | 129 | 123 | 130 | 129 | |
| Ranking | #159 | #180 | #151 | #156 | |
| Conceptual reasoning | SimpleBenchBest score 81.9% | ||||
DTBenchBest score 98.4% | |||||
| Long context | ForecastBenchBest score 62.5 | 59.9 | 57.1 | 59.4 | 57.7 |
| Mathematics | MATH Level 5Best score 98.1% | ||||
OTIS Mock AIME 2024-2025Best score 100.0% | |||||
| Science | GPQA DiamondBest score 95.8% | ||||
| Software engineering | WeirdML (v2)Best score 92.9% | ||||
CybenchBest score 93.0% | |||||
| World knowledge | MMLUBest score 88.1% |
