Llama 3.1-405B | Epoch AI

ECI#162

128

Meta AI

Jul. 23, 2024

Open weights

Llama 3.1-405B is an open-weights model from Meta AI, released July 23, 2024. It has an ECI score of 128 and ranks 162nd of 240 tracked models. At the time of its release, it ranked 3rd of 67 tracked models.

Llama 3.1-405B vs select alternatives

Domain	Benchmark	Llama 3.1-405B Meta AI Jul. 23, 2024 Open weights	Llama 3-70B Meta AI Apr. 18, 2024 Open weights	Claude 3.5 Sonnet Anthropic Jun. 20, 2024 Closed weights	GPT-4o (May 2024) OpenAI May 13, 2024 Closed weights
	ECI scoreBest score 173	128	121	130	128
	Ranking	#162	#181	#157	#161
Conceptual reasoning	SimpleBenchBest score 95%	23%	n/a	28%	n/a
	DTBenchBest score 99%	61%	54%	68%	65%
Long context	ForecastBenchBest score 66.7	62.3	59.9	62.9	61.0
Mathematics	MATH Level 5Best score 98%	50%	23%	52%	51%
	OTIS Mock AIME 2024-2025Best score 100%	10%	4%	7%	6%
Science	GPQA DiamondBest score 97%	51%	41%	54%	49%
Software engineering	CybenchBest score 93%	8%	5%	18%	n/a
	WeirdML (v2)Best score 94%	21%	n/a	31%	n/a
World knowledge	MMLUBest score 88%	85%	79%	87%	84%

Model metadata

General
Organization	Meta AI
Release date	Jul. 23, 2024
Model accessibility	Open weights
Country	United States of America
Knowledge cutoff	2023-12-01
Provider notes	The Llama 3 Herd of Models

Pricing
Input price	$4 / 1M tokens
Output price	$4 / 1M tokens

Training
Parameters	4.1e11
Training compute	3.8e25 FLOP
Training hardware	NVIDIA H100 SXM5 80GB
Training time	89.3 days
Training dataset size	1.6e13 datapoints