Llama 3.1-70B | Epoch AI

ECI#175

125

Meta AI

Jul. 23, 2024

Open weights

Llama 3.1-70B is an open-weights model from Meta AI, released July 23, 2024. It has an ECI score of 125 and ranks 175th of 239 tracked models. At the time of its release, it ranked 8th of 67 tracked models.

Llama 3.1-70B vs select alternatives

Domain	Benchmark	Llama 3.1-70B Meta AI Jul. 23, 2024 Open weights	Llama 3-70B Meta AI Apr. 18, 2024 Open weights	GPT-4o mini OpenAI Jul. 18, 2024 Closed weights	Mistral Large 2 (Jul 2024) Mistral AI Jul. 24, 2024 Open weights
	ECI scoreBest score 174	125	121	124	126
	Ranking	#175	#180	#176	#168
Conceptual reasoning	LMCABest score 68.3	14.8	—	10.4	—
	DTBenchBest score 99%	60%	54%	54%	61%
Games	BALROGBest score 68%	28%	n/a	17%	n/a
Mathematics	MATH Level 5Best score 98%	37%	23%	53%	45%
	OTIS Mock AIME 2024-2025Best score 100%	4%	4%	7%	8%
Science	GPQA DiamondBest score 96%	44%	41%	38%	49%
Software engineering	WeirdML (v2)Best score 94%	9%	n/a	12%	n/a
World knowledge	MMLUBest score 88%	80%	79%	82%	80%

Model metadata

General
Organization	Meta AI
Release date	Jul. 23, 2024
Model accessibility	Open weights
Country	United States of America
Knowledge cutoff	2023-12-01
Provider notes	The Llama 3 Herd of Models

Pricing
Input price	$0.40 / 1M tokens
Output price	$0.40 / 1M tokens

Training
Parameters	7.0e10
Training compute	7.9e24 FLOP
Training hardware	Unknown
Training time	Unknown
Training dataset size	1.5e13 datapoints