Back to models

phi-3-medium 14B

ECI#186
121
Microsoft
Apr. 23, 2024
Open weights

phi-3-medium 14B is an open-weights model from Microsoft, released April 23, 2024. It has an ECI score of 121 and ranks 186th of 249 tracked models. At the time of its release, it ranked 10th of 59 tracked models.

phi-3-medium 14B vs select alternatives

DomainBenchmark
phi-3-medium 14B
Microsoft
Apr. 23, 2024
Open weights
Phi-2
Microsoft
Dec. 12, 2023
Open weights
DeepSeek-V2 (MoE-236B, May 2024)
DeepSeek
May 7, 2024
Open weights
Llama 3-8B
Meta AI
Apr. 18, 2024
Open weights
ECI scoreBest score 166
121108125116
Ranking#186#219#176#205
Conceptual reasoning
BBHBest score 89%
81%
59%
79%
n/a
Language understanding
Adversarial NLIBest score 58%
56%
43%
n/a
57%
HellaSwagBest score 95%
82%
54%
87%
n/a
Mathematics
MATH Level 5Best score 98%
18%
n/a
n/a
6%
Science
OpenBookQABest score 88%
87%
74%
n/a
83%
ARC AI2Best score 95%
92%
76%
92%
83%
GPQA DiamondBest score 96%
28%
n/a
n/a
26%
World knowledge
TriviaQABest score 88%
74%
45%
80%
68%
MMLUBest score 88%
78%
58%
78%
69%

Model metadata

General
OrganizationMicrosoft
Release dateApr. 23, 2024
Model accessibilityOpen weights
CountryUnited States of America
Knowledge cutoffUnknown
Provider notesPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Pricing
Input price$1 / 1M tokens
Output price$1 / 1M tokens
Training
Parameters1.4e10
Training compute4.0e23 FLOP
Training hardwareUnknown
Training timeUnknown
Training dataset size4.8e12 datapoints