Back to models

Gemini 1.5 Flash (May 2024)

ECI#183
121
Google DeepMind
May 23, 2024
Closed weights

Gemini 1.5 Flash (May 2024) is a closed-weights model from Google DeepMind, released May 23, 2024. It has an ECI score of 121 and ranks 183rd (tied with Llama 3-70B and DeepSeek-V2 (MoE-236B, May 2024)) of 242 tracked models. At the time of its release, it ranked 7th of 58 tracked models.

Gemini 1.5 Flash (May 2024) vs select alternatives

DomainBenchmark
Gemini 1.5 Flash (May 2024)
Google DeepMind
May 23, 2024
Closed weights
Google DeepMind
Feb. 21, 2024
Open weights
Google DeepMind
May 14, 2024
Closed weights
Meta AI
Apr. 18, 2024
Open weights
ECI scoreBest score 173
121103125121
Ranking#183#210#175#183
Conceptual reasoning
DTBenchBest score 99%
51%
n/a
56%
54%
Language understanding
BoolQBest score 91%
86%
83%
n/a
n/a
Long context
ForecastBenchBest score 66.6
58.3—58.459.8
Mathematics
GSM8KBest score 95%
82%
46%
n/a
n/a
MATH Level 5Best score 98%
25%
n/a
41%
23%
OTIS Mock AIME 2024-2025Best score 100%
4%
n/a
7%
4%
Multimodal
Video-MMEBest score 80%
70%
n/a
75%
n/a
Science
GPQA DiamondBest score 97%
40%
n/a
46%
41%
World knowledge
MMLUBest score 88%
78%
66%
86%
79%

Model metadata