Back to models

Gemini 1.5 Flash (Sep 2024)

ECI#165
129
Google DeepMind
Sep. 24, 2024
Closed weights

Gemini 1.5 Flash (Sep 2024) is a closed-weights model from Google DeepMind, released September 24, 2024. It has an ECI score of 129 and ranks 165th of 260 tracked models. At the time of its release, it ranked 5th of 89 tracked models.

Gemini 1.5 Flash (Sep 2024) vs select alternatives

DomainBenchmark
Gemini 1.5 Flash (Sep 2024)
Google DeepMind
Sep. 24, 2024
Closed weights
Gemini 1.5 Pro (May 2024)
Google DeepMind
May 14, 2024
Closed weights
o1-mini
OpenAI
Sep. 12, 2024
Closed weights
Claude 3.5 Sonnet
Anthropic
Jun. 20, 2024
Closed weights
ECI scoreBest score 167
129127136130
Ranking#165#179#141#162
Conceptual reasoning
DTBenchBest score 99%
54%
56%
n/a
68%
Mathematics
MATH Level 5Best score 98%
62%
41%
89%
52%
OTIS Mock AIME 2024-2025Best score 100%
16%
7%
47%
7%
Science
GPQA DiamondBest score 96%
47%
46%
62%
54%
Software engineering
WeirdML (v2)Best score 94%
25%
n/a
36%
31%
World knowledge
MMLUBest score 88%
74%
86%
n/a
87%

Model metadata

General
OrganizationGoogle DeepMind
Release dateSep. 24, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2024-05-01
Provider notesGemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Pricing
Input price—
Output price—
Training
ParametersUnknown
Training computeUnknown
Training hardwareGoogle TPU v4
Training timeUnknown
Training dataset sizeUnknown