Back to models

Gemini 1.5 Flash (Sep 2024)

ECI#154
129
Google DeepMind
Sep. 24, 2024
Closed weights

Gemini 1.5 Flash (Sep 2024) is a closed-weights model from Google DeepMind, released September 24, 2024. It has an ECI score of 129 and ranks 154th of 249 tracked models. At the time of its release, it ranked 5th of 89 tracked models.

Gemini 1.5 Flash (Sep 2024) vs select alternatives

DomainBenchmark
Gemini 1.5 Flash (Sep 2024)
Google DeepMind
Sep. 24, 2024
Closed weights
Gemini 1.5 Pro (May 2024)
Google DeepMind
May 14, 2024
Closed weights
o1-mini
OpenAI
Sep. 12, 2024
Closed weights
Claude 3.5 Sonnet
Anthropic
Jun. 20, 2024
Closed weights
ECI scoreBest score 166
129127136130
Ranking#154#169#130#151
Conceptual reasoning
DTBenchBest score 98%
54%
56%
n/a
68%
Mathematics
MATH Level 5Best score 98%
62%
41%
89%
52%
OTIS Mock AIME 2024-2025Best score 100%
16%
7%
47%
7%
Science
GPQA DiamondBest score 96%
47%
46%
62%
54%
Software engineering
WeirdML (v2)Best score 93%
25%
n/a
36%
31%
World knowledge
MMLUBest score 88%
74%
86%
n/a
87%

Model metadata

General
OrganizationGoogle DeepMind
Release dateSep. 24, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2024-05-01
Provider notesGemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Pricing
Input price—
Output price—
Training
ParametersUnknown
Training computeUnknown
Training hardwareGoogle TPU v4
Training timeUnknown
Training dataset sizeUnknown