Back to models

Gemini 1.5 Pro (Sept 2024)

ECI#150
132
Google DeepMind
Sep. 24, 2024
Closed weights

Gemini 1.5 Pro (Sept 2024) is a closed-weights model from Google DeepMind, released September 24, 2024. It has an ECI score of 132 and ranks 150th of 253 tracked models. At the time of its release, it ranked 3rd of 89 tracked models.

Gemini 1.5 Pro (Sept 2024) vs select alternatives

DomainBenchmark
Gemini 1.5 Pro (Sept 2024)
Google DeepMind
Sep. 24, 2024
Closed weights
Gemini 1.5 Pro (May 2024)
Google DeepMind
May 14, 2024
Closed weights
o1-mini
OpenAI
Sep. 12, 2024
Closed weights
Claude 3.5 Sonnet
Anthropic
Jun. 20, 2024
Closed weights
ECI scoreBest score 167
132127136130
Ranking#150#173#134#155
Conceptual reasoning
SimpleBenchBest score 82%
27%
n/a
18%
28%
ARC-AGI-2Best score 95%
1%
n/a
1%
n/a
DTBenchBest score 99%
59%
56%
n/a
68%
Mathematics
MATH Level 5Best score 98%
70%
41%
89%
52%
OTIS Mock AIME 2024-2025Best score 100%
23%
7%
47%
7%
Science
GPQA DiamondBest score 96%
57%
46%
62%
54%
Software engineering
WeirdML (v2)Best score 94%
22%
n/a
36%
31%
World knowledge
MMLUBest score 88%
87%
86%
n/a
87%

Model metadata

General
OrganizationGoogle DeepMind
Release dateSep. 24, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2024-05-01
Provider notesGemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Pricing
Input price—
Output price—
Training
ParametersUnknown
Training computeUnknown
Training hardwareGoogle TPU v4
Training timeUnknown
Training dataset sizeUnknown