Back to models

Gemini 1.5 Pro (May 2024)

ECI#169
127
Google DeepMind
May 14, 2024
Closed weights

Gemini 1.5 Pro (May 2024) is a closed-weights model from Google DeepMind, released May 14, 2024. It has an ECI score of 127 and ranks 169th of 249 tracked models. At the time of its release, it ranked 4th of 63 tracked models.

Input price—
Output price—
ParametersUnknown
Training computeUnknown

Gemini 1.5 Pro (May 2024) vs select alternatives

Reasoning
DomainBenchmark
Gemini 1.5 Pro (May 2024)
Google DeepMind
May 14, 2024
Closed weights
Gemini 1.0 Pro
Google DeepMind
Dec. 13, 2023
Closed weights
GPT-4o (May 2024)
OpenAI
May 13, 2024
Closed weights
Claude 3 Opus
Anthropic
Feb. 29, 2024
Closed weights
ECI scoreBest score 166
127117129127
Ranking#169#203#156#168
Conceptual reasoning
DTBenchBest score 98.4%
56.5%
45.9%
64.5%
61.6%
Long context
ForecastBenchBest score 62.5
58.5—57.758.4
Mathematics
MATH Level 5Best score 98.1%
40.7%
11.2%
51.0%
37.5%
OTIS Mock AIME 2024-2025Best score 100.0%
6.8%
1.1%
6.3%
4.7%
Science
GPQA DiamondBest score 95.8%
45.9%
34.0%
48.9%
47.2%
World knowledge
MMLUBest score 88.1%
85.9%
70.0%
84.2%
84.6%

Model metadata

General
OrganizationGoogle DeepMind
Release dateMay 14, 2024
Model accessibilityClosed weights
CountryUnited States of America
Knowledge cutoff2023-11-01
Provider notesGemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Pricing
Input price—
Output price—
Training
ParametersUnknown
Training computeUnknown
Training hardwareGoogle TPU v4
Training timeUnknown
Training dataset sizeUnknown