Gemini 1.5 Flash (Sep 2024)
ECI#167
127Gemini 1.5 Flash (Sep 2024) is a closed-weights model from Google DeepMind, released September 24, 2024. It has an ECI score of 127 and ranks 167th of 240 tracked models. At the time of its release, it ranked 9th of 78 tracked models.
Gemini 1.5 Flash (Sep 2024) vs select alternatives
| Domain | Benchmark | Gemini 1.5 Flash (Sep 2024) | |||
|---|---|---|---|---|---|
ECI scoreBest score 173 | 127 | 125 | 127 | 127 | |
| Ranking | #167 | #173 | #166 | #165 | |
| Conceptual reasoning | DTBenchBest score 99% | ||||
| Games | BALROGBest score 68% | ||||
| Language understanding | PIQABest score 89% | ||||
| Mathematics | MATH Level 5Best score 98% | ||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Multimodal | GeoBenchBest score 89% | ||||
| Science | GPQA DiamondBest score 97% | ||||
| Software engineering | WeirdML (v2)Best score 94% | ||||
| World knowledge | MMLUBest score 88% |
