Gemini 1.5 Flash (May 2024) is a closed-weights model from Google DeepMind, released May 23, 2024. It has an ECI score of 121 and ranks 183rd (tied with Llama 3-70B and DeepSeek-V2 (MoE-236B, May 2024)) of 242 tracked models. At the time of its release, it ranked 7th of 58 tracked models.
| Domain | Benchmark | ||||
|---|---|---|---|---|---|
ECI scoreBest score 173 | 121 | 103 | 125 | 121 | |
| Ranking | #183 | #210 | #175 | #183 | |
| Conceptual reasoning | DTBenchBest score 99% | ||||
| Language understanding | BoolQBest score 91% | ||||
| Long context | ForecastBenchBest score 66.6 | 58.3 | — | 58.4 | 59.8 |
| Mathematics | GSM8KBest score 95% | ||||
MATH Level 5Best score 98% | |||||
OTIS Mock AIME 2024-2025Best score 100% | |||||
| Multimodal | Video-MMEBest score 80% | ||||
| Science | GPQA DiamondBest score 97% | ||||
| World knowledge | MMLUBest score 88% |