EnigmaEval, created by Scale AI with the Center for AI Safety and MIT, evaluates long-form multimodal reasoning on 1,184 puzzles drawn from puzzle hunts. Solving them requires creative problem solving, synthesizing implicit knowledge, and chaining many deduction steps over mixed text and image content. Puzzles are split into a normal set from beginner-friendly hunts and a hard set drawn from events like the MIT Mystery Hunt, where frontier models still score near zero.

We source results from the public Scale AI EnigmaEval leaderboard. Our chart reports the combined accuracy across all puzzles, scored by exact match of the final answer; the leaderboard’s confidence-interval half-width is kept in the data export.

Model variants such as thinking versus non-thinking modes and reasoning-effort settings appear as separate leaderboard entries distinguished only by name, and we map them to model versions following the same conventions as our Humanity’s Last Exam data, which shares this leaderboard’s format. Scale evaluated most models after the benchmark’s release, and marks those rows with contamination caveats on the leaderboard.