Filter
An index aggregating many different benchmarks into a single, general capability scale.
XX models evaluated modelsExceptionally difficult research-level math problems.
XX models evaluated modelsExpert-written math problems covering advanced undergrad through early career research.
XX models evaluated modelsGitHub issues from real-world Python repos, testing whether models can implement valid code fixes.
XX models evaluated models100 novel puzzles, generated programmatically with a chess engine. Each puzzle has a single best next move.
XX models evaluated modelsA test of AI systems' learning capabilities, measuring whether their scores improve across repeated playthroughs of the relatively obscure game Earthborne Rangers.
XX models evaluated models100 positions from a puzzle-oriented variant of a well-known game, the details of which are deliberately left undisclosed.
XX models evaluated models1,000 factoid questions about politics, science and technology, art, sports, geography, music, and more.
XX models evaluated modelsA challenging multiple-choice question set in biology, chemistry, and physics, authored by PhD-level experts.
XX models evaluated models45 competition-style math problems from OTIS, harder than MATH Level 5 but easier than FrontierMath.
XX models evaluated models