Frontier models can handle well-defined tasks reliably, but fail in the more open-ended aspects that prevent full work automation. Explore Epoch AI's Automation Reports: realistic, open-ended tasks taken directly from our work, manually graded by our own standards.
AI model scores on Epoch AI's IKEA furniture assembly benchmark jumped from 28% to 80% in 10 months, with open-weight models trailing closed-weight models by about 7 months. Epoch AI's test of visual and spatial reasoning across 60 photos from three furniture builds.
A benchmark of 68 significant Erdős problems, open as of August 2026, curated by Thomas Bloom and formalized in Lean. AI systems must prove or disprove them within a fixed budget.
A good benchmark asks more than just “can AI do this specific task?”
Frontier models show no improvement across 30 playthroughs of the board game Earthborne Rangers, scoring far below expert humans. Epoch AI's EBR-bench probes whether AI can learn on the fly.
Give up at least one of: text only, short time horizon, easy to grade, and expert human superiority.
In this episode, Greg Burnham and Tom Adamczewski join Anson Ho to push back on benchmark pessimism and dig into what the next generation of AI benchmarks could look like.
These benchmarks track a wide range of digital work. Progress will correlate with economic utility, but tasks are too self-contained to indicate full automation.
In this episode, Daniel Litt chats with the hosts about AI’s limits in mathematics, accelerating math research, and how to measure progress on open problems.
Is this because skills generalize very well, or because developers are pushing on all benchmarks at once?