We report the proportion of successful runs for each model in both games on their respective maximum difficulties:

• For Slay the Spire 1, this is the “Ascension 20 Heart Kill” setting, where models must not only beat the base game’s three Acts with the maximum number of “Ascension” difficulty modifiers, but also unlock and complete “Act 4” by defeating the final boss, The Heart. Models play 15 runs each on the game’s four characters: Ironclad, Silent, Defect, and Watcher, for a total of 60 runs.

• For Slay the Spire 2, this is the “Ascension 10” setting, where models must complete all three Acts with the maximum number of “Ascension” difficulty modifiers. Slay the Spire 2, as of the launch of this benchmark, lacks an equivalent to The Heart. Models play 15 runs each on the game’s five characters: Ironclad, Silent, Regent, Necrobinder, and Defect, for a total of 75 runs.

We then average the two win rates, weighting both games equally.

For both games, each character’s 15 runs use a fixed set of arbitrarily chosen “seeds”, which determine the set of random outcomes generated during the game. Every model plays the same seeds, so differences in performance are attributable to skill rather than randomness. Additionally, for both games, models play on a profile with all content unlocked.

Each model is run at its respective maximum reasoning setting.

Models play using a basic computer-use harness based on the Inspect ReAct agent. They view the game through screenshots (1280 by 720 for the first game; 1280 by 800 for the second), and interact with it through mouse and keyboard inputs (click, right-click, drag, hover, key, scroll, wait, and a queue of up to eight actions).

All models are run at a compaction threshold of 300,000 tokens. They are allowed to take notes that will persist across compactions. Compaction is performed through the Inspect CompactionAuto policy, which defaults to the model provider’s native compaction endpoint and falls back to summary compaction if the native compaction endpoint is not available. Depending on the provider, we may prune older screenshots of the game from the context before compaction to avoid breaching providers’ image upload limits.

Models are given a web search tool through Tavily, restricted to the Slay the Spire wiki. This helps control for models knowing more information about the first game than the second. We are interested in whether models can solve the games’ tactical puzzles given full information, not whether they have memorized specific trivia about the games.

“Save scumming”, the practice of saving and quitting the game to restart an encounter with advance knowledge, is not prevented mechanically. Instead, we exclude and rerun samples that show evidence of save scumming. We observe this behavior in less than 0.1% of samples.