Progress on the NanoGPT speedrun, a proxy for AI training efficiency. Participants submit code to train a small language model to a validation loss of 3.28 on FineWeb, using 8 NVIDIA H100 GPUs. Progress is measured by reductions in training time.
The NanoGPT speedrun challenges participants to train a small language model to a fixed quality in the least time; the task is closely tied to how efficiently AI models can be trained. While not a direct measure of AI’s impact, its record is a narrow indicator of software progress. Some record holders credit AI assistants with their improvements, so faster progress could partly reflect AI-assisted research.
Between May 2024 and the end of 2025, the record fell from 2,700 seconds (45 minutes) to 114 seconds. Submissions in 2026 cut it to 40 seconds by August, with one record on August 30 alone cutting the time by 41%.
We take the main world-record history from the modded-nanogpt repository, using each record’s date and training time. The series starts with the llm.c baseline from May 28, 2024, followed by the accepted records. Times after record 21 include about four extra seconds because of a timing-rule change. We keep them as reported rather than adjusting them.
Source: modded-nanogpt (Keller Jordan).
Our charts, text and processing are available under the Creative Commons Attribution license. The underlying data comes from modded-nanogpt (Keller Jordan) (https://github.com/KellerJordan/modded-nanogpt) and is subject to its own licence or terms.
Progress on the NanoGPT speedrun, a proxy for AI training efficiency. Participants submit code to train a small language model to a validation loss of 3.28 on FineWeb, using 8 NVIDIA H100 GPUs. Progress is measured by reductions in training time.