The power required to train frontier AI models is doubling annually

Data

We make use of two datasets: Epoch’s AI Models dataset, which collects information on over 1000 notable AI models, as well as our Machine Learning Hardware dataset, which records information on 161 AI accelerators.

After filtering our AI models data to a subset with values for each of `Publication date`, `Training power draw (W)`, we are left with 211 AI models. Our methodology for estimating training power draw can be found here.

Analysis

We focus our analysis on frontier models released after 2010, which we define as those that were among the top 10 models by training compute at the time of their release. After filtering to frontier models released in 2010 or later, we are left with 45 observations.

We fit a log-linear model to estimate the rate of growth for frontier model power draw, and obtain a 90% confidence interval from the coefficient’s standard errors. We estimate that power draw has grown by 2.1x per year, with a 90% confidence interval of 1.9 to 2.2x.

Assumptions and limitations

In general, we define “frontier models” as those in the top 10 models ordered by training compute at the time they were published. We show the robustness of our results against other choices of top-n, as well as in the following table: