Training compute costs are doubling every nine months for the largest AI models

The cost of training large-scale ML models is growing at a rate of 2.6x per year. The most advanced models now cost hundreds of millions of dollars, with expenses measured by amortizing cluster costs over the training period. About half of this spending is on GPUs, with the remainder on other hardware and energy.