Leading ML hardware becomes 40% more energy-efficient each year

Historically, the energy efficiency of leading GPUs and TPUs has doubled every 1.9 years. In tensor-FP16 format, the most efficient accelerators are Meta’s MTIA, at up to 2.1 x 10¹² FLOP/s per watt, and the NVIDIA H100, at up to 1.4 x 10¹² FLOP/s per watt. The upcoming Blackwell series of processors may be even more efficient, depending on their power consumption.