To calculate the average time gap, we proceed day by day over our analysis window, January 1, 2026, through May 28, 2026. For each day, we identify the open-weight model with the highest ECI score available by that date. We then compare that model to the historical state-of-the-art ECI frontier and ask: what is the most recent date on which the SOTA model was not significantly better than this open-weight model? The time gap for that day is the number of days elapsed since that date.

Because ECI scores are estimated with uncertainty, we use bootstrap samples to make this comparison. Bootstrap samples are generated by resampling our full set of benchmark scores with replacement and refitting the ECI model on each resampled dataset. For each bootstrap sample, we compare the open-weight model’s bootstrapped ECI estimate to the bootstrapped ECI estimate of each historical SOTA model, preserving the pairing between bootstrap samples across models. We treat the open-weight model as having plausibly caught up to a previous SOTA model if it outperforms that SOTA model in at least 5% of paired bootstrap samples. Equivalently, this means the previous SOTA model is not significantly better than the open-weight model at the 5% level. We then use the most recent historical SOTA date satisfying this criterion to compute the time gap.

We find an average time gap of four months. This estimate would grow to six months if we required that the open-weight model’s point estimate for ECI be strictly higher than the closed-weight model it is catching up to (instead of better in at least 5% of samples).

Because release dates are observed without uncertainty, calculating the average ECI gap (i.e. the “vertical” gap at a given date) is as simple as observing the average difference between absolute and open-weight SOTA across all dates in our analysis window. We find an average ECI gap of 8 points, with a 90% confidence interval of 7 to 11 units.

We only include models with enough public benchmark coverage to assign an ECI. Leading closed labs do not always release their most capable models — for safety, commercial, or competitive reasons — so this analysis likely understates the true open-vs-closed gap whenever closed labs are sitting on more capable models that have not yet been published.