How we calculate it

We model prefill and decode on separate chip pools, optimizing each independently.

1. Fix the workload. Each request has a cached context of 256Ki tokens (262,144 tokens). We evaluate GLM-5.2, Kimi-K3, Qwen3-Coder 480B and a MAI-Base-2 architecture proxy.
2. Estimate decoding capacity. NVIDIA’s AISimulate estimates execution time from model operations and hardware specifications. We search batch sizes and parallelism layouts that fit in memory, selecting the configuration with the most concurrent requests per allocated chip at each target output speed.
3. Include prefill capacity. We provision prefill to sustain 1.51 new input tokens per output token at each target decode speed. Capacity is concurrent decoding requests divided by the combined prefill and decode chip count.
4. Integrate across output speeds. Higher concurrency generally trades off against per-request output speed. We integrate concurrent decoding requests per total chip from 50 to 400 tokens/s/request on a logarithmic speed axis. Capacity is zero wherever no searched configuration meets the target.
5. Normalize and average. For each model, divide the integrated capacity by GB300’s, then take the arithmetic mean of the four ratios. Each model has equal weight; GB300 scores 1.

A score of 2 means twice GB300’s area under the capacity curve, on average across the four model comparisons. The advantage varies by model and speed.

Key choices and why

• Analytical performance estimates. AISimulate’s “speed of light” (SOL) estimates use compute, memory-bandwidth and interconnect specifications to model operation times. This enables hardware comparisons without equally optimized serving software for each chip. We apply no software-efficiency discount.
• A long, cached context. Agentic coding workloads accumulate long histories that are mostly cached, with comparatively short new input and output per request, as discussed in our previous report. We use 256Ki tokens as a fixed benchmark for this workload pattern. Long-context decode stresses KV-cache capacity and memory bandwidth; actual context lengths depend on how agents manage their history.
• Four equally weighted architectures. The basket spans sparse, latent, grouped-query and sliding-window/global attention. Equal weights provide architectural coverage without estimating traffic shares. Normalizing each model against GB300 before averaging prevents models with higher raw capacity from dominating. We also report individual model scores.
• Prefill based on observed usage. Across about 111,000 selected requests from TraceLab’s Claude Code subset, we counted 1.51 new input tokens per output token, excluding cached input. This measured ratio determines the prefill capacity needed to sustain decoding. We optimize a single prefill configuration for maximum throughput.
• Logarithmic speed weighting. The intervals 50–100, 100–200 and 200–400 tokens/s/request have equal horizontal weight in the integral. This captures the concurrency–speed tradeoff without relying on a single threshold. The range and weighting are benchmark choices.
• Speculative decoding and low precision. We assume three accepted output tokens per speculative-decoding iteration, informed by SemiAnalysis InferenceX’s Kimi-K3 coding benchmark. Low-precision expert weights reduce memory requirements. Acceptance rates and equivalent output quality across precision settings remain modeling assumptions; hardware-specific formats are documented in the repository.
• Versioned scores. Changes to the models, chip specifications or assumptions can change the index. Each release preserves its settings and results so comparisons can use a fixed version.

Accuracy and benchmark checks

AISimulate (previously AIConfigurator) reported approximately 8% average error in decode-token latency using measured kernel-performance data; our index instead uses its hardware-specification-based SOL mode. Our own InferenceX comparisons provide encouraging grounding: for Kimi-K3 on GB300, predictions assuming 50% of SOL performance differed from measured throughput by about 12% on average across five benchmark points, using the benchmark’s deployment layouts. This supports roughly half of SOL as a useful reference for achievable performance on this setup. Agreement was weaker on other hardware.

Interpretation and limits

The score counts active decoding requests with sufficient prefill capacity, assuming separate, fully utilized pools. It excludes tool pauses, queueing, cold-cache misses and KV-cache transfers between pools. It does not measure concurrent users or adjust for model quality, price or power. Architecture proxies, network simplifications and software efficiency affect how closely real systems approach these estimates.

Release and downloads

The v1.0.0 release covers the original 23 hardware profiles. Download its HTML bundle to read the visual guides locally.