Position: LLM Inference Should Be Evaluated as Energy-to-Token Production
Xiang Liu1,∗ Shimiao Yuan2,∗ Zhenheng Tang3 Peijie Dong1 Kaiyong Zhao4 Qiang Wang5 Bo Li3,6 Xiaowen Chu1,†
arXiv:2605.11733v1 [cs.CE] 12 May 2026
1 5
HITSZ
HKUST(GZ) 2 UCAS 3 HKUST 4 XGRIDS 6 Guangzhou HKUST Fok Ying Tung Research Institute ∗
Equal contribution. † Corresponding author. [email protected] Project page: dominic789654.github.io/energy-to-token
Abstract LLM inference is still evaluated mainly as a model or software problem: accuracy, latency, throughput, and hardware utilization. This is incomplete. At deployment scale, the relevant output is a quality-conditioned token produced under joint constraints from effective compute, delivered data-center power, cooling capacity, PUE, and utilization. We argue that the ML community should treat inference as energy-to-token production. We formalize this view with a dimensionally consistent Token Production Function in which token rate is bounded by both compute-per-token and energyper-token ceilings. Listed API prices vary by over an order of magnitude across providers, but we use price dispersion only as directional motivation, not as causal evidence of marginal cost. The core physical question is instead: under fixed quality and service targets, when does the binding constraint move from theoretical peak compute toward delivered power, cooling, and operational efficiency? Under this framing, system optimizations—latent KV-cache compression, sparse or heavily compressed attention, quantization, routing, and difficulty-adaptive reasoning—are not merely local engineering tricks. They are energy-to-token levers because they reduce FLOPs/token, joules/token, memory traffic, or utilization losses under fixed (q ∗ , s∗ ). We therefore call for inference papers and benchmarks to report Joules/token, active binding constraint, PUE-adjusted delivered power, and utilization-adjusted token output alongside accuracy and latency.
1
Introduction
Tokens are becoming the metered output of AI factories. Each generated token converts electricity, accelerators, memory bandwidth, cooling capacity, and software organization into model output subject to quality and service constraints. This is not a metaphorical analogy. As AI data-center electricity demand rises [1, 2] and vendors describe data centers in tokens-per-watt terms [3, 4], inference increasingly resembles an industrial production process whose limiting inputs determine both cost and capacity. Current ML evaluation does not fully reflect this shift. Top-tier inference papers and benchmarks still emphasize accuracy, latency, throughput, and hardware Model FLOPs Utilization (MFU). These metrics remain necessary, but they do not answer the production question: how many qualityconditioned tokens can a deployment produce from a fixed envelope of compute, delivered power, Preprint.
cooling, and utilization? Once that question is asked, system optimizations change meaning. KVcache compression, sparse attention, quantization, routing, and scheduling are not only micro-level ways to win a benchmark; they are interventions that change the energy-to-token frontier. Listed LLM API prices make the physical constraint visible, but they do not identify it causally. As of early 2026, posted prices across major providers still span over an order of magnitude on comparable per-million-token units [5–7]; we use this only as motivation, since the underlying question is whether the binding constraint for generative AI is shifting from theoretical peak compute alone (CapEx) toward delivered data-center power, cooling capacity, PUE, and operational efficiency (OpEx). This position paper argues that LLM inference should be evaluated as energy-to-token production, not merely as model execution. We formalize this view with a Token Production Function: token output is bounded by both compute-per-token and energy-per-token ceilings under fixed quality and service targets. Under that framing, system optimizations become macro-level energy levers because they reduce FLOPs/token, joules/token, memory traffic, or utilization losses without proportional infrastructure expansion. Our contribution is fourfold. First, we diagnose why accuracy/MFU-centered inference evaluation is incomplete under regional power and cooling constraints. Second, we formalize quality- and service-conditioned token output with a dimensionally consistent production function. Third, we map concrete inference optimizations onto the physical variables they change: FLOPs/token, Joules/token, memory traffic, and utilization. Fourth, we propose an evaluation agenda: inference papers and benchmarks should report Joules/token, active binding constraint, PUE-adjusted delivered power, and utilization-adjusted token output alongside accuracy and latency. Our claim is bounded: we do not argue electricity alone determines prices, capability, or geopolitical outcomes, nor treat API prices as causal cost measurements; we argue delivered power and cooling have become binding enough to enter the ML evaluation objective. The paper builds on Green AI [8], carbon-accounting work [9–14], and MLPerf Power [15], adding a (q ∗ , s∗ )-conditioned Leontief production function, a falsifiable ρ − ρ∗ diagnostic with a recommended Kef f convention, and six disclosure dimensions that turn “report J/token” into a comparable benchmark.
2
The Token Production Function
To rigorously analyze LLM inference as an industrial process, we propose the following Token Production Function: Q̇token (t; q , s ) = min ∗
∗
Kef f (t) PIT (t) , ctok (t; q ∗ , s∗ ) etok (t; q ∗ , s∗ )
· U (t; q ∗ , s∗ )
(1)
with PIT (t) =
Pf acility (t) , P U E(t)
Z T Qtoken =
Q̇token (t; q ∗ , s∗ ) dt.
(2)
0
This formulation keeps units explicit: Kef f /ctok and PIT /etok are both tokens/sec, and Qtoken is total tokens over horizon T . Importantly, token output is only comparable across systems when evaluated at fixed quality and service targets (q ∗ , s∗ ); without this conditioning, token quantity alone is not a meaningful production measure. We define each component: • Qtoken : Total quantity of intelligence tokens produced over time period T . • Kef f (t): Effective available compute throughput (FLOPs/sec) at time t, after hardware availability, kernel efficiency, and memory-stall losses, but before demand-side queueing, batching mismatch, regulatory friction, and operational headroom losses captured by U . • Pf acility (t) and PIT (t): Facility-level power and IT-delivered power (watts), linked by P U E(t) ≥ 1. • ctok (t; q ∗ , s∗ ): Compute intensity (FLOPs/token) at fixed quality target q ∗ and service target s∗ . • etok (t; q ∗ , s∗ ): Energy intensity (joules/token) at the same q ∗ , s∗ operating point. 2
• U (t; q ∗ , s∗ ): Effective utilization factor after the physical ceilings are computed (0 < U ≤ 1), capturing queueing, batching mismatch, request-arrival variability, routing, localization/regulatory friction, and operational headroom.1 This separation avoids double counting: Kef f describes hardware- and execution-level effective throughput, while U describes how much of the resulting physical ceiling is converted into realized token output under demand, scheduling, routing, and institutional frictions. Likewise, ctok and etok are related but not interchangeable: ctok is computational work demand (FLOPs/token), whereas etok is measured energy intensity at the operating point (J/token). They therefore define distinct ceilings—compute-throughput capacity and power-delivery capacity—rather than two independent sources of token demand. The min(·, ·) operator instantiates a Leontief (fixed-proportions) production structure [16]: compute and delivered power are co-required at a given operating point, not freely substitutable. We adopt it as a local binding-constraint approximation rather than a claim about all long-run technological substitution: it gives the sharpest analytical predictions about which factor is binding when short-run physical substitution is negligible. The CES family [17] nests both Cobb-Douglas and Leontief as special cases (σ → 0 gives Leontief); we use Leontief as the binding-constraint limit. Under this form, Φsystem improvements do not substitute one factor for another at a fixed technology—they shift the production frontier by simultaneously reducing ctok and etok (or raising U ), rescaling both arms of the min together. This is why Section 4’s architectural gains (MLA, NSA, hybrid linear attention) are consistent with a Leontief structure: they relax both the compute and delivered-power constraints, rather than trading FLOPs for joules at a fixed operating point. As data-center power densities exceed 100 kW/rack [18], P (t) has emerged as the scarce factor in many regions. To avoid over-aggregation, we treat Φsystem as a structured set of mechanisms that parameterize ctok and etok rather than a single free multiplier: Φsystem ≡ {Φpref ill , Φdecode , Φmem , Φcomp , Φsched , Φroute }, with ctok = ctok (m, w, Φsystem ) and etok = etok (m, w, Φsystem , P U E) for model/workload pair (m, w). This decomposition is necessary because some interventions help prefill but not decode, or trade off energy against latency/quality. Operational estimation. Each Φ component admits a ratio-form estimator: Φmem ≈ Q̇obs /Q̇ceil BW with Q̇ceil specs; Φdecode restricts the nuBW = BWHBM /(2Nparam · wbytes ) from hardware batch∗ merator to decode-phase tokens; Φsched ≈ ŪSM /USM from SM-activity counters (e.g., obs DCGM_FI_PROF_SM_ACTIVE) divided by the ideal-batch reference; aggregate Φsystem ≈ eref tok /etok against a dense MHA at FP16 baseline at the same parameter count [19, 20]. Values Φmem < 0.3 indicate memory-bound operation; Φsched ≪ 1 indicates scheduling/batching overhead. This bridges systems engineering, macroeconomics, and energy policy: K(t) ↔CapEx, P (t) ↔OpEx, and Φsystem ↔TFP in the sense of Solow [21]—the residual output gain from better organization rather than raw input expansion. Unlike a pure macroeconomic residual, however, Φsystem is partially decomposable into measurable serving mechanisms. Which constraint binds? The min(·, ·) structure raises a practical question: when is compute the binding factor and when is delivered power? The crossover occurs at the constraint boundary: Kef f PIT = ctok etok
PIT etok = ≡ ρ∗ , Kef f ctok
⇐⇒
(3)
where ρ∗ (joules/FLOP) is the energy-per-FLOP ratio demanded by the workload. If ρ ≡ PIT /Kef f > ρ∗ compute is scarce; if ρ < ρ∗ delivered power is scarce. Eq. 3 extends the Roofline binding-constraint logic [22] from memory bandwidth to delivered data-center power, conditioned on (q ∗ , s∗ ). The regime classification depends on whether Kef f is measured as theoretical peak compute or as realized serving throughput, since memory stalls, insufficient batching, and utilization losses can move the same deployment between regimes. We therefore recommend a fixed 1 U and Φ system are not literally redundant because they are identified from different signals: U is estimated from real-time load (GPU SM activity, queue depth, request arrivals) and captures how much of the deployed capacity is actually in obs use; Φsystem is estimated from J/token relative to a physics-limited reference (eref tok /etok ) and captures how much energy the architecture wastes when fully loaded. A system can have high U (fully booked) and low Φsystem (architecturally wasteful), or vice versa; the two sources of inefficiency respond to different interventions (provisioning vs. algorithmic redesign).
3
Figure 1: The thermodynamics of token generation, illustrating how the Token Production Function converts physical resources (compute K and delivered power P ) into intelligence tokens through system-level optimizations Φsystem . The min(Kef f /ctok , PIT /etok ) constraint creates a “wooden barrel effect” where the limiting rate determines total output.
reporting convention: Kef f should default to realized effective serving throughput at the disclosed (q ∗ , s∗ ) operating point (with batching, context length, and energy-accounting boundary stated), and peak-throughput Kef f may be reported alongside as an upper-bound calibration only. Under this convention ρ − ρ∗ becomes a falsifiable diagnostic: a deployment whose realized ρ exceeds its workload ρ∗ at the stated operating point is, by construction, not power-bound. Appendix B works through a 65B-class anchor on H100 to show how the same hardware can be classified as power-bound under a peak-throughput denominator and effective-compute-bound under a realized-throughput denominator. As context lengths grow and KV-cache bandwidth dominates, ctok and etok shift together with the operating point, and regions with tight grid headroom enter the power-bound regime first. This constraint-switching logic explains why the same model family can appear effective-compute-bound in a well-powered, well-utilized campus and power-bound in a capacity-constrained region. When delivered power is the bottleneck, improvements that reduce measured etok expand effective capacity without additional infrastructure: a memory-efficiency gain that cuts J/token by 50% raises the power-side token ceiling under the same power cap without adding a single watt. What counts as a Φsystem gain. A gain only “counts” when it preserves the operating point: retrieval and reasoning quality must remain within disclosed tolerances of the reference (e.g., MMLU within ϵ and a longcontext benchmark such as RULER or IFEval within δ at the stated context length), latency must stay within the s∗ envelope, and reliability/freshness must not regress; gains that fail these checks shift the operating point and are not directly comparable. Under these fixed targets, inference papers should report not only accuracy, latency, throughput, and MFU, but also J/token, the active binding constraint, PUE-adjusted delivered power, and utilization-adjusted token output.
3
When Power Becomes the Binding Constraint
We use the Token Production Function as an interpretive lens to organize inference history into three epochs. Methodological note: throughout this paper, comparisons between API prices and regions are treated as directional association, not causal identification—posted prices are not normalized for quality, latency SLOs, context windows, caching, or subsidy strategies. Similarly, this section is a theoretical framework illustration, not an empirical validation: annual proxies for Pf acility and Kef f are mapped to public data [1, 23]; Φsystem is inferred qualitatively from documented step-changes. No causal claims are made anywhere in the paper unless explicitly stated. Figure 2 anchors a proxy for average Pf acility (t) to IEA annual electricity consumption (TWh/yr ÷ 8760 h/yr; Eq. 1 uses 4
Table 1: Order-of-magnitude anchors for Eq. 1 variables (2024–2026). Variable (unit)
Anchor value (source)
Kef f (FLOPs/s)
H100 peak: TF32 9.89 × 1014 , BF16 1.979 × 1015 ; 8×H100 BF16 ≈ 1.6 × 1016 [27, 28]. Global data centers: ≈ 47 GW in 2024 (415 TWh/yr), ≈ 108 GW in 2030 IEA central projection (945 TWh/yr) [1, 29]. Industry avg. ≈ 1.56; leading sites ≈ 1.08–1.09 [18]. 65B-regime anchor ≈ 3–4, with large workload/serving variance [19, 24, 25, 14]. Dense proxy ∼ 2 × 109 –∼ 8 × 1011 for 1B–405B models using ctok ≈ 2N [20]; for autoregressive decode under KV reuse, replace with weight-read + per-step attention FLOPs, and for MoE serving substitute Nactive for N . Observed range 0.3–0.7; batching, scheduling, and request mix dominate [24, 25].
Pf acility (avg. GW) P U E (ratio) etok (J/token) ctok (FLOPs/token) U
Table 2: Representative etok values for 65B-class LLM inference at a nominal (q ∗ , s∗ ) anchor (MMLU/IFEval-class quality, 100 ms latency). Rows A–C are measured from the cited independent sources under the listed configurations; row D is a projection composing the KV-compression batch headroom of DeepSeek-V2 [30] with the INT4 energy gains reported by [25], not a matched-stack measurement. Stack, workload mix, batching, and energy-accounting boundary differ across rows. Configuration
Implementation
etok (J)
Rel. MHA
A B C
Standard MHA + FP16 MHA + INT4 quant. MLA (KV compress.)
H100, batch=8 H100, batch=8 H100, batch=24
≈ 3.5 ≈ 1.2 ≈ 1.1
1.0× 0.34× 0.31×
measured [19, 24] measured [25] measured [30, 20]
D
MLA + INT4
H100, batch=24
≈ 0.35
0.10×
projection
Source
power, not energy). Epoch boundaries mark Φsystem step-changes that partially decoupled token output from energy growth. Table 1 gives order-of-magnitude calibration anchors [20, 24–26]. Directional calibration. Table 2 gathers representative etok values for 65B-class inference from independent sources; it is an illustrative compilation, not a single controlled head-to-head benchmark. Rows differ in serving stack and workload mix, and the 65B / 100 ms SLO framing is a nominal anchor rather than a normalized ceteris-paribus comparison. The table’s purpose is to show the direction and rough magnitude of Φsystem effects (architecture and quantization lower etok without expanding Kef f or Pf acility budgets), which is consistent with—though not a controlled test of—the claim that optimization acts as an energy multiplier. The measured A→C spread is ∼3×; the additional 3× implied by composing INT4 onto MLA (row D) is a projection. The framework’s conservative claim is that architecture-side Φsystem levers move PIT /etok by at least the measured 3×, with ∼10× plausible when quantization composes; a controlled cross-stack J/token benchmark closing this gap is what the reporting agenda calls for. 3.1
Epoch 1 (2020–2022): The Pre-Cambrian Era
In the early phase, both K(t) and P (t) were abundant relative to demand. GPT-3-scale models ran on concentrated clusters with Φsystem ≈ 1—no sophisticated memory management or scheduling. The field operated under scaling laws suggesting strong returns from parameters and compute [31–33]; energy costs were buried in operational budgets. 3.2
Epoch 2 (2023–2024): The LLM Explosion
ChatGPT triggered exponential K(t) growth [34, 23] alongside the first wave of Φsystem improvements. FlashAttention [35] reduced attention memory movement from O(N 2 ) to O(N ), lowering both ctok and etok ; PagedAttention/vLLM [36] enabled dynamic KV-cache allocation; INT4/INT8 quantization [37, 38] stretched K(t) within existing P (t) envelopes. Empirical runtime profiling of training, fine-tuning, and inference on commodity hardware confirmed early on that memory traffic, not raw FLOPs, dominates real-world LLM throughput [39]. API pricing remained relatively uniform—energy was not yet the binding constraint. 5
Epoch 3 Context War & Power Wall MLA NSA
Epoch 2 LLM Explosion FlashAttn vLLM INT4
22×
1000
945 TWh
800
101
600 400
415 TWh
200 0
projection begins
2020
2021
2022
Measured electricity (IEA 2025) IEA central projection
2023
2024
100
2025
Year
2026
IEA high scenario Projection uncertainty
2027
2028
2029
2030
Tokens per joule ( system, 2020 =1×, log)
Global data center electricity (TWh/yr)
Epoch 1 Pre-Cambrian dense attention 1
Tokens / joule ( system, illustrative, right axis)
Figure 2: Left axis (blue): global data center electricity (TWh/yr), 2020–2030 (IEA measured, central, and high scenarios with projection band) [1, 29]. Right axis (green, log): illustrative Φsystem proxy normalized to 2020=1, with step-changes anchored to documented system-level deployments (Epoch 2: FlashAttention, vLLM/PagedAttention, INT4/AWQ; Epoch 3: MLA, NSA, sparse-hybrid). Energy grows roughly linearly while Φsystem rises over an order of magnitude—tokens partially decouple from joules. The proxy is a qualitative visualization, not a fitted measurement; methodology and caveats are in §3. 3.3
Epoch 3 (2025–2026): The Context War and Power Wall
Context lengths have reached 1M+ tokens, motivating long-context generation benchmarks [40] for evaluation under sustained-output workloads, and the Power Wall has emerged as a binding constraint. Global data center electricity reached 415 TWh in 2024 and is projected to reach 945 TWh by 2030 [1, 29]; US data centers alone may reach 325–580 TWh by 2028 [2, 41]. US hyperscaler capex has grown ∼72%/yr since Q2 2023, exceeding $400 B in 2025 [42]; on the demand side, China reported ∼140 T daily token calls by March 2026 (∼1000× early 2024; ByteDance Doubao alone ∼120 T/day) [43, 44]. Some regions have hit the P (t) ceiling, and the API price divergence is consistent with this constraint divergence.
4
System Optimizations Are Energy Multipliers
Φsystem summarizes phase- and mechanism-level choices that can reduce ctok and etok under fixed quality/SLO and measurement assumptions. We examine two mechanisms through which micro-level engineering decisions can become macroeconomic energy levers, while treating reported speedups and energy reductions as configuration-dependent rather than universal constants. 4.1
Latent Compression Moves the Memory Boundary
KV-cache memory bandwidth is the dominant bottleneck in long-context inference: saturated HBM leaves compute units idle, wasting both CapEx and OpEx [45]. We use one publicly documented attention lineage to illustrate how memory-side Φsystem levers compose. DeepSeek-V2 introduced Multi-head Latent Attention (MLA) [30] for low-rank KV compression, and NSA added learned sparse selection [46]. The DeepSeek-V4 technical report [47] is one example of a hybrid compressionand-sparsity stack: Compressed Sparse Attention (CSA) compresses KV blocks before top-k selection, Heavily Compressed Attention (HCA) applies more aggressive compression with dense attention over the compressed representation, and these are layered with FP4-trained indexing, multi-head hybrid compression, and heterogeneous KV-cache placement across HBM, CPU memory, and SSD. The report targets 1M-token context serving and lists only ∼27% of V3.2 single-token FLOPs and ∼10% of V3.2 KV cache (developer report, pending third-party replication). Other production stacks combine subsets of the same levers—paged KV management in vLLM [36], FlashAttention IO 6
system effective energy multiplier (same wall-plug power, fixed (q *, s *))
1× baseline
DeepSeek-V2
2.5×
MLA
16× KV bandwidth compression
CSA / HCA
DeepSeek-V4
3.7×
10% KV cache at 1M context
MiniMax
4.0×
Linear (N) local + quadratic global heads
Hybrid Linear
projection (Table 1 row D)
10.0×
Composed: MLA + INT4
54% etok drop (MLA) × 2× INT4 gain
0× 1× 2×
4× 6× 8× 10× Throughput within same power envelope (normalized to 1× standard baseline)
Measured (single-source, configuration-dependent)
Projected (compositional upper anchor, not a head-to-head measurement)
Gains measured/projected at comparable quality and latency targets; actual values vary by workload and serving configuration.
Figure 3: Architectural efficiency comparison across optimization strategies. Bars summarize reported gains from heterogeneous systems papers and developer reports, not a unified head-to-head benchmark; KV-cache compression, sparse/heavily compressed attention, and hybrid attention are Φsystem levers only under fixed quality/SLO assumptions, since degraded retrieval/reasoning/reliability would make the resulting tokens incomparable. scheduling [35], eviction-based KV reduction [48–54], and offloaded inference [55]—and we cite this lineage as one observed instance, not as the recommended architecture. Compression counts as a production-function gain only if retrieval, reasoning, latency, and reliability remain within the fixed (q ∗ , s∗ ) envelope; under that constraint, the family of memory-side optimizations enables: 1. Higher batch sizes: more concurrent sequences within the same memory envelope, potentially increasing throughput per watt under comparable latency targets. 2. Million-token contexts: routinely supporting 1M-token inputs on hardware that would otherwise be memory-bound at far shorter sequence lengths. 3. Improved hardware utilization: reducing the time compute units spend stalled on memory transfers when memory traffic is the binding bottleneck. Prior work on semantic-preserving KV cache compression via eviction and offloading reports up to 50% cache reduction under task-specific quality constraints [48, 49, 55]; the DeepSeek lineage extends this with learned compression and sparse top-k selection. These methods compound Φmem , Φcomp , and Φpref ill only when the reduced cache preserves task-relevant evidence—compression that degrades retrieval is not a pure efficiency gain. Under comparable measurement assumptions the reported direction is an order-of-magnitude reduction in etok and ctok at million-token context. Appendix E gives a worked bandwidth derivation. Cross-vendor price evidence. As of April 2026, the tier-matched output-price gap between frontier Chinese reasoning Pro tiers ($1–$4/M) and frontier US Pro/Sonnet tiers ($12–$30/M) is roughly 5– 10×; the wider 3–30× envelope cited in some reports compares Flash-tier Chinese models to frontier US Opus/GPT-5 tiers and is therefore cross-tier, not like-for-like (Appendix F gives the per-vendor breakdown). The gap is consistent with infrastructure-level Φsystem differences shaping marginal economics, alongside quality, latency-SLO, and business-model variation; we do not attribute it causally to any single factor. 4.2
Sparse and Hybrid Attention Reduce Wasted Work
Dense attention can waste energy by applying O(N 2 ) effort uniformly even when only a subset of token interactions is task-relevant. Multiple lines of work attack this from different angles. Hardware-aligned sparse attention with dynamic chunk selection (e.g., NSA [46]) targets subquadratic long-context complexity; co-designed compression-plus-sparsity stacks (§4.1) push the same direction further by adding heavy compression, low-precision indexing, and heterogeneous KVcache placement. Hybrid linear/quadratic routing [56] sends different heads through O(N 2 ) or O(N ) 7
paths by reasoning need, and difficulty-adaptive token budgets [57] cut token output (22.4% reduction reported, no quality loss) by allocating compute by per-token entropy. Reported speedups (e.g., 6–11× for hardware-aligned sparse attention on 64K+ sequences [46]) are single-source and configurationdependent; we cite them as direction and rough magnitude rather than universal benchmarks. The unifying point is that compression, sparsity, routing, and adaptive computation all act as Φsystem levers that lower ctok and etok at fixed (q ∗ , s∗ ) [20, 25], regardless of vendor. Empirical studies of reasoning-LLM serving further show that long generations and adaptive depth dominate per-query energy under realistic SLOs [58], and the broader compression literature warns that downstream capability—including agentic execution [59] and other “lottery-ticket”-style preserved abilities [60]— depends on which mechanism the optimization preserves, so Φsystem gains must be reported jointly with the relevant (q ∗ , s∗ ) targets. Collectively, these Φsystem improvements can stretch P (t) to produce more quality-conditioned tokens per unit of delivered power. For energy-constrained sites they are therefore a central lever for maintaining capacity at fixed (q ∗ , s∗ ), independent of which specific stack is deployed.
5
Divergent Energy-to-Token Trajectories
The production function yields two stylized archetypes (not exhaustive country classifications; real ecosystems blend both): 5.1
Path A: Infrastructure-Constrained Trajectory
K(t) scales rapidly but P (t) is constrained by grid bottlenecks and legacy infrastructure (high PUE 1.5–2.0) [18]. Limited Φsystem investment means rising token prices as delivered power becomes binding. Outcome: premium tokens, frontier capability emphasis. 5.2
Path B: Efficiency-Optimized Trajectory
K(t) scales carefully while P (t) expands via renewable deployment, grid modernization, and regional corridor infrastructure [61–63]. Aggressive Φsystem maximization (MLA, CSA/HCA, and NSAstyle sparse attention) and low PUE (1.1–1.2) tend to support lower token prices under comparable quality/SLO targets. As an early directional signal from a routing platform rather than a global census, OpenRouter reports rapid growth in open-source and China-developed open-weight model token share, alongside heavy coding and agentic-workflow usage on a 100T-token sample [64]. Outcome: cost-efficient tokens, inference optimization emphasis. A simple strategic interpretation is that ecosystems first accumulate K/P /Φsystem capacity, then providers compete on price/latency/quality with marginal token cost M Citoken ≈ pei ·P U Ei ·etok,i +κi shaped by export controls, energy endowments, and sovereignty rules. Switching costs can turn early adoption into installed-base advantage, so divergence may persist even when posted prices are strategically set.
6
Alternative Views
“Hardware will make energy secondary.” Next-generation hardware (optical interconnects, advanced packaging, new substrates) will improve performance per watt [65, 18]. But hardware cycles span 18–36 months while model scale and context lengths move on 3–6 month product cycles [66, 23]: a 2× more efficient accelerator is absorbed by larger models, longer contexts, and higher request volumes. By Jevons Paradox [67, 68], efficiency gains also stimulate rebound—per-token prices for GPT-4-equivalent capability have fallen sharply since 2023 [69, 70] yet aggregate token consumption has expanded faster. “Renewables and grid expansion will dissolve the Power Wall.” Aggressive renewable buildout, transmission upgrades, and modular nuclear can in principle relax Pf acility [1, 2], but operate on the wrong time constant: grid-scale additions clear permitting and construction over 5–10 years while LLM release cycles measure 3–6 months [42]. Even when new generation lands, it does not directly relax P U E, cooling, on-rack utilization, or routing/queueing inefficiencies—all of which Φsystem governs. The position is complementary, not opposed, to renewable scale-up: Φsystem determines how many quality-conditioned tokens each new megawatt actually produces. 8
Token cost index (normalized, log scale)
10×
6.3×
5.0×
current (Apr 2026)
K grows fast · P grid-bound · high PUE
2.0×
3× 30× listed-price spread (Apr 2026)
1.0×
38× stylized divergence
0.5× 0.2×
system maximized · MLA · CSA/HCA · low PUE
0.14×
0.1× 2024
2025
2026
Path A: Infrastructure-constrained (CapEx-driven)
2027
Year
2028
2029
Path B: Efficiency-optimized (OpEx + system)
2030 Divergence envelope
Stylized archetypes; real ecosystems may combine features of both. Curves anchored to the 3×--30× listed-price spread observed across vendor tiers (Apr 2026, Table 4).
Figure 4: Divergent trajectories of AI ecosystem archetypes. Path A (infrastructure-constrained) tends toward higher token costs when power/cooling bind; Path B (efficiency-optimized) leverages Φsystem for lower-cost tokens despite tighter compute supply. Curves are stylized and anchored to the 3×–30× listed-price spread observed in April 2026 across vendor tiers (Appendix F); the 2030 endpoint is illustrative, not a forecast. “Silicon access determines competitiveness.” Peak silicon matters but is not the only production input. Ecosystems optimizing Φsystem have narrowed capability and cost gaps even under tighter silicon access [65], while electricity-price differentials [71], PUE, grid headroom, scheduling, and routing all shape the delivered cost of tokens. The production-function view does not deny hardware scarcity—it explains why the same silicon budget yields different quality-conditioned token output under different power and system-efficiency regimes. “Vertical integration hides the cost signal.” Hyperscaler custom silicon (TPU, Trainium, Maia) and cross-subsidized APIs can decouple posted prices from marginal cost—which is why we treat API prices as directional motivation only. Vertical integration internalizes the production constraint without removing it: TPU clusters still require delivered electricity, cooling, interconnect, and utilization, so the framework operates at the infrastructure layer where physical constraints persist. “Demand elasticity will erase cost advantages.” Tiered pricing can compete away some energy-cost advantage at the margin [70], but the token market is segmented by API lock-in, migration costs, and compliance constraints [72]. A persistent 2–3× cost advantage shifts market share at the extensive margin even when incumbent workloads remain sticky; elasticity changes how production advantages are monetized but not the underlying physical advantage. “Tokens are not homogeneous.” Quality heterogeneity is real [24, 25], and posted prices are not marginal costs [70]. The proposed reporting standard is therefore not raw tokens per joule but J/token at fixed (q ∗ , s∗ ) with workload, batching, hardware, and energy-accounting boundary disclosed; without those controls token counts are not comparable, with them energy-to-token production becomes measurable.
7
Conclusion and Call to Action
Scope. The Leontief min(·, ·) in Eq. 1 is a short-run binding-constraint approximation, not a structural macro model; the etok anchors are directional under six disclosed measurement dimensions, not ceteris-paribus benchmarks; and ρ − ρ∗ depends on the Kef f convention. Each scoping choice is a feature: every dimension a reviewer asks us to hold fixed is one our reporting agenda already requires authors and benchmarks to disclose. Appendix A elaborates. Position summary. The binding constraint on LLM inference can shift from compute K toward delivered power P , cooling, and utilization; Φsystem optimizations expand capacity without infrastructure expansion; and by Jevons Paradox hardware alone cannot escape the Power Wall. The ML community must elevate “Joules per Token” to first-class evaluation status. Concretely: 9
• Papers and benchmarks should report J/token, the active binding constraint, PUE-adjusted power, and utilization at disclosed (q ∗ , s∗ ) alongside accuracy and latency. • Conferences and leaderboards should add energy-normalized tracks, e.g., MLPerf Power [15] extended to LLM serving. • Funders, operators, and reviewers should treat Φsystem -shifting work as first-class contributions and the absence of ρ, PUE, and Φmem disclosures as a reviewable gap, not a stylistic preference.
References [1] International Energy Agency. Energy and ai. Technical report, IEA Special Report, 2025. URL https://www.iea.org/reports/energy-and-ai. [2] Electric Power Research Institute. Analyzing artificial intelligence and data center energy consumption. Technical Report 3002028905, EPRI, 2024. URL https://www.epri.com/r esearch/products/3002028905. EPRI White Paper No. 3002028905. [3] NVIDIA. Ai factories. NVIDIA solutions page, 2026. URL https://www.nvidia.com/e n-us/solutions/ai-factories/. [4] NVIDIA. Ai inference. NVIDIA solutions page, 2026. URL https://www.nvidia.com/e n-us/solutions/ai/inference/. [5] OpenAI. Api pricing. OpenAI documentation, 2026. URL https://openai.com/api/pri cing/. [6] Anthropic. Models overview and api pricing. Anthropic documentation, 2026. URL https: //docs.anthropic.com/en/docs/models-overview. [7] DeepSeek. Models and pricing. DeepSeek API documentation, 2026. URL https://api-d ocs.deepseek.com/quick_start/pricing. [8] R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni. Green ai. Communications of the ACM, 63 (12):54–63, 2020. doi: 10.1145/3381831. URL https://doi.org/10.1145/3381831. [9] D. Patterson, J. Gonzalez, Q. Le, C. Liang, L. M. Munguia, D. Rothchild, J. Dean, et al. Carbon emissions and large neural network training, 2021. URL https://arxiv.org/abs/2104.1 0350. arXiv preprint arXiv:2104.10350. [10] D. Patterson, J. Gonzalez, U. Hölzle, Q. Le, C. Liang, L.-M. Munguia, J. Dean, et al. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18–28, 2022. doi: 10.1109/MC.2022.3148714. URL https://doi.org/10.1109/MC.2022.3148714. [11] C. J. Wu, R. Raghavendra, U. Gupta, B. Acun, N. Ardalani, K. Maeng, K. Hazelwood, et al. Sustainable ai: Environmental implications, challenges and opportunities. In Proceedings of Machine Learning and Systems (MLSys), volume 4, pages 795–813, 2022. URL https: //arxiv.org/abs/2111.00364. [12] A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres. Quantifying the carbon emissions of machine learning, 2019. URL https://arxiv.org/abs/1910.09700. arXiv preprint arXiv:1910.09700. [13] E. Strubell, A. Ganesh, and A. McCallum. Energy and policy considerations for deep learning in nlp. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3645–3650, 2019. URL https://arxiv.org/abs/1906.02243. [14] A. S. Luccioni, Y. Jernite, and E. Strubell. Power hungry processing: Watts driving the cost of ai deployment? In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2024. doi: 1 0 . 1 1 4 5 / 3 6 3 0 1 0 6 . 3 6 5 8 5 4 2. URL h t t p s : //doi.org/10.1145/3630106.3658542. [15] MLCommons. Mlperf inference v4.1 power results. Technical report, MLCommons, 2024. URL https://mlcommons.org/benchmarks/inference-datacenter/. MLCommons Technical Report. 10
[16] W. W. Leontief. The Structure of American Economy, 1919–1929: An Empirical Application of Equilibrium Analysis. Harvard University Press, 1941. [17] K. J. Arrow, H. B. Chenery, B. S. Minhas, and R. M. Solow. Capital-labor substitution and economic efficiency. The Review of Economics and Statistics, 43(3):225–250, 1961. doi: 10.2307/1927286. URL https://doi.org/10.2307/1927286. [18] Uptime Institute. 2024 global data center survey results. Technical report, Uptime Institute, 2024. Global average PUE: 1.56; industry leaders: 1.08–1.09. [19] S. Samsi, D. Zhao, J. McDonald, B. Li, A. Michaleas, M. Jones, J. Kepner, et al. From words to watts: Benchmarking the energy costs of large language model inference, 2023. URL https://arxiv.org/abs/2310.03003. arXiv preprint arXiv:2310.03003. [20] C. Niu, W. Zhang, J. Li, Y. Zhao, T. Wang, X. Wang, Y. Chen, et al. Tokenpowerbench: Benchmarking the power consumption of llm inference, 2025. URL https://arxiv.org/ab s/2512.03024. arXiv preprint arXiv:2512.03024. [21] R. M. Solow. Technical change and the aggregate production function. Review of Economics and Statistics, 39(3):312–320, 1957. doi: 10.2307/1926047. URL https://doi.org/10.2 307/1926047. [22] S. Williams, A. Waterman, and D. Patterson. Roofline: An insightful visual performance model for multicore architectures. Communications of the ACM, 52(4):65–76, 2009. doi: 10.1145/1498765.1498785. URL https://doi.org/10.1145/1498765.1498785. [23] J. Sevilla and E. Roldán. Training compute of frontier ai models grows by 4-5x per year. Epoch AI Blog, 2024. URL https://epochai.org/blog/training-compute-of-frontier-a i-models-grows-by-4-5x-per-year. [24] J.-W. Chung, R. Wu, J. J. Ma, and M. Chowdhury. Where do the joules go? diagnosing inference energy consumption, 2026. URL https://arxiv.org/abs/2601.22076. arXiv preprint arXiv:2601.22076. [25] J. Delavande, R. Pierrard, and S. Luccioni. Understanding efficiency: Quantization, batching, and serving strategies in llm energy use, 2026. URL https://arxiv.org/abs/2601.22362. arXiv preprint arXiv:2601.22362. [26] H. P. Cavagna, A. Proia, G. Madella, G. B. Esposito, F. Antici, D. Cesarini, Z. Kiziltan, and A. Bartolini. Sweetspot: An analytical model for predicting energy efficiency of llm inference, 2026. URL https://arxiv.org/abs/2602.05695. arXiv preprint arXiv:2602.05695. [27] NVIDIA. Nvidia h100 tensor core gpu: Product specifications. NVIDIA product page, 2026. URL https://www.nvidia.com/en-us/data-center/h100/. [28] NVIDIA. Nvidia hgx platform specifications (hgx h100 4/8-gpu). NVIDIA product page, 2026. URL https://www.nvidia.com/en-us/data-center/hgx. [29] International Energy Agency. Ai is set to drive surging electricity demand from data centres while offering the potential to transform how the energy sector works. IEA News, 2025. URL https://www.iea.org/news/ai-is-set-to-drive-surging-electricity-deman d-from-data-centres-while-offering-the-potential-to-transform-how-the -energy-sector-works. [30] DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. Technical report, DeepSeek-AI, 2024. URL https://arxiv.org/abs/2405.04434. arXiv:2405.04434. [31] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, D. Amodei, et al. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361. arXiv preprint arXiv:2001.08361. [32] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, L. Sifre, et al. Training compute-optimal large language models, 2022. URL https://arxiv.org/abs/22 03.15556. arXiv preprint arXiv:2203.15556. 11
[33] J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, G. Irving, et al. Scaling language models: Methods, analysis and insights from training gopher, 2021. URL https: //arxiv.org/abs/2112.11446. arXiv preprint arXiv:2112.11446. [34] J. Sevilla, L. Heim, A. Ho, T. Besiroglu, M. Hobbhahn, and P. Villalobos. Compute trends across three eras of machine learning. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2022. URL https://arxiv.org/abs/2202.05924. [35] T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems, volume 35, pages 16344–16359, 2022. URL https://arxiv.org/abs/2205.14135. [36] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, I. Stoica, et al. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), pages 611–626, 2023. URL https: //arxiv.org/abs/2309.06180. [37] E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. In Proceedings of the 11th International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2210.17323. [38] J. Lin, J. Tang, H. Tang, S. Yang, X. Dang, and S. Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. In Proceedings of Machine Learning and Systems, volume 6, 2024. URL https://arxiv.org/abs/2306.00978. [39] Longteng Zhang, Xiang Liu, Zeyu Li, Xinglin Pan, Peijie Dong, Ruibo Fan, Rui Guo, Xin Wang, Qiong Luo, Shaohuai Shi, and Xiaowen Chu. Dissecting the runtime performance of the training, fine-tuning, and inference of large language models. arXiv preprint arXiv:2311.03687, 2023. URL https://arxiv.org/abs/2311.03687. [40] Xiang Liu, Peijie Dong, Xuming Hu, and Xiaowen Chu. LongGenBench: Long-context generation benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 865–883. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024 .findings-emnlp.48. URL https://doi.org/10.18653/v1/2024.findings-emnlp.48. [41] U.S. Department of Energy. Doe releases new report evaluating increase in electricity demand from data centers. Technical report, U.S. Department of Energy, 2024. URL https://www.en ergy.gov/articles/doe-releases-new-report-evaluating-increase-electrici ty-demand-data-centers. DOE News. [42] I. Juniewicz. Hyperscaler capex has quadrupled since gpt-4’s release. Epoch AI Data Insights, 2026. URL https://epoch.ai/data- insights/hyperscaler- capex- trend/. Combined Alphabet, Amazon, Meta, Microsoft, and Oracle capex extracted from SEC EDGAR 10-Q/10-K filings. [43] L. Liu. Speech at the china development forum 2026: Token (“ciyuan”) as the value anchor of the intelligent era; daily token-call volume in china exceeds 140 trillion as of march 2026. National Data Administration of China, 2026. URL https://www.nda.gov.cn/sjj/swdt/ mtsy/0325/20260325113132934906079_pc.html. [44] TechNode. Doubao surpasses 120 trillion daily tokens as usage doubles in three months. TechNode, 2026. URL https://technode.com/2026/04/07/doubao-surpasses-120 -trillion-daily-tokens-as-usage-doubles-in-three-months/. [45] W. A. Wulf and S. A. McKee. Hitting the memory wall: Implications of the obvious. ACM SIGARCH Computer Architecture News, 23(1):20–24, 1995. doi: 10.1145/216585.216588. URL https://doi.org/10.1145/216585.216588. [46] J. Yuan, H. Gao, D. Dai, J. Luo, L. Zhao, Z. Zhang, Z. Xie, Y. X. Wei, L. Wang, Z. Xiao, Y. Wang, C. Ruan, M. Zhang, W. Liang, W. Zeng, et al. Native sparse attention: Hardwarealigned and natively trainable sparse attention. In Proceedings of ACL 2025, 2025. URL https://arxiv.org/abs/2502.11089. Best Paper; arXiv:2502.11089. 12
[47] DeepSeek-AI. Deepseek-v4: Towards highly efficient million-token context intelligence. Technical report, DeepSeek-AI, 2026. URL https://huggingface.co/deepseek-ai/De epSeek-V4-Pro/blob/main/DeepSeek_V4.pdf. Technical report. [48] X. Liu, Z. Tang, P. Dong, Z. Li, Y. Liu, B. Li, X. Hu, and X. Chu. Chunkkv: Semanticpreserving kv cache compression for efficient long-context llm inference. In Advances in Neural Information Processing Systems (NeurIPS) 39, 2025. URL https://arxiv.org/abs/2502 .00299. arXiv:2502.00299. [49] Z. Zhang, Y. Sheng, T. Zhou, T. Chen, L. Zheng, R. Cai, B. Chen, et al. H2 o: Heavy-hitter oracle for efficient generative inference of large language models. In Advances in Neural Information Processing Systems, volume 36, 2023. URL https://arxiv.org/abs/2306.14048. [50] Xiang Liu, Hong Chen, Xuming Hu, and Xiaowen Chu. FlowKV: Enhancing multi-turn conversational coherence in LLMs via isolated key-value cache management. In NeurIPS Workshop on Multi-Turn Interactions in Large Language Models, 2025. URL https://arxiv. org/abs/2505.15347. [51] Xiang Liu, Zhenheng Tang, Hong Chen, Peijie Dong, Zeyu Li, Xiuze Zhou, Bo Li, Xuming Hu, and Xiaowen Chu. Semantic integrity matters: Benchmarking and preserving high-density reasoning in KV cache compression. In International Conference on Machine Learning (ICML), 2026. URL https://arxiv.org/abs/2502.01941. [52] Hong Chen, Xiang Liu, Bo Wang, Yuxuan Fan, Yuanlin Chu, Zongluo Li, Xiaowen Chu, and Xuming Hu. SONIC: Segmented optimized nexus for information compression in key-value caching. arXiv preprint arXiv:2601.21927, 2026. URL https://arxiv.org/abs/2601.2 1927. [53] Zeyu Li, Chuanfu Xiao, Yang Wang, Xiang Liu, Zhenheng Tang, Baotong Lu, Mao Yang, Xinyu Chen, and Xiaowen Chu. AnTKV: Anchor token-aware sub-bit vector quantization for KV cache in large language models. arXiv preprint arXiv:2506.19505, 2025. URL https: //arxiv.org/abs/2506.19505. [54] Yuanbing Zhu, Zhenheng Tang, Xiang Liu, Ang Li, Bo Li, Xiaowen Chu, and Bo Han. OracleKV: Oracle guidance for question-independent KV cache eviction. In ICML Workshop on Long-Context Foundation Models, 2025. URL https://openreview.net/pdf?id=KHM2YO GgX9. [55] Y. Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, B. Chen, I. Stoica, et al. Flexgen: Highthroughput generative inference of large language models with a single gpu. In Proceedings of the 40th International Conference on Machine Learning (ICML), pages 31094–31116, 2023. URL https://arxiv.org/abs/2303.06865. [56] MiniMax, A. Li, B. Gong, B. Yang, B. Shan, et al. Minimax-01: Scaling foundation models with lightning attention, 2025. URL https://arxiv.org/abs/2501.08313. arXiv preprint arXiv:2501.08313. [57] X. Liu, X. Hu, X. Chu, and E. Choi. Diffadapt: Difficulty-adaptive reasoning for token-efficient llm inference, 2025. URL https://arxiv.org/abs/2510.19669. arXiv preprint arXiv:2510.19669. [58] Qi Li, Junpan Wu, Xiang Liu, Yuxin Wang, Zeyu Li, Zhenheng Tang, Yuhan Chen, Shaohuai Shi, and Xiaowen Chu. Reasoning language model inference serving unveiled: An empirical study. In International Conference on Learning Representations (ICLR), 2026. URL https: //arxiv.org/abs/2510.18672. [59] Peijie Dong, Zhenheng Tang, Xiang Liu, Lujun Li, Xiaowen Chu, and Bo Li. Can compressed LLMs truly act? an empirical evaluation of agentic capabilities in LLM compression. In International Conference on Machine Learning (ICML), 2025. URL https://arxiv.org/ab s/2505.19433. [60] Zhenheng Tang, Xiang Liu, Qian Wang, Peijie Dong, Bingsheng He, Xiaowen Chu, and Bo Li. The lottery LLM hypothesis, rethinking what abilities should LLM compression preserve? In ICLR Blogposts Track, 2025. URL https://arxiv.org/abs/2502.17535. 13
[61] National Energy Administration. National energy administration releases 2024 national electric power industry statistics. National Energy Administration of China, 2025. URL https: //www.nea.gov.cn/20250121/097bfd7c1cd3498897639857d86d5dac/c.html. [62] Ministry of Industry and Information Technology, State Administration for Market Regulation, and National Energy Administration. Interpretation of the work plan for stabilizing growth in the power equipment industry (2025–2026). Technical report, Ministry of Industry and Information Technology, 2025. URL https://www.miit.gov.cn/zwgk/zcjd/art/2025 /art_44b08c84c6f84feeab80d004460f1003.html. [63] Infocomm Media Development Authority and Singapore Economic Development Board. Launch of second data centre – call for application. Technical report, IMDA / EDB, 2025. URL https://www.imda.gov.sg/resources/press-releases-factsheets-and-speeche s/factsheets/2025/launch-of-second-data-centre. IMDA / EDB Factsheet. [64] OpenRouter. State of ai: token-usage rankings, q1 2026. OpenRouter, 2026. URL https: //openrouter.ai/state-of-ai. [65] Stanford Institute for Human-Centered Artificial Intelligence. 2025 ai index report: Ai model performance gaps narrowing, compute costs plummeting. Technical report, Stanford HAI, 2025. URL https://aiindex.stanford.edu/report/. [66] N. C. Thompson, K. Greenewald, K. Lee, and G. F. Manso. The computational limits of deep learning, 2020. URL https://arxiv.org/abs/2007.05558. arXiv preprint arXiv:2007.05558. [67] W. S. Jevons. The Coal Question: An Inquiry Concerning the Progress of the Nation, and the Probable Exhaustion of Our Coal-Mines. Macmillan and Co., London, 1865. [68] S. Sorrell. Jevons’ paradox revisited: The evidence for backfire from improved energy efficiency. Energy Policy, 37(4):1456–1469, 2009. doi: 10.1016/j.enpol.2008.12.003. URL https: //doi.org/10.1016/j.enpol.2008.12.003. [69] G. Appenzeller. Welcome to llmflation: Llm inference cost is going down fast. Andreessen Horowitz, 2024. URL https://a16z.com/llmflation-llm-inference-cost/. [70] M. Demirer, A. Fradkin, N. Tadelis, and S. Peng. The emerging market for intelligence: pricing, supply, and demand for llms. Technical Report 34608, National Bureau of Economic Research, 2025. URL https://www.nber.org/papers/w34608. NBER Working Paper No. 34608. [71] BusinessEurope. High cost of energy: industrial electricity prices in the eu vs the us and china. BusinessEurope Data Hub, 2024. URL https://www.businesseurope.eu/media-room/ data-hub/high-cost-of-energy/. [72] C. Shapiro and H. R. Varian. Information Rules: A Strategic Guide to the Network Economy. Harvard Business School Press, 1999. [73] U.S. Energy Information Administration. Electric power monthly, table 5.6.b: Average price of electricity to ultimate customers by end-use sector, by state (december 2025 ytd). Technical report, U.S. Energy Information Administration, 2026. URL https://www.eia.gov/electr icity/monthly/epm_table_grapher.php?t=epmt_5_6_b. [74] E. Wu. Sovereignty and data localization. Technical report, Belfer Center for Science and International Affairs, Harvard Kennedy School, 2021. URL https://www.belfercenter.o rg/publication/sovereignty-and-data-localization. [75] P. Villalobos, A. Ho, J. Sevilla, T. Besiroglu, L. Heim, and M. Hobbhahn. Will we run out of data? limits of llm scaling based on human-generated data, 2022. URL htt ps: //arxiv.org/abs/2211.04325. arXiv preprint arXiv:2211.04325; published at ICML 2024. [76] Xiaomi MiMo. Xiaomi mimo api open platform. Xiaomi MiMo platform, 2026. URL https://platform.xiaomimimo.com/. 14
[77] Z.ai (Zhipu AI). Z.ai developer documentation: pricing overview. Z.ai documentation, 2026. URL https://docs.z.ai/guides/overview/pricing. [78] Moonshot AI. Kimi api platform: model inference pricing. Moonshot AI documentation, 2026. URL https://platform.kimi.ai/docs/pricing/chat. [79] Google. Gemini developer api pricing. Google AI documentation, 2026. URL https: //ai.google.dev/gemini-api/docs/pricing.
15
A
Scope, Limitations, and What This Paper Does Not Claim
We list each limitation as already-bounded by the paper rather than as an unaddressed gap, so that anticipated reviewer concerns are met by design rather than patched by rebuttal. Binding-constraint approximation, not a structural macro model. The Leontief min(·, ·) form in Eq. 1 is chosen as a short-run binding-constraint analysis lens, not a long-run substitution model. We do not claim that compute and delivered power are non-substitutable in general; the CES family [17] nests Leontief as σ → 0 and is the appropriate generalization once packaging, photonics, and ondie memory move substitution elasticities into measurable range. The min operator gives sharp predictions about which factor binds in a given measurement window; it does not predict equilibrium token output, equilibrium prices, or country-level capability outcomes. Reviewers searching for a structural prediction will not find one—by design—and a request to swap in CES is consistent with, not contrary to, our framework. Directional anchors, not a controlled benchmark. Tables 2 and 3 are explicitly labeled as illustrative compilations from independent sources, conditioned on six disclosed measurement dimensions (q ∗ , s∗ , workload mix, batching protocol, hardware setup, energy-accounting boundary). The ∼10× spread is a directional upper bound under those dimensions, not a ceteris-paribus result. A single matched cross-stack J/token benchmark is exactly what the paper’s reporting agenda calls for; performing it is future work, not a deficit. The contribution of a position paper is to argue what should be measured; the controlled measurement is the next paper, and the proposed leaderboard standards are designed to make that measurement comparable. ρ − ρ∗ is convention-dependent, and we say so. Appendix B walks through an H100 numerical example showing that the same accelerator can be classified as power-bound under a peak-throughput denominator and effective-compute-bound under a realized-throughput denominator. Our reporting agenda explicitly requires disclosure of the Kef f measurement convention precisely because of this dependence. The diagnostic is meant to be reproducible only when both the Kef f convention and (q ∗ , s∗ ) are stated; isolated J/token numbers without that scaffolding are, by construction, not comparable. Out of scope by design. We do not predict geopolitical outcomes, capability rankings, or which ecosystem “wins.” We do not treat API prices as causal evidence of marginal cost; price dispersion is used as directional motivation only (§5, §6). We do not address training-time energy except where serving-side Φsystem amortizes training cost across more tokens; carbon accounting for training is well-developed in prior work [9–12] and we do not attempt to redo it. Why these caveats strengthen rather than weaken the position. Each caveat is also surfaced inside the main text: the Leontief choice in §2; the directional-only labeling of price evidence in §6; the regime-flip example in Appendix B; the (q ∗ , s∗ )-conditioning of every comparison in §3–§4. The framework is constructed so that a stricter caveat tightens the position rather than relaxes it: every dimension a reviewer asks us to control is a dimension the proposed reporting agenda already requires authors and benchmarks to disclose. The position therefore becomes more defensible as the measurement bar rises.
B
Worked Example: ρ − ρ∗ on H100
This appendix expands the constraint-boundary diagnostic in §2 (Eq. 3) with a concrete numerical anchor. The point is to show how the same accelerator can be classified as power-bound or effectivecompute-bound depending on how Kef f is measured, not to argue that one regime is universally correct. For a dense-attention decoding workload on 65B-class models, the workload-side energy intensity is ρ∗ ≈
etok 3.5 J ≈ ≈ 9 pJ/FLOP, ctok 4 × 1011 FLOPs
using the Table 1 anchors etok ≈ 3.5 J/token and ctok ≈ 2N ≈ 4×1011 FLOPs/token at N ≈ 2×1011 . An H100 GPU at ∼700 W and ∼ 1015 peak BF16 FLOPs/s gives a facility-side ratio ρ ≡ PIT /Kef f ≈ 0.7 pJ/FLOP 16
under a peak-throughput denominator. Since ρ < ρ∗ , this measurement convention classifies the deployment as power-bound: the workload demands more joules per FLOP than the accelerator is delivering at peak. If Kef f is instead measured as realized effective serving throughput, memory stalls, insufficient batching, kernel-launch overhead, and utilization losses cut the denominator. A 5–10× reduction (typical at long context with KV-cache pressure) raises the realized ρ into the same order of magnitude as ρ∗ , and the same hardware now classifies as effective-compute-bound at that operating point. The takeaway is that ρ − ρ∗ is a function of measurement convention as well as physics: reporting whether Kef f is peak or realized, and at what context length, batch size, and quality target, is necessary for the diagnostic to be reproducible across studies.
C
Token Export: The Invisible Commodity Flow
The cross-vendor price divergence in Appendix F suggests that cross-border digital services increasingly inherit the cost structure of local electricity and infrastructure constraints. Industrial electricity tariffs differ by 2–3× across major industrial regions [71], and even within the United States state-level tariffs show large cross-state dispersion [73]; together with PUE and utilization variation, this locational pricing materially shapes the marginal cost of producing a token. We treat these pricing patterns as consistent with the binding-constraint divergence in the main text rather than as a clean identification result. Developer Lock-in and Market Structure. Once a developer architects their application around a particular model’s API, the migration cost grows super-linearly with system complexity [72]. Models from energy-optimized ecosystems have captured significant market share on major routing platforms, suggesting that Φsystem advantages can offset K(t) constraints. Each developer integrated into the ecosystem represents future token demand locked into that infrastructure. Data Sovereignty as Trade Barrier. The localization/regulatory component of U (t) represents data sovereignty as a trade barrier [74]. As regions impose stricter data localization requirements, the global token market fragments into national or regional pools, reinforcing the advantage of ecosystems with domestic energy abundance.
D
Token Abundance, Verification, and Value
Epoch AI projections indicate that high-quality human-generated text data may become increasingly scarce relative to model scale during the 2026–2028 window [75]. Combined with rapid growth in inference capacity, this suggests a future where machine-generated tokens are abundant while trusted, human-curated information remains comparatively scarce. In that regime, scarcity shifts toward verification, curation, provenance, and quality assurance. The operational question moves from “how cheaply can we generate tokens?” to also “how reliably can we filter, validate, and route them?” The energy-to-token conversion metrics we propose remain central even if generation costs fall: systems optimized for Φsystem retain an advantage because they can support both generation and the growing overhead of validation at scale.
E
MLA Worked Example: Bandwidth Derivation
This appendix provides the detailed bandwidth derivation for the MLA case study in Section 4.1, and extends the empirical validation from Table 2 in the main body. In standard multi-head attention (MHA) decoding, each generated token must read keys and values for all prior context positions: roughly 2 · L · nheads · dhead · b bytes per layer, where L is context length and b is bytes per element. For a 65B-class model with L = 1024, nheads = 64, dhead = 128, and FP16 precision (b = 2), this yields approximately 32 MB per layer per decoding step; at H100 HBM bandwidth (∼3.35 TB/s), that single-layer traffic would cap decoding at roughly 105 tokens/s before accounting for all layers, batching, cache layout, and compute. MLA replaces this with a low-rank latent of dimension dc ≪ nheads · dhead ; DeepSeek-V2 uses dc = 512, reducing KV cache bandwidth by approximately 64 × 128/512 ≈ 16× relative to full 17
Table 3: Representative etok values across selected configurations (65B regime, nominal anchor). Rows are visually grouped: measured (A–D, G, I) come from independent published measurements; projection (E, F, J, K) compose independently reported architectural and quantization gains; developer-report only (H) gives relative compute and KV-cache reductions, pending third-party replication. Distillation rows (J, K) trade off model capability and are listed last to flag the explicit quality dimension. Configuration
Implementation
Measured rows (independent sources) A MHA FP16 baseline H100, async batch B MHA INT8 H100, GPTQ C MHA INT4 H100, AWQ D MLA FP16 H100, low-rank KV G NSA (sparse) DeepSeek NSA, attn mask I Hybrid linear MiniMax routing
etok (J)
Batch
Source
3.5 2.1 1.2 1.6 1.8
8 8 8 24 16
measured [19, 24] measured [37, 25] measured [38, 25] measured [30, 20] measured [46]
1.2
20
measured [56]
Projection rows (compose independent gains, not matched-stack measurements) E MLA INT8 H100, MLA+GPTQ 0.95 24 projection F MLA INT4 H100, MLA+AWQ 0.35 24 projection J Distilled 13B 13B student model 0.28 32 projection (capability tradeoff) K Distilled 7B 7B model, MLA+INT4 0.12 64 projection (capability tradeoff) Developer-report only (pending third-party replication) H CSA/HCA + mHC V4-Pro, 1M context relative
–
27% FLOPs / 10% KV [47]
MHA at the same head count [30]. In production-function terms, this maps to a reduction in etok through the Φmem mechanism: lower HBM traffic per token means fewer watt-seconds per token at the same compute utilization. Empirically, this enables 2–3× higher sustainable batch sizes within the same power envelope, which under fixed PIT raises Q̇token by the same factor. A data center limited to 100 MW IT load can thus produce 2–3× more intelligence tokens per hour when deploying MLA-optimized models versus standard MHA—without adding a single watt of infrastructure. The foundational semantic-preserving eviction literature (ChunkKV [48], H2O [49]) demonstrated that attention patterns exhibit persistence across generation steps, enabling dynamic eviction policies that reduce KV cache size by up to 50% without quality degradation. MLA extends these principles through learned compression. E.1
Extended Directional Comparison
Table 3 gathers a broader set of representative configurations from independent measurements and vendor reports to illustrate the direction and rough magnitude of Φsystem variation across implementation choices. Rows A–F are the closest like-for-like comparison under the 65B / 100 ms nominal anchor; rows G–H are technical anchors for sparse and V4 long-context stacks; rows I–K are alternative mechanisms or capability-tradeoff projections and should not be folded into the same controlled spread claim. As with Table 2, the table is illustrative rather than ceteris-paribus. Rows marked “projection” compose independently reported architectural and quantization gains and should be read as back-of-the-envelope estimates. Qualitative patterns visible in Table 3: 1. Quantization alone (A→C): etok drops by ∼66% but batch sizes are largely unchanged— the KV cache remains the bandwidth bottleneck. 2. MLA without quantization (A→D): etok drops by ∼54% and batch size roughly triples, consistent with the KV-compression headroom reported by [30]. 18
Table 4: Listed API prices for frontier LLMs (USD per million tokens, cache-miss input and output prices where applicable, late April 2026; context windows differ, directional and not normalized). Vendor / Model
Origin
Input $/M
Output $/M
DeepSeek-V4-Pro (promo to 2026-05-31) DeepSeek-V4-Pro (regular) DeepSeek-V4-Flash Xiaomi MiMo-V2.5-Pro Xiaomi MiMo-V2.5-Flash GLM-5.1 (Z.ai) Kimi K2.6 (Moonshot)
China [7] China [7] China [7] China [76] China [76] China [77] China [78]
0.44 1.76 0.15 1.00 0.10 1.05 0.95
0.88 3.51 0.29 3.00 0.30 3.50 4.00
Gemini 3.1 Pro (≤200K) Claude Sonnet 4.6 Claude Opus 4.6 GPT-5.5
US [79] US [6] US [6] US [5]
2.00 3.00 5.00 5.00
12.00 15.00 25.00 30.00
3. MLA + INT4 projection (A→F): composing the two mechanisms projects a ∼10× reduction; this is an extrapolation, not a direct measurement. 4. V4 long-context stack (G→H): the developer report gives relative compute and KV-cache reductions at 1M-token context rather than a normalized J/token measurement, so this row should be read as mechanism evidence, not an energy benchmark. 5. Distillation (A→J,K): yields a further 10–35× reduction but trades off model capability and is listed separately to flag the explicit quality dimension. These variations are consistent with—not a controlled test of—the Φsystem decomposition: rows A–F alone show roughly a 10× spread in etok within the same nominal hardware envelope (Pf acility , Kef f ); including the capability-tradeoff projections J and K widens the illustrative range toward 30×. The Token Production Function’s constraint-boundary analysis (Eq. 3) suggests which configurations are likely to be favoured under tight power budgets: power-bound sites will gravitate toward the lowest-etok rows at fixed quality (F is the capability-preserving frontier; J and K trade capability for further energy gains). This directional evidence is consistent with the paper’s central claim that algorithmic optimizations (Φsystem ) function as macro-level energy levers without infrastructure expansion; a full controlled benchmark with matched q ∗ , s∗ and identical serving stacks remains future work.
F
Cross-Vendor Listed-API Pricing (April 2026)
Table 4 compiles listed per-million-token input/output prices for frontier reasoning models across major Chinese and US vendors as of late April 2026. DeepSeek rows use cache-miss input prices and output prices converted from RMB to USD; cache-hit inputs are cheaper and the Pro discount is time-limited. Rows are not normalized for quality, latency SLOs, context window, caching policy, batch discounts, exchange-rate movement, or promotional pricing; the table is provided to support the cross-vendor pattern referenced in §4.1, not as a controlled head-to-head comparison. The ∼3–30× output-price gap is observed across at least four independent Chinese vendors and three independent US vendors, which makes single-firm pricing strategy an incomplete explanation. We treat the gap as consistent with infrastructure-level Φsystem differences shaping marginal API economics, alongside quality differences, latency-SLO variation, caching policies, business-model and subsidy strategies, and exchange-rate movement. No causal identification of any specific cost component is claimed.
19