Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services

IEEE TRANSACTIONS ON SERVICES COMPUTING

1

Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services

arXiv:2611.01371v1 [cs.DC] 1 Nov 2026

Yichiet Aun, Soung-Yue Liew, Yu Liang Lai, Jing Jie Tan, Dong Chen, and Purnomo Husnul Khotimah Abstract—Token-based subscriptions are emerging as a commercial model for large language model inference, but extending them into interoperable utility services requires comparable consumption accounting. Raw token counts do not consistently measure inference resource use because computational demand varies with model architecture, context length, tokenisation, and concurrent serving conditions. This paper presents Token-as-a-Utility, a model-agnostic architecture integrating inference metering, service-level-aware provisioning, and inter-operator settlement. We prove that, under batched serving, no per-request meter can be additive, independent of co-resident traffic, and exactly cost-reflective at once. This motivates separating provisioned capacity, metered consumption, and contracted quality into distinct service products. Consumption is expressed in Reference Inference Units (RIUs), derived from prefill and decode resource requirements, while occupancy-indexed operator ratings recover shared serving costs without altering the common unit. Versioned inference records support auditing and re-rating across operators, while software-enforced capacity leases and quality-constrained routing enable provisioning on shared accelerators. The resource model is validated through measurements on three NVIDIA RTX 5000 Ada GPUs, followed by multi-tenant evaluation using a trace-driven model calibrated to them. For the reference model, occupancy-indexed rating limits statement energy error to 9% under measured traffic, compared with up to 70.5% under fixed-occupancy rating. In calibrated multi-tenant experiments, tenant mispricing decreases from 7.36 times under flat token counting to 2.39 times with RIUs and 1.06 times after occupancy-based reconciliation. These findings demonstrate how portable inference accounting can be combined with load-dependent cost recovery and contract-based provisioning to support heterogeneous inference services across models and operators. Index Terms—Inference serving, service metering, utility computing, service-level agreements, cross-model normalisation, large language models, resource provisioning, settlement.

✦

1

L

I NTRODUCTION

ARGE language model (LLM) inference has acquired the commercial form of a utility. China’s National Data Administration reported that daily token calls exceeded 140 trillion by March 2026, against about 100 billion in early 2024 [1]. The three Chinese mobile operators sell monthly token allowances that resemble mobile data bundles, and China Telecom adds connectivity and security to the same subscription [2]. Singtel launched a token-metered managed inference service in October 2026 [3]. A likely next product bundles a broadband rate, say 100 Mbit/s, with an inference entitlement that applies whichever model answers. A broadband rate can be sold without naming the router, because a bit is a fixed quantity of information and a bit rate is a reservable capacity. Raw token counts have neither

• • • • • •

Y. Aun, S.-Y. Liew and Y. L. Lai are with the Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Kampar, Malaysia. J. J. Tan is with the Saw Swee Hock School of Public Health, National University of Singapore, Singapore. D. Chen is with the School of Physics and Electronics, Nanning Normal University, Nanning, China. P. H. Khotimah is with the Research Center for Data and Information Sciences, National Research and Innovation Agency (BRIN), Indonesia. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. E-mail: {aunyc, syliew, yllai}@utar.edu.my, tanjingjie@nus.edu.sg, chendong@nnnu.edu.cn, purn005@brin.go.id. Corresponding author: Yichiet Aun.

property. The same sentence yields different counts under different tokeniser vocabularies, and the work behind a token depends on architecture, context length and the number of requests sharing each decode step. Raw token counts therefore do not provide a model-independent measure of inference work. This paper builds on three lines of prior work: crossmodel units such as Google’s burndown rates and Azure’s normalized tokens [4], [5], [6], batching-sensitive attribution of serving energy [7], and token-based fairness such as the Virtual Token Counter (VTC) [8], [9]. The vendor units apply within one seller’s boundary, without a published derivation, error bound or procedure for two sellers to settle in the same unit. Network–compute standards call for normalised compute metrics but leave the method open [10], [11], and cross-model signals serve autoscaling rather than billing [12]. This paper presents Token-as-a-Utility, an operational framework for metering, contracting and settling heterogeneous inference services when resource cost depends on batching. It separates a portable reference quantity, the RIU, which never changes with batch occupancy (the number of requests decoded together), from occupancy-dependent operator ratings, a separate adjustment for the conditions under which an operator delivers it. The framework is modelagnostic at the service and accounting interface, where every model is billed in the same unit, while model-specific coefficients, published per model and vintage, convert each

IEEE TRANSACTIONS ON SERVICES COMPUTING

request’s features into that unit. Per-request records keep the features behind both quantities, so a bill can be audited, re-rated and settled between operators. The contributions are as follows. 1)

2)

3) 4)

A service architecture (Section 3) that sells capacity, consumption and quality as separate products, motivated by an impossibility result for per-request metering under batch sharing. A metering method (Section 4) that derives Reference Inference Units (RIUs) from the prefill and decode resource model, with occupancy-indexed operator ratings, declared error bounds (accuracy classes), a bound on gains from repackaging and a re-rating rule between coefficient versions (vintages) that is exact when records carry the feature vector (Proposition 2). Provisioning without hardware partitioning (Section 5) through a software capacity lease, a declared contention ratio and routing that checks legality first. An evaluation calibrated and validated on hardware (Section 6) at Universiti Tunku Abdul Rahman (UTAR). Serving efficiency, energy, mixture-ofexperts weight traffic, swap times and latency under drawn leases are measured against decision rules fixed in advance; the multi-tenant studies, including settlement between operators (Section 7), run on a trace-driven model calibrated with these measurements. Each result is labelled measured, derived, calibrated model or proposed.

2 BACKGROUND, R EQUIREMENTS AND R ELATED W ORK Utility service models. Utility computing treats computation, its pricing and its service-level agreements (SLAs) as services problems [13], [14], [15], [16]. Network pricing separates access capacity, usage and service class [17], and utility tariffs recover shared fixed cost through a capacity charge while the per-unit charge follows marginal cost [18]. Inference vendors sell reserved throughput [4], [19], ondemand tokens and priority tiers [20], each in its own unit. Normalised units and attribution. A standard inference token has been proposed for futures contracts without an operational measurement [21]. A commercial energy unit attributes kilowatt-hours to output tokens through a proprietary engine [22], and MLPerf Endpoints plans cost and power normalisation for systems serving the same model [23]. Abstract credits have also spanned resource types and changing fleets [24], [25]. Concurrent work estimates cost per token under concurrency [26], attributes batch latency with a roofline model [27], measures how request energy depends on batching [28], [29], prices batching externalities and latency tiers [30], [31], and compares token-proportional with Shapley attribution [7], [32], [33]. They leave open a vendor-neutral consumption table with a declared accuracy class and a settlement procedure. Serving and service-level control. Continuous batching and paged key–value (KV) cache memory let one accelerator serve many tenants [34], [35]; prefill (prompt processing) is compute-bound and decode (output generation)

2

TABLE 1 Requirements of an inference utility and coverage by prior work: • addressed, ◦ partly, – not addressed. V vendor units [4], [5]; F fair schedulers [8]; C cost models [26], [28]; N network–compute federation [11], [55]; A cost allocation [7], [56]; T this work. Requirement

V

F

C

N

A

T

R1 comparable consumption across models, vendors R2 published derivation and declared error R3 capacity, consumption, quality as products R4 provisioning without hardware partitioning R5 auditable per-request record, dispute rule R6 settlement across operators and versions

◦

–

◦

–

–

•

–

–

◦

–

–

•

◦

–

–

◦

–

•

–

•

–

–

–

•

–

–

–

◦

◦

•

–

–

–

◦

◦

•

bandwidth-bound [36]; and queueing aware of service-level objectives (SLOs), multi-model pooling, latency isolation and fairness defined over resource cost are well studied [37], [38], [39], [40], [41], [42]. Clients find provider-side token inflation and model substitution hard to detect [43], [44], [45]; this work adds the contracts these mechanisms enforce. Tokenisation disparity. Tokenised length of identical text differs by up to fifteen times across languages [46], at a cost borne by speakers of under-served languages [47], [48]; this paper connects that disparity to a cross-model billing unit. Cost allocation and settlement. Budget balance, additivity and consistency single out Aumann–Shapley prices for shared facilities [49], [50]; on cost functions homogeneous of degree one the same axioms give marginal-cost prices [51], [52], [53]. Shapley settlement between network operators is established [54], and GSMA’s federated operator platform exchanges usage of CPU, memory and storage while leaving settlement out of scope [55]. Selling inference as a metered service across models and operators creates six requirements (Table 1). Tenants need comparable consumption (R1) with a checkable derivation and error (R2). Shared batch cost must be recovered without distorting the unit (R3), often without hardware partitioning (R4). Disputes need per-request evidence (R5), and operators on different coefficient versions must settle (R6). No prior line addresses all six together. Objectives. A cross-model meter can target resource fidelity, language equity (equal charges for equal content in every language) or incentive compatibility (no gain from misreporting tokens). This paper targets resource fidelity, so native-token metering retains the token premium of some languages, which Section 4 discloses as an attribute of the quality class. Character-based pricing addresses incentive compatibility [57], a complementary objective.

3

TOKEN - AS - A -U TILITY S YSTEM M ODEL AND A R -

CHITECTURE

3.1

Three Products

A utility customer buys a reserved capacity, a metered consumption and a contracted quality of supply, and Tokenas-a-Utility keeps these three products. Provisioned capacity

IEEE TRANSACTIONS ON SERVICES COMPUTING

3

is a floor rate of service, enforced by scheduler reservations and billed whether or not it is drawn. Metered consumption is measured in RIUs, which add up across requests, models, operators and billing periods. Contracted quality has two parts. A quality class states which models may serve a request, through floors on capability, latency and tokeniser fertility (tokens per unit of content). A traffic class states when the request is served. Models may substitute for each other only inside the quality class, so the consumption unit need not describe answer quality. Unlike electricity, where kWh = kW × h, a token-rate entitlement converts into reference work only once the model and serving conditions are specified, so a floor of 100 tokens per second is sold as a rate, not a token allowance. 3.2

TABLE 2 Tariff designs permitted by Theorem 1. Design

Gives up

Consequence and examples

Tariff metering

P3

Ex-post allocation Reservation

P2

Separated (this work)

per product

coefficients fixed in advance, operator bears occupancy risk; vendor units [4], [5] shares of realised cost; a charge depends on co-tenants [7], [56] accelerator time of a slice; no perrequest quantity to settle [19] RIU keeps P1, P2 and P3 within a declared error; capacity charge and occupancy-indexed operator rating recover cost

P1

Formal Requirement for Separating the Products

A meter is a map U : (q, σ) 7→ R≥0 from a request q and its serving context σ = (o, m, c) (operator, model and configuration) to RIUs. The ambient state ω comprises the co-resident batch, cache contents and queue, with batch occupancy n(ω), the number of sequences decoded together. A workload Q consumes realised resource R(Q, ω), and the operator admits workloads from a declared envelope E . Throughout, cost means this resource; in the evaluation it is allocated GPU time. Price, the money charged per unit, is set separately by each operator. AP metered unit is expected to satisfy (P1) additivity, U (Q) = q∈Q U (q); (P2) ambient independence, U (q; σ) does P not depend on ω ; and (P3) exact cost-reflectiveness, q∈Q U (q) = λ−1 σ R(Q, ω) for all admissible (Q, ω) with a rate λσ > 0 declared in advance and constant in ω . Assumption 1 (Batch sharing). Each decode step reads the active weights, Wb bytes, once regardless of the number n of sequences in flight, while each sequence reads its own KV state of K(t) bytes at context t, so that the resource per decoded token of one sequence is

ρ(t, n) = Wb /n + K(t).

the serving mechanism that removes homogeneity, its magnitude (one to two orders of magnitude, Section 6) and its design consequence; token-pricing analyses that assume homogeneous production address a different setting [58]. Every tariff gives up one property (Table 2). The separated design remains subject to the theorem and combines existing tariff forms: tariff metering for the quantity, a capacity charge for the shared weight stream (the weight read of each decode step) and an ex-post allocation of the residual by occupancy band. Tenants are quoted, budgeted and settled in RIUs, which keep additivity (P1) and ambient independence (P2) and are cost-reflective (P3) within a declared error. The RIU itself never changes with occupancy: the residual cost of delivery is recovered after the fact (the true-up) through a separate operator rating indexed by the recorded occupancy band, so occupancy dependence is confined to the rating and declared on each record. The charges are disjoint: a floor of F tokens per second costs F β per second, tokens inside it are billed without the β term, and only on-demand tokens carry β and its true-up (worked statement in Supplement S1).

(1)

Theorem 1 (No exact, additive, ambient-independent meter). Fix σ , let Assumption 1 hold with Wb > 0, and let E contain two workloads of identical requests served at occupancies n1 ̸= n2 . No meter satisfies P1, P2 and P3 on E . Proof. Let A serve n identical requests concurrently and B serve them one at a time, each generating Tgen tokens at context t. By (1), R(A) = Tgen (Wb + nK) and R(B) = nTgen (Wb + K), so R(B)/R(A) > 1 for n > 1. By P2 each U (q) is equal in both workloads, so by P1 U (A) = U (B), and P3 with one λσ forces R(A) = R(B), a contradiction. Theorem 1 instantiates a classical result: a budgetbalanced price per metered unit that does not change when aggregate demand is scaled exists only if cost is homogeneous of degree one, which for one metered quantity means linear cost [51], [52], [53]. Decode cost Tgen (Wb + nK) is affine in occupancy with a positive intercept. For mixtureof-experts (MoE) models the weight read grows with n, but the measured read per sequence still falls as n grows (Section 6.2), so the result still applies. The theorem adds

3.3

Architecture

Figure 1 shows the architecture. The gateway meter authenticates the tenant, takes a budget hold in RIUs and records the token counts that form the feature vector ϕ(q). Admission applies the traffic class, the capacity lease and the occupancy band of the accuracy class. Routing checks legality, quality class, contract and cost in that order, and dispatches to a replica on shared accelerators. A second meter in the serving engine, the witness meter, records occupancy, prefill and decode time and preemptions, which the gateway cannot observe; the engine’s own metrics carry no tenant label [35]. The two meters are reconciled per request: token counts must agree exactly, and RIU values within the declared error. The normalisation plane holds the versioned coefficient registry. Each request produces one append-only inference detail record (IDR) carrying ϕ(q), model, configuration, occupancy band, vintage, declared error and, for cross-border routing, the legal basis. It carries no retail price, because the RIU is a wholesale quantity.

IEEE TRANSACTIONS ON SERVICES COMPUTING

4

service

Provisioned capacity floor rate; KV and residency reservations; billed whether or not it is drawn

Gateway meter identity, budget hold, counts ϕ(q)

operator

tenant request

Admission traffic class, lease, occupancy band b

ϕ(q)

Contracted quality quality class K × traffic class; substitution only inside K

Routing LEGAL → CLASS → CONTRACT → COST

Metered consumption Uv (q) = ⟨wm,v , ϕ(q)⟩ in RIU; additive across models and operators

Shared GPUs continuous batching, no hardware partition

Witness meter occupancy n, prefill and decode time n, tpre , tdec

b

Normalisation plane: versioned coefficient registry operator-neutral wm,v for settlement · operator rating wo,c,b for cost recovery · declared accuracy class

settlement

Inference detail record ϕ(q), m, c, b, v, declared error, legal basis; append-only, carries no retail price

Home operator contract, balance, hold

usage push every ∆ entitlement check

Visited operator serves under its own wo,c,b

Neutral clearing: validate → re-rate from ϕ to the settlement vintage → net multilaterally

Fig. 1. Token-as-a-Utility architecture. The three products (top) govern an operator’s mechanisms (middle): capacity is enforced at admission, quality at routing, and the normalisation plane computes consumption from gateway counts and engine-recorded serving conditions. The inference detail record (IDR) carries the feature vector ϕ(q), so settlement can re-rate it to a common coefficient vintage (bottom). Dashed arrows: contractual governance; solid arrows: data flow.

content s in language ℓ τm counts ϕ(q) under the serving model’s tokeniser wm,v quantity Uv (q) in RIU Σ statement ∑i Uv (qi ) across providers

(1) Tokenisation. The same s yields counts that differ by 1.22× (English) to 5.5× (Khmer) across vocabularies. Counts are billed natively; fertility is disclosed as a bounded quality-class attribute. (2) Computation. Equal counts cost different resource by architecture, context and occupancy. ϕ follows the roofline terms; occupancy and configuration enter the operator rating wo,c,b , not the quantity. (3) Aggregation and settlement. Quantities from independent providers add only under a shared vintage v with a declared accuracy class; records carry ϕ, so they can be re-rated when v changes.

Fig. 2. The three normalisation problems between content and a settleable quantity, and the mechanism that handles each.

4

M ODEL -AGNOSTIC I NFERENCE M ETERING

4.1

Three Normalisation Problems

Turning heterogeneous inference into one quantity involves three problems (Fig. 2): tokenisation normalisation, because the same content yields different counts under different vocabularies; computational normalisation, because equal counts consume different resources by architecture, context, precision and occupancy; and commercial normalisation, because independent providers can sum, dispute and settle consumption only under a shared, versioned convention with a known error. 4.2

Resource Model and Reference Inference Unit

For Tin fresh input tokens, prefill performs Fpre = 2Na Tin + 2 2dattn [Lg Tin /2 + Lℓ (wTin − w2 /2)] operations, where Na is

the number of active parameters, Lg and Lℓ the numbers of global and sliding-window layers and w the window, for Tin ≥ w. Attention counts one multiply–add per query– key pair and dimension, the scaling-law convention (Supplement S1). Over Tgen decode steps a sequence starting at context Tctx moves   Wb Tgen T (T −1) + ktok Tctx Tgen + gen 2gen Bdec = + kfix Tgen n (2) bytes, where ktok is the KV size per token, computed from key and value heads rather than query heads, and kfix the per-sequence floor of sliding-window layers. Converting both into time through the device roofline (peak compute and bandwidth) at a declared reference occupancy n0 gives

Uv (q) = ⟨wm,v , ϕ(q)⟩, 2 ϕ(q) = 1, Tin , Tin , Tgen , Tctx Tgen +

2
Tgen 2 , Nimg , V ,

(3)

with coefficients (c0 , α1 , α2 , β, µ, εenc , α1 ) keyed to model m and vintage v , where Nimg images contribute V visual tokens that are prefilled, and so rated, like text. The coefficient of Tgen is β − µ/2 + κfix , with κfix the sliding-window floor in RIUs, so (3) reproduces (2); Supplement S1 gives the cached-prefix term it omits (0.13% of the reference job). The gauge, which sets the size of the unit, fixes one RIU as the decode of one token on a reference configuration: Llama3.1-8B at FP8 with BF16 KV cache on one GPU, at context 2,048 and n0 = 32. Measured on the evaluation node, this is 0.242 J of GPU board energy (Section 6.2); the derived value of 0.34 J also counts an assumed host share. The derivation fixes the structure of the coefficient set and leaves only values empirical. This has three consequences for the common billing form αTin + βTout + γTrsn + δWmulti over input, output, reasoning and multimodal quantities. Reasoning tokens are decoded like output

IEEE TRANSACTIONS ON SERVICES COMPUTING

tokens, so γ = β , as vendors already bill [4]; their counts stay on the record for audit because tenants cannot see them [43]. The multimodal term reduces to an encoder constant plus input tokens. The common form omits the KV cross term Tctx Tgen , which accounts for 42.0% of the cost of a reference job (1,800 input tokens, 30% cached, 420 output tokens on Llama-3.1-8B). A cached input token still carries µTgen /(α1 +µTgen ) of a fresh token’s cost, 0.64 at 420 output tokens; cache rates of 0.05 to 0.1 [4] shift this cost to the output price (derived; Supplement S1). 4.3 Tokenisation: Native Counts and a Disclosed Premium Bound

5

decode step grow with occupancy, the factor n0 /b becomes n0 W (b)/(b W (n0 )), read from a measured weight-traffic table W (b) (Section 6.2); for dense models the two factors are identical. Where hardware differs in its ratio of compute to bandwidth, a configuration-specific coefficient vector replaces the scalar. The split also separates estimation: from latencies measured at controlled occupancy, a third party can recover coefficient ratios within the prefill group and within the decode group, but neither the level nor the ratio between groups (Supplement S2). The quantity’s structure is thus externally checkable, while the operator rating must be declared and certified.

Let Φm (ℓ) be the premium of language ℓ under model m’s tokeniser: the tokens needed for content in ℓ divided by the tokens needed for the same content in the reference language, on a public parallel corpus.

4.5

Proposition 1 (Tokeniser-invariant billing). Suppose a meter charges by content alone and each model’s rating is calibrated on the reference language, so that cost per native token does not depend on the language. If the models serving language ℓ span κℓ = maxm Φm (ℓ)/ minm Φm (ℓ), the meter’s relative error on ℓ is at least (κℓ − 1)/(κℓ + 1) for some model, and an operator paid p per reference token earns a margin p − λΦm (ℓ), negative whenever Φm (ℓ) > p/λ.

Definition 1 (Accuracy class). A meter meets class (ϵb , ϵ) at aggregation level L on E if, over the reference workload, |E[e]| ≤ ϵb and Pr(|e| ≤ ϵ) ≥ 0.95, with e computed per request or per tenant statement (bill).

The best single charge is the harmonic mean of the extreme resources (Supplement S2). With the premium ranges of Fig. 4(a), the error bound is 14% for Malay and 68% for Tamil. Invariant billing is thus least accurate for the languages it is meant to protect, and without cross-subsidy operators have a reason to decline them. The architecture therefore meters native counts and publishes the premium as a bounded quality-class attribute, so a tenant declaring a language sees the eligible models before contracting. The framework therefore does not promise identical charges for equivalent content in different languages; it bounds and discloses the difference. 4.4

Operator-Neutral Quantity and Operator Rating

A settlement unit must give the same number on both sides of an operator boundary. Operators serving the same request may, however, spend different resources: published estimates for one model and query differ by a factor of 8.2 between two hosts [59]. If the operator is added to the ambient state, the argument of Theorem 1 shows that a single rate cannot be both operator-neutral and costreflective. Coefficients are therefore keyed at two levels,

Uv (q) = ⟨wm,v , ϕ(q)⟩,

(b) R̂o (q) = λo,c ⟨wm,v , ϕ(q)⟩. (4)

The RIU, Uv , is the operator-neutral quantity that is aggregated, budgeted and settled, and it is the same for every operator and occupancy. The operator rating R̂o estimates the resource behind it for cost recovery and, of the two, is the only one that depends on occupancy and configuration. It rescales the weight-stream coefficient, the only occupancydependent term of (2), to the recorded occupancy band b, β (b) = βn0 /b, and applies the efficiency λo,c of configuration c. For MoE models, whose weight bytes per

Accuracy Classes and the Repackaging Bound

Since Theorem 1 rules out exact cost-reflectiveness, a meter declares its departure P from it. For a workload Q the relative error is e(Q) = ( q U (q) − λ−1 R)/λ−1 R.

Classes RIU-2, RIU-5, RIU-10 and RIU-25 are named by ϵ in per cent, with bias limits of 0.5, 1, 2 and 5%. Per-request accuracy also bounds what a tenant gains by repackaging a request q into another form g(q): Theorem 2 (Repackaging bound). If |U (q) − λ−1 r(q)| ≤ ϵλ−1 r(q) holds for q and for g(q), any transformation g with r(g(q)) ≥ r(q) satisfies U (g(q)) ≥ 1−ϵ 1+ϵ U (q), so repackaging saves at most 2ϵ/(1 + ϵ). The proof combines the lower bound for g(q) with the upper bound for q (two-region form in Supplement S2). A class bounds only the 95th percentile, so a tenant who can aim at the tail is limited by the worst-case error. Disputes are therefore decided per statement: a correction is admissible only if it changes a statement by more than the declared ϵ. Coefficient error is common to all requests of a statement and does not average out (Supplement S2), so bias is limited separately. A class is accepted on a one-sided test against the upper confidence bound, as for electricity meters [60]. 4.6

Vintages and Re-Rating

A vintage fixes the reference workload, coefficients, envelope, accuracy class and effective and retirement dates. Values may change between vintages; the feature map and gauge may not, since changing them creates a new unit. Scaling every coefficient by one factor changes the gauge but not relative charges, so a uniform efficiency gain needs no recalibration; a change in coefficient ratios does. Proposition 2 (Re-rating). For vintages v, v ′ sharing the feature P ′ map andP any set Q of requests on modelPm, q∈Q U Pv (q) = ⟨wm,v′ , q∈Q ϕ(q)⟩. A scalar T with Uv′ = T Uv for every Q in an envelope whose feature vectors span the feature space exists if and only if wm,v′ = T wm,v . The first statement is linearity of (3); the second follows by evaluating single-request workloads (Supplement S2). A vintage that moves the reference occupancy changes β

IEEE TRANSACTIONS ON SERVICES COMPUTING

6

relative to the other coefficients, so it admits no scalar conversion. Records carrying ϕ(q), or its sum over a settlement bucket, re-rate exactly to any vintage that covers the model. Records of uncovered models wait for an extended vintage.

5

SLA-AWARE U TILITY P ROVISIONING

5.1

The Evaluation Node

The evaluation setting is a shared node of four NVIDIA RTX 5000 Ada accelerators at UTAR; three of them were available for the measurements of Section 6.2. Each card has 32 GB of GDDR6 memory with error correction, 576 GB/s of bandwidth, a 250 W board power limit and a PCIe 4.0 ×16 interface without NVLink [61]. Error correction reserves 6.25% of GDDR memory [62], leaving about 30 GB visible; the calibrated model budgets 27.44 GB after engine reservations. PCIe carries 31.5 GB/s per direction, 18 times less than local memory bandwidth, so models run as single-card replicas. Multi-instance GPU partitioning is supported on data-centre GPUs from Ampere onwards and on Blackwell workstation cards, but not on Ada cards [63]. Under NVIDIA’s MultiProcess Service, a fatal fault in one client is reported to every client on that GPU [64]. On this node every isolation guarantee is therefore a scheduler property, and, with more requested models than cards, which models stay loaded (residency) becomes an operational decision. 5.2

Capacity and Quality Products

TABLE 3 Evidence used in the evaluation. Source

Content

Class

FLORES-200 devtest [66]

1,012 parallel sentences, 18 languages, 16 tokenisers: 291,456 counts 146 problems common to 24 programming languages, 10 tokenisers layers, KV heads, windows, parameters of 13 models 1,429,737 request– response token pairs 19,366 conversation, 8,819 code requests three RTX 5000 Ada, vLLM 0.11, NVML board energy serving, routing, quality, settlement

measured

MultiPL-E [67] Model files

config

BurstGPT v1 [68] Azure trace 2023 [69] Node runs E1–E8 (S7) Calibrated model (S4)

measured measured trace trace measured calibrated model

controller loads a model only when pending work exceeds the work the swap loses by a factor τ . It charges the swap to the operator, applies the legality predicate and counts refused requests as demand. 5.3 Coupling Between Provisioning and Metering Accuracy

Without a hardware partition, a capacity lease combines a scheduler floor that reserves a share of each step’s token budget, a KV-block reservation that keeps other tenants’ long contexts from blocking admission, and a residency reservation that keeps the leased model loaded. The ratio of floors sold to sustainable throughput is published as a declared contention ratio, as broadband access publishes its overbooking. Five traffic classes, adapted from the contractual service categories of Asynchronous Transfer Mode (ATM) networks, carry the quality product: interactive guaranteed (IG) with tight time-to-first-token (TTFT) and time-per-outputtoken (TPOT) bounds, interactive bursty (IB), batch with a deadline (BB), adaptive (AD) and best effort (BE). A quality class is a conjunction of floors, m ∈ K ⇐⇒ θ(m) ≥ θ K ∧ TTFT99 ≤ τ̄ K ∧ Φm (ℓ) ≤ F̄ K for every declared language, so classes form a partial order and no scalar ranking of models is needed. The router selects

The operator chooses a routing policy π , an admission policy a and a declared envelope E to maximise SLO attainment subject to legality, class membership, tenant budgets P q U (q) ≤ bi and

π(r) = arg mink∈A(r) cost(r, k), (5) A(r) = LEGAL(r) ∩ CLASS(r) ∩ CONTRACT(r),

The evaluation answers five research questions. RQ1: Do raw token counts measure inference work consistently across languages, tokenisers and batch occupancies? RQ2: How accurately do RIUs and occupancy-indexed ratings represent resource use across models and loads? RQ3: Which capacity and latency guarantees can a scheduler provide without hardware partitioning, and at what cost? RQ4: How does metering accuracy trade off against routing flexibility and service quality? RQ5 (Section 7): How accurately can operators on different vintages settle? Table 3 lists the evidence: token counts, architecture constants and the serving quantities of Section 6.2 are measured; costs computed from them and published specifications are derived; multi-tenant outcomes (RQ2–RQ5) come from the calibrated model.

where LEGAL(r) holds the replicas a request may lawfully reach under data-residency, transfer-basis and exportcontrol rules, evaluated against a registry at request time. Legality is a filter rather than a penalty, because a cheap enough forbidden route could outbid a penalty. The IDR stores the legal basis, which also meets the per-transfer record duty of Malaysia’s cross-border framework [65]. Residency changes per five-minute epoch rather than per engine step, because a cold load of a non-resident model takes 63–102 s on the node (Section 6.2). Models that must be loaded cold are therefore pinned, and a model held in host memory can be swapped by sleep/wake in about 1 s. The

supp n(π, a) ⊆ E,

(6)

where n(π, a) is the occupancy distribution the two policies induce and supp is its support. Constraint (6) makes metering accuracy an output of the scheduler. The products are coupled through these declared parameters rather than a joint optimiser, because the unit changes allocation mainly as a budget (Section 6).

6

C ASE S TUDY AND E VALUATION

6.1

Setting, Data and Evidence

IEEE TRANSACTIONS ON SERVICES COMPUTING

The calibrated model runs one continuous-batching engine per GPU; each decode round lasts the bytes moved (weights once, every sequence’s KV state) divided by achieved bandwidth, plus admitted prefill time. Tenants draw request shapes from the two traces, and arrivals replay Azure inter-arrival times rescaled to a target load; each 1,800 s run serves 5,000 to 14,600 requests. Ground truth is a reference allocation: a request is charged its own prefill, its own KV reads and an equal share of each weight read it joined. This is exact accounting under the model’s chosen rule, not a uniquely correct economic allocation. The engine omits chunked prefill and per-request overhead. It uses the bandwidth and compute fractions, MoE weight traffic W (n) and cold-load time measured in Section 6.2, and an operator rating without c0 (coefficient vintage RIU2026.M); node power cancels in every reported ratio (earlier assumed constants: Supplement S7). The first study (RQ1, RQ2) serves Llama-3.1-8B at FP8, gpt-oss-20b at MXFP4, and Gemma-3-27B and Qwen3-32B at INT4 to five tenant classes over five arrival seeds. The second (RQ3, RQ4) uses eight single-card replicas on two logical operators of four cards each (the evaluation node and a second node in another jurisdiction), with eleven models over three seeds. Brackets give bootstrap intervals for measured quantities and the range over seeds for model results (details in Supplement S4). 6.2

Hardware Validation

The node was measured first: three RTX 5000 Ada cards ran vLLM 0.11 with the four models of RQ2, with energy read from each card’s NVML counter. Six decision rules were fixed before measurement (Supplement S7). On the gauge model, a decode fit over 18 cells gives ηbw = 0.887 (R2 = 0.993; Fig. 3(a)), and prefill gives ηfl = 0.41. One gauge decode token costs 0.242 J of board energy at occupancy 32 and context 2,048. For gpt-oss20b, weight bytes per decode round grow from 4.1 GB at n = 1 to 13.2 GB at n = 64 (panel (b)), because co-batched requests select different experts. The union-of-experts law W (n) = 2.94 + 8.83[1 − (1 − 4/32)n ] GB fits within 19% and replaces the constant active-weight read. The accelerator time per extra request is 0.65 ms on the gauge model, while a lone one-token request takes 18.5 ms of host and engine time, so c0 moves from the operator rating to the gateway. A cold load takes 63–102 s, so residency is pinned for cold models; vLLM sleep/wake restores a model held in host memory in 0.7–1.0 s. Under drawn leases with a 50 ms TPOT bound, guaranteed-class attainment is 1.00 at 0.31:1, 0.82 at 0.43:1 and 0.00 from 0.57:1 (panel (c)); the calibrated model, which omits chunked prefill, is more conservative and predicts 0.04 at 0.47:1. The paired cache contrast gives d(420) = 0.584, against 0.641 derived. On 199 MMLU-Pro items, accuracy separates the Q3 model (0.437) from the Q1 and Q2 models (0.628–0.719) but does not order Q1 and Q2, so class membership is declared from task-specific evaluation. An energy model fitted on controlled-occupancy cells and frozen was applied to eight 150 s trace runs (panel (d)). On the gauge model, the occupancy-indexed rating kept statement error within +9% at mean occupancies of 1.4 to 27.1, while the reference rating

7

erred by up to −70.5%. For gpt-oss-20b, whose energy model keeps a constant weight stream, the error reached +26.2% at occupancy 1.2, the MoE effect of panel (b). RQ2– RQ5 use a trace-driven model calibrated with these measurements, and RQ2 and RQ4 are reported at this measured vintage. 6.3

RQ1: Raw Token Counts as a Measure of Work

RQ1 establishes that raw token counts cannot serve as a common unit: counts vary with language and tokeniser for fixed content, and the work behind a fixed count varies with occupancy. Figure 4(a) shows that identical content yields different counts (Supplement S3). Malay needs 1.22 to 1.62 times the tokens of English; Tamil shows the widest range, 7.65 times [7.56, 7.73] under the Llama-3 vocabulary, against 1.47 times [1.45, 1.49] under Gemma-3. Panel (b) locates the effect: the spread across tokenisers on identical text is 1.22 times for English and 5.22 to 5.52 times for Tamil and Khmer, whereas every programming language lies between 1.09 and 1.21 times, so the disparity arises in natural-language vocabularies. Panel (c) shows that equal counts can represent unequal work. On one model, card and context of 2,048 tokens, the relative cost of an output token falls from 284.8 at occupancy 1 to 12.65 at occupancy 70. This factor of 22.5 is fixed by published architecture and bandwidth alone, since the efficiency constants cancel. Across 648 combinations of memory fraction, error-correction reserve, precision and scheduler cap at contexts of 1,024 to 4,096 tokens the factor lies between 10.1 and 47.9; it shrinks at long contexts because KV capacity limits the batch (Supplement S3). One model’s occupancy range thus exceeds the output-to-input ratios published across whole menus, 1 to 25 [4] and 4 to 8 [5]. 6.4

RQ2: Normalising Heterogeneous Consumption

RQ2 establishes that recording occupancy, rather than calibrating a better price for each model, keeps the trued-up bill close to cost across models and loads. Six meters share one work-conserving weighted fair queue and differ only in what increments a tenant’s counter: flat input and output counts weighted 1:1 and 1:2, the latter the default of VTC [8]; each model’s catalogue input and output prices, as a multivendor aggregator does [70]; the RIU at reference occupancy, charged at admission or accrued during generation; and the RIU trued up with the occupancy-indexed operator rating of (4). Against the reference allocation (Fig. 5(a)), flat 1:1 counting misprices by 7.36 times [7.01, 7.56], the catalogue price by 5.05 times, the RIU by 2.39 times [2.34, 2.46] and the trued-up bill by 1.06 times [1.03, 1.08], close to an oracle charging realised GPU time. The ordering also holds, with larger magnitudes, when each round’s time is split equally (Supplement S4). GPU time per charged unit across the four models (panel (b)) spreads 3.09 times under flat counting and 3.86 times under the catalogue price. The RIU at one reference occupancy narrows it to 2.55 times [2.39, 2.70]; the remainder arises because the replicas run at mean occupancies of 19.4 (Llama-3.1-8B), 11.5 (Qwen3-32B), 9.6 (Gemma-327B) and 5.3 (gpt-oss-20b) while the coefficients assume 32.

IEEE TRANSACTIONS ON SERVICES COMPUTING

30 20 10 0

ηbw = 0.887, R2 = 0.993

1

4

16

64

12

measured

0.8

10

calibrated model chunked prefill on chunked prefill off

0.6

8 6

0.4

union of experts

4

declared limit

0.2

active-only assumption

2 0

(c)

1.0

statement energy error (%)

context 512 context 2048 context 8192

40

14 (b)

attainment of TPOT ≤ 50 ms

weight bytes per round (GB)

decode round time (ms)

(a) 50

8

0.0 1

4

batch size n

16

64

0

batch size n

1

2

60

reference (n0 = 32) occupancy-indexed Llama-8B gpt-oss-20b

40 20 0

−20 −40 −60 −80

3

(d)

1

3

contention ratio (floors / throughput)

10

30

mean occupancy

Khmer Tamil Burmese Hindi Thai Arabic Chinese Vietnamese Japanese Russian Korean Indonesian Malay German Spanish French Tagalog

(a)

(b) Malay

Burmese Khmer

natural (18) English Chinese

code (24) Llama-3 vocabulary Gemma-3 vocabulary range, 16 tokenisers

1

2

4

1.09–1.21×

8

tokens for identical meaning, relative to English

Tamil

2

4

6

cross-tokeniser spread on identical text

output-token cost / input-token cost

Fig. 3. Hardware validation on the evaluation node. (a) Decode round time against batch size, Llama-3.1-8B, with fitted roofline. (b) gpt-oss20b weight bytes per decode round against batch size, with union-of-experts law and active-only read. (c) Guaranteed-class TPOT attainment against contention ratio, with the calibrated model. (d) Statement energy error against mean occupancy, reference and occupancy-indexed ratings. Measured. 512 context 2k context 8k context 32k context

100

10

published cross-model ratios

1 1

10

(c)

100

batch occupancy n (concurrent sequences)

Fig. 4. Raw token counts as a measure of work. (a) Tokens for identical meaning relative to English, range over 16 tokenisers (measured, FLORES-200 devtest; bootstrap intervals within markers). (b) Cross-tokeniser spread on identical text (measured). (c) Relative output-token cost by batch occupancy, Llama-3.1-8B at FP8 on one RTX 5000 Ada; curves end at KV capacity; band: vendor-published output-to-input ratios (derived).

Indexing by occupancy reduces the spread to 1.08 times; for the MoE model this uses the measured weight-traffic table, without which the spread is 1.79 times. Measured as excess above parity, a model-keyed table for one occupancy, as vendors publish, removes about a quarter of the spread of flat counting, whereas indexing by recorded occupancy removes 96%; occupancy, rather than model identity, makes the charge cost-reflective. The strongest model-keyed alternative, a per-model rate table fitted to the node’s own traffic, misprices by 1.07–1.19 times on four held-out seeds, close to the occupancy-indexed rating (1.03–1.07), but at loads of 0.6 or 1.5 it degrades to 1.48–1.66 times, while the occupancy-indexed rating stays at 1.07–1.08 (Supplement S5). Occupancy therefore belongs in the operator rating rather than in a one-off calibration. The ordering held across nine variants of the assumed constants, with the measured constants inside their ranges (panel (c); Supplement S5). 6.5

RQ3: Provisioning Without Hardware Partitioning

RQ3 establishes that software isolation on shared accelerators can sell a throughput floor, while a latency target also requires a bounded contention ratio. In the calibrated

model (Supplement S5, Fig. S2(a)), routing freedom (in-class models per request) improved availability but not capacity. With each tenant restricted to its preferred model, 7.3% of requests are refused because only eight of eleven models can be resident, and the worst tenant’s TTFT attainment is 0.00. Two admissible models per request remove refusals but push displaced demand onto other tenants’ replicas: throughput drops to 1,031 tok/s and the worst tenant’s attainment stays at 0.003. All tenants recover with three or more models (0.89), reaching 1.00 at 1,680 tok/s under free in-class routing. Checking legality after the cost function would have sent 48.1% of requests to a cheaper replica in a forbidden jurisdiction, while cross-operator routing changed throughput by only 0.5%, so federation is a legal rather than a performance instrument. Among quality-ofservice mechanisms, only the KV-block reservation changed an outcome, raising the bursty class from 0.20 to 0.63 (Supplement S5). Fig. S2(b) shows that residency control pays off when swaps are fast: refusals fall from 7.3% to zero at a 1 s swap, the measured sleep/wake time, and to 5.4% at 10 s, whereas at the measured cold-load times they are 7.0–16.3%. Fig. S2(c) quantifies the capacity product; the contention ratio is floors sold over aggregate throughput in the same

flat tokens 1:1

(a) 7.36

4.71

flat tokens 1:2

5.05

catalogue price 2.39

RIU (n0 = 32)

2.47

RIU, accrued in flight 1.06

RIU, occupancy-indexed

(b) GPU time per charged unit (relative to menu mean)

reference allocation equal time-share

9

4 2

2

5

10

3.09×

2.55× 1.08×

1 0.5 0.25

1

3.86×

20

Llama-3.1-8B gpt-oss-20b

flat 1:1

price

Gemma-3-27B Qwen3-32B

RIU

RIU occ.

tenant mispricing, max/min

IEEE TRANSACTIONS ON SERVICES COMPUTING nominal constants measured constants 9 variants

10

5

2

1

(c)

flat 1:1

price

RIU

RIU occ.

tenant mispricing, max/min across tenants

Fig. 5. Normalisation across models (calibrated model). (a) Tenant mispricing (bill share over GPU-time share, max/min across five tenants); bars: five seeds; filled: reference allocation; open: equal time shares. (b) GPU time per charged unit by model, relative to menu mean (load 1.0; markers: one seed; numbers: five-seed mean max/min). (c) Tenant mispricing under the earlier nominal constants, nine varied constants (meter derived at nominal values) and the measured constants.

run. Leases sold but not drawn leave the guaranteed class unchanged up to 5.5:1. When every lease draws its floor, the model predicts TPOT attainment of 1.00 at 0.15:1, 0.71 at 0.27:1 and 0.04 at 0.47:1, because a larger batch raises throughput but lengthens every decode round. Because the model omits chunked prefill, the prediction is conservative: on the node, attainment is 1.00 up to 0.31:1, 0.82 at 0.43:1 and 0.12 at 0.56:1 (Fig. 3(c)). A lease with a latency target therefore needs a declared ratio below about 0.45:1, set by measurement, with the model as a safe planning bound. Software isolation thus provides a throughput floor and admission priority; the latency target also needs a low contention ratio. 6.6

RQ4: Accuracy, Flexibility and Service

RQ4 establishes that recording occupancy delivers the billing accuracy that controlling occupancy buys only with throughput, as constraint (6) predicts and the separated design of Table 2 assumes. In Fig. 6(a), the reference meter is re-anchored to each (model, operator) key’s median occupancy, with its rating fitted per key. Across 382 keys, its per-request error is ϵ95 = 0.061 (n95 /n5 )1.750 (R2 = 0.634), where the band ratio measures the spread of occupancy. The fitted intercept is 6.1% at a band ratio of one, against 13.4% while the per-sequence constant c0 remained in the operator rating. The tightest per-request class met in these runs was RIU-25, because the worst-served key sets the class: free routing strands some replicas near occupancy one, where the weight-stream term is largest. Meeting the per-request RIU-25 threshold required the tight admission band [9, 12]: at one model per request it cost 38% of throughput, with queues that grow throughout the run (p99 TTFT of 496 to 547 s; panel (c)). This per-request view overstates the error that matters for billing (panel (b)). Disputes concern statements, within which per-request noise averages while systematic error remains. For the reference meter, the worst tenant statement is in error by 0.5% to 4.7% across runs without admission control and the median by at most 2.0%. Admission bands, by contrast, raise statement error to as much as 8.3% by distorting which tenants share a batch. The occupancyindexed rating, reading the occupancy recorded by the witness meter, kept the worst per-request error between 4% and

23% and the worst statement error at or below 2.2% in all 60 runs without restricting routing or admission. With ratings fitted on one seed and frozen, the occupancy-indexed rating’s worst statement error on 40 held-out runs was 0.1– 2.1% (reference meter 1.6–9.3%; transfer across settings in Supplement S5). 6.7

Where Simpler Mechanisms Suffice

Supplement S9 lists eleven configurations in which a simpler alternative performed as well as or better than a proposed mechanism, with the resulting design changes. They share a cause: without hardware partitioning, each guarantee that needed a narrow occupancy distribution was paid for in throughput. Under work-conserving scheduling every unit changed allocation only as a budget (Jain index within 0.0052); RIUs accrued in flight priced no better than RIUs charged at admission; quality-of-service mechanisms did not raise guaranteed-class TPOT attainment, because priority cannot reach a TPOT miss once a batch is formed; admission for per-request accuracy cost 38% of throughput, whereas recording occupancy cost nothing; and residency control gave no benefit at the measured cold-load times, so cold models are pinned and swaps use sleep/wake.

7

R EGIONAL F EDERATION , D ISCUSSION AND L IM -

ITATIONS

7.1

Federated Settlement (RQ5)

Unlike roaming traffic, inference crosses an operator boundary mainly because of law or local saturation, since a token costs almost nothing to transport. Settlement borrows the machinery of roaming: an operator registry, a standard usage record, near-real-time exposure control and neutral clearing. The visited operator, which serves the request, validates entitlement with the home operator, which holds the tenant’s balance; records are pushed home every ∆ seconds, and a clearing service validates, re-rates and nets them multilaterally (Supplement S6). Between pushes, exposure is bounded by the visited operator’s serving rate C , so ∆ ≤ Lr /C for a risk limit Lr . Peak decode of the gauge model at its KV-limited occupancy of 70 on four cards gives

IEEE TRANSACTIONS ON SERVICES COMPUTING

1 RIU-25 RIU-10

0.1

RIU-5 RIU-2

(a) 1

2

4

request, reference statement, reference

10

request, occ.-indexed statement, occ.-indexed

(b)

1 0.1 0.01

1800

(c)

1600 1400 1200 no admission band band [8, 20] band [8, 14] band [9, 12] RIU-25 per request

1000 800

0.001

8

node throughput (tok/s)

worst-case relative error

per-request error e95

e95 = 0.061 (n95 /n5 )1.750 R2 = 0.634, n = 382

10

1

occupancy band ratio n95 /n5 of a key

2

3

4

routing freedom f

free

1

10

100

1000

p99 time to first token (s)

447.5%

none 3.7%

scalar

392.8%

frozen+10% 4.6%

totals only feature sums exact

(a)

per request exact

0.001

0.1

10

settlement error (fraction)

overdraw (% of balance)

Fig. 6. Accuracy against flexibility and service (calibrated model). (a) Reference-meter per-request error against occupancy band ratio (382 (model, operator) keys) and fitted law. (b) Worst per-request and statement error of reference and occupancy-indexed meters by routing freedom (no admission control); bars: three seeds. (c) Throughput against p99 TTFT, 60 runs; filled: reference meter met per-request RIU-25 threshold.

10000

prevents overdraw by a tenant fanning out across operators (panel (b)): overdraw grows in steps with ∆, from 1.1% of the balance at 1 s and 4.4% at 19 s to 603% at four hours. The bound C∆ with the peak one-second rate holds from ∆ = 5 s (Supplement S6).

calibrated model C∆, peak C C∆, mean C

1000 100 10 1

(b) 0.1

1

100

10000

push interval ∆ (s)

Fig. 7. Settlement across three logical operators on different coefficient vintages (calibrated model). (a) Settled-amount error against the same requests rated at the settlement vintage, 95th percentile (bar, with value) and maximum (tick mark) for: no re-rating, scalar factor, frozen coefficients with a 10% haircut, bucket count and token totals, bucket feature sums, per-request feature vectors; dotted lines mark RIU-2 to RIU-25. (b) Maximum overdraw of a tenant opening 7,144 sessions across three operators against the push interval, with the bound C∆.

C ≈ 5.1×103 RIU/s, and Lr = 105 RIU then gives ∆ ≈ 19 s, against four hours for roaming call records [71]. Three logical operators were modelled: one on a current vintage, one with a stale reference occupancy and one fitted before mixture-of-experts models joined the menu. Over 14,390 to 14,577 records per seed, net positions summed to at most 5.6 × 10−17 , showing only that records, re-rating and netting compose without loss. In Fig. 7(a) (two-seed means), settlement without re-rating errs by 448% at the 95th percentile, and a frozen-coefficient fallback with a 10% discount (haircut) by 393% on a model whose cost structure changed, as Proposition 2 predicts; a published scalar factor per model and vintage errs by 3.7% but covers only the few records whose model kept its cost structure. Re-rating from per-request feature vectors or their per-bucket sums is exact by the same proposition, so exact settlement depends on records keeping the feature vector. Buckets keeping only request counts and token totals err by up to 5.6% because the meter is non-linear in totals. Records of a model absent from the settlement vintage, nearly all cross-operator records in this deliberately adverse scenario, must be held until the vintage is extended. The home-side hold bounds rather than

7.2

Deployment and Limitations

Federation needs a neutral registrar for operator identifiers, vintages, sealed evaluation sets and clearing; ratings published under an agreed measurement boundary; and a scalable legal basis for cross-border routing (Supplement S6). A single university facility can deploy the single-operator subset without them. The multi-tenant studies (RQ2–RQ5) run on a trace-driven model, and the meter is derived from the same roofline model. The inputs that could change a decision are measured on the node (Section 6.2), and a frozen calibration was checked on real traffic; that check covers eight runs of 150 s on two models, and its energy model for gpt-oss-20b still keeps a constant weight stream, which leaves errors of up to 26%. The model omits chunked prefill, which makes its latency predictions conservative (Section 6), and the RQ4 ratings were fitted in-sample, with held-out checks in Supplement S5. Energy is GPU board energy read from NVML, so measured joules exclude host and facility shares. The measurements cover one node with three cards, one accelerator generation and one engine version (vLLM 0.11); a new engine version or accelerator calls for re-verification under the rules of Supplement S7. The capacity charge is specified here and remains to be evaluated, settlement is verified in the calibrated model rather than between deployed operators, and a public benchmark separated the lowest quality class from the others but did not order the upper two, so class membership is declared from task-specific evaluation. The tokeniser study uses newsdomain text and excludes two frontier vocabularies that are not public. Both meters sit inside one operator’s trust boundary, so auditability depends on an external auditor and on substitution and count audits that exist as detectors but are not yet deployed as services [45].

IEEE TRANSACTIONS ON SERVICES COMPUTING

8

C ONCLUSION

Raw token counts are not a model-independent measure of inference work: identical content needs 1.2 to 1.6 times more tokens in Malay than in English and up to 7.65 times more in Tamil, and at common context lengths outputtoken cost varies 10 to 48 times with batch occupancy. Under batch sharing, a per-request meter cannot be additive, ambient-independent and exactly cost-reflective at once. Token-as-a-Utility therefore assigns these properties to separate products and recovers residual cost through an occupancy-indexed operator rating. Measurements on the evaluation node confirmed the efficiency, cache-cost and contention assumptions of the design, and on real traffic the occupancy-indexed rating kept the gauge model’s statement energy error within 9%. In the calibrated model, RIUs reduced tenant mispricing from 7.36 times under flat tokens to 2.39 times, and the occupancy true-up reduced it to 1.06 times and stayed accurate on held-out runs and under load shifts. Records carrying the feature vector re-rate exactly across operators and vintages. A per-token latency target held only below a measured contention ratio of about 0.45:1, and accuracy classes are declared per statement. The measurements also added a weight-traffic table for mixtureof-experts models, a gateway-level per-request constant and pinned residency for cold models. A deployed federation awaits the institutions identified above.

DATA AVAILABILITY AND ACKNOWLEDGMENTS Token counts, model constants, hardware measurements and scripts (E1–E8), and the calibrated-model code and outputs are archived on Zenodo, https://doi.org/10.5281/ zenodo.23792887. The concepts, framework, study design and interpretation are the authors’ own. Claude (Anthropic) was used as an assistive tool for code development, data processing and language editing under the authors’ direction; the authors reviewed every analysis and passage, verified all results, and take full responsibility for the content.

R EFERENCES [1]

[2]

[3]

[4]

[5]

Xinhua News Agency, “National data administration: Daily average token calls exceed 140 trillion (scio press conference, 24 march 2026),” news.cn (in Chinese), 2026. [Online]. Available: https://www.news.cn/tech/20260324/ 050774b26bc54f429fce2d21e284949a/c.html GSMA Intelligence, “Telcos and the AI token economy: China sets the pace,” GSMA Intelligence research report, June 2026. [Online]. Available: https://www.gsmaintelligence.com/ research/telcos-and-the-ai-token-economy-china-sets-the-pace Singtel, “Singtel’s RE:AI launches AI tokenas-a-service to accelerate enterprise AI adoption,” News release, 1 October 2026. [Online]. Available: https://www.singtel.com/about-us/media-centre/ news-releases/singtel-reai-launches-ai-token-as-a-service Google Cloud, “Calculate provisioned throughput requirements,” Gemini Enterprise Agent Platform documentation (Vertex AI generative AI), page last updated 2026-10-22 UTC. [Online]. Available: https://docs.cloud.google.com/vertex-ai/generative-ai/ docs/provisioned-throughput/measure-provisioned-throughput Microsoft, “Determine PTU sizing for a workload,” Microsoft Foundry documentation, Microsoft Learn, last updated 202608-21. [Online]. Available: https://learn.microsoft.com/en-my/ Azure/foundry/openai/how-to/provisioned-throughput-sizing

11

[6]

Alibaba Cloud, “Token plan (team edition) – alibaba cloud model studio,” Alibaba Cloud Help Center, published 202607-19, last modified 2026-10-19. [Online]. Available: https: //help.aliyun.com/en/model-studio/token-plan-team-overview [7] Q. Luo, K. Li, Z. Wang, D. Wang, and Y. Chen, “Request-level energy attribution for batched LLM serving,” arXiv preprint arXiv:2608.00026, 2026. [8] Y. Sheng, S. Cao, D. Li, B. Zhu, Z. Li, D. Zhuo, J. E. Gonzalez, and I. Stoica, “Fairness in serving large language models,” in Proc. 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, 2024, pp. 965– 988. [9] R. I. S. Khan, K. Jain, H. Shen, A. Mallick, A. Parayil, A. Kulkarni, S. Kofsky, P. Choudhary, R. St. Amant, R. Wang, Y. Cheng, A. R. Butt, V. Rühle, C. Bansal, and S. Rajmohan, “Ensuring fair LLM serving amid diverse applications,” arXiv preprint arXiv:2411.15997, 2024. [10] C. Li, Z. Du, M. Boucadair, L. M. Contreras, and J. Drake, “A framework for computing-aware traffic steering (CATS),” Internet Engineering Task Force, RFC 10053, 2026. [11] ITU-T, “Computing power network – requirements and functional architecture of the transaction platform,” International Telecommunication Union, Recommendation Y.2504 (04/26), 2026. [12] X. Zhang, X. Xie, Z. He, X. Yin, X. Lin, B. Liang, Z. Cheng, P. Huang, and G. Chen, “Cross-model autoscaling for shared LLM serving,” arXiv preprint arXiv:2609.29160, 2026. [13] M. A. Rappa, “The utility business model and the future of computing services,” IBM Systems Journal, vol. 43, no. 1, pp. 32–42, 2004. [14] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility,” Future Generation Computer Systems, vol. 25, no. 6, pp. 599–616, 2009. [15] G. A. Paleologo, “Price-at-risk: A methodology for pricing utility computing services,” IBM Systems Journal, vol. 43, no. 1, pp. 20–31, 2004. [16] A. Keller and H. Ludwig, “The WSLA framework: Specifying and monitoring service level agreements for web services,” Journal of Network and Systems Management, vol. 11, no. 1, pp. 57–81, 2003. [17] C. Courcoubetis and R. Weber, Pricing Communication Networks: Economics, Technology and Modelling. Chichester, U.K.: John Wiley & Sons, 2003. [18] S. J. Brown and D. S. Sibley, The Theory of Public Utility Pricing. Cambridge, U.K.: Cambridge University Press, 1986. [19] Microsoft, “Provisioned throughput for foundry models,” Microsoft Foundry documentation, Microsoft Learn, last updated 2026-07-15. [Online]. Available: https://learn.microsoft.com/en-my/Azure/foundry/ openai/concepts/provisioned-throughput [20] OpenAI, “API priority processing,” OpenAI product page, 2026. [Online]. Available: https://openai.com/api-priority-processing/ [21] Y. Xing, “AI token futures market: Commoditization of compute and derivatives contract design,” arXiv preprint arXiv:2603.21690, 2026. [22] Antarctica Global Technology and Consulting Pvt. Ltd., “Introducing the one-token model (OTM) v1.0,” Technical note, 1 December 2025. [Online]. Available: https://antarctica.io/ research/one-token-model [23] MLCommons, “MLPerf Endpoints v0.7: A foundation release,” MLCommons blog, 28 July 2026. [Online]. Available: https: //mlcommons.org/2026/07/mlperf-endpoints-v0-7-release/ [24] ACCESS, “Our terminology: ACCESS credits and exchange,” ACCESS documentation (NSF Advanced Cyberinfrastructure Coordination Ecosystem), 2026. [Online]. Available: https://access-ci.atlassian.net/wiki/spaces/ ACCESSdocumentation/pages/467112349/Our+Terminology [25] Amazon Web Services, “Hardware specifications for DB instance classes (Amazon RDS user guide),” Online documentation. [Online]. Available: https://docs.aws.amazon.com/AmazonRDS/ latest/UserGuide/Concepts.DBInstanceClass.Summary.html [26] C. Patil, “Beyond per-token pricing: A concurrency-aware methodology for LLM infrastructure cost estimation,” arXiv preprint arXiv:2606.11690, 2026. [27] S. Jin, X. Liu, J. Shan, L. Xu, T. Zhang, L. Xie, and Z. M. Mao, “LLMVisor: A real-time latency attribution model for multi-tenant LLM serving,” in NeurIPS Workshop on Machine Learning for Systems, 2025.

IEEE TRANSACTIONS ON SERVICES COMPUTING

[28] P. Vellaisamy, V. Lam, S. Blanton, and J. P. Shen, “Characterization of request and token energy costs for LLM inference workloads on GPU platforms,” in Proc. IEEE International Symposium on Workload Characterization (IISWC), 2026. [29] J. Delavande, R. Pierrard, and S. Luccioni, “Understanding efficiency: Quantization, batching, and serving strategies in LLM energy use,” arXiv:2601.22362, 2026. [30] D. Yao and Z. Zhou, “Efficiency and cost alignment in batched LLM serving via resource-fair scheduling,” arXiv preprint arXiv:2608.02244, 2026. [31] I. McDougall and K. Sankaralingam, “Pricing time, not just tokens: Latency-aware mechanism design for LLM inference,” in Proc. 27th ACM Conference on Economics and Computation (EC ’26), 2026. [32] T. Urista, “Who pays for the KV cache? attributing shared AI inference spend across Kubernetes and LLM provider bills,” arXiv preprint arXiv:2609.24991, 2026. [33] X. Yuan, H. Liao, K. Zhang, D. Guo, and G. Tang, “Fine-grained energy accounting in production LLM serving,” Presentation, APNet 2026, Aug. 2026. [Online]. Available: https://conferences.sigcomm.org/events/apnet2026/ [34] G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun, “Orca: A distributed serving system for transformer-based generative models,” in Proc. 16th USENIX Symp. Operating Systems Design and Implementation (OSDI), 2022, pp. 521–538. [35] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” in Proc. 29th Symposium on Operating Systems Principles (SOSP ’23), 2023, pp. 611–626. [36] Y. Zhong, S. Liu, J. Chen, J. Hu, Y. Zhu, X. Liu, X. Jin, and H. Zhang, “DistServe: Disaggregating prefill and decoding for goodputoptimized large language model serving,” in Proc. 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), 2024. [37] S. Chen, Z. Jia, S. Khan, A. Krishnamurthy, and P. B. Gibbons, “SLOs-Serve: Optimized serving of multi-SLO LLMs,” arXiv preprint arXiv:2504.08784, 2025. [38] A. Patke, D. Reddy, S. Jha, H. Qiu, C. Pinto, C. Narayanaswami, Z. Kalbarczyk, and R. Iyer, “Queue management for SLO-oriented large language model serving,” in Proc. ACM Symposium on Cloud Computing (SoCC ’24), 2024, pp. 18–35. [39] Y. Xiang, X. Li, K. Qian, Y. Yang, D. Zhu, W. Yu, E. Zhai, X. Liu, X. Jin, and J. Zhou, “Aegaeon: Effective GPU pooling for concurrent LLM serving on the market,” in Proc. ACM SIGOPS 31st Symposium on Operating Systems Principles (SOSP ’25), 2025, pp. 1030–1045. [40] D. Bali, S. Ponnapalli, Y. Wang, N. Crooks, S. Shenker, and M. Zaharia, “Token latency fairness: Performance isolation for multitenant LLM serving,” arXiv preprint arXiv:2609.18112, 2026. [41] Z. Wei, J. Yen, J. Chen, Z. Zhang, Z. Huang, C. Chen, X. Yu, Y. Gu, C. Wu, Y. Wang, M. Xia, J. Wu, H. Wang, and Z. Qi, “Equinox: Holistic fair scheduling in serving large language models,” arXiv preprint arXiv:2508.16646, 2025. [42] M. Yang, G. Wang, M. Luo, Y. Liu, C. Chen, H. Zhao, Y. Feng, Q. Chen, and M. Guo, “Justitia: Fair and efficient scheduling of task-parallel LLM agents with selective pampering,” arXiv:2510.17015, 2025. [43] S. Hoque, J. Zhang, J. Sun, and F. Suya, “Token inflation: How dishonest providers can overcharge for large language model usage,” arXiv preprint arXiv:2605.30040, 2026. [44] W. Cai, T. Shi, X. Zhao, and D. Song, “Are you getting what you pay for? auditing model substitution in LLM APIs,” arXiv preprint arXiv:2504.04715, 2025. [45] Y. Zhang, Z.-H. Zhang, and H. Qin, “Which model is actually serving you? IRIS: Budgeted black-box auditing of model substitution and routing dilution in LLM gateways,” arXiv preprint arXiv:2607.20860, 2026. [46] A. Petrov, E. La Malfa, P. Torr, and A. Bibi, “Language model tokenizers introduce unfairness between languages,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023. [47] O. Ahia, S. Kumar, H. Gonen, J. Kasai, D. Mortensen, N. Smith, and Y. Tsvetkov, “Do all languages cost the same? tokenization in the era of commercial language models,” in Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2023, pp. 9904–9923. [48] P. Srivastava, “The tokenizer tax: Quantifying and explaining the

12

cross-lingual cost of subword tokenization for Indian languages,” arXiv preprint arXiv:2607.24276, 2026. [49] L. J. Billera and D. C. Heath, “Allocation of shared costs: A set of axioms yielding a unique procedure,” Mathematics of Operations Research, vol. 7, no. 1, pp. 32–39, 1982. [50] L. J. Mirman and Y. Tauman, “Demand compatible equitable cost sharing prices,” Mathematics of Operations Research, vol. 7, no. 1, pp. 40–56, 1982. [51] L. J. Mirman and A. Neyman, “Prices for homogeneous cost functions,” Journal of Mathematical Economics, vol. 12, no. 3, pp. 257–273, 1983. [52] O. Haimanko, “Marginal cost price rule for homogeneous cost functions,” International Journal of Game Theory, vol. 31, no. 1, pp. 19–28, 2002. [53] H. Moulin, “Axiomatic cost and surplus sharing,” in Handbook of Social Choice and Welfare, K. J. Arrow, A. K. Sen, and K. Suzumura, Eds. Elsevier, 2002, vol. 1, ch. 6, pp. 289–357. [54] R. T. B. Ma, D. M. Chiu, J. C. S. Lui, V. Misra, and D. Rubenstein, “Internet economics: The use of Shapley value for ISP settlement,” IEEE/ACM Transactions on Networking, vol. 18, no. 3, pp. 775–787, 2010. [55] GSMA, “Operator platform: Requirements and architecture,” GSM Association, Official Document OPG.02 v12.0, 2026. [56] L. J. Billera, D. C. Heath, and J. Raanan, “Internal telephone billing rates – a novel application of non-atomic game theory,” Operations Research, vol. 26, no. 6, pp. 956–965, 1978. [57] A. Artola Velasco, S. Tsirtsis, N. Okati, and M. Gomez-Rodriguez, “Is your LLM overcharging you? tokenization, transparency, and incentives,” in Proc. International Conference on Machine Learning (ICML), 2026. [58] D. Bergemann, A. Bonatti, and A. Smolin, “The economics of large language models: Token allocation, fine-tuning, and optimal pricing,” in Proc. 26th ACM Conference on Economics and Computation (EC ’25), 2025, p. 786. [59] N. Jegham, M. Abdelatti, C. Y. Koh, L. Elmoubarki, and A. Hendawi, “How hungry is AI? benchmarking energy, water, and carbon footprint of LLM inference,” arXiv preprint arXiv:2505.09598, 2025. [60] International Electrotechnical Commission, “Electricity metering equipment – particular requirements – part 21: Static meters for AC active energy (classes 0,5, 1 and 2),” IEC 62053-21:2020, Edition 2.0. [Online]. Available: https://webstore.iec.ch/en/publication/ 28660 [61] NVIDIA Corporation, “NVIDIA RTX 5000 Ada Generation datasheet,” Datasheet, part no. 2898211 (hosted by HP), 2023. [Online]. Available: https://valuehub.hp.com/static/nvidia/docs/NVIDIA% 20RTX%205000%20Ada%20Generation%20Datasheet.pdf [62] ——, “CUDA C++ best practices guide, release 13.4, sec. 9.2.1 theoretical bandwidth calculation,” Online documentation, 2026. [Online]. Available: https://docs.nvidia.com/cuda/ cuda-c-best-practices-guide/index.html [63] ——, “NVIDIA Multi-Instance GPU User Guide: Supported GPUs,” Online documentation. [Online]. Available: https://docs.nvidia.com/datacenter/tesla/ mig-user-guide/latest/supported-gpus.html [64] ——, “Multi-Process Service: When to use MPS (memory protection and error containment),” Online documentation. [Online]. Available: https://docs.nvidia.com/deploy/mps/latest/ when-to-use-mps.html [65] CMS, “Malaysian guidelines on cross border data transfers,” CMS legal update, 2025. [Online]. Available: https://cms.law/en/int/legal-updates/ malaysian-guidelines-on-cross-border-data-transfers [66] N. Goyal, C. Gao, V. Chaudhary, P.-J. Chen, G. Wenzek, D. Ju, S. Krishnan, M. Ranzato, F. Guzmán, and A. Fan, “The FLORES-101 evaluation benchmark for low-resource and multilingual machine translation,” Transactions of the ACL, vol. 10; arXiv:2106.03193, 2022. [67] F. Cassano, J. Gouwar, D. Nguyen, S. Nguyen, L. Phipps-Costin, D. Pinckney, M.-H. Yee, Y. Zi, C. J. Anderson, M. Q. Feldman, A. Guha, M. Greenberg, and A. Jangda, “MultiPL-E: A scalable and polyglot approach to benchmarking neural code generation,” IEEE Transactions on Software Engineering; arXiv:2208.08227, 2023. [68] Y. Wang, Y. Chen, Z. Li, X. Kang, Y. Fang, Y. Zhou, Y. Zheng, Z. Tang, X. He, R. Guo, X. Wang, Q. Wang, A. C. Zhou, and

IEEE TRANSACTIONS ON SERVICES COMPUTING

X. Chu, “BurstGPT: A real-world workload dataset to optimize LLM serving systems,” arXiv:2401.17644v5; published in Proc. 31st ACM SIGKDD Conf. Knowledge Discovery and Data Mining V.2 (KDD ’25), 2025. [69] Microsoft Azure, “Azure LLM inference trace 2023,” Azure Public Dataset, 2023. [Online]. Available: https://github.com/Azure/ AzurePublicDataset [70] OpenRouter, “Public model catalogue and price list (models api),” https://openrouter.ai/api/v1/models, 2026. [71] GSMA, “GSMA speeds up the transfer of roaming call records,” Press release, 30 August 2007.

13

Related documents

• Large Language Models as Statistical Decision-Makers at Inference Time#634803
• Optimal Model Activation Policies for Inference Networks of Large Language Models#1110390
• Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation#1242544
• Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation#1076515
• LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models#1143744