Which Model Is Actually Serving You? IRIS: Budgeted Black-Box Auditing of Model Substitution and Routing Dilution in LLM Gateways

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Which Model Is Actually Serving You? IRIS: Budgeted Black-Box Auditing of Model Substitution and Routing Dilution in LLM Gateways

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

Which Model Is Actually Serving You? IRIS: Budgeted Black-Box Auditing of Model Substitution and Routing Dilution in LLM Gateways Yuewei Zhang1 , Zhi-Hai Zhang1∗ , Hanzhang Qin2∗

1 Department of Industrial Engineering, Tsinghua University, Beijing, China Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore {[email protected], [email protected], [email protected]}

arXiv:2607.20860v1 [cs.CR] 23 Jul 2026

Abstract Commercial LLM gateways mediate access to hosted models, but the served backend may not match the advertised one: it may substitute a cheaper model on every request or route only a fraction ϵ of requests to it. Prior black-box auditors often need a privileged signal (log-probabilities, token ranks, or reference samples) or a target-specific probe, fix the query budget in advance, and return a yes/no verdict. We present IRIS, an audit that needs only the returned text: it asks endpoints to generate random numbers or strings, fingerprints the backend, and is the first to combine, in one text-only audit, detection of whole-stream substitution and fractional dilution, attribution of the served backend, routing-fraction (ϵ) estimation, and a query budget it sizes itself. A cheap pilot fits the exponential query-error decay and freezes that budget before any suspect query is issued. On an intra-family Qwen3 ladder IRIS verifies the backend at 0.99 AUROC and sharpens attribution as queries accumulate; across a commercial OpenRouter library it catches ϵ=0.3 dilution on margin-qualified pairs at 0.85 mean power (0.017 false-positive rate) and recovers ϵ to within 0.04 for enrolled diluents; and a live cross-provider audit flags 14 of 15 same-model provider pairs by genuine quantization and kernel deviations, corroborated on thirdparty MET traces. Against comparable black-box auditors, IRIS matches or beats detection on shared tasks, and adaptive allocation lifts the matched-budget target-hit rate from 73% to 87%. Further experiments cover adversarial gateways, knob identifiability, unseen diluents, and false-positive control.

Code and data — https://github.com/Photen/IRIS-audit

Introduction

Commercial API gateways and shadow APIs increasingly aggregate LLMs from competing vendors behind one endpoint and in some markets are resold as low-cost commodity routers (Liu et al. 2026). This indirection breaks the link between the model a client requests and the backend that actually serves the call. Measurements document silent downgrades, model switches, billing deviations, and fingerprint failures (Zhang et al. 2026; Lin et al. 2026); formal work studies cheaper, quantized, or randomized substitutes (Cai et al. 2025); and GhostPrint shows a weaker model can be fine-tuned to spoof fingerprinting audits (Zhang, Li, and Wang 2026). We call whole-stream replacement substitution ∗

Corresponding authors.

Task

LLM

Gateway

1 -𝛆

Target Model

𝛆

Weaker Model

IRIS: Information-Rate Inspection from Strings 00101011100 3843648761 ZORAFHQKS

Visible Strings

Fingerprint

Information Rate Estimation

Budget Estimation

𝛆 Dilution Detection

Figure 1: IRIS audits gateway routing from visible strings only: a user task may be served by the intended model or by a weaker substitute; IRIS turns the returned strings into fingerprints, estimates the endpoint information rate, sizes the audit budget, and detects routing dilution. and fractional replacement dilution. Either may arise through misconfiguration, cost control, or misrepresentation, and either requires an audit that infers from returned text alone which model served the request and how much traffic was replaced. Output-only auditors make served-model checking practical (Gao, Liang, and Guestrin 2025; Zhu et al. 2026; Chauvin et al. 2026a; Fang et al. 2026), yet four gaps still separate them from a general endpoint-level audit: (i) probes and references are often endpoint-, reference-, or method-specific rather than reusable across endpoints; (ii) most methods return only a binary substituted-or-not verdict; (iii) the method that does estimate a routing fraction assumes a known substitute under fixed routing; and (iv) query counts are fixed by design, so the auditor cannot tell in advance how many interactions a target error rate requires, or whether a given dilution is detectable within a reasonable budget. IRIS (Information-Rate Inspection from Strings) addresses these gaps by making the audit budget an object of estimation. The auditor probes each endpoint with a randomgeneration challenge whose visible output exposes backendspecific sampling biases (Hopkins, Renda, and Carbin 2023; Zhao, Du, and Wang 2026). Audit-Plan Construction turns these strings into a reusable candidate library, calibrates

thresholds, and freezes the query budget before any suspect traffic is queried. Audit Execution then spends that budget on the gateway, attributes the served backend, and, when the diluting substitute is enrolled, estimates the routing fraction (Figure 1). Contributions. • A reusable audit for cross-model substitution and dilution. To our knowledge, IRIS is the first visible-string random-generation audit that uses reusable probes to detect cross-model gateway substitution, estimate the routing fraction, attribute the served backend, and set the live-query budget before the audit. • An estimate-then-budget strategy for endpoint audits. A cheap pilot fits the exponential rank-error decay, calibrates response-level tell rates (rates of reference-atypical responses), and prescribes the query count a target reliability requires, all before Audit Execution begins. • Theory and guarantees for audit effectiveness. Evidence accumulates across independent queries, not within one long response. When enrollment observes enough response-level tells, dilution cost grows only logarithmically in the target miss probability and roughly inversely with the routing fraction; the measured tail exponent identifies departures from this regime. • Validation on real commercial models. Over claimed– substitute pairs from a K=6 ladder and 17 OpenRouter APIs, the pilot exponent predicts query difficulty and, on the ladder, drives adaptive allocation that meets the target on 87% of pairs versus 73% for a fixed rule at matched budget. At scale, on margin-qualified pairs, IRIS catches ϵ=0.3 dilution at mean power 0.85 with a 0.017 falsepositive rate, names the substitute, and flags an unenrolled one.

Related Work

The market reality of substitution. Measurement studies establish that substitution is real, but they characterize gateway populations rather than giving an error-controlled audit of one endpoint. Zhang et al. (2026) find cross-model impersonation on real shadow APIs, including a GPT-5 endpoint fingerprinting as GLM-4-9B, and report that about 45.8% of endpoints fail fingerprint verification; GateScope records silent downgrades, model switches, and billing deviations across live gateways (Lin et al. 2026); and Liu et al. (2026) broaden the market view to 428 commodity LLM API routers. Model-substitution theory formalizes the same threat class (Cai et al. 2025). Identifying the served model. Black-box identification methods differ mainly in which behavioral signal they read. Input-based methods craft the query, using either discriminative prompt banks (LLMmap; Pasquini, Kornaropoulos, and Ateniese 2025) or adversarial suffixes (TRAP; Gubri et al. 2024). Knowledge-based methods read what the model knows, through factual-capacity probes (IKP; Li 2026), combined trigger-pattern and knowledge-level fingerprints (DuFFin; Yan et al. 2026), or recent-event recall that locates a

training cutoff (LLMLagBench; Pęzik et al. 2025). Outputbased methods read how the model writes, from characteristic wrong-answer patterns (Bradley 2024), word-level and semantic idiosyncrasies (Sun et al. 2025), or provenance and derivation between models (Nikolic, Baluta, and Saxena 2025). Close to our signal, FLIPS (Richardeau et al. 2026) trains a supervised classifier on pseudo-random sequence bias to fingerprint deployment configurations at a fixed query count, while IRIS can further estimate the audit budget and the routing fraction. Most methods require a model-specific probe or reference and return a yes/no verdict, not a routing fraction that a client may tolerate, price, or contest. Detecting substitution and dilution. Substitution and dilution auditors also vary by access assumption. KBF (Fang et al. 2026) uses numerical recall at the knowledge boundary and diagnostically estimates the routing fraction for a known substitute under fixed routing; B3IT (Chauvin et al. 2026a) uses low-temperature border inputs; Log Probability Tracking (Chauvin et al. 2026b) uses a single token’s mean log-probability; MET (Gao, Liang, and Guestrin 2025) and RUT (Zhu et al. 2026) use reference-based equality and rankuniformity. Classical testing theory gives exponential error decay in independent samples (Chernoff 1952; Hoeffding 1965; Cover and Thomas 2006), but dilution is harder than substitution: only an ϵ fraction of responses carries substitute evidence, a low-rate regime that Cai et al. (2025) treat as a reason to fall back on trusted hardware. The novelty is therefore not dilution alone: IRIS pairs reusable randomgeneration probes with an estimate-then-budget plan frozen before any suspect query, and one batch of visible strings supports detection, attribution, and enrolled-diluent rate estimation.

Problem Formulation

We formalize endpoint auditing as follows. A client requests a nominal model through an API gateway. A trusted reference endpoint serves that model faithfully, producing visible outputs from distribution P that the auditor may sample during Audit-Plan Construction. A suspect endpoint advertises the same model but may instead serve a different, cheaper base model R on some or all requests. We model its served distribution as Qϵ = (1 − ϵ) P + ϵ R,

ϵ ∈ [0, 1],

(1)

so ϵ=0 is honest service, ϵ=1 is substitution, and ϵ ∈ (0, 1) is dilution, with the substitute R serving as the diluent. Thus, substitution replaces the advertised model on every request, whereas dilution replaces it on only a fraction ϵ of requests. Temperature, decoding, or reasoning-effort changes may affect output quality, but under token-count billing they do not create extra gateway profit and therefore are not treated as auditing targets. Observation model. The audit is strictly black-box: the auditor fixes a probe (a query prompt) and issues m independent calls to the suspect endpoint, observing only the i.i.d. returned visible strings y1 , . . . , ym ∼ Qϵ , with no weights, log-probabilities, or token ranks exposed. The reference P

and a library {Rj } of candidate substitutes can be sampled under known labels during Audit-Plan Construction, whereas the suspect stream is unlabeled. Audit goals. From the m suspect outputs, the auditor performs three increasingly informative tasks: (1) detect any deviation from the nominal model by testing H0 : ϵ=0 against H1 : ϵ>0 at false-positive rate α (flagging an honest endpoint) and miss probability δ (failing to flag substitution or dilution); (2) estimate the routing fraction ϵ; and (3) given the candidate library {Rj }, attribute the served substitute R. The returned decision is (flag ∈ {0, 1}, ϵ̂, R̂), with R̂ = ∅ when no substitute is named.

The IRIS Audit

IRIS has two stages. Audit-Plan Construction collects labeled responses from the trusted reference and candidate models, builds their fingerprints, and generates a userselected audit plan for either substitution or dilution. Audit Execution uses the plan to query the suspect endpoint, detect the selected deviation type, and identify the served backend when applicable. Although substitution is the ϵ=1 special case of dilution (Eq. (1)), we retain a dedicated mode because market audits often need the simpler, direct decision of whether an endpoint consistently serves the advertised backend. Accordingly, substitution aggregates episode-level evidence under a one-backend assumption, while dilution uses response-level tells to detect and quantify fractional routing.

4.1

Audit-Plan Construction

Probe design and labeled data collection. The audit uses generation probes that ask the endpoint for short random strings: a coin-style binary draw, a single digit, an integer in a range, or a short bit/digit sequence. We write cn,L for a probe with per-draw alphabet or range size n and requested length L, so a single binary draw is c2,1 ; below, c denotes an arbitrary probe. Because these tasks have no factual answer to memorize, the returned strings expose backend-specific sampling biases under black-box access. IRIS uses a task-agnostic probe set C that varies alphabet size and response length; the full set is provided in App. S22. For every c ∈ C, Audit-Plan Construction collects labeled responses from the trusted reference P and candidate substitutes {Rj }Jj=1 . An audit plan covers candidate indices Jaudit ⊆ {1, . . . , J} and follows the user-selected mode z ∈ {s, d}. Visible-string classifier and score. IRIS maps each response y to a fixed 179-dimensional visible-string vector ϕ(y) of format-compliance, symbol-frequency, transition, run-structure, and positional-balance statistics (listed in App. S23). For a fixed probe c, the labeled responses train a multiclass classifier gc whose output is a posterior over the enrolled endpoint labels M ∈ M = {P, R1 , . . . , RJ }: P̂c (M | y) = gc (ϕ(y)) M . (2) The deployed score is the reference negative log-posterior, sc (y) = − log P̂c (P | y).

(3)

Larger values mean that the response is less typical of the trusted reference for probe c. Given m independent responses y1 , . . . , ym , IRIS averages the score into the episode-level evidence statistic m 1 X Sm (c) = sc (yi ). (4) m i=1 When the selected probe is clear from context, we write s(y) and Sm . Audit-Plan Construction also stores the posterior signature ḡM,c , the mean of uc (y) = P̂c (· | y) over endpoint-M enrollment responses. The stored posterior vector, signature, and score are available to either execution mode. Audit-Plan Construction calibrates the two branchspecific budgets independently, but the deployed plan freezes only the branch requested by audit mode z. Substitution audit budget. The substitution branch uses a mean-evidence threshold and a query budget. For each probe c and enrolled pair (P, Rj ), a cheap labeled pilot estimates the rate at which episode-level rank error decays. Define P,c Rj ,c AUROCcj (m) = P{Sm < Sm }

P,c Rj ,c + 21 P{Sm = Sm },

(5)

R ,c

P,c where Sm and Smj are independent m-query episode means from P and Rj . Because 1 − AUROC decays exponentially in m (see Prop. 1), the pilot fits

log(1 − AUROCcj (m)) ≈ âcj − Ibauc,cj m, m ≤ 4, (6)

where âcj is the fitted intercept and Ibauc,cj is the operational rank-error exponent. Because the m ≤ 4 extrapolation is mildly optimistic, IRIS uses a one-sided (1−γ) lower bound I auc,cj and upper bound acj , both t-based over split-level fits (App. S17). With target rank error δauc =δ, the substitution budget is acj − log δauc msub (c) = max . (7) j∈Jaudit I auc,cj The mean-evidence threshold is also generated during Audit-Plan Construction. For a probe c and live budget m, group held-out reference responses into Bmean batches, comP,c pute Sm,b by Eq. (4), and take b 1−α {S P,c }Bmean . τmean (c, m) = Q (8) m,b b=1

After probe selection, the substitution plan freezes τmean (c⋆ , m⋆ ), so an honest reference episode exceeds it with calibrated probability ≤ α.

Dilution audit budget. Dilution needs a response-level test because a substitute may serve only an ϵ fraction of queries. On a calibration split separate from the data used to train gc , a threshold τ turns a response into a tell when sc (y) > τ . IRIS searches only reference quantiles: for a small honest-tail grid Aresp , n o b 1−α {sc (yrP )}nP : α0 ∈ Aresp . Tc = Q (9) 0 r=1

P are held-out reference responses. For each canHere y1:n P R didate substitute Rj with held-out responses y1:nj j , threshold τ has calibrated honest and substitute tell rates nP 1 X α b1,cj (τ ) = 1{sc (yrP ) > τ }, (10) nP r=1 nj

1 X 1{sc (yrRj ) > τ }. qbcj (τ ) = nj r=1

(11)

Let Tcj+ collect thresholds satisfying qbcj (τ ) > α b1,cj (τ ); an empty set marks (c, j) low-margin. At a usable threshold, an ϵ-diluted response is a tell with rate pbϵ,cj (τ ) = (1 − ϵ)b α1,cj (τ ) + ϵ qbcj (τ ). Let Um (b; p) = PK∼B(m,p) (K ≥ b) and bα (m, p0 ) = min{b : Um (b; p0 ) ≤ α}. The resulting level-α power is πϵ,cj (m, τ ) = Um (bα (m, α b1,cj (τ )); pbϵ,cj (τ )) .

For target dilution ϵmin , the query budget at threshold τ is n mdil (c, j; τ ) = min m : πϵmin ,cj (m′ , τ ) ≥ 1 − δ o for all m′ ≥ m , (12)

IRIS sets τcj ∈ arg minτ ∈T + mdil (c, j; τ ) and writes the cj corresponding rates and budget as α1,cj , qcj , mdil (c, j). Plan selection. Plan selection is mode-specific. For z = s, all probes with valid pilot fits are eligible. For z = d, IRIS first forms Cfeas , the probes with Tcj+ ̸= ∅ for every j ∈ Jaudit . The live-query budget for probe c is msub (c), z = s, ⋆ m (c; z) = maxj∈Jaudit mdil (c, j), z = d.

IRIS selects a lowest-budget eligible probe c⋆ for the chosen mode, breaking ties by lower expected response cost, and denotes the selected budget m⋆ (c⋆ ; z) by m⋆ . The selected m⋆ fixes the online standards: τmean (c⋆ , m⋆ ) for substitution and, for each audited j, (τc⋆ j , b⋆j ) for dilution, with b⋆j = bα (m⋆ , α1,c⋆ j ).

4.2

Audit Execution

Audit Execution is the only stage that queries the suspect end⋆ point. It sends probe c⋆ exactly m⋆ times, collects {yi }m i=1 , computes the score and posterior quantities fixed during Audit-Plan Construction, and applies only the decision rule for mode z. Substitution audit execution. When z = s, IRIS computes Sm⋆ (c⋆ ) by Eq. (4) and sets flag = 1{Sm⋆ (c⋆ ) > τmean (c⋆ , m⋆ )}. All responses are treated as coming from one backend and attributed by aggregated log posterior, ⋆

M̂sub = arg max

M ∈M

m X i=1

log P̂c⋆ (M | yi ).

(13)

For a flagged episode, the substitute is named as M̂sub if M̂sub ̸= P , or ∅ if the posterior favors the reference.

Dilution audit execution. Each candidate j ∈ Jaudit uses Pm⋆ ⋆ ⋆ its frozen tell threshold: kj = i=1 1{sc (yi ) > τc j }, fˆj = kj /m⋆ , and flagj = 1{kj ≥ b⋆j }. Let J+ = {j : flagj = 1} and flag = 1{J+ ̸= ∅}. For every flagged candidate with qc⋆ j > α1,c⋆ j , the same tell fraction estimates the routing fraction. Suppressing the fixed (c⋆ , j) indices, ϵ̂ = clip[0,1]

fˆ − α1 . q − α1

(14)

For an enrolled diluent, Eq. (14) gives a calibrated estimate; if the diluent is unenrolled, q is unknown and q ≤ 1 gives the lower bound ϵ̂lb = clip[0,1] ((fˆ− α1 )/(1 − α1 )). A deltamethod interval propagates uncertainty in (fˆ, α1 , q). The branch identifies the diluent from the Pfull posterior vectors of the same batch. Let v̄ = (m⋆ )−1 i uc⋆ (yi ) and abbreviate the frozen signature ḡM,c⋆ as ḡM . The executionstage attribution weights η̂ are the nonnegative solution to the least-squares problem X 2 min v̄ − (1 − 1⊤ w)ḡP − wj ḡRj . (15) 1⊤ w≤1

Among flagged candidates, ĵ = arg maxj∈J+ η̂j . Let h = maxM ∈M v̄M and let τos be its fifth percentile on matched held-out enrolled episodes. The diluent is named as Rĵ when h ≥ τos and left unknown otherwise. Algorithm 1 uses the thresholds defined above: τmean for substitution, and (τcj , b⋆j , τos ) for dilution. Algorithm 1 The IRIS audit Input: P , {Rj }, suspect endpoint, C, Jaudit , z, α, δ, γ, ϵmin Output: (flag, ϵ̂, R̂) ▷ Stage I: construct the audit plan 1: for c ∈ C do 2: collect labeled responses from P and {Rj } 3: train gc ; store score sc and posterior signatures 4: calibrate the thresholds and set mc ← m⋆ (c; z) 5: end for 6: c⋆ ← arg min{mc : c is eligible} 7: m⋆ ← mc⋆ ; freeze the thresholds for c⋆ ▷ Stage II: execute the frozen audit plan ⋆ ⋆ 8: collect {yi }m i=1 from the suspect endpoint using c 9: if z = s then 10: apply the frozen substitution rule to obtain (flag, R̂) 11: ϵ̂ ← flag 12: else 13: for j ∈ Jaudit do 14: compute flagj , ϵ̂j , and lower bound ℓj 15: end for 16: J+ ← {j ∈ Jaudit : flagj = 1} 17: (flag, ϵ̂, R̂) ← (1{J+ ̸= ∅}, 0, ∅) 18: if flag = 1 then 19: compute η̂; set ĵ ← arg maxj∈J+ η̂j 20: (ϵ̂, R̂) ← (ϵ̂ĵ , Rĵ ) if the gate accepts; else (ℓĵ , ∅) 21: end if 22: end if 23: return (flag, ϵ̂, R̂)

Service deployment. When IRIS is exposed as an ondemand audit service, Audit-Plan Construction can run as a low-frequency background job that periodically refreshes. Each user request then triggers only Audit Execution using the latest frozen plan. This temporal decoupling amortizes construction cost across audits and keeps user-triggered service fast and inexpensive.

Audit Sample Complexity

The audit budget is a sample-complexity question: at falsepositive rate α and miss probability δ, how many independent suspect queries are needed, before the audit begins, to distinguish the suspect endpoint from the trusted reference? Evidence accumulates across the number of queries m, not the length of one response (Prop. 2). For substitution, the mean score Sm has a positive error exponent (Prop. 1), so verification error falls exponentially in m and a small pilot can estimate the rate. For dilution, only an ϵ fraction of queries carries substitute evidence, so the response-level tell count becomes the right statistic, and the gap between substitute and reference tell rates sets the cost (Thm. 1): a wide margin gives the favorable m = Θ(ϵ−1 ) regime, a narrow margin the m = Θ(ϵ−2 ) wall. All proofs are in App. S28. Response model. Open-ended generation ranges over a model’s full tokenizer vocabulary, whose support is too large and task-dependent for a direct audit; the probes of Section 4.1 instead concentrate the visible output on a small known alphabet or range, so the parsed symbols support tractable score calibration, tell-rate estimation, and samplecomplexity analysis. Fix a probe context c and a requested response length L; a query to an endpoint serving model M returns a parsed visible response y = y1:L drawn from the autoregressive law QL PM (y | c) = t=1 PM (yt | y<t , c) . (16)

Under the fixed probe c, let P and R denote the reference and substitute response distributions. Symbols within one response are dependent through y<t , whereas separate API calls are independent; the audit budget therefore counts calls m, not tokens within one call.

5.1

Substitution

Four rates play distinct roles: the unattainable oracle benchmark, the information retained by the scalar score, the rate achieved by the deployed fixed-threshold mean test, and the empirical slope IRIS uses for budget prediction. Definition 1 (Audit exponents). For a probe c, response laws P, R, and score s(y) = − log P̂(P | y), define: 1. The oracle exponent is the full-response Chernoff rate X I ⋆ = − min log P (y)1−λ R(y)λ . 0≤λ≤1

2. The score exponent I is the same Chernoff information after mapping the response to the scalar score s(y). 3. The mean exponent I mean is the large-deviations rate at which the deployed fixed-threshold test on the score mean sc

Pm 1 Sm = m i=1 s(yi ) decays, i.e. the maximin crossing ∗ maxτ min{IP∗ (τ ), IR (τ )} of the Cramér rate functions of s under P and R (App. S28, Eq. (10)). 4. The pilot exponent Ibauc is the fitted slope of log(1 − AUROC(m)) on calibration episodes.

The pilot rate is a budgeting fit, not an informationtheoretic lower bound: because AUROC compares an msample statistic from P with an independent one from R, Ibauc is not generally ordered below I sc . IRIS therefore budgets against its lower-confidence bound (Eq. (7)) and verifies the transfer empirically.

Proposition 1 (Mean-score separation). Assume the fixed IRIS score s(y) is bounded and has different means under P and R. Then the episode mean Sm admits a fixed threshold whose error decays as

Perr (Sm ) = exp{−I mean m(1 + o(1))}, I mean > 0. (17) The deployed rate is bounded by the score-level and fullresponse oracle rates: I mean ≤ I sc ≤ I ⋆ .

The oracle rate I ⋆ provides comparison benchmarks: a ⋆ full-string Bayes test would achieve e−I m(1+o(1)) , and Kway identification among enrolled endpoints pays roughly ⋆ m ≳ log(K/δ)/Imin . Proposition 2 (Query accumulation). Let Iseq (L) be the Chernoff information in one full length-L response. For fixed L, m independent API calls accumulate linearly, C(P ⊗m , R⊗m ) = m Iseq (L). Longer same-context responses can only add oracle information, but one autoregressive response is not L independent position tests. The length non-monotonicity observed in Section 6 concerns the deployed score and prompt-length conditions, not the oracle monotonicity in Prop. 2.

5.2

Dilution

Under dilution, each suspect query is drawn i.i.d. from the mixture Qϵ of Eq. (1); the mean statistic Sm still detects large shifts, but low-rate dilution is better read from the tells of Section 4.1. For a fixed response-level threshold τ on the score s(y), the population honest and substitute tell rates are α1 (τ ) = Py∼P (s(y) > τ ) and q(τ ) = Py∼R (s(y) > τ ). Theorem 1 (Tell-rate budget). Fix α, δ, ϵ and a threshold τ . Dilution detection is governed by the honest and substitute tell rates (α1 (τ ), q(τ )): (a) If the suspect routes at least an ϵ fraction to R, the any-tell test (flag if any tell occurs) detects within m queries with probability at least 1 − (1 − ϵq(τ ))m ≥ 1 − e−ϵq(τ )m , so l m m⋆ = ln(1/δ) / ϵ q(τ ) (18)

queries guarantee power at least 1 − δ. (b) On an honest endpoint, the false-positive probability is at most 1 − (1 − α1 (τ ))m ≤ mα1 (τ ); hence α1 (τ ) ≤ α/m controls type-I error at level α.

Probe-design consequences. First, requested length is an unreliable budget axis: the token-optimal length maximizes the per-token tell yield q(L)/L rather than simply growing L (Cor. 2, App. S8). Second, temperature insensitivity is intended specificity, not a blind spot: a pure decodingtemperature retune is a rank-one, on-family move with only second-order separation, and at the greedy boundary repeated single draws add no evidence (Cor. 1, Prop. 1; App. S1, S25), so IRIS treats retunes as out of scope rather than as dilution. A true backend substitution, by contrast, perturbs the logits off that on-family curve and can supply first-order evidence when the pair is measurably separated.

Experiments

No public benchmark targets gateway substitution/dilution auditing, so we build testbeds following prior output-only auditors (Gao, Liang, and Guestrin 2025; Zhu et al. 2026; Richardeau et al. 2026; Cai et al. 2025) and validate on real endpoints, corroborated by the third-party MET corpus (App. S15). The evaluation has two parts: a controlled local testbed (Ollama; 6×A40, Xeon 8358, 2 TiB RAM, Ubuntu 22.04) checks the sample-complexity predictions under same-family substitutions and hosts the baseline headto-head; a commercial OpenRouter testbed then evaluates the same budget laws under constructed gateway dilution at scale.

6.1

Controlled local testbed

The local testbed hosts a deliberately hard K=6 same-family Qwen3 ladder (0.6B–32B). All six probes of C (Section 4.1) run at T =1.0 (120 repeats per model–probe; a sweep covers T ∈ {0, 0.5, 1, 1.5, 2}). The classifier is a 300-tree random forest over the 179 visible-string features; evidence aggregates m independent responses, and each curve averages 40 stratified splits with 95% bootstrap intervals. Substitution/endpoint claims use the content-only (length/formatinvariant) subset; law, budget, and probe-ranking analyses use the full set with a length ablation (Table 9, App. S11).

c100,1

(b) dilution (=0.2) 1.0

c10,8

AUROCdil

1 − AUROC

c2,1

10−1 10

−2

queries m

0.8 0.6

101 dilutions 1/

IRIS RUT chance

queries m

(d) deployed 17-API audit power @ m=80

The near-1/ϵ case is a measured tail event, not a generic consequence of P ̸= R: low-entropy autoregressive conditionals can put most P mass on a thin slice of response space, so an R-typical response may fall deep in the reference tail, with q(τ ) staying Ω(1) while α1 (τ ) is pushed near zero. This is the regime behind many gateway dilutions. With finite enrollment data, confidence bounds on (α1 , q) determine whether the requested cheap audit is certified for the pair or must fall back to the ϵ−2 mean-shift wall of Eq. (19).

(a) error law e−Iauc m

queries to detect

(c) A separating tail with q(τ ) = Ω(1) and α1 (τ ) = o(ϵ) gives the favorable m⋆ = Θ(ϵ−1 log(1/δ)) law. Without such a tail, light-tail low-separation mixtures face the local budget m⋆ = Θ (z1−α + z1−δ )2 ϵ2 χ2 (R∥P ) = Θ(ϵ−2 ), (19) with an all-test Ω(ϵ−2 ) floor for sufficiently lowseparation substitutes.

1.0

0.5

known diluent unknown (LOO) power 0.95

0.0 10−1

routing fraction

100

0.0

0.1

0.2

0.3

0.4

routing fraction

0.5

Figure 2: Experimental overview. (a) Verification error falls b as e−Iauc m : the ladder exemplar c100,1 , the same decay on 17 commercial APIs (c10,8 , dashed), and the entropy-capped c2,1 (flat). (b) Dilution evidence accumulates for IRIS but not for RUT (c2,16 ; MET and FLIPS cannot audit dilution, Table 1). (c) Margin-qualified commercial dilutions cost near1/ϵ queries to detect. (d) Deployed m=80 gateway audit: known and leave-one-out (unknown) diluent power nearly coincide; ϵ=0 realizes the honest FPR 0.017. Metrics. Detection is scored by FPR and power (1−miss), the empirical counterparts of α and 1−δ from Section 3; attribution by accuracy in naming the served model among the K candidates; and quantification by the routing-fraction error |ϵ̂ − ϵ|. AUROC ranks honest against deviating m-query episodes by the evidence Sm (0.5 chance, 1 perfect), with AUROCsub and AUROCdil denoting substitution and dilution (at the stated ϵ), respectively. Hit@B is the fraction of (claimed, candidate) pairs whose audit meets the target reliability at mean query budget B; we write X@m for metric X at budget m. Queries, not length. Verification error decays exponentially in queries, as Prop. 1 predicts: 1 − AUROC falls b log-linearly, matching the empirical e−Iauc m rank-error 2 law (Fig. 2a; c100,1 fit R =0.96), and attribution accuracy rises monotonically with independent queries (Prop. 2; 0.62 → 0.95 over m=1→48). The fingerprint is randomness, not formatting or length: dropping length/format features (179→144) leaves the lead sequence probes c2,16 and c10,8 effectively unchanged at m=8 (≥.998). Budgets and probes from a cheap pilot. Fit on a cheap pilot (m ≤ 4), Ibauc predicts held-out 1−AUROC at larger m to a median | log-ratio| of 0.19 over 224 points, allowing msub to be frozen before any suspect query (conservative LCB; App. S17). The same pilot orders probes by realized efficiency: ρ(Ibauc , acc@8)=0.82 over 13 probes, the six enrolled plus seven further cn,L variants (p<10−3 ; App. S19.1), with high-rate sequence probes reaching AUROC ≥0.95 in a single query while c2,1 never does within 48. The exponent (Def. 1; Prop. 4, App. S24) also follows the predicted

Metric

IRIS

FLIPS

MET

RUT

AUROCsub .993/.9997 .990/.9999 .111/.993 .634/.726 AUROCdil .902/.995 – – .575/.631 ϵ̂ err. .04 – – – Attr. .927/.998 .921/.999 – – Hit@B .87@9 fixed fixed fixed

Table 1: Head-to-head with prior black-box auditors on the shared K=6 Qwen3 ladder and probe c2,16 (30 ordered pairs). AUROC rows give two budgets (m=1/8 for substitution; m=8/32 for dilution at ϵ=0.2); Attr. is closed-set attribution accuracy (m=1/8); bold is the column-wise best. “–” = task not supported or not reported; “fixed” = a fixed, non-estimated budget. entropy–temperature gate (Prop. 1, Cor. 1; Fig. 1, App. S1): it is ≈ 0 for the one-bit c2,1 , peaks near T ≈1, and vanishes under greedy T =0. Because a pure temperature retune is only second order in this gate (Prop. 5), IRIS does not falseflag honest retunes (ϵ̂=0.04 vs. 0.53 for a base-model swap; App. S10, S10.1). Comparison with prior auditors. This head-to-head uses the controlled K=6 Qwen3 ladder, where FLIPS (Richardeau et al. 2026), MET (Gao, Liang, and Guestrin 2025), and RUT (Zhu et al. 2026) can all run (MET needs reference samples, RUT log-ranks); Table 1 lists probe and budget settings. IRIS gives the strongest results on nearly every reported metric and is the only compared method that simultaneously detects substitution, estimates the routing fraction ϵ̂, and attributes the served model. MET-MMD is degenerate at a single draw and RUT stays weak on dilution (Fig. 2b). KBF (Fang et al. 2026) and B3IT (Chauvin et al. 2026a) use target-specific probes (factual recall, border inputs) we cannot reproduce on this shared random probe, so App. S14 compares them qualitatively. Because every compared baseline fixes its query count, we also isolate budget estimation itself: at matched mean budget m≈9 on c10,1 , pilot-driven allocation hits the target (Hit@B) on 0.87 of pairs versus 0.73 for fixed m (decomposition in App. S14).

6.2

Commercial models via OpenRouter

The local laws transfer to the 17 commercial and open models accessed through OpenRouter’s API (Fig. 2a, dashed): c10,8 attribution rises from 0.68 to 0.98 over m=1→32 and AUROC reaches 0.99 by m=8, the same Prop. 1 signature as on the ladder. Entropy gating persists, requested length again shows no stable relationship with accuracy, and the pattern remains consistent when extended to 45 models (App. S9). App. S19 renders the 53 enrolled endpoints (18 families) as an iris gallery. A near-1/ϵ dilution budget. Dilution across distinct commercial models follows near-1/ϵ scaling (Fig. 2c): in the AUROC-rank diagnostic on c10,8 (the median over pairs of the smallest m whose honest-vs-diluted episode AUROC reaches 0.95), ϵ=5/10/20/40% is detected in roughly 64/32/16/8 queries. The deployed fixed-p0 audit below (the

honest tell rate p0 is frozen in advance) instead pays the ϵ−2 rate for type-I robustness, but a pre-fixed-FPR test reproduces the same near-1/ϵ slope given enough reference calibration (App. S4); same-model temperature twins sit at the ϵ−2 wall. This is the Θ(ϵ−1 log(1/δ)) law of Thm. 1 when a separating tail exists, with the 1/ϵ-to-ϵ−2 crossover set by the measured tail exponent (Prop. 6, App. S26). Across the 17and 45-model pools, 62–85% of margin-qualified pairs are consistent with the exact 1/ϵ law, and budgeting uses each pair’s measured exponent (App. S26.1). Detection, ϵ̂, and unseen diluents. We emulate a diluting gateway from real OpenRouter responses: for each ordered (claimed P , diluent R) pair among the 17 models, the diluted stream draws each query i.i.d. from Qϵ in Eq. (1). We fix τ and the honest tell rate p0 on a fresh honest window of P , then run a content-only m=80 two-proportion test on held-out diluted streams; this is the deployed fixed-p0 audit anticipated above, not the any-tell 1/ϵ diagnostic. At ϵ=0.3, IRIS detects the 218/272 margin-qualified pairs at mean power 0.85, names the enrolled substitute, and keeps the pooled in-distribution FPR at 0.017 (Fig. 2d). Sweeping ϵ ∈ [0, 0.5], power rises monotonically and ϵ̂ tracks the truth; App. S3 adds five live OpenRouter audits, FPR hardening, and a held-out marginselection check. The 54 low-margin pairs are known-hard cases: close relatives that may share training lineage (within Qwen3, GPT-4 or Gemini–Gemma), and claimed models whose single-response fingerprint is itself non-distinctive. The fraction estimate tracks truth to |bias| ≈ 0.04 conditional on the response library, with near-nominal though slightly optimistic coverage (App. S2); App. S5 audits the population (α, δ) guarantee (Prop. 2). Detection does not require enrolling the diluent: leave-one-out barely changes power (0.78 vs. 0.85), turns ϵ̂ into a lower bound, and flags the never-enrolled qwen-2.5-72b. Real endpoints and stress tests. A real cross-provider audit (one open-weight model pinned across OpenRouter providers) flags 14/15 provider pairs as distinguishable, real quantization/kernel deviations rather than author-injected, corroborated on the third-party MET corpus (App. S15). The appendix adds the checks a referee expects: probeparaphrase and probe-aware-gateway robustness, non-i.i.d. routing, a q4-for-fp16 cheat, knob identifiability, 45-model scale, exact-binomial FPR hardening, a one-class detector, multi-diluent unmixing, and matched-budget baselines with significance tests (App. S14–S15).

Conclusion

IRIS is a budgeted black-box framework for auditing model substitution and routing dilution in LLM gateways from returned text alone: random-generation probes and an estimatethen-budget pilot predict audit difficulty before querying traffic and reuse those responses for detection, attribution, and routing-fraction estimation. Our analysis explains exponential evidence accumulation and the near-1/ϵ versus ϵ−2 dilution budgets, which a Qwen3 ladder, a 45-model library, a live cross-provider audit of real deviations, and third-party MET traces confirm, with appendices hardening false-positive control and unseen diluents.

References

Bradley, W. F. 2024. LLMs and the Madness of Crowds. arXiv preprint arXiv:2411.01539. Cai, W.; Shi, T.; Zhao, X.; and Song, D. 2025. Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs. arXiv preprint arXiv:2504.04715. Chauvin, T.; Lalanne, C.; Le Merrer, E.; Loubes, J.-M.; Taïani, F.; and Tredan, G. 2026a. Token-Efficient Change Detection in LLM APIs. In International Conference on Machine Learning (ICML). Chauvin, T.; Le Merrer, E.; Taïani, F.; and Tredan, G. 2026b. Log Probability Tracking of LLM APIs. In International Conference on Learning Representations (ICLR). Chernoff, H. 1952. A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations. The Annals of Mathematical Statistics, 23(4): 493–507. Cover, T. M.; and Thomas, J. A. 2006. Elements of Information Theory. Wiley-Interscience, 2nd edition. ISBN 9780471241959. Fang, Y.; Feng, Y.; Li, B.; and Zhou, M. 2026. KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing. arXiv preprint arXiv:2605.29524. Gao, I.; Liang, P.; and Guestrin, C. 2025. Model Equality Testing: Which Model Is This API Serving? In International Conference on Learning Representations (ICLR). Gubri, M.; Ulmer, D.; Lee, H.; Yun, S.; and Oh, S. J. 2024. TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification. In Findings of the Association for Computational Linguistics: ACL 2024, 11496–11517. Bangkok, Thailand: Association for Computational Linguistics. Hoeffding, W. 1965. Asymptotically Optimal Tests for Multinomial Distributions. The Annals of Mathematical Statistics, 36(2): 369–401. Hopkins, A. K.; Renda, A.; and Carbin, M. 2023. Can LLMs Generate Random Numbers? Evaluating LLM Sampling in Controlled Domains. In ICML Workshop on Sampling and Optimization in Discrete Space (SODS). Li, B. 2026. Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity. arXiv preprint arXiv:2604.24827. Lin, G.; Wan, Y.; Pei, S.; Xu, T.; Xu, K.; and Xue, G. 2026. Behavioral Consistency and Transparency Analysis on Large Language Model API Gateways. In Proceedings of the 2026 ACM Internet Measurement Conference (IMC ’26). Liu, H.; Shou, C.; Wen, H.; Chen, Y.; Fang, R. J.; and Feng, Y. 2026. Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain. arXiv preprint arXiv:2604.08407. Nikolic, I.; Baluta, T.; and Saxena, P. 2025. Model Provenance Testing for Large Language Models. In Advances in Neural Information Processing Systems, volume 38, 34126– 34153. Pasquini, D.; Kornaropoulos, E. M.; and Ateniese, G. 2025. LLMmap: Fingerprinting For Large Language Models. In

34th USENIX Security Symposium (USENIX Security 25), 299–318. Pęzik, P.; Kaczyński, K.; Szymańska, M.; Żarnecki, F.; Deckert, Z.; Kwiatkowski, J.; and Janowski, W. 2025. LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models. arXiv preprint arXiv:2511.12116. Richardeau, G.; Dashyan, G.; Le Merrer, E.; and Tredan, G. 2026. FLIPS: Instance-Fingerprinting for LLMs via Pseudorandom Sequences. In International Conference on Machine Learning (ICML). Sun, M.; Yin, Y.; Xu, Z.; Kolter, J. Z.; and Liu, Z. 2025. Idiosyncrasies in Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 57854–57885. PMLR. Yan, Y.; Tang, H.; Yan, S.; and Dai, E. 2026. DuFFin: A DualLevel Fingerprinting Framework for LLMs IP Protection. In Findings of the Association for Computational Linguistics: EACL 2026, 5168–5184. Rabat, Morocco: Association for Computational Linguistics. Zhang, J.; Li, X.; and Wang, S. 2026. Your “Pro” LLM Subscription May Actually Be “Free”: Exposing Fingerprint Spoofing Risks in LLM Inference Services. arXiv preprint arXiv:2606.16100. Zhang, Y.; Jiang, Y.; Chen, Z.; Backes, M.; Shen, X.; and Zhang, Y. 2026. Real Money, Fake Models: Deceptive Model Claims in Shadow APIs. arXiv preprint arXiv:2603.01919. Zhao, M.; Du, Y.; and Wang, M. 2026. Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributions. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 22942– 22959. San Diego, California, United States: Association for Computational Linguistics. Zhu, X.; Ye, Y.; Qiu, T.; Zhu, H.; Tan, S.; Mannan, A.; Michala, J.; Popa, R. A.; and Neiswanger, W. 2026. Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test. In International Conference on Learning Representations (ICLR).

Supplementary Material for IRIS: Budgeted Black-Box Auditing of Model Substitution and Routing Dilution in LLM Gateways Yuewei Zhang, Zhi-Hai Zhang, Hanzhang Qin concerns the increment from repeating an identical single draw. The highest-entropy probe c2,16 is already saturated at m=1 for T ≤0.5—a single draw separates the models—so its accuracy-decay exponent is undefined there (zero unsaturated points to fit) and only its T ≥1 estimates appear in Fig. 1; this is a ceiling on the estimator, not an absence of signal, and is itself consistent with entropy gating (the strongest probe saturates first).

Ibauc peaks near T ≈1, vanishes at T =0

0.4

c2,1 c10,1 c100,1 c2,16

pilot rate Ibauc

0.3 0.2 0.1

S2 0.0 0.0

0.5

1.0

1.5

sampling temperature T

2.0

Figure 1: Pilot rank-error rate Ibauc vs. sampling temperature T for the entropy-ladder probes: Ibauc → 0 under greedy decoding, peaks near T ≈1, and the onset temperature scales with output entropy (c2,1 only starts at T =2).

Temperature Gating

Proposition 1 (Distributional separation). For a single-draw probe at temperature T with categorical laws pTP , pTR on n symbols: 1. One draw carries at most the visible-output information I(Θ; Y ) ≤ H(Y ) ≤ log n, while the testing exponent satisfies I ⋆ ≤ min KL(pTP ∥pTR ), KL(pTR ∥pTP ) . 2. At the greedy boundary T → 0, repeated identical draws have zero additional-query accumulation exponent. Thus a small alphabet caps the mutual information in one draw, but weak evidence also requires the two categorical laws to be close. The c2,1 probe is capped at one bit per draw; it is not automatically useless. The greedy statement concerns accumulation from repetition: a single deterministic draw may still separate two endpoints, but repeated identical draws do not add independent evidence. Corollary 1 (Greedy has no accumulation). Under idealized greedy decoding (T=0), repeated single draws are deterministic and identical, so the accumulation exponent from repeating the same draw is I = 0. Figure 1 shows the per-query information density Ibauc as a function of sampling temperature, for the entropy-ladder probes of Section 6. Ibauc → 0 under greedy decoding (repeated single draws are identical), rises to a probe-dependent interior peak near T ≈1, and falls again as sampling noise grows; the temperature at which accumulation turns on scales with the probe’s output entropy (c2,1 only starts at T =2). This is the empirical basis for the entropy-/temperature-gating claim in the body; we do not assert unimodality of I(T ) in general (Prop. 1), and note that T =0 is not perfectly deterministic on every backend, so the greedy I=0 statement

Dilution Estimation

Theorem 1 flags that a suspect is diluted; here we detail the routing-fraction estimator ϵ̂ of Eq. (14). Under Qϵ the perquery tell probability at a threshold τ is pϵ = (1 − ϵ)α1 + ϵ q, with α1 = PP (s > τ ) and q = PR (s > τ ); inverting the observed tell fraction fˆ over m queries gives the method of-moments estimate ϵ̂ = clip[0,1] (fˆ − α1 )/(q − α1 ) of Eq. (14), identifiable whenever the diluent separates (q − α1 > 0). We fix τ at the (1−α0 ) reference quantile (α0 =0.1, so there is no max-gap snooping) and estimate α1 , q on an enrollment split disjoint from the audited responses; a deltamethod 95% interval propagates the binomial tell variance and the anchor (α1 , q) uncertainty. On controlled mixtures of the K=6 ladder (30 reference– diluent pairs passing the margin check, m=128, probe c2,16 , 6000 episodes/cell), ϵ̂ tracks the true fraction with mean |bias| ≤ 0.02 and 95%-CI coverage 0.92–0.96 across ϵ ∈ [0, 0.75], and the ϵ=0 control returns ϵ̂≈0.02 (Fig. 2)—IRIS reports how much a stream is diluted, not merely that it is. Low-margin pairs (q → α1 ) are excluded as non-identifiable.

Coverage on the gateway. We re-audit the interval on the full 17-endpoint gateway (content-only features, m=80, the 268/272 identifiable pairs with q>α1 —a looser screen than the deployed q−p0 ≥ 0.15 margin that retains 218 in App. S3—held-out mixtures at known ϵ). The point estimate stays essentially unbiased (mean ϵ̂ = 0.110/0.204/0.303/0.501 at ϵ=0.1/0.2/0.3/0.5; signed bias ≤ 0.010, per-pair |bias| ≈ 0.04), but the delta-method interval is mildly anti-conservative: empirical coverage is 0.86–0.87 at the nominal 90% level and 0.92–0.93 at 95%— the low end of the K=6 figure, under-covering by ≈ 3–4 points on the harder content-only gateway. Dropping the anchor-uncertainty terms (a Wilson interval on fˆ alone) worsens it to 0.79–0.86 at 90%, so the (α1 , q) variance is a non-negligible part of the honest width and must be retained. We therefore describe the interval as approximately calibrated in-distribution, but slightly optimistic rather than exact.

Dilution at Scale

We scale the online controlled-dilution audit (Section 6.2; dilutions we inject, not gateways found diluting) to the full 17endpoint OpenRouter library (10,200 real responses) and add

(a) dilution estimate (c10,8 , m=128) 0.8

0.8

CI coverage

0.6

estimate ˆ

(b) calibration of the CI 1.0

identity 10–90% over pairs ˆ (mean)

0.4

0.6 0.4

0.2 0.2 0.0

nominal 0.95 95% CI coverage

0.0 0.0

0.2

0.4

true routing fraction

0.6

0.0

0.2

0.4

true routing fraction

0.6

Figure 2: Routing-fraction estimation on controlled mixtures (K=6 ladder, 30 margin-qualified pairs, m=128, c2,16 ). Left: ϵ̂ vs. true ϵ (identity dashed; band is the 10–90% spread over pairs). Right: empirical 95%-CI coverage vs. the nominal 0.95. the unknown-diluent regime. Protocol. Posteriors P̂(· | y) come from a 5-fold out-of-fold content-only (length/formatinvariant) random forest over the three sequence probes, scored as s = − log P̂(P | y). For each ordered (claimed P , diluent R) pair we use three disjoint splits of P : a calibration split fixes τ at the (1−α) quantile (α=0.05); a fresh honest window re-measures the null tell rate p0 before the audit; and an audit split supplies the honest queries. The diluted stream draws each of m queries from R w.p. ϵ else P , and detection is a one-sided two-proportion test of the audit tell rate against p0 (declared at p<0.01); the realized FPR is the empirical ϵ=0 row of Table 1 (0.02), not the union-bound α/m used in the m⋆ derivation. This at-scale audit therefore runs the fixedp0 two-proportion test (local complexity Θ(ϵ−2 ) by Prop. 2), not the shrinking-threshold any-tell whose 1/ϵ budget is validated separately in App. S4—these are distinct tests, and we do not claim the at-scale numbers exhibit the 1/ϵ law. The calibrated estimate is ϵ̂=(fˆ − p0 )/(q − p0 ) with q the diluent’s tell rate (needs R enrolled); the deployed lower bound ϵ̂lb =(fˆ − p0 )/(1 − p0 ) uses only P . We report pairs passing the pre-audit margin check q − p0 ≥ 0.15 (a combinedprobe tell-rate margin; this differs from the per-probe tailplateau used in Table 23, which is why the count is 218/272 here versus its 248/164/176 for a single probe). The remaining 54/272 pairs are marked low-margin/indeterminate; over all 272 ordered pairs the ϵ=0.3 reliable-detection rate (f.95 ) is correspondingly ≈ 0.51 (0.64×218/272). These 54 are not random: they concentrate on a few claimed models (qwen3-32b, qwen3-8b, gpt-4.1, gpt-4o-mini, gemini-2.5-flash account for 48) and split into two kinds. First, close relatives that plausibly share training lineage—within the Qwen3 family (8b/32b/235b), within the GPT-4 series (gpt-4.1/gpt-4o-mini), within Gemini 2.5, and Gemini–Gemma (which share a tokenizer)— whose random-generation fingerprints are genuinely similar. Second, claimed models whose own single-response fingerprint is not distinctive, so a foreign diluent rarely crosses the tell threshold (q ≈ 0); this is directional (a property of the defended model) and is not size-specific, as it includes the flagship gpt-4.1 and the mainstream gemini-2.5-flash. Both are limits of the per-response tell margin at the deployed budget, not of identifiability: full-query attribution still sep-

arates all 17 models (verification AUROC 0.99 by m=8). Honest FPR. The pooled mean above hides a right tail: over the 218 margin-qualified pairs the per-pair honest (ϵ=0) FPR has median 0.000, mean 0.017, 95th percentile 0.090, and maximum 0.557, with 7.8% of pairs above 0.05. The tail is the 22 pairs whose fresh honest window draws zero tells (p̂0 =0), where the normal-approximation two-proportion test compares against an empirical null of 0 and is anticonservative. Replacing it with a one-sided exact binomial test of the audited tell count against a Clopper–Pearson (1−γ) upper bound on p0 (γ=0.1, with a khw =0 guard giving p̄0 ≈ 0.023 rather than 0) cuts the tail—max 0.39, 95th percentile 0.018, 2.8% above 0.05, pooled mean 0.0065—while preserving power (known-diluent ϵ=0.3: 0.85 → 0.78 under the exact test—a separate effect from the equal-valued leaveone-out power in Table 1). One residual heavy-tailed pair still has FPR 0.39: its honest-window split draws zero tells but the disjoint audit split has true honest rate 0.062, and the countonly Clopper–Pearson bound corrects for sampling error in the window count, not for this audit-side dispersion—so we report the whole distribution, not the mean. Margin validity. To rule out that choosing the reported subset and evaluating power on the same data inflates it, we re-apply the margin check on a third disjoint selection split and measure power and FPR on the disjoint test split. Membership is stable (223/272 vs. the 218 in-sample, 210 shared) and the held-out honest FPR is if anything stricter (0.0085 vs. 0.0167). The held-out ϵ=0.3 power is 0.66 versus the in-sample 0.85; but a matched control that applies the margin check in-sample on the same half-size audit bank also gives 0.66, so the gap is a data-budget artifact of halving the bank (less score diversity), not selection bias (≤ 0.002 power attributable to disjoint selection). We therefore report the fullbank numbers as the operating point, with this experiment establishing that selection circularity is not their source. Unknown diluent (leave-one-out). To model a substitute we have never enrolled, we delete R from the library and retrain on the remaining 16 endpoints; R’s responses are scored out-of-distribution by the fold classifiers. Detection uses only P , so it is unchanged in principle; Table 1 confirms it degrades only slightly in practice. The foreign responses are also flagged out-of-library by open-set rejection (knownness maxc P̂(c | y) below the claimed model’s 95%-accept gate) at mean rate 0.19 versus a 0.04 self-rate. The deployed ϵ̂lb stays below the true ϵ in aggregate (Table 1, last column), as it must when q ≤ 1.

Query Complexity

Validating the 1/ϵ law under a controlled error rate, and deciding whether the measured tail exponent κ is a population property or a finite-sample plateau, both turn on reference calibration size. We address them on fresh, large reference batches (qwen3:8b@T 1 as reference; 0.6b/4b/14b@T 1 and the temperature twin 8b@T 0.9 as substitutes; ≈3000 responses each on c2,16 /c10,8 ). Protocol. The reference is split three ways—a calibration split fixes τ at the per-query level α/m, a disjoint certification split

Table 1: Dilution audit over all margin-qualified (claimed, diluent) pairs of the 17-endpoint gateway, m=80, content-only features, 400 episodes/cell. Known: diluent enrolled (218 pairs). Unknown: leave-one-out, diluent deleted from the library (198 pairs). “power” is the fraction of audits flagged at p<0.01; “f.95 ” the fraction of pairs detected at power≥0.95; ϵ̂ the calibrated estimate, ϵ̂lb the claimed-only lower bound. known diluent ϵ 0.00 0.05 0.10 0.20 0.30 0.50

power f.95 0.02 0.11 0.30 0.67 0.85 0.96

med. p

0.00 5×10 0.00 2×10−1 0.01 5×10−2 0.31 3×10−3 0.64 4×10−5 0.87 2×10−9 −1

unknown (leave-one-out) ϵ̂

ϵ̂lb

0.03 0.02 0.06 0.04 0.10 0.07 0.20 0.14 0.30 0.22 0.50 0.36

power f.95

ϵ̂lb

0.03 0.14 0.32 0.61 0.78 0.91

0.02 0.04 0.07 0.14 0.20 0.34

0.00 0.01 0.03 0.31 0.53 0.80

claimed

diluent

result

ϵ̂

offline, real gateway gpt-4o-mini gemma-3-4b gemini-2.5-pro claude-3.5-haiku

llama-3.1-8b gemma-3-27b qwen3-235b gpt-4o-mini

0.95 1.00 1.00 0.00

2×10−4 4×10−10 9×10−10 6×10−1

0.33 0.33 0.31 —

live, end-to-end (m=80) gpt-4o-mini gpt-4o-mini llama-3.3-70b gpt-4.1 gpt-4o-mini

honest (ϵ=0) llama-3.1-8b gemma-3-4b qwen3-8b qwen-2.5-72b†

clean diluted diluted diluted diluted

0.31 3×10−3 3×10−5 2×10−4 <10−3

— 0.26 0.30 0.43 0.24‡

Live ϵ̂ is a single-stream point estimate, noisier than the offline calibration. † never enrolled (unknown diluent), a moderate twin: borderline at m=80 (p ≈ 0.003–0.01 across draws), robustly flagged at m≥160 (shown: m=240, p=10−3 ). ‡ lower bound ϵ̂lb .

Table 2: Representative pairs at ϵ=0.3, m=80 (top), and five online end-to-end audits of controlled mixtures we inject through OpenRouter (bottom; author-constructed, not gateways found diluting; honest window measured first). The unknown substitute (qwen-2.5-72b) is not enrolled, so only its lower bound is available. bounds α1 by a one-sided Clopper–Pearson upper bound, and a held-out audit split measures the realized FPR—so the null is never tested on the data that chose it (the data-dependentthreshold objection). We report queries-to-detect at power 0.95 under a pre-fixed total FPR (1% and 5%), not AUROC, fit its ϵ-exponent, and re-estimate κ as the calibration grows from n=50 to n=3000. (1) The 1/ϵ law holds at a controlled FPR, not via AUROC. Queries-to-detect at power 0.95 under a pre-fixed total FPR follow a log–log slope of −1.0 to −1.12 for the distinct substitutes (0.6b, 4b) on both probes and at both α=0.05 and 0.01—the 1/ϵ law, now against a fixed error rate rather than AUROC. The temperature twin (8b@T 0.9) is undetectable at power 0.95 within 256 queries at every ϵ (the wall), so the dichotomy survives a controlled-FPR test. (2) But certifying the FPR needs thousands of reference samples. Realizing the union-bound level α1 ≤ α/m requires the reference’s deep tail; a 1000-sample calibration overshoots (realized total FPR 0.09–0.14 on c2,16 , though ≤ 0.04 on the better-behaved c10,8 ). The certifiable budget is m ≤ α/q 0 (N ) with q 0 (N ) the one-sided Clopper–Pearson floor at N samples assuming zero tail events (a best-case envelope): m ≤ 16 at N =1000, m ≤ 50 at N =3000,

m ≤ 166 at N =10,000 (α=0.05; an order of magnitude smaller at α=0.01). These are the zero-event envelope; with the tail events a real reference exhibits, the achievable certified m is smaller still for atom-heavy probes like c2,16 , whose realized FPR already exceeds nominal at the N =1000 calibration above. The 1/ϵ regime is therefore deployable with a certified FPR only down to the ϵ whose budget N supports— ϵ ≳ 0.1 at a few thousand samples, smaller ϵ needing tens of thousands. This is the calibration cost the 1/ϵ budget demands, stated explicitly rather than hidden behind AUROC; all the FPR/power figures here resample one finite held-out reference bank into episodes, so the intervals are conditional on that bank (as in App. S3). (3) The tail exponent κ is stable in the calibration size. Re-estimating κ as the reference grows from n=50 to 3000, the distinct substitutes settle at a small, stable value (4b at κ ≈ 0, 0.6b at 0.13) without moving toward 1, while the adjacent-temperature stress test stays near κ ≈ 1 at every n. We therefore present 1/ϵ as a measured, finite-ϵ, calibrationbounded regime when the margin check passes—never an asymptotic guarantee. (4) Continuous one-class scores. To rule out that κ ≈ 0 is a random-forest discretization artifact, we re-measure κ

(a) deployed audit after margin check

(b) fraction estimate on the gateway

power 0.95

0.5

0.8

identity calibrated ˆ (known diluent) lower bound ˆlb (unknown)

0.4

estimate ˆ

detection power @ m=80

1.0

0.6 0.4

0.3 0.2 0.1

0.2

known diluent unknown diluent (leave-one-out)

0.0 0.0

0.1

0.2

0.3

routing fraction

0.4

0.5

0.0 0.0

0.1

0.2

0.3

true routing fraction

0.4

0.5

Figure 3: Deployed dilution audit on the real 17-endpoint gateway. (a) Detection power vs. routing fraction ϵ at m=80 under the fixed-p0 two-proportion test, pooled over pairs passing the margin check (band = 25–75% over pairs): the unknown-diluent leave-one-out curve nearly coincides with the known curve—detection needs only the claimed model’s fingerprint. (b) Routing-fraction recovery: the calibrated ϵ̂ tracks the identity, while the deployed lower bound ϵ̂lb (valid for an unknown diluent) sits conservatively below it. Table 3: Which statistic backs which claim, and at what guarantee level. IRIS couples a ranking/difficulty estimator to a type-I-valid deployed test; the prescribed budget is a feasibility diagnostic, not a coverage guarantee. (Referenced from the body.) claim / use

statistic

attribution, full- AUROC-slope Ibauc sub. verification (rank) dilution detection fixed-p0 two(deployed) proportion 1/ϵ at pre-fixed any-tell, CPFPR certified α1 routing fraction ϵ̂ moments + delta interval end-to-end estimate-then(α, δ, ϵ) budget

guarantee difficulty score; no coverage certificate type-I valid given null; Θ(ϵ−2 ) local needs ∼thousands refs to certify approx. calibrated indist. (optimistic) type-I held; power met on a minority (diagnostic)

with two continuous, reference-only scores—a standardized Mahalanobis density and an isolation-forest anomaly. The adjacent-temperature stress test remains low-separation (κ ≈ 0.9–1.1), while distinct substitutes keep a much smaller exponent under the Mahalanobis density and interpolate under isolation forests. We accordingly report 1/ϵ as the behaviour of IRIS’s deployed score after the margin check, not as a score-independent law.

End-to-End Bound

Proposition 2 (Binomial guarantee). Fix targets α, δ ∈ (0, 1), a minimum routing fraction ϵ, and a per-response threshold τ with population rates α1 = PP (s>τ ) and q = PR (s>τ ), q > α1 . The level-α one-sided binomial test of the tell count against null rate α1 controls type-I error for every m. If m⋆ is the smallest stable budget whose binomial power against pϵ = (1 − ϵ)α1 + ϵq is at least 1 − δ for every m ≥ m⋆ , then PQϵ′ (flag) ≥ 1 − δ for all ϵ′ ≥ ϵ and m ≥ m⋆ .

Proposition 2 is a population power calculation. It reduces to the any-tell m⋆ = Θ(ϵ−1 log(1/δ)) regime only when a separating threshold gives q = Ω(1) and α1 = o(ϵ); at a fixed null rate α1 = p0 > 0, the local budget is Θ p0 (1 − p0 )/(ϵ2 (q − p0 )2 ) . With estimated rates, the threshold must be fixed on a split separate from the one estimating the null, and Clopper–Pearson upper/lower bounds on (α1 , q) preserve the statement up to the chosen confidence. Certifying α1 ≤ α/m is reference-sample intensive, which is why the empirical check below distinguishes the population guarantee from finite-calibration performance. Empirical check. Proposition 2 is type-I valid for the population α1 (by the binomial test’s validity, not a union bound); m⋆ sized for binomial power ≥ 1 − δ at pϵ controls type-II. The finite-sample question is whether the estimated (τ, α1 , q) deliver this. We check it on the 17-endpoint gateway (c10,8 , content-only), with train/calibration/audit splits disjoint at the raw-response level so no episode is reused, at (α=0.05, δ=0.1, ϵ=0.3), calibrating against Clopper– Pearson bounds (γ=0.1). Across the 258/272 feasible pairs the realized type-I is controlled in aggregate—mean 0.013, within α on 90% of pairs—confirming the test-validity argument, at mean budget m⋆ =112; realized power averages 0.73 but meets the pre-registered 0.9 target (δ=0.1) on only 17.8% of feasible pairs (12.6% at ϵ=0.2)—population-exact yet finite-sample under-powered on the median pair at this m⋆ . The per-pair type-I carries a tail (mean 0.013 but maximum 0.475, with 10% of pairs above α), induced by the winner’s-curse m⋆ -minimization over the calibration grid a0 ∈ {0.02, 0.05, 0.1, 0.15} at the full confidence γ; a Bonferroni split (each Clopper–Pearson bound at γ/4 before minimizing) restores it—maximum 0.032 at ϵ=0.2 (∼0% exceedance) and 0.168 at ϵ=0.3—at a larger budget (mean m⋆ 151 → 223 at ϵ=0.2), whereas fixing a single a0 a priori does not control it (maximum 0.76–0.93, ∼18% exceedance), confirming the tail is finite-calibration variance the union bound addresses, not grid optimism alone. The shortfall is thus a finite-sample effect: each pair calibrates (τ, α1 , q) on only ∼33 responses, so the 1−2γ confidence is loose and clumped scores transfer imperfectly between the calibration and audit halves. The guarantee is exact at the population level and as tight as the calibration sample allows, so a deployment should size its honest window accordingly. The infeasible 14 pairs are returned indeterminate rather than flagged.

Multiple Diluents

A gateway may dilute one reference a mixture of P with PJ PJ J substitutes, Q = 1 − j=1 ϵj P + j=1 ϵj Rj , with P ϵ = (ϵ1 , . . . , ϵJ ) and ϵ0 = 1 − j ϵj ≥ 0. Both halves of IRIS extend without new machinery. Proposition 3 (Multi-diluent audit). Under Q above with i.i.d. routing, at any threshold τ the tell probability is X p(τ ) = ϵ0 α1 (τ ) + ϵj qj (τ ) ≥ ϵtot qmin (τ ), j

where ϵtot = j ϵj and qmin = minj qj . If the same threshold controls the honest tail, any-tell detection scales

Detection. At P a per-response level τ the tell probability is p = ϵ0 α1 + j ϵj qj with qj = PRj (s > τ ), so p ≥ P ( j ϵj ) minj qj once every diluent separates (qj > α1 ). The any-tell budget of Theorem P 1(a) therefore holds in the total foreign fraction ϵtot = j ϵj : the 1/ϵ law detects any foreign traffic at a cost set by ϵtot and the weakest separating diluent, and—as in the single-diluent case—needs no enrollment of the Rj . Nonnegative unmixing. Let the classifier map a response to its posterior vector u(y) = P̂(· | y) ∈ ∆K over the K enrolled endpoints, and let the signature of endpoint M be ḡM = Ey∼M [u(y)]P(estimated on enrollment). Under Q, EQ [u] = ϵ0 ḡP + j ϵj ḡRj , so from the observed mean posterior v̄ over m audit queries the simplex-constrained least squares is η̂ = arg min v̄ − (1 − 1⊤ w)ḡP w≥0, 1⊤ w≤1

wj ḡRj

2 2

The optimizer η̂ is estimated from audit responses, not supplied as a ground-truth quantity. Under this multi-diluent model and the full-rank condition below, it estimates the true fraction vector ϵ; in the single-diluent main audit, IRIS uses it only to rank candidate identities and reports the tell-based ϵ̂. This is the standard quantification-learning / mixture-proportion problem. The fractions are identifiable iff the shift vectors {ḡRj − ḡP }Jj=1 are linearly independent (the matrix G = [ḡR1 − ḡP , . . . , ḡRJ − ḡP ] has full column rank J)—the multi-diluent analogue of the single-diluent condition q > α1 (at J=1, ḡR1 ̸= ḡP ). Because posterior signatures live in the K-simplex, every column sums to zero and J ≤ K − 1 is necessary. Near-twin diluents collapse columns of G and are non-identifiable, exactly mirroring the ϵ−2 wall; a delta-method or bootstrap over the signatures propagates the interval. Empirical validation: on the 17-endpoint gateway (c10,8 , content-only, m=200), simultaneously diluting a reference with J=2 (ϵ=.2, .2) or J=3 (ϵ=.15 each) full-rank diluents and solving the simplexconstrained least squares above recovers every per-diluent fraction to a median worst-case error of 0.03–0.04 (90th percentile 0.07; 98% of configurations within 0.1), with all 120 sampled triples identifiable (none rank-deficient). The single-machinery claim thus holds: detection and per-diluent quantification extend to several simultaneous diluents at the same accuracy as the single-diluent case (App. S2).

Backend Separability

Extending Section 6: the same nominal model served by local Ollama vs. a commercial gateway is distinguishable from visible strings at T =1, on content-only (length/formatinvariant) features, so the signal is a sampling-distribution

(a) attribution, 17 commercial & open APIs

(b) reference-vs-suspect verification

1.0

0.8

0.9

verification AUROC

attribution accuracy

with the total foreign fraction as ⌈ln(1/δ)/(ϵtot qmin )⌉. For quantification, if u(y) ∈ ∆K , ḡM = EM [u(y)], and G = [ḡR1 − ḡP , . . . , ḡRJ − ḡP ], then EQ [u] = ḡP + Gϵ, and the fractions are identifiable iff G has full column rank.

0.6

0.4

0.2

0.8

0.7

0.6 chance 1/17

0.0 2

independent queries m

c2,1

c100,1

0.5

c2,16

independent queries m

c10,8

Figure 4: External validation on 17 commercial and open APIs (chance 1/17, dotted). (a) attribution accuracy and (b) reference-vs-suspect verification AUROC vs. independent queries m; shaded bands are 95% bootstrap intervals over 40 splits. The e−Im law and entropy gating replicate off the local testbed onto frontier closed APIs (Section 6). difference rather than a serving-format artifact. Table 4 gives per-probe verification AUROC (local vs. gateway) and the raw output diversity (distinct responses over 120 local / 100 gateway draws), which exposes the mechanism: local quantized, top-k-truncated decoding is markedly more modecollapsed. Table 4: Backend detectability of one nominal model (local Ollama vs. gateway), T =1. AUROC is per-response (m=1) on content-only features; all probes reach ≈ 1.0 by m=8. Probe labels abbreviate c10,1 , c100,1 , c2,16 , and c10,8 ; diversity is distinct/total responses. model

probe

Qwen3-8B digit num100 bits16 digits8 Qwen3-32B num100 bits16 digits8

AUROCm=1

div. (loc/gw)

.76 .86 .91 .95 .93 .90 .97

1/120 vs 11/100 3/120 vs 24/100 42/120 vs 88/100 34/120 vs 97/100 16/120 vs 17/100 62/120 vs 40/100 82/120 vs 29/100

Queries, Not Length

The body claims that evidence accumulates in the number of independent queries m, not within-response length L (Prop. 2). Table 5 is the direct empirical support: singleresponse attribution accuracy (K=3 Qwen3 sizes, chance 1/3) as a function of the requested L, under two prompt styles (chat-templated and direct). Accuracy is non-monotone in L—it wanders up and down (e.g. chat: .94 → 1.00 → .97 → .83 → .89 over L=16–256)—and a linear fit of accuracy on L has leave-one-condition-out cross-validated R2 < 0 under both styles (−1.59 chat, −4.73 direct)—worse than predicting the mean (an in-sample OLS-with-intercept R2 is non-negative by construction; the negative value is the heldout score, our point being predictive uselessness of L, not in-sample fit). These conditions vary the requested length, so they are different contexts rather than truncations of one fixed response (Prop. 2(i)); the non-monotonicity is consistent with but not predicted by the same-context monotonicity (full argument in the proof of Prop. 2, App. S28). Length still

(a)

(b)

per-position tell

(a) attribution across 45 gateway endpoints

global-statistic tell

(b) reference-vs-suspect verification

1.0

0.8

0.9

L? = Lmin interior L

0.6

tell prob. q(L) yield q(L)/L (scaled)

0.4

verification AUROC

0.8

attribution accuracy

probability / yield

1.0

0.6

0.4

0.2

0.8

0.7

0.6

0.2

chance 1/45

0.0

0.0 5

response length L

Figure 5: Token-optimal length maximizes the per-token tell yield q(L)/L, not L (Cor. 2); the yield is rescaled to its own maximum so the panels share an axis. (a) A per-position tell makes q(L) concave with q(0)=0, so q(L)/L is decreasing and the optimum is the shortest admissible length (L⋆ =Lmin ): many short independent queries dominate. (b) A global-statistic tell (compression, n-gram entropy, χ2 ) is information-starved at small L, so q(L) is S-shaped and the yield peaks at an interior L⋆ . Which regime holds is empirical (Table 5); either way the accumulation axis is the query count m, not L. carries evidence—a long response is not worthless—but it does not accumulate linearly and is an unreliable budget axis, whereas the error exponent tensorizes exactly across independent queries m (the e−Im law, Fig. 2a). This is why IRIS spends its budget on more short independent queries rather than longer responses. The same accounting fixes the token-optimal length: turning a length-L query into a per-token yield question makes many short independent queries beat one long response whenever they give larger tell probability per token. Corollary 2 (Token yield). Fix a token budget B = mL as m independent queries of length L, where a length-L query has tell probability q(L). The miss probability is at most e−ϵq(L) B/L , so this bound is minimized by maximizing q(L)/L. The exact geometric miss gives the same optimizer to first order when the false-positive probability is controlled. Figure 5 contrasts the two yield shapes that fix where L⋆ lands, and the shape follows from how the tell is computed rather than from any extra assumption. A tell that can fire at each position gives a concave q(L) whose chord slope q(L)/L through the origin only falls, so the budget is best spent on the shortest admissible queries; a tell that reads a whole-response statistic needs a few symbols before it engages, giving an S-shaped q(L) and an interior token-optimal length. In both regimes the linear-in-m accumulation of the e−Im law (Prop. 1) is the reliable lever, while L is not.

45-Model Scaling

independent queries m

0.5

independent queries m

We re-run the full pipeline on a pooled library of 45 commercial/open models accessed via OpenRouter (27,000 responses). Attribution (chance 1/45=2.2%) reaches accuracy 0.85 at m=8 and 0.90 at m=16 on c10,8 , with reference-vssuspect verification AUROC 0.98 (Fig. 6); per-model recall is broad (median 0.91, and 0.82 even excluding the five easiest models), so it is not carried by a few trivial outliers.

c2,1

c100,1

c10,1

c10,8

c2,8

c2,16

Figure 6: Scaling to 45 models via OpenRouter (chance 1/45=2.2%, dotted). (a) attribution accuracy and (b) reference-vs-suspect verification AUROC vs. independent queries m, per probe; shaded bands are 95% bootstrap intervals over 10 splits. The e−Im law and entropy gating hold at K=45: c10,8 reaches accuracy 0.85 at m=8 (0.96 at m=48) and verification AUROC 0.98, while c2,1 stays weak. Because ∼13 models emit a verbose preamble whose digits the character parser retains, length-ablation alone is not a complete control here; the decisive de-confounding is a clean-format control restricting to the K=32 models with indistinguishable output length (7–12 chars), which still gives accuracy 0.83 at m=8 (chance 1/32=3.1%) and AUROC 0.97—confirming the signal is the per-token sampling distribution, not response length. Dilution across six frontier models follows an approximate inverse-power wall; on contentonly features the log–log slope is ≈−1.4 and the median length-matched pair is detected at ϵ=0.1 in ≈64 queries (the smaller 32-query figure on the full feature set is inflated by one length-distinguishable model). Detectability is strongly pair-dependent: several frontier-vs-frontier dilutions remain undetectable at ϵ=5–10% even at 256 queries. Open-set rejection (random 20% of models held out as unknown, knownness = maxc P̂(c | y)) degrades modestly with pool size on a like-for-like protocol (same data, same splits, content-only features): c10,8 open-set AUROC@m=8 falls 0.73 → 0.70 from K=17 to K=45 (reject-at-95%-accept 0.26 → 0.11), as an unknown grows likelier to overlap some model in a denser enrolled set.

S10

Base Specificity

The economic threat is covert substitution of the base model (Section 3); the audit must therefore be specific—flagging a model substitution but not an honest same-model sampler/effort retune. Table 7 runs the dilution audit on a substitution (ϵ=1, m=80, content-only) for each suspect type. An adjacent-temperature retune of the same model yields ϵ̂=0.04 (flagged 8%, barely above the false-positive floor), while a base-model substitution yields ϵ̂=0.53—a 13× separation; only the extreme, observable T =0 greedy collapse registers among samplers. Reasoning effort leaves a residual (ϵ̂=0.14) but is economically transparent and directly observable from returned token counts (and removable by length-matching), so it is not a covert dilution vector. We next confirm which knobs leave any visible-string signature, varying one knob on a fixed model on content-only (length-invariant) features (Fig. 7); in every case content-

Table 5: Single-response attribution accuracy vs. response length L (K=3, chance 1/3), two prompt styles. Accuracy is nonmonotone in L and a linear fit has R2 < 0 (worse than the mean)—length is not an accumulation axis, unlike independent queries m (Fig. 2). “—” = length not in that style’s grid. L

chat

direct

16 24 32 48 64 96 128 192 256

.94 — 1.00 — .97 — .83 — .89

.90 .92 .83 .93 .83 .89 .95 .88 —

R2 (acc vs. L)

−1.59

−4.73

Table 6: Length/format ablation at K=45 (179 → 144 features): attribution accuracy and verification AUROC at m=8, full vs. length-ablated. The high-entropy sequence probes are near-invariant (≤ 0.03 drop), so their signal is the per-token sampling distribution, not response length; the lower-entropy c100,1 relies more on length. accuracy @ m=8 AUROC @ m=8 probe

full

abl.

full

abl.

c10,8 c2,16 c100,1 c10,1

.855 .725 .611 .446

.844 .691 .502 .356

.976 .967 .893 .860

.973 .955 .871 .831

only AUROC tracks full-feature AUROC, so none of these effects is a length artifact. Quantization—a cost-relevant precision downgrade, the one borderline same-family cheat—is probed on the real gateway by pinning OpenRouter providers that serve one model at different declared precision. Provider × precision. Two production endpoints serving the nominally identical model at different declared precision on different providers are distinguishable from visible strings (content-only AUROC@m=8: qwen3-235b bf16/fp8 1.00, llama-3.3-70b 0.85, deepseek-v3.2 fp4/fp8 0.67). Because provider and precision are perfectly confounded in this design (no same-provider precision contrast), this evidences endpoint-configuration identifiability, not weight precision in isolation. The strong end is partly an output-diversity split (the qwen3-235b fp8 endpoint collapses to 1–2 distinct strings per 100, the bf16 endpoint stays diverse), and the deepseek fp4-vs-fp8 pair—both fully diverse—sits near chance, so identifiability is modeldependent and degenerates when both endpoints sample broadly. Temperature and effort. Sampler choice perturbs the audit only at the extremes; a temperature retune over the normal operating range is essentially invisible. The one strong sampler tell is low-temperature collapse: T =0 vs. any T >0 separates at AUROC ≈0.99, because greedy decoding re-

Table 7: Specificity of the dilution audit: substitution (ϵ=1, m=80, content-only) by suspect type. ϵ̂ is the reported routing fraction (deployed lower bound), “flag” the p<0.01 detection rate. A specific audit flags a base-model substitution, not an honest same-model retune. The cross-model row is the 3-model temperature pool (moderate margin); the 17endpoint matrix (App. S3) gives the full base-model picture. suspect (whole stream)

ϵ̂

same config (false-positive floor) 0.01 same model, adjacent-T retune 0.04 same model, far T (→ 1.5/2.0) 0.11 same model, T =0 greedy collapse 0.30 same model, reasoning effort (min↔high) 0.14 different base model (substitution) 0.53

flag rate 0.00 0.08 0.30 0.60 0.61 0.78

peats a single string (llama-4 at T =0 emits one distinct 16-bit string over 80 draws). Between adjacent operating temperatures the signal nearly vanishes: AUROC falls to ≈0.58 and T1.5-vs-T2.0 is at chance, because the acrosssample diversity that carries the signal has already saturated by T ≈1. These two regimes are exactly the second-order analysis of App. S25. In the interior, adjacent temperatures separate only at O((∆β)2 )—the interior Fisher quadratic, which gives the observed ≈ 0.58—while T =0 vs. T >0 is a separate support-collapse tell. So a temperature retune is, correctly, near-invisible to a model-dilution audit, while a backend substitution is not. Reasoning effort (minimal vs. high) likewise leaves a fingerprint: pooled-m=8 content-only AUROC is 0.92, though a single response gives only 0.72–0.76. The mechanism is mode/prefix concentration, not a flatter distribution. Minimal effort anchors on preferred openings—on the digit probe, gpt-oss-120b’s prefix 274 appears 41% vs. 8% of the time, and first-digit entropy drops 2.76 → 1.88. The overall character entropy is unchanged, so we do not claim an entropy law: only the mode mass moves. The take-away is that the audit’s signal is dominated by the base model: the sampler contributes only at the extremes (greedy collapse), and reasoning effort—though it leaves a residual mode-concentration signature—is observ-

distributional shift

collapse / mode concentration

0.90

Backend (local vs. gateway)

0.92

Reasoning effort (min vs. high)

0.99

Temperature: T =0 vs. T >0 Temperature: adjacent T >0

0.58

Provider × precision

0.67–1.00 (model-dep.) deepseek

llama

qwen

chance

0.5

0.6

0.7

0.8

0.9

content-only verification AUROC @ m=8

probe features

1.0

Figure 7: Which serving knobs leave a visible-string signature: content-only (length-invariant) verification AUROC at m=8 for same-model variants, colored by mechanism. The audit is specific to the base model—adjacent operating temperatures fall to chance (an honest retune is not flagged), provider/precision is model-dependent, and the only strong sampler signal is the extreme T =0 collapse. able from token counts and billed transparently, so the alarm is correctly reserved for a change of the underlying model (Table 7).

S10.1

Table 8: Auditing a precision cheat (q4 vs. fp16, ref qwen3:0.6b). Per-query tell rate q (mean [95% CI]) and budget m⋆ at ϵ=0.1; “detect” marks q clearing the α≈0.05 false-positive floor. The last column is a fp16-vs-fp16 negative control. Detectable only on the high-entropy probe, and the tell survives length ablation; the low-entropy signal is a length artifact that ablation removes.

Precision Retunes

Quantization is the cost-relevant same-weights cheat: serving a 4-bit model where the reference is 16-bit. We audit it directly under controlled mixtures (reference qwen3:0.6b-fp16, substitute the default q4), measuring the per-query tell rate q and the resulting budget m⋆ = ln(1/δ)/(ϵq) against the false-positive floor α≈0.05 (Table 8). Three findings, each guarded by a control. First, the cheat is detectable, but only on a high-entropy probe: on c2,16 , q=0.42 clears the floor (m⋆ ≈72 at ϵ=0.1), whereas on low-entropy c100,1 it does not (q=0.06). Second, the signal is genuine per-token randomness, not response length: under a length/format ablation (179→144 features) the c2,16 tell barely moves (q:0.42→0.37, m⋆ ≈82), while the apparent c100,1 signal collapses to q=0—it was a length artifact. Third, a fp16-vs-fp16 negative control (two independent fp16 runs) stays below the floor on both probes (q ≤ 0.03), so the audit does not manufacture a tell between identical configurations. We therefore do not claim a clean entropy law for quantization: detectability is probe- and model-dependent, but on the right probe a precision downgrade is a real, length-robust, control-validated tell. Across model sizes. The 0.6B audit above leaves open whether the precision cheat is detectable at scale. We collected a fresh q4-vs-fp16 pair for qwen3 at 0.6B, 1.7B and 4B (100 repeats each) and find detectability grows with size, on length-ablated (content-only) features: on the highentropy c2,16 the per-query tell rate is q=0.35 at 0.6B, 0.74 at 1.7B (AUROC 0.95), and 1.00 at 4B (AUROC 1.00), so the budget m⋆ = ln(1/δ)/(ϵq) at ϵ=0.1 falls from ≈86 to ≈30. At 4B the cheat is caught on every probe; at smaller sizes it is probe-dependent (the low-entropy c100,1 signal is a length artifact that ablation removes, q→0). Larger models have sharper, more idiosyncratic sampling laws that quanti-

c2,16 c2,16 c100,1 c100,1

full (52) len-abl. (28) full (165) len-abl. (132)

q (q4 vs. fp16) det. m⋆ϵ=.1 .42 [.27, .52] .37 [.17, .51] .06 [.02, .11] .00 [.00, .00]

yes yes no no

72 82 — —

fp16-vs-fp16 negative control (length-ablated): c2,16 abl. (28) .03 [.01, .07] no — c100,1 abl. (132) .00 [.00, .01] no —

zation perturbs more visibly, so the same-weights precision downgrade—the realistic covert cheat—is auditable and increasingly so with scale, on the high-entropy sequence probe IRIS already leads with.

S11

Feature Sets

IRIS uses two feature groups (Section 3): the full 179-feature set and the length/format-invariant content-only subset. Table 9 states which set produces each headline result. The rule is principled: any claim that a substitution occurred— one model/endpoint/serving-config standing in for another— is made on content-only features, so it cannot be an artifact of length or formatting; the information-rate, budgeting, and probe-ranking results use the full set and are separately shown robust to a length/format ablation (179→144).

S12

Paraphrase Robustness

The probe family is conspicuous, so an adaptive router could keyword-detect “generate random bits” and serve it honestly while diluting other traffic. We test how far paraphrasing defeats this by collecting responses to four stealth paraphrases of c2,16 that preserve the 16-i.i.d.-bit semantics but reword the surface (A/B-bucket assignment, train/validation tagging, a protocol nonce, fair coin tosses; 80 repeats × the K=6 ladder). Three findings. (i) The probe is detectable: a TF–IDF + logistic classifier trained on obvious randomgeneration prompts flags 13/13 of them and still flags 3/4 stealth paraphrases (only the nonce framing slips through), so paraphrasing only partially evades a keyword router. (ii) The fingerprint is in the sampling distribution, not the wording: models stay near-perfectly identifiable under every paraphrase (attribution accuracy 0.99–1.00 at m=8, chance 1/6). (iii) It partially transfers across phrasings: a classifier trained on the plain probe attributes the reworded responses at 0.50–0.75 accuracy at m=8 (chance 0.17), so each framing shifts the served distribution somewhat and is best enrolled separately. Together, an auditor can rotate among many semantically-equivalent framings—each accumulating evidence—to raise a static router’s cost, but cannot fully hide the request from a determined classifier; a fully

Table 9: Feature set per headline result. “Content-only” = length/format-invariant subset; “full” = all 179 features. Every substitution/endpoint/config claim is content-only. Result

Feature set

full; replicated content-only (144); R2 <0 in L I entropy/temperature gating full Budget-from-Ibauc (Tab. 16, 17) full (calibration AUROC) FPR calibration (Tab. 15) full Probe choice by Ibauc full Dilution 1/ϵ (Fig. 14) full; 45-pool slope contentonly Baselines (Tab. 10) full (clean attribution) External 17 APIs (Fig. 4) full; length-ablated check Endpoint-vs-model / backend content-only (Tab. 4) Dilution audit at scale + live content-only (detection, (§6.2, App. S3) tell-rate, ϵ̂) 45-pool open-set (App. S9) content-only; attribution de-confounded by lengthmatched K=32 (.83@8) Serving-knob identifiability content-only (Fig. 7)

−Im

law, m-not-L (§6)

adaptive adversary that recognizes the randomness semantics (not the wording) remains the key open threat. Output-side special-casing. The keyword router above acts on the prompt; a gateway could instead special-case the probe by its output shape (“the response is a bare 0/1 string”). We test both evasion surfaces on 5 distinct commercial endpoints (gpt-5.4-mini, claude-haiku-4.5, gemini-3.5-flash, llama-4-maverick, qwen3-max; 50 repeats each), comparing the bare probe, three keyword-free disguises (unit-test, A/B-bucket, protocol-nonce framings), and an embedded-output disguise that asks for the 16 bits inside a templated device-log line. Three findings. (i) The fingerprint survives every disguise on commercial models: within-disguise K=5 attribution is 0.87–0.94 at m=1 and 1.00 at m=8 (chance 0.20) for all variants, including the embedded one. (ii) No single static filter catches everything: an output-shape classifier trained to flag bare bit-strings (vs. genuine traffic) flags the bare probe and the clean disguises (0.78–1.00) but is completely defeated by the embedded-output disguise (flag rate 0.00, identical to honest traffic), which nonetheless still carries the fingerprint (m=8 attribution 1.00); symmetrically, the prompt keyword router misses the nonce framing. So the two cheap filters are each defeated by a different disguise while detection is unaffected—to special-case the probe a gateway must run both a semantic prompt parser and an output parser, and even then an auditor composing an unseen framing with an embedded output slips past. (iii) Honest limits: cross-phrasing transfer (train on the bare probe, attribute a disguise) is only partial (0.40–0.74 at m=8), so the auditor should enroll under the deployed phrasing rather than rely on transfer; and some framings depress a model’s output diversity (the unit-test framing collapses distinct-string rate

to 0.16 for some endpoints, versus 0.37 bare), so a disguise’s entropy must be validated before use. This corroborates on real gateways that static probe-detection—prompt- or output-side—is an arms race the auditor can keep winning by rotation, while a fully adaptive semantic router stays the key open threat. Task-embedded probe prefix. The experiments above show that IRIS remains effective under several disguises, but after publication we cannot exclude a gateway that recognizes standalone audit prompts and deliberately routes only those requests to the advertised model. A practical hardening is to embed the probe into the real task request: prepend or append an instruction requiring the response to begin with a short random string (for example, several digits or bits) before answering the substantive prompt. The first few visible tokens are then concentrated on a finite known alphabet, so the same visible-string features and thresholds can be calibrated on that prefix while the downstream content remains the user’s task. The auditor sends the identical task-embedded template to a trusted endpoint and to the suspect endpoint, using the trusted responses as the matched control for the prefix distribution. This does not rule out a fully semantic adaptive router, but it turns the failure mode “route obvious probes honestly” into an auditable deployment variant: a gateway must now both recognize the embedded probe inside ordinary traffic and make the task-conditioned prefix match the trusted model.

S13

Sticky Routing

The dilution theory and the deployed binomial test assume each request is routed independently; real gateways may route stickily (sessions, bursts, time-of-day, batches). We simulate sticky routing on a gateway pair passing the margin check (gpt-4o-mini vs. llama-3.1-8b, m=80, ϵ=0.3) with autocorrelation ρ at a fixed marginal fraction, giving effective block length L≈1/(1−ρ). Three findings, at L=1/4/16: (i) type-I is robust by construction—under an honest stream the tells are i.i.d. Bernoulli(α1 ) regardless of ρ (no substituted block exists), so sticky routing cannot inflate the FPR; this is definitional, not a simulation discovery (the simulated 0.033/0.050/0.050 are the i.i.d. null). (ii) power degrades with burstiness (1.00 → 0.97 → 0.77) as the effective independent count m/L falls. (iii) the i.i.d. interval breaks: the binomial ϵ̂ CI coverage collapses (0.92 → 0.64 → 0.35) because it ignores within-block correlation, while a block bootstrap restores it substantially (0.93 → 0.77 → 0.57, limited at L=16 by only ≈5 blocks in m=80). The point estimate ϵ̂ stays unbiased (0.30/0.29/0.31). So sticky routing does not break type-I but invalidates the i.i.d. confidence interval, which a block-aware interval repairs given enough blocks; a per-session or longer audit horizon supplies them. Contentadaptive routing (serving the probe honestly) is the separate, harder threat of App. S12.

S14

Black-Box Auditors

Table 10 positions IRIS against the auditors of Section 2. IRIS, FLIPS, KBF, B3IT and GateScope are all black-box API auditors, so the distinguishing axis is not white-box

Matched-budget comparison. The quantitative head-tohead in the body (Table 1) includes only auditors that run on the shared random-generation probe (FLIPS, MET, RUT). KBF and B3IT instead use target-specific probe families (knowledge-boundary factual recall, engineered border inputs) whose construction we cannot faithfully reconstruct from their papers, so we treat them qualitatively here and reproduce what is reproducible of each: B3IT’s fixed-budget strategy and KBF’s detection regime. A faithful head-tohead isolates the budgeting strategy by holding the probe and classifier fixed and varying only how the query count is set. Reimplementing a B3IT-style fixed-budget decision rule on our c100,1 probe at target AUROC 0.99, a fixed m=3 meets the target on only 70% of 30 model pairs, whereas IRIS’s estimate-then-budget loop meets it on 87%. This particular contrast is not cost-matched—IRIS spends a larger mean budget here (11 queries, ≈ 3 on easy pairs)—so it mixes adaptivity with budget; App. S14 isolates the two at matched cost (the canonical 87% vs. 73%). Two head-to-heads we deliberately do not run: (i) a faithful reimplementation of KBF’s knowledge-boundary factual-recall probe is a different probe family whose construction we cannot reproduce from the paper, so a probe-matched KBF comparison is out of scope; and (ii) on quantification, B3IT and GateScope report no routing fraction while KBF estimates one only for a known/candidate substitute under fixed routing and on its own knowledge-boundary probe, so a probe-matched ϵ̂ comparison again reduces to reproducing KBF’s probe family (above); the budgeting axis, not access, is the point. On the shared detection task IRIS matches KBF’s regime (KBF detects 5–10% routing for well-separated pairs; IRIS detects 5% in ≈64 queries, Section 6).

(a) substitution [c2,16 ]

(b) dilution = 0.2

1.0

0.9

verification AUROC

model access. It is twofold: IRIS (i) estimates its query budget from an on-data per-query rate rather than fixing it, and (ii) quantifies dilution on a universal probe—returning a calibrated ϵ̂ and naming the diluent by attribution—whereas the closest black-box auditors fix their query count and first build a target-specific probe: KBF estimates a routing fraction but only for a known (or candidate-pool) substitute under fixed routing and on a per-model knowledge-boundary probe, B3IT flags a temporal change from per-target border inputs, and GateScope measures gateways at the fleet level. IRIS and FLIPS share the core idea of using random generation as a model-discrimination signal (Richardeau et al. 2026; Hopkins, Renda, and Carbin 2023; Zhao, Du, and Wang 2026); our contribution is what we build on top of it. At the probe level, IRIS does not fix one pseudorandom probe scored by randomness-test features but selects a probe from a parameterized random-generation family {cn,L } by its on-data information rate Ibauc . At the algorithm level, IRIS replaces fixed-budget instance classification with an estimate-then-budget loop, a dilution tell-count test, and a calibrated routing-fraction estimate ϵ̂—turning a deployment-config identifier into a budgeted cross-model dilution auditor. (We similarly build on, rather than reinvent, B3IT’s exponential error law and temperature phase transition, Section 5.) The verification-AUROC accumulation curves underlying the comparison are in Fig. 8.

0.8

0.7

0.6

0.8

0.7

0.6 IRIS FLIPS MET-MMD RUT

0.5 20

queries m

IRIS RUT

0.5 20

queries m

Figure 8: Method comparison on the shared randomgeneration probe (c2,16 ), verification AUROC vs. queries m. (a) Substitution: IRIS ties a FLIPS-style classifier (AUROC ≈ 0.99 at m=1, both saturating to 1.0 by m=8): the shared random probe ties, and IRIS’s contribution is the budgeting and dilution capability it adds, while both dominate MET (maximum-mean-discrepancy two-sample test, degenerate at m=1, 0.11) and an adapted RUT (rankuniformity, plateauing near 0.74). (b) Dilution (ϵ=0.2): IRIS accumulates toward 1.0 while RUT stays near chance—the capability gap is the budgeting axis, not access (Table 10). Cost-matched comparison. The headline “87% vs. 70% for a B3IT-style fixed m=3” is not cost-matched: IRIS spent a mean budget ≈11 there, so part of the gap is simply 3.7× more queries. We decompose it by comparing, on the per-pair verification-AUROC(m) curves, three allocation rules at the same average budget B: fixed (m=B for every pair), oracle (per-pair minimal m to the target), and pilot-driven IRIS (rank pairs by a cheap pilot AUROC and waterfill the same pool onto the predicted-harder pairs, no target peeking). On the hardest probe (c10,1 , target AUROC 0.99, 30 pairs), at the matched budget m≈9 the fixed-m rule meets the target on 0.73 of pairs versus the pilot-driven IRIS allocation’s 0.87 (oracle 0.90); the fixed rule at m=3 meets only 0.60. So of IRIS’s advantage over fixed-m=3, roughly half is the larger budget its estimator prescribes (0.60→0.73) and roughly half is the adaptive allocation (0.73→0.87 at matched cost). The same holds on c100,1 (0.77 fixed vs. 0.80 pilot at m≈10, oracle 0.87; the per-pair estimate-then-budget loop of App. S14 reaches 0.87 on this probe by sizing each pair rather than waterfilling a fixed pool); on saturated easy probes (c10,8 ) all rules tie near 1.0. We caution that this compares IRIS’s classifier and allocation against adapted fixed-budget rules on our probe, not against the original KBF/B3IT/RUT/MET implementations (KBF’s knowledge-boundary probe we cannot reproduce), so it is a capability, not a head-to-head method, comparison. Significance of the head-to-head. We judge every performance gap in Table 1 with a two-sided Wilcoxon signedrank test on the paired per-split measurements behind the same cells (40 stratified splits, identical probe/seed/protocol to the table). IRIS’s margin over MET and RUT is significant on every metric they support: substitution AUROC at m=1 (p < 4 × 10−41 versus each), at m=8 (p=2 × 10−31 versus MET, p < 4 × 10−41 versus RUT), and dilution AUROC at m=8 and m=32 (p < 4 × 10−41 versus RUT). Against FLIPS the result is deliberately not a significant gap: on clean

Method

Access

Probe

Decision

Budget

IRIS FLIPS MET RUT KBF B3IT GateScope

text text ref. samples log-ranks text text text

rand-gen rand-gen natural natural knowledge border content/bill

attr+verify+ϵ̂ instance ID equality equality+mix (det.) routing+π̂ † change-det. fleet measure

est. m⋆ fixed fixed fixed fixed fixed —

Table 10: Where IRIS sits among black-box auditors. The output-only auditors (IRIS, FLIPS, KBF, B3IT, GateScope) share access; IRIS alone estimates the query budget (all others fix it) and quantifies dilution (ϵ̂ plus diluent ID) on a universal probe. “(det.)” marks detect-only mixture handling; † marks a routing-fraction estimate valid only for a known or candidate-pool substitute under fixed routing. substitution the two stay within 0.003 AUROC and the sign of the difference flips with the budget (IRIS higher at m=1, p=2 × 10−9 ; FLIPS higher at m=8, p=0.05), and closed-set attribution is a flat tie (p=0.14 at m=1, p=1.0 at m=8). IRIS’s separation from FLIPS is thus structural rather than metric—only IRIS scores dilution and estimates ϵ̂ (the “–” cells)—which no signed-rank test on a shared metric can express. The cost-matched allocation gap above (0.73→0.87) is reported as a decomposition of where the advantage originates.

S15

Extended Method Comparison

Section 2 and App. S14 place IRIS among the black-box substitution/dilution auditors that share its random-generation probe (FLIPS, MET, RUT) or its access model (KBF, B3IT, GateScope). We complete the picture along three axes a reader may raise, model attribution, concurrent single-token fingerprinting, and sequential change tests, and ground the dilution rate in classical mixture-testing theory. We first correct one framing from App. S14: KBF and B3IT both ship public code, so the barrier to a numeric head-to-head is not reproducibility but probe family. A faithful run puts each on its native probe (knowledge-boundary recall, engineered border inputs) at matched total budget, not on the shared C. Table 11 lays out where every comparable method sits across the audit’s axes, making concrete that no prior method jointly covers substitution, fractional dilution, open-set attribution, routing-fraction estimation, and a pre-committed budget from a single text-only universal probe. Attribution baselines and a generic text classifier. Output-based identification is a mature line: LLMmap (Pasquini, Kornaropoulos, and Ateniese 2025) learns a discriminative prompt bank, Model Provenance Testing (Nikolic, Baluta, and Saxena 2025) pairs a candidate library with a multiple-testing decision (the framing closest to IRIS’s attribute-then-budget loop), and the idiosyncrasy school (Sun et al. 2025) names a model from surface text alone. These read a natural-language response, not a randomgeneration challenge, so they are complementary rather than probe-matched. The sharpest question they pose is whether IRIS’s engineered visible-string features are needed at all, or whether a generic text classifier over the same randomgeneration responses already names the backend. We test this

directly with a Text-TFIDF attributor, character n-gram TFIDF over the raw response string with a logistic-regression head (the idiosyncrasy recipe of Sun et al. 2025), evaluated on the identical splits, seed, and m-aggregation as Table 1. Table 12 shows the engineered features earn their place exactly where a budgeted audit operates: at a single query IRIS leads the generic classifier by 0.07 on the ladder and 0.06 on the 17-endpoint gateway. By m=8 the three converge; the text classifier even edges ahead at m=32 on the gateway (0.999), but only by reading raw formatting and whitespace tells that IRIS’s content-only (length/format-invariant) subset deliberately discards for robustness to formatting drift and paraphrase (App. S11, App. S12). The operating regime of a pre-budgeted audit is small m, where the engineered signal wins. A real cross-provider deviation audit (not injected). Every dilution in the body is author-injected. To show IRIS flags a genuine same-advertised-model deviation in the wild, we audit open-weight models that OpenRouter serves through several upstream providers. OpenRouter lets a client pin the provider (provider.order with allow_fallbacks:false), and providers frequently serve identical weights at different quantization or with different kernels, so provider A vs. provider B for one slug is a real deviation test with a quasi-ground-truth precision label. We collect 80 probe repeats per (model, provider) on c2,16 and c10,8 for three models across their real providers (1,906 live responses) and test each provider pair for distributional equality with IRIS’s content-only detector (a 5-fold random-forest ROC-AUC with a Mann–Whitney U test on the out-of-fold scores; no reference weights, only returned strings). Table 13 reports the more discriminative probe per pair. IRIS flags 14 of the 15 provider pairs as distinguishable: DeepInfra’s fp4 serving of deepseek-chat-v3 separates from its fp8 peers at AUROC 0.97, two providers advertising mistral-nemo at the same nominal fp8 separate at AUROC up to 1.00 (a kernel- rather than precisionlevel difference), and the first-party Mistral endpoint is distinguishable from every reseller. The one indistinguishable pair is the two highest-precision llama-3.3-70b servings (bf16 vs. fp16), i.e. IRIS does not false-alarm when two providers really do serve near-identical distributions. This is a two-sample cross-provider consistency test

Table 11: Complete method comparison across the audit’s axes. Access: what the method reads (text=visible strings only; ref=reference samples from trusted weights; ranksw =log-ranks needing a white-box local reference; lp=log-probabilities; TEE=trusted hardware). Signal: probe family. Sub/Dil/Attr: detects whole-stream substitution / fractional dilution / attributes the served model (chg=change-only vs. the endpoint’s own past; det=detect-only mixture handling; p=partial). ϵ̂: estimates the routing fraction († only for a known/candidate substitute under fixed routing). Budget: est=estimated before the audit, fix=fixed by design, any=anytime-valid. API/Data: evaluated on real commercial endpoints / releases a public dataset or code (p=code or partial). Venues are in Section 2. Method

Access

Signal

Sub Dil Attr

ϵ̂

IRIS (ours)

text

rand-gen

est

FLIPS text rand-gen Y – MET text+ref natural Y – RUT ranksw natural Y det KBF text knowledge Y Y B3IT text border chg – Cai et al. text+lp natural Y Y Log-Prob Track lp natural chg – LLMmap text prompts p – Provenance text shared Y – Idiosyncrasies text natural – – IKP text knowledge – – One Token text rand-gen Y – BSA text natural chg –

Y – – p – – – Y Y Y Y Y –

– – – Y† – – – – – – – – –

fix fix fix fix fix fix fix fix fix fix fix fix any

– Y p Y Y Y Y Y – Y Y Y –

Y Y – Y Y p p Y Y Y Y p p

Table 12: Adding a generic output-text attributor (TextTFIDF: character n-gram TF-IDF + logistic regression over the raw response, the idiosyncrasy recipe Sun et al. 2025) to the closed-set attribution head-to-head, on the same splits/seed/m-aggregation as Table 1. Cells are attribution accuracy at budget m; IRIS and FLIPS reproduce that table’s Attr. row up to resampling. Bold marks the best method in the low-budget operating regime (m≤8). Method

Budget API Data

m=1

m=8

m=32

Local ladder c2,16 (K=6 Qwen3) IRIS .928 .999 — FLIPS .923 .999 — Text-TFIDF .857 .997 — Commercial gateway c10,8 (K=17) IRIS .688 .965 .985 FLIPS .670 .958 .976 Text-TFIDF .627 .943 .999

(are A and B the same?), a slightly different question from claimed-model verification (no trusted reference is assumed), and it corroborates on real endpoints what MET (Gao, Liang, and Guestrin 2025) found for production APIs and Bruckner (2026) for a flagship endpoint: the same advertised model is routinely served as measurably different distributions across a gateway. External corroboration on the released MET corpus. As a third-party real-data check we run a cheap char ngram detector on the released MET dataset (Gao, Liang, and Guestrin 2025): genuine completions from 9 commercial providers plus controlled local distortions, prompt-aligned so topic is balanced across the compared classes (Table 14).

Three findings. First, a clean negative control: the near-honest fp16 and fp32 servings of Llama-3-70B are not separable (AUROC 0.37), so the detector reads serving behaviour, not a formatting artifact. Second, quantization leaves a visiblestring signature on the larger model: nf4 and int8 separate from fp16 at AUROC 1.00 and 0.99 on Llama-3-70B (weaker on 8B, ≤0.53), and the watermark separates on both. Third, and most telling for our probe design, a generic classifier cannot attribute the 9 real providers from these naturallanguage completions (0.13/0.09 accuracy vs. 0.11 chance). This is precisely why IRIS probes with high-entropy random generation rather than natural language: on the same class of real commercial endpoints, the random-generation cross-provider audit above separates 14 of 15 provider pairs (Table 13), and MET’s own many-sample MMD test likewise flags provider deviations, whereas a cheap per-response classifier on natural text sits at chance. Concurrent single-token fingerprinting. Concurrent with this work, Bruckner (2026) fingerprint and verify commercial endpoints from the single-token output distribution of trivial random-generation prompts, a census of 165 models over OpenRouter at roughly one token per query, reporting a 7.3% verification equal-error rate and 59.5% leave-one-out lineage recovery. The shared premise, visible text, no logprobabilities, and random generation as the discriminator, makes it the closest point of comparison to IRIS’s probe, and it independently corroborates that sampling bias identifies backends at scale. IRIS is distinguished not by the probe but by what it builds on top: fractional dilution detection with a calibrated routing fraction ϵ̂, open-set attribution across a candidate library, an estimate-then-budget plan that fixes m before any suspect query, and sample-complexity guarantees (e−Im verification, the 1/ϵ dilution wall). Single-token ver-

Table 13: Real cross-provider audit: the same advertised open-weight model served by different real OpenRouter providers (routing pinned, allow_fallbacks:false), tested for distributional equality by IRIS’s content-only detector. AUROC≈0.5 = indistinguishable; AUROC≫0.5 (Mann–Whitney p<0.05) = a real same-model deviation. Each row uses the more discriminative of c2,16 /c10,8 ; “(q)” is OpenRouter’s advertised quantization. Deviations are provider-side, not author-injected.

Table 14: External corroboration on the released MET corpus (Gao, Liang, and Guestrin 2025) (genuine provider completions, prompt-aligned). Left block: controlled-distortion detection versus the fp16 reference (AUROC; fp16-vsfp32 is a negative control, wm=watermark). Right: 9way real-provider attribution accuracy from natural-language completions. A generic char-n-gram classifier catches quantization on the larger model but cannot fingerprint providers from natural text, motivating IRIS’s random-generation probe.

Provider A (quant) Provider B (quant) AUROC verdict llama-3.3-70b-instruct DeepInfra (fp8) Novita (bf16) DeepInfra (fp8) Together (fp8) DeepInfra (fp8) WandB (fp16) Novita (bf16) Together (fp8) Novita (bf16) WandB (fp16) Together (fp8) WandB (fp16) deepseek-chat-v3 DeepInfra (fp4) Novita (fp8) DeepInfra (fp4) StreamLake (unk.) Novita (fp8) StreamLake (unk.) mistral-nemo DeepInfra (fp8) DekaLLM (fp8) DeepInfra (fp8) Mistral (unk.) DeepInfra (fp8) Novita (fp8) DekaLLM (fp8) Mistral (unk.) DekaLLM (fp8) Novita (fp8) Mistral (unk.) Novita (fp8)

.68 .67 .71 .74 .53 .72

different different different different indist. different

.97 .97 .67

different different different

1.00 .77 .84 1.00 1.00 .97

different different different different different different

Llama-3 70B 8B †

fp32†

nf4

int8

prov. attr.

.37 .53

1.00 .53

.99 .45

.72 .64

.13 .09

negative control (near-honest pair, should not separate); attribution chance = 0.11.

Verifiable-inference contrast. A separate, cooperative line certifies inference cryptographically: TOPLOC (Ong et al. 2025) commits to locality-sensitive hashes of activations so a validator can later detect a model, prompt, or precision swap. Such schemes give strong guarantees but require provider participation (commitments, shared seeds, or trusted hardware, as in the TEE fallback of Cai et al. 2025); IRIS is a unilateral audit a client runs from returned text alone, with no provider cooperation.

S16

ification is essentially the m=1, ϵ=1 special case of IRIS’s substitution head. Sequential vs. fixed-budget tests. A parallel design choice is whether to test at a fixed budget or sequentially. Behavioralshift auditing (Richter et al. 2025) and nonparametric twosample testing by betting (Shekhar and Ramdas 2023) accumulate an anytime-valid e-process and stop when evidence crosses 1/α, controlling type-I error at every stopping time, while “You’ve Changed” (Dima et al. 2025) runs a featuredistribution two-sample test against an endpoint’s own past. IRIS deliberately pre-commits m: the audit budget is itself the object we estimate (Section 4), so a client can price and contest an audit before issuing any suspect query, which an anytime rule does not promise. The two are composable, a sequential back-end could spend the estimated m⋆ as its horizon, and we treat the fixed-budget guarantee (App. S5) as the deployable contract. Theory lineage of the dilution rate. IRIS’s dilution model Qϵ = (1 − ϵ)P + ϵR is Huber’s ϵ-contamination (Huber 1964), and the 1/ϵ-versus-ϵ−2 dichotomy we measure (Prop. 6, App. S26) is the sparse-mixture detection boundary studied by higher criticism (Donoho and Jin 2004): a separating tail admits an adaptive Θ(1/ϵ) detector, while its absence forces the ϵ−2 two-proportion rate. We therefore present IRIS’s rate as a measured instance of this classical boundary rather than a new phenomenon, which is why we report the tail exponent κ (App. S26.1) instead of asserting 1/ϵ unconditionally.

One-Class Detector

IRIS’s deployed score s = − log P̂(P | y) is a multiclass posterior whose negatives are the enrolled candidates, so the leave-one-out unknown-diluent test (App. S3) still trains on the other K−1 endpoints. To isolate how much detection owes to those negatives—and to test the principle that an unknown substitute is flagged from the reference alone— we replace the score with a genuine one-class detector: an isolation forest fit on the claimed model’s content-only responses only, scored as the out-of-distribution anomaly. On the 17-endpoint gateway (c10,8 ), across the pairs passing the multiclass margin check, the reference-only detector still separates—single-query AUROC median 0.75 (vs. 0.99 multiclass) and per-query tell rate q ≈ 0.13 (vs. 0.98)—clearing the α=0.05 false-positive floor (realized FPR 0.07) on about half of them at a single query and accumulating over m like any tell. The price of dropping the candidate negatives is thus a real but bounded ∼ q −1 budget inflation (≈ 7× here). The no-enrollment guarantee is therefore correct for detection—the alarm is raised from the claimed fingerprint alone, as the one-class detector confirms—while the candidate roster earns its place by sharpening the score and naming the diluent, not by enabling detection.

S17

LCB Budgeting

IRIS budgets against a (1−γ) lower-confidence bound I on the fitted exponent rather than inflating the point budget by a flat safety factor csafe (Section 4); we detail the rule here and compare it against the flat-factor baseline it replaces. The flat rule scales the predicted m⋆ by csafe , trading queries for a

Table 15: Same-endpoint FPR calibration (nominal α=0.05): realized false-positive and true-positive rates vs. budget m (Section 6). At operational budgets (m=4–8) realized FPR matches nominal; larger budgets require proportionally stronger calibration. m=4

m=8

m=16 m=32

Table 16: Flat-factor baseline (K=6): scaling the predicted m⋆ by a safety factor csafe trades a few queries for a higher budget-rule hit rate. IRIS replaces this with the LCB rule (Table 17).

1.0 1.5 2.0

target AUROC 0.95

target AUROC 0.99

mean m

hit rate

mean m

hit rate

6.7 9.5 11.3

0.80 0.89 0.91

10.9 14.6 16.8

0.69 0.74 0.83

10−3

P R 1 − AUROC = P(Sm ≥ Sm )

P R ≥ P(Sm ≥ τmean ) P(Sm ≤ τmean )

10−2

10−3

higher budget-rule hit rate (Table 16). The budgeting rests on a predictive base: the error a cheap pilot (m ≤ 4) extrapolates tracks the realized held-out error closely (Fig. 9), which is exactly why a conservative bound on the same fit yields a calibrated budget. Rank error as a miss-probability surrogate. The deployed substitution head thresholds the episode mean Sm at the reference (1−α)-quantile τmean (Eq. (8)), so its miss probR ≤ τmean ). Since the pilot draws ability is δmean := PR (Sm the reference and substitute episodes independently,

≥ α δmean ,

Budget-from-Ibauc is predictive (median | log ratio| = 0.19, 68% within 2×)

10−1

c2,16 .05/1.00 .06/1.00 .09/1.00 .12/1.00 c100,1 .08/0.83 .08/0.90 .11/0.96 .14/0.99 c10,1 .06/0.82 .09/0.86 .13/0.89 .18/0.90

safety csafe

c2,16 c2,8 c10,8 c2,1 c10,1 c100,1 2× band perfect

predicted error from m ≤ 4 pilot

Probe

10−2

(1)

the quantile definition. Thus δmean ≤ (1 − AUROC)/α: driving the rank error to δauc forces the calibrated miss to δmean ≤ δauc /α, so the rank-error exponent lower-bounds the calibrated-miss exponent and budgeting on Ibauc is conservative in rate. The bound is exact in m but loose by the constant 1/α, so setting δauc =δ pins the miss exponent and leaves only the constant to the hit-rate calibration below. The optimism is in the fit, not this inequality: Ibauc extrapolated from m ≤ 4 is a two-sample rate not provably below I sc (Prop. 1), which is why IRIS budgets against its lowerconfidence bound I auc . The LCB rule replaces csafe by a (1−γ) lower-confidence bound I on the fitted exponent (bootstrap or t-based over calibration splits), budgeting m⋆ = ⌈(â − log δauc )/I⌉ (the rank-error target of Eq. (7), set to the miss probability δ). On the K=6 ladder (γ=0.1) this reaches a matched hit rate at fewer queries than the flat factor at the 0.95 target (Table 17); it is not a coverage certificate—at the 0.99 target no

100

Figure 9: Budget-from-Ibauc is predictive (K=6). Predicted verification error from an m≤4 pilot vs. realized error at held-out larger m, per (pair, probe); the dashed line is perfect prediction and the band is a 2× tolerance. Across 224 validation points the median | log-ratio| is 0.19 with 68% inside the band—the small-m extrapolation is slightly optimistic, motivating the lower-confidence-bound budget below. Table 17: Per-pair adaptive budgeting (K=6, γ=0.1): an LCB on Ibauc vs. the flat safety factor csafe . Mean budget m and budget-rule hit rate (realized AUROC ≥ target). rule

P R P ≥ ≤ τmean } ⊆ {Sm ≥ τmean , Sm using the inclusion {Sm P R Sm } with independence, and P(Sm ≥ τmean ) ≥ α from

10−1

realized error at held-out m

flat s=1 flat s=2 LCB (bootstrap) LCB (t-based)

target AUROC 0.95

target AUROC 0.99

mean m

hit

mean m

hit

5.5 8.4 5.8 6.6

0.81 0.91 0.86 0.88

8.4 12.4 8.7 10.0

0.68 0.77 0.67 0.72

rule attains nominal coverage, as part of the gap is intrinsic AUROC saturation on low-information probes (c10,1 , c100,1 ) rather than estimator variance. A calibrated coverage guarantee, and a resampling unit that does not reuse overlapping splits, remain future work. (r) (r) Concretely, from split-level pilot fits {(âcj , Ibauc,cj )}R r=1 , the one-sided t-based bounds that enter the substitution budget msub (Eq. (7)) are I auc,cj = Ibauc,cj − t1−γ,R−1 se b R (Icj ), acj = âcj + t1−γ,R−1 se b R (acj ),

(2) (3)

with t1−γ,R−1 the one-sided Student-t quantile and √ (r) se b R (Icj ) = sd({Ibauc,cj }R r=1 )/ R the across-split standard error (likewise for acj ).

(a) detection by audit margin (=0.3, c2,16 )

(b) early stopping vs the fixed budget

betting e-process SPRT

1.0

200

S19.1

fixed m? (Thm. 1a) betting E[τ ] horizon 200 SPRT E[τ ]

0.8

expected queries to stop E[τ ]

detection rate within horizon

175

0.6

0.4

no decision by horizon

150 125 100 75 50

0.2 25 honest FPR 0.004

0.0

same-family (size)

distinct model

low-sep. (±temp)

same-family (size)

distinct model

low-sep. (±temp)

Figure 10: Anytime-valid sequential auditing on the separation ladder (ϵ=0.3, c2,16 ). (a) Detection rate within the horizon for the betting e-process vs. the SPRT; honest streams sit at the false-positive floor (dotted). (b) Expected stopping time E[ν] vs. the fixed budget m⋆ of Thm. 1(a). The e-process stops early on detectable suspects and does not stop within the horizon on the low-separation stress test, which is the sequential analogue of an indeterminate audit.

S18

Sequential Auditing

IRIS fixes a query budget m⋆ in advance; a natural alternative is a sequential test that stops at a data-dependent stopping time ν, as soon as the accumulated evidence is decisive. We implement two—a betting e-process (anytime-valid, sound under optional stopping at any time) and a sequential probability ratio test (SPRT)—both at level α=0.05 against the three-rung separation ladder (reference qwen3:8b@T =1, probe c2,16 , ϵ=0.3 dilution, horizon 200, 2000 streams; Fig. 10). The betting e-process stops early on detectable suspects—expected stopping time E[ν]≈28 on a samefamily size substitution and ≈68 on a distinct model, comparable to the fixed budget—while detecting at rate ≥0.99. On the low-separation temperature stress test it does not stop (detection 0.0095, E[ν] pinned at the horizon), which is the sequential analogue of returning indeterminate. Honest streams almost never trip (FPR 0.0045 for the e-process, 0.0235 for the SPRT). Under substitution (ϵ=1.0) stopping is faster still (E[ν]≈8 same-family, ≈17 distinct). The e-process thus delivers an always-valid decision without committing to m in advance—complementary to the estimate-then-budget loop, at the price of a modest E[ν] overhead over the oracle fixed m⋆ on the easiest pairs.

S19

Fingerprint Library

Figure 11 renders 53 enrolled endpoints (8 local Ollama models and 45 commercial/open gateway endpoints) spanning 18 model families as a gallery of fingerprints. Each “iris” is one endpoint: radial fibers encode its robust-normalized visiblerandomness signature (aggregated over c2,8 /c2,16 and c10,8 ), hue marks the model family, and the pupil names the endpoint. The view is illustrative—same-family endpoints share a hue and a broadly similar corona, while cross-family endpoints are plainly distinct. The gallery is a superset enrolled only for visualization: several shown frontier endpoints enter no quantitative metric; the 8 local serves are the K=6 Qwen3 ladder plus two extras, and the 45 gateway endpoints are the App. S9 pool.

Probe Ranking

The probe is selected, not asserted: ranking probes by the calibration-estimated Ibauc from a cheap pilot predicts their realized query efficiency (Fig. 12). The two quantities are monotonically related—high-Ibauc sequence probes reach verification AUROC ≥0.99 in a single query while the zeroinformation c2,1 (Ibauc ≈ 0) never does—so the auditor reads Ibauc off calibration and picks the probe before spending its audit budget. On the expanded 13-condition grid this ordering is significant (Spearman(Ibauc , accuracy@m=8)=0.82, p<10−3 ; Section 6).

S20

The Two IRIS Stages

Audit-Plan Construction may query the trusted reference and candidate endpoints under known labels, whereas Audit Execution queries the unlabeled suspect and applies only the frozen plan. Table 18: Data access in the two IRIS stages. Audit-Plan Construction fixes Ibauc , m⋆ , and the mode-specific decision thresholds from labeled data; Audit Execution uses unlabeled live suspect responses. Some deployments additionally remeasure the null tell rate on a fresh honest window of the claimed model (App. S3); this window is the one executionstage use of reference-side data, and its trust assumption is discussed in App. S10. Audit-Plan Construction Endpoints Labels Sample P ? Observes Produces

Audit Execution

trusted P , candi- live suspect (+ opdates {Rj } tional honest window of P ) known none for suspect yes optional pre-audit honest window visible strings visible strings score library, cal- suspect responses, ibrated thresholds, decisions, ϵ̂, R̂ frozen budget plan

S21

Dilution Calibration

This appendix writes the dilution budget of Section 4.1 out in full: it gives the explicit threshold grid, level-α cutoff, diluted-binomial power, and cheapest-threshold selection behind the compact definition of mdil (c, j; τ ) in Eq. (12), with all (c, j) indices retained. Dilution budget. Over a fixed honest-tail grid Aresp , the b 1−α ({sc (yrP )}nP ) : α0 ∈ threshold grid is Tc = {Q 0 r=1 Aresp }, with honest/substitute tell rates α b1,cj (τ ), qbcj (τ ) from Eq. (10). For target dilution ϵmin the diluted tell probability is pbϵ,cj (τ ) = (1 − ϵmin )b α1,cj (τ ) + ϵmin qbcj (τ ). With Um (b; p) = PK∼B(m,p) (K ≥ b) the binomial upper tail, the level-α cutoff and the diluted-binomial power are bα (m, p0 ) = min{b : Um (b; p0 ) ≤ α}, πϵ,cj (m, τ ) = Um (bα (m, α b1,cj (τ )); pbϵ,cj (τ )) .

(4)

A Library of Visible-String Fingerprints each iris is one endpoint; radial fibers encode its random-generation signature, hue marks the model family

Qwen

Qwen3 0.6B

Qwen3 1.7B

Qwen3 14B

Qwen3 235B

Qwen3 30B

Qwen3 32B

Qwen3 32B API

Qwen3 4B

Qwen3 8B

Qwen3 8B API

Qwen3 Max

GPT-4.1

GPT-4o mini

GPT-5.4 mini

GPT-5.5

gpt-oss 120B

gpt-oss 20B

gpt-oss 20B API

Claude 3.5 Haiku

Claude Haiku 4.5

Claude Opus 4.8

Claude Sonnet 4.6

Gemini 2.5 Flash

Gemini 2.5 Pro

Gemini 3.1 Pro

Gemini 3.5 Flash

Gemma 27B

Gemma 4B

Hermes 3 Llama 405B

Llama 3.1 8B

Llama 3.2 1B

Llama 3.3 70B

Llama 4 Maverick

DeepSeek Chat

DeepSeek V4 Flash

DeepSeek V4 Pro

Phi-4

Mistral Large

Mistral Medium 3

Grok 4.20

Grok 4.3

Kimi K2.5

GLM-5.2

MiniMax M3

Command A

Nova Lite

Nova Micro

Nova Pro

Jamba Large

Nemotron Super 120B

Nemotron Ultra 550B

Hunyuan A13B

MiMo v2.5 Pro

GPT

Claude

Gemini

Llama

DeepSeek

Phi

Mistral

Grok

Kimi

GLM

MiniMax

Cohere

Nova

Jamba

Nemotron

Hunyuan

MiMo

Figure 11: A library of visible-string fingerprints for 53 enrolled endpoints (8 local, 45 gateway) across 18 model families. Each iris is one endpoint; radial fibers encode its random-generation signature (robust-normalized across endpoints), hue marks the model family, and the pupil names the endpoint.

Estimated Ibauc predicts probe efficiency Spearman ρ = 0.89 c2,16 c10,8

1.000

c2,8

realized verification AUROC at m=8

0.975

c100,1

0.950 c10,1

0.925 0.900 0.875 0.850 0.825 c2,1

0.800 0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

calibration-estimated Ibauc (from m ≤ 4 pilot)

Figure 12: Calibration-estimated Ibauc (from an m≤4 pilot) predicts realized verification AUROC at m=8, one point per probe family. The estimate orders probes by efficiency before any audit budget is spent: high-Ibauc sequence probes saturate at m=8 while c2,1 (Ibauc ≈ 0) lags. IRIS keeps only the positive-separation thresholds Tcj+ = {τ ∈ Tc : qbcj (τ ) > α b1,cj (τ )},

(5)

marks the pair low-margin if Tcj+ = ∅, and otherwise sizes the budget and selects the cheapest surviving threshold, mdil (c, j; τ ) = min{m ∈ N+ : πϵ,cj (m′ , τ ) ≥ 1 − δ

for every m′ ≥ m}, τcj ∈ arg min+ mdil (c, j; τ ), (6) τ ∈Tcj

with mdil (c, j) = mdil (c, j; τcj ).

S22

Probes and Sampling

For full reproducibility we give the verbatim probe prompts and the exact decoding parameters behind every response in the paper. Each query is a single user turn whose content is the probe string below; no system prompt is used, and the prompt is fixed (no per-query randomization), so the only stochasticity is the endpoint’s own sampling. The target distribution is uniform over each probe’s alphabet. We state the prompts in the clear by design: IRIS’s budget is set by the realized per-query information rate Ibauc , not by probe secrecy, and the matched-budget and routing analyses of Section 6 hold when the probe is known to the provider; a secrecy-dependent deployment can instead draw from a held-out probe bank using the same templates. Primary probe set. Table 19 lists the six probes (three single-draw, three short-sequence) behind all headline results using the cn,L notation of Section 4.1. Here n is the alphabet/range size, L the requested sequence length, and npred the decode-token cap (num_predict/max_tokens); singledraw probes use L=1. Expansion probes. The probe-ranking / information-law experiment (Sec. 6, 13 probe conditions) adds seven probes spanning a wider entropy × length grid; Table 20 gives them verbatim.

Decoding parameters. We use two collection paths, both with temperature 1.0 on the main run and top_p=1.0 throughout. Local endpoints are served by Ollama via /api/generate with think=false (reasoning disabled), num_predict per Table 19, and a greedy singletoken prewarm call before each model; the temperature sweep covers T ∈ {0, 0.5, 1, 1.5, 2}, and the main run draws 120 repeats per model–probe. Gateway endpoints are queried through an OpenAI-compatible /chat/completions interface with temperature=1.0, top_p=1.0, and max_tokens equal to the same npred for models that allow reasoning to be turned off and max(npred , 700) for mandatory-reasoning models; reasoning is disabled with {"enabled": false} where supported, otherwise forced to the lowest setting ({"effort": "low"}) for models that cannot disable it (e.g. Gemini 3.x, gpt-oss). Gateway pools draw 100 repeats per model–probe. These settings yield the ∼24k local and 10,200 + 27,000 gateway responses in the release bundle. We log and analyze only the visible response string (and runtime metadata), never weights, logits, or token ranks.

Probe code

parser

npred

c10,1

char

c100,1

100

number

c2,1

char

c2,8

char

c2,16

char

c10,8

char

Verbatim prompt (single user turn; no system prompt) Output one uniformly random digit from 0 to 9. Output only the single digit, nothing else. Choose one uniformly random integer between 1 and 100 inclusive. Output only the integer. Flip one fair coin. Output only a single character: H for heads or T for tails. Generate exactly 8 symbols by independently choosing uniformly between 0 and 1. Output the sequence only, with no spaces and no explanation. Generate exactly 16 symbols by independently choosing uniformly between 0 and 1. Output the sequence only, with no spaces and no explanation. Generate exactly 8 independent uniformly random digits from 0 to 9. Output only the compact digit string, no separators.

Table 19: The six primary probes. Prompts are reproduced verbatim. Decode cap rule: npred =48 for single-draw probes and max(72, 3L+40) for short-sequence probes.

Probe code

parser

npred

c26,1

char

c16,1

char

c6,1 c2,4

6 2

1 4

char char

48 72

c2,32

char

136

c10,16

char

c6,8

char

Verbatim prompt Output one uniformly random lowercase letter from a to z. Output only the single letter. Output one uniformly random hexadecimal digit (0-9 or a-f). Output only the single character. Roll one fair six-sided die. Output only a single digit from 1 to 6. Generate exactly 4 symbols by independently choosing uniformly between 0 and 1. Output the sequence only, with no spaces and no explanation. Generate exactly 32 symbols by independently choosing uniformly between 0 and 1. Output the sequence only, with no spaces and no explanation. Generate exactly 16 independent uniformly random digits from 0 to 9. Output only the compact digit string, no separators. Generate exactly 8 outcomes of a fair six-sided die (digits 1 to 6). Output only the compact string, no separators.

Table 20: The seven expansion probes used only for the Ibauc probe-ranking law.

S23

Feature Dictionary

Table 21 lists the fixed 179-dimensional vector ϕ(y) used by the main IRIS classifiers. The list contains the numeric training features only: identity fields, temperature/top-p, repeat index, requested length, and API runtime counters are excluded. Coordinates that do not apply to a particular probe are left missing and are zero-imputed inside the training pipeline. Table 21: Complete visible-string feature dictionary. Each row gives two coordinates of the 179-dimensional vector ϕ(y). # Feature

Definition

# Feature

Definition

1 alphabet_size

Probe alphabet size n.

2 response_chars

3 response_non_ws_chars

Character length after 4 contains_newline whitespace removal. Indicator that the raw re- 6 contains_alpha_text sponse contains whitespace. Non-separator characters 8 checked_chars outside the target alphabet. 10 parsed_len invalid_chars/ checked_chars, zero if none checked. Indicator that parsed_len 12 parse_success equals requested length.

Raw response character length. Indicator that the raw response contains a newline. Indicator for a long alphabetic/CJK text run. Non-separator characters checked against the alphabet. Number of target-alphabet symbols recovered.

5 contains_space 7 invalid_chars 9 invalid_rate 11 exact_len 13 length_error

14 abs_length_error

39 count_0

parsed_len minus requested length. abs_length_error divided by requested length. Shannon entropy of recovered symbol counts. ℓ1 distance from target symbol probabilities. Smallest observed symbol frequency. Most frequent symbol count divided by parsed_len. Mean length of constantsymbol runs. Maximum constant-symbol run length. Fraction of adjacent parsed symbols that differ. Entropy of adjacent twosymbol blocks. zlib compressed parsedstring length divided by parsed_len. Autocorrelation of alphabetindexed symbols at lag 2. Alphabet index of the first parsed symbol. Count of parsed symbol 0.

41 count_1

Count of parsed symbol 1.

42 freq_1

43 count_2

Count of parsed symbol 2.

44 freq_2

45 count_3

Count of parsed symbol 3.

46 freq_3

47 count_4

Count of parsed symbol 4.

48 freq_4

15 rel_length_error 17 entropy 19 l1_target 21 min_freq 23 mode_count_ratio 25 mean_run 27 longest_run 29 alternation_rate 31 bigram_entropy 33 compression_ratio 35 autocorr_lag2 37 first_symbol_code

16 chi2_target 18 entropy_norm 20 max_freq 22 freq_range 24 run_count 26 std_run 28 longest_run_ratio 30 expected_alternation 32 trigram_entropy 34 autocorr_lag1 36 autocorr_lag3 38 last_symbol_code 40 freq_0

Indicator for nonempty parse and invalid_rate≤ 0.25. Absolute value of length_error. Pearson χ2 distance from target symbol counts. entropy divided by log2 n. Largest observed symbol frequency. max_freq minus min_freq. Number of constant-symbol runs. Standard deviation of run lengths. longest_run divided by parsed_len. P Target value 1 − a p2a . Entropy of adjacent threesymbol blocks. Autocorrelation of alphabetindexed symbols at lag 1. Autocorrelation of alphabetindexed symbols at lag 3. Alphabet index of the last parsed symbol. Frequency of parsed symbol 0. Frequency of parsed symbol 1. Frequency of parsed symbol 2. Frequency of parsed symbol 3. Frequency of parsed symbol 4. Continued on next page

# Feature

Definition

# Feature

Definition

49 count_5

Count of parsed symbol 5.

50 freq_5

51 count_6

Count of parsed symbol 6.

52 freq_6

53 count_7

Count of parsed symbol 7.

54 freq_7

55 count_8

Count of parsed symbol 8.

56 freq_8

57 count_9

Count of parsed symbol 9.

58 freq_9

59 trans_0_0

Adjacent transition frequency 0→0. Adjacent transition frequency 1→0. Mean mismatch to the alternating pattern starting with symbol 0. Symbol-0 frequency in the first half. Absolute first-half vs secondhalf symbol-0 frequency gap.

60 trans_0_1

Frequency of parsed symbol 5. Frequency of parsed symbol 6. Frequency of parsed symbol 7. Frequency of parsed symbol 8. Frequency of parsed symbol 9. Adjacent transition frequency 0→1. Adjacent transition frequency 1→1. Mean mismatch to the alternating pattern starting with symbol 1. Symbol-0 frequency in the second half. Standard deviation of symbol-0 frequencies across quarters. Adjacent transition frequency 0→3. Adjacent transition frequency 0→5. Adjacent transition frequency 0→7. Adjacent transition frequency 0→9. Adjacent transition frequency 1→3. Adjacent transition frequency 1→5. Adjacent transition frequency 1→7. Adjacent transition frequency 1→9. Adjacent transition frequency 2→1. Adjacent transition frequency 2→3. Adjacent transition frequency 2→5. Adjacent transition frequency 2→7. Adjacent transition frequency 2→9. Adjacent transition frequency 3→1. Adjacent transition frequency 3→3. Adjacent transition frequency 3→5. Adjacent transition frequency 3→7. Adjacent transition frequency 3→9. Adjacent transition frequency 4→1.

61 trans_1_0 63 alt0_mismatch 65 first_half_symbol0_ freq 67 half_freq_gap 69 trans_0_2 71 trans_0_4 73 trans_0_6 75 trans_0_8 77 trans_1_2 79 trans_1_4 81 trans_1_6 83 trans_1_8 85 trans_2_0 87 trans_2_2 89 trans_2_4 91 trans_2_6 93 trans_2_8 95 trans_3_0 97 trans_3_2 99 trans_3_4 101 trans_3_6 103 trans_3_8 105 trans_4_0

Adjacent transition quency 0→2. Adjacent transition quency 0→4. Adjacent transition quency 0→6. Adjacent transition quency 0→8. Adjacent transition quency 1→2. Adjacent transition quency 1→4. Adjacent transition quency 1→6. Adjacent transition quency 1→8. Adjacent transition quency 2→0. Adjacent transition quency 2→2. Adjacent transition quency 2→4. Adjacent transition quency 2→6. Adjacent transition quency 2→8. Adjacent transition quency 3→0. Adjacent transition quency 3→2. Adjacent transition quency 3→4. Adjacent transition quency 3→6. Adjacent transition quency 3→8. Adjacent transition quency 4→0.

62 trans_1_1 64 alt1_mismatch 66 second_half_symbol0_ freq 68 quarter_freq_std

fre- 70 trans_0_3 fre- 72 trans_0_5 fre- 74 trans_0_7 fre- 76 trans_0_9 fre- 78 trans_1_3 fre- 80 trans_1_5 fre- 82 trans_1_7 fre- 84 trans_1_9 fre- 86 trans_2_1 fre- 88 trans_2_3 fre- 90 trans_2_5 fre- 92 trans_2_7 fre- 94 trans_2_9 fre- 96 trans_3_1 fre- 98 trans_3_3 fre- 100 trans_3_5 fre- 102 trans_3_7 fre- 104 trans_3_9 fre- 106 trans_4_1

Continued on next page

# Feature 107 trans_4_2 109 trans_4_4 111 trans_4_6 113 trans_4_8 115 trans_5_0 117 trans_5_2 119 trans_5_4 121 trans_5_6 123 trans_5_8 125 trans_6_0 127 trans_6_2 129 trans_6_4 131 trans_6_6 133 trans_6_8 135 trans_7_0 137 trans_7_2 139 trans_7_4 141 trans_7_6 143 trans_7_8 145 trans_8_0 147 trans_8_2 149 trans_8_4 151 trans_8_6 153 trans_8_8 155 trans_9_0 157 trans_9_2 159 trans_9_4 161 trans_9_6 163 trans_9_8 165 number_found

Definition

# Feature

Adjacent transition fre- 108 trans_4_3 quency 4→2. Adjacent transition fre- 110 trans_4_5 quency 4→4. Adjacent transition fre- 112 trans_4_7 quency 4→6. Adjacent transition fre- 114 trans_4_9 quency 4→8. Adjacent transition fre- 116 trans_5_1 quency 5→0. Adjacent transition fre- 118 trans_5_3 quency 5→2. Adjacent transition fre- 120 trans_5_5 quency 5→4. Adjacent transition fre- 122 trans_5_7 quency 5→6. Adjacent transition fre- 124 trans_5_9 quency 5→8. Adjacent transition fre- 126 trans_6_1 quency 6→0. Adjacent transition fre- 128 trans_6_3 quency 6→2. Adjacent transition fre- 130 trans_6_5 quency 6→4. Adjacent transition fre- 132 trans_6_7 quency 6→6. Adjacent transition fre- 134 trans_6_9 quency 6→8. Adjacent transition fre- 136 trans_7_1 quency 7→0. Adjacent transition fre- 138 trans_7_3 quency 7→2. Adjacent transition fre- 140 trans_7_5 quency 7→4. Adjacent transition fre- 142 trans_7_7 quency 7→6. Adjacent transition fre- 144 trans_7_9 quency 7→8. Adjacent transition fre- 146 trans_8_1 quency 8→0. Adjacent transition fre- 148 trans_8_3 quency 8→2. Adjacent transition fre- 150 trans_8_5 quency 8→4. Adjacent transition fre- 152 trans_8_7 quency 8→6. Adjacent transition fre- 154 trans_8_9 quency 8→8. Adjacent transition fre- 156 trans_9_1 quency 9→0. Adjacent transition fre- 158 trans_9_3 quency 9→2. Adjacent transition fre- 160 trans_9_5 quency 9→4. Adjacent transition fre- 162 trans_9_7 quency 9→6. Adjacent transition fre- 164 trans_9_9 quency 9→8. Indicator that the number 166 number_value parser found an integer.

Definition Adjacent transition frequency 4→3. Adjacent transition frequency 4→5. Adjacent transition frequency 4→7. Adjacent transition frequency 4→9. Adjacent transition frequency 5→1. Adjacent transition frequency 5→3. Adjacent transition frequency 5→5. Adjacent transition frequency 5→7. Adjacent transition frequency 5→9. Adjacent transition frequency 6→1. Adjacent transition frequency 6→3. Adjacent transition frequency 6→5. Adjacent transition frequency 6→7. Adjacent transition frequency 6→9. Adjacent transition frequency 7→1. Adjacent transition frequency 7→3. Adjacent transition frequency 7→5. Adjacent transition frequency 7→7. Adjacent transition frequency 7→9. Adjacent transition frequency 8→1. Adjacent transition frequency 8→3. Adjacent transition frequency 8→5. Adjacent transition frequency 8→7. Adjacent transition frequency 8→9. Adjacent transition frequency 9→1. Adjacent transition frequency 9→3. Adjacent transition frequency 9→5. Adjacent transition frequency 9→7. Adjacent transition frequency 9→9. First integer recovered by the number parser. Continued on next page

# Feature 167 number_log10 169 number_parity 171 number_mod100 173 freq_H 175 freq_T 177 trans_H_T 179 trans_T_T

Definition

# Feature

log10 of number_value, 168 number_num_digits when positive. number_value modulo 2. 170 number_mod10 number_value modulo 172 count_H 100. Frequency of parsed symbol 174 count_T H. Frequency of parsed symbol 176 trans_H_H T. Adjacent transition fre- 178 trans_T_H quency H→T. Adjacent transition frequency T→T.

Definition Decimal digit count of number_value. number_value modulo 10. Count of parsed symbol H. Count of parsed symbol T. Adjacent transition quency H→H. Adjacent transition quency T→H.

frefre-

Score Information

Definition 1 distinguishes the oracle exponent I ⋆ , the score exponent I sc , and the plug-in estimate Ibauc . We make I sc precise and bound it.

Definition 1 (Score exponent). Let µP , µR be the pushforward laws of the score s(y) = − log P̂(P | y) under y ∼ P and y ∼ R. The score exponent is the Chernoff information of these scalar laws, I sc (c; P, R) = R − min0≤λ≤1 log µP (s)1−λ µR (s)λ ds.

Proposition 4 (Score compression). Let s = s(y) be a deterministic measurable score with push-forward laws µP = s# P and µR = s# R. Then I sc ≤ I ⋆ . If P ̸= R, equality holds iff s is sufficient for {P, R}, equivalently iff the likelihood ratio P/R is constant on each level set of s. The equality condition says exactly when the scalar score loses no response-level evidence. Calibrated log-odds that are affine and strictly monotone in log(P/R) attain the oracle rate; miscalibration or a missing feature that breaks this measurability gives I sc < I ⋆ . The operational Ibauc (slope of log(1−AUROC)) is the decay rate of a two-sample rank statistic on 2m observations; it is not in general bounded by I sc and can exceed it (e.g. I auc = 2 I sc for Ber(.25)/Ber(.75) scores). What Prop. 4 bounds is the one-sample mean-threshold exponent, I mean ≤ I sc ≤ I ⋆ ; we do not claim Ibauc = I sc , and its exponential law is the empirical regularity of Section 6.

S25

Temperature Retunes

This appendix derives the magnitude of a temperature change’s effect on a visible-string audit and shows why IRIS’s insensitivity to it is the intended behavior. All quantities here are per single draw (per position); the per-response budget follows through Prop. 2 and is non-additive in length. Throughout we hold the served weights fixed, so the perposition logit vector z is the same at both temperatures—this is exactly what separates a temperature retune from a model change. Proposition 5 (Second-order retunes). Hold the served logits z ∈ Rn (n < ∞) fixed and temper at inverse temperaP ture β = 1/T , pβi ∝ eβzi . With A(β) = log i eβzi and V = A′′ (β0 ) > 0, a small gap ∆β = β1 − β0 gives I ⋆ = 18 (∆β)2 V + O(∆β 3 ),

KL(pβ0 ∥pβ1 ) = 12 (∆β)2 V + O(∆β 3 ), χ2 (pβ1 ∥pβ0 ) = (∆β)2 V + O(∆β 3 ).

Thus a pure temperature retune has only second-order separation in ∆β; the companion identity is VarpT (z) = T 3 dH/dT ≥ 0.

Resolving a temperature retune therefore needs m = Θ((∆β)−2 ) draws, while a base-model change perturbs z itself and can supply first-order evidence. The expansion is an interior statement; at the greedy boundary V → 0, and support collapse is the relevant observable (Cor. 1). The derivation, summarized next, is the basis for Eqs. (7)–(8).

(a)

(b)

on-family vs. off-family move

separation order

1.0

backend swap ∼ δ temperature ∼ (∆β)2

pz0 (backend swap)

off family: I ? ∼ δ temperature: I ? ∼ (∆β)2

family {pβ ∝ eβz }

pβ0 (T0 )

per-draw separation I ?

S24

0.8 0.6 0.4 0.2 0.0 0.0

0.2

0.4

0.6

perturbation magnitude

0.8

1.0

Figure 13: Why a temperature retune is second order while a backend substitution is first order. (a) With the logits z fixed, the tempered laws pβ ∝ eβz trace a one-parameter curve; a temperature change moves β along it (rank-one, on-family), whereas a model or backend change perturbs z and leaves the curve. (b) An on-family step separates only at second order, I ⋆ ≈ 81 (∆β)2 V , flat at the origin, so resolving it costs m = Θ((∆β)−2 ) draws; an off-family perturbation can separate at first order in its magnitude. IRIS therefore waives a temperature retune and fires on a separating backend substitution. Tempering family. Decoding at temperature T draws token i with pTi = softmax(z/T )i ∝ ezi /T . In the inverse temperature β = 1/T this is the exponential family pβi ∝ eβzi with natural parameter β, sufficient statistic P the logit (“energy”) z, and log-partition A(β) = log i eβzi . Its cumulants are A′ (β) = Epβ [z] and A′′ (β) = Varpβ (z), the Fisher information of the family. A temperature change moves β along this curve; a model change moves the vector z itself, generically off it (Fig. 13). Second-order gap. For a small gap ∆β = β1 − β0 every standard divergence collapses to the same Fisher quadratic (directional asymmetry enters only at O(∆β 3 )): KL ≈ 12 (∆β)2 V,

χ2 ≈ (∆β)2 V,

I ⋆ ≈ 18 (∆β)2 V,

with V := Varpβ0 (z) and the Chernoff information I ⋆ the Bhattacharyya value at the optimal tilt λ⋆ = 21 . (The χ2 follows from the exp-family identity χ2 (pβ1 ∥pβ0 ) = eA(2β1 −β0 )−2A(β1 )+A(β0 ) − 1, whose exponent has zero value, zero first derivative, and second derivative 2A′′ (β0 ) at β1 =β0 .) Converting to the temperature gap, ∆β = 1/T1 − 1/T0 ≈ −∆T /T02 , gives (∆T )2 dH +· · · , 8 T0 dT T0 (7) using the identity below. Temperature thus separates only at second order in the gap, I ⋆ = Θ((∆β)2 ), so an audit needs m = Θ((∆β)−2 ) independent draws to resolve a retune. I ⋆ (T0 , T1 ) = 81 (∆β)2 V +O(∆β 3 ) =

Heat capacity. The tempered logit fluctuation equals an entropy slope. With H(T ) the Shannon entropy of pT , the relation H = A(β) − βA′ (β) gives dH/dβ = −βA′′ (β), and β = 1/T (dβ/dT = −1/T 2 ) yields VarpT (z) = T 3

dH ≥ 0, dT

(8)

nonnegative because output entropy rises with temperature; it is the “heat capacity” of the next-token distribution. Crucially V is the variance of z under the tempered law pT , not the spread of the raw logit vector: the softmax de-weights the large-magnitude tail, so V is bounded by the entropy slope and is O(1) nat2 near T ≈1 (for plausible next-token profiles V ≈ 1–5), not the O(10–100) a raw logit range might suggest. Numeric. Between T0 =1 and T1 =1.5, ∆β = − 13 and I ⋆ ≈ V /72 ≈ 0.01–0.07 nat√per draw. A Chernoff-todetectability heuristic (d′ ≈ 2 I ⋆ , AUROC ≈ Φ(d′ /2)) maps this to single-response AUROC ≈ 0.57–0.59, matching the measured ≈ 0.58 between adjacent operating temperatures (App. S10); the per-position rate does not multiply by response length L, since AR positions are correlated and share the backbone z (Prop. 2). Boundary case. Eq. (7) governs the interior (T ≳ 0.5, non-degenerate pT ). At the greedy boundary T → 0, β → ∞ and V → 0 (point mass), so the per-draw rate vanishes (Cor. 1)—yet T =0 vs. T >0 is easily distinguishable (AUROC ≈ 0.99) by a different, single-draw observable: the support/entropy collapse (greedy repeats one string). The two mechanisms act at different sample sizes and do not conflict. Sampler retunes. A dilution audit asks whether the backend model changed, not whether a scalar sampler knob moved. Tempering is a rank-one, on-family move and is forced to be second order; a model change is instead an offfamily perturbation of z. Thus IRIS can waive a temperature retune while catching a separating substitute. The operational caveat is that pairs with too little visible-string separation require substantially larger budgets and fall outside the deployed margin-check regime (Thm. 1(c), App. S26).

S26

Tail Phase Boundary

Theorem 1(c) shows the regime is set by whether a separating tail exists. We make the interpolation between 1/ϵ and 1/ϵ2 explicit through a tail-separation function. Definition 2 (Tail separation). For reference distribution P with score CDF FP , the tail-separation function of a substi tute distribution R is qα (R) = Py∼R s(y) > FP−1 (1 − α) , i.e. the tell rate at the per-query false-positive level α. Proposition 6 (Tail budget). Suppose FP is atomless and qα ≍ c ακ as α → 0 for a tail exponent κ ∈ [0, 1) and c > 0, and the auditor controls type-I at level α via the union bound (per-query level α/m). Then, with α, δ, c, κ fixed and ϵ → 0, the detection budget is m⋆ = Θ ϵ−1/(1−κ) , 1/(1−κ) m⋆ ≍ ln(1/δ) ϵ−1/(1−κ) . c ακ

Thus κ = 0 gives the 1/ϵ law, κ = 12 gives ϵ−2 scaling, and

κ ↑ 1 is the no-tail wall. The κ = 21 exponent matches the aggregate mean-shift budget of Thm. 1(c), although the tests and constants are different.

We measure qα directly on the reference endpoint qwen3:8b@T =1 over a separation ladder (probe c2,16 ), and re-measure on scale-invariant (length/format-ablated, 179→144) features. The tail exponent κ (the log–log slope of qα in α) orders the ladder monotonically and the ordering survives ablation (Table 22): distinct/same-family substitutes have κ < 12 (near the 1/ϵ law, exact only at κ=0), temperature twins sit near the κ= 21 crossover that Prop. 6 maps to 1/ϵ2 , and near-greedy twins have κ → 1 (no tail; m⋆ =ϵ−1/(1−κ) diverges, an effective wall). We report the q-controlled continuum and the exponent κ, not a fitted ϵ−2 slope. When κ=0. Exact 1/ϵ (κ=0, qα → q0 > 0 as α → 0) requires the substitute to retain positive mass where the reference’s score is in its extreme upper tail—in the population, a region P makes vanishingly unlikely yet R does not. For samplers over a shared vocabulary at finite temperature, where every visible string has positive probability under both, the population qα generically vanishes as α → 0 (κ > 0): a strict κ=0 would need a genuine support difference or a singular component, not merely two distinct models. An empirical q0 > 0 estimated from a finite reference can instead reflect empirical zero-probability regions a high-capacity classifier mistakes for true support separation. The discrete score (atoms from short strings and from the forest’s predict_proba) further makes the atomless hypothesis of Prop. 6 idealizing: above the top atom q can drop to 0, and with atom-randomization a bounded likelihood ratio gives qα = O(α), i.e. κ=1. We therefore read the measured κ < 21 as a finite-ϵ, finite-sample statement and test its stability: re-estimating κ as the reference calibration set grows from n=50 to n=3000 (App. S4) shows whether κ settles at a positive value or moves toward 1. The 1/ϵ regime is thus reported as the measured behavior over the audited ϵ range, never an asymptotic law. Table 22: Tail-separation ladder on c2,16 (ref qwen3:8b@T 1): plateau q0 =qα=.01 and tail exponent κ, full → scale-invariant (length-ablated) features. The κ-ordering survives ablation; exact 1/ϵ holds only at κ=0 and κ= 12 is the 1/ϵ2 crossover (Prop. 6). substitute

q0 full→abl κ full→abl

qwen3:0.6b qwen3:32b qwen3:14b 8b@T 2 8b@T 1.5 8b@T 0.5 8b@T 0

.58 → .33 .60 → .45 .87 → .56 .08 → .07 .08 → .04 .01 → .01 .00 → .00

.14 → .31 .13 → .22 .03 → .18 .48 → .60 .22 → .57 .81 → .97 −

regime near-1/ϵ (κ< 12 ) near-1/ϵ (κ< 21 ) near-1/ϵ (κ< 21 ) near 1/ϵ2 (κ≈ 21 ) interpolating wall (κ→1) wall (κ→1)

The same continuum read off a second probe (Fig. 15): the plateau tell rate q slides smoothly from same-model temperature twins (at the floor α, undetectable) through same-family sizes to a distinct cross-family model (q=1). When the null rate is pushed to α1 → 0 (a separating tail), the budget is the any-tell 1/ϵ law m⋆ = ln(1/δ)/(ϵq) with constant set by q. At a fixed non-zero null p0 , however, this is a different

(a) commercial swaps with audit signal

(b) margin-checked swaps plus stress test

1.0

undetectable within 256

0.8

0.4

0.2

0.7

AUROC-rank power

routing fraction

0.8

0.1

queries to detect (≥ 0.95)

0.9 102

101

100

0.6 0.05 1

audit queries m

128

256

0.5

1/ 1/2 distinct models (commercial) supplementary stress test (same model, ±0.1 temp)

10−1

routing fraction

Figure 14: Dilution (referenced from Section 6). (a) AUROCrank diagnostic power over (ϵ, m) for commercial basemodel dilutions with enough enrollment signal. (b) Queries to detect vs. ϵ: dilutions passing the margin check track the near-1/ϵ wall, while the qwen3:8b T =1.0-vs-T =0.9 same-model supplementary stress test (c2,16 ) is undetectable within 256 queries (per pair, not median). The pre-fixed-FPR validation is in App. S4. (a) q: near-twin → distinct

(b) one 1/ law, constant set by q distinct (q=1.0) same-family (q=0.32) near-twin (q=0.16)

qwen3:0.6b

budget m? = ln(1/δ)/(q)

gpt-oss:20b 8b @T2.0 qwen3:32b qwen3:14b 8b @T1.5 8b @T0.5

102

101

α (floor)

8b @T0 (greedy)

0.0

0.2

0.4

0.6

per-query tell rate q

0.8

1.0

10−1

routing fraction

Figure 15: The tell-rate continuum on c100,1 (ref qwen3:8b@T =1). Here q is the per-query tell rate Py∼R [s(y)>τ ] and α the per-query false-positive floor. (a) Per-query tell rate q per substitute, ascending; bars below the false-positive floor α (greedy and near-greedy twins) are undetectable. (b) The unified budget m⋆ = ln(1/δ)/(ϵq) for three representatives—one 1/ϵ law on log–log axes, vertically offset by q. Distinct models sit lowest (cheapest), lowseparation variants highest.

is the typical behavior once the margin check passes—the implied budget exponent 1/(1 − κ) has median 1.02–1.10, and 62–85% of such pairs are statistically consistent with κ=0. A right tail remains (2–9% with κ ≥ 0.4), so IRIS budgets against the per-pair κ read off calibration rather than assuming a universal κ=0.

S27

Artifact Manifest and Infrastructure

We release, as a single reproducibility bundle: (i) all 147,070 frozen response records as JSON Lines, covering the controlled Ollama pools, the 17- and 45-endpoint gateway pools, and the reported robustness and negative-control analyses; (ii) the exact probe prompts and deterministic parser; (iii) all 179 visible-string feature definitions with the content-only and length-ablated subsets; (iv) the deterministic stratifiedsplit procedures and fixed seeds used by the analysis code; and (v) the code for response collection, feature extraction, the IRIS estimate-then-budget loop, dilution simulation, atscale and live dilution auditing, and figure generation. The bundle is available at https://github.com/Photen/IRIS-audit. Local models ran under Ollama 0.12.5 on Ubuntu 22.04.4 LTS with an Intel Xeon Platinum 8358 CPU, 2 TiB system memory, and six NVIDIA A40 GPUs (46–49 GiB reported per GPU; driver 550.54.15); the 17-API set was queried through a commercial OpenAI-compatible gateway. Analysis used scikit-learn random forests (300 trees, √ max_features = ·, balanced classes) under a fixed integer seed (20260617); reported curves average 40 stratified splits with 95% bootstrap intervals. Classifier posteriors P̂ (· | y) are random-forest predict_proba outputs clipped to [10−12 , 1] and used as a score, not as calibrated probabilities; every operating threshold and budget is set empirically on held-out calibration data, so posterior calibration is not assumed. Software: Python 3.12, scikit-learn 1.5.1, NumPy 1.26.4, and SciPy 1.13.1. No weights, logits, or token ranks were accessed.

test: the any-tell formula ignores the honest tells arriving at rate p0 and would not control type-I, so the deployed twoproportion test instead has local signal ϵ(q − p0 ) and budget m⋆ = Θ p0 (1 − p0 )/(ϵ2 (q − p0 )2 ) (Prop. 2 proof). The ϵ−2 wall is thus not the q → α limit of the any-tell law (which would give the finite ln(1/δ)/(ϵα)); it is the distinct fixed-null regime that happens to share the ϵ−2 exponent, the two budgets being endpoints of the measured-κ continuum m⋆ =Θ(ϵ−1/(1−κ) ) (App. S26).

This section collects a rigorous, self-contained proof of every formal claim in the paper—the propositions, the theorem, and the corollaries of Section 5, together with the two analytic results stated in the appendix (the temperature second-order law, App. S25, and the multi-diluent identifiability claim, App. S6). Each subsection restates what is proved, lists the standing assumptions, and gives the argument.

S26.1

S28.1

Tail Stability

Table 22 measures κ on one reference (qwen3:8b@T 1) and one probe. To test whether the 1/ϵ regime is a property of that single calibration point or holds across the library, we recompute the tail exponent κ (the log–log slope of qα in α, as in Table 22) for every ordered (reference, substitute) pair in the 17- and 45-endpoint gateways, on content-only features, averaging over four out-of-fold classifier/split seeds. Table 23 reports the distribution over pairs passing the enrollment margin check (plateau q0 ≥ 0.15). The picture is consistent and quantifies the headline’s scope: the 1/ϵ regime (κ ≈ 0)

S28

Proofs

Proof of Prop. 1 (mean score)

Restatement. Fix a probe c and let P, R be the reference and substitute response laws. The deployed claim is: if the bounded score s(y) P = − log P̂(P | y) has EP [s] ̸= 1 ER [s], then Sm = m i s(yi ) has a fixed-threshold error −m I mean (1+o(1)) mean e with I > 0 and I mean ≤ I sc ≤ I ⋆ . The comparison facts used around the proposition are: ⋆ (1) the full-string Bayes test has error e−I m(1+o(1)) and ⋆ I > 0 ⇐⇒ P ̸= R; (2) K-endpoint separation is guaran⋆ teed by m ≳ log(K/δ)/Imin .

Table 23: Tail-exponent κ distribution over ordered (reference, substitute) pairs passing the margin check of the 17- and 45endpoint gateways (content-only features, four seeds). “cons. κ=0” is the fraction of margin-qualified pairs statistically consistent with κ=0 (the exact 1/ϵ law); “1/(1−κ)” is the implied budget exponent; “twin” the fraction with κ ≥ 0.4. pool probe 17 17 17 45

sep.

c10,8 248/272 c2,16 164/272 c100,1 176/272 c10,8 1544/1980

κ med. [25,75] cons. κ=0 1/(1−κ) med. twin .02 [.0, .17] .08 [.01, .18] .02 [.0, .08] .09 [.01, .26]

Throughout, queries are independent, so under each fixed P hypothesis y1 , . . . , ym are i.i.d. Write ψ(λ) = log y P (y)1−λ R(y)λ and λ⋆ = arg min[0,1] ψ, with I ⋆ = −ψ(λ⋆ ) < ∞. Part (1). Achievability. For any λ ∈ [0, 1] the likelihoodratio (Bayes) test between P ⊗m and R⊗m with prior (πP , πR ) obeys X 1−λ λ πP P ⊗m πR R⊗m Pe (m) ≤ y1:m

≤

P (y)1−λ R(y)λ

= emψ(λ) ,

λ ≤ 1 and tensorisation over the m independent using πP1−λ πR coordinates. At λ = λ⋆ ,

Pe (m) ≤ e−mI

⋆

(∀m),

(9)

a genuine finite-m bound (the Bhattacharyya statement) and the ≤ half of the asymptotic law. R(y) Converse. Let ℓ(y) = log P (y) be the log-likelihood ratio. Under the regularity hypothesis that the cumulant generating function ΛP (λ) = log EP [eλℓ ] = ψ(λ) is finite on an open interval containing [0, 1] (automatic when the per-probe alphabet is finite, |supp| = n < ∞, the regime of all single-draw probes) and ℓ is non-degenerate, the Chernoff-information theorem (method of types in the finitealphabet case, Cover and Thomas (2006, Thm. 11.9.1)) gives ⋆ ⋆ Pe (m) ≥ e−mI (1+o(1)) . With (9), Pe (m) e−I m(1+o(1)) . P =1−λ Positivity. By Hölder, Rλ ≤ yP P P 1−λ λ ⋆ ( y P) ( y R) = 1, so ψ ≤ 0 and I ≥ 0, with ψ ≡ 0 when P = R. If P ̸= R then ℓ is non-degenerate under P , so ψ (its cumulant generating function, CGF) is strictly convex; together with ψ(0) = ψ(1) = 0 this forces ψ(λ) < 0 on (0, 1), hence I ⋆ > 0. The argument needs no common support: on atoms with P (y) = 0 < R(y) the summand vanishes for λ ∈ (0, 1), only deepening the strict inequality. Thus I ⋆ > 0 ⇐⇒ P ̸= R. □

Part (2). Let {Mi }K i=1 have laws {Pi } and prior πi ≥ πmin > 0. The MAP classifier errs only if some wrong j has joint posterior at least that of the true i; hence X X P(err) ≤ πi PP ⊗m πj Pj⊗m ≥ πi Pi⊗m i

≤

j̸=i

XX i̸=j

1−λ⋆ ij

λ⋆

Pj ij

.77 .74 .85 .62

1.02 1.09 1.02 1.10

.06 .02 .02 .09

where each inner term is the Bhattacharyya/Chernoff bound (9) for the ordered pair (i, j) (the prior ratio πj /πi is absorbed by the same π 1−λ π λ ≤ 1 step). Bounding each by ⋆ ⋆ e−mImin with Imin = mini̸=j I ⋆ (c; Mi , Mj ) > 0 (all endpoints distinct) gives ⋆

⋆

P(err) ≤ K(K − 1) e−mImin ≤ K 2 e−mImin .

Requiring the right side ≤ δ holds once m ≥ (2 log K + ⋆ ⋆ log(1/δ))/Imin , i.e. m ≳ log(K/δ)/Imin . This is a union bound layered on top of the binary exponent of Cover and Thomas (2006, Thm. 11.9.1), not that theorem alone. □ Part (3). Regularity. With posteriors clipped to [ϵ0 , 1−ϵ0 ], s ∈ [− log(1 − ϵ0 ), log(1/ϵ0 )] is bounded, so the CGFs ΛP (t) = log EP [ets ] and ΛR (t) = log ER [ets ] are finite for all t ∈ R, in particular near 0, which is the hypothesis of Cramér’s theorem. (Without clipping, EP [ets ] = EP [P̂(P | y)−t ] may diverge for every t > 0 and the law can fail, so boundedness is the regularity condition we assume.) Exponential law of Sm . Let µP = EP [s] < µR = ER [s] (relabel otherwise). By Cramér, for fixed τ ∈ (µP , µR ), ∗ 1 m log PP (Sm ≥ τ ) → −IP (τ ), ∗ 1 m log PR (Sm < τ ) → −IR (τ ),

∗ with IP∗ (x) = supt {tx − ΛP (t)}, IR (x) = supt {tx − ΛR (t)} the Legendre–Fenchel transforms. Both are convex, non-negative, vanish only at the respective means; ∗ on (µP , µR ), IP∗ increases and IR decreases, crossing at † a unique τ . The Bayes (or balanced) error of the meanthreshold rule decays at

I mean =

max

τ ∈(µP ,µR )

∗ min{IP∗ (τ ), IR (τ )}

∗ = IP∗ (τ † ) = IR (τ † ) > 0,

(10)

the maximum attained at the unique crossing τ † (where moving τ either way lowers the smaller exponent), establishing the exponential verification law for Sm at rate I mean . Optimality among sum-based tests (corrected identification). The empirical mean Sm is the one-dimensional sufficient statistic of the i.i.d. sample for the exponential family {pt (y) ∝ P (y)ets(y) }t∈R generated by tilting in s. Consequently every P test that depends on (y1 , . . . , ym ) only through the sum i s(yi ) is a (possibly randomized) function of Sm , and since the threshold rule is the likelihood-ratio test among functions of the 1-D statistic Sm (monotone likelihood-ratio (LR) under the tilt family), its best Bayes exponent is exactly

(10), attained by thresholding Sm at τ † . (We do not claim a common tilt equating the two distinct score-CGF derivatives; the exponent is the crossing value (10), not a Chernoff value of the two-sided tilt.) Comparison with I sc . Let µP , µR now denote the score push-forward of Def. 1, with I sc = R 1−λlaws λ − min0≤λ≤1 log µP µR the Chernoff information of the full scalar score, attained by the score-level likelihood-ratio test (LRT) on log(µR /µP ). The mean-threshold test is one particular non-LRT function of the scores, so by optimality of the LRT (Neyman–Pearson at the score level), I

mean

≤I

⋆

≤I ,

the second inequality being Prop. 4 (data processing under y 7→ s). Equality I mean = I sc holds iff thresholding Sm coincides with the score-LRT, i.e. iff s(y) = a + b log µµPR (s(y)) (s(y)) for constants a, b (an affine reparametrisation of the scorelevel log-LR). Strictness. The inequality is generically strict. With scores in {−1, 0, 1} and µP = (0.45, 0.10, 0.45),

µR = (0.05, 0.90, 0.05),

one has EP [s] = ER [s] = 0, so Sm → 0 a.s. unmean der both hypotheses = 0, while I sc = P 1−λ λand I − minλ log s µP µR = 0.511 (at λ⋆ = 12 ). Hence 0 = I mean < I sc = 0.511. Even restricting to score laws with unequal means, I mean /I sc can be as small as ≈ 10−3 , so the gap is not a measure-zero artefact; and on a monotoneLR example with s affine in log(µR /µP ) the two coincide (I mean ≈ I sc ≈ 0.236, up to grid resolution), confirming the equality condition. Hence the mean-statistic exponent is I mean ≤ I sc , strict in general and equal only in the affine case. □ Remark. In summary: (i) a positive oracle exponent I ⋆ exists iff P ̸= R; (ii) K-way separation costs m ≳ ⋆ log(K/δ)/Imin ; and (iii) IRIS’s mean statistic Sm decays exponentially in m at the bounded rate I mean ≤ I sc ≤ I ⋆ . The separately-fit ranking rate I auc of Def. 1 (slope of log(1 − AUROC)) is a two-sample quantity on 2m observations and is not bounded by I sc in general—it can be as large as 2 I sc (e.g. Ber(.25)/Ber(.75) scores, where I auc = 2D(0.5∥0.25) = 0.288 while I sc = 0.144)—so it is the empirically-validated budgeting rate, not a proven lower bound.

S28.2

Proof of Prop. 2 (length)

Restatement. Fix a probe c and letPIseq (L) := C P (· | c), R(· | c) = − min0≤λ≤1 log y1:L P (y1:L | c)1−λ R(y1:L | c)λ be the Chernoff information carried by one full length-L response under (16). Then (i) Iseq is nondecreasing in L; (ii) it admits no per-position conditionalsum (“chain”) decomposition and Iseq (L) ̸= L Iseq (1) in general (with the gap possibly of either sign); (iii) across m independent queries the exponent is exactly additive, C(P ⊗m , R⊗m ) = m Iseq (L). The empirical nonmonotonicity in L is a property of the operational exponents I sc , I auc and the yield q(L)/L, not of Iseq (L).

Assumptions. A fixed probe c; all per-position alphabets finite (or countable with Iseq (L) < ∞), so the Hellinger integral and the min over λ ∈ [0, 1] are well defined; cross-query independence (independent API calls). No boundedness P of Chernoff by log n is used. Write ψP,Q (λ) = log y P (y)1−λ Q(y)λ and fλ (P, Q) = P 1−λ Q(y)λ , so C(P, Q) = − min0≤λ≤1 ψP,Q (λ). y P (y) Lemma 1 (Chernoff data processing). For any Markov kernel W (· | y) with pushforwards P̃ , Q̃, C(P̃ , Q̃) ≤ C(P, Q).

Proof. Fix λ ∈ (0, 1). For each output z, Hölder’s inequality 1 with exponents 1−λ , λ1 applied to ay = P (y)W (z | y), by = Q(y)W (z | y) gives X 1−λ X λ X P̃ (z)1−λ Q̃(z)λ = ay by ≥ a1−λ bλy y y

X y

1−λ

P (y)

Q(y) W (z | y),

since W (z | y)1−λ W (z | y)λ = W (z | y). Summing over z P and using z W (z | y) = 1 yields fλ (P̃ , Q̃) ≥ fλ (P, Q), hence ψP̃ ,Q̃ (λ) ≥ ψP,Q (λ). The endpoints λ ∈ {0, 1} give ψ = 0 on both sides. Taking minλ and negating, C(P̃ , Q̃) ≤ C(P, Q). (i) Monotonicity. Truncation T : y1:L 7→ y1:L−1 is a deterministic kernel whose pushforwards of P, R are the length-(L − 1) marginals (again AR responses under (16)). Lemma 1 gives Iseq (L − 1) ≤ Iseq (L). Also C ≥ 0: at λ = 0, ψ = 0, so minλ ψ ≤ 0. Thus Iseq is non-negative and non-decreasing; the oracle exponent of the full response cannot fall as L grows. (ii) No chain rule. The factorization (16) gives the PL KL chain rule KL(P ∥R) = t=1 Ey<t ∼P [KL(P (· | y<t )∥R(· | y<t ))], a property of KL alone. Since ψP,R is convex in λ with ψ(0) = ψ(1) = 0 and (when the relevant KL terms are finite) ψ ′ (0+ ) = −KL(P ∥R), ψ ′ (1− ) = KL(R∥P ), one gets the regularity-light sandwich 0 ≤ C(P, R) ≤ min{KL(P ∥R), KL(R∥P )}

(⋆)

(both bounds vacuously valid if a KL is +∞). Chernoff is thus only bounded by, never equal to, the conditional sum. Concretely, on {0, 1}2 let P = 12 δ(0,0) + 21 δ(1,1) and R be the product law with marginal (0.9, 0.1). Then Iseq (2) = C(P, R) ≈ 0.347 nats, the per-position value Iseq (1) = C((.5, .5), (.9, .1)) ≈ 0.112, the naive “Chernoff chain” (first-symbol Chernoff plus the P -average of conditional-symbol Chernoffs) ≈ 1.316 ̸= 0.347, while the genuine KL chain matches to machine precision (1.7148 = 1.7148). So Chernoff information does not decompose into a sum of per-position terms. The example also gives Iseq (2) ≈ 0.347 > 2 Iseq (1) ≈ 0.225 (super-additivity under positive within-response association), whereas in the i.i.d. product case P = p⊗L , R = r⊗L one has fλ (P, R) = fλ (p, r)L , a shared minimizer λ⋆ , and C = L C(p, r). So the safe statement is non-equality, with the gap of either sign and no sub-additivity claim.

(iii) Accumulation in m. The m queries are independent, so the transcript law is P ⊗m vs R⊗m over i.i.d. length-L responses. The product identity (now across the query index, where independence genuinely holds) gives fλ (P ⊗m , R⊗m ) = fλ (P, R)m , hence C(P ⊗m , R⊗m ) = m Iseq (L), and by Prop. 1 the Bayes error decays as e−m Iseq (L)(1+o(1)) . The exponent grows linearly and exactly in m, while within one response the length contribution is the bounded, non-multiplicative Iseq (L) ≤ min{KL(P ∥R), KL(R∥P )} of (i)–(ii). A length-L response is therefore not L independent draws.

q ≥ α1 , so p′ ≥ p ≥ ϵq and the same bound holds uniformly over routing fractions ≥ ϵ. Setting e−ϵqm ≤ δ gives m ≥ ln(1/δ)/(ϵq), i.e. m⋆ = ⌈ln(1/δ)/(ϵq)⌉ guarantees power ≥ 1 − δ.

Observed non-monotonicity. By (i), Iseq (L) cannot decrease in L, so the measured non-monotonicity is not about it. IRIS uses a fixed score s(y) = − log P̂(P | y) rather than the length-L LRT; its score/plug-in exponents I sc , I auc (Def. 1) carry no data-processing monotonicity in L and may rise then fall as the classifier saturates, and the per-token yield q(L)/L of Cor. 2 is non-monotone by construction (concave q gives a decreasing yield; an S-shaped q from a global statistic gives an interior peak). These operational quantities, with prompt-regime shifts of the served distribution, are where the non-monotonicity lives. Crucially, the L-comparison in Section 6 varies the prompt’s requested length, so successive conditions are different contexts c—not the truncation of one fixed length-L response that part (i) governs—and the first 16 symbols of a “generate 32” response need not share the marginal of a “generate 16” response. The non-monotonicity is therefore an operational property across prompt-length conditions, consistent with but not predicted by the samecontext monotonicity of part (i).

the inequality being 1 − (1 − x)m ≤ mx for x ∈ [0, 1] (convexity of x 7→ (1 − x)m , equivalently Bonferroni). Choosing τ with α1 (τ ) ≤ α/m bounds the type-I error by α. Joint operating point. Parts (a) and (b) share one τ , and (a)’s budget m⋆ = ln(1/δ)/(ϵq(τ )) is exactly the m that (b) must control. Substituting into α1 (τ ) ≤ α/m⋆ gives the feasibility condition

S28.3

Proof of Thm. 1 (dilution)

Throughout, the m queries are i.i.d.: query i is routed to the substitute distribution R with probability ϵ independently across i, and otherwise to the reference distribution P , so iid yi ∼ Qϵ = (1 − ϵ)P + ϵR. The deployed score s(y) and the level τ are fixed before the audit (no data-dependent threshold). Write α1 = α1 (τ ) = PP (s > τ ) and q = q(τ ) = PR (s > τ ), and define the per-query crossing event Ei = {s(yi ) > τ } with per-query crossing probability under Qϵ p := PQϵ (s > τ ) = (1 − ϵ)α1 + ϵq.

The events E1 , . . . , Em are i.i.d. Bernoulli(p).

(a) First crossing. The any-tell rule flags iff at least one Ei occurs, soQ the miss probability over m queries is exactly m P(no hit) = i=1 P(Eic ) = (1 − p)m . Since α1 ≥ 0 we have p ≥ ϵq, hence 1 − p ≤ 1 − ϵq and P(no hit in m) = (1 − p)

≤ (1 − ϵq)

−ϵq m

≤e

the last step from 1 − x ≤ e−x . Equivalently the firsthit time N = min{i : Ei } is Geom(p) and, because p ≥ ϵq, is stochastically dominated by a Geom(ϵq) variable; the resulting (1 − ϵq)m is therefore a lower bound on the power 1 − P(N > m) (the bound is conservative, in the safe direction). For a suspect with routing fraction ϵ′ ≥ ϵ, p′ = α1 + ϵ′ (q − α1 ) is nondecreasing in ϵ′ whenever

(b) Type-I error. On an honest endpoint ϵ = 0, so the Ei are i.i.d. Bernoulli(α1 ) and the false-positive probability over m queries is exactly m [ P Ei = 1 − (1 − α1 )m ≤ m α1 , i=1

α1 (τ ⋆ ) ≤

α α ϵ q(τ ⋆ ) = . m⋆ ln(1/δ)

Raising τ lowers α1 (good) but also lowers q (shrinking the RHS), so the existence of a feasible τ ⋆ is a nontrivial joint property of (P, R), not implied by (a) or (b) alone, and is verified on calibration data. (c) Two regimes. Regime (i): separating tail. Suppose there is τ with q(τ ) = Ω(1), α1 (τ ) → 0, and α1 (τ ) = o(ϵ). By (a) the power budget is m⋆ = ln(1/δ)/(ϵq(τ )) = Θ(ϵ−1 log(1/δ)). For a valid audit the feasibility condition of (b) must also hold at this τ : since q(τ ) = Θ(1) the RHS α ϵ q(τ )/ ln(1/δ) is Θ(ϵ), so feasibility holds precisely because α1 (τ ) = o(ϵ) (in particular when α1 is a fixed small constant the auditor enforces by pushing τ into the tail, where by the low-entropy sharpness of (16) Rtypical strings are P -improbable, so α1 collapses while q stays Ω(1)). Both the power and type-I requirements are then met at m⋆ = Θ(ϵ−1 log(1/δ)). The chi-square identity. Since Qϵ (y) − P (y) = ϵ R(y) − P (y) , χ2 (Qϵ ∥P ) =

X (Qϵ (y) − P (y))2 P (y)

= ϵ2

X (R(y) − P (y))2 y

P (y)

= ϵ2 χ2 (R∥P ),

valid whenever P (y) = 0 ⇒ R(y) = 0 (i.e. R ≪ P ); otherwise both sides are +∞ and the identity holds in [0, ∞]. This is the only divergence manipulated; no chain rule, additivity, or alphabet-size bound for Chernoff information is used anywhere. Regime (ii): low-separation local mixtures. Assume P ̸= R and the finite-variance condition V := χ2 (R∥P ) < ∞. We first bound the sample complexity of the optimal meanshift (linear-score) test of H0 : yi ∼ P against H1 : yi ∼ Qϵ from m i.i.d. responses in the local (ϵ → 0) regime; the

matching all-test floor below then shows the resulting ϵ−2 order is optimal over all tests, so it bounds what any auditor (IRIS included) can achieve. Introduce the locally-most-powerful (score) statistic T (y) :=

R(y) − 1, P (y)

ϵ the Fréchet derivative of log dQ dP = log(1 + ϵT ) at ϵ = 0. A direct computation gives X EP [T ] = (R − P ) = 0,

VarP (T ) = EP [T 2 ] =

X (R − P )2 y

= χ2 (R∥P ) = V, X EQϵ [T ] = (1 − ϵ) (R − P ) y

{z } X R +ϵ −1 R P |

= ϵ χ2 (R∥P ) = ϵV, so the per-sample squared standardized separation of T between P and Qϵ is (EQϵ [T ] − EP [T ])2 VarP (T ) 2 (ϵV ) = = ϵ2 V = ϵ2 χ2 (R∥P ) V = χ2 (Qϵ ∥P ),

d2 :=

using the identity above. (Optimality: for any score g, Cauchy–Schwarz in L2 (P ) gives (EQϵ [g] − EP [g])2 /VarP (g) = ⟨g−EP g, dQϵ /dP −1⟩2P /VarP (g) ≤ ∥dQϵ /dP − 1∥2L2 (P ) = χ2 (Qϵ ∥P ), with equality at g ∝ T . Thus T is optimal and χ2 (Qϵ ∥P ) is the largest achievable per-sample separation.) Regularity for the central limit theorem (CLT). Let T̄m = P 1 T (y ) and reject for T̄m > t. The score T is fixed (it i i m does not depend on ϵ); only the sampling law Qϵ changes with ϵ. We require a triangular-array CLT for {T (yi )}i≤m under Qϵ as ϵ → 0, m → ∞. A clean sufficient condition P (R−P )2 R < ∞ (slightly stronger than is ER [T 2 ] = y P2 χ2 (R∥P ) < ∞): then supϵ≤ϵ0 EQϵ [T 2 1{|T | > R}] → 0 as R → ∞ (uniform integrability of T 2 along the family, since Qϵ = (1−ϵ)P +ϵR is a convex combination of two laws each with finite E[T 2 ]), which yields p the Lindeberg condition and √ hence m(T̄m −EQϵ T )/ VarQϵ (T ) ⇒ N (0, 1) under Qϵ and the analogous statement under P . Moreover VarQϵ (T ) = V + O(ϵ) → V . p Sample complexity. A level-α test sets t = z1−α V /m (null variance V ). Its power at Qϵ is ≥ 1 − δ as soon as q p ϵV − z1−α V /m ≥ z1−δ VarQϵ (T )/m p = z1−δ V /m (1 + o(1)),

i.e.

√

√ m ϵ V ≥ (z1−α + z1−δ )(1 + o(1)), giving (z1−α + z1−δ )2 ϵ2 V (z1−α + z1−δ )2 = Θ(ϵ−2 ), = 1 + o(1) χ2 (Qϵ ∥P )

m⋆ = 1 + o(1)

the Θ being in ϵ at fixed (P, R); the hidden constant is 1/χ2 (R∥P ), which blows up as the pair approaches the twin wall χ2 (R∥P ) → 0 (so the budget is really Θ(ϵ−2 /χ2 (R∥P )), matching App. S26’s κ = 12 crossover). IRIS’s deployed test. P IRIS does not run the LMP test; it 1 thresholds Sm = m i s(yi ) with its fixed deployed score 2 s. Assume σP2 := VarP (s) < ∞, σR := VarR (s) < ∞, and ER [s] ̸= EP [s] (the analogous light-tail and nonzero-meanseparation conditions, logically distinct from χ2 < ∞). Then the mean shift is EQϵ [s]−EP [s] = ϵ(ER [s]−EP [s]) = Θ(ϵ) and VarQϵ (s) = σP2 + O(ϵ), so the same CLT/Neyman– Pearson argument gives 2 z1−α σP + z1−δ σP ⋆ −2 m = 1 + o(1) 2 = Θ(ϵ ), ϵ(ER [s] − EP [s])

with the model-dependent constant (z1−α σP + z1−δ σP )2 /(ER [s] − EP [s])2 in place of 1/χ2 (R∥P ). By the optimality (Cauchy–Schwarz) bound above, (EQϵ [s] − EP [s])2 /σP2 ≤ χ2 (Qϵ ∥P ), so this constant is ≥ 1/χ2 (R∥P ): IRIS shares the ϵ−2 scaling but pays a larger, empirically calibrated constant. □ All-test floor (matching lower bound). The ϵ−2 order is not an artifact of restricting to mean-shift tests; it is a floor over all level-α tests, by an elementary information-theoretic bound (no LAN machinery). For any test, the excess of its power at Qϵ over its size at P is at most the total vari⊗m ation TV(Q , P ⊗m ). Write the Bhattacharyya affinity P pϵ P (y)Qϵ (y) = 1 − H 2 (P, Qϵ ) with H 2 the A := y squared Hellinger distance; using H 2 ≤ 12 χ2 (Qϵ ∥P ) and the identity χ2 (Qϵ ∥P ) = ϵ2 χ2 (R∥P ) proved above, and the , P ⊗m ) = Am , tensorization A(Q⊗m ϵ A ≥ 1 − 21 ϵ2 χ2 (R∥P ),

Am ≥ 1 − 12 m ϵ2 χ2 (R∥P )

(the √ second bypBernoulli’s inequality), so from TV ≤ 1 − A2m ≤ 2(1 − Am ) applied to the product measures, p p TV(Q⊗m , P ⊗m ) ≤ 2 (1 − Am ) ≤ ϵ m χ2 (R∥P ). ϵ

A level-α test with power ≥ 1 − δ requires TV ≥ 1 − α − δ, hence (1 − α − δ)2 m ≥ 2 2 = Ω(ϵ−2 ). ϵ χ (R∥P )

So no auditor—linear-score or not—detects routing fraction ϵ in fewer than Ω(ϵ−2 ) queries once χ2 (R∥P ) < ∞: the low-separation wall is information-theoretic, matching the mean-shift upper bound up to the constant, and the meanshift test of this regime is therefore order-optimal. □

Remark (regularity). (1) The CLT requires finite second moments: χ2 (R∥P ) < ∞ (indeed ER [T 2 ] < ∞ for the uniform-integrability/Lindeberg step) for the optimal test, 2 and σP2 , σR < ∞ for IRIS’s test. These are genuine restrictions: χ2 (or Var(log P )) can be +∞ even when the Chernoff information I ⋆ is finite, in which case the normal approximation fails and one must use exact tail bounds; the ϵ−2 law is the light-tail case. (2) Non-degeneracy P ̸= R (so V > 0) is needed, else m⋆ = ∞. (3) Θ(ϵ−2 ) is the leading-orderin-ϵ scaling at fixed (P, R); uniformity over pairs fails near the twin wall, where the true budget is Θ(ϵ−2 /χ2 (R∥P )). (4) The ϵ−2 order is a floor over all tests (not just meanshift), established byp the all-test lower bound above (an elementary TV ≤ ϵ mχ2 Hellinger-tensorization argument, no LAN needed); the mean-shift test of this regime is hence order-optimal. (5) No additivity, chain rule, or alphabet-size bound for Chernoff information is invoked; the only divergence pushed around is χ2 , via the exact identity χ2 (Qϵ ∥P ) = ϵ2 χ2 (R∥P ).

S28.4

Proof of Prop. 2 (binomial guarantee)

The threshold τ , the null rate, and the budget m⋆ are fixedP from calibration before the audit, so the m⋆ tells k = i 1{s(yi ) > τ } are a sum of i.i.d. Bernoullis whose rate is α1 under P and pϵ′ = (1 − ϵ′ )α1 + ϵ′ q under Qϵ′ . (i) Type-I. The one-sided binomial test of H0 : rate ≤ α1 at level α rejects iff k ≥ kα (m), where kα (m) = min{k : PBin(m,α1 ) (K ≥ k) ≤ α}. By construction PBin(m,α1 ) (K ≥ kα (m)) ≤ α for every m, so under P the flag probability is ≤ α exactly—type-I is controlled by the test’s validity, with no union bound and no dependence on m. With α1 estimated, using a (1−γ) Clopper–Pearson upper bound α1 ≥ α1 as the null only raises kα (m), so the bound holds with probability ≥ 1 − γ over calibration. (ii) Power. Since q > α1 , pϵ′ = α1 + ϵ′ (q − α1 ) is strictly increasing in ϵ′ , so pϵ′ ≥ pϵ for ϵ′ ≥ ϵ, and the binomial right-tail PBin(m,p) (K ≥ kα (m)) is nondecreasing in the success rate p at fixed m; hence the power at any ϵ′ ≥ ϵ is at least the power at ϵ. The exact-test power is not monotone in m—the discrete critical value kα (m) jumps, so power can dip at a budget increment (e.g. at p0 =0.05, p1 =0.1, α=0.05 it falls from ≈ 0.150 at m=7 to ≈ 0.038 at m=8)—but PBin(m,pϵ ) (K ≥ kα (m)) → 1 as m → ∞. We therefore define m⋆ as the smallest budget for which power ≥ 1 − δ holds at every m ≥ m⋆ (a stable budget; equivalently, a randomized test restores exact monotonicity in m), giving power ≥ 1−δ for all ϵ′ ≥ ϵ at every budget ≥ m⋆ . Replacing q by q ≤ q in the sizing lowers pϵ , only increasing m⋆ , so power is preserved; a union bound over the two calibration events gives joint confidence 1 − 2γ. Regimes. Write ∆ = pϵ − α1 = ϵ(q − α1 ). Separating threshold (q = Ω(1), α1 → 0): the null rate → 0, so kα (m) = 1 for all moderate m and the test fires on the first tell—it is exactly the any-tell rule of Thm. 1(a), with m⋆ = ⌈ln(1/δ)/(ϵq)⌉ = Θ(ϵ−1 log(1/δ)). Near-twins (q → α1 , ∆ → 0): the p normal approximationp to the binomial gives m⋆ ≈ (z1−α α1 (1 − α1 ) + z1−δ pϵ (1 − pϵ ))2 /∆2 = Θ(∆−2 ) = Θ ϵ−2 (q − α1 )−2 , the ϵ−2 wall of Thm. 1(c).

The same i.i.d.-tell count underlies both, so a single test interpolates the two budgets. □

S28.5

Proof of Cor. 2 (length choice)

Assumptions. (A1) Integral split: L | B, so m = B/L ∈ Z>0 ; the continuous maximization in (ii) is the natural relaxation of this integer-constrained problem. (A2) I.i.d. queries: the m responses are independent, each drawn from Qϵ (· | L) = (1 − ϵ)P (· | L) + ϵR(· | L) with identical length-L laws (independent calls, independent ϵ-routing). (A3) Fixed rule per length: the score s and level τL are fixed, defining q(L), α1 (L) with q(L) > 0. No within-response positional independence is assumed: by the autoregressive law (16) the L positions of one response are dependent, which is exactly why q(L) is a measured primitive of L (Prop. 2), not 1 − (1 − q(1))L . Only crossquery independence (A2) is used; no Chernoff additivity or chain rule is invoked anywhere. Step 1 (tell probability). Fix L and let E = {s(y) > τL }. By (A2) and total probability, pϵ (L) = PQϵ (·|L) (E) = (1 − ϵ)α1 (L) + ϵq(L) ≥ ϵq(L), (∗) the inequality because α1 (L) ≥ 0. Dropping (1 − ϵ)α1 (L) only discards detections from reference responses that happen to cross τL , so it makes the subsequent bound conservative. Step 2 (miss bound). By (A2) the m = B/L events {E at query i} are i.i.d. Bernoulli(pϵ (L)), so “no crossing in m queries” has probability m B/L P(miss) = 1 − pϵ (L) = 1 − pϵ (L) ,

an equality in pϵ (L). Since t 7→ (1 − t)m is decreasing on [0, 1], (∗) gives P(miss) ≤ (1 − ϵq(L))B/L . Finally 1 − x ≤ e−x for all real x (here x = ϵq(L) ∈ [0, 1]) yields B/L P(miss) ≤ 1 − ϵq(L) ≤ e−ϵq(L) B/L = exp − ϵB q(L) . L

This is part (i): the displayed e−ϵq(L)B/L in the paper is the rightmost quantity, the result of two stacked relaxations ((∗) and 1 − x ≤ e−x ), hence an upper bound on the miss, not an equality. Setting the bound ≤ δ recovers m⋆ = ⌈ln(1/δ)/(ϵq)⌉ of Eq. (18) with m = B/L, confirming the corollary is a direct specialization of Theorem 1(a). Step 3 (optimal length). Fix ϵ, B. The bound factors through ρ(L) := q(L)/L as e−ϵBρ(L) . Because x 7→ e−ϵBx is strictly decreasing for ϵB > 0, q(L) arg min e−ϵB q(L)/L = arg max L: q(L)>0, L|B L: q(L)>0, L|B L =: L⋆ , independent of ϵ, B (they enter only through the L-free decreasing factor). This is the corollary’s claim, corrected to “minimizes the miss bound.”

Step 4 (exact optimum). Minimizing (1 − pϵ (L))B/L is, via the strictly decreasing transform − B1 log(·), equivalent to maximizing g(L) = − L1 log(1 − pϵ (L)). Using − log(1 − t) = t + 12 t2 + O(t3 ), p (L)2 pϵ (L) ϵ +O g(L) = L L p (L)2 q(L) α1 (L) ϵ =ϵ . + (1 − ϵ) +O L L L Thus arg maxL g(L) = arg maxL q(L)/L = L⋆ provided both pϵ (L) = o(1) (small per-query tell / small ϵ) and α1 (L) = o(ϵq(L)) (FPR controlled at level τL , Theorem 1(b)); without the latter the leading term keeps the (1 − ϵ)α1 (L)/L contribution and the optimizer maximizes pϵ (L)/L, not q(L)/L. When ϵq(L) = Θ(1) the 12 pϵ (L)2 /L term can shift the optimizer, so “maximize q(L)/L” is exact for the exponential bound for all ϵ, and for the exact geometric miss only to first order in pϵ . □ Remark (yield shape). If the tell is per-position, q(L) is concave with q(0) = 0, so ρ(L) = q(L)/L is nonincreasing (chord slope of a concave function through the origin) and L⋆ is the smallest admissible length. If the tell is a global statistic (χ2 , compression, n-gram entropy), q(L) is S-shaped (information-starved at small L) and ρ peaks at an interior L⋆ . Both are consistent with Step 3; which holds is empirical (Prop. 2, Sec. 6). No additivity of within-response information across positions is used.

S28.6

Proof of Prop. 1 (distributional separation)

Fix a single-draw probe with response alphabet Y, |Y| = n, and categorical laws p and r for the reference and substitute at the chosen temperature. P For λ ∈ [0, 1] write the Chernoff coefficient Cλ := y∈Y p(y)1−λ r(y)λ , so the per-draw Chernoff information is I ⋆ = − min0≤λ≤1 log Cλ . All logarithms are natural. Step 0 (alphabet caveat). Take n=2 and, for d ∈ (0, 12 ), p = (1 − d, d), r = (d, 1 − d). The swap symmetry r(y) = p(σy) gives Cλ = C1−λ , and λ 7→ log Cλ is convex (Hölder), so its minimiser is the fixed point λ⋆ = 12 . Hence Xp min Cλ = C1/2 = p(y)r(y) λ

p = 2 d(1 − d), p I ⋆ = − log 2 d(1 − d) −−−−→ +∞. + d→0

The crossover I ⋆ = log 2 occurs at the unique d0 ∈ (0, 12 ) √ p solving 2 d0 (1 − d0 ) = 12 , i.e. d0 = 12 1 − 23 ≈ 0.067; for every d < d0 the bound I ⋆ ≤ log n fails, e.g. at d = 0.01, I ⋆ ≈ 1.6145 > log 2 ≈ 0.6931. Since here H(p) = H(r) = Hb (d) → 0 while I ⋆ → ∞, this simultaneously refutes the variants I ⋆ ≤ log n and I ⋆ ≤ min(H(p), H(r)). (For disjoint supports Cλ = 0 on (0, 1) and I ⋆ = +∞.) Thus the conflation of low output entropy with low distinguishability is the conceptual error: a sharply biased coin whose bias differs between p and r is a low-entropy yet high-exponent probe.

Step 1 (MI cap). Let Θ ∈ {P, R} carry the uniform prior, with Y | Θ=P ∼ p and Y | Θ=R ∼ r, so Y has marginal p̄ = 21 (p + r). Then I(Θ; Y ) = H(Y ) − H(Y | Θ)

= H(p̄) − 12 H(p) + H(r) = 12 KL(p∥p̄) + 12 KL(r∥p̄) = JS(p, r),

the (uniform-prior) Jensen–Shannon divergence. Two elementary bounds give (1)

I(Θ; Y ) = H(Y ) − H(Y | Θ) (2)

(3)

≤ H(Y ) ≤ log |Y| = log n,

(4)

I(Θ; Y ) ≤ H(Θ) = log 2,

where (2) uses H(Y | Θ) ≥ 0 (discrete conditional entropy is nonnegative), (3) is the maximum-entropy bound on n atoms, and (4) is the symmetric bound I(Θ; Y ) ≤ H(Θ) with Θ binary and uniform. Hence a single Bayesian draw transmits at most min(log 2, log n) = log 2 nat about model identity—one bit, “which of two models”—so for n > 2 the operative cap is log 2, not the alphabet, and log n binds only at n=2 (a coin). This is the honest carrier of the “smallalphabet / low-entropy probe is weak” message — a bound on an averaged f -divergence, not on the error exponent. In the Step-0 family JS(p, r) ≤ log 2 for all d (e.g. 0.637 < log 2 at d = 0.01), consistent with the bound even where I ⋆ diverges: a sharp model-discriminating coin is a high-exponent but low-mutual-information probe. Step 2 (KL gating). For every λ ∈ [0, 1], apply Jensen to the concave map log under the law p: h i − log Cλ = − log Ep (r/p)λ h i ≤ −Ep λ log(r/p) = λ KL(p∥r),

and symmetrically (Jensen under r with exponent 1 − λ) − log Cλ ≤ (1 − λ) KL(r∥p). Since 0 ≤ λ ≤ 1, I ⋆ = max − log Cλ ≤ min KL(p∥r), 0≤λ≤1 KL(r∥p) .

In particular I ⋆ → 0 whenever p → r in KL: a probe is weak precisely when the two model laws are close, regardless of n. (This bound is sharp in spirit but not the log n statement: in Step 0, min(KL) = 4.50 nat at d = 0.01, large and diverging, so the KL bound does not carry the alphabet message — only the MI bound of Step 1 does.) Step 3 (temperature limit). Model temperature as inverse-temperature tempering of fixed logits zP , zR ∈ Rn (App. S25): for M ∈ {P, R}, pTM (y) = P ′ eβzM (y) / y′ eβzM (y ) , β = 1/T . Assume each arg max is ⋆ ⋆ zM (y) > 0, unique with gap ∆M := zM (yM ) − maxy̸=yM ⋆ yM := arg maxy zM (y).

⋆ (a) Point-mass degeneration. For y ̸= yM , ⋆ eβzM (y) −β(zM (yM )−zM (y)) ⋆ ) = e βz (y M M e ≤ e−β∆M −−−−→ 0,

pTM (y) ≤

β→∞

T ⋆ in total variation at geometric rate, H(p so pTM → δyM M) → 3 0, and by Eq. (8) VarpTM (zM ) = T dH/dT → 0. ⋆ (b) Same-mode branch yP⋆ = yR =: y ⋆ . Both laws share ⋆ the dominating atom y with mass → 1, so ∥pTP − pTR ∥TV → 0 and, by Step 2, 0 ≤ I ⋆ (T ) ≤ min KL(pTP ∥pTR ), KL(pTR ∥pTP ) −−−→ 0, T →0

with rate I ⋆ (T ) = O e−β min(∆P ,∆R ) (the off-atom contributions to either KL are O(e−β∆ ) and the on-atom log-ratio log(pTP (y ⋆ )/pTR (y ⋆ )) → log 1 = 0). The optimal exponent of the m-draw Bayes test is m I ⋆ (T ), so the increment contributed by the (m+1)-th draw is exactly I ⋆ (T ) → 0: additional independent queries asymptotically add no exponent. (c) Degenerate identical-draw statement (Cor. 1). At T = ⋆ 0 (greedy) the response equals the constant yM determinis⋆ tically, so y1 = · · · = ym = yM . On the same-mode branch the log-likelihood ratio between P and R of m identical greedy draws is m log p0P (y ⋆ )/p0R (y ⋆ ) = m log(1/1) = 0 and does not grow with m: repeating an identical draw contributes zero additional exponent, the per-additional-query rate is exactly 0, matching App. S1. ⋆ (honest caveat). Here (d) Distinct-mode branch yP⋆ ̸= yR the limits δyP⋆ , δyR⋆ have disjoint support, so Cλ → 0 for λ ∈ (0, 1) and I ⋆ (T ) → +∞: a single draw separates P from R almost surely (the support-collapse tell of App. S25). The two branches do not conflict because they concern different observables: part (ii)’s “rate vanishes” is the increment from repeating an already-observed identical draw (zero in both branches), whereas distinct-mode separation is delivered by the first draw, not by accumulation. The branch-independent invariant is therefore: under greedy decoding, repeating an already-observed identical draw yields no additional exponent, so any nonzero accumulation rate from independent repeats requires T > 0. ■ Conclusion. A ceiling holds on the per-draw mutual information, I(Θ; Y ) = JS(p, r) ≤ H(Θ) = log 2 (one bit of model identity, with log n a looser bound binding only at n=2; Step 1), but not on the Chernoff exponent I ⋆ (Step 0), which is instead gated by closeness, I ⋆ ≤ min(KL(p∥r), KL(r∥p)) (Step 2). For part (ii) (Step 3): as T → 0 each law degenerates to a point mass, and on the same-mode branch the per-additional-query exponent vanishes, with the strict T =0 greedy case giving exactly zero accumulation from identical repeats.

S28.7

Proof of Cor. 1 (greedy decoding)

Precise statement. Fix a probe c and consider greedy decoding (T =0) of either the reference or substitute endpoint, indexed by M ∈ {P, R}. Assume the idealized greedy decoder: at every decoded position the next-token argmax of

the endpoint law PM (· | y<t , c) is unique (or ties are broken by a fixed deterministic rule), and the backend is bit-exact, so that for each endpoint M the decoded response is a single deterministic string ⋆ yM = arg max PM (y | c) y

(realized identically on every query).

Let s(y) = − log P̂(P | y) be the fixed (enrollment(M ) frozen) i.i.d. queries let Sm = Pm score, and for m nominally 1 T =0 i=1 s(yi ) with yi ∼ PM (· | c) (Eq. 4). Let m 1 (P ) (R) (P ) (R) AUROC(m) = P Sm < Sm + 2 P Sm = Sm

be the rank functional of Section 3, and let the operational accumulation exponent be the slope d Ibauc := − log 1 − AUROC(m) , dm b fit over m. Then Iauc = 0. Equivalently: repeating an identical single greedy draw adds no evidence, so 1−AUROC(m) is constant in m and its decay exponent is exactly zero. (This is a statement about the increment from repetition; it does not assert that a single greedy draw fails to separate P and R.) Step 1: point-mass score. Under the idealized greedy de⋆ . coder, every query to model M returns the same string yM Because s is a deterministic function, every query yields the same score ⋆ ⋆ s⋆M := s(yM ) = − log P̂(P | yM ),

T =0 i.e. the push-forward law µM of s under PM (· | c) is the Dirac mass δs⋆M . Its variance is 0.

Step 2: no m dependence. For any m ≥ 1 and any model ⋆ M , the draws y1 = · · · = ym = yM are identical, hence m 1 1 X (M ) s(yi ) = · m s⋆M = s⋆M Sm = m i=1 m (deterministically, for every m).

(M ) Thus Sm is the same degenerate random variable δs⋆M for

all m; averaging copies of a constant returns the constant. (Equivalently, in the empirical-mean / large-deviations language of Prop. 1: the cumulant generating function of s under µM = δs⋆M is the linear ΛM (θ) = θ s⋆M , whose Legendre transform is 0 at x = s⋆M and +∞ elsewhere; the Cramér rate function is degenerate and the error exponent governing Sm is 0. The mean does not concentrate further with m because there is no fluctuation to suppress.) Step 3: constant rank functional. Since Sm ≡ s⋆P and (R) Sm ≡ s⋆R are constants independent of m, the joint law (P ) (R) of (Sm , Sm ) is the product of two Dirac masses and is identical for all m. Therefore AUROC(m) = 1{s⋆P < s⋆R } + 12 1{s⋆P = s⋆R } (P )

= AUROC(1)

for all m ≥ 1,

a fixed value in {0, 12 , 1} that does not vary with m. In particular 1 − AUROC(m) = 1 − AUROC(1) is constant in m.

Step 4: zero exponent. Becausem 7→ 1 − AUROC(m) is constant, log 1 − AUROC(m) is constant in m (with the convention log 0 = −∞ treated as a constant when AUROC ≡ 1, i.e. the two greedy strings already separate perfectly at m=1). Its slope in m is therefore identically 0, so d I = − log 1 − AUROC(m) = 0. ■ dm Remark 1 (meaning of I = 0). The exponent measures only the rate of improvement with repetition, not the level. If ⋆ yP⋆ ̸= yR then AUROC(1) may equal 1 and a single greedy draw perfectly separates the two endpoints; nonetheless I = 0 because additional identical draws contribute nothing. Greedy decoding therefore collapses the independent-repeat channel that the e−Im law of Prop. 1 relies upon: the auditor cannot drive a residual error toward 0 by issuing more queries with T =0. This is exactly the boundary of the temperature gating of Prop. 1(ii): as T → 0 the per-query law tends to a point mass and the marginal value of an extra query vanishes; at T =0 it is exactly 0. Remark 2 (temperature consistency). The companion second-order analysis sends T → 0 (β → ∞) so that V = VarpT (z) → 0 and the per-draw oracle rate of Eq. (7) vanishes; that is the same degeneracy as Step 1 viewed through the exponential family. The separate fact that T =0 vs. T >0 is itself trivially distinguishable (AUROC ≈ 0.99 by support/entropy collapse) is not a contradiction: it is a singledraw observable about whether decoding is greedy, not an accumulation rate over repeated identical greedy draws. Remark 3 (why T > 0). The contrapositive of Steps 1– 4 is that a strictly positive accumulation exponent requires the per-query score law µM to be non-degenerate (positive variance), since a degenerate µM forces Sm ≡ const and hence I = 0. For the paper’s sampling-based decoders the lever that lifts µM off a point mass is temperature, so T > 0 is the operative requirement; more generally any decoding configuration that renders PM (· | c) non-degenerate (and a probe c on which the conditional is genuinely random) suffices. Real-backend caveat. The clean I = 0 uses the idealized greedy decoder of the Statement. On real backends T =0 is not perfectly deterministic (batching, non-associative floating-point reductions, MoE routing, tie-breaking), so µM may carry a tiny but nonzero variance and the strict equalities of Steps 1–2 hold only approximately; the conclusion then degrades gracefully to a near-zero accumulation rate. Accordingly the corollary is to be read as a statement about the increment from repeating an identical single draw (cf. App. S1), which is what T =0 controls.

S28.8

Proof of Prop. 4 (score compression)

Write the Chernoff (Rényi/Hellinger) affinities X Φ⋆ (λ) = P (y)1−λ R(y)λ , y

Φsc (λ) =

µP (s)1−λ µR (s)λ ds,

λ ∈ [0, 1],

where µP = ϕ# P, µR = ϕ# R are the push-forwards of the deterministic measurable score s = ϕ(y), so that I ⋆ = − minλ log Φ⋆ (λ) and I sc = − minλ log Φsc (λ). Step 1 (pointwise data-processing via Hölder). Fix λ ∈ (0, 1); the endpoints give Φ⋆ (0) = Φsc (0) = Φ⋆ (1) = Φsc (1) = 1 (total masses) trivially. Since ϕ is determin−1 istic, the level P sets {ϕ (s)}s partition P response space and µP (s) = y∈ϕ−1 (s) P (y), µR (s) = y∈ϕ−1 (s) R(y) (read as integrals against a common dominating measure in the continuous case). On one level set put ay = P (y), by = R(y). Hölder’s inequality with conjugate exponents p = 1 1 1−λ , q = λ gives X 1−λ X λ X a1−λ bλy ≤ ay by y y∈ϕ−1 (s)

y∈ϕ−1 (s)

= µP (s)1−λ µR (s)λ .

(11) (This is superadditivity of the weighted geometric mean: coarsening cannot decrease affinity. It is the Hölder/Hellinger-affinity data-processing inequality, not the log-sum/Jensen inequality used for f -divergences.) Summing (11) over level sets, X X Φ⋆ (λ) = P 1−λ Rλ s

≤

y∈ϕ−1 (s)

µP (s)1−λ µR (s)λ

(12)

= Φsc (λ),

∀ λ ∈ [0, 1].

Step 2 (− log min reversal). For every λ, Φsc (λ) ≥ Φ⋆ (λ) ≥ minλ′ Φ⋆ (λ′ ), so minλ Φsc (λ) ≥ minλ Φ⋆ (λ). As t 7→ − log t is strictly decreasing, I sc = − log min Φsc (λ) ≤ − log min Φ⋆ (λ) = I ⋆ . λ

If P, R have disjoint supports then Φ⋆ (λ) = 0 for λ ∈ (0, 1), I ⋆ = +∞, and the bound holds vacuously. Step 3 (interior optimizer under P ̸= R). log Φ⋆ is convex (Hölder) with Φ⋆ (0) = Φ⋆ (1) = 1, hence Φ⋆ ≤ 1 on [0, 1], with M := minλ Φ⋆ < 1 whenever P ̸= R (else Φ⋆ ≡ 1 would force P = R). Thus the oracle minimizer λ⋆ ∈ (0, 1), and likewise any minimizer of Φsc attaining a value < 1 is interior. Step 4 (equality ⇒ sufficiency). Assume P ̸= R and I sc = ⋆ I , i.e. minλ Φsc = minλ Φ⋆ = M < 1. Let λ1 attain Φsc (λ1 ) = M . By (12), M = min Φ⋆ (λ) ≤ Φ⋆ (λ1 ) ≤ Φsc (λ1 ) = M, λ

so Φ⋆ (λ1 ) = Φsc (λ1 ) = M < 1, whence λ1 ∈ (0, 1) (Step aggregate equality 1 ) = Φsc (λ1 ) means P 3). The P Φ⋆ (λ1−λ 1−λ1 λ1 1 λ1 µ (s) µ (s) − a by = 0 with ev−1 P R s ϕ (s) y ery summand ≥ 0 by (11); hence each summand vanishes and (11) is an equality on every level set. Hölder equality at the interior exponent λ1 ∈ (0, 1) forces the vectors 1 (a1−λ )y and (bλy 1 )y to be proportional on the level set, i.e. y ay ∝ by there (with the convention that on the support both are positive; positions where one vanishes do not contribute

and carry P/R ∈ {0, ∞} constant by support agreement). Thus the likelihood ratio P/R is constant on each ϕ−1 (s), i.e. σ(s)-measurable; by Neyman–Fisher factorization s is a sufficient statistic for {P, R}. Step 5 (sufficiency ⇒ equality). Conversely, if P/R ≡ r(s) on ϕ−1 (s) then ay = r(s)by there, so for every λ, X X a1−λ bλy = r(s)1−λ by y y∈ϕ−1 (s)

ϕ−1 (s)

y∈ϕ−1 (s)

1−λ X

ϕ−1 (s)

so (11) is an equality; summing over s gives Φ⋆ ≡ Φsc and I sc = I ⋆ . Step 6 (calibration corollary). If logit P̂(P | y) = a + b ℓ(y) with ℓ = log(P/R) and b ̸= 0, then s(y) = − log σ(a + b ℓ(y)) is a strictly monotone (injective) function of ℓ, so the level sets of s coincide with those of ℓ, on which P/R = eℓ is constant; Step 5 gives I sc = I ⋆ . If b = 0 (uninformative) or s merges ℓ-distinct responses (lost feature / miscalibration), then P/R is not σ(s)-measurable, (11) is strict on some level set at the interior optimizer λ1 , and Step 4 yields I sc < I ⋆ . □

S28.9

Proof of Prop. 5 (temperature retunes)

Standing assumptions. Fix a context c and the served logit (“energy”) vector z ∈ Rn over a finite alphabet of size n < ∞, held fixed across temperatures. For an inverse temperature β = 1/T ∈ (0, ∞) define the tempered next-token law n X eβzi pβi = P βzj , eβzi . A(β) = log je i=1 Because n < ∞ and z is finite, A is real-analytic (C ∞ ) on all of R, pβi > 0 for every i and every finite β, and {pβ }β∈R is a regular one-parameter exponential family with natural parameter β and sufficient statistic z. We write β0 = 1/T0 , β1 = 1/T1 , ∆β = β1 − β0 , V := A′′ (β0 ) = Varpβ0 (z). The interior expansions below assume the non-degeneracy V > 0 (equivalently z not constant on the support, i.e. pβ0 not a point mass); the degenerate boundary V → 0 is treated separately in Step 6.

Step 1 (cumulants). By direct differentiation of A, P zi eβzi ′ A (β) = Pi βzi = Epβ [z], ie 2 A′′ (β) = Epβ [z 2 ] − Epβ [z] = Varpβ (z) ≥ 0. The nonnegativity of A′′ is the convexity of A; it holds unconditionally because a variance is nonnegative. Step 2 (χ2 identity). For any two members pβ1 , pβ0 of the family, X (pβi 1 )2 χ2 pβ1 ∥pβ0 = −1 pβi 0 i X e2β1 zi /Z(β1 )2 − 1, Z(β) := eA(β) , = β0 zi /Z(β ) e 0 i

and 2

ie β1

= Z(2β1 − β0 ) = eA(2β1 −β0 ) , so

(2β1 −β0 )zi

χ p ∥p

β0

A(2β1 −β0 )−2A(β1 )+A(β0 ) g(β1 )

−1

=e − 1, g(β1 ) := A(2β1 − β0 ) − 2A(β1 ) + A(β0 ). (13) Note 2β1 − β0 is finite, so the right side is finite. Differentiating g in β1 , g(β0 ) = 0, g ′ (β1 ) = 2A′ (2β1 − β0 ) − 2A′ (β1 ), g ′ (β0 ) = 0,

g ′′ (β1 ) = 4A′′ (2β1 − β0 ) − 2A′′ (β1 ), g ′′ (β0 ) = 2A′′ (β0 ) = 2V.

Hence g(β1 ) = V (∆β)2 + O(∆β 3 ) and, since eg − 1 = g + O(g 2 ), (14) χ2 pβ1 ∥pβ0 = (∆β)2 V + O(∆β 3 ). Step 3 (KL quadratic). For an exponential family, KL(pβ1 ∥pβ0 ) = A(β0 ) − A(β1 ) − A′ (β1 )(β0 − β1 ), a Bregman divergence of the convex A. Taylor expansion of A about β0 (legitimate since A ∈ C ∞ ) gives KL pβ1 ∥pβ0 = 12 A′′ (β0 )(∆β)2 + O(∆β 3 ) (15) = 12 (∆β)2 V + O(∆β 3 ), and identically KL(pβ0 ∥pβ1 ) = 21 (∆β)2 V + O(∆β 3 ); the two directions coincide through second order, the directional asymmetry entering only at O(∆β 3 ) (the Fisher metric is symmetric). Here A′′ (β0 ) = V is the Fisher information of the family at β0 , so (15) is the standard 12 gββ (∆β)2 Fisher quadratic.

Step 4 (Chernoff tilt).PFor two laws p, q on the finite alphabet define ψ(λ) = log i p1−λ qiλ and the Chernoff informai ⋆ tion I = − min0≤λ≤1 ψ(λ). With p = pβ0 , q = pβ1 in the same exponential family, p1−λ qiλ ∝ e(β0 +λ∆β)zi , whence i ψ(λ) = A β0 + λ∆β − (1 − λ)A(β0 ) − λA(β1 ). (16) ψ is smooth in λ, ψ(0) = ψ(1) = 0, and ψ is convex in λ because A is convex; thus the minimum is interior and unique whenever V > 0. Its location satisfies ψ ′ (λ⋆ ) = ∆β A′ (β0 + λ⋆ ∆β) − A(β1 ) − A(β0 ) = 0, i.e. A′ (β0 + λ⋆ ∆β) = A(β1 ) − A(β0 ) /∆β. Expanding both sides to first order in ∆β gives λ⋆ = 21 +O(∆β): the optimal tilt is the midpoint only in the small-gap limit, not exactly. However, the value is insensitive to this O(∆β) error: by the envelope theorem (or simply because ψ ′ (λ⋆ ) = 0 kills the first-order sensitivity), evaluating ψ at λ = 21 instead of λ⋆ changes ψ by O (λ⋆ − 12 )2 · ψ ′′ = O(∆β 2 ) · O(∆β 2 ) = O(∆β 4 ), negligible at the order considered. Therefore I ⋆ = −ψ(λ⋆ ) = −ψ( 21 ) + O(∆β 4 ), 1 −ψ( 12 ) = 12 A(β0 ) + A(β1 ) − A β0 +β , 2 the Bhattacharyya value. Taylor-expanding the convex A about β0 , the first-order terms cancel and 1 I ⋆ = 12 A(β0 ) + A(β1 ) − A β0 +β + O(∆β 4 ) 2 = 18 A′′ (β0 )(∆β)2 + O(∆β 3 ) = 18 (∆β)2 V + O(∆β 3 ). (17)

(Numerically, I ⋆ / 18 (∆β)2 V → 1 as ∆β → 0, and the minimizing λ⋆ → 0.499 at small gaps.) Equations (14), (15), (17) establish the displayed triple KL ≈ 12 (∆β)2 V, χ2 ≈ (∆β)2 V, I ⋆ ≈ 18 (∆β)2 V. P Step 5 (heat capacity). Let H(T ) = − i pTi log pTi be the Shannon entropy of the tempered law, pT = pβ with β = 1/T . For an exponential family the entropy in the natural parameter is the Legendre-type relation H = A(β)−βA′ (β) (since log pβi = βzi − A(β) gives H = −E[βz − A] = A − βA′ ). Differentiating, dH = A′ (β) − A′ (β) − βA′′ (β) = −βA′′ (β). dβ

Remark (scope). This is a supplementary low-separation stress test, not a robustness guarantee against a deliberately matched cheaper model. The guarantee is conditional on the backend being measurably separated from the reference.

S28.10

With β = 1/T , dβ/dT = −1/T 2 , the chain rule gives A′′ (β) dH dβ dH , i.e. = = (−βA′′ (β)) − 1/T 2 = dT dβ dT T3

dH ≥ 0, (18) dT the nonnegativity being that of the variance; equivalently dH/dT = V /T 3 ≥ 0, so output entropy is nondecreasing in temperature. VarpT (z) = A′′ (β) = T 3

Step 6 (temperature gap). The map T 7→ β = 1/T gives ∆β = 1/T1 − 1/T0 = −∆T /(T0 T1 ) = −∆T /T02 + O(∆T 2 ), so (∆β)2 = (∆T )2 /T04 + O(∆T 3 ). Substituting into (17) and using (18) at T0 , V = T03 dH/dT |T0 , I ⋆ (T0 , T1 ) = 18 (∆β)2 V + O(∆β 3 ) (∆T )2 = 18 + ··· T03 dH 4 dT T0 (∆T )2 dH = + ··· , 8 T0 dT T0

c0 zP (·) for a single global scalar c0 at all contexts—i.e. R is a temperature reparametrization of P , not a distinct model. Hence IRIS’s second-order insensitivity to temperature is the intended dilution invariance: it waives an on-family sampler retune (unchanged weights) while an off-family substitute incurs first-order (Θ(1) tell-rate, 1/ϵ) cost whenever it is measurably separated. □

(19)

which is exactly Eq. (7). Thus along the tempering family the response-level separation is I ⋆ = Θ (∆β)2 = Θ (∆T )2 : a strictly second-order effect in the gap, vanishing quadratically as T1 → T0 . By the standard Chernoff–Stein bound, distinguishing pβ0 from pβ1 from m i.i.d. single-draw probes to fixed error needs m = Θ(1/I ⋆ ) = Θ (∆β)−2 draws.

Greedy boundary. As T → 0 (β → ∞) the law pβ concentrates on arg maxi zi ; if that argmax is unique, V = Varpβ (z) → 0 and (19) gives a vanishing per-draw rate, consistent with dH/dT → 0. The interior expansion’s nondegeneracy hypothesis V > 0 thus fails exactly at the greedy boundary; there the appendix’s separate support-collapse observable (greedy repeats a single string) — not the secondorder tempering rate — governs separability. This is a change of statistic, not a contradiction.

Geometric conclusion. The reachable set { softmax(βz) : β > 0 } is a one-dimensional analytic curve in the simplex ∆n−1 , hence Lebesgue-measure zero for n ≥ 3. A pure temperature retune moves β along this curve, incurring only the second-order cost (19); a genuine model change perturbs the vector z itself, generically off the curve. For R to mimic P by tempering alone at every context c would require zR (·) =

Proof of Prop. 6 (tail budget)

Restatement. Fix a reference distribution P with continuous (atomless) score CDF FP (t) = Py∼P (s(y) ≤ t), and define the tail-separation function qα = Py∼R s(y) > FP−1 (1 − α) . Assume there are constants c > 0 and a tail exponent κ ∈ [0, 1) with qα ≍ c ακ as α → 0 (i.e. qα /(cακ ) → 1). The auditor runs the any-tell rule of Thm. 1, controlling overall type-I error at level α by the union bound, i.e. using per-query level α/m (threshold τm = FP−1 (1 − α/m), perquery tell rate qα/m ). Then, with α, δ ∈ (0, 1) and c, κ held fixed while ϵ → 0, the minimal budget attaining power ≥ 1−δ against any suspect with routing fraction ≥ ϵ satisfies m⋆ (ϵ) = Θ ϵ−1/(1−κ) , 1/(1−κ) m⋆ (ϵ) ≍ ln(1/δ) ϵ−1/(1−κ) . κ cα

In particular κ = 0 gives Θ(ϵ−1 ), κ = 12 gives Θ(ϵ−2 ) (the same scaling exponent as the aggregate mean-shift budget of Thm. 1(c), a coincidence of exponents between two distinct tests, not of optimality), and κ ∈ (0, 21 ) interpolates. As κ ↑ 1 the exponent and constant diverge; κ = 1 is the degenerate wall, where no finite m attains power once ϵ c α < ln(1/δ).

Write Lδ := ln(1/δ) > 0; a ≍ b means a/b → 1 and a = Θ(b) means 0 < lim inf a/b ≤ lim sup a/b < ∞, both as ϵ → 0.

Step 0 (any-tell power). By Thm. 1, at per-response threshold τ the rule flags at the first query with s(y) > τ . Under Qϵ each query crosses with probability r(τ ) := PQϵ (s > τ ) = (1 − ϵ)α1 (τ ) + ϵq(τ ), and since queries are i.i.d. the within-m power is exactly powm (τ ) = 1 − (1 − r(τ ))m ,

r(τ ) ≥ ϵq(τ ).

(20)

Step 1 (type-I threshold). On an honest endpoint the mquery false-positive probability is 1 − (1 − α1 (τ ))m ≤ m α1 (τ ) (Bernoulli/union bound), so it suffices to take α1 (τ ) ≤ α/m. As FP is atomless, τm := FP−1 (1 − α/m) gives exactly α1 (τm ) = α/m and, by definition of the tailseparation function, q(τm ) = qα/m . Thus for each candidate budget m the auditor is forced to the single threshold τm ; all statements below are read at this m-dependent threshold, which is self-consistent because τm is determined by m alone. (Discrete FP : take the smallest τ with α1 (τ ) ≤ α/m; equalities become “≤”, lowering the achievable q and enlarging only the constant.)

Step 2 (upper bound). At τ = τm , powm ≥ 1 − (1 − ϵqα/m )m by (20), so (1 − ϵqα/m )m ≤ δ suffices for power ≥ 1 − δ. Using − ln(1 − x) ≥ x on [0, 1) (verified: h(x) = − ln(1 − x) − x has h(0) = 0, h′ (x) = x/(1 − x) ≥ 0), this holds whenever m ϵ qα/m ≥ Lδ . (21)

Substituting qα/m ≍ c(α/m)κ (valid since α/m → 0, justified a posteriori below) gives ϵcακ m1−κ ≳ Lδ ; since κ < 1, x 7→ x1−κ is increasing and invertible, so 1/(1−κ) m ≥ m+ (ϵ) := cLαδκ ϵ−1/(1−κ) (1 + o(1)). (22)

As ϵ → 0, m+ → ∞, retroactively validating the α/m → 0 tail asymptotic. Hence m⋆ ≤ ⌈m+ ⌉ = O(ϵ−1/(1−κ) ).

Step 3 (lower bound). Fix the auditor’s only admissible threshold τm from Step 1. Power ≥ 1 − δ requires, by (20), (1 − r(τm ))m ≤ δ, i.e. −m ln(1 − r(τm )) ≥ Lδ . Now α α r(τm ) = (1−ϵ) m +ϵqα/m ≤ m +ϵqα/m → 0 (as α/m → 0 and ϵqα/m ≤ ϵ → 0). Applying − ln(1 − x) ≤ x/(1 − x) on [0, 1) (verified: g(x) = x/(1 − x) + ln(1 − x) has g(0) = 0, g ′ (x) = x/(1 − x)2 ≥ 0), Lδ ≤ −m ln(1 − r(τm )) ≤ =

m r(τm ) 1 − r(τm )

1 (1 − ϵ)α +ϵ m qα/m . 1 − r(τm ) | {z } →α

Since r(τm ) → 0 the prefactor → 1, and the honest term (1−ϵ)α ≤ α is an m-independent constant that is dominated by Lδ m+ (ϵ) as ϵ → 0; hence Lδ ≤ (1 + o(1)) ϵ m qα/m ,

i.e.

m ϵ qα/m ≥ Lδ (1 − o(1)).

This is (21) up to 1 − o(1), so repeating Step 2 with the reversed inequality gives m ≥ m− (ϵ) = m+ (ϵ)(1 − o(1)). (The worst-case honest contribution α1 (τm ) ≤ α/m is what makes the lower bound match the upper bound to leading order; it adds at most O(α) = o(Lδ m+ ) to the budget.) Combining Steps 2–3, 1/(1−κ) m⋆ (ϵ) ≍ cLαδκ ϵ−1/(1−κ) = Θ ϵ−1/(1−κ) . The two-sided Θ uses − ln(1 − x) = x(1 + O(x)) with x = ϵqα/m → 0, exactly the regime ϵ → 0, q ≤ 1.

Step 4 (regimes). (i) κ = 0: qα/m → q0 := c > 0, 1 − κ = 1, so m⋆ = Θ(Lδ /(ϵc)) = Θ(ϵ−1 ), α-independent (constant α−κ/(1−κ) = α0 = 1); reproduces Thm. 1(a)/(c) first regime, eq. (18). (ii) κ = 21 : 1/(1 − κ) = 2, so m⋆ = √ Θ(ϵ−2 ) with constant (Lδ /(c α))2 = L2δ /(c2 α) (scaling −1 α ). This matches the scaling exponent ϵ−2 of the aggregate mean-shift / Neyman–Pearson budget of Thm. 1(c), eq. (19); the two budgets come from different tests (any-tell first-crossing vs. aggregate Sm mean-shift), so the agreement is of exponents only, not of constants or optimality, and κ = 21 is an empirical tail property (Table 22), not

a derived consequence of χ2 (R∥P ) being small. (iii) Interpolation/wall: for κ ∈ (0, 12 ), 1 < 1/(1 − κ) < 2, monotone increasing. As κ ↑ 1, 1/(1 − κ) → ∞ and the constant (Lδ /(cακ ))1/(1−κ) → ∞. At κ = 1, (21) becomes ϵcα · m0 = ϵcα ≥ Lδ , independent of m: if ϵcα < Lδ no finite m attains power ≥ 1 − δ (the wall). Hence κ ∈ [0, 1) is necessary for the finite closed form. (The ϵ-power −1/(1 − κ) and α-power −κ/(1 − κ) equal 0, − 13 , −1, −3 at κ = 0, 14 , 12 , 34 .) □

Remark (scope). The Θ is in ϵ with (α, δ, c, κ) fixed; the hidden constant carries an α−κ/(1−κ) and a (ln(1/δ))1/(1−κ) factor, both diverging as κ ↑ 1. The result is a statement about the any-tell first-crossing test with a Bonferroni per-query level α/m adapted to the horizon; a per-query level held fixed in m would give a different (non-quantile-shrinking) calculation. Atomlessness of FP is idealizing for the discrete short-string probes, where one reads qα/m at the achievable level α1 (τ ) ≤ α/m, affecting only the constant; for the population κ=0 this further requires R to retain mass on the extreme upper tail of s under P (a support difference), not merely P ̸= R (App. S26). One more constant correction: the honest reference itself contributes crossings at rate α1 = α/m, so the no-substitute zero-tell probability is (1 − α/m)m → e−α rather than 1, and the sufficiency condition (21) sharpens to m ϵ qα/m ≥ ln(1/δ) − α + o(1); this shifts the leading constant by the O(α) term ln(1/δ) − α in place of ln(1/δ), leaving the ϵ-exponent unchanged.

S28.11

Proof of Prop. 3 (multi-diluent audit)

We prove the detection bound and the identifiability criterion at the population (noiseless) level, the regime in which Prop. 3 is stated. Setup. A gateway dilutes one reference P with J substitutes R1 , . . . , RJ , routing each query independently to P with probability ϵ0 and to Rj with probability ϵj , where ϵ = (ϵ1 , . . . , ϵJ ) ∈ Θ := {ϵ ∈ RJ : ϵj ≥ 0, 1⊤ ϵ ≤ 1}, ϵ0 := 1 − 1⊤ ϵ ≥ 0,

so the per-query response law is the mixture Q = ϵ0 P + PJ K j=1 ϵj Rj . The posterior map u(y) = P̂(· | y) ∈ ∆ (the simplex on the K enrolled endpoints) is a fixed measurable map, identical at enrollment and audit; signatures ḡM = Ey∼M [u(y)] and the audit mean v̄ = EQ [u] are exact expectations (integrable since u ∈ [0, 1]K ). No calibration of u is used. Detection. Fix a per-response threshold τ and let α1 (τ ) = PP (s > τ ), qj (τ ) = PRj (s > τ ). The event Aτ = {y : s(y) > τ } does not depend on the law, so applying Q to the fixed set Aτ and using additivity over the mixture components, p(τ ) = PQ (s > τ ) = ϵ0 α1 (τ ) +

J X

ϵj qj (τ )

j=1

≥

J X j=1

ϵj qj (τ ) ≥ ϵtot min qj (τ ), 1≤j≤J

P with ϵtot = j ϵj ; the first inequality drops the nonnegative term ϵ0 α1 and the second uses ϵj ≥ 0 and qj ≥ minj ′ qj ′ . This bound is unconditional (no separation hypothesis). If moreover a threshold τ ⋆ exists with qmin := minj qj (τ ⋆ ) > 0 and α1 (τ ⋆ ) ≤ α/m, then under i.i.d. routing the tells are i.i.d. Bernoulli(p(τ ⋆ )) and the any-tell argument of Thm. 1(a,b) applies verbatim with ϵ → ϵtot , q → qmin : P(no tell in m) ≤ (1 − ϵtot qmin )m ≤ e−ϵtot qmin m , so m⋆ = ⌈ln(1/δ)/(ϵtot qmin )⌉ gives power ≥ 1 − δ, while the honest false-positive probability is ≤ m α1 (τ ⋆ ) ≤ α. Detection thus runs at the 1/ϵ law in the total foreign fraction ϵtot , set by the weakest separating diluent, and needs no enrollment of the Rj . The hypothesis “every diluent separates” is used only to guarantee the existence of τ ⋆ (so qmin > 0 and α1 → 0), not for the inequality itself. Affine mean model. With G := [ ḡR1 − ḡP , . . . , ḡRJ − ḡP ] ∈ RK×J , linearity of expectation over the mixture and ϵ0 = 1 − 1⊤ ϵ give X v̄ = EQ [u] = ϵ0 ḡP + ϵj ḡRj = ḡP + G ϵ. j

Because u(y) ∈ ∆ pointwise, 1⊤ u(y) = 1, hence 1⊤ ḡM = 1 for every endpoint M and 1⊤ (ḡRj − ḡP ) = 0. Every column of G therefore lies in the hyperplane H0 = {x : 1⊤ x = 0}, so rank(G) ≤ dim H0 = K − 1. In particular, identifying J fractions requires J ≤ K − 1: at least J+1 enrolled endpoints are necessary. K

Identifiability. Let Φ : Θ → RK , Φ(ϵ) = ḡP + Gϵ, be the (affine) mean map, with linear part G, so Φ(ϵ) − Φ(ϵ′ ) = G(ϵ − ϵ′ ). We show Φ is injective on Θ (equivalently ϵ is identifiable from v̄) iff G has full column rank J. (⇐) Full rank ⇒ injective. If G has full column rank then Gd = 0 ⇐⇒ d = 0; hence Φ(ϵ) = Φ(ϵ′ ) ⇒ G(ϵ − ϵ′ ) = 0 ⇒ ϵ = ϵ′ , injective on all of RJ , a fortiori on Θ. Moreover, since G⊤ G ≻ 0, for v̄ = Φ(ϵ⋆ ) the noiseless simplex-constrained least squares objective ∥v̄ − Φ(ϵ)∥22 = (ϵ − ϵ⋆ )⊤ (G⊤ G)(ϵ − ϵ⋆ ) is a strictly convex quadratic vanishing only at ϵ = ϵ⋆ , so ϵ⋆ is its unique minimizer over Θ. (⇒) Injective ⇒ full rank. Contrapositive: if G is rank1 deficient, pick d ̸= 0 with Gd = 0. The point ϵ⋆ = 2(J+1) 1 J ⋆ ⊤ ⋆ ⋆ satisfies ϵ > 0 and 1 ϵ = 2(J+1) < 1, so ϵ lies in the (nonempty, open) interior Θ◦ ; for small t ̸= 0, ϵ′ := ϵ⋆ + td ∈ Θ◦ with ϵ′ ̸= ϵ⋆ and Φ(ϵ′ ) − Φ(ϵ⋆ ) = t Gd = 0. Thus Φ is not injective. (The necessity direction uses that Θ is full-dimensional; at a boundary corner a rank-deficient model could be locally identified.) Signature collapse. If two diluents share a signature, ḡRa = ḡRb (a ̸= b), columns a, b of G coincide, so d = ea − eb ̸= 0 satisfies Gd = 0: G is rank-deficient and only the sum ϵa +ϵb is identifiable, not the individual fractions (ϵ and ϵ + t(ea − eb ) are indistinguishable for small feasible P t). More generally null(G) = {d : j dj (ḡRj − ḡP ) = 0} is exactly the set of unidentified directions, the multi-diluent analogue of a low-separation wall. At J = 1 full rank reads ḡR1 ̸= ḡP , the mean-posterior analogue (not a literal specialization) of the single-diluent tell separation q > α1 . □

References Bruckner, T. 2026. One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions. arXiv preprint arXiv:2607.10252. Cai, W.; Shi, T.; Zhao, X.; and Song, D. 2025. Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs. arXiv preprint arXiv:2504.04715. Cover, T. M.; and Thomas, J. A. 2006. Elements of Information Theory. Wiley-Interscience, 2nd edition. ISBN 9780471241959. Dima, A.; Foulds, J.; Pan, S.; and Feldman, P. 2025. You’ve Changed: Detecting Modification of Black-Box Large Language Models. arXiv preprint arXiv:2504.12335. Donoho, D.; and Jin, J. 2004. Higher Criticism for Detecting Sparse Heterogeneous Mixtures. The Annals of Statistics, 32(3): 962–994. Gao, I.; Liang, P.; and Guestrin, C. 2025. Model Equality Testing: Which Model Is This API Serving? In International Conference on Learning Representations (ICLR). Hopkins, A. K.; Renda, A.; and Carbin, M. 2023. Can LLMs Generate Random Numbers? Evaluating LLM Sampling in Controlled Domains. In ICML Workshop on Sampling and Optimization in Discrete Space (SODS). Huber, P. J. 1964. Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35(1): 73–101. Nikolic, I.; Baluta, T.; and Saxena, P. 2025. Model Provenance Testing for Large Language Models. In Advances in Neural Information Processing Systems, volume 38, 34126– 34153. Ong, J. M.; Di Ferrante, M.; Pazdera, A.; Garner, R.; Jaghouar, S.; Basra, M.; Ryabinin, M.; and Hagemann, J. 2025. TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference. arXiv preprint arXiv:2501.16007. Pasquini, D.; Kornaropoulos, E. M.; and Ateniese, G. 2025. LLMmap: Fingerprinting For Large Language Models. In 34th USENIX Security Symposium (USENIX Security 25), 299–318. Richardeau, G.; Dashyan, G.; Le Merrer, E.; and Tredan, G. 2026. FLIPS: Instance-Fingerprinting for LLMs via Pseudorandom Sequences. In International Conference on Machine Learning (ICML). Richter, L.; He, X.; Minervini, P.; and Kusner, M. J. 2025. An Auditing Test to Detect Behavioral Shift in Language Models. In International Conference on Learning Representations (ICLR). Shekhar, S.; and Ramdas, A. 2023. Nonparametric TwoSample Testing by Betting. IEEE Transactions on Information Theory. Sun, M.; Yin, Y.; Xu, Z.; Kolter, J. Z.; and Liu, Z. 2025. Idiosyncrasies in Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 57854–57885. PMLR.

Zhao, M.; Du, Y.; and Wang, M. 2026. Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributions. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 22942– 22959. San Diego, California, United States: Association for Computational Linguistics.

Record · ID 394317 · SHA-256 0829aab1cbe84a98
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.