Self-Certification of Representation Adequacy: Sequential Certification at Minimum Task Loss

Self-Certification of Representation Adequacy: Sequential Certification at Minimum Task Loss Zijie Huang

750691178@qq.com

Independent Researcher

arXiv:2608.02267v1 [cs.AI] 3 Aug 2026

Abstract Agents that act on a compressed representation of their history face a structural risk: if the representation aliases histories with different optimal actions, no rule measurable with respect to the representation can avoid an irreducible per-round loss, and the agent may be unable to detect this from its own transcript. This paper develops a four-layer theory of self-certification of representation adequacy. The static layer defines decision-theoretic adequacy through a Bayes-risk grouping identity and prices a one-shot external verification by an exact total-variation threshold. The sequential layer poses certification as an optimalstopping problem in the currency of task loss: we define an environment-wise certification complexity constant Ci through a covering linear program, prove an information–task-loss lower bound R(π, Ei ) ≥ Ci kl(δ, 1 − δ) for every δ-correct strategy, and give a Certification Track-and-Stop (CTS) policy whose cost matches the bound asymptotically. A final boundary layer gives an explicit kernel-switching example and identifies the open theorem needed to cover policy switching or representation repair; it does not claim that the fixedkernel guarantees extend to representation revision. The proofs of the two main theorems are given in full in the appendices, with all concentration, tracking, and plug-in stability steps written out. Keywords: representation adequacy, self-certification, sequential testing, track-and-stop, pure exploration, Bayes risk, dual control

1 Introduction 1.1 Motivation: when does an agent know that its representation still suffices? Agents that must act on a compressed view of their history—a finite memory state, a feature map, a learned embedding—face a structural risk that is easy to state and hard to detect from the inside. If the representation f maps two histories with different optimal actions to the same memory state, then no rule measurable with respect to f (H) can separate them, and the agent pays an irreducible per-round loss. The static layer of this paper formalizes the resulting gap as a Bayes-risk grouping identity (Theorem 1 of Section 2, a definitional rearrangement of Bayes risk over representation cells, in the tradition of comparative experiments (Blackwell, 1951, 1953; Le Cam, 1964); we do not claim it as a new result) and uses it to define the adequacy label  ∗ ∗ Θ(E) = 1 RE (f (H)) = RE (H)

∈ {0, 1},

which records whether acting on the representation loses nothing relative to acting on the full history. The label is deliberately decision-theoretic: it depends on the loss function and on the environment, not only on the information structure. An agent whose representation ©2026 Huang. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/.

Huang

is inadequate may not be able to discover this from its own transcript, because inadequacy is precisely a statement about what the transcript cannot express. The static layer then asks a one-shot question: should the agent buy a single external verification now? Under an equal label prior, Theorems 3 and 6 of Section 2 answer this with an exact threshold: a one-shot verification is worth its audit cost c if and only if c ≤ c∗ = (T r/2) (1 − TV), where T is the remaining horizon, r the per-round aliasing regret under inadequacy, and TV the total variation distance between the label-aggregated internal transcript laws. The argument is a standard application of the Le Cam two-point method and the Bayes error–total variation identity (Tsybakov, 2009; Le Cam, 1986); the contribution of that layer is the choice of the tested label—decision-theoretic adequacy— together with the threshold characterization and its boundary analysis, not the testing machinery. Deployment, however, rarely offers a single buy/no-buy decision with a fixed signal quality. Evidence about adequacy accumulates gradually, and it accumulates through the agent’s own behavior: every action the agent takes produces both an observation and a task consequence. This paper studies the sequential form of the certification problem: how can an agent reach a prescribed confidence δ about the adequacy of its representation at minimum cumulative task loss? The one-shot threshold c∗ prices a single purchase; the sequential model prices the entire acquisition policy. The framework is motivated by agents with an explicit decision memory whose action repertoire mixes task actions with epistemic actions. In the environments for sequential decisions (ESD) background of this work, the repertoire includes Speak/Silence (act on the current representation), Ask (clarify the latent state), Verify (purchase an external adequacy audit), and Defer (abstain and wait); a Repair action, which modifies the representation itself, is discussed only as an open direction in Section 4. Model M1 below abstracts this repertoire as a finite action set in which each action a simultaneously (i) generates an observation from a channel KEa that is informative about the adequacy label to a degree measured by Kullback–Leibler (KL) divergence, and (ii) incurs a task loss gE (a) that depends on the unknown environment—so that the very act of gathering certification evidence may itself be the source of the aliasing regret the agent fears. This dual role of actions is the modeling decision from which everything else in the paper follows. 1.2 The four-layer structure of the paper The paper is organized as a chain of four layers, and we state explicitly which parts are standard tools and which parts we claim as contributions, because the layers differ sharply in this respect. 1. Static representation layer (Section 2). Theorem 1 decomposes the Bayes-risk gap of a lossy representation into cell-wise aliasing terms. This is a grouping identity—a definitional tool, not a new result—and we label it as such. Its role is to make the adequacy label Θ(E) a well-defined, environment-dependent but action-independent object of inference. 2. One-shot certification layer (Section 2). Theorems 3 and 6 characterize the value of a single verification purchase. The underlying mathematics is the two-point method and 2

Self-Certification of Representation Adequacy

the Bayes error–TV identity, standard tools (Tsybakov, 2009); the layer’s contribution is the combinatorial step of taking the tested hypothesis to be the adequacy label and deriving the closed-form threshold c∗ = (T r/2)(1−TV) together with the assumptions under which each component is indispensable. 3. Information–task-loss trade-off layer (Section 3, with proofs in Appendices A and B). This is the core of the present paper. For the sequential certification model M1, we prove a lower bound on the cumulative task loss of any δ-correct certification policy (Theorem 8), give an explicit policy, Certification Track-and-Stop (CTS, Algorithm 1), whose loss matches the lower bound to first order as δ → 0 (Theorem 9), and conclude that the optimal certification cost satisfies limδ→0 Rδ∗ (Ei )/ log(1/δ) = C(Ei , f ) (Corollary 10), where the constant C(Ei , f ) is a cost-weighted characteristic time with two equivalent forms (Definition in Section 3.2). The technical ingredients—a transportation inequality (Kaufmann et al., 2016) for the lower bound and the Track-and-Stop template (Garivier and Kaufmann, 2016) for the upper bound—are credited where they are used; what is new is the model in which the hypotheses are adequacy labels and the cost currency is task loss, and the resulting constant. 4. Audit-and-repair layer (Section 4). Certification that drives policy switching (act on f while adequate, fall back when certified inadequate) and Repair actions that change the representation itself both violate the fixed-kernel premise of Model M1. We formalize the boundary precisely, exhibit a kernel-switching example whose achievable cost is O(1) rather than Θ(log(1/δ)), and state the resulting open problems without claiming results for them. 1.3 Statement of the main results (preview) Model M1, in one sentence: an agent interacts with one of K known environments E1 , . . . , EK , each carrying a binary adequacy label Θi ∈ {0, 1} induced by a fixed representation f ; in every round it chooses an action At from a finite set A, observes Yt ∼ KiAt , and pays task loss gi (At ); it must stop at an almost surely finite time τ and output a label estimate Θ̂Pwith maxiPi (Θ̂ ̸= Θi ) ≤ δ, minimizing the expected cumulative task loss R(π, Ei ) = Eπi t≤τ gi (At ) . The answer is governed by a single constant per environment, the certification complexity Ci = C(Ei , f ), which we give in two provably equivalent forms (Section 3.2): a covering linear program (LP), nP o
P a a Ci = min (1) a gi (a) na : na ≥ 0, a na d Ki ∥ Kj ≥ 1 ∀ j ∈ Alt(i) , where d(·∥·) is KL divergence and Alt(i) = {j : Θj ̸= Θi } is the set of environments whose adequacy label disagrees with that of Ei ; and a max-min, cost-weighted characteristic-time form, h
i−1 P wa Ci = sup inf d Kia ∥ Kja , (2) a gi (a) w∈∆(A) j∈Alt(i) whose equivalence with (1) follows from LP strong duality and a minimax argument. With the degenerate conventions Ci = 0 (a zero-loss action already separates all alternatives) and 3

Huang

Ci = +∞ (the LP is infeasible, i.e., some alternative is statistically indistinguishable under every action), the main results of the paper are: • Lower bound (Theorem 8): every δ-correct policy π satisfies R(π, Ei ) ≥ Ci kl(δ, 1−δ) ≥
Ci log 1/(2.4δ) for each environment Ei , where kl denotes Bernoulli KL divergence. • Matching upper bound (Theorem 9): the CTS policy (Algorithm 1)—maximumlikelihood estimation of the environment, plug-in computation of the cost-ratiooptimal allocation w∗ , track-and-stop forced exploration, and a GLR stopping rule with threshold β(t, δ) = log(2t(K − 1)/δ)—is δ-correct and satisfies lim supδ→0 R(π ∗ , Ei )/ log(1/δ) ≤ Ci . • Limit (Corollary 10): under the union of the assumptions of the two bounds,  lim Rδ∗ (Ei ) log(1/δ) = Ci ,

δ→0

where Rδ∗ (Ei ) is the infimum of R(π, Ei ) over δ-correct policies. In words: the minimal task loss required to certify representation adequacy to confidence δ grows exactly like C(E, f ) log(1/δ), and the constant is the value of a zero-sum game in which the agent chooses an action mix and an adversarial alternative environment chooses the hardest label-disagreeing hypothesis, with information measured per unit of task loss. 1.4 Positioning relative to prior work: an audit-based assessment Because the constants in our main theorem have the shape of classical sequential-design constants, a careful account of what is and is not covered by prior work is part of the contribution statement itself. We summarize a five-direction audit of the nearest literatures; the full version with theorem-level cross-checks is given in the related-work section. (a) Sequential design of experiments and controlled sensing. The line of Chernoff (1959) establishes that, for testing a finite set of hypotheses with experiment-dependent information, the expected total cost of an asymptotically optimal procedure grows like log(1/δ) times the reciprocal of a max-min information rate. Nitinawarat and Veeravalli (2015) resolve the problem with per-action costs that are exogenous fees: their Theorem 5.1 gives a lower bound on the expected accumulated cost with coefficient equal to a maxmin ratio of information rate to cost rate, together with a matching test. Nitinawarat et al. (2013) develop the controlled-sensing version in which actions are chosen for their observation kernels, and Naghshvar and Javidi (2013) give an active sequential test whose total-cost asymptotics have the same shape; in all of these models, observation actions carry sensing costs while the classification decision is made by a separate terminal action. We state plainly: if Model M1 is specialized to a binary environment class with known kernels, exogenous per-action fees, and a terminal decision isolated from the loss stream, then the resulting lower bound and matching asymptotics are already contained in this literature, and renaming the hypothesis “adequacy” would add nothing. Our statements are therefore not claimed as new in that simple form. 4

Self-Certification of Representation Adequacy

(b) Pure exploration and best-arm identification. The fixed-confidence pureexploration literature provides the technical template we use. Garivier and Kaufmann (2016) prove, for best-arm identification, a transportation-based lower bound E[τ ] ≥ T ∗ (µ) kl(δ, 1 − δ) and a Track-and-Stop algorithm whose sample complexity matches it to first order; the transportation inequality itself is due to Kaufmann et al. (2016). Degenne and Koolen (2019) handle arbitrary answer maps (including identification of properties beyond the optimal arm) but with homogeneous sampling costs, while Kanarios et al. (2024) study heterogeneous per-arm costs with the answer map given by reward means, arriving at a cost-weighted characteristic time of the same max-min shape as (2). Neither covers the intersection treated here: an answer map defined by a decision-theoretic adequacy property of a representation, with costs that are task losses coupled to the same unknown environment that defines the label. (c) Controlled sensing, POMDP scheduling, and dual control. In the partially observed control literature, sensor-scheduling and controlled-sensing models choose actions for their observation kernels (Krishnamurthy, 2016; Kaelbling et al., 1998), and POMDP bandits share actions between reward and observation (Krishnamurthy and Wahlberg, 2009)— but without a certification target and stopping rule. The dual-control line originating with Feldbaum (Feldbaum, 1963; Bar-Shalom and Tse, 1974; Tse and Bar-Shalom, 1975; Witsenhausen, 1971; Yame, 1987) couples identification and control in a different, control-oriented sense. Our audit finds the same information–control overlap but no adequacy-label, taskloss-currency, fixed-confidence stopping result in the references examined; the switching extension remains open and is stated as such in Section 4. (d) Information-directed sampling. In information-directed sampling, actions do carry both an immediate loss and an information gain: Russo and Van Roy (2018) define the information ratio of revealing actions, and the satisficing variant of Russo and Van Roy (2022) is conceptually close to certification. That framework is regret-based, √ however: its information object is the (endogenous) optimal action, the guarantees are T -type regret bounds, and there is no δ-certification target, no stopping rule, and no log(1/δ) constant to match. Our lower bound therefore follows the additive, covering route of Chernoff (1959) rather than the ratio route. The three differences we rely on. Against this background, the results of this paper rest on three modeling differences that the audited literatures do not cover, individually or in combination: (i) the alternative set Alt(i) is defined by the adequacy label Θj ̸= Θi —a self-referential, decision-theoretic property of the representation and loss, for which the audit found no precedent in any of the four directions; (ii) the cost gi (a) is task loss incurred under the same environment that defines the label, and the environment class is composite: distinct environments may share a label yet induce different loss profiles, so the price of certification involves a worst case within the label class rather than an expected fee; and (iii) the objective is δ-correct certification of that label with the cost currency of (ii), an intersection left open by the cost-heterogeneous (Kanarios et al., 2024) and answermap (Degenne and Koolen, 2019) generalizations alike. The constant C(E, f ) acquires its decision-theoretic semantics from (i)–(ii); its mathematical shape is inherited from the templates credited above. 5

Huang

1.5 From a falsified conjecture to the present formulation It is worth recording how the present problem formulation arose, because it disciplines the scope of our claims. The investigation began from a broader conjecture—that an agent could, from inside a fixed representation, detect and characterize its own inadequacy (a “state-closure” property of the decision memory). That conjecture is false: the static layer exhibits an explicit counterexample (Counterexample A of Section 2) in which two environments induce identical internal transcript laws under every internal certification rule, so internal evidence alone provably cannot separate adequacy from inadequacy, while an external audit separates them at once. What survives the counterexample is a more modest and, we believe, correct question: the agent cannot close its own state, but it can price its uncertainty about the adequacy label and purchase evidence about it optimally. The one-shot layer prices a single purchase; the present sequential layer prices the optimal acquisition policy; and the audit-and-repair layer of Section 4 marks exactly where the pricing framework again meets its boundary—when the act of certification changes the process being certified. We confine the historical conjecture to this motivational role and state no results about it.

1.6 Organization Section 2 (static layer) defines the Bayes-risk framework, the adequacy label, and oneshot certification. Section 3 defines Model M1 and the certification complexity C(E, f ), and states the main Theorems 8 and 9 with Corollary 10, the CTS policy (Algorithm 1), and the non-triviality and boundary statements NT1–NT2 and B1–B3. Appendices A and B prove the lower and upper bounds respectively. Section 4 analyzes kernel switching, Verify/Repair actions, and policy-dependent transcripts, and collects the open directions. Section 5 collects the numerical work on fully specified finite instances: exact LP values of the certification complexity, a simulated convergence study of the CTS policy, and the boundary statements in numbers. All numerical claims in the paper concern finite instances checked by exact LP solves, exhaustive enumeration, or simulation of the stated policy, as recorded where they occur.

2 The Static Theory: Representation Adequacy and One-Shot Certification This chapter develops the first two layers of the theory: the static representation layer, which defines decision-theoretic adequacy and decomposes its cost exactly (Section 2.2), and the one-shot certification layer, which asks whether adequacy can be certified from internal transcripts alone and prices an external verification channel when it cannot (Sections 2.3– 2.5). Everything here is static in two senses: transcript laws are fixed independently of the agent’s decisions, and certification is a single purchase decision. The sequential problem— buying accuracy δ at minimum task cost over time—is taken up in Section 3, which builds on the threshold characterization of Theorem 6. 6

Self-Certification of Representation Adequacy

2.1 The Certification Framework Single-shot decision problem. Let the history space H be finite, with elements h ∈ H called histories (or states), and let the action set A be finite. An environment E specifies a distribution pE on H and a loss ℓE : H × A → [0, 1] (and, in the sequential model of Section 3, a transition law). A (possibly randomized) P decision P rule is a map δ : H → ∆(A) into the simplex over A, with risk R(δ; pE , ℓE ) := h pE (h) a δ(a | h) ℓE (h, a); the Bayes risk of observing the full history is X ∗ RE (H) := min R(δ; pE , ℓE ) = pE (h) min ℓE (h, a), (3) δ

h∈H

a∈A

where the minimum over all randomized rules is attained deterministically: for each h, P the inner sum a δ(a | h) ℓE (h, a) is a convex combination of the numbers {ℓE (h, a)}a∈A , hence bounded below by mina ℓE (h, a), with equality whenever δ(· | h) is supported on arg mina ℓE (h, a), which is nonempty since A is finite. Bayes risk as a difficulty measure is standard (DeGroot, 1970; Blackwell and Girshick, 1954). Representation and aliasing cells. A representation map is a deterministic f : H → U into a representation space U ; the aliasing cell of u ∈ U is Hu := {h ∈ H : f (h) = u}.

(4)

The cells partition H; two histories in one cell are indistinguishable under the agent’s representation (aliasing). An f -based rule δ̃ : U → ∆(A) acts as δ(h) = δ̃(f (h)), constant within cells, and its Bayes risk is X   ∗ RE (U ) := min pE (h) Ea∼δ̃(f (h)) ℓE (h, a) . (5) δ̃:U →∆(A)

h∈H

∗ (U ) ≥ R∗ (H); the gap is the aliasing regret, Since {δ̃ ◦ f } ⊆ {δ : H → ∆(A)}, one has RE E decomposed cell by cell by Theorem 1.

The certification quintuple. The meta-decision problem “can the agent certify the adequacy of its own representation” is specified by the quintuple ⟨ E, F, T , V, c ⟩.

(6)

The environment class E collects the candidate environments; |E| ≥ 2 with label disagreement is the premise of every unidentifiability result below (a singleton class makes adequacy directly computable, Proposition 5(b)(i)). The representation class F contains deterministic maps; f ∈ F is fixed and known throughout—we certify a given representation, we do not select one. The internal transcript channel T is the cost-free information source: t periods of interaction produce the internal transcript Tt := (X1 , A1 , . . . , Xt , At ),

(7)

with Xs the period-s observation and As ∈ A the period-s action, and PEt the induced transcript law under environment E. 7

Huang

Key assumption (no loss feedback). Except where explicitly stated, Tt contains no feedback on the realized loss ℓE . Every unidentifiability result in this chapter sources from this assumption. The external verify channel V: the action verify costs c ≥ 0 and returns a signal Y ∼ Q(· | E) on a signal space Y, with Q known; perfect verify (Y reveals E, hence the adequacy label) and non-augmenting verify (Q(· | E) does not depend on E) are the two recurring special cases. Theorem 6 additionally uses the horizon T and the per-period aliasing regret r > 0. The adequacy label. Certification tests a label, and the choice of label is a frameworklevel decision. This paper tests decision-theoretic adequacy,  ∗  ∗ AdeqE (f ) := 1 RE (U ) = RE (H) ⇐⇒ aliasing regret = 0, (8) rather than the environment identity itself. Adequacy is risk-level, not information-level: f may discard much about h and remain adequate whenever the optimal attainable risk is unchanged; and it is task- and environment-dependent through both pE and ℓE —there is no “absolutely adequate” representation. Certification is non-trivial only when the label Θ(E) := AdeqE (f ) takes both values on E. Label priors and aggregated mixtures. Certification is a binary testing problem over the label Θ(E) = AdeqE (f ) ∈ {0, 1}, whatever the size of the environment class. Let E be an arbitrary finite class with prior q; the prior induces the label prior πi := q(Θ = i) and the label-aggregated transcript mixtures X P̄it := P (Tt ∈ · | Θ = i) = q(E | Θ = i) PEt , i ∈ {0, 1}, (9) E: Θ(E)=i

through which every bound in this chapter is stated. The two-point class E = {E0 , E1 } with disagreeing labels is the special case |E| = 2, where each mixture reduces to a single transcript law, P̄it = PEt i . Equal label priors π0 = π1 = 1/2 are a standing assumption wherever the closed form (1−TV)/2 appears (Sections 2.3 and 2.5); Theorem 6(e)(iii) shows this assumption is indispensable to the shape of the closed form, and gives the general-prior formula. Relation to Blackwell sufficiency. The experiment induced by f is a deterministic garbling of the identity experiment, so f is less informative in Blackwell’s order (Blackwell, ∗ (U ) ≥ R∗ (H); but Blackwell sufficiency 1951, 1953; Le Cam, 1964), consistent with RE E requires risk preservation for all losses and priors, whereas AdeqE (f ) requires it only for the fixed pair (pE , ℓE )—the specialization that makes adequacy environment-dependent and certification non-trivial. 2.2 Aliasing Regret Decomposition This subsection states the definitional tool of the static layer—the cell-wise decomposition of aliasing regret, a standard Bayes-risk grouping identity under a partition (attribution below; no novelty claimed)—and fixes the cell-level source of the per-period regret r that Theorem 6 prices. 8

Self-Certification of Representation Adequacy

Theorem 1 (Aliasing regret decomposition). For any environment E and representation f : H → U, #

" ∗ ∗ RE (U ) − RE (H) =

X u∈U

min a∈A

X

pE (h) ℓE (h, a) −

X

pE (h) min ℓE (h, a) .

h∈Hu

h∈Hu

(10)

a∈A

Attribution. A representation map is a deterministic garbling of the identity experiment, ∗ (U ) ≥ R∗ (H) is a direct corollary of the risk-domination characterizaso the inequality RE E tion of Blackwell’s comparison of experiments (Blackwell, 1951; Le Cam, 1964), and (10) is its finite, cell-explicit form; the cellwise equality condition below is the single-period, risklevel analogue of value-preserving aggregation criteria in the state-abstraction literature (Givan et al., 2003; Li et al., 2006). We claim no novelty for the identity itself. Proof of Theorem 1 We proceed in three steps. ∗ (U )). An f -based rule acts within each cell H through a Step 1 (cell expansion of RE u single (possibly mixed) action q(· | u) := δ̃(u) ∈ ∆(A); grouping its expected loss by cell, " X h∈H

pE (h)

X

XX

q(a | f (h)) ℓE (h, a) =

a

q(a | u)

u∈U a∈A

# X

pE (h) ℓE (h, a) .

h∈Hu

The bracket for cell u depends only on q(· | u), so the minimization over all f -based rules separates into independent cellwise minimizations. For each cell, the mix P P p (h)ℓ (h, a) is a convex combination, with weights q(a | u) ture risk E E h∈Hu a P q(· | u), of the pure-action cell risks p (h)ℓ (h, a); a convex combination of E h∈Hu E real numbers is bounded below by their minimum, so the cell contribution is at least P mina h∈Hu pE (h)ℓE (h, a),Pwith equality exactly when q(· | u) is supported on the cell’s Bayes action set arg mina h∈Hu pE (h)ℓE (h, a) (nonempty, A finite). Hence the minimum over the class of all—including randomized—f -based rules is attained by a deterministic rule, and X X ∗ RE (U ) = min pE (h) ℓE (h, a). (11) u∈U

a∈A

h∈Hu

∗

∗

P Step 2 (cell expansion of RE (H)). By the deterministic attainment of (3), RE (H) = h∈H pE (h) mina ℓE (h, a); regrouping this finite sum by cell, ∗ RE (H) =

X X

pE (h) min ℓE (h, a).

u∈U h∈Hu

a∈A

(12)

Step 3 (subtraction). Subtracting (12) from (11), both sides being sums over u ∈ U , yields (10) term by term. The proof uses only finiteness of H and A (all minima attained, all sums finite); the identity holds verbatim for any real-valued loss on finite spaces, the restriction ℓE ∈ [0, 1] being used only later, when per-period regrets are compared across environments in Theorem 6. Two properties are read off (10) by comparing the two terms within each cell. 9

Huang

Property (i): nonnegativity P and the equality condition. Fix a cell u and a cell Bayes action a∗u ∈ arg mina h∈Hu pE (h)ℓE (h, a). For every h ∈ Hu , mina ℓE (h, a) ≤ ℓE (h, a∗u ) by the definition of the pointwise minimum; multiplying by pE (h) and sum∗ (U ) ≥ R∗ (H). Equality ming over h ∈ Hu shows each cell term is nonnegative, hence RE E ∗ (U ) = R∗ (H) holds if and only if every cell term vanishes, and a single cell term vanRE E ishes if and only if there exists an action a∗ with a∗ ∈ arg mina ℓE (h, a) for all h ∈ Hu with pE (h) > 0: if such an a∗ exists, the cell Bayes action achieves the pointwise minimum on all positive-mass histories and the two terms coincide; conversely, if no action is simultaneously Bayes on all positive-mass histories, then for every candidate a′ some positive-mass h has ℓE (h, a′ ) > mina ℓE (h, a) and the cell term is strictly positive. Zero-mass histories neither contribute nor constrain, which is why adequacy depends on pE as well as ℓE . Property (ii): positive contribution ⇐⇒ action conflict. Say cell u exhibits an action conflict if its positive-mass histories share no common Bayes action. By (i), the cell’s contribution is strictly positive if and only if a conflict occurs. Together, (i) and (ii) give the cell-level characterization used throughout: AdeqE (f ) = 1 ⇐⇒ within every cell, histories share a Bayes action pE -a.s.

(13)

This reduces adequacy to a risk-level event—“no cell contains an action conflict”—and thereby reduces certification to testing (Section 2.3). Proposition 2 (Binary closed form). Suppose cell Hu contains exactly twoP histories, written h = 0, 1; A = {0, 1}; and ℓE (h, a) = 1[a ̸= h]. With cell mass m := h∈Hu pE (h) and within-cell posterior p := P (h = 1 | u), the cell’s contribution to (10) is m · min(p, 1 − p),

(14)

maximized at p = 1/2 with value m/2. Proof of Proposition 2 For each history, mina ℓE (h, a) = 0 (take a = h), so the second cell term in (10) vanishes. For the first term, action a = 1 errs exactly on h = 0, giving cell risk pE (0) = m(1 − p); action a = 0 errs exactly on h = 1, giving cell risk pE (1) = mp. Hence the cell contribution is min{mp, m(1 − p)} = m min(p, 1 − p). Finally, min(p, 1 − p) is the pointwise minimum of two affine functions of p ∈ [0, 1], hence concave, strictly below 1/2 on both sides of p = 1/2, and equal to 1/2 there. Attribution. Equation (14) is the standard binary Bayes-error form L∗ = E[min(η(X), 1− η(X))] (Devroye et al., 1996, Theorem 2.1) restricted to one cell. The reading that matters later: aliasing is most destructive at p = 1/2, where any constant action errs on half the cell mass, and harmless at p ∈ {0, 1}; the balanced cell is the canonical worst case in the constructions of Theorem 6. Remark (Relation to state aggregation and to weighted forgetting). The equality condition of property (i) overlaps with value-preserving state aggregation (Givan et al., 2003; Li et al., 2006); (13) is its single-period, risk-level statement, and we claim no novelty. The difference is usage: those works check the condition forward under a known model, whereas we fix it as the label AdeqE (f ) and ask whether it is verifiable from loss-free internal transcripts. 10

Self-Certification of Representation Adequacy

The concurrent preprint of Bennett (2026) proves a cell-level aliasing decomposition of the same type as Theorem 1: its Kρ is the minimal margin-weighted deletion cost making every (representation, test) cell’s optimal bet single-valued, and its own Remark 14 identifies Kρ as a weighted Bayes error on the partition—consistent with our attribution of Theorem 1 as a standard grouping identity. The strict differences are fourfold: (i) loss convention— normalized betting regret with margin weights there, unnormalized Bayes risk difference with ℓ ∈ [0, 1] here; (ii) cells—joint (representation × diagnostic test) cells there, representation cells Hu here; (iii) object—a policy-wise decomposition (realized regret = Kρ+ within-cell misprediction) there, a Bayes-risk difference (optimal U -rule minus optimal H-rule) here; (iv) goal—that work connects weighted weakness to generalization probability and does not study certification; this paper defines the adequacy label from the decomposition and studies its certifiability and the value of external verification. 2.3 One-Shot Impossibility of Internal Certification Can the agent determine the label AdeqE (f ) from Tt alone? For an arbitrary finite environment class the answer is no, with the degree of impossibility controlled exactly by the total variation distance between the two label-aggregated transcript laws. We declare in advance: the theorem of this subsection is a direct application of Le Cam’s two-point method and the Bayes error–TV identity (Tsybakov, 2009; Le Cam, 1986; Devroye et al., 1996); the only new element is the test label, AdeqE (f ) rather than the environment itself. The certification problem. Let E be an arbitrary finite environment class with prior q, f fixed, and Θ(E) = AdeqE (f ) taking both values on E; equal label priors π0 = π1 = 1/2 are assumed wherever the closed form (1 − TV)/2 appears. The data are governed by the label-aggregated mixtures P̄0t , P̄1t of (9)—laws of observation–action sequences only, by the key no-loss-feedback assumption. Definition (Internal certification rule). An internal certification rule is any measurable C : T t → {0, 1} depending only on Tt ; C = 1 certifies “f adequate”, C = 0 rejects. Randomized rules (extra internal randomization independent of environment and data) are allowed. The error events are {C = 1} under Θ = 0 (false certification) and {C = 0} under Θ = 1 (false rejection). Structurally this is two-point testing between the mixtures P̄0t and P̄1t , whose error bounds are given completely by Le Cam’s method (Le Cam, 1986; Tsybakov, 2009); the sole difference from the classical setup is the label: not the environment identity E but AdeqE (f ), environments sharing a label being aggregated into one mixture. Conditioning on the binary label collapses any finite class to a two-hypothesis problem, so the classical closed form survives for every finite E—in contrast to multi-class environment identification, which no single pairwise TV governs (see the remark after Theorem 6). Theorem 3 (TV lower bound for internal certification). In the setup above, write TV(P, Q) := supB |P (B) − Q(B)| over measurable events. For any internal certification rule C (including randomized rules),  max P̄0t (C = 1), P̄1t (C = 0) 11

≥

1 − TV(P̄0t , P̄1t ) . 2

(15)

Huang

Under equal label priors π0 = π1 = 1/2, the Bayes error satisfies err(C) := 12 P̄0t (C=1) + 12 P̄1t (C=0) ≥

1 − TV(P̄0t , P̄1t ) , 2

(16)

with equality attained by the likelihood-ratio test. In particular, if P̄0t = P̄1t , every internal certification rule has Bayes error exactly 21 —no better than random guessing. Attribution. The minimax bound (15) is Theorem 2.2(i) of Tsybakov (2009), resting on the Scheffé identity (their Lemma 2.1), applied here to the label-aggregated mixtures; the equal-prior identity also appears in §3.2 of Devroye et al. (1996). The only contribution of this paper here is the label. Proof of Theorem 3 Write P0 = P̄0t and P1 = P̄1t . For a randomized rule, condition on each realization of its internal randomization: every inequality below holds pointwise on each realization, and taking expectation over the randomization preserves the inequalities; hence it suffices to treat deterministic C. Let A = {C = 1}. Step 0 P (conditioning on the label). By the definition (9) and the tower property, P (C=1 | t t Θ=0) = E:Θ(E)=0 q(E | Θ=0) PE (C=1) = P̄0 (C=1), and likewise P (C=0 | Θ=1) = P̄1t (C=0). Hence err(C) is exactly the equal-prior Bayes error of C for the two-hypothesis problem (P0 , P1 ), whatever the size of E; no step below uses the cardinality of E. Step 1 (sum of the two error probabilities). Substituting P1 (Ac ) = 1 − P1 (A) and then the definition of TV,
P0 (A) + P1 (Ac ) = 1 − P1 (A) − P0 (A) ≥ 1 − sup P1 (B) − P0 (B) = 1 − TV(P0 , P1 ). (17) B

Step 2 (minimax bound). The maximum of two  numbers is at least their average, so (17) gives max{P0 (A), P1 (Ac )} ≥ P0 (A) + P1 (Ac ) /2 ≥ (1 − TV)/2, which is (15).   Step 3 (Bayes bound). Under equal label priors, err(C) = 21 P0 (A) + P1 (Ac ) ≥ (1 − TV)/2 by (17). Step 4 (achievability). Let p0 , p1 be densities of P0 , P1 with respect to a common dominating measure µ (one may take µ = (P0 + P1 )/2), and consider the likelihood-ratio test A∗ = {p1 > p0 }. Writing out its Bayes error pointwise, the integrand equals p0 on A∗ and p1 on A∗c , and in both cases equals min(p0 , p1 ): Z h Z i ∗ 1 1 err(A ) = 2 p0 1{p1 > p0 } + p1 1{p1 ≤ p0 } dµ = 2 min(p0 , p1 ) dµ. R R Now substitute the Scheffé identity TV(P0 , P1 ) = 12 |p0 − p1 | dµ = 1 − min(p0 , p1 ) dµ (Tsybakov, 2009, Lemma 2.1), whose second equality follows from the pointwise identity R |p0 − p1 | = p0 + p1 − 2 min(p0 , p1 ) and (p0 + p1 ) dµ = 2: hence err(A∗ ) = (1 − TV)/2, attaining the lower bound of Step 3, which applies to every rule. The minimal equal-prior Bayes error is therefore exactly (1 − TV)/2. Step 5 (indistinguishable case). If P0 = P1 then TV = 0, so every rule has Bayes error at least 21 by Step 3; the constant rules C ≡ 0 and C ≡ 1 each attain exactly 12 (one error probability is 1, the other 0). Hence every rule has Bayes error exactly 12 : no rule beats random guessing.

12

Self-Certification of Representation Adequacy

Corollary (Internal unidentifiability). If P̄0t = P̄1t , then for every t and every internal certification rule C, the equal-label-prior Bayes error is exactly 21 : internal transcripts cannot statistically separate “f adequate” from “f inadequate”. This case is non-vacuous, because transcript laws depend only on observation–action sequences while AdeqE (f ) also depends on losses. Explicit construction (the two-point special case |E| = 2): let H = {h0 , h1 } be aliased into a single cell by a constant f , with the cell index as the only observation and identical policies in both environments, so the transcript laws coincide, PEt 0 = PEt 1 for every t; let E0 carry the loss ℓ(h, a) = 1[a ̸= h] (the cell contains an action conflict, so AdeqE0 (f ) = 0 by property (ii) of Theorem 1) and E1 carry ℓ(h, a) = 1[a ̸= 0] (action a = 0 is Bayes on both histories, so AdeqE1 (f ) = 1). Then AdeqE0 (f ) ̸= AdeqE1 (f ) while P̄0t = P̄1t : adequacy is unidentifiable from internal data in the strong sense of the corollary. Information-theoretic form. Under equal priors, the same distinguishability has an exact information-theoretic statement. With base-2 logarithms (units of bits), the Jensen– Shannon divergence is (Lin, 1991)
1
JS(P, Q) := 21 KL P P +Q + 2 KL Q P +Q , (18) 2 2 R where KL(P ∥Q) := p log2 (p/q) dµ for densities p, q against a common dominating µ. Proposition 4 (Mutual-information form). Under equal label priors π0 = π1 = 12 , I(Θ; Tt ) = JS(P̄0t , P̄1t ).

(19)

Consequently P̄0t = P̄1t implies I(Θ; Tt ) = 0: the internal transcript carries no information whatsoever about adequacy. Attribution. Proposition 4 is a direct rewriting of the JS-divergence definition (Lin, 1991) for an equal-prior two-component mixture, not a new information-theoretic principle; it makes explicit that certification at P̄0t = P̄1t is information-theoretically impossible (I(Θ; Tt ) = 0), not merely worst-case hard. Proof of Proposition 4 Write M = (P̄0t + P̄1t )/2, let p0 , p1 be the densities of P̄0t , P̄1t against a dominating µ, and set η(t) := p1 (t)/(p0 (t) + p1 (t)). Under the equal label prior, η(t) is exactly the posterior P (Θ = 1 | Tt = t), and the Radon–Nikodym derivatives against M are dP̄0t /dM = 2(1 − η) and dP̄1t /dM = 2η. Substituting into (18) and expanding log2 (2u) = 1 + log2 u term by term, Z h Z i
  JS(P̄0t , P̄1t ) = (1 − η) log2 2(1 − η) + η log2 (2η) dM = 1 − h2 (η) dM, (20) where h2 (u) = −u log2 u − (1 − u) log2 (1 − u) is the binary entropy function and the last step uses (1−η)+η = 1. On the other hand, by the entropy representation of mutual information, the equal label prior H(Θ) = h2 ( 21 ) = 1, and the marginal law of the transcript being M , Z
I(Θ; Tt ) = H(Θ) − Et∼M h2 P (Θ=1 | Tt =t) = 1 − h2 (η) dM. (21) 13

Huang

Comparing (21) with (20) yields (19). Finally, JS(P, Q) = 0 if and only if both KL terms vanish; each KL divergence is nonnegative and vanishes only at equality of its arguments (Jensen’s inequality applied to the strictly convex − log2 ), so JS(P̄0t , P̄1t ) = 0 if and only if P̄0t = P̄1t , in which case I(Θ; Tt ) = 0. 2.4 The Honest Boundary: A Counterexample and Possibility Conditions Theorem 3 is a binary testing statement about two observed, label-conditional transcript laws. Its theorem-level requirements are that both adequacy labels occur with positive prior mass and that the certifier is restricted to the stated transcript; the bound then holds regardless of which variables the transcript contains. The loss-free, aliased construction above is an important mechanism for making the two laws coincide: adequacy depends on losses, while the transcript need not reveal them. It is not an additional hypothesis of the total-variation inequality. The counterexample of part (a) is load-bearing: an early stage of this research program conjectured a closure claim for all finite-state systems, and the counterexample below refutes it in a precise form. We report the refutation in the body, not an appendix, because it pins down the actual scope of the claim. Proposition 5 (Boundary and counterexample package). (a) (Counterexample A.) There exists a finite Markov decision process whose optimal actions change the world state with probability one every period, yet whose state representation is decision-theoretically adequate and internally certifiable; hence the closure claim “for all finite-state systems, if the agent’s actions can change the world state, then the system cannot internally certify the decision-theoretic adequacy of its state representation” is false. (b) (Sufficient conditions.) Adequacy is certifiable or directly computable whenever any one of the following holds: (i) the environment class is a singleton (the model is known); (ii) the transcript carries loss feedback, with structure permitting counterfactual comparison; (iii) the representation is aliasing-free (f bijective, or the state fully observable). (c) (Belief-state note.) In partially observable environments the belief state is a sufficient information state for optimal control (Åström, 1965; Kaelbling et al., 1998); representation inadequacy can therefore arise only when the agent’s representation is coarser than the belief state. Proof of (a): Counterexample A, full analysis. Construction. Consider the finite MDP with world state Wt ∈ {0, 1} fully observed (Zt = Wt ), action set A = {0, 1}, loss ℓ(W, a) = 1[a ̸= W ],

(22)

Wt+1 = 1 − At ,

(23)

and transition law so the next world state is fully determined by the current action, independently of the current state. The representation f is the identity map, and the environment class E is a singleton: the model (22)–(23) is fully known to the agent. The value-iteration argument below follows the standard treatment of finite MDPs in Sutton and Barto (2018). Analysis. The Bellman optimality equation for this system is n o V ∗ (w) = min 1[a ̸= w] + V ∗ (1 − a) , w ∈ {0, 1}. (24) a∈{0,1}

14

Self-Certification of Representation Adequacy

Substituting V ∗ ≡ 0 into the right-hand side gives mina 1[a ̸= w] = 0, matching the lefthand side, so V ∗ ≡ 0 solves (24). To confirm optimality without appeal to uniqueness of the Bellman solution: the per-period loss is nonnegative under every policy, so the expected total loss of any policy is bounded below by 0; the policy a∗t = Wt attains loss 0 in every period (its action always equals the current state), hence it is optimal and the optimal value function is identically zero. Along the optimal trajectory a∗t = Wt , the transition law (23) gives Wt+1 = 1 − At = 1 − Wt , and therefore
P Wt+1 ̸= Wt = 1 (25) in every period: the agent’s optimal action changes the world state with probability one. Meanwhile, every aliasing cell of the identity f is a singleton, so no cell contains an ac∗ (U ) = R∗ (H), i.e. Adeq (f ) = 1. tion conflict; by Theorem 1 every cell term vanishes, RE E E Moreover, since the environment class is a singleton and the model is known, the agent computes AdeqE (f ) directly—both Bayes risks are explicit optimizations over known objects— without consulting any transcript statistics. Internal certification is therefore trivially feasible. The falsified claim, stated precisely. Claim (falsified). For all finite-state systems: if the agent’s actions can change the world state, then the system cannot internally certify the decision-theoretic adequacy of its state representation. Counterexample A satisfies the antecedent—by (25), the optimal policy changes the world state with probability one every period—and violates the consequent—AdeqE (f ) = 1, computable internally. A closure claim for all finite-state systems whose antecedent holds and whose consequent fails on a single instance is false; the historical heuristic conjecture referred to as the “state-closure paradox” is refuted in precisely this form. The refutation does not touch Theorem 3: a singleton, known environment class induces no label-discrimination problem at all. It also lies outside the loss-free aliasing construction used above, because f is the identity. Any meaningful replacement for the rejected universal claim must state an explicit observational indistinguishability premise, not merely that actions can alter the world. Proof of (b): three sufficient conditions for certifiability. Each condition describes a regime in which adequacy is computable or the observed transcript can become informative. They are useful exits from the motivating loss-free aliasing construction, but they are not logical negations of the hypotheses of Theorem 3. (i) Singleton class (known model). If |E| = 1, then pE , ℓE (and any transition law) are known, and ∗ (U ) = R∗ (H)] is directly computable: both Bayes risks are explicit opAdeqE (f ) = 1[RE E timizations over known objects. No hypotheses need to be distinguished, the two-point premise of Theorem 3 fails, and the certification problem does not arise; Counterexample A falls into the possibility region exactly along this route. (ii) Loss feedback. If the internal transcript contains realized losses, the realized loss of the f -based rule is an observable sequence, and under additional structure permitting counterfactual comparison—the agent can estimate, within each cell, the losses of the actions it did not take—adequacy testing can be converted into online estimation of the aliasing regret. Such feedback may make 15

Huang

the two label-conditional laws distinguishable, but it does not do so automatically; Theorem 3 still applies to the resulting observed laws. By way of contrast, windowed adaptive methods for concept drift such as the FLORA family of Widmer and Kubat (1996) presuppose observable prediction-error signals to detect environmental change; their feasibility rests precisely on loss feedback and is orthogonal to the no-loss-feedback assumption of this paper. (iii) Aliasing-free representation. If f is a bijection—in particular, if the state is fully observable and f is the identity—every cell is a singleton, no cell contains an action conflict, and by Theorem 1 the aliasing regret vanishes identically: AdeqE (f ) ≡ 1 holds a priori as a definitional fact and requires no certification. These examples are not a converse characterization. For Theorem 3, a non-vacuous certification problem needs both labels to occur and needs the two label-conditional transcript laws; the numerical lower bound is determined entirely by their total variation. A multi-environment class with label disagreement, a loss-free internal record, and an aliased representation describe a central ESD route to indistinguishability, but loss feedback may still be insufficient without counterfactual structure, and aliasing alone need not make the two transcript laws close. Conversely, any feedback mechanism that changes those laws simply changes the total variation in the theorem. □ Proof of (c): belief-state note. Åström (1965) proved that in a partially observable Markov process the conditional distribution of the state given the action–observation history—the belief state—is a sufficient information state, so the partially observable optimal control problem can be reformulated on the space of these conditional distributions; Kaelbling et al. (1998) formalized the resulting belief-MDP reduction for planning under partial observability, and Krishnamurthy (2016) gives a comprehensive modern treatment including controlled sensing. It follows that representation inadequacy can occur only when the representation the agent actually maintains is strictly coarser than the belief state—for instance, when a memory system applies lossy semantic compression on top of the history. The certification problem of this paper is therefore not a consequence of partial observability itself, but of an additional lossy representation layer below the belief state; this delimits the problem domain precisely. □ 2.5 The Value of One-Shot External Verification We now state the main theorem of the static theory: on an arbitrary finite environment class, under equal label priors and symmetric alignment, the value of an external verify action is characterized by a TV-weighted closed-form threshold. The claim is scoped in advance to the combination “adequacy label + channel augmentation + threshold characterization”; every component tool is standard and is credited where used. Setup. E is an arbitrary finite environment class with prior q, f fixed, and the label Θ(E) = AdeqE (f ) taking both values; the induced label prior is equal, π0 = π1 = 1/2 (standing assumption; item (e)(iii) of the theorem shows it is indispensable to the closedform shape). Transcript laws enter through the label-aggregated mixtures P̄0t , P̄1t of (9); the two-point class is the special case |E| = 2. By property (ii) of Theorem 1, in every inadequate environment (Θ = 0) the f -rule incurs a per-period aliasing regret r > 0, whose cell-level source is the decomposition (10); in every adequate environment (Θ = 1) it incurs 16

Self-Certification of Representation Adequacy

per-period regret 0. The transcript laws PEt are fixed and independent of the switching decision (transcript exogeneity); certification is performed once, after which the policy is fixed for the T -period horizon (one-shot certification). Assumption (A1 (symmetric alignment)). Certification drives a policy switch: C = “adequate” ⇒ use the f -rule; C = “inadequate” ⇒ use a fallback rule. Matched rule–label pairs have per-period regret 0 (the f -rule under any environment with Θ = 1, by AdeqE (f ) = 1; the fallback under any environment with Θ = 0, by construction); mismatched pairs incur per-period regret exactly r > 0, equal in both directions: misuse of the f -rule in an inadequate environment costs the same r per period as misuse of the fallback in an adequate environment. “Optimal” in item (a) below means optimal within the A1 certification–switching scheme class. The fallback rule of A1 is exogenously given, known to be optimal in inadequate environments; its information source and acquisition cost lie outside the model (for instance, supplied by a designer or obtained through an independent fully observable channel). Theorem 6 prices the certification of the label, not the acquisition of the fallback. Asymmetry is why A1 must be symmetric: with mismatch costs r0 ̸= r1 in the two directions, the expected total regret of any certifier is i Th r0 P̄1t (C=0) + r1 P̄0t (C=1) , (26) 2
R a weighted testing problem whose optimum T2 min r0 dP̄1t , r1 dP̄0t , attained by the weighted likelihood-ratio test A∗ = {r1 p̄0 > r0 p̄1 } (the pointwise argument of Theorem 3, Step 4), depends on the full likelihood-ratio geometry and the ratio r0 /r1 , not on the scalar TV alone; the closed form (1 − TV)/2 and the threshold of item (b) then fail. R(C) =

The verify channel. verify costs c ≥ 0 and returns Y ∼ Q(· | E): perfect (Y reveals the label) or noisy, in which case we assume Y is conditionally independent of the internal transcript given E, so that under label i the agent faces the label-aggregated product law J¯i :=

X

  q(E | Θ = i) PEt ⊗ Q(· | E) ,

i ∈ {0, 1}.

(27)

E: Θ(E)=i

Theorem 6 (Decision-theoretic value of external verification). In the setup above (arbitrary finite E, equal label priors π0 = π1 = 1/2, A1), write TV := TV(P̄0t , P̄1t ) over the labelaggregated mixtures (9) and B ∗ (·) for the minimal equal-label-prior Bayes error of the binary label test on the experiment indicated. Then: (a) The internal-only optimal expected total regret is ∗ Rint =

Tr 1 − TV(P̄0t , P̄1t ) , 2

(28)

attained within the A1 scheme class by the certifier induced by the likelihood-ratio test (Tsybakov, 2009; Devroye et al., 1996). 17

Huang

Table 1: Case bookkeeping for A1 (symmetric alignment) in the proof of Theorem 6(a). Matched rows incur zero per-period regret; mismatched rows incur per-period regret r, hence total regret T r over the horizon. Label Θ

Certification

Rule used (A1 switch)

Per-period / total regret

1 (adequate) 1 (adequate) 0 (inadequate) 0 (inadequate)

C=1, correct C=0, error C=1, error C=0, correct

f -rule (match) fallback (mismatch) f -rule (mismatch) fallback (match)

0/0 r/Tr r/Tr 0/0

(b) The net value of a perfect verify is Vverify =

Tr (1 − TV) − c, 2

(29)

and verify strictly dominates internal-only certification if and only if c < c∗ :=

Tr 1 − TV(P̄0t , P̄1t ) . 2

(30)

(c) Two extremes: TV = 0 (P̄0t = P̄1t ) gives c∗ = T r/2, verification maximally valuable; TV → 1 gives c∗ → 0, internal certification asymptotically sufficient and verify valueless. (d) Noisy verify: with Y conditionally independent of Tt given E (product laws (27)), the (1 − TV) factor in the threshold is replaced by the marginal distinguishability gain ¯ ≥ 0, B ∗ (P̄ t ) − B ∗ (J)

(31)

  ¯ . If i.e. noisy verify strictly dominates if and only if c < T r B ∗ (P̄ t ) − B ∗ (J) Q(· | E) does not depend on E (the signal is uninformative about the label and verify does not augment the experiment), the gain is 0 and verify should not be used at any c > 0 (minimal counterexample) (Blackwell, 1951; Le Cam, 1964). (e) Indispensability: removing any one of (i) the A1 symmetry, (ii) the experimentaugmentation property of verify, (iii) the equal label prior invalidates the formula of (a) or the closed-form threshold of (b); the counterexamples are constructed in Appendix C. For (iii), a closed form survives under a general label prior: with π0 ̸= π1
R the minimalR Bayes error of the
label test is min π0 dP̄0t , π1 dP̄1t and the threshold is c∗ = T r min π0 dP̄0t , π1 dP̄1t ; what is lost is the (1 − TV)/2 shape, in which the prior cancels. Proof of Theorem 6 (a). Take any certifier C in the A1 scheme class. The certification output selects a rule through the A1 switch, and the adequacy label determines whether the selection matches; Table 1 spells out the four joint cases. 18

Self-Certification of Representation Adequacy

As Table 1 shows, the regret of the whole certification–switching scheme is governed entirely by the certification error event: in the two matched rows the per-period regret is 0 (in the first row by Θ = 1, i.e. AdeqE (f ) = 1 for every environment carrying that label; in the fourth by the construction of the fallback under A1), while in the two mismatched rows it is exactly r, the symmetric clause of A1 asserting that the two mismatch directions carry the same price r. Since certification is performed once and the policy is then fixed for the whole horizon, the total regret is T · r on the error event and 0 on its complement, so the expected total regret of C is T · r · P (err),

P (err) = 21 P̄0t (C=1) + 21 P̄1t (C=0) = err(C),

exactly the equal-label-prior Bayes error of Theorem 3. By that theorem, P (err) ≥ (1 − TV)/2, so the expected total regret of C is at least T2r (1 − TV). Attainability: Step 4 of the proof of Theorem 3 shows the likelihood-ratio test A∗ = {p̄1 > p̄0 } attains Bayes error exactly (1 − TV)/2; coupling it with the A1 switch yields expected total regret exactly (28). The underlying identity is the standard Le Cam two-point / Bayes error–TV relation (Tsybakov, 2009; Devroye et al., 1996); the only addition is the transcription, via A1, of error rate into regret units, with the label taken to be AdeqE (f ). (b). A perfect verify fully reveals the adequacy label, so the certification error is zeroed; by A1 the matched rule is used under each label, the total regret is 0, at the one-time cost c. The net value is the internal-only benchmark minus the cost, ∗ Vverify = Rint −c=

Tr (1 − TV) − c, 2

which is (29), and Vverify > 0 if and only if c < c∗ with c∗ as in (30). This is a direct instantiation of the value-of-information structure “value = with-information benchmark − without-information benchmark − cost,” evaluated before the information is acquired (Howard, 1966; DeGroot, 1970). (c). Substitute the extremes of TV into (30): at TV = 0, c∗ = T r/2, the maximum of the net-value expression since 1 − TV ≤ 1—this is the case in which internal certification cannot beat random guessing (Theorem 3, Step 5), so external verification recovers the entire aliasing loss; as TV → 1, the optimal internal-only expected regret tends to 0, so there is asymptotically nothing left for verification to recover. (d). The joint observation is the pair (Tt , Y ), whose law under label i is J¯i of (27). Applying to (Tt , Y ) the Markov kernel that discards Y —keeping only Tt —maps J¯i to P̄it for each i ∈ {0, 1}; hence the experiment {J¯0 , J¯1 } is at least as informative as {P̄0t , P̄1t } in Blackwell’s order, and by the comparison theorem of Blackwell (1951) a more informative experiment has pointwise no larger Bayes risk on every decision problem (Blackwell, 1951; ¯ ≤ B ∗ (P̄ t ), i.e. Le Cam, 1964). Specializing to the equal-label-prior binary test gives B ∗ (J) the gain (31) is nonnegative. With noisy verify used, the agent performs the one-time optimal certification on the joint experiment (A1 unchanged), incurring expected total ¯ + c; relative to the internal-only benchmark T r B ∗ (P̄ t ), the net value is regret T r B ∗ (J) ¯ − c, strictly positive if and only if c < T r [B ∗ (P̄ t ) − B ∗ (J)]. ¯ Vverify = T r [B ∗ (P̄ t ) − B ∗ (J)] t ¯ Finally, if Q(· | E) = Q(·) is the same in every environment, then Ji = P̄i ⊗ Q for both labels; writing p̄j for the density of P̄jt and q for that of Q, the posterior odds of Θ = 1 19

Huang

against Θ = 0 given (t, y) equal p̄1 (t)q(y)/[p̄0 (t)q(y)] = p̄1 (t)/p̄0 (t) whenever q(y) > 0— the factor q(y) cancels—so the posterior given (Tt , Y ) coincides with the posterior given ¯ = B ∗ (P̄ t ), the gain is 0, and Vverify = −c < 0 at any c > 0: verify Tt alone, B ∗ (J) should not be performed. This identically-distributed verify is simultaneously the minimal counterexample to (b). (e). The explicit constructions are deferred to Appendix C; we summarize each item and keep the self-contained case (ii) in full. (i) Drop A1 symmetry (construction in Appendix C.1). There is a completely explicit two-point instance (|E| = 2, f constant onto a single representation value) in which the fallback rule coincides with the f -rule’s cell Bayes action under the adequate environment, so that mismatch direction costs 0, not r: “always fallback, never certify” has total regret identically 0 under both environments, yet TV(P01 , P11 ) = 21 < 1 and (28) prescribes the strictly positive value T4r . The formula no longer describes the optimum, and the threshold of (b) collapses with it; the counterexample bites exactly when the internal channel is not perfectly distinguishing. The full construction and its arithmetic are given in Appendix C.1. (ii) Drop augmentation. If Q(· | E0 ) = Q(· | E1 ), the final paragraph of the proof of (d) gives gain 0 and Vverify = −c < 0 at any c > 0: the “buy” conclusion of (b) fails even though the closed-form threshold (30) is well defined—(b) silently presupposes that verify augments the experiment, and an identically distributed signal violates exactly that presupposition. (iii) Drop equal label priors (construction in Appendix C.2). Under a general label Rprior π the pointwise Bayes-error expansion gives the weighted threshold c∗ =
t t T r min π0 dP̄0 , π1 dP̄1 , attained by the weighted likelihood-ratio test: still a closed form, but one in which the prior R sits inside the minimum. The equal label prior is exactly the case in which it cancels, 21 min(p̄0 , p̄1 ) dµ = (1 −TV)/2, the shape of (a) and (b). That the shape, not merely the value, is genuinely lost is witnessed by a cyclic three-environment instance on which the equal-prior shape prescribes 41 while the true optimum is 13 ̸= 14 ; the general derivation and the complete cyclic computation are given in Appendix C.2. Remark (Why cost additivity is not among the assumptions). A perfect verify exhausts the label’s information in a single purchase: conditional on a fully revealing signal, the posterior over Θ degenerates to a point mass, so any repeat purchase of the same perfect channel has marginal information gain exactly zero, at any price. Consequently, under any nondecreasing cost structure γ(k) for k purchases—additive or not—the optimal number of perfect verifications is at most one, and the purchase decision compares the single-purchase cost γ(1) against the internal-only benchmark of (a), which is exactly the threshold comparison of (b) with c = γ(1). Cost additivity therefore plays no role in Theorem 6 and does not appear among the assumptions in (e); a cost structure becomes decision-relevant only in extended models where verification is noisy and repeatable, an open direction connected to the sequential model of Section 3. Relation to classical VOI; scoped contribution; numerical check. Theorem 6 instantiates the with-minus-without information value of Howard (1966) and DeGroot (1970), differing in the decision object (purchasing verification of one’s own representation adequacy), in sharpness (a (1 − TV)-weighted closed-form threshold), and in boundary treatment (the TV extremes and the counterexamples of (e) are part of the result). Its claim 20

Self-Certification of Representation Adequacy

is limited to the combination “labeling + channel augmentation + threshold characterization”: the component tools (Tsybakov, 2009; Devroye et al., 1996; Blackwell, 1951; Le Cam, 1964; Howard, 1966; DeGroot, 1970) are standard, and (e) marks the validity boundary. All identities and thresholds of this chapter were machine-checkably verified on finite instances (algebraic identities to below 10−12 error, or by exhaustive pointwise enumeration; for Theorem 6, c∗ and the switching decision were checked pointwise in three TV regimes, with indifference at c = c∗ , and the cyclic-class computations of (e)(iii) and of the remark in Appendix C.3 were checked by exact rational arithmetic and exhaustive enumeration over all 23 deterministic label rules). 2.6 Instantiation: Decision-Theoretic Memory (ESD) We instantiate the static framework to the Event–State–Decision memory (ESD), in which maintaining a dialogue memory is state-conditioned decision making rather than retrieval. The instantiation involves no dialogue data and no empirical claims. Setup. Fix an environment. A history has two components: the semantic observation X ∈ X (the semantic unit) and a latent state Z ∈ Z, with joint law p(x, z) and loss ℓ((x, z), a) ∈ [0, 1]; the agent’s representation retains only X. Write R∗ (X) and R∗ (X, Z) for the Bayes risks of observing only X and the full pair (X, Z). For each x ∈ X , define the conditional state value (the normalized regret of the cell {x} × Z) X X g(x) := min p(z | x) ℓ((x, z), a) − p(z | x) min ℓ((x, z), a) ≥ 0, (32) a∈A

z

a∈A

z

the conflict set C := {x ∈ X : g(x) > 0} (by property (ii) of Theorem 1, g(x) > 0 if and only if the cell {x} × Z contains an action conflict), and the conflict mass µ := P (X ∈ C). Proposition 7 (µ–g conflict decomposition). In the setup above, X   ∆info := R∗ (X) − R∗ (X, Z) = p(x) g(x) = µ · E g(X) X ∈ C .

(33)

x∈C

Boundary convention: if µ = 0 (C = ∅) then ∆info = 0 and the conditional-expectation term is treated as 0. Attribution. Proposition 7 is a special case and instantiation of Theorem 1; no novelty is claimed. Its nonnegativity also has a classical source: retaining only X garbles (X, Z) deterministically, so ∆info ≥ 0 follows from Blackwell risk domination (Blackwell, 1951), and g(x) matches the cell-restricted binary Bayes error of Devroye et al. (1996, Theorem 2.1). Proof of Proposition 7 Apply Theorem 1 with H = (X, Z) and U = X: the history space is X × Z with law p(x, z), and the representation map is the projection (x, z) 7→ x, whose aliasing cells are exactly {x} × Z, x ∈ X . The cell decomposition (10) then reads " # X X X R∗ (X) − R∗ (X, Z) = min p(x, z) ℓ((x, z), a) − p(x, z) min ℓ((x, z), a) . (34) x∈X

a∈A

z

z

a∈A

For each x with p(x) > 0, factoring p(x, z) = p(x) p(z | x) shows the bracket equals p(x) g(x) with g(x) as in (32); for x with p(x) = 0 the bracket vanishes and p(x)g(x) = 0 21

Huang

P as well. Hence ∆info = x∈X p(x)g(x). By the definition of C, g(x) = 0 for x ∈ / C, so the sum restricts to C, giving the first equalityP of (33). For the second, if µ > 0 write P p(x) = µ · p(x | C) for x ∈ C and factor out µ: x∈C p(x)g(x) = µ x∈C p(x | C) g(x) = µ · E[g(X) | X ∈ C]. If µ = 0 then every summand vanishes, ∆info = 0, and the conditionalexpectation term is treated as 0 by convention. Equation (33) separates the extent of inadequacy (the conflict mass µ: how often the agent lands on a semantic unit whose cell contains an action conflict) from its severity (the conditional expectation E[g(X) | X ∈ C]: the normalized within-cell regret on those units); the certification problem is indifferent to this profile only because AdeqE (f ) is the binary indicator of ∆info = 0. The five-action decision memory; two types of value, kept separate. stantiates the action set A = {speak, silence, ask, verify, defer},

ESD in-

(35)

with readings tied to (33). Off the conflict set (g(x) = 0), property (ii) of Theorem 1 guarantees only that the cell admits a common Bayes action—some action, not necessarily speak; in the ESD instance, whatever the common Bayes action is, it is executed by default. Under conflict (g(x) > 0), silence/ask/defer postpone or transfer the decision. Two of the five actions are queries, and they answer different questions: ask is the state-clarification action, a Vstate -type query about the current state Z; verify is the adequacy-certification action, a Vcert -type query about the label Θ = AdeqE (f ), priced by Theorem 6. The two value types are not conflated, and the connecting chain is exact: ∆info is a Vstate -type measure—the ex ante information value of the state Z for semantic-memory decisions— and its cell-level source µ · E[g(X) | X ∈ C] prices the per-period aliasing regret r that an inadequate representation inflicts (the per-period expected cost of the within-cell action conflicts); Theorem 6 then takes that r as an input and prices certification through c∗ = (T r/2)(1 − TV), a Vcert -type quantity about the label, not the state. Proposition 7 is an identity under a known joint law, not a learning result. 2.7 From One-Shot to Sequential Certification The static theory answers a deliberately narrow question: whether to buy one verify, once, at a closed-form threshold c∗ . Two idealizations bound its reach. First, transcript exogeneity: the laws P̄0t , P̄1t are fixed independently of the agent’s conduct, so the model cannot express that waiting, exploring, or acting differently would change what the internal channel reveals. Second, the one-shot decision: c∗ compares two fixed schemes over a fixed horizon and says nothing about how to buy a prescribed certification accuracy δ at minimum task cost—how long to rely on the internal channel as the transcript lengthens, when to stop waiting and purchase verification, and how these choices interact with the task loss accumulating meanwhile. Both idealizations are load-bearing in the proofs above, and both must be relaxed to address the sequential question. Section 3 takes up exactly this relaxation: it introduces the policy-dependent sequential certification model and proves the matching lower and upper bounds of Theorems 8 and 9, for which the present chapter supplies the label, the TV-based distinguishability measure, and the threshold logic that the 22

Self-Certification of Representation Adequacy

sequential analysis refines. Adjacent tools for controlled observation and active sequential testing exist (Naghshvar and Javidi, 2013; Krishnamurthy, 2016), but they assume exogenous estimable feedback or a known model; the interface to adequacy certification under loss-free transcripts is built in Section 3.

3 The Sequential Certification Model and Main Theorem This section gives the full definition of the sequential certification model, defines the certification complexity constant that governs it, states the two main theorems and their corollary, specifies the matching policy CTS, and records the non-triviality and boundary statements that delimit the results. Proofs of the lower and upper bounds are deferred to Appendices A and B respectively; the boundary statements B1–B3 are proved in Appendix A. 3.1 Model M1: sequential adequacy certification under a fixed representation Model M1 (sequential certification).

The model consists of the following ingredients.

(M1.1) Environments and labels. A finite environment class E = {E1 , . . . , EK } with K ≥ 2 is fixed, together with a fixed representation f . Each environment Ei carries a binary adequacy label  ∗ ∗ Θi := 1 RE (f (H)) = RE (H) ∈ {0, 1}, (36) i i defined through the Bayes-risk framework of the static layer (Section 2): Θi = 1 if acting optimally on the representation achieves the Bayes risk of acting on the full history. The labels are known as a function of the (known) environments; what is unknown to the agent is which environment Ei it is in, and hence the realized label. (M1.2) Actions, channels, losses. The action set A is finite. Each action a ∈ A in environment Ei carries an observation channel Kia , a probability distribution on a finite observation space Y, and a task loss gi (a) ≥ 0, finite. Audit costs are folded into the task loss: a Verify action, Pwhen present, has gi (Verify) equal to its audit price caudit . We write d(P ∥Q) = y∈Y P (y) log(P (y)/Q(y)) for the KL divergence between two distributions on Y, with the conventions 0 log 0 = 0 and d(P ∥Q) = +∞ if P is not absolutely continuous with respect to Q. (M1.3) Interaction protocol. A policy π is a sequence of (possibly randomized, via an independent seed) decision rules; at round t it selects At as a measurable function of the history Ht−1 = (A1 , Y1 , . . . , At−1 , Yt−1 ). In environment Ei , the observation then follows Yt ∼ KiAt , conditionally independent of the past given the action sequence (Assumption (H1) below). Let Ft = σ(A1 , Y1 , . . . , At , Yt ) be the natural filtration. Each pair (Ei , π) induces a probability measure Pπi on interaction sequences, with expectation Eπi . (M1.4) Stopping and decision. The policy commits to a stopping time τ with respect to (Ft ) and a terminal decision Θ̂ ∈ {0, 1} that is Fτ -measurable. 23

Huang

(M1.5) δ-correctness (fixed confidence), including admissibility. For a target confidence δ ∈ (0, 12 ), a policy π is δ-correct if

Pπi τ < ∞ = 1 for every i ∈ {1, . . . , K} and max Pπi Θ̂ ̸= Θi ≤ δ. (37) i

The first requirement is an admissibility condition: certification must terminate under every environment, not only under favorable ones. (The lower bound below is stated for policies with Eπi [τ ] < ∞; Appendix A treats the remaining case by a standard truncation argument.) (M1.6) Cost. The performance measure is the expected cumulative task loss until certification, Eπi

R(π, Ei ) :=

τ hX

i gi (At ) ,

t=1

so that R(π, Ei ) = π δ-correct}.

Na (τ ) :=

X

1{At = a},

(38)

t≤τ

π a gi (a) Ei [Na (τ )].

P

We write Rδ∗ (Ei ) := inf{R(π, Ei ) :

(M1.7) Alternative sets. For each environment Ei , the set of label-disagreeing alternatives is Alt(i) := { j ∈ {1, . . . , K} : Θj ̸= Θi }.

(39)

That the alternatives are defined by the adequacy label (36)—a decision-theoretic property of the representation—rather than by an exogenous hypothesis partition is the modeling feature that distinguishes this formulation from the sequential-testing formulations audited in Section 1.4; it is also what makes same-label environments j∈ / Alt(i) irrelevant to the certification constraint (they matter only for estimation, via Assumption (H2+ )). Action semantics. The abstract action set instantiates the ESD repertoire described in Section 1.1: Act (execute the task policy induced by f ; informative about the label to the extent that task observations reveal aliasing, and lossy exactly when the representation is inadequate), Ask (query the latent state, at a query fee), Verify (purchase an adequacy audit, at the audit price), and Defer (wait, at low or zero loss and low or zero information). The Repair action is excluded from Model M1 deliberately: it changes the representation, hence the label itself, and is treated in Section 4. Assumptions. We collect the standing assumptions, each with a one-line statement of what it buys and where it is used. (H1) Conditional independence of the channels. Given the action sequence, the observations satisfy Yt ∼ KiAt conditionally independent of the past. This makes the loglikelihood ratio of any two environments a sum of action-indexed increments and is used by both bounds. (H2) Feasibility. For every i and every j ∈ Alt(i) there exists an action a with 0 < d(Kia ∥Kja ) < +∞: every label-disagreeing alternative can be statistically separated from Ei by some action. The two failure modes are asymmetric. If some j ∈ Alt(i) satisfies d(Kia ∥Kja ) = 0 for every a, then no amount of evidence separates Ei from Ej and δcertification is impossible; this is the hopeless boundary B1 below, and the LP (40) is 24

Self-Certification of Representation Adequacy

exactly infeasible in this case. The opposite failure, d(Kia ∥Kja ) = +∞ for the available actions, is harmless—a single observation may then separate the two environments perfectly. Theorem 8 in fact only requires Ci < ∞, and hopelessness is characterized exactly by infeasibility of the LP. (H2+ ) Pairwise distinguishability of all environments. For every pair i ̸= j (including same-label pairs) there exists an action a with 0 < d(Kia ∥Kja ): the true environment itself is identifiable. This is stronger than (H2), which constrains only label-disagreeing pairs, and it is needed only for the upper bound, where the maximum-likelihood estimator of the environment must stabilize exactly. The lower bound does not use it. (H3) Losses. 0 ≤ gi (a) < ∞ for all i, a. The degenerate case gi (a) = 0 is handled by continuity of the LP (boundary B2 below). (H4) Full support. Kia (y) > 0 for every i, a, and y ∈ Y. Consequently all log-likelihoodratio increments are uniformly bounded, which the concentration arguments of the upper bound require. The lower bound does not use it. Assumption dependence of the results. Theorem 8 is proved under (H1)–(H3) alone. Theorem 9 uses the full set (H1), (H2), (H2+ ), (H3), (H4). Corollary 10 holds under the union. We flag the dependence at each statement rather than absorbing it into a global assumption block, because the two bounds genuinely differ in what they need. 3.2 The certification complexity Definition (certification complexity). For each environment Ei , define the certification complexity Ci = C(Ei , f ) by the covering linear program o nX X
gi (a) na : na ≥ 0 ∀a, na d Kia ∥ Kja ≥ 1 ∀ j ∈ Alt(i) , (40) Ci := min a∈A

a∈A

and by the cost-ratio characteristic-time representation " Ci =

#−1 P inf j∈Alt(i) a∈A qa d Kia ∥ Kja P , sup q∈∆(A) a∈A qa gi (a)

(41)

where ∆(A) is the probability simplex over actions. The allocation q ∗ in this display is the one tracked by CTS. There is also a value-equivalent reparameterization: for positive costs, set X va qa gi (a) va = P , Ψi (v) := inf d(Kia ∥Kja ). g (a) q g (b) j∈Alt(i) i i b b a Then Ci = [supv∈∆(A) Ψi (v)]−1 , but v and q are different coordinates. In general arg max Ψi is not the cost-optimal allocation q ∗ ; the inverse map is qa ∝ va /gi (a). Zero-cost actions and infinite KL terms use the perturbation/truncation conventions stated in Appendix A. Conventions: if the LP (40) is infeasible we set Ci = +∞ (boundary B1); if its value is 0 we set Ci = 0 (boundary B2). Why the two value representations agree (proof pointer). Feasibility P under (H2) makes the LP bounded and feasible, so strong duality gives the dual value maxλ j∈Alt(i) λj 25

Huang

P j j a a subject to j λj da ≤ gi (a) for all a, where we abbreviate da := d(Ki ∥Kj ). Writing M for the bracketed sup-inf in the v-reparameterized display above, Sion’s minimax theorem—both ∆(A) and ∆(Alt(i)) are compact and convex and the objective is bilinear and continuous—gives X . M = inf max µj dja gi (a), µ∈∆(Alt(i))

a

j

and a direct comparison of constraints shows the dual value equals 1/M . The full verification is recorded in Appendix A; the two value representations have also been checked to agree on randomly generated finite instances by exact LP solves and grid-based evaluation. This is an equality of optimal values, not an assertion that the two displayed optimizers are the same allocation. Two-point special case (bang-bang; boundary B3). the infimum in (41) collapses and both forms reduce to

If |Alt(i)| = 1, say Alt(i) = {j},

gi (a)
, a∈A d K a ∥ K a i j

(42)

Ci = min

so the optimal design concentrates on the single action with the best loss-per-nat ratio. This is exactly the binary form of the controlled-experiment constant of Nitinawarat and Veeravalli (2015, Theorem 5.1): with a single alternative, our model specializes to a known one, and the theorem of Section 3.3 correctly contains it. We treat this consistency check as a boundary requirement (B3), not as a contribution. Comparison with cost-weighted characteristic times. The form (41) has the shape of the cost-aware characteristic time of Kanarios et al. (2024) and the classic characteristic time of Garivier and Kaufmann (2016), and we inherited the shape from those templates. Two differences are load-bearing for us and were not present in either: the inner infimum ranges over alternative sets defined by the adequacy label (36), and the weights gi (a) are task losses incurred under the same environment that indexes the channels, not exogenous sampling fees. A caution about (41) is equally load-bearing and easy to miss: the arg sup of the (w/g)-weighted max-min form is in general not the allocation. The
cost-optimal P P j
asymptotic loss constant of an allocation w is g (a)w / inf w d a j a i a a a , and maximizing the (w/g)-weighted objective in (41) can yield a strictly larger constant than Ci : on a 2 × 2 instance (two environments, two actions) the (w/g)-optimal allocation attains constant ≈ 4.17 against the LP optimum Ci ≈ 2.73 (checked numerically by exact LP solve and exhaustive grid evaluation; the gap is structural, not numerical noise). The matching policy of Section 3.4 therefore tracks the cost-ratio optimizer (48)—equivalently, the normalized LP solution—and not the arg sup of (41). 3.3 Main results Theorem 8 (Environment-wise lower bound). Suppose (H1)–(H3) hold. Then for every δ ∈ (0, 12 ), every δ-correct policy π with Eπi [τ ] < ∞, and every environment Ei , R(π, Ei ) ≥ Ci kl δ, 1 − δ 26

≥ Ci log

1 , 2.4 δ

(43)

Self-Certification of Representation Adequacy

where kl(p, q) = p log(p/q) + (1 − p) log((1 − p)/(1 − q)) is the Bernoulli KL divergence and Ci = C(Ei , f ) is the certification complexity of Section 3.2. In particular, lim inf δ→0

Rδ∗ (Ei ) ≥ Ci . log(1/δ)

(44)

The proof (Appendix A) combines the transportation inequality of Kaufmann et al. (2016)—applied with actions in the role of arms and channel outputs in the role of arm observations—with a change of measure against each j ∈ Alt(i) on the event {Θ̂ = Θi }, followed by the scale reparametrization na = Eπi [Na (τ )]/L, which turns the information constraints into exactly the LP (40) with value Ci kl(δ, 1 − δ). The second inequality in (43) is the standard Bernoulli-KL bound kl(δ, 1 − δ) ≥ log(1/(2.4δ)) of Kaufmann et al. (2016). The technical route is standard and is credited as such; the content of the theorem is that the certification currency g and the label-defined alternatives Alt(i) pass through the argument unchanged, so the classical constant is replaced by C(Ei , f ) with its decisiontheoretic semantics. Theorem 9 (Matching upper bound: CTS). Suppose (H1), (H2), (H2+ ), (H3) and (H4) hold, and let π ∗ be the CTS policy of Section 3.4 (Algorithm 1) with stopping threshold β(t, δ) = log(2t(K − 1)/δ). Then: (a) π ∗ is δ-correct in the sense of (37); moreover, the stopping and decision rules alone guarantee maxi Pi (Θ̂ ̸= Θi ) ≤ δ under any sampling rule satisfying (H1); (b) for every environment Ei , lim sup δ→0

R(π ∗ , Ei ) ≤ Ci . log(1/δ)

(45)

The proof (Appendix B) adapts the Track-and-Stop template of Garivier and Kaufmann (2016) with the cost-weighted plug-in design of Section 3.4: part (a) is a mixturemartingale argument over the K − 1 challengers and a geometric time grid; part (b) combines almost-sure stabilization of the environment estimator (which is where (H2+ ) and the forcedP exploration
are used),
of the plug-in allocation, and the LP optimality P tracking ∗ dj = C at the cost-ratio optimizer w ∗ . ∗ / min w g (a)w identity i j a a a a a i Corollary 10 (Limit of the optimal certification cost). Under the union of the assumptions of Theorems 8 and 9, namely (H1), (H2), (H2+ ), (H3), (H4), the optimal certification cost satisfies, for every environment Ei , Rδ∗ (Ei ) = C(Ei , f ), δ→0 log(1/δ) lim

(46)

and the CTS policy attains the limit. If C(Ei , f ) = 0 (boundary B2), (46) reads Rδ∗ (Ei ) = o(log(1/δ)); the case C(Ei , f ) = +∞ is excluded by (H2) and corresponds to the hopeless boundary B1, where no δ-correct policy with finite expected loss exists for small δ. Corollary 10 is a direct conjunction of (44) and (45). We emphasize the reading given in Section 1.3: the price of certifying representation adequacy to confidence δ is C(E, f ) log(1/δ) + o(log(1/δ)), with C(E, f ) computable as a finite LP from the model primitives—and the statement is exactly as strong as its assumptions, which the boundaries B1–B3 below show to be individually sharp in the senses stated there. 27

Huang

3.4 The CTS strategy We now define the Certification Track-and-Stop policy π ∗ (CTS) completely. It has four components: an environment estimator, a plug-in allocation rule, a tracking rule with forced exploration, and a stopping rule with terminal decision. Throughout, dja (ı̂) := d(Kı̂a ∥Kja ) denotes the plug-in divergence profile at the estimated environment. Estimator (maximum likelihood / KL projection). At time t, CTS computes X log KiAs (Ys ), (47) ı̂t ∈ arg max i∈{1,...,K}

s≤t

the environment whose induced channel law maximizes the likelihood of the realized transcript (equivalently, the KL projection of the empirical action-conditional observation histograms onto the model class); ties are broken by a fixed priority rule. Under (H1), (H2+ ) and the forced exploration below, ı̂t equals the true environment eventually almost surely (Appendix B); since E is finite, this is the only stability property the plug-in step needs. Plug-in allocation (cost-ratio optimizer).

w∗ (ı̂) ∈ arg sup

inf

Given ı̂t = ı̂, CTS solves X
wa d Kı̂a ∥ Kja

w∈∆(A) j: Θj ̸=Θı̂

a∈A

X

,

(48)

gı̂ (a) wa

a∈A

the allocation maximizing the worst-case information rate per unit Pof task loss. Two equivalent readings are useful: w∗ (ı̂) is the normalized LP solution n∗ / a n∗a of (40) evaluated at
P P ∗ j
∗ ı̂; and its value is the reciprocal of the plug-in loss constant a g wa / inf j a wa da = C(Eı̂ , f ). As recorded in Section 3.2, (48) is deliberately not the arg sup of the (w/g)weighted form (41): the two criteria share the same value 1/Ci but their optimizers can differ, and using the wrong one inflates the asymptotic constant (the 2 × 2 instance quoted there gives 4.17 against the optimum 2.73). If the denominator of (48) can vanish (a zeroloss informative action, boundary B2), the ratio is read by continuity and the policy behaves as described under B2 below. P Sampling rule (C-tracking with forced exploration). Writing Na (t) = s≤t 1{As = a}, CTS plays h
Na (t) i At+1 ∈ arg max wa∗ ı̂t − , (49) a∈A t subject to the forced-exploration override: if after the choice some action would violate p Na (t + 1) ≥ t + 1 + K 2 − 2K, (50) the most under-sampled such action is played instead. The tracking rule (49) together with (50) is the template of Garivier and Kaufmann (2016, Lemma √ 7): it guarantees Na (t)/t → wa∗ whenever the tracked allocation stabilizes, while the t-rate exploration makes the estimator (47) consistentp and contributes only o(log(1/δ)) to the loss (Appendix B); this is also the source of the Ω( log(1/δ)) residual cost recorded under boundary B2. 28

Self-Certification of Representation Adequacy

Algorithm 1 Certification Track-and-Stop (CTS), π ∗ Require: environment class E = {E1 , . . . , EK } with labels (Θi ), channels (Kia ), losses (gi (a)); confidence δ ∈ (0, 12 ) 1: initialize Na (0) ← 0 for all a; play each action once (in an arbitrary order) to initialize (47) 2: for t = 1, 2, . . . do 3: compute the estimator ı̂t by (47) 4: solve the plug-in allocation w∗ (ı̂t ) by (48) 5: if maxi minj:Θj ̸=Θi LLRi,j (t) > β(t, δ) with β as in (52) then 6: set τ ← t; return Θ̂ = Θı̂τ 7: else 8: choose At+1 by C-tracking (49), overridden by forced exploration (50) if violated; observe Yt+1 ∼ K At+1 9: end if 10: end for

Stopping rule and terminal decision (GLR). (i, j) define the log-likelihood ratio LLRi,j (t) :=

X

log

s≤t

For each ordered pair of environments

KiAs (Ys ) KjAs (Ys )

,

(51)

which is well-defined and has uniformly bounded increments under (H4). CTS stops at τ := inf

n

t : max i

o min LLRi,j (t) > β(t, δ) ,

j: Θj ̸=Θi

β(t, δ) := log

2t (K − 1) , δ

(52)

and outputs Θ̂ := Θı̂τ .

(53)

The threshold (52) is of the Garivier and Kaufmann (2016, Theorem 10) type: it absorbs a union bound over the K − 1 challengers and a geometric time grid, which is what makes the δ-correctness of Theorem 9(a) independent of the sampling rule. The threshold is conservative relative to mixture thresholds (Kaufmann and Koolen, 2021); we keep it because the matching first-order constant is already delivered and the tightness of second-order terms is left to the open problems of Section 4. 3.5 Non-triviality and boundary behavior We record why the results are not special cases of the classical statements they resemble (NT1, NT2), and the three boundary instances that fix the exact scope of the assumptions (B1–B3). B1–B3 are proved in Appendix A; all numerical claims below were checked on finite instances by exact LP solves, exhaustive enumeration, or direct evaluation (agreement to solver precision). 29

Huang

NT1: equal information, different losses, different optimal policies. There exist instances with two actions a1 , a2 such that Kia1 = Kia2 for every environment Ei —the two actions are statistically indistinguishable under all hypotheses—yet gi (a1 ) ̸= gi (a2 ). Since Ci depends on the loss profile gi through (40), the optimal allocations differ across the two actions, and flipping the loss assignment flips which action the LP selects. Any criterion built from observation kernels alone (a pure Chernoff-type information rate) assigns the two actions identical value and therefore cannot recover the optimal design. A concrete instance exhibiting the swap was constructed and checked numerically. NT1 isolates the task-loss coupling: the certification problem posed in the currency of task loss is genuinely different from the same problem posed in the currency of sampling fees, already at the level of the constant. NT2: kernel switching escapes the static formula. Model M1 assumes each action carries a fixed channel Kia . Certification-driven behavior violates this: a policy that acts on the representation while uncertified and switches to a fallback rule once inadequacy is certified changes the law of its subsequent transcript—the available channels after a Verify outcome depend on the policy’s own history. The constant C(E, f ), computed from fixed kernels, does not characterize the achievable cost of this switching policy class: in the example of Section 4.1, a single near-perfect verification followed by a zero-loss fallback achieves total cost O(1) uniformly in δ, while the static formula for the same channel and loss primitives is positive and predicts Θ(log(1/δ)). This is a boundary statement about the scope of Theorems 8 and 9—they apply to the fixed-kernel policy class exactly as assumed— and it motivates the open program of Section 4; we claim no result for the switching class here. B1: hopeless instances (Ci = +∞). If for some j ∈ Alt(i) every action satisfies d(Kia ∥Kja ) = 0, then the LP (40) is infeasible, Ci = +∞, and δ-certification of the label at Ei is impossible for any δ < 1/2: environments Ei and Ej induce identical transcript laws under every policy. Infeasibility of the LP is exactly the hopeless case, in the same sense in which partial-monitoring games with no informative feedback structure are hopeless (Bartók et al., 2014): the analogy is structural (identical observation laws under a label disagreement), and we record it as such rather than as a reduction. ◦

◦

B2: free rides (Ci = 0). If some action a◦ has gi (a◦ ) = 0 and d(Kia ∥Kja ) > 0 for every j ∈ Alt(i), the LP value is 0 and Corollary 10 reads Rδ∗ (Ei ) = o(log(1/δ)): the upper bound delivers the little-o, and the lower bound’s lim inf ≥ 0 is trivial. The phenomenon is the free ride of Section 1.1: when the task action the agent would execute anyway is informative about the label, certification information is a by-product of task execution, a configuration with no analogue in classical experiment design where every experiment carries a positive fee. The exact sub-logarithmic scale is left open: CTS on such an instance pays p √ Ω( log(1/δ)) because of its t-rate forced exploration (50), and whether Rδ∗ (Ei ) = O(1) is achievable in this regime is recorded as open item (iii) of Section 4.4. B3: two-point degeneration (|E| = 2). With two environments, Alt(i) is a singleton for each i and both forms of Ci reduce to the bang-bang ratio (42). Theorems 8 and 9 then recover the binary controlled-testing constants of Nitinawarat and Veeravalli (2015, Theorem 5.1) (with task loss in the role of the control cost), so the results correctly contain 30

Self-Certification of Representation Adequacy

the known special case rather than competing with it. This boundary also fixes the sense in which the composite environment class matters: with K ≥ 3 and shared labels, the maxmin form (41) is genuinely a game, and single-action (bang-bang) allocations are generically suboptimal.

4 Switching Kernels, VERIFY/REPAIR, and Open Directions The main results of Section 3 are proved for a fixed-kernel model: every action carries an a priori specified channel Kia and loss gi (a), and the cost account stops at the certification time τ . This final section marks, as precisely as we can, where that framework ends. We first work out the kernel-switching boundary NT2 in detail (Section 4.1), then discuss policydependent transcripts in general and their neighbors in the control literature (Section 4.2), then the Repair action and representation dynamics (Section 4.3). Section 4.4 collects the open problems. Everything in this section is a boundary analysis or a research direction; we claim no theorems here, and the honesty conventions of Section 1.2 apply with doubled force. 4.1 A kernel-switching example: NT2 in detail Example (kernel switching). Consider two environments sharing the same representation f : E0 is adequate (Θ0 = 1) and E1 is inadequate (Θ1 = 0). Before any certification, the available actions are Act and Verify. Acting executes the task policy induced by f : under E0 it costs g0 (Act) = 0 and generates observations from K0Act ; under E1 the same action incurs the aliasing regret g1 (Act) = r > 0 per round and generates observations from K1Act . Verify purchases an external audit at price caudit > 0; we take the audit to be perfect—its channel is a point mass on the true label—and remark on the noisy case below. Finally, and this is the ingredient that leaves Model M1: the environment admits an exogenously given fallback rule that is known to be optimal under inadequacy (zero loss under E1 ), and once the agent has certified Θ̂ = 0 it may switch to this rule permanently, after which its observations follow a fallback channel K fb and its task loss is zero. The switching policy π ↔ is: play Verify at t = 1; if the audit returns Θ = 1, act on f thereafter; if it returns Θ = 0, switch to the fallback rule thereafter. Its certification error is zero, hence δ-correct for every δ ≥ 0, and its total cost is

R π ↔ , E0 = R π ↔ , E1 = caudit = O(1), uniformly in δ. In particular Rδ∗ = o(log(1/δ)) in the strongest possible sense. Cost structure: where the O(1) comes from. It is instructive to decompose the total cost of π ↔ against the static benchmark. Suppose first that the perfect audit is removed, so that label evidence must come from task observations alone: then the instance is a two-environment member of the fixed-kernel class, Theorem 8 applies with Ci = mina gi (a)/d(Kia ∥Kja ) (boundary B3), and—whenever acting is informative but costly under E1 —every δ-correct policy pays Θ(log(1/δ)) in environment E1 , where the per-round aliasing regret r keeps accumulating until the GLR statistic separates the two environments. The switching policy escapes both terms of that accounting: the audit replaces the log(1/δ)-scaling evidence accumulation by a single fixed payment caudit , and the fallback 31

Huang

replaces the continuing loss stream r per round by zero under E1 . Neither escape is available within Model M1 as axiomatized: the first is available statically only when the audit is perfect (the B2 degeneracy), and the second is excluded outright because (38) prices only the pre-stopping trajectory of a kernel family that (M1.2) declares immutable. The example’s O(1) is thus the sum of two distinct phenomena—degenerate certification cost and policy-dependent loss truncation—and only the first is captured by the static constant C(E, f ). What the static formula does and does not say here. Two separate points must be kept apart. First, the collapse of the certification cost is not, by itself, outside the static framework: a perfect audit has d(KiVerify ∥KjVerify ) = +∞ for j ∈ Alt(i), so the LP (40) already assigns Ci = 0 to such instances, and Corollary 10 reads Rδ∗ = o(log(1/δ)) via boundary B2—consistently with the example. Second, the genuinely non-static element is the switch: after the Verify outcome, the law of the agent’s subsequent transcript depends on the policy’s own history—K0Act or K1Act before certification, K fb after a negative certification—and the objective that the example prices (one audit fee, then a zero-loss regime under both environments) couples the certification purchase to the post-certification loss stream. Model M1 excludes both features by construction: (M1.2) fixes one channel and one loss per action for the entire horizon, and (38) stops the accounting at τ . The change-of-measure argument behind Theorem 8 requires the fixed-kernel premise exactly here: the transportation inequality compares transcript laws generated by a single, policyindependent kernel family (Kaufmann et al., 2016), and no per-environment constant computed from fixed kernels can describe a class in which the kernel family is itself a function of the policy. There is therefore no contradiction between the example and Theorem 8; there is a scope statement. The static complexity C(E, f ) characterizes the fixed-kernel policy class, and the switching class can dominate it by an unbounded factor (from Θ(log(1/δ)) to O(1)) precisely when certification unlocks a regime change. The noisy-audit variant. If the audit is imperfect, one purchase certifies the label only to the confidence the audit channel affords, and repeated purchases or hybrid evidence accumulation become necessary. How the optimal number of purchases scales with δ in that regime is not covered by any result in this paper; it is recorded as open item (iii) below, together with the related B2 scaling question. 4.2 Policy-dependent transcripts in general The example is an instance of a general phenomenon: whenever the certification outcome is acted upon—act on the representation while adequate, fall back when inadequate—the certification process and the task process cease to be separable. Three couplings arise simultaneously. (i) Information–loss coupling along the path: before certification, the frule is both the cheapest task action (under E0 ) and a source of label evidence; its value therefore depends on the posterior, not on its static loss alone. (ii) Regime switching: the action set effectively available after certification is a function of the terminal decision, so the model class for the post-certification phase is policy-dependent. (iii) Objective coupling: the natural performance measure is total—certification expenditure plus subsequent task loss—so a verification is worth buying exactly when the regime it unlocks dominates its 32

Self-Certification of Representation Adequacy

price, which is the sequential analogue of the one-shot threshold c∗ = (T r/2)(1 − TV) of the static layer. In that layer, Theorem 6 of Section 2 prices a single certification-driven switch; the sequential model of Section 3 generalizes the acquisition side of that picture (the one-shot model is the special case nVerify = 1 with δ fixed by the audit channel’s total variation), while the switching side remains open here. A compact way to see why the static lower-bound technology does not extend by a routine generalization is to follow the change-of-measure step of Appendix A under a switching policy. The transportation inequality controls, for each alternative j, the log-likelihood ratio accumulated along the realized action sequence; with policy-dependent kernels, the likelihood ratio between Ei and Ej is no longer a function of the action counts Na (τ ) alone, because the same action belongs to different kernel families before and after the switch. The lower-bound variable is therefore no longer an allocation na over a fixed action set but an allocation over regime-action pairs, with regime transitions triggered by the certification statistic itself; the covering LP (40) would have to be replaced by a dynamic program over posterior-dependent regimes, and we do not know whether a finite-constant characterization in the spirit of C(E, f ) survives. We record this as the technical content of open item (i) rather than as a conjectured theorem. The nearest classical frame for this phenomenon is dual control, where control actions serve probing and regulation simultaneously (Feldbaum, 1965, 1963; Bar-Shalom and Tse, 1974; Tse and Bar-Shalom, 1975; Witsenhausen, 1971; Yame, 1987); the switching problem above can be read as dual control with a certification terminal and a label-dependent regime. We have now completed a theorem-level audit of this neighboring line and of POMDP bandits. The audit finds substantial overlap—dual control allows an action to change both the controlled state and the information available for future control—but no result in the audited set combines a decision-theoretic adequacy label, task loss of the same action as the certification currency, and a fixed-confidence stopping objective. This is an audited-scope statement, not a claim that dual control is unrelated or that no unexamined result could narrow the gap. POMDP bandits likewise share actions between reward and observation (Krishnamurthy and Wahlberg, 2009), but without a certification target, stopping rule, or label-defined alternative structure; planning under belief-state sufficiency (Kaelbling et al., 1998; Åström, 1965) presumes the representation question settled rather than pricing it. The lower-bound/matching-policy pair for certification-driven switching therefore remains open, as stated in item (i). 4.3 REPAIR and representation dynamics The Repair action goes one step beyond switching: instead of replacing the policy on a certified negative, it modifies the representation itself, f 7→ f ′ , attempting to enlarge the adequate region of the environment class. This breaks the framework at a deeper level than kernel switching, for two distinct reasons. First, the object of inference moves: the ∗ (f (H)) = R∗ (H)}, a time-varying, policy-dependent adequacy label becomes Θt = 1{RE t E quantity, so the hypothesis being tested changes as a consequence of the agent’s own actions. Second, the alternative sets (39), the channels, and the loss profiles are all indexed by the representation (aliasing regret is a property of f ), so a Repair action rewrites the model class mid-stream. Certification with a moving label is outside fixed-hypothesis sequential 33

Huang

testing in the Chernoff (1959) line, outside the fixed-confidence pure-exploration template of Garivier and Kaufmann (2016)—both presume a static hypothesis and a static experiment family—and outside Model M1 as defined. We know of no standard template that absorbs a self-modifying hypothesis, and we therefore treat representation dynamics strictly as a research direction, consistent with the limitation recorded in the static layer, where the fallback and repair mechanisms are explicitly exogenous inputs whose acquisition is not priced. Two partial observations delimit the direction without resolving it. First, the one-shot layer already contains the germ of the difficulty: the value of verification in Theorem 6 of Section 2 is priced against a fixed fallback, and the static layer is explicit that the acquisition of that fallback—its information source and cost—is outside the model; Repair is precisely the attempt to bring that acquisition inside the decision problem, and the analysis above indicates that doing so changes the hypothesis space itself. Second, representation dynamics interact with the certification label in both directions: a repair that succeeds changes the set of adequate environments and hence the alternative sets (39) of all future certification problems, while a failed repair consumes the same audit currency that certification uses. Any future theory of this interaction will therefore have to price, jointly, evidence about a label and modifications of the object the label describes; we state this as the content of open item (ii) and make no further claim. 4.4 Open problems (i) Coupling theorem for certification-driven switching. Extend the lower-bound/matchingpolicy pair of Theorems 8 and 9 to policy classes whose post-certification kernels depend on the certification outcome (f-rule ↔ fallback), with a total objective combining certification expenditure and post-switch task loss. The example of Section 4.1 shows the constant can collapse from Θ(log(1/δ)) to O(1); a characterization of the general scaling is missing. (ii) Repair and representation dynamics. Formulate certification when the label Θt = AdeqE (ft ) is itself policy-dependent, including the pricing of representation modifications against their certification value. (iii) Repeated purchases of a noisy Verify; the exact B2 scale. In the free-ride p regime (Ci = 0), Corollary 10 delivers Rδ∗ = o(log(1/δ)) while CTS pays Ω( log(1/δ)) through forced exploration; whether O(1) is achievable, and more generally how repeated noisy-audit purchases scale with δ, is open. (iv) Dual-control/POMDP extension. The literature audit is complete for the references listed in Section 6.6; the open problem is now to build and analyze a dual-control/POMDP model with a moving adequacy label and policy-dependent kernels, rather than to repeat the audit. (v) Non-asymptotic remainders. Refine Corollary 10 to Rδ∗ (Ei ) = Ci log(1/δ) + r(δ) with an explicit rate for r(δ) = o(log(1/δ)); already in pure exploration this is recognized as delicate, and we expect no shortcut from the adequacy-label structure. 34

Self-Certification of Representation Adequacy

4.5 Summary The sequential layer of this paper prices the acquisition of adequacy evidence in a fixedkernel world: Theorem 8 shows that every δ-correct policy pays at least C(E, f ) kl(δ, 1−δ) in task loss, Theorem 9 shows the CTS policy pays exactly that to first order, and Corollary 10 identifies the limit Rδ∗ (Ei )/ log(1/δ) → C(E, f ) with C(E, f ) computable as a finite LP. The boundaries of the statement are as much a part of the result as the limit: hopelessness is exactly LP infeasibility (B1), free rides are exactly zero LP value (B2), and the two-point case recovers the classical controlled-testing constant (B3). This section has argued that the framework’s edge is sharp and locatable: the moment certification is allowed to change the process being certified—through policy switching, and a fortiori through representation repair—the fixed-kernel constant ceases to govern, and the achievable cost can collapse to O(1). Characterizing that regime is the natural continuation of the present work, and we have stated it as such, without claiming it.

5 Numerical Illustrations on Finite Instances This section collects numerical work on small, fully specified instances of Model M1. Its role is deliberately limited, and we state the limitation before any numbers: everything reported here is machine-checkable verification on finite instances and a demonstration of the behavior predicted by Theorems 8 and 9; it is auxiliary evidence and is not a substitute for the proofs. The proofs stand on their own; the computations below serve three narrower purposes. First, they make the abstract objects of Section 3 concrete: the certification complexity C(E, f ) is a finite linear program, and we exhibit one instance in full—every channel, every loss, and the exact LP solutions—so that the reader can recompute every constant we quote. Second, they illustrate the predicted asymptotics: the simulated cost of the CTS policy, normalized by log(1/δ), approaches the LP constant from above as δ decreases, at the pace that the conservative stopping threshold leads one to expect. Third, they exercise the boundary statements B1–B3 and the non-triviality statements NT1–NT2 on explicit numbers, so that the scope of the assumptions is checked rather than merely asserted. All instance parameters are given explicitly in the tables below, every linear program was solved exactly (to solver precision) from those tabulated parameters, and the simulation protocol is specified completely in Section 5.7. We report only what the computations show; where the displayed behavior falls short of the limit theorems (for instance, convergence that requires smaller δ than we simulate), we say so. 5.1 A four-environment instance, specified completely Our running instance has K = 4 environments sharing a fixed representation f , with adequacy labels (Θ0 , Θ1 , Θ2 , Θ3 ) = (1, 1, 0, 0), (54) so that E0 , E1 are adequate, E2 , E3 are inadequate, and the alternative sets of Section 3.1 are Alt(0) = Alt(1) = {2, 3} and Alt(2) = Alt(3) = {0, 1}. The action set is A = {Act, Ask, Verify} and the observation space has three elements, Y = {y1 , y2 , y3 }. Table 2 gives the task losses, including the label-coupled structure of Act: acting on the representation is cheap (0.05 per round) when the representation is adequate and expensive 35

Huang

Table 2: Task losses gi (a) and adequacy labels of the four-environment instance. Act carries the label-coupled loss of Section 3: cheap under adequacy, priced at the per-round aliasing regret under inadequacy. The Verify loss is the audit price cv , equal to 1.0 in the base instance and swept in Figure 2.

E0 E1 E2 E3

Θi

gi (Act)

gi (Ask)

gi (Verify)

1 1 0 0

0.05 0.05 0.60 0.60

0.35 0.35 0.35 0.35

cv = 1.0 cv = 1.0 cv = 1.0 cv = 1.0

Table 3: Observation channels Kia of the four-environment instance, as probability rows on Y = {y1 , y2 , y3 }. All entries are positive (Assumption (H4)); every ordered pair of environments is separated by at least one action (Assumptions (H2) and (H2+ )), with Verify alone already separating every pair. The Act rows of an adequate environment and of its binding inadequate alternative are close but not equal, which is what makes acting nearly uninformative about the label in this instance. Action

Environment

Kia (y1 )

Kia (y2 )

Kia (y3 )

Act

E0 E1 E2 E3

0.5000 0.5200 0.4500 0.4800

0.3000 0.2800 0.3000 0.3200

0.2000 0.2000 0.2500 0.2000

Ask

E0 E1 E2 E3

0.7000 0.6800 0.3000 0.3200

0.2000 0.2200 0.4000 0.3800

0.1000 0.1000 0.3000 0.3000

Verify

E0 E1 E2 E3

0.9000 0.8800 0.1000 0.1200

0.0700 0.0900 0.2000 0.1800

0.0300 0.0300 0.7000 0.7000

(0.60 per round, the per-round aliasing regret) when it is not, while Ask costs a flat query fee 0.35 and Verify costs the audit price cv , which is 1.0 in the base instance and is varied in Figure 2. Table 3 gives all twelve observation channels. The tables are the instance: every number quoted in this section is recomputed from them. Solving the covering LP (40) for each environment at cv = 1.0 gives C(E0 , f ) = 0.6050,

C(E1 , f ) = 0.6264,

C(E2 , f ) = 0.4657,

C(E3 , f ) = 0.4783, (55) ∗ each attained by a bang-bang LP solution that places all weight on Verify: nVerify = Ci and n∗Act = n∗Ask = 0. Consequently the cost-ratio optimizer (48) that CTS tracks is the 36

Self-Certification of Representation Adequacy

Table 4: Divergences d(Kia ∥Kja∗ ) to the binding alternative j ∗ (i) (in parentheses) and the resulting loss-per-nat ratios gi (a)/d, computed from Tables 2 and 3. Verify has the best ratio in every environment, which is why the LP (40) puts all weight on it. Act E0 (j ∗ = 3) E1 (j ∗ = 3) E2 (j ∗ = 1) E3 (j ∗ = 1)

Ask

Verify

d

g/d

d

g/d

d

g/d

0.0010 0.0042 0.0114 0.0043

47.64 11.81 52.53 139.23

0.3097 0.2825 0.3232 0.2961

1.130 1.239 1.083 1.182

1.6528 1.5965 2.1471 2.0906

0.6050 0.6264 0.4657 0.4783

pure Verify allocation, w∗ (i) = eVerify for all four environments. The values (55) are exact LP outputs; recomputing the LP from the four-decimal channel values of Table 3 reproduces them to three decimals (solver agreement within 2.5 × 10−4 ), and a grid evaluation of the max-min form (41) agrees with the LP value to the same tolerance, consistent with the duality of Section 3.2. Table 4 explains why the optimum is bang-bang on Verify in this instance: it reports, for each environment and action, the divergence to the binding alternative (the alternative attaining minj∈Alt(i) d(KiVerify ∥KjVerify )) and the associated loss-per-nat ratio gi (a)/d(Kia ∥Kja ). Verify dominates because it buys several times more divergence per round than Ask, at less than three times the price; Act, despite being nearly free under adequacy, is nearly uninformative about the label—its channels under E0 and its binding alternative E3 differ by only 0.001 nats—so its loss-per-nat ratio is the worst of the three. This is the regime of Section 1.1 in which certification must be purchased rather than picked up as a by-product of acting; the free-ride regime is exercised separately as boundary B2 below. 5.2 Convergence of the CTS cost to the LP constant Figure 1 plots the simulated expected cost of the CTS policy π ∗ (Algorithm 1) on the instance of Tables 2–3, normalized by log(1/δ), against log(1/δ) for the five confidence levels δ = 0.1, 0.05, 0.02, 0.01 and 0.005. Each (environment, δ) combination was simulated with 120–150 independent replications; the sampling rule is C-tracking (49) of the plug-in cost-ratio optimizer with the forced exploration
(50), and the stopping rule is the GLR rule (52) with threshold β(t, δ) = log 2t(K − 1)/δ at K = 4. Horizontal lines mark the LP constants (55), which by Corollary 10 are the limits the curves should approach; Table 5 reports the endpoint values underlying the plot. Three readings of Figure 1 and Table 5 are warranted, and we state them with the conservatism they deserve. First, the qualitative prediction of Corollary 10 is visible: all four curves decrease monotonically and are approaching their LP constants from above, with no sign of leveling off at any other value. Second, the simulation is not in the asymptotic regime: at δ = 0.01 the normalized cost is between 1.9 and 2.5 times C(Ei , f ). The bulk 37

Huang

CTS in E0 (simulated) CTS in E1 (simulated) CTS in E2 (simulated) CTS in E3 (simulated)

1.8

R( * , Ei) /log(1/ )

1.6 1.4 1.2 1.0 0.8

C(E1, f)=0.626 C(E0, f)=0.605 C(E3, f)=0.478 C(E2, f)=0.466

0.6 0.4

2.5

3.0

3.5

4.0

4.5

5.0

log(1/ )

5.5

6.0

Figure 1: Simulated certification cost of CTS on the four-environment instance, normalized by log(1/δ), as a function of log(1/δ), with 120–150 independent replications per (environment, δ) combination. Horizontal lines mark the LP constants C(Ei , f ) of (55). All four curves decrease monotonically toward their respective constants; at δ = 0.005 (log(1/δ) ≈ 5.3) they lie in the band 1.0–1.2, still above the limits 0.466–0.626 predicted by Corollary 10.

Table 5: Simulated values of R(π ∗ , Ei )/ log(1/δ) underlying Figure 1 at the two ends of the simulated range, together with the LP constants (55). The third row is the ratio of the δ = 0.01 entry to the corresponding constant: the simulated cost at δ = 0.01 is still between 1.9 and 2.5 times the asymptotic constant, and decreasing. E0

E1

E2

E3

R/ log(1/δ) at δ = 0.1 R/ log(1/δ) at δ = 0.01 ratio to C(Ei , f ) at δ = 0.01

1.770 1.146 1.89

1.863 1.217 1.94

1.592 1.157 2.48

1.642 1.151 2.41

C(Ei , f ) (LP, (55))

0.6050

0.6264

0.4657

0.4783

of this gap is explained by two standard and identified mechanisms, not by a failure of the constant. The GLR threshold β(t, δ) = log(2t(K − 1)/δ) of Garivier and Kaufmann (2016, Theorem 10) absorbs a union bound over the K − 1 challengers and a geometric time grid, and is known to be conservative relative to mixture-based thresholds (Kaufmann and Koolen, 2021); and the stopping time inherits the usual overshoot—the LLR process crosses the boundary with positive excess, so the expected cost exceeds the first-order approximation by a quantity that decays only relative to log(1/δ). Both effects shrink 38

Self-Certification of Representation Adequacy

Certification complexity as a function of the verification cost C(E0, f) C(E1, f) C(E2, f) C(E3, f)

1.2 1.0

C(E, f)

0.8 0.6 0.4 0.2 0.0

0.0

0.5

1.0

1.5

VERIFY cost cv

2.0

2.5

Figure 2: Certification complexity C(Ei , f ) of the four-environment instance as a function of the Verify price cv , computed by re-solving the LP (40) on a grid in [0.1, 2.5]. Each curve is piecewise linear: a pure-Verify segment through the origin up to an environment-dependent kink, after which Ask-based plans take over and the curve flattens toward the no-Verify LP value. The values at cv = 1.0 are those of (55).

as δ decreases, which is exactly what the figure shows. We therefore report the honest conclusion: asymptotic convergence to C(E, f ) requires smaller δ than we simulate, and the trend displayed over the simulated range is consistent with the theorem. Third, the certification guarantee holds with room to spare: the empirical error frequency was at most 0.01 in every cell, including those with target δ = 0.1, reflecting the same threshold conservativeness; the δ-correctness of Theorem 9(a) is satisfied in all runs. 5.3 The complexity frontier in the audit price Figure 2 shows how the certification complexity of the same instance moves with the audit price: we re-solve the LP (40) with gi (Verify) = cv for cv ∈ [0.1, 2.5], holding the channels and the remaining losses of Tables 2–3 fixed. Each curve is the value of a parametric linear program, hence continuous, nondecreasing, and concave piecewise linear in cv ; the figure displays exactly this structure. Two structural effects are visible. First, the initial segment of every curve is the bangbang line cv /dVerify , where dVerify = minj∈Alt(i) d(KiVerify ∥KjVerify ): while the audit is i i cheap enough, buying divergence through Verify is optimal and the complexity is exactly linear in its price. The kink occurs where the line meets the cost of cheaper plans; numerically the curves stay on the pure-Verify line up to environment-dependent kink points in the range cv ≈ 1.9–2.5 (≈ 1.9 for E0 , ≈ 2.0 for E1 , ≈ 2.3 for E2 and ≈ 2.5 for E3 ), beyond which Ask-based plans take over and each curve flattens onto its no-Verify LP value— 1.130, 1.239, 1.083, 1.182 respectively, the price of covering both alternative constraints 39

Huang

with Ask alone. The two adequate curves saturate first: at cv = 2.0 they already sit at 1.130 and 1.239, while the inadequate curves are still on the pure-Verify line there (0.931 and 0.957) and keep rising past cv ≈ 2.2–2.3. Second, the ordering of the curves reflects the label-coupled loss structure rather than the channels alone: the inadequate environments E2 , E3 are cheaper to certify than the adequate ones throughout, not because their channels are more informative in an absolute sense, but because the LP prices information in the currency of the environment’s own task loss, and the divergence profile of Verify against the adequate alternatives is larger in that direction (d values 2.15 and 2.09 against 1.65 and 1.60; see Table 4). The frontier thus makes concrete the central feature of the constant C(E, f ): it couples the statistical distinguishability of the environments to the loss in which the certification bill is paid, and moving a single price (cv ) re-optimizes the whole covering plan rather than rescaling a fixed design. 5.4 NT1 on numbers: equal kernels, different losses, different optima Figure 3 instantiates the non-triviality statement NT1 of Section 3.5. The instance has two environments, E0 adequate and E1 inadequate, and two informative actions a1 , a2 whose observation channels are identical under both environments: on a two-point observation space Y = {y1 , y2 }, d K0a

K0a1 = K0a2 = (0.8, 0.2), K1a1 = K1a2 = (0.3, 0.7),
K1a = 0.5341 for a ∈ {a1 , a2 }.

(56)

The two actions are therefore statistically indistinguishable: any criterion built from the observation kernels alone—in particular the Chernoff-type most-informative-action rule of the classical sequential-design line (Chernoff, 1959; Nitinawarat and Veeravalli, 2015)— assigns them the same value. What differs is the loss: a1 costs a fixed 0.5 per round in both environments, while a2 is environment-dependent. Under loss profile A, g0 (a2 ) = 0.01 and g1 (a2 ) = 1.0; under profile B these two values are swapped, g0 (a2 ) = 1.0 and g1 (a2 ) = 0.01. Since |Alt(i)| = 1 the bang-bang form (42) applies and the optimal certification action in environment i is arg mina gi (a)/0.5341 = arg mina gi (a): under profile A, E0 certifies through a2 (C0 = 0.01/0.5341 ≈ 0.019) and E1 through a1 (C1 = 0.5/0.5341 ≈ 0.936); under profile B the choices interchange. The point is quantitative, not cosmetic. A kernel-only criterion sees one instance where there are two; in at least one of the two profiles it prescribes an action whose loss-per-nat ratio exceeds the optimum by a factor of 0.5/0.01 = 50 in the affected environment—the same statistical evidence at fifty times the asymptotic cost (and if the kernel-only rule happens to favor a2 , profile B inflates the factor to 1.0/0.01 = 100). The certification problem posed in the currency of task loss is therefore genuinely different from the same problem posed in the currency of sampling fees, already at the level of the constant: the g-dependence of C(E, f ) in the main theorems is not an artifact of the proof technique but a property of the problem, and the main theorems’ accounting in the currency g is loadbearing. We remark that the same phenomenon is what the comparison with the active sequential testing formulation of Naghshvar and Javidi (2013) and the controlled-sensing constants of Nitinawarat and Veeravalli (2015) must respect: in those models the cost of an action is an exogenous fee, whereas here it is the environment’s own task loss, and (56) shows the two cost models are not interchangeable even when the kernels coincide. 40

Self-Certification of Representation Adequacy

Identical observation kernels (fixed DKL), different task losses loss profile A optimal: a1

task loss per step

1.0

optimal: a1 loss profile B action a1 (fixed cost) action a2 (env-dependent)

0.8 0.6

different optimal designs

optimal: a2

optimal: a2

0.4 0.2 0.0

E0 (adequate)

E1 (inadequate)

E0 (adequate)

E1 (inadequate)

Figure 3: The NT1 instance (56): per-round task losses of the fixed-cost action a1 and the environment-dependent action a2 under the two loss profiles. The observation kernels—and hence every kernel-only information criterion—are identical across profiles and across the two actions (d = 0.5341 nats), yet the LP-optimal certification action, marked in each panel, swaps: profile A selects a2 at E0 and a1 at E1 , and profile B selects the reverse.

5.5 The sequential certification loop, schematically Figure 4 summarizes the model that all of the above numbers come from: the interaction loop of Model M1 with the four components of the CTS policy marked at the points where they act. The diagram is a rendering of definitions (38)–(52) rather than new content, and is included to fix the data flow: the unknown environment index i selects a row of the channel and loss tables; the policy, driven by the history, picks the action; the environment returns an observation through the selected channel and charges the selected loss; the transcript feeds the environment estimator, the plug-in allocation, and the GLR statistic; and on stopping, the certified label is read off the estimated environment. The schematic makes one modeling choice visible that the equations keep implicit: the loss is charged per round, in parallel with the observation, and the certification constraint couples only to the transcript, never directly to the loss stream. It also localizes precisely where the boundary phenomena of the next subsection enter: B1 removes the divergence between two rows of the channel box, so no transcript can separate them; B2 sets a loss entry to zero, so the accumulator stops charging for informative rounds; B3 collapses the environment box to two rows; and NT2, which the model excludes by fiat, would allow the certification output to rewire the channel box itself after a Verify outcome, making the channel family a function of the policy’s own history (Section 4). 5.6 Boundary behavior in numbers: B1, B2, B3, and NT2 We close the numerical tour by evaluating the four boundary statements of Sections 3.5 and 4 on explicit instances derived from Tables 2–3. Each boundary is reported in the same format: the modification of the base instance, the numerical outcome, and the consistency check against the theorems. 41

Huang

channel/loss Policy π rows At from Ht−1 ; CTS: MLE ı̂t , index i unknown, Θi realized w∗ (ı̂t ), C-tracking

Environment Ei

historydriven

At

Transcript Ht

Yt ∼ KiAt

(A1 , Y1 , . . . , At , Yt )

Action output triple channel KiAt , loss gi (At ), label information

gi (At )

Cost accumulator R = Ei [

P

t≤τ gi (At )]

LLR increments

GLR stopping maxi minj∈Alt(i) LLRi,j (t) > β(t, δ)

τ

Certification output Θ̂ = Θı̂τ ∈ {0, 1}

Figure 4: The sequential certification loop of Model M1 with the CTS components in place. The unknown environment fixes the channel and loss rows (Tables 3 and 2); the policy selects actions from the history; each action returns an observation through its channel and charges its loss, which accumulates into the objective (38); the transcript drives the estimator (47), the plug-in allocation (48), and the GLR statistic; stopping (52) triggers the terminal decision (53). Solid arrows are the data flow; the dashed arrow is the information feedback that makes the policy history-dependent.

B1: hopeless instances, LP infeasibility. Modify the base instance by replacing every channel of E3 with the corresponding channel of E0 : K3a ← K0a for a ∈ {Act, Ask, Verify}. Then 3 ∈ Alt(0) and d(K0a ∥K3a ) = 0 for every action, so the covering LP (40) for E0 has an unsatisfiable constraint row. Solved from the tabulated parameters, the LP returns infeasible (linprog status 2), i.e. C(E0 , f ) = +∞ under the convention of Section 3.2. This matches Theorem 8 and the B1 statement of Section 3.5 exactly: E0 and E3 induce identical transcript laws under every policy, no δ-correct policy with δ < 1/2 and finite expected loss exists at E0 , and the failure is certified by the same object that computes the constant—the LP itself—rather than by a separate diagnostic. The situation is the hopelessness of partial-monitoring games without informative feedback (Bartók et al., 2014), transplanted to the certification currency. B2: free rides, LP value zero. Keep the channels of the base instance and change one loss: g0 (Act) ← 0. Since both divergences d(K0Act ∥K2Act ) = 0.0081 and d(K0Act ∥K3Act ) = 0.0010 are strictly positive, the zero-loss action covers every alternative constraint, and the LP value drops to exactly 0 (solved value 0.0, status optimal). Corollary 10 reads Rδ∗ (E0 ) = o(log(1/δ)) on this instance, with the lower bound lim inf ≥ 0 trivially tight: certification information is a free by-product of an action the agent would execute anyway, the free-ride phenomenon of Section 1.1. We repeat the honest caveat of Section 3.5: the 42

Self-Certification of Representation Adequacy

p √ exact sub-logarithmic scale is open, and CTS itself pays Ω( log(1/δ)) here because its trate forced exploration (50) keeps buying Ask and Verify rounds at positive loss; nothing in our results decides whether O(1) is achievable, and we do not claim it. B3: two-point degeneration, consistency with the binary constant. Restrict the environment class to {E0 , E2 }, so that Alt(0) = {2} and the bang-bang form (42) applies. The three loss-per-nat ratios, computed from the tabulated channels, are g0 (Act) 0.05 = ≈ 6.21, 0.0081 d(K0Act ∥K2Act ) 0.35 g0 (Ask) = ≈ 1.016, Ask Ask 0.3446 d(K0 ∥K2 ) 1.0 g0 (Verify) = ≈ 0.5526, Verify Verify 1.8095 d(K0 ∥K2 )

(57)

so on the restricted class C(E0 , f ) = mina g0 (a)/d(K0a ∥K2a ) = 0.5526, attained by Verify. In the reverse direction the ratios are 0.6/0.0084 ≈ 71.65 (Act), 0.35/0.3527 ≈ 0.993 (Ask) and 1.0/2.1952 ≈ 0.4555 (Verify), so C(E2 , f ) = 0.4555. Both binary constants lie below their four-environment counterparts (55): in the full instance the binding alternatives are E3 for E0 (d = 1.6528) and E1 for E2 (d = 2.1471), and restricting the class to {E0 , E2 } deletes exactly those binding constraints from the covering LP. This is the binary specialization of the controlled sequential testing constant of Nitinawarat and Veeravalli (2015, Theorem 5.1), with the task loss in the role of the control cost: with a single alternative there is no covering problem to balance and the cheapest nat wins. The numerical agreement is a boundary requirement, not a contribution—the theorems correctly contain the known binary case, and the genuinely combinatorial content of (40) appears only when |Alt(i)| ≥ 2. NT2: kernel switching escapes the static formula, in numbers. Instantiate the switching example of Section 4.1 with explicit channels: two environments, E0 adequate and E1 inadequate; before any certification the only task action is Act, with g0 (Act) = 0, g1 (Act) = r = 0.6, and channels

d K1Act

K0Act = (0.7583, 0.1249, 0.1168),
K0Act = 1.2000.

K1Act = (0.1168, 0.1249, 0.7583),

(58)

Viewed as a fixed-kernel instance (the audit removed, as in Section 4.1), it falls under B3: C(E1 , f ) =
r/1.2 = 0.5 per nat, and Theorem 8 gives R(π, E1 ) ≥ 0.5 kl(δ, 1 − δ) ≥ 0.5 log 1/(2.4δ) for every δ-correct fixed-kernel policy—numerically ≥ 1.86 at δ = 0.01 and ≥ 4.17 at δ = 10−4 , growing without bound as δ → 0. The switching policy of Section 4.1— one perfect Verify purchase at price caudit = 1.0, then act on f if certified adequate and switch to the zero-loss fallback otherwise—has certification error zero and total cost exactly 1.0 at every δ. Hence already at δ = 0.01 the switching class beats the fixed-kernel optimum at E1 by a factor of at least 1.87, at δ = 10−4 by at least 4.17, and by an unbounded factor asymptotically, while the static constant for the same channel and loss primitives predicts Θ(log(1/δ)). This is the numerical content of NT2: C(E, f ) characterizes the fixed-kernel policy class exactly as assumed, and fails to describe any class in which the certification 43

Huang

outcome rewires the subsequent channels. We record it as a scope boundary of Theorems 8 and 9 and as the motivation for the open program of Section 4; no claim is made for the switching class here. (For completeness: in the fixed-kernel reading, E0 itself is the B2 case, C(E0 , f ) = 0, since acting there is free and informative—so the static formula already sees the free ride; what it cannot see is the truncation of the loss stream under E1 after a negative audit.) 5.7 Reproducibility All instance parameters are given in Tables 2–3 and in the displayed equations (56) and (58) of this section; no other inputs are used. Every linear program reported here—the constants (55), the frontier of Figure 2, the per-nat ratios of Table 4, and the boundary computations B1–B3—was solved with scipy.optimize.linprog (HiGHS dual simplex) directly from the tabulated four-decimal parameters; the quoted values agree with the full-precision computation to within 2.5 × 10−4 , and the LP and max-min forms (40)– (41) were cross-checked on the instance by grid evaluation of the simplex and on randomly generated five-environment three-action instances, with agreement within the grid tolerance (relative error below 1%). The CTS curves of Figure 1 were produced by Monte Carlo simulation of Algorithm 1: C-tracking (49) of the plug-in cost-ratio optimizer (48) with forced exploration (50), GLR stopping (52) at β(t, δ) = log(2t(K − 1)/δ), terminal decision (53); 120–150 independent replications per (environment, δ) cell with independent random streams seeded per cell, and empirical error frequencies recorded alongside costs (all ≤ 0.01, as reported in Section 5.2). All identities, LP values, and thresholds quoted in this section were verified by exact LP solves, exhaustive enumeration, or direct evaluation from the tabulated parameters, with agreement to solver precision; the simulation is the only Monte Carlo ingredient, and it is reported as such.

6 Related Work We organize the comparison along the five directions of the literature audit conducted for this paper—sequential experimental design and active hypothesis testing; pure exploration and best-arm identification; controlled sensing and POMDPs; dual control and adaptive control; information-directed sampling—preceded by the classical statistical machinery that our static layer uses, and followed by three adjacent lines (partial monitoring and its classification, concept drift and robust dynamic programming, and a concurrent aliasingdecomposition preprint). For each cluster we give the trajectory of the research line, a substantive description of what the key papers establish, the overlap with the present paper, and the strict difference. We deliberately do not claim the absence of precedents in general terms; for the closest works we instead state concretely which element of our setting they lack. Table 6 summarizes the resulting matrix; its reading is given immediately below, and Section 6.9 states the exact scope of the combination we do claim. Read column-wise, Table 6 locates the paper by elimination rather than by declaration. The first two rows achieve δ-certification with stopping and matching log(1/δ) constants— the strongest known analogues of our Theorems 8 and 9—but their inference object is an exogenous hypothesis or answer, and their cost is either an exogenous sensing fee decoupled from any task or a homogeneous time count; the certification problem and the control 44

Self-Certification of Representation Adequacy

Table 6: The audit matrix. Three features distinguish the present setting: (i) the inference object is a decision-theoretic adequacy label rather than an exogenous parameter; (ii) the cost charged is the task loss of the same action that carries the discriminating information; (iii) the objective is a δ-certification with stopping, matched at the log(1/δ) scale. No audited line combines all three; each column is discussed in the text. Line of work

Inference object

Cost semantics

Objective

δ-certify + stop

log(1/δ) match

Seq. design / active testing (Chernoff, 1959; Nitinawarat et al., 2013; Nitinawarat and Veeravalli, 2015; Naghshvar and Javidi, 2013) Pure exploration / BAI (Kaufmann et al., 2016; Garivier and Kaufmann, 2016; Degenne and Koolen, 2019; Kanarios et al., 2024) Structured-bandit regret (Graves and Lai, 1997; Combes et al., 2015, 2017) Information-directed sampling (Russo and Van Roy, 2016, 2018, 2022) Controlled sensing / POMDP bandits (Krishnamurthy, 2016; Krishnamurthy and Wahlberg, 2009) Dual control / adaptive control (Feldbaum, 1963; Bar-Shalom and Tse, 1974; Tse and Bar-Shalom, 1975; Witsenhausen, 1971; Yame, 1987) Partial monitoring (Rustichini, 1999; Cesa-Bianchi et al., 2006; Bartók et al., 2014)

environment hypothesis

exogenous sensing fee

expected sensing cost

yes

yes

answer under (semi-) arbitrary answer map

unit time; exogenous arm cost (Kanarios et al., 2024)

E[τ ] / expected cost

yes

yes

unknown parameter

regret of actions

cumulative regret

no

LP-form regret bounds

optimal action A∗ (endogenous)

one-step loss coupled with info gain

Bayesian regret

no

ratio-type √ ΨHT

latent state / arm state

sensing cost or reward

discounted cost / reward

no

partial (structural)

unknown dynamics / latent parameters

probing value coupled with control loss

expected control cost / reward

generally no

dual-effect and separation results

hidden outcome via signal matrix

loss through feedback only

cumulative regret

no

rate classification

This paper (model M1)

adequacy label Θ (decisiontheoretic)

task loss until δcertification

yes

yes (Ci ; Theorems 8 and 9)

45 task loss gi (a) coupled with label information

Huang

problem live on separate actions. The fourth row is the only audited line in which one action simultaneously carries a task loss and an information gain, the same coupling as our model M1, but it is a regret framework without a certification target, a stopping rule, or a terminal decision, and its information object is the optimal action rather than a property of the agent’s representation. The fifth row shares the action–observation-kernel mechanism, and its bandit variant shares the coupled action, yet neither poses a testing objective. The sixth row is the closest control-theoretic neighbor: it studies the dual effect and policy-dependent information, but not a fixed-confidence adequacy label or a task-loss price for certifying that label. The seventh row supplies the algebraic notion of structural uninformative feedback (hopeless games) that our boundary instance B1 instantiates in KL terms, but it classifies regret rates rather than pricing a certification decision. The present paper occupies the last row: adequacy label, coupled task-loss cost, and δ-certification simultaneously. 6.1 Comparison of Experiments, Sufficiency, and Value of Information: the Static Layer’s Tools Trajectory. Statistical decision theory recast inference as choice under loss, with experiments—families of distributions indexed by a parameter—as the objects of comparison. The line runs from the game- and decision-theoretic foundations through Blackwell’s informativeness order to Le Cam’s quantitative, deficiency-based generalization; a parallel line priced information before acquisition (value of information, VOI). Together with the two-point lower-bound technology and the Jensen–Shannon divergence, this cluster supplies every tool our static layer uses, and the paper credits each at the point of use. Key papers. Blackwell (1951) introduced the “more informative” partial order on experiments and proved its equivalence to garbling and to pointwise domination of risk functions on all decision problems—the comparison theorem behind our channel-augmentation argument. Blackwell (1953) characterized when two experiments are equivalent (mutual garblings), connecting the order to sufficiency and randomization criteria. Le Cam (1964) gave the measure-theoretic treatment of sufficiency and approximate sufficiency, introducing the deficiency that quantifies how far one experiment is from dominating another, and Le Cam (1986) embedded the order into the asymptotic theory of statistical decision problems. Blackwell and Girshick (1954) systematized the underlying framework—games against nature, risk functions, Bayes and minimax rules—on which all of the above rests. Tsybakov (2009) is the standard textbook source for the two-point method: its Lemma 2.1 is Scheffé’s identity and its Theorem 2.2 is the two-point minimax lower bound whose equal-prior form reads Bayes error = (1 − TV)/2. Devroye et al. (1996) develop the probabilistic theory of pattern recognition; their Theorem 2.1 gives L∗ = E[min(η, 1 − η)] and their §3.2 the equal-prior total-variation form. Fano (1961) is the source of Fano’s inequality linking error probability to conditional entropy, the classical route from information to error for many hypotheses. Lin (1991) introduced the Jensen–Shannon divergence. Howard (1966) defined the value of an information source as the difference between the with-information and without-information optimal decision values, computed prior to acquisition; DeGroot (1970) developed the Bayesian decision-theoretic foundations—prior-to-posterior analysis, Bayes risks, and the worth of experiments—in the form used throughout sequential decision analysis. 46

Self-Certification of Representation Adequacy

Overlap. The representation map f is a deterministic garbling of the identity experiment, so the nonnegativity of the aliasing regret in Theorem 1 and the monotonicity of the verify gain in Theorem 6 are direct corollaries of Blackwell risk domination. Theorem 1 itself is a Bayes risk grouping identity on a partition, used as a definitional tool and not claimed as new. Theorem 3 is a standard application of the two-point method of Tsybakov (2009) with the Bayes-error forms of Devroye et al. (1996); the paper’s contribution there is confined to taking the tested label to be the decision-theoretic adequacy AdeqE (f ) rather than the parameter. Theorem 4 is a direct rewriting of the Jensen–Shannon definition of Lin (1991), not a new information-theoretic principle. The threshold in Theorem 6 instantiates the Howard–DeGroot with-minus-without template (Howard, 1966; DeGroot, 1970), with the uninformed benchmark computed by the two-point bound. Fano-type many-hypothesis bounds (Fano, 1961) are explicitly not needed: the certification problem is binary, so the label test aggregates to two mixtures and retains the total-variation closed form. Strict difference. This cluster compares exogenously given experiments from the analyst’s viewpoint, with the parameter as the label of interest, and VOI prices information about an externally given state. The present paper keeps the order-theoretic tools but changes what is tested and who tests it: the label is the adequacy of the agent’s own representation relative to its task—an event derived from the decision problem itself—and the question is whether an agent confined to a loss-feedback-free internal channel can certify it, and at what price an external channel should be purchased against the resulting indistinguishability floor. No work in this cluster takes a decision-derived property as the tested label or prices the certification of one’s own representation. 6.2 Sequential Experimental Design and Active Hypothesis Testing Trajectory. This is the oldest and, mathematically, the strongest of the four audited directions. Beginning with Chernoff’s sequential design of experiments, the line asks how to allocate observations among available experiments so as to identify the true hypothesis at minimal expected cost, subject to an error constraint that is driven to zero. The mature form of the answer is now classical: the leading term of the optimal expected cost scales as log(1/δ) divided by a max-min, over observation allocations, of Kullback–Leibler information rates, and explicit policies achieve the same constant. This cluster is the principal reason the present paper restricts its novelty claim to the semantics of the label and the cost rather than to the asymptotic form of the bounds. Key papers. Chernoff (1959) founded the area: for testing among finitely many hypotheses when the experimenter chooses at each stage which experiment to perform, his procedure A is asymptotically optimal as the per-observation cost tends to zero, and the leading term of the risk is the per-observation cost times log(1/δ) divided by a max-min, over experiment allocations, of KL information rates—the dual of an allocation program, i.e., precisely the shape of the linear program that defines our certification complexity Ci . Nitinawarat et al. (2013) formulated controlled sensing for multihypothesis testing in a dynamic-systems setting, where a controller selects at each step one of several sensing actions with control-dependent observation kernels, and established asymptotically optimal policies with the corresponding max-min (Chernoff-type) constant. Nitinawarat and Veer47

Huang

avalli (2015) solved the cost-heterogeneous extension: with non-uniform sensing costs, their Theorem 5.1 lower-bounds the expected cumulative sensing cost by − log δ divided by the maximum over allocations of the minimum over alternative hypotheses of the informationrate-to-cost-rate ratio, up to a vanishing (1 + o(1)) factor, and exhibits a test achieving the same constant. In other words, the cost-heterogeneous, information-constrained allocation program—with a matching upper bound—is established there to first-order precision. Naghshvar and Javidi (2013) studied active sequential hypothesis testing in a Bayesian framework, where the decision maker adaptively selects sensing actions on the basis of an information state; they established asymptotic optimality of their proposed policies and lower bounds on the total cost, and their Remark 2 records exactly the coverage-type necessary condition—accumulated expected information must exceed log(1/δ)—that our lower-bound argument also passes through. We therefore do not claim that inequality, or the log(1/δ)/max–min KL scaling, as new. Overlap. The overlap is substantial and we state it plainly. Our model M1 inherits the mechanics of this cluster—actions induce observation kernels, kernels induce KL rates, rates determine the log(1/δ) scale—and our Theorem 8 is proved with the same change-of-measure technology, via the transportation lemma of Kaufmann et al. (2016) discussed in Section 6.3 below. The audit verdict on the simple form is a no-go: were the environment class a simple binary pair with known kernels, i.i.d. observations, and deterministic exogenous sensing fees, renaming the hypothesis as an adequacy event would be an isomorphism, and the problem would be covered by Chernoff (1959) and Nitinawarat and Veeravalli (2015) with constants matching to first order. Our boundary instance B3 makes the inclusion explicit and checkable: in the two-environment special case, our constant reduces to the bang-bang form Ci = mina gi (a) / d(Kia ∥ Kja ), which coincides with the binary form of the bound of Nitinawarat and Veeravalli (2015, Theorem 5.1); the sequential theorem of this paper properly contains that known special case. Strict difference. Three elements of model M1 are absent from every work in this cluster, and each is load-bearing. (i) Cost semantics. The sensing cost in Chernoff (1959); Nitinawarat et al. (2013); Nitinawarat and Veeravalli (2015); Naghshvar and Javidi (2013) is an exogenous fee attached to the observation action and decoupled from any task; the task loss, if any, is borne by a separate terminal or classification action. In M1 the cost gi (a) is the task loss incurred by the very action whose observation carries the label information: certification expense and task performance are one object, so the lower bound prices certification in units of decision quality. (ii) Object of inference. The hypothesis in this cluster is the iden∗ (f (H)) = R∗ (H)] tity of the environment, an exogenous parameter. Our label Θi = 1[RE Ei i is a property of the agent’s representation relative to the loss structure; it is undefined without the decision problem, and two environments that are statistically far apart can share a label while statistically close environments can differ. (iii) The alternative set. Accordingly, Alt(i) is defined here by adequacy-label disagreement rather than by hypothesis distinctness. This changes the meaning, though not the algebra, of the max-min constant: Ci is the price of certifying adequacy, and hopeless configurations (boundary instance B1, where Ci = +∞) acquire a representation-level reading—the agent’s own description of the world is unverifiable—that has no counterpart in this cluster. The composite extension, in which 48

Self-Certification of Representation Adequacy

several environments share a label and the certification cost involves the worst case within the alternative set, is likewise outside the simple-hypothesis framework of these works. 6.3 Pure Exploration and Best-Arm Identification Trajectory. Fixed-confidence best-arm identification (BAI) developed, over the last decade, the sharpest form of the same log(1/δ) theory: non-asymptotic lower bounds via change of measure, asymptotically matching allocation-tracking algorithms, and generalizations to arbitrary answer structures and to heterogeneous action costs. On the regret side of the bandit literature, linear-program-form lower bounds with matching algorithms are even older and by now standard. This cluster is the source of our proof templates; it is also the cluster whose two nearest sub-lines leave, provably, an empty intersection with our setting. Key papers. Kaufmann et al. (2016) established the sample-complexity theory of BAI: the transportation lemma (their Lemma 1) converts a δ-correctness constraint into an inequality between expected pull counts and KL divergences, yielding E[τ ] ≥ T ∗ (µ) kl(δ, 1−δ) with T ∗ (µ) an allocation program, together with the elementary bound kl(δ, 1 − δ) ≥ log(1/(2.4δ)) that we also use. Garivier and Kaufmann (2016) gave the matching algo√ rithm: Track-and-Stop, which tracks a plug-in estimate of the optimal allocation with trate forced exploration and stops by a generalized likelihood ratio rule, achieves the lower bound’s constant asymptotically; a key structural fact of their analysis (their Lemma 3) is that the pairwise separation achievable between two bandit models itself depends on the allocation, which makes the lower-bound program genuinely max-min over a continuum of alternatives. Degenne and Koolen (2019) generalized the answer structure to multiple correct answers under an (almost) arbitrary answer map, characterized when a sample complexity exists, and identified the continuity requirements that general answer maps impose. Kanarios et al. (2024) introduced cost-aware BAI, in which each arm carries a cost distribution in addition to a reward distribution and the goal is to identify the best-reward arm at minimum expected cost; their lower bound has the cost-weighted characteristic-time form—a max-min information rate normalized by the expected cost of the allocation— and their CTAS algorithm matches it asymptotically, with a simpler rule proved optimal in two-armed models. On the regret side, Graves and Lai (1997) established asymptotic allocation-program lower bounds for adaptively controlled Markov chains, and Combes et al. (2015, 2017) made LP-form regret lower bounds with asymptotically matching algorithms standard for combinatorial and general structured bandits. Finally, Kaufmann and Koolen (2021) systematized mixture-martingale techniques for sequential tests and anytime confidence intervals, the source of the time-uniform threshold technology used in our stopping-rule analysis alongside the thresholds of Garivier and Kaufmann (2016). Overlap. Our lower bound is a direct application of the transportation lemma of Kaufmann et al. (2016), with actions playing the role of arms and channel outputs the role of arm observations; the δ-correctness constraint enters exactly through their change-of-measure inequality. Our matching policy (certification track-and-stop, CTS) is the Track-and-Stop template of Garivier and Kaufmann (2016) adapted to√the cost-weighted design: plugin estimation of the optimal allocation, C-tracking with t-rate forced exploration, and a 49

Huang

GLR-type stopping rule with a time-uniform threshold of the type of Kaufmann and Koolen (2021). The certification complexity Ci has two equivalent forms—a covering linear program and a cost-weighted max-min characteristic time—whose second form parallels the characteristic time of Kanarios et al. (2024); the equivalence itself is a strong-duality and minimax computation that we carry out for our setting. None of these tools is claimed as new. Strict difference. The two nearest sub-lines each cover one of our two distinguishing features and miss the other, so their intersection is empty. Degenne and Koolen (2019) admit essentially arbitrary answer maps—our adequacy-label partition could be expressed in their framework—but price time homogeneously: their objective is E[τ ], and there is no notion of an action whose cost is the task loss it incurs. Kanarios et al. (2024) admit heterogeneous costs but their answer map is parameterized by reward means (the identity of the best arm), an exogenous object; the alternative set is defined by mean comparisons, not by a property derived from a decision problem. In M1, simultaneously: (a) the alternative set Alt(i) = {j : Θj ̸= Θi } is defined by the adequacy label, a decision-theoretic, representationand loss-dependent object—the answer P map is decision-theorized ; and (b) the objective is the cumulative task loss R(π, Ei ) = Ei [ t≤τ gi (At )] of the same actions that generate the discriminating evidence—the cost is task-loss-ized. Both modifications are load-bearing: the question a certifying agent faces is precisely how much task performance it must spend to buy a δ-certified answer about its own representation, and neither the answer-map generality of Degenne and Koolen (2019) nor the cost generality of Kanarios et al. (2024) addresses that coupling. Two further cautions recorded by the audit delimit the claim from below. First, a single-constraint linear program with fixed per-action information values is only the |Alt| = 1 degenerate special case; in general the correct lower-bound program is the costweighted max-min characteristic time, which is the form we adopt. Second, the LP-form lower bound and its matching-algorithm technology are standard on the regret side (Graves and Lai, 1997; Combes et al., 2015, 2017); what is not standard is the object the program computes a price for. 6.4 Information-Directed Sampling and the Value of Information Trajectory. The information-ratio line measures the price of information inside a regretminimization problem: each action is evaluated by the ratio of its expected squared regret to the information it provides about a target quantity, and Bayesian regret bounds follow from entropic arguments. Within all four audited directions, this is the only line in which a single action explicitly carries both a task loss and an information gain—the same coupling as our model M1—and its satisficing variant is conceptually the closest to our certification motivation. Key papers. Russo and Van Roy (2016) developed the information-theoretic analysis of Thompson sampling, introducing the information ratio and proving Bayesian regret bounds p of the form Ψ · H(A∗ ) · T , which tie regret to the entropy of the optimal action and to a worst-case ratio Ψ. Russo and Van Roy (2018) turned the ratio into an algorithm, information-directed sampling (IDS), which selects actions minimizing expected squared regret per unit of information gain; their revealing-action construction makes explicit that 50

Self-Certification of Representation Adequacy

one action simultaneously incurs one-step loss and reduces uncertainty about A∗ . Russo and Van Roy (2022) studied satisficing in time-sensitive bandit learning, where the learner need only identify an action that is good enough rather than the optimal one; the satisficing viewpoint is the nearest analogue, in this literature, of certifying adequacy instead of identifying an exact answer. The classical VOI template of Howard (1966); DeGroot (1970), discussed in Section 6.1, underlies our static verify threshold. Overlap. The coupling structure of M1—one action, one task loss, one piece of label evidence—is the defining feature of IDS, and the satisficing objective of Russo and Van Roy (2022) shares our motivation that an agent should be able to settle for a certified “adequate” state of knowledge rather than an exact identification. The information-theoretic idiom (entropy of the target, information gained per action) is common ground with our coveragestyle arguments. Strict difference. Three separations, each checkable. (i) Framework. IDS √is a regret framework: its guarantees are ratio-based Bayesian regret bounds of the form Ψ · H · T , with no δ-certification target, no stopping time, and no terminal decision. Our bounds are additive coverage statements at the Chernoff scale log(1/δ), with an explicit stopping rule and a δ-correct terminal answer; the audit notes that ratio-based and additive analyses answer genuinely different questions, and neither type of bound implies the other. (ii) Information object. IDS learns about the optimal action A∗ , an object endogenous to the reward function; our label Θ concerns the adequacy of the agent’s representation, defined through a Bayes-risk equality under the agent’s own f , and it can vary independently of which action is optimal. (iii) What is priced. Classical VOI prices an exogenously given experiment for an external decision maker; our static layer prices an external verify channel against an internal indistinguishability floor, yielding the closed-form threshold c∗ = (T r/2)(1 − TV) of Theorem 6, and the sequential layer prices certification itself in task-loss units. Neither the ratio line nor classical VOI poses the question “what does it cost, in task performance, to certify one’s own representation to level δ.” 6.5 POMDPs, Controlled Sensing, and State Abstraction Trajectory. Partially observable control was made tractable by the discovery that the posterior state distribution is a sufficient information state, reducing the problem to a fully observable MDP on belief space; the line then developed planning algorithms, the control of the observation process itself (controlled sensing), and, on the representation side, formal criteria for which state abstractions preserve value. This cluster defines the classical answer to “what should the agent’s state be,” against which our adequacy question is positioned. Key papers. Åström (1965) proved that the conditional distribution of the state given the action–observation history is a sufficient information state for Markov processes with incomplete state information, enabling the belief-space reformulation of partially observable optimal control. Kaelbling et al. (1998) brought POMDPs into AI planning, formalizing the belief-MDP reduction and developing exact and approximate solution algorithms. Krishnamurthy (2016) gives a comprehensive modern treatment, from Bayesian filtering through structural properties of POMDP value functions to controlled sensing, where the agent actively selects sensors or observation channels. Krishnamurthy and Wahlberg (2009) studied 51

Huang

multi-armed bandits whose arms are partially observed Markov decision processes—playing an arm yields a reward and an informative observation at once—and established structural results (monotone and threshold policies, myopic bounds) for the resulting discountedreward problem. On the abstraction side, Givan et al. (2003) developed equivalence notions and model minimization for MDPs, showing that stochastic-bisimulation-minimized models preserve optimal value functions; Li et al. (2006) unified five state-abstraction notions into a single framework with their implication structure; Littman et al. (2002) introduced predictive state representations, replacing belief over latent states by vectors of predictions of future tests; and Sutton and Barto (2018) is the standard reinforcement-learning textbook whose treatment of finite MDPs and value iteration our analysis of Counterexample A follows. Overlap. Belief-state sufficiency delimits our problem domain: by Åström (1965), inadequacy can arise only when the representation the agent actually maintains is strictly coarser than the belief state, which is exactly where our boundary analysis (Theorem 5) locates the problem. Controlled sensing shares the mechanics of M1—actions induce observation kernels, kernels induce KL rates—and Krishnamurthy and Wahlberg (2009) shares the further feature that the played action produces reward and information together. Strict difference. In every controlled-sensing model we audited (Krishnamurthy, 2016; Nitinawarat et al., 2013; Nitinawarat and Veeravalli, 2015), observation actions carry only sensing costs, and the task loss is borne by a separate terminal, stopping, or classification action; the coupling of task loss and label evidence on a single action is absent from this cluster. Krishnamurthy and Wahlberg (2009) does share the coupled action but pursues a discounted-reward objective: there is no hypothesis to certify, no δ-correctness constraint, and no stopping problem, so no certification complexity arises. The abstraction works (Givan et al., 2003; Li et al., 2006; Littman et al., 2002) answer “which representations preserve value” forward, from the analyst’s viewpoint, under a known model: given the model, one verifies value preservation or minimizes the model. We fix the preservation condition as the definition of the label Θ and ask the inverse question—whether the agent can certify this unknown event by itself, under an unknown environment, and at what taskloss price. Controlled sensing chooses among known observation channels to reduce state uncertainty; it does not test whether the state representation itself is decision-theoretically adequate, which is the label under test here. 6.6 Dual Control and Adaptive Control Trajectory. Dual control studies systems in which an action has two simultaneous roles: it changes the physical or economic state, and it changes what the controller can learn about an unknown state or parameter. Feldbaum introduced this “dual effect” viewpoint and formulated control actions as experiments whose value must be traded against regulation loss. Bar-Shalom and Tse analyzed the dual effect, certainty equivalence, and separation; their results identify special conditions under which the probing role disappears. Witsenhausen showed, in a discrete-time setting, that separation of estimation and control is not a generic principle. Later work, including information-theoretic dual adaptive control, made the information value of probing explicit. 52

Self-Certification of Representation Adequacy

Key papers. Feldbaum (1963) is the early explicit statement of the dual-control problem: the experiment chosen by the controller also changes the controlled state. Bar-Shalom and Tse (1974) and Tse and Bar-Shalom (1975) formalized the dual effect and generalized certainty equivalence, while Witsenhausen (1971) established a sharp limitation on a naive separation principle. Yame (1987) connected dual adaptive control to information-theoretic value criteria. These works are about control under unknown dynamics or parameters; they do not use the decision-theoretic adequacy of the controller’s own representation as the hypothesis label. Overlap. The policy-dependent transcript in Section 4 is closest to this line. In both settings an action can be informative and costly at the same time, the posterior affects which action is attractive, and a probing action can change the subsequent control problem. The dual effect also explains why a routine transplantation of the fixed-kernel transportation proof is not available once certification changes the regime. Strict difference and audit boundary. The comparison must be kept precise. The dual-control papers above infer an external plant state, parameter, or model and optimize expected control performance (often over a finite or discounted horizon); they do not stop and declare a fixed-confidence label that the agent’s representation is decision-theoretically ∗ (f (H)) = R∗ (H)] is the inference object, adequate. In Model M1, by contrast, Θi = 1[RE Ei i Alt(i) is defined by disagreement in that label, and the cost charged to an informative action is its task loss under the unknown environment. The present paper therefore does not claim that dual control is unrelated; it claims only that, within the dual-control, adaptive-control, and POMDP references audited here, we found no lower-bound/matching-policy result that combines this adequacy label, this task-loss currency, and a δ-correct stopping objective. The certification-driven switching and Repair problems remain open precisely because they require a dual-control/POMDP model with a moving label and policy-dependent kernels. 6.7 Partial Monitoring, Concept Drift, and Robust Dynamic Programming Trajectory. Three neighboring lines study decision making under imperfect or shifting information about the environment: partial monitoring, where feedback is a noisy function of the hidden outcome; concept drift, where the environment changes and must be tracked from observable errors; and robust dynamic programming, where the transition law is known only to lie in a given uncertainty set. Each relaxes one idealization of the classical setting, and each makes assumptions orthogonal to ours. Key papers. Rustichini (1999) founded the general theory of regret minimization under partial monitoring, characterizing when per-period regret can be driven to zero from an exogenously given feedback structure. Cesa-Bianchi et al. (2006) established the canonical O(T 2/3 ) regret bound for finite partial-monitoring games. Bartók et al. (2014) completed the classification: every finite game is trivial, easy, hard, or hopeless, with √ partial-monitoring 2/3 e minimax regret Θ(1), Θ( T ), Θ(T ), or Θ(T ) respectively, and the hopeless class is characterized algebraically—the signal matrices cannot distinguish two Pareto-optimal actions, so the feedback is structurally uninformative and linear regret is unavoidable. Widmer and Kubat (1996) introduced windowed adaptive learning for concept drift and hidden context (the FLORA family), presupposing observable prediction errors as the drift signal. Iyengar 53

Huang

(2005) developed robust dynamic programming, solving finite- and infinite-horizon dynamic programs against a rectangular uncertainty set of transition kernels. Overlap. Our boundary instance B1 is the fixed-confidence analogue of the hopeless class of Bartók et al. (2014). When some j ∈ Alt(i) satisfies d(Kia ∥ Kja ) = 0 for every action a, the covering linear program is infeasible, Ci = +∞, and δ-certification is infeasible at any cost; both their criterion and ours are algebraic certificates of structural information insufficiency, theirs expressed in signal-matrix terms, ours in KL terms. More broadly, all three lines share our theme of acting under limited or shifting information about the environment, and partial monitoring shares the superficial feature that realized losses are not directly observed. Strict difference. Each line differs on a specific axis. Partial monitoring takes the feedback structure as exogenous and classifies the regret rate attainable within it; it poses no certification meta-decision—whether to stop and declare a label with a δ guarantee, and at what price—which is the object our Theorems 8 and 9 compute, and outside the hopeless class its feedback, however noisy, remains loss-correlated, whereas our internal transcripts carry no loss feedback at all. Concept-drift methods of the type of Widmer and Kubat (1996) detect change from observable prediction-error signals; their feasibility rests exactly on the loss feedback that our model excludes, which is why drift adaptation is listed in our boundary analysis as a sufficient condition for certification rather than a competing method. Robust dynamic programming (Iyengar, 2005) optimizes against a known uncertainty set; unidentifiability—two environments that no transcript statistic can separate, the TV = 0 or d = 0 regime in which our floor sits at chance level or Ci = +∞—lies outside its scope by construction. 6.8 A Concurrent Aliasing Decomposition: Regret as Weighted Forgetting The work. The concurrent preprint Bennett (2026) proves a cell-level aliasing decomposition of the same type as our Theorem 1: for a fixed representation M and a finite evaluation distribution of history–test pairs, the minimal average normalized (betting) regret over M based policies equals Kρ(M ), the minimum margin-weighted deletion cost that renders the optimal bet single-valued on every (representation, test) cell; the regret of an actual policy decomposes policy-wise as Kρ(M ) plus within-cell misreporting; and the same quantity is bridged to the weighted weakness of Stack Theory and to a generalization probability e−Kρ . Overlap. The irreducible regret in both works arises from action conflict inside aliasing cells, and their Remark 14 identifies Kρ as a weighted Bayes error on the partition. This is consistent with our attribution of Theorem 1 as a standard grouping identity used as a definitional tool; both works, in different idioms, assign the same epistemic status to the identity. Bennett (2026) is the closest same-type neighbour we are aware of, and the separation below is exact. Strict difference. Four axes separate the two results. (i) Loss convention. Their regret is a normalized betting regret with margin weights ρi ; Theorem 1 uses the unnormalized Bayes-risk difference with losses in [0, 1] (their Remark 7 records that the unnormalized variant changes the weights, to 2mi , while the cell structure survives). (ii) Cell definition. 54

Self-Certification of Representation Adequacy

Their cells are (representation × diagnostic test) pairs; Theorem 1 groups by the representation cells Hu alone. (iii) Decomposition object. Theirs is policy-wise (actual-policy regret equals Kρ plus within-cell misreporting); ours is a difference of two Bayes optima, ∗ (U ) − R∗ (H). (iv) Goal. They pursue Stack-Theory and generalizability consequences RE E and do not study internal certification of adequacy under unidentifiability, the value of an external verification channel, or sequential certification complexity; our impossibility bound (Theorem 3), verification threshold (Theorem 6), and certification lower and upper bounds (Theorems 8 and 9) have no counterpart there. Conversely, their Stack-Theory consequences have no counterpart here; the two papers share a lemma-type identity and diverge from it in orthogonal directions. 6.9 Positioning: The Exact Scope of the Present Combination We close by stating precisely what is claimed, what is credited, and what the closest works lack. The claim is a combination, scoped to the four audited directions: a model in which (a) the alternative set is defined by a decision-theoretic adequacy label, Alt(i) = {j : Θj ̸= Θi } ∗ (f (H)) = R∗ (H)]; (b) the cost charged is the cumulative task loss of with Θi = 1[RE Ei i the same actions that carry the discriminating information; and (c) the objective is a δcertification with stopping, matched at the log(1/δ) scale by an explicit policy. Within the sequential-design (Chernoff, 1959; Nitinawarat et al., 2013; Nitinawarat and Veeravalli, 2015; Naghshvar and Javidi, 2013), pure-exploration (Kaufmann et al., 2016; Garivier and Kaufmann, 2016; Degenne and Koolen, 2019; Kanarios et al., 2024), controlled-sensing (Krishnamurthy, 2016; Krishnamurthy and Wahlberg, 2009), and information-directed (Russo and Van Roy, 2016, 2018, 2022) lines surveyed above, no same-type result contains all three elements, and the reason in each case is concrete rather than rhetorical: Nitinawarat and Veeravalli (2015) solves the cost-heterogeneous allocation problem with a matching bound but its cost is an exogenous sensing fee and its label an exogenous hypothesis; Kanarios et al. (2024) has heterogeneous costs but an answer map parameterized by reward means; Degenne and Koolen (2019) has arbitrary answer maps but a homogeneous time objective; IDS (Russo and Van Roy, 2018, 2022) has the loss–information coupling but no certification target or stopping; the POMDP-bandit line (Krishnamurthy and Wahlberg, 2009) has the coupled action but no testing objective; and the partial-monitoring classification (Bartók et al., 2014) has the structural-uninformativeness criterion but no certification meta-decision. We also record the audit’s verdict on the degenerate form: with a simple binary hypothesis pair, known kernels, and deterministic exogenous costs, the problem is covered by Chernoff (1959) and Nitinawarat and Veeravalli (2015), and our constant reduces to their binary special case (boundary instance B3)—the paper’s contribution lies in the decision-theoretic label, the task-loss semantics of the cost, and the boundaries (B1–B3), not in the asymptotic form of the bounds. Accordingly, every individual technique is credited as a standard tool at the point of use: the grouping identity of Theorem 1 and its instantiation are Bayes-risk bookkeeping on a partition (Blackwell, 1951; Le Cam, 1964; Devroye et al., 1996); Theorem 3 is the Le Cam two-point method with the Bayes-error–TV identity (Tsybakov, 2009; Devroye et al., 1996; Le Cam, 1986); Theorem 4 is a rewriting of the Jensen–Shannon definition (Lin, 1991); the verify threshold of Theorem 6 is the Howard–DeGroot with-minus-without computation 55

Huang

(Howard, 1966; DeGroot, 1970) with a Blackwell augmentation argument (Blackwell, 1951; Le Cam, 1964); Theorem 8 applies the transportation lemma of Kaufmann et al. (2016); and the matching policy of Theorem 9 is the Track-and-Stop template of Garivier and Kaufmann (2016) with time-uniform thresholds of the type of Kaufmann and Koolen (2021), adapted to the cost-weighted design of Kanarios et al. (2024). The sole incremental claim is the combination: the adequacy label as the object of a sequential test, the task loss as the currency of its certification, the matching constant Ci read as the decision-theoretic price of self-certification, and an explicit boundary package (Counterexample A; B1–B3) delimiting where the combination degenerates into, or diverges from, the known results above.

7 Conclusion This paper studies a narrow but consequential question: given a fixed, possibly lossy representation of history, what task loss is necessary to certify whether that representation remains decision-theoretically adequate? The answer is deliberately not a universal impossibility claim about agents that act on changing worlds. Counterexample A shows why that broader statement is false. The relevant obstruction is observational: two environments can induce the same internal transcript laws while assigning different adequacy labels to the same representation. For that fixed-representation problem, the static layer makes three points precise. First, ∗ (f (H)) = R∗ (H), and its failure has an exact cell-wise Bayesadequacy is the equality RE E risk decomposition. Second, the difficulty of internally certifying that label is governed by the total variation distance between the two label-conditional transcript laws; equality of those laws yields both Bayes error 1/2 and zero mutual information. Third, under the explicit A1 switching convention, a one-shot external verification is worth purchasing exactly up to the corresponding avoidable loss. These are statements about a fixed observed transcript and a fixed fallback convention, not claims that an agent can reconstruct or repair its own state space from internal evidence. The sequential layer gives the paper’s main fixed-kernel result. In Model M1, the agent chooses actions that simultaneously incur environment-indexed task losses and generate environment-indexed observations. For every confidence level δ ∈ (0, 12 ), the certification complexity C(Ei , f ) is the minimum task loss needed to accumulate one unit of evidence against every label-disagreeing alternative. Theorem 8 and Theorem 9 establish the matching first-order characterization
Rδ∗ (Ei ) = C(Ei , f ) log(1/δ) + o log(1/δ) (δ → 0), under the stated finite-class, conditional-independence, identifiability, and full-support assumptions. The proof techniques are standard controlled-testing and Track-and-Stop tools; the model contribution is to make the tested property decision-theoretic adequacy and to charge the information-gathering actions in the currency of their task loss. The limits matter as much as the theorem. Certification-driven switching changes the available kernels and post-certification loss stream; Repair changes the representation and therefore the label being certified. Neither is covered by Model M1, and no claim in this manuscript extends the fixed-kernel lower or upper bound to either setting. The kernelswitching construction in Section 4 is a boundary example, not a dynamic theorem. A 56

Self-Certification of Representation Adequacy

genuine extension must jointly model evidence, post-certification control, and changes in the representation itself. The nearest dual-control and POMDP-bandit references have now been audited at the theorem and model level; establishing the corresponding moving-label, policy-dependent result is the substantive next mathematical problem.

Appendix A. Proof of the Lower Bound (Theorem 8) Guide to this section. This section gives a self-contained proof of Theorem 8 for model (M1), including the transportation lemma, the information constraint, the LP scaling argument, the equivalence of the two forms of the certification complexity Ci , the lim inf statement, the three boundary examples B1/B2/B3, and the non-triviality note NT1. The logical dependencies are as follows. Subsection A.1 formalizes the model and restates the hypotheses. Subsection A.2 proves the transportation Lemma 14, built up through the finitehorizon density (Lemma 15), the likelihood-ratio martingale (Lemma 16), the stopped density (Lemma 17), a Wald-type identity (Lemma 18), and Bernoulli contraction (Lemma 19). Subsection A.3 turns δ-correctness into the information constraint (Lemma 21, via the monotonicity Lemma 20) and proves the estimate kl(δ, 1 − δ) ≥ log(1/(2.4δ)) (Lemma 23); combined with Lemma 14 this yields the per-alternative bound of Corollary 24. Subsection A.4 carries out the LP scaling and proves Theorem 8. Subsection A.5 proves the equivalence of the LP and max–min forms of Ci (Theorem 27), the statement used by the upper-bound design of Appendix B. Subsection A.6 derives the lim inf form (Corollary 29), Subsection A.7 treats the boundary examples B1/B2/B3 (Propositions 31, 32, 34), and Subsection A.8 proves the non-triviality note NT1 (Proposition 35). Subsection A.9 collects the completeness self-check and the residual statements.

Provenance and claims. All lower-bound techniques used here are standard tools from the sequential identification literature: the transportation (change-of-measure) inequality is that of Kaufmann et al. (2016, Lemma 1) applied under the isomorphism action ↔ arm, channel output ↔ arm reward ; the inequality kl(δ, 1 − δ) ≥ log(1/(2.4δ)) is the standard estimate of Kaufmann et al. (2016); the two-point degeneration B3 is compared against Nitinawarat and Veeravalli (2015, Theorem 5.1); the hopeless boundary B1 corresponds to the hopeless class of partial monitoring (Bartók et al., 2014). The contribution of the present work is not the lower-bound machinery; it is the model itself—the alternative set Alt(i) defined through adequacy labels and the coupling to task loss g—and the decisiontheoretic semantics of the constant C(E, f ).

Conventions. All logarithms are natural. For probability vectors p, q on a finite space, P d(p∥q) := y p(y) log p(y) q(y) with 0 log 0 = 0, 0 log(0/0) = 0, and d(p∥q) = +∞ if p ̸≪ q. 2 For Bernoulli laws, kl(p, q) := p log pq + (1 − p) log 1−p 1−q for (p, q) ∈ (0, 1) , extended to 2 [0, 1] by continuity where possible and by kl(p, 0) = +∞ for p > 0, kl(1, q) = log(1/q), kl(0, q) = log(1/(1 − q)), kl(0, 0) = kl(1, 1) = 0. Products with scalars follow 0 · (+∞) = 0 and c · (+∞) = +∞ for c > 0.

57

Huang

A.1 Model (M1): Complete Formalization Environments and labels. The environment class is finite: E = {E1 , . . . , EK }, K ≥ ∗ (f (H)) = 2. The representation f is fixed. For each i, the adequacy label Θi := 1{RE i ∗ (H)} ∈ {0, 1} is a deterministic constant determined by the Bayes-risk framework RE i of the companion paper; the only randomness in the model comes from the interaction transcript. We write Θ(i) and Θi interchangeably. The alternative set is Alt(i) := { j ∈ {1, . . . , K} : Θj ̸= Θi }. If Alt(i) = ∅ (all environments share the label of i), the certification question is trivial; we adopt the convention Ci = 0 in that case (the LP below has no constraints; the max–min form has inf over an empty set equal to +∞, whose reciprocal is 0). Henceforth Alt(i) ̸= ∅. Actions, channels, losses. The action set A is finite. Under environment Ei , action a produces an observation through the channel Kia , a probability distribution on the finite observation space Y, with mass function kia (y) := Kia ({y}), and incurs a task loss gi (a) ∈ [0, ∞) (audit costs are absorbed into g; e.g. a verify action carries caudit inside g). Canonical probability space. randomization sequence. Let

Randomized policies are allowed through an external

Ω := Y N × [0, 1]N ,

ω = (y1 , y2 , . . . ), (u1 , u2 , . . . ) ,

equipped with the product σ-field F generated by cylinders (Y finite carries the discrete σ-field; [0, 1] the Borel σ-field). Coordinate maps are Yt (ω) = yt , Ut (ω) = ut . Policies. A policy is a sequence π = (πt )t≥1 of measurable maps πt : (A × Y)t−1 × [0, 1] → A. The action process is defined recursively:
A1 (ω) := π1 (u1 ), At (ω) := πt A1 , Y1 , . . . , At−1 , Yt−1 , ut , t ≥ 2. Since all spaces involved are standard Borel (finite or [0, 1]), measurability is preserved by the recursion. Environment measures. For each i, Pi is the unique probability on (Ω, F) such that: (i) (Ut )t≥1 are i.i.d. Unif[0, 1]; (ii) conditionally on (A1 , Y1 , . . . , At−1 , Yt−1 , Ut ), the observation Yt is independent of the past and of Ut+1 , Ut+2 , . . . and satisfies Pi (Yt = y | ·) = kiAt (y). Existence and uniqueness follow from the Ionescu–Tulcea theorem: the transition kernel from π (history,u) histories to (Ut , Yt ) is the Markov kernel Unif[0, 1](du) ⊗ Ki t (dy); all kernels are regular because every space is standard Borel (finite spaces and [0, 1]). Point (ii) is precisely hypothesis (H1) (conditional independence of channel outputs given the action history). Because the Ut ’s have the same law under every Pi , the policy-induced randomization does not enter likelihood ratios between environments (Lemma 15). Filtration.

Set F0 := {∅, Ω} and
Ft := σ U1 , . . . , Ut+1 , A1 , . . . , At+1 , Y1 , . . . , Yt ,

t ≥ 1.

Then At+1 is Ft -measurable, Yt is Ft -measurable, and As , Ys , Us are Ft -measurable for s ≤ t. This is the natural filtration of the interaction (with the usual harmless enlargement that reveals the next randomization seed; it does not affect any argument below). 58

Self-Certification of Representation Adequacy

Stopping and terminal decision. A strategy is a triple (π, τ, Θ̂) where τ is a stopping time with respect to (Ft ), and Θ̂ : Ω → {0, 1} is Fτ -measurable, with Fτ := { F ∈ F : F ∩ {τ ≤ t} ∈ Ft ∀t ≥ 0 }, the standard stopped σ-field (it is a σ-field: closed under complement and countable unions, checked directly from the definition). A strategy is admissible if τ < ∞ Pi -a.s. for every i = 1, . . . , K. (Without admissibility, Θ̂ is undefined on {τ = ∞} and error probabilities are not well-defined; admissibility is therefore part of the correctness requirement, not an extra restriction. It is also what excludes the vacuous “never stop, never err” strategy. A relaxed variant in which the decision is issued only on {τ < ∞} is used in exactly one place—the free-rider construction B2, Subsection A.7.2—and is flagged there; the lower bound itself is proved for admissible strategies.) P P Counts and risk. For a ∈ A, Na (τ ) := τt=1 1{At = a} = t≥1 1{t ≤ τ, At = a} ∈ [0, ∞] (the sum is well-defined since {t ≤ τ } = {τ ≤ t − 1}c ∈ Ft−1 and At are measurable). The expected certification cost (risk) of strategy (π, τ, Θ̂) in Ei is R(π, Ei ) := Ei

τ hX

i X gi (At ) = gi (a) Ei [Na (τ )],

t=1

a∈A

where the exchange of sum and expectation is Tonelli’s theorem for nonnegative summands (valid also when the value is +∞). δ-correctness (fixed-confidence PAC).

A strategy is δ-correct if it is admissible and

max Pi Θ̂ ̸= Θi

1≤i≤K

≤ δ.

Recall Θi is deterministic, so {Θ̂ ̸= Θi } ∈ Fτ is an ordinary event. Hypotheses (restated). The lower bound uses hypotheses (H1)–(H3) of the main text, restated here for convenience in the numbered environments below. Assumption 11. Channel conditional independence, as built into the construction of Pi above. (This is hypothesis (H1) of the main text.) Assumption 12. Feasibility: for every i and every j ∈ Alt(i) there exists a ∈ A with 0 < d(Kia ∥Kja ) < +∞. (This is hypothesis (H2) of the main text. Only the B1-type violation—some j ∈ Alt(i) with dja = 0 for all a—implies hopelessness; see Subsection A.7.1. A violation in the d = +∞ direction is harmless, since infinite divergence only eases discrimination. Theorem 8 in fact needs only Ci < ∞; the exact characterization of hopelessness is the infeasibility of (LP) (Subsection A.5, infeasibility step).) Assumption 13. gi (a) ∈ [0, ∞) for all i, a. (This is hypothesis (H3) of the main text. The degenerate case gi (a) = 0 is treated by LP limits: Part 2 of Subsection A.5 and Subsection A.7.2.) 59

Huang

KL notation.

Throughout, for j ∈ Alt(i) we abbreviate
dja := d Kia Kja ∈ [0, +∞],

the environment i being fixed by context. A.2 The Transportation Lemma Source. This is Lemma 1 of Kaufmann et al. (2016) under the isomorphism action ↔ arm, channel output ↔ arm reward sample; the proof below is a complete, self-contained reproduction adapted to model (M1). The condition checklist showing that the isomorphism is legitimate appears in Subsubsection A.2.6. Lemma 14 (Transportation). Fix i ̸= j. Let (π, τ, Θ̂) be admissible and let F ∈ Fτ . Then X
Ei [Na (τ )] · dja ≥ kl Pi (F), Pj (F) , a∈A

with the conventions of the preamble (in particular the left side is +∞ if any summand is, and the inequality is then trivial). The proof occupies Subsubsections A.2.1–A.2.5. Fix the pair (i, j) throughout. A.2.1 Reduction to the Absolutely Continuous Case Let A0 := {a : dja = +∞}. If Ei [Na (τ )] > 0 for some a ∈ A0 , the left side of Lemma 14 is +∞ and there is nothing to prove. Otherwise Ei [Na (τ )] = 0, hence Pi (Na (τ ) = 0) = 1, for every a ∈ A0 ; i.e., Pi -almost surely only actions in Aij := {a : dja < ∞} are played before τ . For a ∈ Aij , dja < ∞ implies kia ≪ kja , so the per-step log-likelihood ratio Zt := log

kiAt (Yt ) kjAt (Yt )

is well-defined and bounded on the Pi -full event {At ∈ Aij } :

indeed |Zt | ≤ Bij := maxa∈Aij maxy: kia (y)>0 log(kia (y)/kja (y)) < ∞, because Y and Aij are finite and kja (y) > 0 whenever kia (y) > 0. We work on the Pi -full event that no action of A0 is played up to time τ ; all statements below are understood modulo this event. A.2.2 Finite-Horizon Likelihood Ratio Lemma 15. For every t ≥ 0, on the event Et := {As ∈ Aij ∀s ≤ t} that no action of A0 is played in the first t steps, Pi is absolutely continuous with respect to Pj on Ft : precisely, Pi (F ∩ Et ) = Ej [1F ∩Et Lt ] for every F ∈ Ft , with density Lt =

t Y k As (Ys ) i

As s=1 kj (Ys )

,

L0 := 1.

(Whenever Ei [Na (τ )] = 0 for all a ∈ A0 , Subsubsection A.2.1 gives Pi (Et ) = 1 for every t, so this is a restriction only modulo a Pi -null event; equivalently, one may without loss of generality replace the strategy by one that never plays A0 up to τ .) 60

Self-Certification of Representation Adequacy

Proof By induction on t. The case t = 0 is trivial (F0 = {∅, Ω}, E0 = Ω, L0 = 1). Assume the claim at t − 1. The induction step below is applied with G replaced by G ∩ Et : note Et = Et−1 ∩ {At ∈ Aij }, where both factors are Ft−1 -measurable (Subsection A.1: At is Ft−1 -measurable), and on {At ∈ Aij } one has kiAt ≪ kjAt , so the absolute-continuity computation is legitimate. The cylinder rectangles
F = G∩{Yt ∈ CY }∩{Ut+1 ∈ CU }, G ∈ σ (As , Ys )s≤t−1 , U1 , . . . , Ut , CY ⊆ Y, CU Borel, form a π-system generating Ft . For such F , h i Pi (F ) = Ei 1G 1{Ut+1 ∈ CU } KiAt (CY ) (11; def. of Pi )   = P (Ut+1 ∈ CU ) · Ei 1G KiAt (CY ) (Ut+1 ⊥ Ft−1 ; same U -law under Pi , Pj )   At = P (Ut+1 ∈ CU ) · Ej Lt−1 1G Ki (CY ) (induction hypothesis) h i = Ej Lt−1 1G 1{Ut+1 ∈ CU } KiAt (CY ) , where the induction hypothesis applies because 1G KiAt (CY ) is Ft−1 -measurable (G ∈ σ((As , Ys )s≤t−1 , U1 , . . . , Ut ) ⊆ Ft−1 and At is Ft−1 -measurable), and the last step reabsorbs the independent factor Ut+1 under Pj . Next, for each a ∈ Aij and CY ⊆ Y, h k a (Y ) i X X k a (y) Kia (CY ) = kia (y) = kja (y) ia = Ej ia 1{Y ∈ CY } , Y ∼ Kja , kj (y) kj (Y ) y∈CY

y∈CY

since terms with kja (y) = 0 satisfy kia (y) = 0 by absolute continuity. Using that At is measurable with respect to σ((As , Ys )s≤t−1 , Ut ) and 11 under Pj , h i   k At (Yt ) Pi (F ) = Ej Lt−1 1G 1{Ut+1 ∈ CU } iAt 1{Yt ∈ CY } = Ej 1F Lt . kj (Yt ) Both sides are measures in F agreeing on a generating π-system that contains Ω (take G = Ω, CY = Y, CU = [0, 1]), hence they agree on all of Ft by the π–λ (uniqueness of measure) theorem. Lemma 16 (Likelihood-ratio martingale). (Lt )t≥0 is a nonnegative (Ft )-martingale under Pj , with Ej [Lt ] = 1. Proof Lt ≥ 0 is Ft -measurable and bounded by eBij t < ∞ on the relevant event (Subsubsection A.2.1), hence integrable. Since Lt = Lt−1 · eZt and Lt−1 is Ft−1 -measurable, Ej [Lt | Ft−1 ] = Lt−1 Ej [eZt | Ft−1 ], X X Ej [eZt | Ft−1 ] = 1{At = a} a

y: kja (y)>0

k a (y) X = 1{At = a} · 1 = 1, kja (y) ia kj (y) a

P a using 11, the Ft−1 -measurability of At , and kia ≪ kja (so y:kja >0 ki (y) = 1). Taking F = Ω in Lemma 15 (restricted to the Pi -full event Et of Subsubsection A.2.1) gives Ej [Lt ] = Pi (Et ) = 1.

61

Huang

A.2.3 Stopped Density Lemma 17. Let P σ be a stopping time bounded by a constant n. Then Pi ≪ Pj on Fσ with density Lσ := nt=0 1{σ = t}Lt . Proof For F ∈ Fσ and each t ≤ n, F ∩ {σ = t} ∈ Ft (definition of Fσ : F ∩ {σ ≤ t} ∈ Ft , and {σ = t} = {σ ≤ t} \ {σ ≤ t − 1}). Hence Pi (F ) =

n X

n
X     Pi F ∩ {σ = t} = Ej 1F ∩{σ=t} Lt = Ej 1F Lσ ,

t=0

t=0

where the second equality is Lemma 15 applied on Ft . A.2.4 A Wald-Type Identity for the Information Accumulated by Bounded Stopping Times Lemma 18. For every n ≥ 1, X     Ei log Lτ ∧n = dja · Ei Na (τ ∧ n) . a∈A

P Proof Write log Lτ ∧n = nt=1 1{t ≤ τ ∧ n} Zt , a finite sum of bounded variables (|Zt | ≤ Bij , Subsubsection A.2.1), so all expectations below are finite and the interchange of sum and expectation is legitimate. The event {t ≤ τ ∧ n} = {τ ∧ n ≤ t − 1}c = {τ ≤ t − 1}c lies in Ft−1 (since τ is a stopping time). By 11 and the Ft−1 -measurability of At , Ei [Zt | Ft−1 ] =

h k a (Y ) i X = 1{At = a} dja , 1{At = a} EY ∼Kia log ia k (Y ) j a a

X

the inner expectation being exactly dja (finite for a ∈ Aij ; on the Pi -null complement the convention of Subsubsection A.2.1 applies). Therefore     X j   Ei 1{t ≤ τ ∧ n}Zt = Ei 1{t ≤ τ ∧ n} Ei [Zt | Ft−1 ] = da Ei 1{t ≤ τ ∧ n, At = a} . a

Summing over t = 1, . . . , n and noting gives the claim.

Pn

t=1 1{t ≤ τ ∧ n, At = a} = Na (τ ∧ n) term by term

A.2.5 Data Processing on Fτ ∧n , and Passage to the Limit Lemma 19 (Bernoulli contraction). Let σ be bounded by n and let E ∈ Fσ . Then
Ei [log Lσ ] ≥ kl Pi (E), Pj (E) . Proof Let ψ(x) = x log x on (0, ∞), ψ(0) := 0; ψ is convex (ψ ′′ (x) = 1/x > 0). By Lemma 17, Ei [g] = Ej [Lσ g] for every Fσ -measurable g with Ei |g| < ∞; taking g = log Lσ 62

Self-Certification of Representation Adequacy

gives Ei [log Lσ ] = Ej [ψ(Lσ )] (finite by Lemma 18). For any E ∈ Fσ with Pj (E) > 0, Jensen’s inequality for the regular conditional law Pj (· | E) gives  
Ei [1E log Lσ ] = Ej [1E ψ(Lσ )] = Pj (E) Ej ψ(Lσ ) E ≥ Pj (E) ψ Ej [Lσ | E]  P (E)  Pi (E) i = Pj (E) ψ = Pi (E) log , Pj (E) Pj (E) where Ej [Lσ | E] = Pi (E)/Pj (E) follows from Lemma 17 (Pi (E) = Ej [1E Lσ ]). If Pj (E) = 0 then Pi (E) = 0 (absolute continuity, Lemma 17) and both sides of the last inequality are 0 by convention. Apply to E = E and E = E c and add: Ei [log Lσ ] ≥ Pi (E) log

Pi (E) Pi (E c ) + Pi (E c ) log = kl Pi (E), Pj (E) . c Pj (E) Pj (E )

Proof of Lemma 14 For each n ≥ 1, σn := τ ∧ n is bounded, and the event En := F ∩ {τ ≤ n} lies in Fτ ∧n : indeed, for t < n, En ∩ {τ ∧ n ≤ t} = F ∩ {τ ≤ t} ∈ Ft because F ∈ Fτ ; for t ≥ n, {τ ∧ n ≤ t} = Ω and En ∈ Fn ⊆ Ft (since F ∈ Fτ implies F ∩ {τ ≤ n} ∈ Fn , the σ-field at time n). Combining Lemmas 18 and 19 with σ = σn , E = En : X
dja Ei [Na (τ ∧ n)] ≥ kl Pi (En ), Pj (En ) ∀n. (59) a

Left side. Since τ < ∞ Pi -a.s. (admissibility), Na (τ ∧ n) ↑ Na (τ ) Pi -a.s., so by monotone convergence Ei [Na (τ ∧ n)] ↑ Ei [Na (τ )] ∈ [0, ∞], and the left side of (59) converges to P j a da Ei [Na (τ )] (monotone limit of a finite sum; conventions of the preamble). If this limit is +∞, Lemma 14 holds trivially. Right side. Since τ < ∞ Pi -a.s. and Pj -a.s. (admissibility under every environment, in particular j), En ↑ F, so by continuity of probability measures from below, pn := Pi (En ) → p := Pi (F),

qn := Pj (En ) → q := Pj (F).

The function kl : [0, 1]2 → [0, ∞] (with our conventions) is jointly lower semicontinuous: on (0, 1)2 it is continuous (sum of continuous terms); at boundary points with finite value (p = 0, q ∈ (0, 1), etc.) it is extended by continuity; at (p, 0) with p > 0, any sequence (pn , qn ) → (p, 0) satisfies kl(pn , qn ) ≥ pn log(pn /qn ) + (1 − pn ) log(1 − pn ) → +∞ = kl(p, 0) (the first term dominates: pn log pn + (−pn log qn ) → +∞ since −pn log qn → +∞, and the second term converges); the remaining boundary points are analogous. Hence lim inf n kl(pn , qn ) ≥ kl(p, q). Taking lim inf in (59): X
dja Ei [Na (τ )] ≥ lim inf kl(pn , qn ) ≥ kl Pi (F), Pj (F) . a

n→∞

A.2.6 Isomorphism with Kaufmann et al. (2016, Lemma 1): Condition Checklist Lemma 1 of Kaufmann et al. (2016) states, for a bandit model with arms a whose reward laws νa , νa′ satisfy KL(νa , νa′ ) < ∞, and any almost surely finite stopping time τ : P ′ a Eν [Na (τ )] KL(νa , νa ) ≥ kl(Pν (F), Pν ′ (F)) for F ∈ Fτ . The correspondence is exact: 63

Huang

KCG16 bandit arm a reward law νa (env. ν) rewards of arm a i.i.d., independent across arms given the pull sequence sampling rule selects At from Ht−1

KL(νa , νa′ ) < ∞

a.s. finite stopping time τ w.r.t. the interaction filtration

model (M1) action a ∈ A channel Kia 11: Yt ∼ KiAt conditionally on the history, independent of the past given At policy πt (Ht−1 , Ut ) (external randomization Ut is environment-independent and cancels in likelihood ratios, Lemma 15) dja < ∞ on the Pi -a.s. played action set; otherwise the inequality is trivial (Subsubsection A.2.1) admissible τ (a.s. finite under all Pi )

One technical strengthening relative to the statement in Kaufmann et al. (2016): our truncation argument (Subsubsection A.2.5) requires only Pi -, Pj -a.s. finiteness of τ —no integrability assumption on τ is needed—because the limit is taken with monotone convergence on the left of (59) and lower semicontinuity of kl on the right. Finite observation space Y gives bounded per-step log-ratios (Subsubsection A.2.1), which makes the Wald identity (Lemma 18) hold without extra moment conditions. A.3 From δ-Correctness to the Information Constraint Lemma 20 (Monotonicity of kl). (a) For fixed q ∈ (0, 1), p 7→ kl(p, q) is strictly increasing on [q, 1]. (b) For fixed p ∈ (0, 1), q 7→ kl(p, q) is strictly decreasing on (0, p]. Proof

Differentiate kl(p, q) = p log pq + (1 − p) log 1−p 1−q : ∂ kl p 1−p p(1 − q) = log − log = log , ∂p q 1−q q(1 − p)

∂ kl p 1−p q−p =− + = . ∂q q 1−q q(1 − q)

For (a): if p > q then p(1 − q) − q(1 − p) = p − q > 0, so the log is positive and ∂kl/∂p > 0; strict increase follows, and extends to the endpoint p = q (where kl = 0) by continuity. For (b): if q < p then ∂kl/∂q = (q − p)/(q(1 − q)) < 0. Lemma 21 (Information constraint). Let δ ∈ (0, 21 ), let (π, τ, Θ̂) be δ-correct, fix i, and set F := {Θ̂ = Θi }. Then F ∈ Fτ and, for every j ∈ Alt(i),
kl Pi (F), Pj (F) ≥ kl(1 − δ, δ) = kl(δ, 1 − δ) > 0. Proof F ∈ Fτ because Θ̂ is Fτ -measurable by definition of a strategy. By δ-correctness, Pi (F) = 1 − Pi (Θ̂ ̸= Θi ) ≥ 1 − δ. For j ∈ Alt(i), Θj ̸= Θi , so {Θ̂ = Θi } ⊆ {Θ̂ ̸= Θj } and Pj (F) ≤ Pj (Θ̂ ̸= Θj ) ≤ δ. Since δ < 12 we have Pi (F) ≥ 1 − δ > δ ≥ Pj (F); if Pj (F) = 0 then kl(Pi (F), 0) = +∞ and the claim is trivial; otherwise two applications of Lemma 20 (first (a) with p ↓ to 1 − δ, then (b) with q ↑ to δ) give

kl Pi (F), Pj (F) ≥ kl 1 − δ, Pj (F) ≥ kl(1 − δ, δ). 64

Self-Certification of Representation Adequacy

δ Symmetry: kl(1 − δ, δ) = (1 − δ) log 1−δ δ + δ log 1−δ = kl(δ, 1 − δ) by swapping the two summands. Positivity: δ ̸= 12 , and kl(p, q) = 0 iff p = q (Gibbs’ inequality: kl(p, q) = d(Ber(p)∥Ber(q)) ≥ 0 with equality iff the laws agree).

Remark 22 (The role of δ < 12 ). The constraint set {p ≥ 1 − δ, q ≤ δ} intersects the diagonal {p = q} when δ > 12 (e.g. p = q = 12 is admissible for δ = 34 and gives kl = 0), so no positive information lower bound follows from δ-correctness alone for δ > 12 . The fixed-confidence regime is δ ∈ (0, 21 ), and Theorem 8 is stated in that regime; this restriction is used only through Lemma 20. The boundary δ = 12 gives kl = 0 and a vacuous bound. Lemma 23 (KCG16 estimate). For every δ ∈ (0, 1), kl(δ, 1 − δ) ≥ log

1 . 2.4 δ

Proof If δ ≥ 1/2.4 the right side is ≤ 0 while kl ≥ 0, so assume δ ∈ (0, 1/2.4). We in fact prove the inequality on all of (0, 1). Define, for δ ∈ (0, 1), h(δ) := kl(δ, 1 − δ) + log(2.4δ). δ Using kl(δ, 1 − δ) = δ log 1−δ + (1 − δ) log 1−δ δ , add log δ = δ log δ + (1 − δ) log δ:

δ2 + (1 − δ) log(1 − δ) + log 2.4. h(δ) = δ log 1−δ | {z } =: ϕ(δ)

Critical points. With u := δ/(1 − δ) ∈ (0, ∞) (strictly increasing in δ),  d δ2 δ δ  δ2 +δ· log −log(1−δ)−1 = 2 log + 1+ = 2 log u+1+u, ϕ′ (δ) = log 1−δ dδ 1−δ 1−δ 1−δ
d δ2 1 1 δ where dδ log 1−δ = 2δ + 1−δ and δ 2δ + 1−δ = 2 + 1−δ . The function δ 7→ 2 log u + 1 + u is strictly increasing (both u and log u are strictly increasing in δ), tends to −∞ as δ → 0 and to +∞ as δ → 1; hence ϕ, and therefore h, has a unique critical point δ ∗ ∈ (0, 1), which is the global minimizer (note h(0+ ) = log 2.4 > 0 and h(1− ) = +∞). Let u∗ := δ ∗ /(1 − δ ∗ ) be the unique root of ψ(u) := u + 2 log u + 1 = 0, u > 0, unique because ψ ′ (u) = 1 + 2/u > 0, ψ(0+ ) = −∞, ψ(+∞) = +∞. At the critical point 2 log u∗ = −(1 + u∗ ), so with δ ∗ = u∗ /(1 + u∗ ), 1 − δ ∗ = 1/(1 + u∗ ): 2u∗ log u∗ − (1 + u∗ ) log(1 + u∗ ) −u∗ (1 + u∗ ) − (1 + u∗ ) log(1 + u∗ ) = 1 + u∗ 1 + u∗ ∗ ∗ = −u − log(1 + u ),   where the first equality rewrites ϕ(δ) = 2u log u−(1+u) log(1+u) /(1+u) using δ 2 /(1−δ) = u2 /(1 + u) and δ = u/(1 + u). Therefore ϕ(δ ∗ ) =

min h(δ) = log 2.4 − u∗ − log(1 + u∗ ) = log δ∈(0,1)

Locating u∗ . ψ is strictly increasing, and: 65

2.4 . (1 + u∗ ) eu∗

Huang

• ψ(0.47) = 1.47 + 2 log 0.47 < 0 because log 0.47 < −0.735: indeed log 0.47 < 1 −0.735 ⇐⇒ e0.735 < 0.47 = 2.12766, and truncating the exponential series with a geometric tail bound, e0.735 ≤

14 X 0.735k k=0

k!

+

0.73515 /15! = 2.08548 . . . < 2.12766; 1 − 0.735/16

1 • ψ(0.48) = 1.48 + 2 log 0.48 > 0 because log 0.48 > −0.74 ⇐⇒ e0.74 > 0.48 = 2.08334, P 12 0.74k 0.74 and the (termwise positive) partial Taylor sum already gives e ≥ k=0 k! = 2.09594 . . . > 2.08334.

Hence u∗ ∈ (0.47, 0.48). Finally, 8 X 0.48k 0.489 /9!  ∗ (1 + u∗ ) eu ≤ 1.48 e0.48 ≤ 1.48 + k! 1 − 0.48/10 k=0

= 1.48 × 1.616075 . . . = 2.39179 . . . < 2.4, so min h > 0, i.e. h(δ) > 0 for all δ ∈ (0, 1), which rearranges to the claim. (The slack is small but strictly positive: numerically min h ≈ 0.00733 at δ ∗ ≈ 0.3233; the Taylor bounds above constitute a rigorous proof.) Corollary 24 (Per-alternative information bound). Under the hypotheses of Lemma 21, for every j ∈ Alt(i), X

Ei [Na (τ )] · dja ≥ kl(δ, 1 − δ) ≥ log

a∈A

Proof

1 . 2.4δ

Lemma 14 with F = {Θ̂ = Θi }, then Lemma 21, then Lemma 23.

A.4 LP Scaling: Proof of Theorem 8 Definition 25 (Certification complexity, LP (covering) form). The certification complexity of environment i is nX o X Ci = inf gi (a) na : na ≥ 0 ∀a, na dja ≥ 1 ∀j ∈ Alt(i) , (LP) a

a

with the convention Ci = +∞ if (LP) is infeasible and Ci = 0 if Alt(i) = ∅. Fix i. Under 12, (LP) is feasible: for each j ∈ Alt(i) pick a(j) with 0 < dja(j) < ∞ and P P set n := t j ea(j) with t := maxj 1/dja(j) ; then a na dja ≥ t dja(j) ≥ 1 for every j, and the objective is finite. Hence 12 implies Ci < ∞; also Ci ≥ 0 always. Proof of Theorem 8 The second inequality is Lemma 23 combined with the first (and Ci ≥ 0). We prove the first. If Ci = 0 there is nothing to prove (R ≥ 0); if Alt(i) = ∅, 66

Self-Certification of Representation Adequacy

Ci = 0 by convention. So assume Ci > 0 and Alt(i) ̸= ∅. Write na := Ei [Na (τ )] ∈ [0, ∞] and L := kl(δ, 1 − δ) > 0. By Corollary 24, X na dja ≥ L ∀j ∈ Alt(i). (60) a

Case P A: Ei [τ ] < ∞. Then na ≤ Ei [τ ] < ∞ for every a. Define P ma := na /L ∈ [0, ∞). By (60), a ma dja ≥ 1 for all j, i.e. m is feasible for (LP); hence a gi (a)ma ≥ Ci . Multiplying by L and using the Tonelli identity of Subsection A.1, X X gi (a)ma ≥ L Ci . (61) gi (a) na = L R(π, Ei ) = a

a

Case B: Ei [τ ] = ∞. Two sub-cases. (a) g := mina gi (a) > 0. Then R(π, Ei ) =

X

gi (a)na ≥ g

X

na = g Ei [τ ] = +∞ ≥ L Ci ,

a

a

since Ci < ∞ under 12. The bound holds trivially. (b) Some gi (a) = 0. Apply Lemma 14 to the bounded stopping time τ ∧ m and the event Em := F ∩ {τ ≤ m} ∈ Fτ ∧m , where F = {Θ̂ = Θi } (membership proved in Subsubsection A.2.5). Exactly as in the proof of Lemma 14, X
Ei [Na (τ ∧ m)] dja ≥ kl pm , qm , pm := Pi (Em ), qm := Pj (Em ), a

for every j ∈ Alt(i) and every m ≥ 1. Since Em ↑ F (admissibility under Pi and Pj ), pm → Pi (F) ≥ 1 − δ and qm → Pj (F) ≤ δ (continuity from below; the inequalities are Lemma 21’s). Fix ε ∈ (0, 12 − δ). There exists m0 such that for all m ≥ m0 : pm ≥ 1 − δ − ε and qm ≤ δ + ε, with 1 − δ − ε > δ + ε; hence by Lemma 20 (and the boundary convention as in Lemma 21), kl(pm , qm ) ≥ kl(1 − δ − ε, δ + ε) =: Lε > 0

∀m ≥ m0 .

(m)

Now Ei [Na (τ ∧ m)] ≤ m < ∞, so ma := Ei [Na (τ ∧ m)]/Lε is feasible for (LP), giving X gi (a) Ei [Na (τ ∧ m)] ≥ Lε Ci ∀m ≥ m0 . a

Let m → ∞: Ei [Na (τ ∧ m)] ↑ na by monotone convergence, the (finite) sum passes to the limit with values in [0, ∞], and we obtain R(π, Ei ) ≥ Lε Ci . Finally let ε ↓ 0: (1 − δ − ε, δ + ε) → (1 − δ, δ) ∈ (0, 1)2 is an interior point where kl is continuous, so Lε → kl(1 − δ, δ) = L, and R(π, Ei ) ≥ L Ci . Remark 26. Case B shows that the hypothesis Ei [τ ] < ∞, often included in the statement of Theorem 8 for convenience, can be dropped entirely: if Ei [τ ] = ∞ the bound is either trivial (sub-case (a)) or recovered by truncation and monotone convergence (sub-case (b)). Concerning the role of 12: the quantifier “for every δ-correct strategy” is vacuously correct 67

Huang

only when 12 fails in the B1 way—i.e. some j ∈ Alt(i) has dja = 0 for every a (Subsection A.7.1: no δ-correct admissible strategy exists for δ < 12 , consistently with (LP) being infeasible and Ci = +∞). Failure of 12 in the d = +∞ direction (dja = +∞ for some a, while some other a′ has dja′ > 0) is harmless—infinite divergence only makes i and j easier to distinguish—and the proof of Theorem 8 in fact requires only Ci < ∞; the exact characterization of hopelessness is the infeasibility of (LP) (Subsection A.5, infeasibility step), not the failure of 12 per se. A.5 Equivalence of the Two Forms of Ci Fix i with Alt(i) = {j1 , . . . , jm } ̸= ∅; write ga := gi (a), and dja as before. Define X wa M := sup inf dj , ga a w∈∆(A) j∈Alt(i)

(MM)

a∈A

where ∆(A), ∆(Alt) are probability simplices, and the ratio wa /ga is defined by the limit convention   wa wa := lim ∈ [0, +∞] = +∞ if wa > 0, ga = 0; = 0 if wa = 0 , ε↓0 ga + ε ga and likewise dja ∈ [0, +∞] terms are read as limT →∞ min(dja , T ). The max–min form of the complexity is CiMM := [M ]−1 , with 1/0 = +∞ and 1/(+∞) = 0. Theorem 27 (Equivalence of the two forms of Ci ). Under 12 and 13, CiLP = CiM M , i.e. h X wa i−1 Ci = sup inf dja . g j∈Alt(i) a w∈∆(A) a Moreover, whenever all dja < ∞ (and ga ≥ 0) the LP value is attained; the sup in (MM) is attained in the regular case below. With dja = +∞ allowed, the LP value need not be attained—e.g. dj = (+∞, 1), g = (1, 1): every ε e1 is feasible with objective ε ↓ 0, so Ci = 0 but no feasible n achieves it—yet the identity Ci = 1/M under the limit conventions, and the equivalence Ci = +∞ iff (LP) is infeasible iff M = 0, remain valid. The proof proceeds in three parts: Part 1 treats the regular case ga > 0 and dja < ∞ for all a, j; Part 2 extends to ga ≥ 0 by perturbation; Part 3 extends to dja ∈ [0, ∞] by truncation and settles the infeasibility convention. A.5.1 Part 1: Regular Case (ga > 0, dja < ∞ for all a, j) Step 1.1 (primal LP: attainment, finiteness, positivity). Feasibility holds as shown above (without 12’s strictness, since now all dja are finite, P feasibility needs for each j some a j with da > 0, which 12 supplies). The objective n 7→ a ga na is continuous; the feasible set P {n ≥ 0 : P a na dja ≥ 1 ∀j} is closed and nonempty; since ga > 0 for all a, the sublevel set {n ≥ 0 : a ga na ≤ c} is bounded, hence its intersection with the feasible set is compact for c larger than the objective value at any feasible point; the infimum C is therefore attained P ∗ ∗ at some n , with C < ∞. Moreover C > 0: if C = 0 then a ga na = 0 forces n∗ = 0 (all P ga > 0), which violates every constraint a n∗a dja ≥ 1. 68

Self-Certification of Representation Adequacy

Step 1.2 (reparametrization of the LP as a max–min). For w ∈ ∆(A) define X wa M (w) := min dja ∈ [0, ∞). j g a a P There is a bijection between feasible n with t := a ga nP a > 0 and pairs (w, t) ∈ ∆(A) × (0, ∞) with M (w) ≥ 1/t: set wa := ga na /t (then w ≥ 0, a wa = 1) and compute X

na dja ≥ 1 ⇐⇒ t

X wa a

a

ga

dja ≥ 1 ⇐⇒

X wa a

1 dja ≥ ; ga t

minimizing over j turns the family of constraints into the single inequality M (w) ≥ 1/t. Since every feasible n ̸= 0 has t > 0 and n = 0 is infeasible,  C = inf t : ∃ w ∈ ∆(A), M (w) ≥ 1/t

=

1 1 . = supw M (w) w: M (w)>0 M (w) inf

(62)

P The sup is attained: M (w) = minj a (wa /ga )dja is the minimum of finitely many linear (hence continuous) functions of w on the compact simplex, henceP continuous, so w∗ ∈ 1 arg max M (w) exists; and M ∗ := M (w∗ ) > 0, because with w◦ := m j ea(j) (uniform over the distinguishing actions supplied by 12), M (w◦ ) = min

X w◦

j

a

a j d ga a

1 dj > 0. m ga(j) a(j)

≥ min j

Hence C = 1/M ∗ ∈ (0, ∞). Step 1.3 (Sion minimax: condition checklist). game on ∆(A) × ∆(Alt) with payoff F (w, µ) :=

XX a

wa µj

j

Consider the two-player zero-sum dja . ga

Checklist for Sion’s minimax theorem (Sion 1958): (i) ∆(A) and ∆(Alt) are compact convex subsets of Euclidean spaces (finite-dimensional simplices); (ii) F is jointly continuous (bilinear with finite coefficients dja /ga ); (iii) w 7→ F (w, µ) is convex for every fixed µ (it is linear); (iv) µ 7→ F (w, µ) is concave for every fixed w (it is linear). All conditions hold, so sup inf F (w, µ) = inf sup F (w, µ). µ

w

µ

w

Both extremizations over a simplex P of a linear function are attained at vertices: inf µ F (w, µ) = minj F (w, ej ) = minj a (wa /ga )dja = M (w) (the minimum of a linear function over the simplex equals the minimum over its vertices ej : F (w, µ) = P j µj F (w, ej ) ≥ minj F (w, ej ) with equality at the minimizing vertex); similarly P supw F (w, µ) = maxa j µj dja /ga . Therefore j j µj da

P ∗

M = sup M (w) =

inf

w

µ∈∆(Alt)

69

max a

ga

.

(63)

Huang

Step 1.4 (explicit dual LP and equality of values). The Lagrangian/LP dual of (LP) is nX o X D∗ := sup λj : λj ≥ 0 ∀j, λj dja ≤ ga ∀a . (D) j

j

Weak duality (one line, no assumptions): for primal-feasible n and dual-feasible λ, X

X

λj ≤

j

λj

X

na dja =

X

X

a

a

j

na

X

λj dja ≤

na ga ,

a

j

P P the first inequality using a na dja ≥ 1 and λj ≥ 0, the second using j λj dja ≤ ga and na ≥ 0; hence D∗ ≤ C. Strong duality is obtained constructively through Steps 1.2–1.3: P reparametrize dual-feasible λ ̸= 0 as λ = sµ with s := j λj > 0 and µ := λ/s ∈ ∆(Alt); the dual constraints read s

X

j j µj da

P µj dja ≤ ga

∀a ⇐⇒ s max a

j

ga

ji j µj da −1

P

h

≤ 1 ⇐⇒ s ≤ max a

ga

(the max is positive for every µ: for µ = ej it is maxa dja /ga > 0 by 12, and a general µ puts P mass µj > 0 on some j, giving j µj dja ≥ µj dja , positive at a = a(j)). Hence ∗

D =

sup µ∈∆(Alt)

h

ji j µj da −1

P max a

ga

ji j µj da −1 (63)

P

h

= inf max µ

a

ga

=

1 (62) = C. M∗

(64)

This proves strong duality D∗ = C and simultaneously the identity C = [M ∗ ]−1 = CiM M in the regular case. (The justification for strong duality is therefore: primal feasible with finite value ⇒ the reparametrization (62) ⇒ Sion minimax (63) ⇒ explicit dual value (64); no appeal to a black-box duality theorem is needed, though the conclusion agrees with standard LP strong duality, e.g. Bertsimas–Tsitsiklis, Theorem 4.4.) A.5.2 Part 2: Extension to ga ≥ 0 (Zero-Cost Actions) (ε)

Let A0 := {a : ga = 0} ̸= ∅ possibly. For ε > 0Pset ga := ga + ε > 0; let C(ε) be the (LP) value with costs g (ε) and M (ε) := supw minj a wa dja /(ga + ε). Part 1 applies for each ε > 0: C(ε) = 1/M (ε), with C(ε) ∈ (0, ∞). Step 2.1 (attainment of the LP value with g ≥ 0). A finite linear program that is feasible P and whose objective is bounded below attains its value: the feasible set F = {n ≥ 0 : a na dja ≥ 1 ∀j} is a nonempty closed polyhedron; if F contained a line {n + tv : t ∈ R} (v ̸= 0) the linear objective would have to be constant along it (otherwise it is unbounded below on F ); quotienting out the lineality space, F becomes a pointed polyhedron, i.e. the Minkowski sum of the convex hull of its extreme P P points and its recession cone {v ≥ 0 : j A , on which the objective v d ≥ 0 ∀j} = R + a a a a ga va ≥ 0 is bounded below; hence the minimum is attained at an extreme point. Denote the value by C(0) and a minimizer by n∗ . 70

Self-Certification of Representation Adequacy

′

Step 2.2 (continuity of the LP value as ε ↓ 0). Since g (ε) ≥ g (ε ) ≥ g for 0 < ε ≤ ε′ , C(ε) is nonincreasing as ε ↓ 0 and C(ε) ≥ C(0) (same feasible set, larger objective). Conversely, feasibility of n∗ is independent of ε, so X X n∗a −−→ C(0), ga(ε) n∗a = C(0) + ε C(ε) ≤ ε↓0

a

a

hence C(ε) ↓ C(0). Step 2.3 (the max–min P side converges to the limit convention). For each w and j j, the bracket bj (w, ε) := a wa da /(ga + ε) is nondecreasing as ε ↓ 0 and converges to P j bj (w, 0) := a wa da /ga ∈ [0, ∞] under the limit convention (each summand converges monotonically). Hence minj bj (w, ε) ↑ minj bj (w, 0) (finite min of increasing sequences converges to the min of the limits), and M (ε) = sup min bj (w, ε) ↑ sup sup min bj (w, ε) = sup min bj (w, 0) =: M (0), w

j

w ε>0

j

w

j

where interchanging supw and supε is legitimate (suprema over independent index sets commute), and supε minj bj (w, ε) = minj bj (w, 0) by the monotone convergence just noted. Therefore 1 1 C(0) = lim C(ε) = lim = = CiM M , ε↓0 ε↓0 M (ε) M (0) where the reciprocal commutes with the monotone limits (including the cases C(0) = 0 ⇐⇒ M (0) = +∞, since C(ε) ↓ 0 ⇐⇒ M (ε) ↑ ∞). Step 2.4 (free-rider sanity check). If there exists a∗ ∈ A0 with dja∗ > 0 for all j ∈ Alt(i), then n = t ea∗ with t := 1/ minj dja∗ is feasible with objective 0, so C(0) = 0; and w = ea∗ gives bj (w, 0) = dja∗ /0 = +∞ for every j, so M (0) = +∞ and 1/M (0) = 0: the two forms agree (this is B2, Subsection A.7.2). Conversely, if every a ∈ A0 has dja = 0 for some j, then only costly actions can cover that j, and 12 guarantees some a ∈ / A0 with dja > 0 does, keeping the restricted LP feasible. A.5.3 Part 3: Extension to dja ∈ [0, ∞] and the Infeasibility Convention Step 3.1 (truncation in d). For T > 0 let dja (T ) := min(dja , T ) < ∞. Parts 1–2 apply to the truncated data (feasibility is preserved: 12 provides 0 < dja(j) , hence dja(j) (T ) > 0 for all T ), giving C(T ) = 1/M (T ) for the corresponding values. As T ↑ ∞: feasible sets of (LP) grow, so C(T ) is nonincreasing and C(T ) ≥ C. (The untruncated LP value need not be attained when some dja = +∞—see the example in the statement of Theorem 27—so the following near-optimality argument avoids selecting a minimizer.) Let n be any feasible solution (LP). For each j, monotone convergence in each summand P of the untruncated P gives a na dja (T ) ↑ a na dja ≥ 1 as T ↑ ∞; if the limit sum is finite then dja (T ) = dja on {a : na > 0} for all large T and the constraint holds with the same value, and if the limit sum is +∞ the truncated sum eventually exceeds 1; in either case there is Tj (n) with nPfeasible for the truncated problem for all PT ≥ Tj (n). Hence for T ≥ maxj Tj (n), C(T ) ≤ a ga na , and therefore limT →∞ C(T ) ≤ a ga na ; taking the infimum over all untruncated-feasible n yields limT C(T ) ≤ C. Combined with C(T ) ≥ C, C(T ) ↓ C. On the max–min side, 71

Huang

P bj (w, T ) := a (wa /ga ) min(dja , T ) ↑ bj (w, ∞) for each (w, j) by monotone convergence of each summand, and supw minj passes to the limit exactly as in Step 2.3, giving M (T ) ↑ M . Hence C = 1/M with the stated conventions. Step 3.2 (infeasibility ⇐⇒ M = 0 ⇐⇒ C = +∞). (LP) is infeasible iff there exists j0 with dja0 = 0 for all a (constraint j0 reads 0 ≥ 1). If this holds, then for every w: P j0 j0 a (wa /ga )da = 0 (each term is 0, including ∞ · 0 = 0 under the convention, since da = 0 kills the term), so minj ≤ 0 and M = 0. Conversely, if (LP) is feasible, the construction ◦ dj below Step 1.2 (using 12) produces w◦ with M (w◦ ) ≥ minj wa(j) a(j) /ga(j) > 0—with the convention that if ga(j) = 0 the term is +∞, which only helps—so M > 0. Hence C = +∞ ⇐⇒ M = 0 ⇐⇒ 1/M = +∞, consistent with the conventions. (B1, Subsection A.7.1, shows this case is genuinely hopeless information-theoretically.) This completes the proof of Theorem 27. □ Remark 28 (What the equivalence buys). The lower bound (Theorem 8) is stated against the LP form. The matching upper bound (Theorem 9, CTS) uses as its sampling design the cost-ratio maximizer P w dj ∗ Pa a a , w (i) ∈ arg sup inf w∈∆(A) j∈Alt(i) a ga wa P i.e. the normalized LP solution n∗ / a n∗a (the characteristic-time design of the CABAI template (Kanarios et al., 2024)). Theorem 27 certifies that the constant driving both bounds is the same, Ci = 1/M . The link n∗a = Ci wa∗ /ga between an LP optimizer and a maximizer w∗ of the (w/g)-weighted max–min form (MM) is mathematically valid wherever both exist (the reparametrization of Step 1.2 at the optimum) and may be retained; but is in general a different allocation—its asymptotic cost constant
 P the (w/g)-maximizer P j
g w inf w d j a a a a a a can exceed Ci (numerical 2 × 2 counterexample: 4.17 vs. the optimal 2.73, see the main text)—and it is not the sampling design of Theorem 9. A.6 From the Pointwise Bound to the lim inf Statement Define the optimal risk at confidence δ:  Rδ∗ (Ei ) := inf R(π, Ei ) : (π, τ, Θ̂) δ-correct and admissible ,

δ ∈ (0, 21 ).

Corollary 29 (lim inf form of Theorem 8). Under 11–13, for every i, lim inf δ→0

Rδ∗ (Ei ) ≥ Ci . log(1/δ)

Proof If Ci = 0 the statement is Rδ∗ ≥ 0, trivial. If Ci = +∞ (only possible when 12 fails), the infimum is over an empty set for δ < 12 (B1, Subsection A.7.1: no admissible δ-correct strategy exists), and the statement is vacuous; the convention Ci = +∞ is consistent. Assume 0 < Ci < ∞. By Theorem 8 (valid for every δ-correct admissible strategy, Case B of Subsection A.4 removing any integrability requirement), for every δ ∈ (0, 12 ), Rδ∗ (Ei ) kl(δ, 1 − δ) ≥ Ci · . log(1/δ) log(1/δ) 72

(65)

Self-Certification of Representation Adequacy

It remains to compute the asymptotics of the kl factor. Expand: kl(δ, 1 − δ) = δ log

δ 1−δ 1 δ + (1 − δ) log = (1 − δ) log + (1 − δ) log(1 − δ) + δ log . 1−δ δ δ 1−δ

As δ → 0: (1 − δ) log(1 − δ) → 0 (since x log x → 0 as x → 1− ; precisely (1 − δ) log(1 − δ) ∈ δ [−1/e, 0]); δ log 1−δ = δ log δ − δ log(1 − δ) → 0 − 0 = 0; and (1 − δ) log(1/δ) = log(1/δ) +
o log(1/δ) since δ log(1/δ) → 0. Hence
1 kl(δ, 1 − δ) = 1 + o(1) log , δ

i.e.

kl(δ, 1 − δ) = 1. δ→0 log(1/δ) lim

(66)

Taking lim inf δ→0 on both sides of (65) (the right side is Ci times a convergent sequence with limit 1) gives the claim.
Remark 30. The alternative form R(π, Ei ) ≥ Ci log(1/(2.4δ)) = Ci log(1/δ) − log 2.4 loses only an additive Ci log 2.4, negligible on the log(1/δ) scale; the sharp first-order constant is Ci . A.7 Boundary Examples B1, B2, B3 (Formal Statements and Proofs) A.7.1 B1: Hopeless Pairs Proposition 31. Suppose that for some i there exists j ∈ Alt(i) with d(Kia ∥Kja ) = 0 for every a ∈ A. Then: (i) for every admissibleSstrategy, the transcript laws coincide: Pi |Ft = Pj |Ft for all t, and hence on F∞ := σ( t Ft ); (ii) no strategy is δ-correct for any δ < 21 ; (iii) (LP) is infeasible, consistently with the convention Ci = +∞; in the max–min form, M = 0. Proof (i) d(Kia ∥Kja ) = 0 iff kia = kja pointwise (Gibbs’ inequality on a finite space: the KL divergence vanishes only between equal distributions). Hence KiAt = KjAt as kernels at every t. We show Pi (F ) = Pj (F ) for all F ∈ Ft by induction on t, exactly as in Lemma 15 but with all likelihood ratios equal to 1: the case t = 0 is trivial; for the step, on cylinder rectangles F = G ∩ {Yt ∈ CY } ∩ {Ut+1 ∈ CU },     Pi (F ) = Ei 1G 1{Ut+1 ∈ CU }KiAt (CY ) = Ej 1G 1{Ut+1 ∈ CU }KjAt (CY ) = Pj (F ), using the induction hypothesis for the first equality (together with KiAt = KjAt and the identical Uniform law of Ut+1 ), and agreement on a generating π-system extends to Ft by S uniqueness of measure. Equality on F∞ follows since t Ft is an algebra (it is closed under finite unions because the filtration is nested) generating F∞ , and two probability measures agreeing on it agree on the generated σ-field (π–λ theorem). (ii) Let (π, τ, Θ̂) be admissible. Since τ < ∞ Pi - and Pj -a.s., Θ̂ is defined a.s. under both laws, and {Θ̂ = Θi } ∈ Fτ ⊆ F∞ . Because Θj ̸= Θi (as j ∈ Alt(i)), {Θ̂ ̸= Θj } ⊇ {Θ̂ = Θi }. 73

Huang

By (i), and because Fτ -measurable events have equal probabilities S under Pi , Pj (for F ∈ Fτ : F ∩ {τ ≤ t} ∈ Ft has equal probabilities for every t, and F = t (F ∩ {τ ≤ t}) up to null sets since τ < ∞ a.s. under both laws; continuity from below gives Pi (F ) = Pj (F )):



Pj Θ̂ ̸= Θj ≥ Pj Θ̂ = Θi = Pi Θ̂ = Θi = 1 − Pi Θ̂ ̸= Θi . Therefore max Pi (Θ̂ ̸= Θi ) ≥ i

1 2

h

i
≥ Pi Θ̂ ̸= Θi + Pj Θ̂ ̸= Θj

1 2.

So δ-correctness is impossible for every δ < 12 : certification between i and j is informationtheoretically hopeless. (ii) is exactly the statement that the transportation lower bound is +∞: with dja = 0 for all a, Lemma 14 imposes no constraint, but no strategy meets the correctness requirement either. P (iii) The constraint of (LP) indexed by this j reads a na · 0 ≥ 1, impossible for any n ≥ 0; hence P (LP) is infeasible and Ci = +∞ by convention. For the max–min form: for every w, a (wa /ga )dja = 0, so the infimum over j is 0 and M = 0, i.e. CiM M = 1/0 = +∞ (Theorem 27, infeasibility step). Correspondence with partial monitoring (Bartók et al., 2014). In the taxonomy of finite partial monitoring of Bartók et al. (2014), a game is hopeless when two Pareto-optimal environments/actions are indistinguishable under every available observation structure; the minimax regret then grows linearly and no consistent learning is possible. B1 is the analogue for fixed-confidence certification: the pair (i, j) is indistinguishable under every action, so the certification question between the adequacy labels Θi ̸= Θj admits no δ-correct answer at any confidence better than chance, and the certification complexity is +∞. The LP feasibility hypothesis 12 is precisely the exclusion of hopeless pairs. A.7.2 B2: Free Riding on a Zero-Cost Discriminating Action Proposition 32. Suppose that for some i there exists a∗ ∈ A with gi (a∗ ) = 0 and 0 < ∗ ∗ d(Kia ∥Kja ) < ∞ for every j ∈ Alt(i). Then Ci = 0; moreover there is a family of strategies {πδ }δ∈(0,1/2) , each stopping a.s. under Pi , with
Ei [τ ] = O log(1/δ) , R(πδ , Ei ) = 0 for every δ, and with error probability ≤ δ under every environment (decision issued at τ ; precise sense below). Within the fully admissible strategy
class of Subsection A.1, the CTS strategy of Theorem 9 delivers Rδ∗ (Ei ) = o log(1/δ) on this instance, consistent with limδ→0 Rδ∗ (Ei )/ log(1/δ) = 0 = Ci ; the bound of Corollary 29 (lim inf ≥ 0 = Ci ) holds with equality. Proof Value of the LP. Set t := 1/ minj∈Alt(i) dja∗ < ∞ and n := t ea∗ ≥ 0. For every P j j ∗ j: a na da = t da∗ ≥ 1, so n is feasible with objective gi (a ) t = 0; hence Ci = 0 (and by Theorem 27, free-rider step, the max–min form agrees: w = ea∗ makes every bracket +∞). Strategy. Write Alt(i) = {j1 , . . . , jm }. Play At = a∗ for all t, and for j ∈ Alt(i) define the log-likelihood-ratio walk LLRj (t) :=

t X

∗

Xs(j) ,

Xs(j) := log

s=1

74

kia (Ys ) . ∗ kja (Ys )

Self-Certification of Representation Adequacy

∗

(j)

Under Pi , given the (constant) action sequence, 11 makes (Ys ) i.i.d. Kia , hence Xs are i.i.d. ∗ ∗ (j) (j) with mean Ei [Xs ] = dja∗ =: dj > 0 and |Xs | ≤ Bj := maxy: ka∗ (y)>0 | log(kia (y)/kja (y))| < ∗

i

∗

∞ (finite Y, dj < ∞ so kja > 0 on supp kia ). Set β := log(1/δ), define the first-passage times τj := inf{t ≥ 1 : LLRj (t) > β}, and stop at τ := max τj , j∈Alt(i)

Θ̂ := Θi (constant).

T τ is a stopping time: {τ ≤ t} = j {τj ≤ t} ∈ Ft as each τj is a stopping time; the constant decision is Fτ -measurable on {τ < ∞} and defined on all of Ω. Correctness and stopping. Under Pi : each LLRj (t)/t → dj > 0 a.s. (strong law), so each τj < ∞ a.s., hence τ < ∞ a.s.; and Θ̂ = Θi surely, so Pi (Θ̂ ̸= Θi ) = 0. Under Pj with j ∈ Alt(i): a (wrong) decision is issued iff τ < ∞, and n o {τ < ∞} ⊆ {τj < ∞} = sup LLRj (t) > β . t≥1

Q ∗ ∗ The process Lt := exp(LLRj (t)) = s≤t kia (Ys )/kja (Ys ) is the likelihood-ratio martingale of Lemma 16 specialized to the constant action a∗ : nonnegative, with Ej [Lt ] = 1. Doob’s maximal inequality for nonnegative martingales gives   Pj max Lt > eβ ≤ e−β Ej [Ln ] = e−β ∀n, t≤n

and the events {maxt≤n Lt > eβ } increase to {supt Lt > eβ }, so continuity from below yields Pj (supt LLRj (t) > β) ≤ e−β = δ; hence Pj (Θ̂ ̸= Θj ) ≤ δ. Under environments e with Θe = Θi (e ̸= i), any issued decision equals Θe , so Pe (Θ̂ ̸= Θe ) = 0. Expected sample size and risk. Wald’s identity applied to the bounded stopping time τj ∧ n gives Ei [LLRj (τj ∧ n)] = dj Ei [τj ∧ n] (i.i.d. increments with finite mean). On {τj ≤ n}, LLRj (τj ) ≤ β + Bj (the overshoot is at most the increment bound); on {τj > n}, LLRj (n) ≤ β. In either case LLRj (τj ∧ n) ≤ β + Bj , hence dj Ei [τj ∧ n] ≤ β + Bj , and monotone convergence as n → ∞ gives Ei [τj ] ≤ (β + Bj )/dj . Therefore Ei [τ ] ≤

X j

Ei [τj ] ≤

X log(1/δ) + Bj
= O log(1/δ) . dj

j∈Alt(i)

Since only a∗ is ever played, R(πδ , Ei ) = gi (a∗ ) Ei [τ ] = 0. Admissibility caveat and the o(log(1/δ)) conclusion. The strategy stops a.s. under Pi (the environment whose risk is bounded) with Ei [τ ] < ∞; under j ∈ Alt(i) it may abstain (τ = ∞) with probability ≥ 1 − δ—which is exactly how the error probability is kept below δ; under environments sharing the label of i, the decision, whenever issued, is correct. Thus πδ certifies the label of Ei at zero expected cost and is δ-correct in the decision-at-τ sense. Converting it into a fully admissible strategy (a.s. stopping under every environment, as required in Subsection A.1) requires the GLR/time-uniform machinery: the CTS
strategy of Theorem 9 is fully admissible and δ-correct, and delivers Rδ∗ (Ei ) = o log(1/δ) on this instance (Theorem 9, applied with Ci = 0, gives lim supδ→0 R(π ∗ , Ei )/ log(1/δ) ≤ Ci = 0; the residual vanishing-rate cost comes from the forced exploration, cf. the main text, B2). 75

Huang

For the purposes of the present lower bound, what matters is: Ci = 0, the pointwise bound R ≥ 0 = Ci · kl(δ, 1 − δ) holds, and limδ→0 Rδ∗ (Ei )/ log(1/δ) = 0 = Ci , so the bound of Corollary 29 is tight.
Remark 33. B2 is the regime where Theorem 9 degenerates to R = o log(1/δ) : certification is achievable at zero asymptotic cost rate because a free action is informative against every alternative. The lower bound lim inf ≥ 0 is trivially tight. Note the strategy above uses only the martingale maximal inequality (a special case of the threshold argument of (Garivier and Kaufmann, 2016)), not the full Track-and-Stop machinery. A.7.3 B3: Two-Point Degeneration and Consistency with Nitinawarat and Veeravalli (2015) Proposition 34 (Bang-bang). Suppose |Alt(i)| = 1, say Alt(i) = {j} (in particular this holds whenever |E| = 2). Then gi (a) Ci = min , (67) j dja a: da >0 with the convention that the minimum is +∞ (infeasible) if no a has dja > 0—excluded by 12—and 0 if some a∗ with gi (a∗ ) = 0 has dja∗ > 0. Proof Let ρ := mina gi (a)/dja over {a : dja > 0}, nonempty and all ratios in [0, ∞) by 12/13 (if gi (a) = 0 the ratio is 0). Lower bound: for any (LP)-feasible n, X a

gi (a) na =

X gi (a) a:dja >0

dja

X

dja na +

a:dja =0

gi (a)na ≥ ρ

X

dja na ≥ ρ,

a

P j since the second sum is ≥ 0 and feasibility gives a da naP≥ 1. Upper bound: with j j ∗ ∗ ∗ ∗ a ∈ arg mina gi (a)/da and n := (1/da∗ )ea , n is feasible ( a n∗a dja = 1) with objective gi (a∗ )/dja∗ = ρ. Hence Ci = ρ, attained by a single-action (“bang-bang”) plan. Max– P min cross-check: with a single alternative, M = supw a wa dja /gi (a) = maxa dja /gi (a) (supremum of a linear function over the simplex, attained at a vertex; the limit convention for gi (a) = 0 gives dja /0 = +∞ when dja > 0, consistent with ρ = 0), and 1/M = mina gi (a)/dja = ρ. Edge case dja∗ = +∞. If the minimizing action has dja∗ = +∞, the value formula (67) remains valid under the limit convention of Subsection A.5 (the ratio gi (a∗ )/dja∗ is read as 0, and Theorem 27, Part 3, gives Ci = 1/M with M = +∞), but attainment is limited to finite d: no finite plan n realizes the value, which is only approached along the truncation d ∧ T . In the lower-bound chain (60), terms with dja = +∞ are read under the 0 · (+∞) = 0 convention (preamble): they contribute +∞ whenever na > 0, making the corresponding constraint trivially satisfiable. Consistency with Nitinawarat and Veeravalli (2015, Theorem 5.1). Nitinawarat and Veeravalli (2015) study controlled sequential multihypothesis testing with controldependent observation kernels and nonuniform control costs (see also (Naghshvar and Javidi, 2013) for active sequential hypothesis testing). Specialized to (a) two hypotheses, (b) 76

Self-Certification of Representation Adequacy

i.i.d. (non-Markovian) observation channels Kia , Kja under control a, and (c) control cost c(a) = gi (a), the asymptotic (small error probability) optimal expected cost under hypothesis i in their Theorem 5.1 is governed by the single divergence vector a 7→ d(Kia ∥Kja ), and the optimal static control solves exactly the bang-bang program mina c(a)/d(Kia ∥Kja ): with one alternative there is no trade-off to balance, all information comes from the single pair (i, j), and the cheapest unit of KL divergence wins. Proposition 34 shows the constant Ci of Theorem 8 reduces to exactly this expression in the two-point case: our result properly contains the known binary controlled-sensing constant. The genuinely new content of the general formula appears only when |Alt(i)| ≥ 2—where the adequacy-label-defined alternative set forces a covering/balancing problem (the LP) rather than a single bang-bang ratio—and when zero-cost actions create the B2 degeneracy, a case outside the positive-cost assumption of Nitinawarat and Veeravalli (2015). A.8 Non-Triviality Note NT1: Identical Kernels, Different Losses, Different Optima Proposition 35. There exist instances of model (M1) and environments i with actions a1 ̸= a2 such that Kea1 = Kea2 for every environment e,

gi (a1 ) ̸= gi (a2 ),

P and the set of optimal certification allocations w∗ (i) ∈ arg supw inf j∈Alt(i) a (wa /gi (a)) dja (equivalently, the LP optimizers) changes when the loss profile is altered while the observation kernels are held fixed; in particular, every criterion that depends only on the kernels (Kea )e,a (e.g. Chernoff-type most-informative-action rules (Chernoff, 1959)) is unable to distinguish the two situations and is suboptimal in at least one of them. Proof (explicit numerical construction, all quantities in closed form) Take |E| = 2 with labels Θ1 = 1, Θ2 = 0, so Alt(1) = {2}; Y = {0, 1}; A = {a1 , a2 , a3 } with Bernoulli channels (writing Ber(p) for the law on {0, 1} with P (1) = p):



1 K1a1 = K1a2 = Ber 21 , K2a1 = K2a2 = Ber 14 ; K1a3 = Ber 21 , K2a3 = Ber 20 . Note a1 , a2 have identical kernels under both environments. The divergences from environment 1 to environment 2 are
1/2 1 1/2 4 d1 := d Ber( 21 ) Ber( 14 ) = 12 log + log = 12 log ≈ 0.143841, 1/4 2 3/4 3
1/2 1/2 100 1 d3 := d Ber( 21 ) Ber( 20 ) = 12 log + 12 log = 12 log ≈ 0.830366, 1/20 19/20 19 with d2a1 = d2a2 = d1 , d2a3 = d3 (and d1 , d3 ∈ (0, ∞), so 12 holds). Since |Alt(1)| = 1, Proposition 34 applies: C1 = mina g1 (a)/da , attained by a single best action. Loss profile G(1) : g1 = (1, 5, 10). The three ratios are 1 2 = ≈ 6.95212, d1 log(4/3)

5 10 = ≈ 34.76059, d1 log(4/3) 77

10 20 = ≈ 12.04289, d3 log(100/19)

Huang

(1)

so the unique optimal action is a1 and C1 = 2/ log(4/3) ≈ 6.95212. Loss profile G(2) : g1 = (5, 1, 10) (the losses of a1 , a2 swapped). The kernels are unchanged, so d1 , d3 are unchanged; the ratios are now 34.76059, 6.95212, 12.04289, and the (2) unique optimal action is a2 , with C1 = 2/ log(4/3). Consequences. (i) The kernels (Kea )—and hence every functional of them alone, such as the Chernoff-type allocation arg maxa da —are identical under G(1) and G(2) , yet the optimal certification action flips from a1 to a2 : the optimal strategy depends on the task loss beyond the observation kernels. (ii) The kernel-only most-informative-action rule selects a3 in both profiles (d3 > d1 ), achieving ratio 20/ log(100/19) ≈ 12.04289, which is worse than optimal by the factor 20/ log(100/19) 10 log(4/3) = ≈ 1.73228, 2/ log(4/3) log(100/19) i.e. a ≈ 73% excess asymptotic cost, in both profiles. Hence no pure Chernoff/kernel criterion can attain C1 here, and the constant C(E, f ) genuinely couples information content with task loss. All numerical values above are in closed form and were verified computationally (see the main text, numerical log of 2026-08-03). A.9 Proof-Completeness Self-Check The following self-check list records, for each technical point of the proof, where it is carried out and its status. # Technical point 1 Canonical space, filtration, policies with external randomization; existence/uniqueness of Pi (Ionescu–Tulcea); all conditioning via regular Markov kernels (standard Borel spaces) 2 Stopped σ-field Fτ ; admissibility (τ < ∞ a.s. under all environments) as part of δ-correctness; F ∩ {τ ≤ n} ∈ Fτ ∧n verified 3 Transportation lemma: density on Ft by induction + π–λ (15); LR martingale (16); stopped density (17); Wald-type identity with bounded increments (18); data processing via Jensen/ψ(x) = x log x (19); limit n → ∞ via MCT + joint l.s.c. of kl 4 Reduction for dja = +∞ actions (triviality of LHS vs Pi -a.s. exclusion) 5 Isomorphism with Kaufmann et al. (2016, Lemma 1): condition-by-condition checklist; source annotated; strengthening noted (only a.s. finiteness of τ needed) 6 δ-correctness ⇒ information constraint: kl monotonicity proved (20); kl(1 − δ, δ) = kl(δ, 1 − δ) verified; δ < 21 necessity made explicit with counterexample for δ > 21 (Remark 22) 7 kl(δ, 1 − δ) ≥ log(1/(2.4δ)): full analytic proof (unique ∗ critical point, min h = log[2.4/((1 + u∗ )eu )] > 0) with rigorous Taylor/rational bounds (u∗ ∈ (0.47, 0.48), 1.48e0.48 ≤ 2.39180 < 2.4); slack ≈ 0.0073

78

Where Subsection A.1

Status closed

Subsections A.1, A.2.5

closed

Subsection A.2

closed

Subsubsection A.2.1

closed

Subsubsection A.2.6

closed

Subsection A.3

closed

Lemma 23

closed

Self-Certification of Representation Adequacy

8

9

10

11

12

13 14

15

16

17

LP scaling R ≥ kl(δ, 1 − δ) Ci : Case A (E[τ ] < ∞); Case B(a) (E[τ ] = ∞, min g > 0 ⇒ R = ∞); Case B(b) (zero-cost entries: truncation τ ∧ m + continuity of kl at interior point + MCT) LP strong duality: weak duality one line; strong duality derived constructively via reparametrization (62) + Sion (63) + explicit dual (64); attainment/finiteness/positivity of primal value argued Sion minimax conditions verified explicitly (compact convex simplices, bilinear continuous payoff, linear ⇒ convex/concave in each variable); vertex attainment of inf µ , supw gi (a) = 0 handled by ε-perturbation with proof of C(ε) ↓ C(0) and M (ε) ↑ M (0) (sup/sup interchange + monotone min); LP attainment with g ≥ 0 via polyhedral argument; free-rider consistency check dja = +∞ handled by truncation d ∧ T : near-optimal feasible solutions + per-constraint monotone convergence give C(T ) ↓ C (no claim that C(T ) = C eventually; untruncated LP attainment may fail, example recorded in Theorem 27); attainment of the LP value guaranteed whenever all dja < ∞; infeasibility ⇐⇒ M = 0 ⇐⇒ C = +∞ proved both directions lim inf statement: kl(δ, 1 − δ)/ log(1/δ) → 1 proved term by term; Ci = 0, +∞ edge cases disposed B1: transcript-law equality by induction; error complementarity ⇒ max error ≥ 21 ; LP infeasibility and M = 0 matched; Bartók et al. (2014) correspondence annotated B2: LP value 0; explicit one-sided strategy τ = maxj τj ; δ-correctness via LR martingale + Doob maximal inequality under each j ∈ Alt(i); Ei [τ ] = O(log(1/δ)) via truncated Wald + overshoot bound; R(πδ , Ei ) = 0; admissibility caveat stated honestly (a.s. stopping under all environments delegated to CTS/Theorem 9, which delivers Rδ∗ = o(log(1/δ)) on this instance, matching lim Rδ∗ / log(1/δ) = 0 = Ci ; no O(1)-scale claim is made) B3: bang-bang formula proved both directions + max–min cross-check; formal consistency with Nitinawarat and Veeravalli (2015, Theorem 5.1) stated at the level of the constant in the binary, i.i.d.-channel, positive-cost specialization (not a re-derivation of the Markovian theorem); new content isolated to |Alt| ≥ 2 and zero-cost cases NT1: formal statement + closed-form numerical construction; both loss profiles; kernel-only criterion shown suboptimal by factor ≈ 1.73228; numbers verified computationally

79

Subsection A.4

closed

Subsection A.5, Part 1

closed

Subsection A.5, Step 1.3

closed

Subsection A.5, Part 2

closed

Subsection A.5, Part 3

closed

Subsection A.6

closed

Subsubsection A.7.1

closed

Subsubsection A.7.2

closed with caveat

Subsubsection A.7.3

closed

Subsection A.8

closed

Huang

18 Provenance: transportation (Kaufmann et al., 2016), kl inequality (Kaufmann et al., 2016), Nitinawarat and Veeravalli (2015) (B3), Bartók et al. (2014) (B1), Sion 1958 (minimax); contribution restricted to the adequacy-label Alt + task-loss coupling model and the decision-theoretic semantics of C(E, f )

preamble, Subsubsection A.2.6, Subsections A.7, A.8

closed

Known limitations and honest boundaries. (i) Theorem 8 is stated for δ ∈ (0, 21 ); for δ > 12 the information constraint is vacuous (Remark 22). (ii) The comparison with Nitinawarat and Veeravalli (2015) in B3 is a formal consistency check of constants in the common specialization, not an embedding of the Markovian controlled-sensing model. (iii) In B2, the elementary one-sided strategy stops a.s. only under Pi (it abstains, rather than errs, under alternatives); fully admissible certification at rate o(log(1/δ)) is delegated to the CTS strategy of Theorem 9 (the exact O(1) scale is left as an open note, see the main text, B2). (iv) The kernel-switching caveat NT2 of the main text (kernel-switching actions making transcripts policy-dependent) is outside the scope of this lower bound and is an open direction, as stated in the specification. Hypothesis dependency. Theorem 8 requires only 11–13; it does not rely on 36 (pairwise distinguishability of the true environment from all others, including samelabel environments) or 37 (full support of all channels), which are additional hypotheses of the upper bound in Appendix B. The hypotheses of the combined limit theorem limδ→0 Rδ∗ (Ei )/ log(1/δ) = Ci (Corollary 10) are the union of the two documents’ hypotheses: 11, 12, 36, 13, 37.

Appendix B. The CTS Strategy and Proof of the Upper Bound (Theorem 9) Guide to this section. This section gives a complete, self-contained proof of Theorem 9: part (a) (δ-correctness of the CTS policy under any sampling rule) and part (b) (asymptotic cost matching lim supδ→0 R(π ∗ , Ei )/ log(1/δ) ≤ Ci ). Every technical step (martingale verification, union-bound series, concentration constants, tracking error, plug-in stability, cost assembly) is written out in full. Deviations from / additions to the companion statement that were necessary to close the proof are flagged inline and collected in the completeness checklist of Subsection B.8. The structure and logical dependencies are as follows. Subsection B.1 fixes notation and states the two additional hypotheses 36 and 37. Subsection B.2 defines the CTS policy π ∗ : likelihood and MLE, the plug-in design w∗ (·) (well-posedness in Lemma 39), C-tracking sampling with forced exploration, and the GLR stopping rule. Subsection B.3 proves part (a) (Theorem 43): the test supermartingale (Lemma 44), Doob’s maximal inequality (Lemma 45), the geometric-grid upgrade (Lemma 46), and the final union bound. Subsection B.4 proves part (b) in four steps: deterministic forced-exploration 80

Self-Certification of Representation Adequacy

coverage and budget (Lemmas 49 and 51); Step (i), MLE consistency and exact plug-in stabilization (Lemmas 52–55); Step (ii), the tracking lemma (Lemma 57); Step (iii), per-action LLR concentration (Lemmas 59 and 61) and the stopping-time upper bound (Theorem 62); Step (iv), cost assembly via the LP identity (Lemma 64) yielding Theorem 65. Subsection B.5 assembles Theorem 9 and Corollary 10. Subsection B.6 treats the degenerate cases (Lemma 67, Theorem 68). Subsections B.7 and B.8 record the technical provenance and the completeness self-check with the residual statements. Provenance disclaimer (stated up front, per project discipline). The proof techniques are adaptations of known templates: Track-and-Stop / C-tracking / GLR stopping / forced exploration and the β(t, δ) threshold are from Garivier and Kaufmann (2016, Theorems 10 and 14); the cost-weighted characteristic-time design follows Kanarios et al. (2024); the tighter threshold of Kaufmann–Koolen (2021) is mentioned as an optional improvement and is not used here (we keep the more conservative β for simplicity of analysis). The contribution of the present work is the model (adequacy-label-defined alternative + taskloss coupling) and the decision-theoretic semantics of the constant C(E, f ), not the proof technology. B.1 Preliminaries B.1.1 Model and Notation (Recap of (M1)) We work in model (M1); all notation below is fixed for the whole section. • Environments. Finite class E = {E1 , . . . , EK }, K ≥ 2, with fixed representation f . ∗ (f (H)) = R∗ (H)] ∈ Each environment carries a binary adequacy label Θi := 1[RE Ei i {0, 1}. The alternative set of environment i is Alt(i) := {j : Θj ̸= Θi }. • Actions and channels. Finite action set A; we write K = |E| for the number of environments and A := |A| for the number of actions where the two could be confused (in most formulas K appears as the number of arms in the tracking lemmas, i.e. |A|; we keep the symbol KA := |A| explicit in the tracking lemmas to avoid ambiguity). Under Ei , action a produces an observation Y ∼ Kia , a probability distribution on the finite observation space Y, and incurs task loss gi (a) ≥ 0. • Interaction. A policy π picks At based on Ht−1 = (A1 , Y1 , . . . , At−1 , Yt−1 ); then Yt ∼ KiAt . Assumption 11: conditionally on the action sequence, observations are independent; equivalently, for the natural filtration Ft := σ(A1 , Y1 , . . . , At , Yt ), Pi (Yt ∈ · | Ft−1 , At ) = KiAt (·). • Stopping and decision. τ is a stopping time w.r.t. (Ft ); the terminal decision b ∈ {0, 1} is Fτ -measurable. Counts Na (t) := P 1[As = a]. Θ s≤t   P P τ • Cost. R(π, Ei ) := P Eπi gi (a) Eπi [Na (τ )] (the equality is Tonelli’s t=1 gi (At ) = aP theorem applied to t≥1 gi (At )1[t ≤ τ ] = a gi (a)Na (τ ); no integrability condition is needed for the identity itself, finiteness of the right-hand side is proved in Subsubsection B.4.6). 81

Huang

• KL divergences. For j ∈ Alt(i) and a ∈ A we abbreviate X K a (y) dja (i) := d(Kia ∥Kja ) = Kia (y) log ia , Kj (y) y∈Y

and we drop the argument (i) when i is the true environment under study. • Assumptions from the statement. 11 channel conditional independence; 12 feasibility: ∀i, ∀j ∈ Alt(i), ∃a : 0 < d(Kia ∥Kja ) < +∞; 13 g finite, nonnegative. P P j • The constant. Ci := min a gi (a)ma : ma ≥ 0, a ma da (i) ≥ 1 ∀j ∈ Alt(i) (LP covering form, Definition 25), with the convention Ci = +∞ if infeasible and Ci = 0 if the LP value is 0. By LP strong duality (feasible and bounded under 12), P h h i−1 X w dj (i) i−1 j wa Pa a a = sup inf Ci = sup inf d (i) , gi (a) a w∈∆(A) j∈Alt(i) a w∈∆(A) j∈Alt(i) a gi (a) wa where the first form is the cost-ratio characteristic time (proved in Lemma 64 below; it is the form the algorithm uses) and the second is the (w/g)-weighted max– min form stated in the main text (proved there via Sion’s minimax theorem; the two suprema coincide in value, see Remark 41). Throughout, i∗ denotes the true environment when we analyze part (b), and we write Θ∗ := Θi∗ , Alt∗ := Alt(i∗ ), dja := dja (i∗ ), g(a) := gi∗ (a). B.1.2 Additional Technical Assumptions (Minimal Sufficient Additions, Flagged) The proof requires two assumptions beyond 11–13. Both are flagged in the checklist of Subsection B.8. Assumption 36 (Pairwise distinguishability of the class). For every ordered pair (i, j) with i ̸= j (whether or not j ∈ Alt(i)) there exists a with d(Kia ∥Kja ) > 0. For Theorem 9(b) at a fixed i∗ , only the pairs (i∗ , j) are needed; the full pairwise version is used to guarantee τ < ∞ a.s. under every environment (Remark 63), as required for the δ-correctness corollary of Theorem 9(a). (This is hypothesis (H2+) of the main text.) Why 36 is needed. 12 only separates i∗ from environments with the opposite label. The MLE bıt must identify i∗ among all environments, including same-label ones. If j ̸= i∗ satisfied Kja = Kia∗ for all a (same channels, same label) but gj ̸= gi∗ , then no data can separate i∗ from j, while the plug-in designs w∗ (j) and w∗ (i∗ ) differ, so the C-tracking target stabilizes at a wrong design w∗ (j) ̸= w∗ (i∗ ) (under the fixed tie-break the MLE stabilizes at j within the inseparable pair, rather than oscillating) and Na (t)/t would not converge to w∗ (i∗ ). 36 excludes exactly this pathology. It is minimal in the sense that if it fails for the pair (i∗ , j) with w∗ (j) ̸= w∗ (i∗ ), the conclusion of Step (i) below is false for every policy of CTS type. (If Kj = Ki∗ and gj = gi∗ and Θj = Θi∗ , then the two design problems coincide and the wrong plug-in limit is harmless; 36 is the clean sufficient condition that removes the need for this case distinction.) Assumption 37 (Full support / bounded log-likelihood increments). There exists qmin > 0 such that Kia (y) ≥ qmin for all i, a, y. (This is hypothesis (H4) of the main text.) 82

Self-Certification of Representation Adequacy

Consequences used. (i) All divergences are finite: d(Kia ∥Kja ) ≤ log(1/qmin ). (ii) The LLR increments are uniformly bounded: B := max max log i,j,a

y∈Y

1 Kia (y) < ∞. ≤ log a Kj (y) qmin

(68)

(iii) The maps P 7→ d(P ∥Kja ) are continuous on ∆(Y) with an explicit modulus (Subsubsection B.4.2, Eq. (86)). 37 can be relaxed to “Kia∗ ≪ Kja for all a, j with d(Kia∗ ∥Kja ) < ∞” at the price of replacing B by the maximum log-ratio on the support; we keep the clean version. Remark 38 (On the rate of MLE stabilization). A natural alternative route to MLE consistency is to analyze the sign of LLRi∗ ,j (t) directly. Under forced exploration alone, a √ same-label competitor j may be separated from i∗ only at drift rate Θ( t), and converting this into an “eventually always positive” sign statement requires uniform control of the LLR fluctuations over the random time changes Na (t) and over the finitely many challengers j simultaneously—a detour we do not take. The route used in Subsubsection B.4.2 (mirroring the use of consistent per-arm empirical means by Garivier and Kaufmann (2016)) is the empirical-distribution KL projection view of the MLE: √ the lack-of-fit of the true model is O(log t) while every wrong model has lack-of-fit ≥ c t, with stable (eventually permanent) separation. This KL-projection route yields exact stabilization directly from the weaker entry-wise concentration of Lemma 53, together with the explicit tail bound (88), and makes the finite-class simplification “bıt = i∗ eventually” fully rigorous. B.1.3 Conventions • log is the natural logarithm; 0 log 0 = 0; 1[·] is the indicator. • ∆(A) := {w ∈ RA ≥0 :

P

a wa = 1}.

• L := log(1/δ). • “a.s.” without qualification means Pi∗ -a.s. in part (b) and Pj -a.s. in part (a) when the environment is j. • Constants c1 , c2 , . . . and C1 , C2 , . . . are positive and independent of δ and t; they may depend on (E, A, Y, g, qmin ) and on the free parameter ε where indicated. • In the tracking lemmas we write KA := |A| for the number of actions (the “K” of Lemma 7 of Garivier and Kaufmann (2016)), to avoid confusion with K = |E|. B.2 The CTS Policy π ∗ : Complete Definition The policy has three components (sampling, stopping, decision), defined in Subsubsections B.2.1–B.2.4. The definition coincides with the one in the main text except for the design map w∗ (·), see Remark 41. 83

Huang

B.2.1 Likelihood and the MLE bıt Fix the interaction protocol of (M1). Given the history Ht , the likelihood of environment i is XX X b ta (y) log Kia (y), Na (t) K (69) log KiAs (Ys ) = ℓi (t) := a

s≤t

y

b a (y) := 1 where K t s≤t 1[As = a, Ys = y] is the per-action empirical channel (defined Na (t) arbitrarily, say uniform, when P Na (t) = 0; forced exploration will make Na (t) ≥ 1 for all b ta (y) log K a (y) = −H(K b ta ) − d(K b ta ∥K a ) and dropping t ≥ t0 , see Lemma 49). Using y K i i the i-independent entropy term, X
b a ∥ Ka . Na (t) d K (70) bıt := arg max ℓi (t) = arg min Di (t), Di (t) := t i P

i

i

a

Ties in the arg min are broken by a fixed deterministic rule (e.g. smallest index). This KL-projection form of the MLE is what makes the consistency proof of Subsubsection B.4.2 possible. B.2.2 The Plug-in Design w∗ (i): Well-Posedness Definition (main case gi > 0). For an environment i with gi (a) > 0 for all a and Alt(i) ̸= ∅, define the cost-weighted information rate of an allocation w ∈ ∆(A): P minj∈Alt(i) Dj (w; i) w dj (i) Pa a a , (71) = Φi (w) := inf c(w; i) j∈Alt(i) a gi (a)wa P P where Dj (w; i) := a wa dja (i) and c(w; i) := a gi (a)wa . Then w∗ (i) := the lexicographically smallest element of arg max Φi (w).

(72)

w∈∆(A)

Lemma 39 (Well-posedness). Assume gi (a) > 0 for all a and 12 for i. Then (i) Φi is continuous on ∆(A); (ii) the supremum in (72) is attained, so the arg max is nonempty and w∗ (i) is well-defined; (iii) Φi (w∗ (i)) = 1/Ci > 0, in particular minj Dj (w∗ (i); i) = c(w∗ (i); i)/Ci > 0. Proof (i) c(w; i) ≥ mina gi (a) =: gmin > 0, so w 7→ 1/c(w; i) is continuous. Each w 7→ Dj (w; i) is linear, hence continuous; the pointwise minimum of finitely many continuous functions is continuous; the ratio of two continuous functions with positive denominator is continuous. (ii) ∆(A) is compact and Φi continuous, so the supremum is attained (Weierstrass). (iii) is the content of the LP identity, proved as Lemma 64 below (it is used only from Subsubsection B.4.6 onward; no circularity, since Subsubsections B.4.1–B.4.5 use only minj Dj (w∗ ; i) > 0, which follows from Φi (w∗ ) ≥ Φi (w◦ ) > 0 where w◦ is any allocation with minj Dj (w◦ ; i) > 0—such w◦ exists by 12: take the uniform mixture of the discriminating actions aj , one per j ∈ Alt(i), which is a finite set). Definition (degenerate cases). (i) If Alt(i) = ∅: the ratio design is undefined (the inf in (71) is over the empty set); set w∗ (i) := uniform on A (irrelevant: the GLR rule stops 84

Self-Certification of Representation Adequacy

immediately, Subsubsection B.6.2). (ii) If Alt(i) ̸= ∅ but gi (a) = 0 for some a: define w∗ (i) := w∗ (i; gi + η0 ) as the ratio design (72) computed with the fixed perturbed costs gi + η0 , η0 := 1 (well-defined by Lemma 39 since gi + η0 > 0). Two consequences: (a) under 12 for i, minj Dj (w∗ (i); i) > 0 also in degenerate environments—used to guarantee τ < ∞ a.s. under every environment (Remark 63); (b) when the true environment i∗ is itself degenerate, the sharper schedule η = ηδ ↓ 0 is needed to recover the exact constant Ci∗ : this is Theorem 68 (the regime Ci∗ = 0 included). Remark 40 (The plug-in map). The CTS policy uses w∗ (bıt ) with w∗ (·) given by (72) on {i : gi > 0, Alt(i) ̸= ∅} and the convention above otherwise. Since E is finite and bıt is eventually exactly equal to i∗ (Subsubsection B.4.2), no continuity of the map i 7→ w∗ (i) is ever needed—this is the finite-class simplification, used explicitly. Remark 41 (Deviation from thePcompanion statement, flagged). The main text defines the plug-in design as arg supw inf j a (wa /gi (a)) dja (i) (the (w/g)-weighted max–min). That optimization problem has the same value 1/Ci as (71)–(72) (both equal the LP value by duality), but its maximizer is in general not cost-optimal. Counterexample: g = (1, 2), Alt = {1, 2}, d1 = (1, 0.1), d2 = (0.1, 1). Then Ci = 30/11 ≈ 2.73, attained by the ratio maximizer w = (1/2, 1/2) (normalized LP solution); the (w/g)-weighted maximizer is w = (1/3, 2/3), whose realized cost ratio is c(w)/ minj Dj (w) = (5/3)/(0.4) = 25/6 ≈ 4.17 > Ci . With the (w/g)-weighted design, part (b) of Theorem 9 would therefore be false. The ratio P ∗ ∗ design (71)–(72)—equivalently, the normalized LP solution m / a ma , see Lemma 64—is exactly the cost-weighted characteristic-time design of Kanarios et al. (2024), and it is what makes the terminal identity c(w∗ )/ minj Dj (w∗ ) = Ci true by construction. B.2.3 Sampling: C-Tracking with Forced Exploration q 2 −2K (the forced-exploration curve of Garivier and Kaufmann (2016); Let f (t) := t + KA A note f is nondecreasing, f (0) = −KA < 0, and f (t) − f (t − 1) ≤ 2K1 A for all t ≥ 1, since √ √ √ √ 2 − 1 ≥ K 2 ). Define the u − u − 1 = 1/( u + u − 1) ≤ 1/(2KA ) for u = t + KA A under-explored set  Ut := a ∈ A : Na (t) < f (t) . The sampling rule is:  if Ut ̸= ∅ (forced exploration; ties by fixed order),  arg min Na (t) a∈Ut h At+1 = N (t) i  arg max wa∗ (bıt ) − a otherwise (C-tracking; ties by fixed order). t a∈A (73) (At t = 0 we have U0 = ∅ since f (0) < 0, and Na (0)/0 is read as 0; the argmax is then over w∗ (bı0 ) with bı0 arbitrary. Any initialization convention works; the proofs only use t ≥ 1.) Remark 42. This is exactly the C-tracking rule of Garivier and Kaufmann (2016) with target w∗ (bıt ) and their forced exploration curve. Two deterministic properties are proved in Subsubsection B.4.1: coverage N√ a (t) ≥ f (t) − O(KA ) for all a, t (Lemma 49), and a forced-exploration budget F (t) = O( t) (Lemma 51). 85

Huang

B.2.4 Stopping and Decision: GLR Rule For i, j ∈ {1, . . . , K}, i ̸= j, define the log-likelihood ratio process LLRi,j (t) :=

X

log

s≤t

KiAs (Ys ) KjAs (Ys )

,

LLRi,j (0) := 0,

(74)

with the convention log(0/0) = 0 and log(x/0) = +∞ for x > 0 (under Pj , KjAs (Ys ) > 0 a.s., so the ratio is a.s. well-defined; see Subsubsection B.3.1). The threshold is β(t, δ) := log The stopping time and decision are n τ := inf t ≥ 1 : max i

bı := arg max i

2t(K − 1) , δ

min

j: Θj ̸=Θi

(75)

o LLRi,j (t) > β(t, δ) ,

min LLRi,j (τ ),

j: Θj ̸=Θi

t ≥ 1.

(76) b := Θbı , Θ

with the conventions min over an empty set equals +∞ (so if Alt(i) = ∅ for the maximizer, b ̸= Θj , τ < stopping is immediate) and τ := +∞ if the set is empty. Part (a) bounds Pj (Θ ∞); part (b) shows τ < ∞ a.s. and bounds E[τ ]. This is the GLR stopping rule of Garivier and Kaufmann (2016, Theorem 10 threshold family), in the lineage of Chernoff’s sequential tests (Chernoff, 1959) and active sequential hypothesis testing (Naghshvar and Javidi, 2013), specialized to a finite composite alternative defined by adequacy labels. B.3 Part (a): δ-Correctness Under an Arbitrary Sampling Rule Theorem 43 (Theorem 9(a)). Fix any sampling rule (any sequence of Ft−1 -measurable b be defined by (76) with threshold (75). Then under action selections, e.g. (73)), and let τ, Θ 11, for every environment j:
b ̸= Θj , τ < ∞ ≤ δ. Pj Θ In particular, provided τ < ∞ a.s. (proved in Subsubsection B.4.5 for the CTS sampling b ̸= Θj ) ≤ δ. rule under every environment), π ∗ is δ-correct: maxj Pj (Θ Template: Garivier and Kaufmann (2016, Theorem 10). The proof has four steps: (Subsubsection B.3.1) test supermartingale; (Subsubsection B.3.2) Doob’s maximal inequality; (Subsubsection B.3.3) time-uniform control of a growing threshold via a geometric grid, with the series computed explicitly; (Subsubsection B.3.4) error decomposition and union bound over challengers. B.3.1 The Test Supermartingale Fix a pair (i, j) with i ̸= j (no label condition needed). Define Y K As (Ys )
i Mt := exp LLRi,j (t) = , As K j (Ys ) s≤t 86

M0 := 1.

(77)

Self-Certification of Representation Adequacy

Lemma 44. Under Pj , (Mt , Ft )t≥0 is a nonnegative supermartingale with Ej [Mt ] ≤ 1 for all t. If Kia ≪ Kja for all a (in particular under 37), it is a martingale: Ej [Mt | Ft−1 ] = Mt−1 . Proof (no steps skipped) Nonnegativity is clear. Under Pj , KjAs (Ys ) > 0 a.s. (since Ys | As ∼ KjAs ), so each factor is a.s. finite (with the convention of Subsubsection B.2.4) and Mt is a.s. finite. For the supermartingale property, condition on Fs−1 . The sampling rule makes As Fs−1 -measurable, so on {As = a}, h K As (Y ) Ej

s i As Kj (Ys )

i h K a (Y ) i s i Fs−1 = Ej F , A = a s−1 s Kja (Ys ) X X K a (y) Kja (y) ia = = Kj (y) a a

(78) Kia (y)

≤ 1,

y: Kj (y)>0

y: Kj (y)>0

where the second equality uses 11: the conditional law of Ys given Fs−1 and As = a is Kja . The inequality is an equality iff Kia ({y : Kja (y) = 0}) = 0, i.e. Kia ≪ Kja ; under 37 this holds for all a, giving the martingale property. Since As is Fs−1 -measurable, (78) holds on every atom {As = a}, hence Ej [Ms | Fs−1 ] = Ms−1 Ej

h

KiAs (Ys ) KjAs (Ys )

Fs−1

i

≤ Ms−1

a.s.

(79)

Iterating (79) from M0 = 1 gives Ej [Mt ] ≤ 1. Remark. The verification (78) is exactly where 11 enters part (a): the likelihood ratio of the whole adaptive transcript factorizes, and each factor has conditional expectation ≤ 1 under j regardless of how As was chosen. This is why the result holds for any sampling rule. B.3.2 Doob’s Maximal Inequality Lemma 45. For every β > 0 and every finite horizon T ≥ 1,     Pj sup LLRi,j (t) > β ≤ e−β , and Pj sup LLRi,j (t) > β ≤ e−β .

(80)

t≥1

1≤t≤T

Proof (Mt ) is a nonnegative supermartingale (Lemma 44). Doob’s maximal
inequality for nonnegative supermartingales states: for x > 0, x P max0≤t≤T Mt ≥ x ≤ E[M0 ] = 1. Apply with x = eβ and takeSlogarithms to get the first statement. For the second, note {supt≥1 LLRi,j (t) > β} = T {supt≤T LLRi,j (t) > β} is an increasing union, so its probability is the limit of the finite-horizon bounds, each ≤ e−β . B.3.3 Time-Uniform Control of the Growing Threshold β(t, δ) Lemma 45 controls the crossing of a fixed level. The stopping rule uses the time-varying threshold β(t, δ) = log(2t(K − 1)/δ), which is nondecreasing in t. We upgrade to a uniform statement on a geometric grid. 87

Huang

Lemma 46. Let tk := 2k for k = 0, 1, 2, . . . . Then for the pair (i, j):   X δ Pj ∃ t ≥ 1 : LLRi,j (t) > β(t, δ) ≤ e−β(tk ,δ) = . K −1

(81)

k≥0

Proof (full computation) Since β(·, δ) is nondecreasing, for t ∈ [tk , tk+1 ) we have β(t, δ) ≥ β(tk , δ), hence n o  ∃ t ∈ [tk , tk+1 ) : LLRi,j (t) > β(t, δ) ⊆ sup LLRi,j (t) > β(tk , δ) . 1≤t<tk+1

The intervals [tk , tk+1 ) = [2k , 2k+1 ), k ≥ 0, partition {1, 2, 3, . . . }. A union bound over k, followed by Lemma 45 with horizon T = ⌈tk+1 ⌉ − 1, gives  X  X
Pj ∃t : LLRi,j (t) > β(t, δ) ≤ Pj sup LLRi,j (t) > β(tk , δ) ≤ e−β(tk ,δ) . t<tk+1

k≥0

k≥0

2·2k (K−1)

Now insert β(tk , δ) = log : δ X X X δ δ δ δ e−β(tk ,δ) = = 2−k = ·2 = . 2(K − 1) 2(K − 1) K −1 2(K − 1) 2k k≥0

k≥0

k≥0

Remark 47 (Where the constants in P β go). The factor 2 inside β(t, δ) = log(2t(K − 1)/δ) pays exactly for the geometric series k 2−k = 2; the factor (K − 1) pays for the union over at most K − 1 challengers in Subsubsection B.3.4; the factor t pays for the linear growth of the grid index. This is the Theorem 10 threshold of Garivier and Kaufmann (2016); the tighter, time-uniform thresholds of Kaufmann–Koolen (2021) (based on mixture martingales) could replace β(t, δ) by log(1/δ) + O(log log t), but we deliberately keep the conservative closed-form threshold to keep the analysis elementary. B.3.4 Error Decomposition and the Final Union Bound b ̸= Θj }, let i := bı; Proof of Theorem 43 Fix the true environment j. On {τ < ∞, Θ ′ then Θi ̸= Θj , i.e. j ∈ {j : Θj ′ ̸= Θi }, and by the definition of the stopping rule and of bı as the GLR maximizer, min

j ′ : Θj ′ ̸=Θi

LLRi,j ′ (τ ) > β(τ, δ)

=⇒

LLRi,j (τ ) > β(τ, δ).

Therefore 

b ̸= Θj τ < ∞, Θ

⊆

[



∃ t ≥ 1 : LLRi,j (t) > β(t, δ) .

i: Θi ̸=Θj

Taking Pj and applying a union bound over i (there are at most K −1 indices with Θi ̸= Θj ), then Lemma 46 to each pair (i, j): X
δ b ̸= Θj ≤ Pj τ < ∞, Θ ≤ δ. K −1 i: Θi ̸=Θj

Remark 48. The proof used nothing about the sampling rule except Ft−1 -measurability of At (needed for (78)); in particular it does not use 12, 13, 37, or the convergence of Na (t)/t. This is the sense in which Theorem 9(a) holds “for any sampling rule”. 88

Self-Certification of Representation Adequacy

B.4 Part (b): Asymptotic Cost Matching Standing setup for Subsection B.4. The true environment is i∗ ; 11–13, 36, 37 hold; g(a) := gi∗ (a) > 0 for all a (degenerate case in Subsection B.6); Alt∗ ̸= ∅ (otherwise Ci∗ = 0, trivial, see Subsubsection B.6.2). We write w∗ := w∗ (i∗ ), dja := d(Kia∗ ∥Kja ), P P Dj (w) := a wa dja , c(w) := a g(a)wa , and S ∗ := min∗ Dj (w∗ ) > 0, j∈Alt

s∗ :=

1 , S∗

(82)

where S ∗ > 0 by Lemma 39(iii). The goal is lim supδ→0 R(π ∗ , Ei∗ )/L ≤ Ci∗ , achieved through four steps: (i) MLE consistency and plug-in stability (Subsubsection B.4.2); (ii) tracking convergence (Subsubsection B.4.3); (iii) stopping-time upper bound (Subsubsections B.4.4–B.4.5); (iv) cost assembly (Subsubsection B.4.6). Subsubsection B.4.1 collects the deterministic forced-exploration lemmas used everywhere. B.4.1 Forced Exploration: Coverage and Budget Lemmas q 2 − 2K and the rule (73). Both lemmas are deterministic: they Recall f (t) = t + KA A hold for every realization, regardless of the targets w∗ (bıt ). Lemma 49 (Coverage). For all t ≥ 0 and all a ∈ A: √ Na (t) ≥ f (t) − KA ≥ t − 3KA . (83) √ In particular Na (t) → ∞ at rate t uniformly over a. q √ 2 − 2K ≥ Proof The second inequality uses f (t) = t + KA t − 2KA . For the first, A define the deficit of arm a at time t by df a (t) := f (t) − Na (t) and consider the potential X + S(t) := df a (t) − 1 ≥ 0. a∈A

We show S(t) ≤ KA − 1 for all t by induction on t; the claim then follows from df a (t) − 1 ≤ [df a (t) − 1]+ ≤ S(t) ≤ KA − 1. Base: df a (0) = f (0) = −KA < 0, so S(0) = 0. Inductive 1 √ step t − 1 → t: write f ′ := f (t) − f (t − 1) = √ ≤ 2K1 A ≤ 12 , and note the 2 2 t+KA +

t−1+KA

key structural fact: when forced exploration is active (Ut−1 ̸= ∅), the rule plays a violator of maximum deficit, because arg minc∈Ut−1 Nc (t − 1) = arg maxc∈Ut−1 df c (t − 1) (the same f (t − 1) is subtracted from every count). In one step an unplayed arm’s deficit grows by exactly f ′ and the played arm’s deficit changes by −1 + f ′ . Distinguish three cases. Case 1: Ut−1 = ∅ (tracking step). Every arm is a non-violator: df c (t − 1) ≤ 0. The played arm’s new deficit is ≤ −1 + f ′ < 0; every unplayed arm’s new deficit is ≤ f ′ ≤ 12 < 1. Hence S(t) = 0. Case 2: Ut−1 ̸= ∅ and D∗ := maxc∈Ut−1 df c (t − 1) < 1. Every arm’s deficit at t − 1 is < 1 (violators by assumption, non-violators since their deficit is ≤ 0). Hence every arm’s new deficit is < 1 + f ′ , i.e. its new excess over 1 is at most f ′ ; and the played arm (a violator) has new deficit < 1 − 1 + f ′ = f ′ < 1, excess 0. Therefore S(t) ≤ (KA − 1) f ′ ≤ 89

KA − 1 ≤ KA − 1. 2KA

Huang

Case 3: Ut−1 ̸= ∅ and the played arm b := At has eb := df b (t − 1) − 1 ≥ 0 (i.e. D∗ ≥ 1). Unplayed violators c satisfy df c (t − 1) ≤ D∗ = 1 + eb , so their excess over 1 grows by at most f ′ and their new excess is at most eb + f ′ ; non-violators keep excess 0 (their new deficit is ≤ f ′ ≤ 1). • Sub-case 3a: eb ≥ 1 − f ′ . The played arm’s new excess is exactly eb − (1 − f ′ ) ≥ 0: it drops by 1 − f ′ . Each unplayed violator’s excess grows by at most f ′ , and there are at most KA − 1 of them, so S(t) ≤ S(t−1)−(1−f ′ )+(KA −1)f ′ = S(t−1)−1+KA f ′ ≤ S(t−1)−

1 ≤ KA −1, 2

using S(t − 1) ≤ KA − 1 (induction) and f ′ ≤ 2K1 A . • Sub-case 3b: 0 ≤ eb < 1 − f ′ . The played arm’s new excess is 0, and every unplayed violator’s new excess is at most eb + f ′ < 1, so S(t) ≤ (KA − 1) (eb + f ′ ) < KA − 1. In all cases S(t) ≤ KA − 1, completing the induction. Remark 50. The proof is tie-break-agnostic (any maximizer of the deficit works) and uses nothing about the targets w∗ (bıt ): coverage holds uniformly over the pre-stabilization phase as well. The potential argument is a self-contained version of the Lemma 7 analysis of Garivier q and Kaufmann (2016); the curve

2 − 2K is chosen precisely so that the one-step t + KA A

deficit growth f ′ ≤ 2K1 A is dominated by the unit decrease from playing the maximum-deficit violator. (Numerical spot-check: with adversarially switching targets the observed maximum deficit is far below KA .) P Lemma 51 (Forced-exploration budget). Let F (t) := s≤t 1[Us−1 ̸= ∅] be the number of forced-exploration rounds up to t. Then √

F (t) ≤ KA f (t) + 1 ≤ KA t + KA . (84) Proof A forced play of arm a at time s requires a ∈ Us−1 , i.e. Na (s − 1) < f (s − 1) ≤ f (t), and increases Na by one. Let sr ≤ t be the last forced-play time of arm a up to t: then the number of forced plays of a up to t is at most Na (sr ) = Na (sr − 1) + 1, with Na (sr − 1) < f (sr − 1) ≤ f (t). Since Na is integer-valued, nondecreasing and starts at 0, this number is at most ⌊f (t)⌋ + 1 ≤ q f (t) + 1. Summing over the KA arms and using q √ √ 2 2 ≤ t + KA − 2KA + 1 ≤ t + KA (since t + KA t + KA ) gives (84). B.4.2 Step (i): MLE Consistency and Exact Plug-in Stabilization This subsubsection proves that bıt = i∗ for all sufficiently large t, a.s., with a quantitative tail bound on the stabilization time. The argument uses only 11, 37, 36 and the coverage Lemma 49 — not the tracking rule — so there is no circularity with Subsubsection B.4.3. 90

Self-Certification of Representation Adequacy

Lemma 52 (Per-action observations are i.i.d.). Under Pi∗ , for each action a, the observations (Ys )s: As =a are i.i.d. ∼ Kia∗ ; more precisely, for every n and every measurable set B, n
Y Kia∗ (Br ), Pi∗ Ys1 ∈ B1 , . . . , Ysn ∈ Bn = r=1

where s1 < · · · < sn are the first n times action a is played (stopping times w.r.t. the enlarged filtration including As ). Proof By 11, Pi∗ (Ys ∈ B | Fs−1 , As = a) = Kia∗ (B). The claim follows by induction on n: condition on the history up to sn − 1; sn and Asn = a are determined by that history, so P(Ysn ∈ Bn | Fsn −1 , sn , Asn ) = Kia∗ (Bn ), which is constant, hence independent of (Ys1 , . . . , Ysn−1 ). p Lemma 53 (Uniform empirical deviation bound). Let h(n) := 5 log n/n for n ≥ 2. For m ≥ 3, define the event n o emp b a (y) − Kia∗ (y) ≤ h(n) , Gm := ∀a ∈ A, ∀y ∈ Y, ∀n ≥ m : K (n) b a is the empirical distribution of the first n observations of action a (Lemma 52). where K (n) Then
2KA |Y| emp c (m − 1)−9 . (85) Pi∗ (Gm ) ≤ 9 Proof (full computation) For fixed (a, y), the indicators 1[Ysr = y] are i.i.d. Bernoulli b a (y) is their empirical mean. Hoeffding’s inequality with mean Kia∗ (y) (Lemma 52), and K (n) for i.i.d. [0, 1]-bounded variables gives, for each fixed n,


b a (y) − Kia∗ (y)| > h(n) ≤ 2 exp − 2n h(n)2 = 2 exp − 10 log n = 2n−10 . P |K (n) R∞ P Union bound over n ≥ m (with n≥m n−10 ≤ m−1 x−10 dx = 91 (m − 1)−9 ) and then over the KA |Y| pairs (a, y) gives (85). Lemma 54 (KL modulus of continuity under 37). For any P, P ′ , Q ∈ ∆(Y) with Q ≥ qmin :
d(P ∥Q)−d(P ′ ∥Q) ≤ ω ∥P −P ′ ∥1 ,

ω(u) := u log

u u + log(|Y|−1)+h2 , (86) qmin 2 2 1

where h2 (p) := −p log p − (1 − p) log(1 − p); in particular ω(u) → 0 as u → 0. Moreover d(P ∥Q) ≤

X (P (y) − Q(y))2 y

Q(y)

≤

|Y| max (P (y) − Q(y))2 . qmin y

(87)

P P P Proof Since d(P ∥Q) = y P (y) log P (y)− y P (y) log Q(y) = −H(P )− y P (y) log Q(y),   P
we have d(P ∥Q) − d(P ′ ∥Q) = H(P ′ ) − H(P ) + y P ′ (y) − P (y) log Q(y). The second term is bounded by ∥P − P ′ ∥1 maxy | log Q(y)| ≤ u log(1/qmin ). For the entropy 91

Huang

term, the Audenaert sharp bound gives |H(P ) − H(P ′ )| ≤ T log(|Y| − 1) + h2 (T ) with P P (y) ≤ T := 12 ∥P −P ′ ∥1 . This yields (86). For (87), use log x ≤ x−1: d(P ∥Q) = y P (y) log Q(y) P P (P (y)−Q(y))2 P (y)−Q(y) = (writing P = (P − Q) + Q and simplifying), then y P (y) y Q(y) Q(y) bound 1/Q ≤ 1/qmin and the sum over y by |Y| maxy (·)2 . Lemma 55 (Exact stabilization of the MLE). Under 11, 37, 36 and the sampling rule (73), there exist (deterministic) t3 and c4 > 0 such that
Pi∗ ∃ s ≥ t : bıs ̸= i∗ ≤ c4 t−9/2 ∀ t ≥ t3 . (88) Consequently T0 := sup{t : bıt ̸= i∗ } is a.s. finite with P(T0 > t) ≤ c4 t−9/2 , E[T0 ] < ∞ (indeed E[T0p ] < ∞ for p < 7/2), and bıt = i∗ ∀t > T0 a.s. =⇒ w∗ (bıt ) = w∗ (i∗ ) ∀t > T0 a.s. (89) √ √ Proof Set m(t) := ⌈ t⌉ − 3KA − 2; by Lemma 49 (Eq. (83)), Na (s) ≥ s − 3KA ≥ emp m(s) ≥ m(t) for all a and all s ≥ t (for t large enough that m(t) ≥ 3). Work on Gm(t) . By P b a ∥K a ). (70), bıs = arg mini Di (s) with Di (s) = a Na (s)d(K s i emp ba = K ba True model. By (87) and the definition of Gm(t) (note K s (Na (s)) with Na (s) ≥ m(t)), for every s ≥ t: X X |Y| |Y| 5 log Na (s) Di∗ (s) ≤ Na (s) · h(Na (s))2 ≤ · Na (s) · qmin qmin Na (s) a a (90) 5KA |Y| log s =: C5 log s. ≤ qmin a

a

emp Wrong models. Fix j ̸= i∗ . By 36 choose aj with daj := d(Ki∗j ∥Kj j ) > 0. On Gm(t) , for s ≥ t:


d b saj − K a∗j ∥1 ≥ da − ω |Y| h(m(s)) ≥ aj , b saj ∥ K aj ≥ da − ω ∥K d K (91) j j i j 2 where the second inequality uses ∥ · ∥1 ≤ |Y| maxy | · | and monotonicity of ω for small arguments, and the last holds for all s ≥ t ≥ t3 with t3 chosen (deterministically) so that ω(|Y|h(m(t3 ))) ≤ 21 minj̸=i∗ daj — possible since h(m(t)) → 0 and ω(u) → 0. Since all terms in Dj (s) are nonnegative (KL ≥ 0),

√
daj m(s) daj m(t) d+ min ≥ ≥ ≥ cd t − 3KA − 2 , (92) 2 2 2 √ √ with d+ s, there is a min := minj̸=i∗ daj > 0. Since C5 log s = o( s) while (92) grows like deterministic t4 ≥ t3 such that for all s ≥ t ≥ t4 : Di∗ (s) < minj̸=i∗ Dj (s), hence bıs = i∗ (the tie-break does not trigger since the inequality is strict). Therefore Dj (s) ≥ Naj (s) ·

2KA |Y| emp c
Pi∗ ∃s ≥ t : bıs ̸= i∗ ≤ P (Gm(t) ) ≤ (m(t) − 1)−9 ≤ c4 t−9/2 , 9 √ P using m(t) − 1 ≥ t/3 for t large. E[T0 ] = t P(T0 > t) < ∞ and the moment statement follow from the polynomial tail. Finally (89): w∗ (·) is a fixed deterministic function (lexicographic selection, Subsubsection B.2.2), so bıt = i∗ implies w∗ (bıt ) = w∗ (i∗ ) exactly. 92

Self-Certification of Representation Adequacy

Remark 56 (The finite-class simplification, made explicit). Because E is finite, MLE consistency in the discrete topology means exact stabilization, which turns the plug-in map w∗ (bıt ) into the constant w∗ (i∗ ) eventually. No continuity or differentiability of w∗ (·) in any parameter is needed—in contrast to the continuous-parameter setting of Garivier and Kaufmann (2016), where one must prove regularity of θ 7→ w∗ (θ). The price is 36: exact identification must be possible against every competitor, including same-label ones; see Subsubsection B.1.2 for why this is minimal, and Remark 38 for why the proof goes through the KL-projection form (70) rather than through LLR signs. B.4.3 Step (ii): The Tracking Lemma By Lemma 55, the tracking target of rule (73) is frozen at w∗ := w∗ (i∗ ) for all t > T0 . The following lemma is deterministic given T0 ; it is the content of Lemmas 8–9 of Garivier and Kaufmann (2016) adapted to our notation, proved in full. Lemma 57 (C-tracking with frozen target). Define the (random) clock shift Ba (u) := ∗ (T P0 + u) wa − Na (T0 + u) for u ≥ 0, where T0 is the stabilization time of Lemma 55. Then a Ba (u) = 0 for all u, and for every t ≥ T0 and every a: √

Na (t) − t wa∗ ≤ (KA − 1) T0 + 1 + (KA − 1) KA t + KA . (93) {z } | =: ξ(t)

Consequently (deterministically) Na (t)/t → wa∗ for all a on {T0 < ∞}, hence Pi∗ -a.s.; and p 

E Na (τ ) − τ wa∗ E[τ ] + KA + E[T0 ]. (94) ≤ (KA − 1) E[T0 ] + 1 + (KA − 1)KA Proof Bookkeeping. For t = T0 +u write Ba (u) := twa∗ −Na (t); then If arm b is played at time t + 1 (forced or tracking), then

P

a Ba (u) = t−t = 0.

Bc (u + 1) = Bc (u) + wc∗ (c ̸= b).

Bb (u + 1) = Bb (u) − (1 − wb∗ ),

(95)

Minimum bound. On a tracking step (non-forced), theP played arm maximizes wa∗ − Na (t)/t, equivalently maximizes Ba (u) (multiply by t); since a Ba (u) = 0, the maximizer satisfies Bb (u) ≥ 0, hence after the step Bb (u + 1) ≥ −1 + wb∗ ≥ −1; unplayed arms only increase (wc∗ ≥ 0). On a forced step, the played arm’s B decreases by1 − wb∗ ≤ 1. Hence, with F
(u) := number of forced steps in (T0 , T0 + u] and M0 := max 1, − mina T0 wa∗ − Na (T0 ) ≤ T0 + 1 (using Na (T0 ) ≤ T0 ), √

min Ba (u) ≥ − M0 + F (u) ≥ − T0 + 1 + KA ( t + KA ) , (96) a

√ where F (u) ≤ F (t) ≤ KA ( P t + KA ) by Lemma 51. Maximum P bound. Since Pa Ba (u) = 0, at most KA − 1 terms are negative, so for each a: Ba (u) = − c̸=a Bc (u) ≤ c̸=a [Bc (u)]− ≤ (KA − 1)(M0 + F (u)). Combined with (96), √ |Ba (u)| ≤ (KA − 1)(M0 + F (u)) ≤ (KA − 1)(T0 + 1) + (KA − 1)KA ( t + KA ), which is (93). Convergence. On {T0 < ∞}, ξ(t)/t → 0 as t → ∞ (T0 is a finite constant for the √ realization, t/t → 0), so Na (t)/t → wa∗ ; by Lemma 55, T0 < ∞ a.s. For (94): on {τ ≤ T0 } use |Na (τ ) − τ wa∗ | ≤ τp ≤ T0 ; on {τ > T0 } use (93) at t = τ ; then take expectations and √ apply Jensen E[ τ ] ≤ E[τ ] (valid since E[τ ] < ∞, proved in Subsubsection B.4.5).

93

Huang

Remark 58. The proof needs no continuity of the target √ map: the target is exactly constant after T (finite-class simplification). Note the error is O( t) deterministically given T0 , and √ 0 O( Eτ ) in expectation—this is what replaces a uniform-integrability argument in the cost assembly (Subsubsection B.4.6). B.4.4 Step (iii), Part 1: Per-Action LLR Concentration (Good Events with Explicit Constants) a a P a , where X a := log Ki∗ (Yr ) and (Y a ) are For j ∈ Alt∗ and a ∈ A, let Sja (n) := nr=1 Xj,r a a r j,r K (Y ) j

r

a ] = dj and |X a | ≤ B a.s. by the i.i.d. observations of action a (Lemma 52). Then E[Xj,r a j,r 37, Eq. (68). Note X
LLRi∗ ,j (t) = Sja Na (t) . (97) a

Lemma 59 (Uniform deviation bound, explicit constants). For every ε > 0, x > 0, and each pair (a, j):   Pi∗ ∃ n ≥ 1 : Sja (n) − n dja > εn + x ≤

2

 ε  x . exp − B2 1 − e−ε2 /(2B 2 ) | {z }

(98)

=: c1 (ε)

Proof (full computation) Hoeffding’s inequality applied to the i.i.d. [−B, B]-bounded increments: for each fixed n, u > 0, P |Sja (n) − ndja | > u

u2  . 2B 2 n

 ≤ 2 exp −

Set u = εn + x and use (εn + x)2 = ε2 n2 + 2εnx + x2 ≥ ε2 n2 + 2εnx: (εn + x)2 ε2 n εx ≥ + 2 2 2 2B n 2B B Union bound over n ≥ 1: (98).

=⇒

2 2 2 P |Sja (n) − ndja | > εn + x ≤ 2 e−εx/B e−ε n/(2B ) .

−ε2 n/(2B 2 ) n≥1 e

P

2

=

2

e−ε /(2B ) 2 2 1−e−ε /(2B )

≤

1 2 2 , 1−e−ε /(2B )

which gives

Definition 60 (Good event). For ε > 0 and t ≥ 1, define Et (ε) :=

n εt o ∀a ∈ A, ∀j ∈ Alt∗ : Sja (Na (t)) − Na (t) dja ≤ εNa (t) + . KA

(99)

Lemma 61 (Bad-event probability). With c3 (ε) := KA (K − 1) c1 (ε) and c4 := 1/(KA B 2 ), Pi∗ Et (ε)c

2

≤ c3 (ε) e−c4 ε t .

(100)

Moreover on Et (ε) ∩ {t ≥ T0 }, for every j ∈ Alt∗ : LLRi∗ ,j (t) ≥ t Dj (w∗ ) − 2εt − KA d¯ ξ(t), 94

1 , d¯ := max dja ≤ log a,j qmin

(101)

Self-Certification of Representation Adequacy

and consequently √
min∗ LLRi∗ ,j (t) ≥ t S ∗ − 2ε − C6 t

j∈Alt

on Et (ε) ∩ {T0 ≤

√

t} ∩ {t ≥ t̄},

(102)

where C6 and t̄ are deterministic constants (depending on (E, A, Y, g, qmin ) only). Proof (100): apply Lemma 59 with x = εt/KA to each of the KA |Alt∗ | ≤ KA (K − 1) pairs (a, j) and take
a union bound; the exponential factor per pair is exp(−εx/B 2 ) = exp − ε2 t/(KA B 2 ) , whence the stated value c4 = 1/(KA B 2 ); note the bound is uniform in n, so it applies to the random count Na (t) without any conditioning. (101): on Et (ε), summing the deviation bound over a: X X X X Na (t)dja − 2εt, Na (t) − εt = Na (t)dja − ε Sja (Na (t)) ≥ LLRi∗ ,j (t) = a

a

a

a

P j and by Lemma 57 (for t ≥ T0 ), Na (t) √ ≥ twa∗ − ξ(t), so a Na (t)dja ≥ tDj (w∗ ) − ξ(t) a da ≥ √ ∗ ¯ tD√ t}, ξ(t) = (KA −1)(T0 +1)+(KA −1)KA ( t+KA ) ≤ j (w )−KA d ξ(t). (102): on {T0 ≤ C6′ t for t ≥ t̄ with deterministic C6′ , t̄; take C6 := KA d¯C6′ . P

B.4.5 Step (iii), Part 2: Stopping-Time Upper Bound Theorem 62. Under the standing setup, with s∗ = 1/S ∗ (Eq. (82)) and L = log(1/δ):
Ei∗ [τ ] ≤ s∗ L 1 + o(1) (δ → 0), (103) where the o(1) depends on (E, A, Y, g, qmin ) but not on the sampling history. Moreover τ < ∞ a.s. and E[τ p ] < ∞ for every p < 45 (in fact P(τ > t) = O(t−9/4 ) beyond the O(L) scale, plus an exponentially small term). More precisely, for fixed δ the tail bound is 2 the sum of the exponential term c3 (εδ )e−c4 εδ t — which by itself would permit moments of every order — and the polynomial term O(t−9/4 ), which alone permits all p < 49 (the stated p < 54 is conservative; higher prescribed moments are available by raising the exponent in Lemma 53, cf. Remark 66). Proof Set ε = εδ := L−1/8 (any schedule with εδ → 0 and √ ε−4 δ = o(L) works). Step 1: the deterministic crossing time. On the event Et (ε) ∩ {T0 ≤ t} ∩ {t ≥ t̄}, by (102) the stopping statistic satisfies √
max min LLRi,j (t) ≥ min∗ LLRi∗ ,j (t) ≥ t S ∗ − 2ε − C6 t. i

j:Θj ̸=Θi

j∈Alt

√ Hence τ ≤ t as soon as t(S ∗ − 2ε) − C6 t > β(t, δ) = L + log(2t(K − 1)). Define √

t1 (δ, ε) := min t ≥ t̄ : t S ∗ − 2ε − C6 t ≥ L + log 2t(K − 1) . √ The constraint function ψ(t) := t(S ∗ − 2ε) − C6 t − β(t, δ) is nondecreasing on t ≥ t̄′ for a deterministic t̄′ : its continuous derivative (S ∗ − 2ε) − 2C√6t − 1t is positive for all sufficiently large t since S ∗ − 2ε > 0. Enlarging the deterministic t̄ to max{t̄, t̄′ } if needed, the crossing inequality therefore holds for every t ≥ t1 (δ, ε), which is exactly what the contrapositive of 95

Huang

Step 2 below uses. Inversion computation. Any t satisfying the constraint √ obeys t ≥√L/S ∗ ; ∗ ∗ for t ≤ 2s L we have log(2t(K − 1)) ≤ log(4s L(K − 1)) =:√clog and C6 t ≤ C6 2s∗ L. Hence the constraint is implied by t(S ∗ − 2ε) ≥ L + clog + C6 2s∗ L, i.e. √
√ c C6√ 2s∗ s∗ L 1 + log + L + clog + C6 2s∗ L L L =: t̄1 (δ, ε), t ≥ = S ∗ − 2ε 1 − 2ε/S ∗ provided t̄1 ≤ 2s∗ L (true for L large, since the numerator is L(1 + o(1)) and S ∗ − 2ε → S ∗ ).

log L So t1 (δ, εδ ) ≤ t̄1 (δ, εδ ) = s∗ L 1P + O(εδ ) + O L =P s∗ L(1 + o(1)). Step 2: tail sum. E[τ ] = t≥0 P(τ > t) ≤ t1 + t≥t1 P(τ > t). For t ≥ t1 (≥ t̄), the contrapositive of Step 1 gives √ {τ > t} ⊆ Et (ε)c ∪ {T0 > t}, so by Lemma 61 and Lemma 55, X X X X √ c3 (ε) 2 ′ P(τ > t) ≤ c3 (ε)e−c4 ε t + + c c′4 ( t)−9/2 ≤ t−9/4 . 4 c4 ε2 t≥t1

t≥1

t≥1

(104)

t≥1

2 −2 ) 2 2 = O(ε 1−e−ε /(2B ) as ε → 0 (since 1 − e−u ∼ u), so cc34(ε) = O(ε−4 ) = O(L1/2 ) = o(L) by the choice εδ = L−1/8 . ε2 ∗ Hence E[τ ] ≤ t1 + o(L) = s L(1 + o(1)) + o(L) = s∗ L(1 + o(1)).

The second sum is a finite constant. For the first: c3 (ε) = KA (K −1)·

Remark 63 (τ < ∞ under every environment). Fix any environment i′ with Alt(i′ ) ̸= ∅ as the truth. The machinery of Subsubsections B.4.1–B.4.5 is environment-generic: forcedexploration coverage and budget (83) are sampling-rule properties; MLE stabilization (Subsubsection B.4.2) uses only 11, 37, 36 for the pairs (i′ , j) — all assumed; the tracking lemma (Subsubsection B.4.3) holds for any frozen target; the good events (Subsubsection B.4.4) involve only the channels. After stabilization, the target is w∗ (i′ ), and by 12 for i′ together with the design convention of Subsubsection B.2.2 (ratio design (72) when gi′ > 0, fixed perturbed design w∗ (i′ ; gi′ +η0 ) otherwise), Lemma 39(iii) gives S ∗ (i′ ) := minj∈Alt(i′ ) Dj (w∗ (i′ ); i′ ) > 0 in all cases. The crossing-time analysis of Theorem 62 then applies verbatim and yields Pi′ (τ < ∞) = 1 (indeed Ei′ [τ ] ≤ s∗ (i′ )L(1 + o(1))). If Alt(i′ ) = ∅, then τ = 1 by the empty-min convention of (76). Hence Pj (τ < ∞) = 1 for every j, which closes the “δcorrect” statement of Theorem 43. This mirrors the Proposition 13 structure of Garivier and Kaufmann (2016) (good event + forced exploration + tail sum), with the tail computed explicitly in (104). B.4.6 Step (iv): Cost Assembly and the LP Identity Lemma 64 (LP identity for the ratio design). Let g > 0 and 12 hold for i∗ . Then min w∈∆(A)

c(w) = Ci∗ , minj∈Alt∗ Dj (w)

the minimum is attained at w∗ (Definition (72)), and therefore P ∗ c(w∗ ) a g(a)wa = = Ci∗ . P S∗ minj∈Alt∗ a wa∗ dja 96

(105)

(106)

Self-Certification of Representation Adequacy

P wa j −1 supw inf j a g(a) da (the (w/g)-weighted max–min form of the main P P j text), and Ci∗ = max j λj : j λj da ≤ g(a) ∀a, λ ≥ 0 (LP dual form). Moreover Ci∗ =



Proof Step 1 (inf ≥ Ci∗ ). Let w ∈ ∆(A) with ρ := minj Dj (w) > 0. Set ma := wa /ρ. P j Then a ma da = Dj (w)/ρ ≥ 1 for all j, so m is feasible for the LP defining Ci∗ , and P a g(a)ma = c(w)/ρ ≥ Ci∗ . (If ρ = 0, the ratio is +∞.) Taking the infimum over w gives inf w minc(w) ≥ Ci∗ . j Dj (w) P Step 2 (min ≤ Ci∗ , attained). The LP is feasible (12)Pand its objective m 7→ a g(a)ma is coercive on the feasible set (g > 0: ∥m∥ → ∞ gm → ∞), and the feasible set P⇒ is closed; hence an LP optimizer m∗ exists with a g(a)m∗a = Ci∗ . Since the constraints P ∗ down strictly reduces the objective; if scale, minj a m∗a dja = 1 (if it were > 1, scaling mP ◦ ∗ any constraint failed, infeasible). Set wa := ma / b m∗b (note m∗ ̸= 0). Then Dj (w◦ ) = P ∗ j a ma da P ≥ P 1m∗ ∗ b mb b b

◦

c(w ) ∗ with equality at the minimizing j, and c(w◦ ) = PCim ∗ , so min D (w ◦ ) = j j b b Ci∗ . Hence the infimum in (105) equals Ci∗ and is attained at w◦ (and w∗ is a maximizer of minj Dj (·) , i.e. a minimizer of the ratio, by Definition (72); the lexicographic selection Φ i∗ = c(·) only picks among optimizers). Step 3 (other LP strong duality for the covering LP (feasible, bounded) gives P P forms). Ci∗ = max{ j λj : j λj dja ≤ g(a) ∀a}. For the (w/g)-weighted form, apply Sion’s miniP wa P j ∗ max theorem to (w, µ) 7→ a g(a) j µj da on ∆(A)×∆(Alt ) (both compact convex, payoff P P wa j a (µ) da = inf µ maxa D̄g(a) with D̄a (µ) := j µj dja . The bilinear continuous): supw inf j a g(a)   a (µ) −1 dual LP value equals supµ mina D̄g(a) = inf µ maxa D̄g(a) (restrict the dual constraint a (µ) P P wa j j 1 j λj da ≤ g(a) to λ = Λµ and optimize Λ). Hence supw inf j a g(a) da = Ci∗ : the two max–min forms coincide in value (their maximizers differ in general — Remark 41).

Theorem 65 (Cost assembly; Theorem 9(b)). Under the standing setup of Subsection B.4, lim sup δ→0

R(π ∗ , Ei∗ ) ≤ Ci∗ . log(1/δ)

(107)

P Proof By Tonelli’s theorem (Subsubsection B.1.1), R(π ∗ , Ei∗ ) = a g(a) E[Na (τ )]; all terms are finite since E[Na (τ )] ≤ E[τ ] < ∞ (Theorem 62) and g is finite. For each a, (94) of Lemma 57 gives p

E[Na (τ )] ≤ wa∗ E[τ ] + (KA − 1) E[T0 ] + 1 + (KA − 1)KA E[τ ] + KA + E[T0 ]. Multiply by g(a) and sum over a: h i p

R(π ∗ , Ei∗ ) ≤ c(w∗ ) E[τ ] + KA gmax (KA −1) E[T0 ]+1 +(KA −1)KA E[τ ]+KA +E[T0 ] . p The bracket is O(1) + O( E[τ ]) = o(L): E[T0 ] < ∞ is independent of δ (Lemma 55; ∗ the p sampling√rule does not depend on δ), and E[τ ] ≤ s L(1 + o(1)) (Theorem 62) gives E[τ ] = O( L). Inserting the stopping-time bound:


c(w∗ ) R(π ∗ , Ei∗ ) ≤ c(w∗ ) s∗ L 1 + o(1) + o(L) = L 1 + o(1) = Ci∗ L 1 + o(1) , ∗ S 97

Huang

where the last equality is Lemma 64, Eq. (106). Remark 66 (Why no uniform-integrability detour is needed). The assembly above controls E|Na (τ√ ) − τ wa∗ | directly by the pointwise tracking error (93) plus Jensen’s inequal√ ity E τ ≤ Eτ ; the only integrability inputs are E[T0 ] < ∞ (polynomial tail (88)) and E[τ ] < ∞ (Theorem 62). Higher moments of τ are also available if wanted: the tail bounds of Theorem 62 give E[τ p ] < ∞ for p < 5/4 as stated (in fact p < 9/4 from the polynomial summand alone, the exponential p summand at fixed δ permitting all orders), and raising the exponent q = 5 in h(n) = q log n/n of Lemma 53 yields any prescribed finite moment at the price of larger constants. Since the Jensen route suffices for the cost assembly, we do not need that strength. Proposition 13 of Garivier and Kaufmann (2016) uses the same good-event/tail-sum structure. B.5 Synthesis Theorem 9 is the conjunction of the results above: part (a) is Theorem 43 together with the a.s.-stopping statement of Remark 63; part (b) is Theorem 65, with the degenerate cases treated in Subsection B.6. Corollary 10 then combines Theorem 9(b) with Theorem 8 of Appendix A. □ B.6 Degenerate Cases B.6.1 Vanishing Costs: gi∗ Has Zeros (Ci∗ = 0 Included) Suppose g(a) = 0 for some actions (13 still holds). The ratio design (72) may then be illdefined (c(w) = 0 denominators). The following perturbation device recovers Theorem 9(b) in full. Definition (perturbed CTS). For η > 0 let g η (a) := g(a) + η, and let πη∗ be the CTS policy of Subsection B.2 with the design map computed from the perturbed costs: wη∗ (i) := arg maxw minj Dj (w; i)/cη (w; i) (well-defined by Lemma 39 since g η > 0), everything else unchanged. Lemma 67 (LP value continuity). Let C(η) be the LP value with costs g η := g + η. Then for every η > 0, Ci∗ ≤ C(η) ≤ Ci∗ + η ∥m∗ ∥1 , (108) where m∗ is an optimizer of the unperturbed LP. P P j Proof The covering LP min{ a g(a)ma : m ≥ 0, a ma da ≥ 1 ∀j} is feasible by (12) and bounded below by 0; by the fundamental theorem of linear a feasible P programming ∗ ∗ LP with finite value attains its optimum, so m exists with P a g(a)ma = Ci∗ . (i) m∗ is feasible for the perturbed LP (same constraints), so C(η) ≤ a g η (a)m∗a = Ci∗ + η∥m∗ ∥1 . (ii) g η ≥ g pointwise, so the perturbed objective dominates the unperturbed one on the common feasible set, giving C(η) ≥ Ci∗ .

98

Self-Certification of Representation Adequacy

Theorem 68 (Degenerate-cost upper bound). Let 11–13, 36, 37 hold and Alt∗ ̸= ∅. Run πη∗δ with ηδ := 1/ log L (and the good-event parameter εδ := 1/(log L)2 ). Then lim sup δ→0

R(πη∗δ , Ei∗ ) ≤ Ci∗ log(1/δ)

(possibly Ci∗ = 0).

(109)

Proof All of Subsections B.3–B.4.5 apply to πη∗ verbatim with g replaced by g η in the design (the channels, LLR, concentration, MLE stabilization, forced exploration, and the tracking lemma never used g; only the design wη∗ and the cost assembly did). Write Sη∗ := minj Dj (wη∗ ), s∗η := 1/Sη∗ . By Lemma 64 with costs g η : cη (wη∗ ) = C(η) Sη∗ . Hence the true cost rate satisfies c(wη∗ ) = cη (wη∗ ) − η = C(η) Sη∗ − η. (110) Also C(η) = cη (wη∗ )/Sη∗ ≥ η/Sη∗ , i.e. s∗η ≤ C(η)/η ≤ (Ci∗ +η∥m∗ ∥1 )/η — and Sη∗ ≥ η/C(η) ≥ η/(Ci∗ + η∥m∗ ∥1 ), which for η ≤ 1 is ≥ η/(Ci∗ + ∥m∗ ∥1 ) =: S η with a constant S > 0. Stopping time. Theorem 62 applied with εδ = (log L)−2 : the inversion factor is 1 −
−1

(log L)−2
= 1 + O εδ /(Sηδ ) = 1 + O (log 2εδ /Sη∗δ = 1 + O (log L)−1 , and the tail sum −1 L)
8 = o(L). Hence (104) is O(ε−4 δ ) = O (log L) E[τ ] ≤ s∗ηδ L 1 + O (log L)−1

+ o(L).

(111)

(Note s∗ηδ ≤ C(ηδ )/ηδ = O(log L), so the additive o(L) terms with η-dependent constants— all polynomial in 1/εδ and 1/Sη∗δ , i.e. polylogarithmic in L—remain o(L).) P ∗ ] + o(L), where the Cost. As in Theorem 65, R = a g(a)E[Na (τ )] ≤ c(wηδ ) E[τ √ √ o(L) tracking-error term is unchanged (it depends on K , E[T ], Eτ only, and Eτ = 0 A √ p O( s∗ηδ L) = O( L log L) = o(L)). Insert (110) and (111): R ≤

C(ηδ )Sη∗δ − ηδ s∗ηδ L 1 + o(1) + o(L) ≤ C(ηδ ) L (1 + o(1)) + o(L),

using C(ηδ )Sη∗δ s∗ηδ = C(ηδ ) and ηδ s∗ηδ ≥ 0. By Lemma 67, C(ηδ ) → Ci∗ (in particular C(ηδ ) ≤ ηδ ∥m∗ ∥1 → 0 when Ci∗ = 0). Remark 69 (The B2 corner). When Ci∗ = 0—equivalently, when a set of zero-cost actions covers all of Alt∗ (boundary case B2 of the main text)—(109) says R = o(log(1/δ)): the log(1/δ) scale is not the right one, matching the companion statement’s “R = O(1)-type” behavior claim up to the exact scale (we prove o(log(1/δ)), which together with the trivial lim inf ≥ 0 of Theorem 8 gives lim Rδ∗ / log(1/δ) = 0 = Ci∗ ). B.6.2 Empty Alternative: Alt(i∗ ) = ∅ b = Θ∗ is 0Then every environment has the same label Θ∗ , and the constant decision Θ correct. In the GLR rule the inner min is over the empty set (= +∞ by convention), so τ = 1 a.s., R ≤ maxa g(a) = O(1), and the LP defining Ci∗ has no constraints, so Ci∗ = 0: both Theorem 9(a) and Theorem 9(b) hold trivially. 99

Huang

B.7 Technical Provenance Component C-tracking sampling rule, forced-exploration curve √ t + K 2 − 2K, coverage/budget/tracking lemmas

GLR stopping rule, threshold β(t, δ) = log(2t(K − 1)/δ), supermartingale + Doob + geometric grid correctness argument Good-event + tail-sum stopping-time analysis Cost-weighted characteristic time supw inf j Dj (w)/c(w) as the optimal design Transportation lemma, kl(δ, 1 − δ) ≥ log(1/(2.4δ)) (lower bound only, cited) Tighter time-uniform thresholds (mixture martingales)

Audenaert entropy bound (continuity of H)

Where here Source template Subsubsection B.2.3, Garivier and Kaufmann Lemmas 49–51, 57 (2016, Lemmas 7–9) (proofs redone self-contained: potential argument for coverage; positive/negative-part bookkeeping for tracking) Subsubsection B.2.4, Garivier and Kaufmann Subsection B.3 (2016, Theorem 10) (same threshold family; our grid computation is spelled out in Lemma 46) Subsubsections B.4.4–Garivier and Kaufmann B.4.5 (2016, Theorem 14 / Proposition 13 template) Subsubsection B.2.2, Kanarios et al. (2024) Lemma 64 (CABAI), cost-weighted T&S variant Subsection B.5 Kaufmann et al. (2016, Lemma 1) not used; Remark 47

Lemma 54

Kaufmann–Koolen 2021 (optional improvement; we keep the conservative closed-form β for simpler analysis) standard (Audenaert 2007)

Statement of contribution. The proof technology above is an adaptation of known templates (the Track-and-Stop machinery of Garivier and Kaufmann (2016); the cost-weighting of Kanarios et al. (2024)). The contribution of the present work is the model —the alternative set defined through adequacy labels Θi and the coupling of observation channels to task losses gi —and the decision-theoretic semantics of the constant C(E, f ), not new concentration or tracking technology. B.8 Proof-Completeness Self-Check Checklist Legend: [done] = proved in full in this section; [cite] = standard result cited with reference; [added] = assumption/convention added beyond the companion statement, explicitly flagged; [dev] = deviation from the companion statement, explicitly flagged. # Technical point 1 Likelihood factorization under adaptive sampling; MLE = KL-projection form

100

Status [done]

Location / remark Subsubsection B.2.1, Eqs. (69)–(70); uses 11

Self-Certification of Representation Adequacy

w∗ (i) well-posed: Φi continuous on compact ∆, sup attained, tie-break fixed Design-map correction: cost-ratio optimizer arg sup inf j Dj /c (= normalized LP solution) instead of the (w/g)-weighted argmax of the companion statement

[done]

Lemma 39

[dev]

4

Forced-exploration coverage Na (t) ≥ f (t) − KA

[done]

5 6

√ Forced-exploration budget F (t) = O( t) exp(LLR) nonnegative (super)martingale under j, any sampling rule

[done] [done]

Remark 41, with explicit 2 × 2 counterexample showing the (w/g)-weighted argmax achieves cost ratio 25/6 ≈ 4.17 > Ci = 30/11 ≈ 2.73; values of the two max–min problems agree (Lemma 64, Step 3) Lemma 49, self-contained potential proof (analogue of Garivier and Kaufmann (2016, Lemma 7)) Lemma 51 Lemma 44, conditional-expectation computation (78) written out; 11 used Lemma 45

2 3

Doob maximal inequality ⇒ fixed-level crossing bound Geometric time grid + series P −β(2k ,δ) = δ/(K − 1) ke

[done]

Error event decomposition + union bound over ≤ K − 1 challengers 10 Per-action i.i.d. structure of observations 11 Uniform empirical-deviation bound with explicit p exponent (h(n) = 5 log n/n, tail O(m−9 )) 12 MLE exact stabilization bıt = i∗ eventually a.s., tail P(T0 > t) ≤ c4 t−9/2 , E[T0 ] < ∞

[done]

7 8

9

[done]

[done] [done] [done]

13 Pairwise distinguishability incl. same-label environments

[added]

14 Full support / bounded LLR increments (|X| ≤ B, KL modulus ω, χ2 bound (87))

[added]

15 Plug-in stability w∗ (bıt ) = w∗ (i∗ ) eventually—no continuity of w∗ (·) needed (finite class) 16 Tracking convergence Na (t)/t → wa∗ a.s. with √ pointwise error ≤ (KA − 1)(T0 + 1) + O( t)

[done]

101

[done]

Lemma 46, full computation; constant accounting in Remark 47 Subsubsection B.3.4 Lemma 52 Lemma 53, full computation Lemma 55 via KL projection√(Di∗ = O(log t) vs Dj ≥ c t); Remark 38 explains why the KL-projection route is used instead of the LLR-sign route Assumption 36, Subsubsection B.1.2; minimal-sufficiency justification given there Assumption 37, Subsubsection B.1.2; used in Lemmas 54, 59 (89), Remark 56 Lemma 57 (analogue of Garivier and Kaufmann (2016, Lemmas 8–9), proved in full)

Huang

17 Per-action LLR concentration, uniform over random counts, explicit constants c1 (ε), c4 2

18 Good-event probability P(Etc ) ≤ c3 (ε)e−c4 ε t with c4 := 1/(KA B 2 ) (exponent ε2 t/(KA B 2 ) from x = εt/KA in Lemma 59) and LLR lower bound (101)–(102) ∗ 19 Stopping-time bound √ E[τ ] ≤ s L(1 + o(1)): inversion ∗ of t(S − 2ε) − C6 t ≥ β(t, δ), tail sums P P −9/4 2 c3 e−c4 ε t = O(ε−4 ), t < ∞, schedule εδ = L−1/8 20 τ < ∞ a.s. under every environment P 21 Cost identity R = g E[Na (τ )] (Tonelli; no Wald needed) 22 E|Na (τ ) − τ wa∗ | = o(L) without heavy UI: pointwise tracking error + Jensen + E[T0 ] < ∞

[done]

[done]

[done]

Theorem 62, all series computed

[done]

Remark 63; uses full pairwise 36 (item 13) Subsubsection B.1.1, Theorem 65 (94), Remark 66; E[τ 2 ] < ∞ available by raising the exponent in Lemma 53 but not needed Lemma 64 (three steps, including Sion + LP duality for the value equivalence) Subsubsection B.6.1, Lemma 67, Theorem 68

[done] [done]

23 LP identity c(w∗ )/ minj Dj (w∗ ) = Ci∗ ; equivalence of [done] LP / cost-ratio / (w/g)-weighted / dual forms (values) 24 Degenerate g with zeros / Ci∗ = 0 (B2): perturbed design πη∗δ , ηδ = 1/ log L, εδ = (log L)−2 ; LP value continuity 25 Alt(i∗ ) = ∅ corner 26 Synthesis with Theorem 8 ⇒ lim Rδ∗ / log(1/δ) = Ci

Lemma 59 (Hoeffding + dyadic-free union over n, AM–GM split of (εn + x)2 ) Lemma 61

[done]

[done] [done]

Subsubsection B.6.2 Subsection B.5; Theorem 8 cited from the companion lower-bound appendix (Appendix A), not re-proved

Explicitly declared residual items (nothing hidden). 1. 36 and 37 are additions to the 11–13 of the companion statement. 36 is necessary for any CTS-type plug-in analysis (counterexample in Subsubsection B.1.2); 37 is a convenience that can be relaxed to per-pair absolute continuity with support-restricted constants (Subsubsection B.1.2). 2. Design-map deviation (item 3) is a correction to the companion statement’s plugin formula; the theorem statements (a), (b) and the constant Ci are unaffected. 3. In the degenerate case (Subsection B.6), the policy is modified to the perturbed design πη∗δ with ηδ → 0; this is a convention addition (Garivier and Kaufmann (2016) handle degeneracies similarly via limiting designs). 4. The tail exponent −9/2 in (88) (via q = 5 in h(n)) limits the moments of T0 to p < 7/2 and the available τ -moments accordingly; any finite moment can be obtained by increasing q. Only E[T0 ] is used. 102

Self-Certification of Representation Adequacy

5. Theorem 8 is used as stated in the companion lower-bound appendix (Appendix A) and is not reproved here.

6. Exact scale in the B2 corner (Ci∗ = 0). This proof delivers R = o(log(1/δ)) (i.e. lim Rδ∗ / log(1/δ) = 0 = Ci , Subsubsection B.6.1); the companion statement √ once claimed O(1). On this instance the t forced exploration of CTS alone conp
tributes Ω log(1/δ) , so an O(1) statement would require modifications such as MLE-triggered adaptive exploration; left as an open remark.

Appendix C. Detailed Constructions for Theorem 6(e) This appendix collects the explicit constructions deferred from the proof of Theorem 6(e) in Section 2.5: the two-point instance showing that the A1 symmetry is indispensable (item (e)(i), Appendix C.1), the general label-prior form and the cyclic three-environment instance showing that the equal label prior is indispensable to the closed-form shape (item (e)(iii), Appendix C.2), and the companion remark separating environment identification from label identification (Appendix C.3). All computations use the notation of Section 2.

C.1 Item (e)(i): dropping A1 symmetry, an explicit two-point instance We give a completely explicit instance in the two-point special case |E| = 2: H = {h0 , h1 }; f constant onto a single representation value, so the unique cell is all of H; A = {0, 1}; ℓ(h, a) = 1[a ̸= h] in both environments; E = {E0 , E1 } with pE0 (h1 ) = 1 and pE1 (h0 ) = pE1 (h1 ) = 21 . Under E0 the cell Bayes action is a = 1 with cell risk 0, so AdeqE0 (f ) = 1; under E1 the within-cell posterior is p = 12 with cell mass m = 1, so by Proposition 2 the f -rule’s per-period aliasing regret is m min(p, 1 − p) = 12 =: r and AdeqE1 (f ) = 0. Take the fallback to be the full-history Bayes rule δfb (h) = h: matched under E1 with regret 0, as A1 requires. But under E0 , h = h1 almost surely, so the fallback plays a = 1 almost surely—exactly the f -rule’s cell Bayes action—and the mismatch direction “wrongly using the fallback under E0 ” costs 0, not r: the symmetric clause fails, and “always fallback, never certify” has total regret identically 0 under both environments. Yet with the one-step transcript equal to the observed history, P01 is the point mass on h1 and P11 = ( 21 , 12 ), so TV(P01 , P11 ) = 12 < 1 and (28) prescribes the strictly positive value T4r for the optimum: the formula no longer describes the optimum, and the threshold of (b) collapses with it, there being nothing to buy when certification itself is unnecessary. (Consistently, (26) here evaluates to 0, since min(r0 dP̄1t , r1 dP̄0t ) = 0 when r0 = 0.) The counterexample bites exactly when the internal channel is not perfectly distinguishing. 103

Huang

C.2 Item (e)(iii): general label priors and the cyclic instance Under a general label prior π, the Bayes error of a certifier C with certification event A = {C = 1} expands pointwise as Z h i t c t π0 p̄0 1{x ∈ A} + π1 p̄1 1{x ∈ / A} dµ errπ (C) = π0 P̄0 (A) + π1 P̄1 (A ) = Z (112)
≥ min π0 p̄0 , π1 p̄1 dµ, because at each x the integrand equals one of the two weighted densities, each at least their minimum; equality holds for the weighted likelihood-ratio test A∗ = {π1 p̄1 > π0 p̄0 }, which selects the minimum everywhere (the pointwise argument is identical to Step 4 of the proof of Theorem 3). Under A1 the expected total regret is T r errπ (C), so the optimal internal-only benchmark and the perfect-verify threshold are Z
∗ ∗ Rint = c = T r min π0 dP̄0t , π1 dP̄1t , (113) still a closed form, but one in which the prior sits inside the minimum. The equal label prior is exactly Rthe case in which it cancels: substituting π0 = π1 = 12 and the Scheffé identity gives 12 min(p̄0 , p̄1 ) dµ = (1 − TV)/2, the shape of (a) and (b). That the shape, not merely the value, is genuinely lost is shown by the cyclic three-environment instance on the transcript space {0, 1, 2} with


pE0 = 21 , 21 , 0 , pE1 = 0, 12 , 12 , pE2 = 12 , 0, 12 , (114) equal environment prior q = ( 13 , 31 , 13 ), and labels Θ = (0, 0, 1), so π = ( 23 , 13 ). The mixtures are P̄0 = ( 41 , 12 , 41 ) and P̄1 = ( 12 , 0, 12 ), with TV(P̄0 , P̄1 ) = 21 , so the equal-prior shape would prescribe (1 − TV)/2 = 14
; but the weighted pointwise minimum is min( 23 · 14 , 13 · 12 ), min( 32 · 1 1 2 1 1 1 1 1 1 1 2 , 3 · 0), min( 3 · 4 , 3 · 2 ) = ( 6 , 0, 6 ), integrating to 3 ̸= 4 . The weighted likelihood-ratio 1 test attains 3 (it declares Θ = 0 at x = 1 and breaks the ties at x ∈ {0, 2} arbitrarily). Hence no expression of the shape (1 − TV)/2 describes the optimum under this non-equal label prior, while (113) does: the equal label prior is indispensable to the shape of the closed forms in (a) and (b). C.3 Environment identity versus label identity Remark (Environment identity versus label identity). Identifying which environment obtains and identifying the adequacy label are different testing problems already at |E| = 3. On the R cyclic class (114) with equal environment prior, the multi-class Bayes error 1 is 1 − 3 max{p0 , p1 , p2 } dµ (at each x the indicator sum collapses to the single term pδ(x) (x) ≤ maxj pj (x), with equality for any pointwise-maximum rule); the pointwise maximum is 12 at all three transcript points, so environment identification has Bayes error 12 , while every pairwise total variation equals 12 and any two-point formula predicts 41 . The label test instead aggregates the class into the two mixtures P̄0t , P̄1t and retains the binary closed form at every finite |E|: on the same class with prior q = ( 14 , 14 , 12 ) inducing equal label priors, Theorem 3 prescribes (1 − TV)/2 = 14 , attained by the likelihood-ratio rule. The 104

Self-Certification of Representation Adequacy

two problems return genuinely different answers ( 12 versus 14 ) on the same class, which is why the framework fixes its test object: adequacy label, not environment identity. Unlike Fano-type multi-class identification, where the error depends on the full joint structure of K hypotheses (Fano, 1961), label aggregation restores the two-density situation in which a single scalar—the mixture TV—is exactly informative.

References Karl J. Åström. Optimal control of Markov processes with incomplete state information. Journal of Mathematical Analysis and Applications, 10(1):174–205, 1965. doi: 10.1016/ 0022-247X(65)90154-X. Yaakov Bar-Shalom and Elliot Tse. Dual effect, certainty equivalence, and separation in stochastic control. IEEE Transactions on Automatic Control, 19(5):494–500, 1974. doi: 10.1109/TAC.1974.1100635. Gábor Bartók, Dean P. Foster, Dávid Pál, Alexander Rakhlin, and Csaba Szepesvári. Partial monitoring—Classification, regret bounds, and algorithms. Mathematics of Operations Research, 39(4):967–997, 2014. doi: 10.1287/moor.2014.0663. Michael Timothy Bennett. Regret is weighted forgetting. Preprint, under review, 2026. URL https://www.preprints.org/manuscript/202603.1546. preprints.org, v2 July 2026; under double-blind review at TMLR. David Blackwell. Comparison of experiments. In Jerzy Neyman, editor, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, pages 93– 102, Berkeley and Los Angeles, 1951. University of California Press. URL https:// projecteuclid.org/euclid.bsmsp/1200500222. David Blackwell. Equivalent comparisons of experiments. The Annals of Mathematical Statistics, 24(2):265–272, 1953. doi: 10.1214/aoms/1177729032. David H. Blackwell and Meyer A. Girshick. Theory of Games and Statistical Decisions. Wiley Publications in Statistics. John Wiley & Sons, New York, 1954. Reprinted by Dover Publications, New York, 1979. Nicolò Cesa-Bianchi, Gábor Lugosi, and Gilles Stoltz. Regret minimization under partial monitoring. Mathematics of Operations Research, 31(3):562–580, 2006. doi: 10.1287/ moor.1060.0206. Herman Chernoff. Sequential design of experiments. The Annals of Mathematical Statistics, 30(3):755–770, 1959. doi: 10.1214/aoms/1177706205. Richard Combes, Mohammad Sadegh Talebi, Alexandre Proutière, and Marc Lelarge. Combinatorial bandits revisited. In Advances in Neural Information Processing Systems 28 (NIPS 2015), pages 2116–2124. Curran Associates, 2015. Richard Combes, Stefan Magureanu, and Alexandre Proutière. Minimal exploration in structured stochastic bandits. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 1763–1771. Curran Associates, 2017. Rémy Degenne and Wouter M. Koolen. Pure exploration with multiple correct answers. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019), pages 14564– 14573. Curran Associates, 2019. arXiv:1902.03475. 105

Huang

Morris H. DeGroot. Optimal Statistical Decisions. McGraw-Hill, New York, 1970. Reprinted in the Wiley Classics Library, Wiley-Interscience, Hoboken, NJ, 2004, with a foreword by J. B. Kadane. Luc Devroye, László Györfi, and Gábor Lugosi. A Probabilistic Theory of Pattern Recognition, volume 31 of Stochastic Modelling and Applied Probability. Springer, New York, 1996. doi: 10.1007/978-1-4612-0711-5. Robert M. Fano. Transmission of Information: A Statistical Theory of Communications. MIT Press, Cambridge, MA, 1961. A. A. Feldbaum. Dual control theory problems. IFAC Proceedings Volumes, 1(2):541–550, 1963. doi: 10.1016/S1474-6670(17)69687-3. A. A. Feldbaum. Optimal Control Systems. Academic Press, New York, 1965. Translated from the Russian by A. Kraiman; Russian original: Fizmatgiz, Moscow, 1963. Aurélien Garivier and Emilie Kaufmann. Optimal best arm identification with fixed confidence. In Proceedings of the 29th Conference on Learning Theory (COLT 2016), volume 49 of Proceedings of Machine Learning Research, pages 998–1027. PMLR, 2016. URL http://proceedings.mlr.press/v49/garivier16a.html. arXiv:1602.04589. Robert Givan, Thomas Dean, and Matthew Greig. Equivalence notions and model minimization in Markov decision processes. Artificial Intelligence, 147(1–2):163–223, 2003. doi: 10.1016/S0004-3702(02)00376-4. Todd L. Graves and Tze Leung Lai. Asymptotically efficient adaptive choice of control laws in controlled Markov chains. SIAM Journal on Control and Optimization, 35(3):715–743, 1997. Ronald A. Howard. Information value theory. IEEE Transactions on Systems Science and Cybernetics, SSC-2(1):22–26, 1966. doi: 10.1109/TSSC.1966.300074. Garud N. Iyengar. Robust dynamic programming. Mathematics of Operations Research, 30 (2):257–280, 2005. doi: 10.1287/moor.1040.0129. Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2):99–134, 1998. doi: 10.1016/S0004-3702(98)00023-X. Kellen Kanarios, Qining Zhang, and Lei Ying. Cost aware best arm identification. In Reinforcement Learning Conference (RLC 2024), 2024. URL https://arxiv.org/abs/ 2402.16710. arXiv:2402.16710. Emilie Kaufmann and Wouter M. Koolen. Mixture martingales revisited with applications to sequential tests and confidence intervals. Journal of Machine Learning Research, 22 (246):1–44, 2021. URL http://jmlr.org/papers/v22/18-798.html. Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On the complexity of best-arm identification in multi-armed bandit models. Journal of Machine Learning Research, 17 (1):1–42, 2016. URL http://jmlr.org/papers/v17/kaufmann16a.html. Vikram Krishnamurthy. Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing. Cambridge University Press, Cambridge, 2016. doi: 10.1017/ CBO9781316471104. 106

Self-Certification of Representation Adequacy

Vikram Krishnamurthy and Bo Wahlberg. Partially observed Markov decision process multiarmed bandits—Structural results. Mathematics of Operations Research, 34(2):287– 302, 2009. Lucien Le Cam. Sufficiency and approximate sufficiency. The Annals of Mathematical Statistics, 35(4):1419–1455, 1964. doi: 10.1214/aoms/1177700372. Lucien Le Cam. Asymptotic Methods in Statistical Decision Theory. Springer Series in Statistics. Springer-Verlag, New York, 1986. doi: 10.1007/978-1-4612-4946-7. Lihong Li, Thomas J. Walsh, and Michael L. Littman. Towards a unified theory of state abstraction for MDPs. In Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics (ISAIM 2006), Fort Lauderdale, FL, 2006. URL http: //anytime.cs.umass.edu/aimath06/proceedings/P21.pdf. Jianhua Lin. Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1):145–151, 1991. doi: 10.1109/18.61115. Michael L. Littman, Richard S. Sutton, and Satinder Singh. Predictive representations of state. In Advances in Neural Information Processing Systems 14 (NIPS 2001), pages 1555–1561. MIT Press, 2002. URL https://proceedings.neurips.cc/paper/2001/ hash/1e4d36177d71bbb3558e43af9577d70e-Abstract.html. Mohammad Naghshvar and Tara Javidi. Active sequential hypothesis testing. The Annals of Statistics, 41(6):2703–2738, 2013. doi: 10.1214/13-AOS1144. Sirin Nitinawarat and Venugopal V. Veeravalli. Controlled sensing for sequential multihypothesis testing with controlled Markovian observations and non-uniform control cost. Sequential Analysis, 34(1):1–24, 2015. doi: 10.1080/07474946.2014.961864. Sirin Nitinawarat, George K. Atia, and Venugopal V. Veeravalli. Controlled sensing for multihypothesis testing. IEEE Transactions on Automatic Control, 58(10):2451–2464, 2013. Daniel Russo and Benjamin Van Roy. An information-theoretic analysis of Thompson sampling. Journal of Machine Learning Research, 17(68):1–30, 2016. Daniel Russo and Benjamin Van Roy. Learning to optimize via information-directed sampling. Operations Research, 66(1):230–252, 2018. doi: 10.1287/opre.2017.1663. Daniel Russo and Benjamin Van Roy. Satisficing in time-sensitive bandit learning. Mathematics of Operations Research, 47(4):2815–2839, 2022. Aldo Rustichini. Minimizing regret: The general case. Games and Economic Behavior, 29 (1–2):224–243, 1999. doi: 10.1006/game.1998.0690. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 2 edition, 2018. Elliot Tse and Yaakov Bar-Shalom. Generalized certainty equivalence and dual effect in stochastic control. IEEE Transactions on Automatic Control, 20(6):817–819, 1975. doi: 10.1109/TAC.1975.1101108. Alexandre B. Tsybakov. Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, New York, 2009. doi: 10.1007/978-0-387-79052-7. Gerhard Widmer and Miroslav Kubat. Learning in the presence of concept drift and hidden contexts. Machine Learning, 23(1):69–101, 1996. doi: 10.1007/BF00116900. 107

Huang

Hans S. Witsenhausen. Separation of estimation and control for discrete time systems. Proceedings of the IEEE, 59(11):1557–1566, 1971. doi: 10.1109/PROC.1971.8488. J. Yame. Dual adaptive control of stochastic systems via information theory. In Proceedings of the 26th IEEE Conference on Decision and Control, pages 316–320, 1987. doi: 10.1109/ CDC.1987.272811.

108

Related documents

• Self-Certification of Representation Adequacy: Sequential Certification at Minimum Task Loss#1070903
• Self-Supervised Representation Learning as Mutual Information Maximization#1131415
• 7 U.S.C. § 4505 — Certification of organizations#560607
• Problems of representation I: nature and role#764658
• Which Tasks Survive Self-Supervised Learning?#1142994