arXiv CSopen access

Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment

Position: A Three-Layer Probabilistic Assume–Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment

arXiv:2605.18672v1 [cs.AI] 18 May 2026

Saddek Bensalem1

Yi Dong2

Dejan Nickovic4 1

CSX-AI, France 4

2

Martin Fränzle3

Ayoub Nouri5

University of Liverpool, UK

Xiaowei Huang2

Rajarshi Roy2 3

Janis Kröger3

Changshun Wu5

Carl von Ossietzky Universität Oldenburg, Germany

Austrian Institute of Technology, Austria

5

Université Grenoble Alpes, France

Abstract This position paper argues that enforcing LLM agent safety within a single abstraction layer is not merely suboptimal but categorically insufficient for deployed LLM agents—a structural consequence of how agent execution works, not a contingent limitation of current systems. The three dimensions that jointly constitute safe operation—semantic intent and policy compliance, environmental validity, and dynamical feasibility—each depend on a strictly distinct information set that becomes available at a different stage of execution. No single guardrail can certify all three. We argue the community must respond with a contract-based architecture in which each safety dimension is enforced by an independently certified layer whose probabilistic guarantee satisfies the assumption of the next. We sketch such an architecture and derive the compositional system-level safety bounds it admits via the chain rule of probability. Three open problems stand between this and a deployable standard: bound estimation from non-i.i.d. traces, graceful degradation of contracts under deployment drift, and extension to multi-agent settings—the most important unfinished business in LLM agent runtime assurance.

1

Introduction

The deployment of LLM agents in safety-critical settings poses a qualitatively new challenge for runtime assurance. An LLM agent uses a language model as its reasoning engine to implement task-oriented functionality: it receives instructions from human users, conducts multi-step reasoning and planning, and interacts with the environment through tool invocation or physical execution. Unlike conventional software, whose behaviour is formally specified, and unlike static language models, whose outputs do not act on the world, LLM agents combine non-deterministic reasoning with closed-loop execution in which generated plans influence the world and future observations. We argue a strong claim: safe deployment of LLM agents is structurally inadequate under any single-layer enforcement design, no matter how systematically engineered [17]. The three dimensions of safe operation (semantic intent alignment and policy compliance, environmental validity, and dynamical feasibility) become verifiable only at distinct stages of execution, so any architecture that collapses them into a single abstraction necessarily leaves at least one dimension uncertified. What is required is not a stronger guardrail, but a staged, contract-based architecture in which three independently certified layers (user, operational, functional) compose sequentially through assume–guarantee reasoning. The limitation is not empirical but structural: no amount of prompt engineering, model fine-tuning, or execution monitoring confined to a single abstraction can certify properties whose truth depends on information becoming available only later in the execution chain. Preprint.

LLMs are vulnerable to hallucinations, prompt injection, distributional instability, and adversarial manipulation [27]. When embedded in agents, these vulnerabilities are amplified [21]: the consequences of an incorrect or unsafe generation are no longer confined to textual output but may propagate to physical actions, API calls, or system state transitions. The problem of safety in LLM agents is therefore not merely one of content filtering but of runtime assurance in dynamic and uncertain environments. Because the space of possible LLM agent failures is open-ended and cannot be enumerated at design time, empirical evaluation alone cannot establish the systematic assurance that safety-critical deployment demands. Throughout this paper, guarantee refers to probabilistic assume–guarantee contracts—bounds on satisfaction probability rather than deterministic proofs—a distinction that matters for interpreting the framework’s contributions against the broader literature on formal verification of agents [38, 29]. That this gap has practical consequences is visible in the benchmark record. Of sixteen popular agents evaluated on AgentSafetyBench, none achieves a safety score above 60% [64], with behavioural safety (30.4%) significantly lagging behind content safety (68.4%). While these failures cannot be attributed to a single cause—such as weak guardrails or distributional mismatch—their pattern is instructive: failures concentrate in environmental and physical dimensions, beyond the reach of purely semantic controls. A similar limitation appears from a security perspective. Single-layer defences leave critical gaps, as reflected in attack success rates exceeding 84% on the Agent Security Bench. Taken together, these observations do not constitute proof of the structural argument developed below, but they are consistent with it: both safety and security failures emerge precisely where semantic-layer protections cease to apply. This convergence motivates the need for a multi-layer architecture, as developed in Section 2. Recent work has begun to cross layer boundaries: VeriGuard [38] achieves near-zero attack success rates by combining offline policy synthesis and formal verification with lightweight runtime monitoring, and Agent-C [29] reaches perfect temporal-constraint conformance via SMT-enforced generation. Yet both remain scoped to semantic intent and do not address environmental validity or dynamical feasibility. Further complementary benchmarks, AgentHarm [3] and BAD-ACTS [41], have confirmed that agentic frameworks dramatically amplify misuse risk and that only multi-step monitoring protocols begin to render deployment plausible—consistent with the layered structure advocated here. While security and its limitations are briefly mentioned, this paper primarily focus on safety aspects. The structure of the paper is as follows. Section 2 develops the structural argument formally. Section 3 presents the three-layer instantiation and its safety bounds. Section 4 identifies four open problems that stand between that architecture and a deployable standard. It discusses alternative views in Section 5 and concludes in Section 6.

2

From Safety Dimensions to Contract-Based Design

This section surveys existing approaches to LLM agent safety, identifies the structural characteristic that motivates our framework, and argues for the three-layer architecture as the minimal sufficient response. We focus on safety mechanisms operating externally to the LLM weights, excluding fine-tuning approaches such as RLHF [13], SFT [42], and DPO [44], which require parameter modification and are often impractical when model weights are inaccessible. Instead, we focus on external guardrails that enforce safe behaviour at runtime, independently of the underlying model. Unlike static text generation, LLM agents perform multi-step reasoning and planning, invoke tools, and interact with dynamic environments, causing errors to propagate beyond linguistic outputs to external systems, physical processes, or long-term state trajectories [21]. Existing Approaches and Their Limits The largest body of work targets semantic intent alignment and policy compliance. System-level approaches enforce constraints externally. AgentSpec [57] introduces a lightweight domain-specific language for runtime constraint enforcement, preventing unsafe executions in over 90% of code-agent cases while remaining computationally lightweight. Agent-C [29] encodes temporal safety requirements as first-order logic constraints enforced via SMT solving, achieving perfect conformance on retail and airline benchmarks. ShieldAgent [12] structures verifiable rules extracted from policy documents into probabilistic rule circuits, achieving 90.1% rule recall. These are genuine advances: they demonstrate that specification-driven runtime guardrails are both practical and effective at the user layer. The most systematic proposal within 2

this single-layer paradigm is that of Dong et al. [17], who advocate a rigorous requirement-driven design process—covering neural-symbolic implementation, statistical certification, and a full systems development lifecycle—applied to a guardrail conceived as an input-output filter on the LLM. We do not dispute the value of this engineering rigour; we argue that no single-layer design, however carefully constructed, can certify all three safety dimensions, because the information required to do so becomes available only at strictly distinct stages of execution. A second strand addresses the execution environment. Out-of-distribution detection [48, 63] identifies when model inputs deviate from training distribution, though without determining whether the current world lies within the Operational Design Domain (ODD) [28, 50, 10] under which system-level guarantees were derived. Inner Monologue [26] feeds environmental feedback back into the LLM to support replanning, but reactively, once execution has begun. RoboGuard [47] and Safety Chip [62] use world-model information to instantiate action-level constraints via temporal logic—instrumentally, rather than as an independent determination of whether execution is authorised at all given the current world state, i.e., whether the system is operating within its ODD. A third strand targets low-level execution safety. Specification-based runtime monitoring [6] tracks system trajectories against temporal logic specifications. Control barrier functions [2] enforce forward invariance of a safe set with minimal intervention, though their guarantees degrade when environmental assumptions break at runtime. Shielding in reinforcement learning [1, 61] synthesises correctors from temporal logic specifications; both absolute and probabilistic variants require knowledge of MDP safety dynamics—an assumption LLM agent environments structurally violate. Recent work integrating CBFs with LLM planners [30, 59, 65] demonstrates that functional-layer enforcement is independently necessary and cannot be subsumed by user-level guardrails. Recent efforts attempt broader coverage. Agent Behavioral Contracts [7] introduce probabilistic (p, δ, k)-satisfaction notions—meaning a contract holds with probability at least p, confidence 1−δ, over windows of k steps—with a Drift Bounds Theorem, detecting 5.2–6.8 soft violations per session that uncontracted baselines miss entirely. PRO2 GUARD [58] extends enforcement into the probabilistic domain via learned Discrete-Time Markov Chains, achieving PAC-correctness on risk estimates. Shamsujjoha et al. [52] contribute a systematic taxonomy of runtime guardrails for Foundation Model based agents, grounded in a systematic literature review and structured around three dimensions: quality attributes, pipeline stages, and artefacts. This is the closest antecedent to the present work and clearly motivates multi-layered thinking. The key difference is organisational principle: their taxonomy is artefact-driven, decomposing guardrail design by which pipeline objects (goals, plans, tools, outputs) are being guarded at each stage. Our architecture is information-driven, deriving layer boundaries from a structural argument about when each category of safety claim can first be verified—given user instructions, policies and rules, before any world observation (User Layer), given current sensor data (Operational Layer), or continuously during actuation (Functional Layer). This distinction is not merely taxonomic: it grounds the contract-chain structure and the compositional probability bounds of Section 3, which depend on the sequential availability of information rather than on the artefact being inspected. An artefact-driven decomposition cannot ensure that certifications relying on temporally distinct information sets remain consistent. This breaks the assumption–guarantee chain, as guarantees are no longer established against the correct information domain. The information-driven decomposition avoids this issue by construction and enables compositional safety bounds 3.2. Contract-Based Design as the Unifying Principle The structural characteristic identified above demands a response with three properties simultaneously, each corresponding to a structural failure mode of single-layer enforcement: it must reason across heterogeneous verification methods without forcing them into a common formalism; it must compose guarantees across layers whose dynamics are non-deterministic and non-stationary; and it must yield modular statistical bounds certifiable layer by layer. We argue that probabilistic assume–guarantee (A/G) contracts [16, 15, 24, 8] provide exactly this foundation. Heterogeneous verification methods are accommodated by the contract structure itself. Each safety component is specified by an assumption on its inputs and a guarantee on its outputs, scoped to its own abstraction level and information set. Semantic, operational, and dynamical dimensions need not be unified into a single model [22]: each layer carries an independently verifiable contract, and the heterogeneity of verification methods and specification formalisms across layers is a feature, not a limitation. This matters in practice: existing CPS assumptions are typically informal and split

3

between deterministic and probabilistic formalisations [33], a fragmentation that worsens when one layer is an LLM—a gap we begin to address here. Principled composition across non-stationary layers follows from the quantitative semantics of contracts. Satisfaction is probabilistic: each contract holds with some probability, meaning that executions satisfying the assumption also satisfy the guarantee with that probability. Sequential composition is then well-defined: the chain rule decomposes system-level safety into a product of conditional probabilities aligned with the architectural stages [16], and the Fréchet–Bonferroni bound yields a non-trivial lower bound from marginals alone whenever individual probabilities are sufficiently high. Critically, this rule does not presuppose deterministic or stationary subsystem dynamics, a necessary distinction from existing CPS contract frameworks—which assume well-characterised noise models— rather than the discrete token sampling and context-window non-determinism characteristic of LLMs. Modular statistical guarantees follow from the same decomposition: A/G contracts compose locally certifiable bounds into a system-level guarantee without global analysis, complementing PAC-style statistical interpretability as a refinement adapted to the non-stationary, finite-trace setting of LLM agents; the estimation challenges this raises are developed in Section 3.2. Within each layer, neural-symbolic methods should be adopted [12, 29]: the symbolic component renders contract assumptions and guarantees formally checkable, while the neural component handles the perceptual and linguistic complexity that purely symbolic approaches cannot scale to. The Three-Layer Architecture Is the Necessary Response Let IU , IO , and IF denote the three information sets available respectively (i) prior to world observation, (ii) after world-state estimation but before actuation, and (iii) during control-loop execution. IU contains user intent, policy, and role metadata but no sensor-derived state. IO contains the estimated world state ŵ and ODD configuration but lacks real-time dynamical trajectory information. IF contains full state trajectories and control inputs but operates after both intent validation and ODD authorisation. These sets are strictly ordered, IU ̸⊆ IO ̸⊆ IF (Appendix B), and mutually non-substitutable as certification domains. Let ΦU (e), ΦO (e), and ΦF (e) denote the Boolean safety predicates for semantic, operational, and dynamical safety respectively (formally defined in Appendix B). Because ΦU , ΦO , and ΦF depend on strictly distinct information sets that become available at different stages, any architecture that collapses them into fewer than three independently certified stages must either violate the desired semantic, operational or dynamical safety (see the desiderata D1–D3 of Appendix B) or implicitly reconstruct the three-stage structure within a single component (Proposition 1). The minimality of three layers is therefore not a design choice but a consequence of the temporal ordering τU < τO < τF of certifiable information, where τi denotes the earliest time at which Φi can be certified: fewer layers break the contract chain or hide its boundaries; additional layers may refine it but cannot reduce this minimum (Appendix B, Corollary 1).

3

Three-Layer Framework

The three-layer framework, illustrated in Figure 1, instantiates the contract-based design of Section 2. A plan p passes sequentially through three assurance barriers before and during execution. Each barrier i ∈ {U, O, F } carries a probabilistic assume–guarantee (A/G) contract Σi = (Ai , Γi ): if the environment satisfies assumption Ai , the layer guarantees property Γi with probability pi = Pr(Γi ) [16]. These probabilistic assume-guarantee frameworks or their refinements addressing conditional distributions [9] facilitate the relevant modular reasoning. The guarantees chain, derived from the temporal ordering as ΓU ⇒ AO and ΓO ⇒ AF , enables a modular decomposition of the system-level safety probability via the chain rule introduced in Section 3.2. Running example. A caregiver instructs a service robot (LLM-orchestrated): “Visit Rooms 12, 15, and 18 before 15:00; check hydration and wellbeing; bring water if needed; do not disturb during rest periods.” The LLM produces plan p—a sequence of visit, assessment, and conditional-delivery actions—which passes through the three layers as developed below (see Figure 1). 3.1

The Three Layers

User Assurance Layer (ΣU ). The first barrier validates p against intent, policy, and ethics before any world observation is made, across three orthogonal dimensions that together span the space of pre-execution failures [21, 52]: (i) cognitive alignment—does the plan faithfully realise the user’s 4

plan p (LLM output)

IU Intent · policy · role metadata no world observation sO

User assurance layer ΣU

Semantic

AU : authorised request ΓU : intent · policy · ethics ✓

Cognitive · regulatory ethical alignment

ΓU ⇒ AO

IO Sensor data ξ · world state ŵ ODD spec · envelope E sF

IF

Running example Caregiver instructs robot: “Visit Rooms 12, 15, 18; check hydration; bring water if needed; no disturbance at rest.” Blocks: rest-hour entry, camera-active routing

(filtered plan descends)

Operational Assurance layer ΣO

Operational

AO : ΓU holds (upstream filter) ΓO : w ∈ WODD · envelope E

ODD membership autonomy level

ΓO ⇒ AF

Trajectory x(t) · control u(t) system dynamics (f, g)

Dimension certified

plan infeasible → recompute p′

Information available

safety signals (bottom-up)

revised plan p′

Layer & A/G contract

World-state check: Room 15: clinical procedure → sub-plan blocked Corridor to R12: under-lit → valid at v ≤ vmax

ΓO violated

Runtime enforcement: Resident enters corridor → hold (d < dmin ) R18: furniture rearranged → escalate, update map Handover: CBF constrains arm force f ≤ fmax

ΓF violated

(authorised execution descends)

Functional Assurance layer ΣF

Dynamical

AF : ΓO holds (upstream filter) ΓF : no collision · f ≤ fmax

CBF · STL monitor safe-stop

System-level safety bound (B4) Pr(safe) ≥ pU · pO|U · pF |OU pO|U , pF |OU independently estimable (compositional certification via chain rule)

Physical world / actuator

execution time pre-execution (IU only)

pre- & during (IO added)

control-loop runtime (IF added)

IU ̸⊆ IO ̸⊆ IF — each layer certifies only what its information permits

User Layer

Operational Layer

Functional Layer

System output

Running example

contract chain / safety signal plan recomputation p′

Figure 1: Three-layer probabilistic assume–guarantee architecture with running example (right column). Each layer Σi is certified against the information set Ii first available at its execution stage. Forward contract propagation is shown by dashed downward arrows (ΓU ⇒ AO , ΓO ⇒ AF ). Left margin: upward safety signals (sF → sO → sU ). Right margin: when ΓO or ΓF is violated the User Layer recomputes a revised plan p′ , making the architecture a live bidirectional assurance loop. intent, neither under- nor over-delivering? [57]; (ii) regulation alignment—does it conform to domainspecific rules and policies? [29, 57]; and (iii) ethical alignment—does it respect ethical constraints such as prohibiting cameras in private spaces? [4, 12]. Authorisation of the issuing user is verified in parallel. A plan failing any check is blocked or, where recoverable, returned for clarification. Beyond gating, the layer derives quantitative constraints—bounds capsU on continuous variables such as speed, force, and proximity, and exclusions zonesU over spatial regions, time windows, and task types—from user roles and ethical rules, passing them downstream. Assuming AU (authorised request), the layer guarantees ΓU : the emitted plan targets only authorised locations, encodes conditional actions correctly, and contains no policy or ethical violations; ΓU constitutes AO . Example. The layer rejects plans omitting the conditional water-delivery logic (cognitive), scheduling room entry during rest hours without override (regulation), or routing through a bathroom with cameras active (ethical). Key open problems include automated formalisation of informal intent and domain policies into verifiable representations—initial attempts in ShieldAgent [12] and Agent-C [29] but not yet closed. Operational Assurance Layer (ΣO ). The second barrier asks not “what should the agent do?” but “is this a world in which we are certified to act, and under what degree of autonomy?” Let W denote the space of all possible world states reachable by the agent during deployment. The Operational Design Domain (ODD) is a certified subset WODD ⊆ W encoding five independently certifiable dimensions: physical configuration (geometry, obstacles, dynamics), perceptual configuration (sensor availability, visibility), contextual configuration (time of day, occupancy, access restrictions), governance configuration (authorisation regimes, policy zones), and social or normative context (restricted roles, privacy constraints). The first three follow established ODD formalisms [28]; the latter two extend them to the institutional embedding of LLM agents. Execution outside WODD invalidates all downstream guarantees regardless of physical capability. 5

Beyond ODD membership, the Operational Layer governs the agent’s autonomy envelope: the degree of freedom under which the plan may be executed in the current operational context. We model this envelope as E(ŵ) = (L, Π, H), where L denotes the autonomy level, Π the permitted action and tool set, and H the maximum execution horizon before checkpoint. While ODD validity determines whether execution is authorised at all, the autonomy envelope determines how much autonomy is granted within the valid domain. The layer ingests sensor data ξ, computes world-state estimate ŵ, and issues a deterministic validity verdict. Determinism preserves audit trails and compositional invariants [16]; probabilistic uncertainty is handled upstream, but the verdict itself is binary. Note that, while the verdict is binary for every input, a probabilistic guarantee is needed when dealing with a population of inputs. When the ODD is invalid or degraded, a fallback policy triggers plan restriction or human takeover. Assuming AO (i.e., ΓU holds), the layer guarantees ΓO : (i) execution remains within WODD , (ii) all ODD-invalid subplans are blocked, and (iii) autonomy-envelope updates are conservative with respect to operational degradation (i.e., autonomy reduction, restriction of permitted actions, or activation of a Minimal Risk Condition (MRC) [50] when required). This guarantee constitutes AF for the Functional Layer. Example. Room 15 is restricted (clinical procedure): that sub-plan is blocked. The corridor to Room 12 is under-lit but within the ODD subject to v ≤ vmax . Key open problems include maintaining guarantees under distributional shift: when ŵ drifts from the distribution under which WODD was certified, methods such as [66] may be needed. Functional Assurance Layer (ΣF ). The third barrier enforces how the system executes at controlloop frequency. Three complementary mechanisms share the load. Specification-based runtime monitoring [6] tracks the trajectory against safety specifications, producing robustness margins ρF (t)—the signal temporal logic (STL) robustness degree [6] of the current trajectory against the safety specification—and triggering intervention when a violation is detected or anticipated. Control barrier functions (CBFs) [2] project any candidate control command udes (t) onto the admissible safe set, correcting it minimally; recent work confirms CBF quadratic programs embed inside LLM planning loops without sacrificing task completion [30, 65], and probabilistic extensions now bound safety failure probability for learned barriers [53, 37]. Simulation-based synthesis [1, 61] pre-computes safe envelopes via world models when the environment is large or unknown; CBF projection takes precedence when the two conflict, as it provides a certified instantaneous correction independent of model accuracy. Assuming AF (ΓO holds: w ∈ WODD , v ≤ vmax ), the layer guarantees ΓF : no collision occurs, all physical interactions satisfy f ≤ fmax , and any violation triggers corrective action or safe-stop. Safety signals si are structured status reports emitted by each layer upon constraint violation or execution failure, indicating whether operation is nominal or requires upstream intervention. They propagate bottom-up: the functional safety signal sF is reported to the Operational Layer, which propagates sO to the User Layer [22], enabling plan recomputation when execution cannot complete safely. The contract chain is thus a live, bidirectional assurance loop. Example. A resident entering the corridor triggers a hold; a CBF constrains arm force during the handover; rearranged furniture in Room 18 causes escalation with an updated obstacle map, exercising the bottom-up channel. Key open problems include real-time safety under non-stationary dynamics [6] and mid-execution adaptation of safety envelopes when updated caps or zones arrive from upper layers. 3.2

End-to-End Safety Guarantee

Since ΓU ⇒ AO and ΓO ⇒ AF , the three contracts compose sequentially [16, 22]. Let Fi = ¬Γi be the failure event of layer i. The system-level safety probability Pr(system safe) = Pr(ΓU ∩ΓO ∩ΓF ): admits four characterisations of increasing precision: Pr(safe) ≥ max(0, pU + pO + pF − 2) P Pr(safe) ≥ max(0, pU + pO + pF − 2 + i<j Pr(Fi ∩ Fj ))

(B1) (B2)

Pr(safe) = 1 − Pr(FU ∪ FO ∪ FF ) Pr(safe) = pU · pO|U · pF |OU

(B3) (B4)

6

(B1) provides a conservative lower bound from marginals alone, while (B2) incorporates pairwise co-failure probabilities. (B3) is exact via inclusion–exclusion. (B4), derived from the chain rule, is the most informative form, as its factors align with the architectural stages and correspond to layer-wise conditional guarantees. These bounds do not assume independence. Instead, they reflect different levels of available statistical information, from marginals to conditional probabilities. The architecture’s key contribution is to make the factors of (B4) well-defined and independently estimable by aligning them with the sequential availability of information across layers. Any weakening of any single layer degrades all four bounds; no strengthening of one compensates for a gap in another [16]. Appendix C instantiates these bounds on the running example with illustrative estimates; Proposition 2 in Appendix B.5 establishes that no genuine two-stage design yields (B4) as a product of independently estimable, auditable layer-level quantities. The bounds are further elaborated in Appendix A. The running example instantiates the contract chain ΓU delivers AO , ΓO delivers AF , and ΓF .The Room 18 escalation illustrates the architecture’s bidirectional character: the Operational Layer reissues revised constraints on a plan already certified by ΣU , and the Functional Layer operates on an execution context already authorised by both upper layers. Whether these upstream certifications translate into quantitative improvements (pO|U > pO , pF |OU > pF ) is deployment-dependent; what the architecture guarantees is that these conditionals are well-defined, independently estimable, and auditable—properties that any two-stage collapse would forfeit (Proposition 2).

4

Limitations and Future Directions

From A/G Guarantee to Estimable Bounds The framework treats the LLM as a fixed black box, which has a direct consequence for certification: if the underlying model is updated or replaced, the contract structure carries over but the layer-level probabilities pU , pO , pF and their conditionals must be re-derived from fresh execution traces, since contract satisfaction rates will generally change. More fundamentally, even for a fixed model, the bounds (B1)–(B4) require estimating the marginal probabilities pU , pO , pF ; the pairwise co-failure rates Pr(Fi ∩ Fj ); and the conditionals pO|U , pF |OU . None of these is analytically available for LLM agents, because token sampling introduces non-stationarity—identical instructions may yield different plans across invocations— making standard i.i.d. assumptions untenable for bound estimation. The numerical instantiation of Appendix C illustrates what is at stake: a 4-point estimation error in pO|U propagates directly to the (B4) certificate, motivating tight rather than merely valid bounds. Standard PAC theory [54] is therefore not directly applicable, for three compounding reasons. The first is non-i.i.d. traces. Each step in an LLM agent trace conditions on prior context, violating the independence assumption that PAC bounds require. For the marginal estimates required by (B1), martingale bounds [34] and mixing-process bounds [39, 40] offer the most applicable partial remedies, tolerating within-sequence dependence without stationarity assumptions. For the conditional quantities required by (B4), non-exchangeable conformal prediction [5] and the survival-analysis calibration [14]—which reframes safety-probability estimation as a time-to-unsafe-sampling problem and constructs distribution-free lower predictive bounds—are better suited. Anytime-valid inference via e-processes [46] applies across all quantities but yields wider intervals. None of these frameworks has yet been extended to variable-length, layer-level satisfaction events across heterogeneous layers, and ScenicProver [55] offers the closest available blueprint, though extension to non-stationary token-sampling remains open. The second obstacle is compositional interval width. Composing interval estimates [p̂i ± εi ] across layers produces a bound whose width grows with each additional layer and whose coverage depends on the joint distribution of estimation errors—a problem neither the probabilistic contract literature [16, 24] nor PRO2 GUARD [58] resolves across layers. PAC-Bayes bounds [36] treating δ as uncertain, combined with dependency-graph decompositions [45] exploiting structured inter-layer dependence, offer a promising direction but have not been applied in this setting. The third obstacle is correlated backbone failures. When a single LLM backbone underlies multiple layers, a systematic model failure—distributional shift, adversarial prompt, or a hallucination regime [27]—may cause all the relevant layers to fail simultaneously. Under positive correlation, 7

assuming independence overestimates the joint failure probability, so all bounds become pessimistic: valid but potentially too conservative to certify a system that genuinely meets its safety target. The open problem is therefore on the tightness rather than correctness. Two partial architectural strategies provide partial decoupling: deriving the Functional Layer’s CBF certificate from a dynamics model that does not share parameters with the LLM reasoning engine [43], and instantiating distinct LLMs at different layers—though residual correlation persists in the latter case, since frontier LLMs share substantial pretraining data and all layers process the same plan p. Formalising partial decoupling as a quantitative bound on cross-layer correlation is the work required to tighten the system-level certificate to the point where it supports a practical deployment decision. Graceful Degradation of Contracts Under Deployment Drift The bounds (B1)–(B4) are meaningful only if the contracts Σi = (Ai , Γi ) remain stable under realistic deployment perturbations. In practice, this stability is challenged both by violations of assumptions at the semantic level and by distributional drift at runtime. We argue that these seemingly distinct failure modes admit a common treatment through robust contracts that enable controlled, quantitative degradation of guarantees. In classical assume/guarantee (A/G) reasoning, guarantees hold only under strict satisfaction of assumptions: any violation of Ai renders Γi vacuous. This binary semantics is ill-suited for LLMbased systems, where inputs and plans are inherently approximate and semantically structured. We therefore propose a robust contract formulation in which assumptions may be violated up to a tolerance ε, resulting in an at most proportional degradation of the guarantee Γi . This replaces brittle logical validity and enables graceful degradation across all architectural layers. This perspective directly addresses semantic non-robustness, albeit currently only for applications domains where the space of observed behaviours can be equipped with an adequate metric quantifying distance between behaviours. Conceptually, this metric is used for measuring distance between an undesired and a set of desired behaviours, as in robust variants of temporal logic [18, 19]. An instantiation of such metric approaches to the rich semantic spaces manipulated by LLMs is missing hitherto, and it certainly requires more fundamental semantic concepts than logics talking about the inherently metric spaces of time and scalar signal values can currently offer. In LLMbased pipelines, small perturbations at the surface level may leave semantics unchanged, while syntactically similar inputs may differ substantially in meaning. Existing robustness techniques such as SmoothLLM [49] and CluCERT [60] provide certified radii for token-level noise, but do not capture semantic equivalence at the plan level required for contract stability. Within a robust contract framework, such mismatches can be absorbed as bounded assumption violations, allowing guarantees to degrade proportionally rather than fail catastrophically. Initial steps in this direction are taken by the Agent Behavioral Contracts framework [7], which treats soft violations as early indicators of behavioral drift, though a quantitative robustness theorem remains open. Crucially, the same mechanism provides a principled response to distributional drift at runtime. We distinguish between environmental drift, where the operational distribution diverges from the certified WODD , and generative drift, where LLM stochasticity produces previously unseen plan structures. In both cases, the underlying issue can be interpreted as a violation of layer assumptions Ai . Robust contracts allow such deviations to be captured quantitatively: instead of invalidating the guarantees entirely, the system tracks how far the observed behavior departs from the certified assumption space and adjusts Γi accordingly. This unified view does not eliminate the need for detection and re-certification. Environmental drift may still cause hard violations (e.g., leaving WODD ), making ODD monitoring [48, 63] a prerequisite for safe operation. Similarly, the estimates pU , pO|U , pF |OU must be revalidated under sustained drift. However, robust contracts ensure that, prior to re-certification, the system degrades in a controlled and interpretable manner rather than failing abruptly. Fully realizing this approach requires capabilities that are currently missing for LLM agents: (i) automated inference of assumption relaxations from observed violations and feedback, and (ii) principled incremental updates of contracts and compositional bounds under changing assumptions. The anytime-valid inference framework [46] is a promising foundation for such incremental recertification, as its confidence sequences remain valid under continuous data collection, though its integration into multi-layer contract systems remains an open problem.

8

Extension to Multi-Agent Settings The framework is restricted to single-agent architectures by construction. In multi-agent deployments, an additional class of unsafe behaviour emerges that no single-agent layer addresses: cross-agent belief manipulation, where an agent’s reasoning is corrupted by peer-generated content without any single instruction being overtly malicious. This can occur through faulty instructions that exploit implicit inter-agent trust [35], through coordinated factual fragments that steer agents toward false beliefs via their own reasoning tendencies [25], or through accumulated reasoning misalignment as chain-of-thought traces diverge from humanpreferred reasoning paths with depth [56]—and the literature on such vectors is growing. All three vectors share a common structural cause: the ΓU guarantee certifies semantic authorisation against a single human principal but provides no certification over the content or intent of inter-agent messages. A multi-agent extension would need to treat inter-agent messages as a fourth information domain IM , introducing a provenance layer that validates the full authorisation chain of incoming instructions back to a verified human principal. This interacts non-trivially with the system-level bounds of Section 3.2, since co-failure correlation between agents sharing infrastructure would enter the bound. MAST [11] identifies 14 failure modes across system design, inter-agent misalignment, and task verification that such an extension would need to address. We regard this as the highest-priority structural extension once the single-agent open problems above are resolved.

5

Alternative Views

Learned end-to-end safety. An alternative is to learn safety properties directly (e.g., RLHF [13], Constitutional AI [4], or learned probabilistic models such as P RO 2 G UARD [58]). We view these approaches as complementary: learning handles perceptual complexity, while contracts provide certification and compositional reasoning. Whether learning can replace this structure remains open; we argue certification constraints make this unlikely. Probabilistic vs. boolean guarantees. Boolean certification is preferable where achievable. At the system level, however, probabilistic contracts are the correct target on two independent grounds: LLM non-determinism makes boolean system-level certification ill-defined, and the deployment environment is itself irreducibly probabilistic—sensor noise, partial observability, and non-stationary dynamics mean that boolean guarantees derived from formal models degrade under real-world conditions regardless of the underlying model [33]. Specifically, the User Layer’s interaction with the LLM is inherently non-deterministic, so alignment assurance can only be assessed probabilistically, while the Operational and Functional Layers focus respectively on system autonomy and safe action execution, both tightly coupled to an environment characterised by imperfect observation—partial observability, unreliable sensors, and imperfect perception—each introducing sources of uncertainty that must likewise be assessed probabilistically. Latency and deployability. Sequential certification across three layers imposes cumulative latency potentially incompatible with fast control loops. We scope the framework to deliberative agents where planning horizons exceed certification overhead, and note that partial parallelisation across layers is possible; full latency characterisation remains future work. Correctness vs. optimality. The architecture certifies safety but says nothing about task performance: a maximally conservative layer could certifiably block all actions. The tension between safety guarantees and task utility is not addressed here; VeriGuard [38] demonstrates empirically that nearzero attack success rate can be achieved without substantially degrading task success rate, suggesting the tension may be practically manageable, but no theoretical treatment of this tradeoff within the A/G contract framework currently exists.

6

Conclusions

The three-layer architecture proposed here is the minimum viable certification structure that follows from this, and whether it becomes a deployable standard depends on the community treating the open problems of Section 4 not as limitations to acknowledge but as the primary research agenda to pursue: the field is currently producing safety mechanisms faster than the theoretical infrastructure to compose and certify them. This asymmetry is not unusual in the early stages of a safety-critical engineering discipline, such as aviation and automotive. The difference is that LLM agents are being deployed now, in safety-critical settings, before that theory exists. 9

References [1] M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu. Safe reinforcement learning via shielding. In S. A. McIlraith and K. Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 2669–2678. AAAI Press, 2018. [2] A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control barrier functions: Theory and applications. In 17th European Control Conference, ECC 2019, Naples, Italy, June 25-28, 2019, pages 3420–3431. IEEE, 2019. [3] M. Andriushchenko, A. Souly, A. Sezener, E. Cubuk, R. Prenger, C. Rahtz, J. Steinhardt, J. Kolter, X. Davies, et al. Agentharm: A benchmark for measuring harmfulness of llm agents. arXiv preprint arXiv:2410.09024, 2024. [4] Y. Bai et al. Constitutional ai: Harmlessness from ai feedback. ArXiv, abs/2212.08073, 2022. [5] R. F. Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani. Conformal prediction beyond exchangeability. Annals of Statistics, 51(2):816–845, 2023. [6] E. Bartocci, J. V. Deshmukh, A. Donzé, G. Fainekos, O. Maler, D. Nickovic, and S. Sankaranarayanan. Specification-based monitoring of cyber-physical systems: A survey on theory, tools and applications. In Lectures on Runtime Verification, volume 10457 of Lecture Notes in Computer Science, pages 135–175. Springer, 2018. [7] V. P. Bhardwaj. Agent behavioral contracts: Formal specification and runtime enforcement for reliable autonomous AI agents. arXiv preprint arXiv:2602.22302, 2026. cs.AI. [8] P. Blohm, M. Fränzle, P. Herber, P. Kröger, and A. Remke. Towards probabilistic contracts for intelligent cyber-physical systems. In International Symposium on Leveraging Applications of Formal Methods, pages 26–47. Springer, 2024. [9] P. Blohm, M. Fränzle, P. Herber, P. Kröger, and A. Remke. Towards probabilistic contracts for intelligent cyber-physical systems. In T. Margaria and B. Steffen, editors, Leveraging Applications of Formal Methods, Verification and Validation. Specification and Verification 12th International Symposium, ISoLA 2024, Crete, Greece, October 27-31, 2024, Proceedings, Part III, Lecture Notes in Computer Science, pages 26–47. Springer, 2024. [10] British Standards Institution. PAS 1883: Operational design domain (ODD) taxonomy for an automated driving system (ADS)—specification. Technical report, BSI, Aug. 2020. [11] M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran, M. Zaharia, J. E. Gonzalez, and I. Stoica. Why do multi-agent LLM systems fail? In Advances in Neural Information Processing Systems (NeurIPS), volume 38, 2025. Datasets and Benchmarks Track, Spotlight. arXiv:2503.13657. [12] Z. Chen, M. Kang, and B. Li. ShieldAgent: Shielding agents via verifiable safety policy reasoning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 8313–8344. PMLR, 2025. [13] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. [14] H. Davidov et al. Calibrated predictive lower bounds on time-to-unsafe-sampling in LLMs. 2026. arXiv preprint arXiv:2506.13593. [15] B. Delahaye, B. Caillaud, and A. Legay. Probabilistic contracts: A compositional reasoning methodology for the design of stochastic systems. In L. Gomes, V. Khomenko, and J. M. Fernandes, editors, 10th International Conference on Application of Concurrency to System Design, ACSD 2010, Braga, Portugal, 21-25 June 2010, pages 223–232. IEEE Computer Society, 2010. 10

[16] B. Delahaye, B. Caillaud, and A. Legay. Probabilistic contracts: a compositional reasoning methodology for the design of systems with stochastic and/or non-deterministic aspects. Formal Methods in System Design, 38(1):1–32, 2011. [17] Y. Dong, R. Mu, G. Jin, Y. Qi, J. Hu, X. Zhao, J. Meng, W. Ruan, and X. Huang. Position: Building guardrails for large language models requires systematic design. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 11475–11492. PMLR, 2024. [18] A. Donzé and O. Maler. Robust satisfaction of temporal logic over real-valued signals. In K. Chatterjee and T. A. Henzinger, editors, Formal Modeling and Analysis of Timed Systems 8th International Conference, FORMATS 2010, Klosterneuburg, Austria, September 8-10, 2010. Proceedings, Lecture Notes in Computer Science, pages 92–106. Springer, 2010. [19] M. Fränzle and M. R. Hansen. A robust interpretation of duration calculus. In D. Van Hung and M. Wirsing, editors, Theoretical Aspects of Computing – ICTAC 2005, pages 257–271, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. [20] M. Fréchet. Généralisations du théorème des probabilités totales. Fundamenta Mathematicae, 25:379–387, 1935. [21] Y. Gan, Y. Yang, Z. Ma, P. He, R. Zeng, Y. Wang, Q. Li, C. Zhou, S. Li, and T. Wang. Navigating the risks: A survey of security, privacy, and ethics threats in LLM-based agents. arXiv preprint arXiv:2411.09523, 2025. [22] D. Giannakopoulou, K. S. Namjoshi, and C. S. Păsăreanu. Compositional reasoning. In Handbook of Model Checking, pages 345–383. Springer, 2018. [23] J. Y. Halpern. Actual Causality. MIT Press, Cambridge, MA, 2016. [24] A. Hampus and M. Nyberg. A theory of probabilistic contracts. In Leveraging Applications of Formal Methods, Verification and Validation. Specification and Verification (ISoLA 2024), volume 15221 of Lecture Notes in Computer Science, pages 296–319. Springer, 2024. [25] J. Hu, X. Huang, Y. Sun, Y. Dong, and X. Huang. Lying with truths: Open-channel multi-agent collusion for belief manipulation via generative montage. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), 2026. [26] W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. In Conference on Robot Learning, pages 1769–1782. PMLR, 2023. [27] X. Huang, W. Ruan, W. Huang, G. Jin, Y. Dong, C. Wu, S. Bensalem, R. Mu, Y. Qi, X. Zhao, K. Cai, Y. Zhang, S. Wu, P. Xu, D. Wu, A. Freitas, and M. A. Mustafa. A survey of safety and trustworthiness of large language models through the lens of verification and validation. Artificial Intelligence Review, 57(7):175, 2024. [28] ISO. ISO 34503: Road vehicles—taxonomy and definitions for operational design domain for automated driving systems, 2023. [29] A. Kamath et al. Enforcing temporal constraints for LLM agents, 2025. ICLR 2026 Workshop VerifAI. [30] A. A. Khan, M. Andrev, M. A. Murtaza, S. Aguilera, R. Zhang, J. Ding, S. Hutchinson, and A. Anwar. Safety aware task planning via large language models in robotics. arXiv preprint arXiv:2503.15707, 2025. cs.RO. [31] S. M. Kwerel. Bounds on the probability of the union and intersection of m events. Advances in Applied Probability, 7(2):431–448, 1975. [32] S. Lei et al. OffTopicEval: When large language models enter the wrong chat, almost always! In Proceedings of the 14th International Conference on Learning Representations (ICLR), 2026. arXiv:2509.26495. 11

[33] C. Li, S. Faghfoorian, and I. Ruchkin. What does it take to get guarantees? Systematizing assumptions in cyber-physical systems. arXiv preprint arXiv:2511.15952, 2025. [34] S. Lotfi, Y. Kuang, B. Amos, M. Goldblum, M. Finzi, and A. G. Wilson. Unlocking tokens as data points for generalization bounds on larger language models. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. Spotlight presentation. arXiv:2407.18158. [35] M. Lupinacci, F. A. Pironti, F. Blefari, F. Romeo, L. Arena, and A. Furfaro. The dark side of LLMs: Agent-based attacks for complete computer takeover. arXiv preprint arXiv:2507.06850, 2025. v5, revised November 2025. [36] D. A. McAllester. Pac-bayesian model averaging. Proceedings of the 12th Annual Conference on Computational Learning Theory (COLT), pages 164–170, 1999. [37] P. Mestres et al. Probabilistic control barrier functions: Safety in probability for discrete-time stochastic systems, 2025. [38] L. Miculicich, M. Parmar, H. Palangi, K. D. Dvijotham, M. Montanari, T. Pfister, and L. T. Le. Veriguard: Enhancing llm agent safety via verified code generation, 2025. [39] M. Mohri and A. Rostamizadeh. Rademacher complexity bounds for non-I.I.D. processes. In Advances in Neural Information Processing Systems, volume 21, 2009. [40] M. Mohri and A. Rostamizadeh. Stability bounds for stationary ϕ-mixing and β-mixing processes. Journal of Machine Learning Research, 11:789–814, 2010. [41] J. Nöther, A. Singla, and G. Radanovic. Benchmarking the robustness of agentic systems to adversarially-induced harms. arXiv preprint arXiv:2508.16481, 2025. [42] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 27730– 27744, 2022. [43] R. Pandya. Influence-aware safety for human-robot interaction. Pittsburgh, PA, Oct. 2025. CMU-RI-TR-25-95. [44] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct preference optimization: your language model is secretly a reward model. NIPS ’23, Red Hook, NY, USA, 2023. Curran Associates Inc. [45] L. Ralaivola, M. Szafranski, and G. Stempfel. Chromatic PAC-Bayes bounds for non-IID data. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), pages 416–423, 2009. [46] A. Ramdas, P. Grünwald, V. Vovk, and G. Shafer. Game-theoretic statistics and safe anytimevalid inference. Statistical Science, 38(4):576–601, 2023. [47] Z. Ravichandran, A. Robey, V. Kumar, G. J. Pappas, and H. Hassani. Safety guardrails for llm-enabled robots. IEEE Robotics and Automation Letters, 2026. [48] J. Ren, J. Luo, Y. Zhao, K. Krishna, M. Saleh, B. Lakshminarayanan, and P. J. Liu. Out-ofdistribution detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations (ICLR), 2023. [49] A. Robey, E. Wong, H. Hassani, and G. J. Pappas. SmoothLLM: Defending large language models against jailbreaking attacks. In Proceedings of the 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 23–38, 2025. [50] SAE International. J3016: Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Technical Report J3016_202104, SAE International, Apr. 2021. [51] K. Salako. Constructive proofs of generalized Boole–Fréchet bounds: A dynamic programming approach, 2025. arXiv:2512.09161 [math.PR], 9 December 2025. 12

[52] M. Shamsujjoha, Q. Lu, D. Zhao, and L. Zhu. Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents. In Proceedings of the 22nd IEEE International Conference on Software Architecture (ICSA), pages 37–48, 2025. [53] C. Urrea. Probabilistic safety guarantees for learned control barrier functions: Theory and application to multi-objective Human–Robot collaborative optimization. Mathematics, 14(3):516, 2026. [54] L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, 1984. [55] E. Vin, K. A. Miller, I. Incer, S. A. Seshia, and D. J. Fremont. ScenicProver: A framework for compositional probabilistic verification of learning-enabled systems. arXiv preprint arXiv:2511.02164, 2025. Full version of a paper submitted to TACAS 2026. cs.LO, cs.AI, cs.LG. [56] B. Wang, Z. Li, X. Huang, X. Huang, and Y. Dong. Chain-of-thought as a lens: Evaluating structured reasoning alignment between human preferences and large language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), 2026. [57] H. Wang, C. M. Poskitt, and J. Sun. AgentSpec: Customizable runtime enforcement for safe and reliable LLM agents. In Proceedings of the 48th IEEE/ACM International Conference on Software Engineering (ICSE 2026), pages 1–12, Rio de Janeiro, Brazil, 2026. ACM. [58] H. Wang, C. M. Poskitt, J. Sun, and J. Wei. Pro2 Guard: Proactive runtime enforcement of LLM agent safety via probabilistic model checking. arXiv preprint arXiv:2508.00500, 2025. cs.SE, cs.AI. [59] L. Wang, Z. Ying, X. Yang, Q. Zou, Z. Yin, T. Li, J. Yang, Y. Yang, A. Liu, and X. Liu. RoboSafe: Safeguarding embodied agents via executable safety logic. arXiv preprint arXiv:2512.21220, 2025. cs.AI, cs.CV, cs.RO. [60] Z. Wang, C.-H. Cheng, G. Jin, et al. CluCERT: Certifying LLM robustness via clustering-guided denoising smoothing. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025. arXiv:2512.08967. [61] W. Yang, G. Marra, G. Rens, and L. D. Raedt. Safe reinforcement learning via probabilistic logic shields. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pages 5739–5749. ijcai.org, 2023. [62] Z. Yang, S. S. Raman, A. Shah, and S. Tellex. Plug in the safety chip: Enforcing constraints for llm-driven robot agents. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14435–14442. IEEE, 2024. [63] A. Zhang, T. Z. Xiao, W. Liu, R. Bamler, and D. Wischik. Your finetuned large language model is already a powerful out-of-distribution detector. In The 28th International Conference on Artificial Intelligence and Statistics, 2025. [64] Z. Zhang, S. Cui, Y. Lu, J. Zhou, J. Yang, H. Wang, and M. Huang. Agent-SafetyBench: Evaluating the safety of LLM agents. arXiv preprint arXiv:2412.14470, 2024. [65] W. Zhao, Y. Zhao, G. Liu, Z. Jiang, D. Ma, Z. Li, and X. Li. Sage-llm: Towards safe and generalizable llm controller with fuzzy-cbf verification and graph-structured knowledge retrieval for uav decision, 2026. [66] Y. Zhao, B. Hoxha, G. Fainekos, J. V. Deshmukh, and L. Lindemann. Robust conformal prediction for STL runtime verification under distribution shift. In 15th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2024, Hong Kong, May 13-16, 2024, pages 169–179. IEEE, 2024.

13

A

Three-Layer Framework

This Appendix section presents the technical details that correspond to those in Section 3. The three-layer framework processes a plan p through three sequential validation layers before and during execution. Each layer is associated with a contract consisting of an assumption on its inputs and a guarantee on its outputs; the guarantee of each layer satisfies the assumption of the next, enabling compositional reasoning (Section A.2). The framework operates in a layered validation and enforcement mode: the User Layer validates plan p prior to execution against intent, policy, and ethics; the Operational Layer validates both prior to and during execution against the current world state; and the Functional Layer enforces safety continuously at runtime. Example 1 (Setup). A caregiver instructs a service robot deployed in an elderly-care facility: “Visit Rooms 12, 15, and 18 before 15:00 to check each resident’s hydration and wellbeing; bring water if needed; do not disturb residents during rest periods.” The robot operates indoors across two floors with moderate human traffic, equipped with a mobile base, a 6-DoF compliant arm, and RGB-D / LiDAR perception. The LLM agent produces a plan p — a sequence of room-visit, assessment, and conditional deliver actions — which passes sequentially through the three layers, as developed in Examples 2–4. A.1

The Three Layers

Each layer is a quadruple (Mi , Gi , πi , Σi ) for i ∈ {U, O, F }: a monitor Mi observing state and context, a guardrail Gi enforcing local constraints, a fallback policy πi activated when Gi becomes infeasible or a safety signal is violated, and a probabilistic assume–guarantee contract Σi = (Ai , Γi ) as defined in Section 3. In the main text, Σi serves as shorthand for the layer as a whole; the quadruple here makes the internal structure of each layer explicit without altering the contract semantics. The three contracts are chained: the guarantee of each layer satisfies the assumption of the next (ΓU ⇒ AO , ΓO ⇒ AF ), so the global invariant Σglobal = ΓU ∧ ΓO ∧ ΓF follows by sequential local certification, without global analysis across heterogeneous layers. Each layer additionally emits two constraint outputs: caps—quantitative limits on continuous variables such as speed or force—and zones—spatial or semantic regions whereSoperation is forbidden. These are fused conservatively as capsfinal = mini capsi and zonesfinal = i zonesi (where caps i and zones i for each layer i ∈ {U, O, F } are defined in Sections A.1.1–A.1.3 below), so that the tightest numerical limit and any locally prohibited region dominate globally. Each of the three layers instantiates this template for a distinct safety dimension, as detailed in the subsections below. A.1.1

User Assurance Layer

The User Layer is the first assurance barrier, validating the plan p against intent, policy, and ethics before any world observation is made. Its responsibility spans three orthogonal alignment dimensions. Cognitive alignment checks whether the plan faithfully realises the user’s expressed intent—neither under-delivering nor exceeding it. Regulation alignment checks compliance with domain-specific rules and standards, covering both syntactic well-formedness and semantic adherence to applicable policies. Ethical alignment checks that the plan respects ethical constraints, such as prohibiting camera use in private spaces. The layer additionally verifies the request itself: that the issuing user holds the required authorisation and permissions. The quadruple (MU , GU , πU , ΣU ) is instantiated as follows. The monitor MU observes incoming intent y and flags unsafe or ambiguous content—refusal keywords, Protected Health Information (PHI) exposure, permission mismatches, and policy violations. It also enforces four levels of output checks: syntactic well-formedness, semantic feasibility, semantic compliance with domain policies, and semantic fidelity to user intent; a plan failing any of them is handled by the guardrail GU , e.g. fixed or returned for refinement with a guidance. When GU cannot resolve a violation, the fallback policy πU issues a plan disposition sU ∈ {ok, clarify, deny}, indicating whether the plan is approved, returned for clarification, or denied outright. Unlike sF and sO , which are emitted during execution and propagate upward through the layer stack, sU is a pre-execution gate decision. It constitutes the terminal output of the bottom-up channel: when sO reaches the User Layer, the layer issues sU ∈ {ok, clarify, deny} to notify the user or trigger plan recomputation, but sU is not propagated to any higher layer (none exists). 14

The safety contract ΣU = (AU , ΓU ) formalises the layer’s assurance commitment. Given intent y, policy database Puser , and context metadata, the layer produces pU , capsU , zonesU = U (y, Puser , context). The outputs capsU and zonesU are not pre-existing inputs but are derived through policy reasoning: from user roles, permissions, and ethical rules, GU produces quantitative limits on continuous variables—such as maximum speed or proximity bounds—and semantic exclusions over locations and tasks, together defining the boundaries within which the Operational Layer must operate. Assuming AU —the request is issued by an authorised user—the layer guarantees ΓU —the emitted plan pU targets only authorised locations, encodes conditional actions correctly, and contains no policy or ethical violations. This guarantee directly constitutes assumption AO for the Operational Layer. Example 2 (User Layer). The layer validates plan p against intent, policy, and ethics before any world observation is made. It rejects plans that omit the conditional water-delivery logic (cognitive alignment), that schedule room entry during rest hours without caregiver override (regulation alignment), or that route the robot through a bathroom with cameras active (ethical alignment). Given that the request is issued by an authorised caregiver (AU holds), the guarantee ΓU is that pU visits only rooms in Rauth (the set of locations authorised for the issuing user under the current policy), encodes conditional delivery correctly, and contains no policy or ethical violations. The approved plan thus targets only authorised rooms, constituting AO for the Operational Layer. Key open problems include the automated formalisation of user intent and domain policies into verifiable representations, and the design of neural-symbolic validation tools that can bridge informal intent with formal constraint checking. A.1.2

Operational Assurance Layer

The Operational Layer is the middle assurance barrier. Where the User Layer constrains what the system is permitted to do and the Functional Layer constrains how it executes, the Operational Layer constrains where and under what world conditions execution is authorised—answering not “what will happen?” but “is this a world in which we are authorised and certified to act?” The ODD is not merely a list of environmental constraints but a structured object encoding five dimensions—physical configuration (geometry, obstacles, dynamics), perceptual configuration (sensor availability, visibility), contextual configuration (time of day, density, access restrictions), governance configuration (authorisation regimes, policy zones), and social or normative context (restricted roles, privacy constraints)—which may interact but are treated as independently certifiable for the purposes of ODD membership evaluation. The Operational Design Domain is a certified subset WODD ⊆ W that encodes not only physical feasibility but legitimacy: a robot may be dynamically capable of navigating a crowded hospital corridor yet barred by institutional policy during certain hours, and execution outside WODD invalidates all downstream guarantees regardless of physical capability. The quadruple (MO , GO , πO , ΣO ) is instantiated as follows. The monitor MO ingests sensor and perception data ξ and computes the world state estimate ŵ. The guardrail GO enforces a deterministic validation decision over ŵ: identical estimates must yield identical verdicts, ensuring reproducibility, auditability, and preservation of compositional invariants. Determinism is an engineering choice rather than a logical requirement of the probabilistic A/G framework [16]; it is adopted here because stochastic verdicts complicate audit trails in safety-critical deployments. Probabilistic uncertainty is handled upstream—a world-risk score RO (ŵ) = P (ŵ ∈ / WODD ) informs margin selection and conservative thresholds—but the final verdict of GO remains deterministic, consistent with safetycritical control architectures and required for assume–guarantee composability. When the verdict is invalid or the ODD degrades, the fallback πO issues sO ∈ {nominal, degraded, MRC}, where MRC denotes a Minimal Risk Condition [50]—a pre-planned, system-initiated response that brings the agent to a safe state upon ODD exit or critical failure, triggering plan restriction or human takeover. The safety contract ΣO = (AO , ΓO ) formalises the layer’s commitment. Given validated plan pU , user-level constraints capsU and zonesU , sensor data ξ, and ODD specification Podd , the layer produces: pO , capsO , zonesO = O(pU , ξ, Podd ), where pO = pU when ODDvalid = true, and pO = fODD (pU )—a restriction of the plan to the valid subset of the domain—otherwise. Assuming AO (i.e., ΓU holds), the layer guarantees ΓO : execution remains within WODD , autonomy decisions respect the envelope E, ODD-invalid sub-plans are blocked, and MRC is triggered upon ODD exit. This guarantee constitutes AF for the Functional Layer. 15

Example 3 (Operational Layer). The layer checks whether ŵ lies in WODD . Room 15 is restricted (clinical procedure in progress): the verdict is invalid; that sub-plan is blocked and the caregiver is notified. The corridor to Room 12 is under-lit but remains within WODD subject to v ≤ vmax , issued as a conditional validity decision. Given that the plan targets only rooms in Rauth (AO holds), the guarantee ΓO is that execution stays within WODD at v ≤ vmax and all ODD-invalid sub-plans are blocked, constituting AF for the Functional Layer. A key open problem is establishing guarantees under distributional shift: when ŵ is drawn from a distribution that drifts from the one under which WODD was certified, the deterministic verdict may be unreliable, and robust conformal methods [66] offer a promising direction. A.1.3

Functional Assurance Layer

The Functional Layer is the final assurance barrier, constraining how the system executes at controlloop frequency. Once the User and Operational Layers have validated intent and authorised the deployment context, the Functional Layer takes over to enforce the physical, spatial, geometric, and temporal constraints imposed by the deployment environment dynamically—guaranteeing that the system remains within its safe envelope even under dynamically changing conditions. The layer draws on three complementary enforcement techniques. Specification-based runtime monitoring tracks the executed trajectory against safety specifications, producing robustness margins ρF (t) and verdicts that trigger intervention when a violation is detected or anticipated. Control barrier functions (CBFs) project any candidate control command onto the admissible safe set via Πsafe , correcting it minimally—if the command already satisfies all constraints, Πsafe leaves it unchanged. Recent work confirms that CBF-constrained quadratic programs can be embedded inside LLM-driven planning loops without sacrificing task completion, provided the dynamics model is sufficiently accurate [30, 65], and probabilistic extensions now provide explicit bounds on safety failure probability for learned barriers [53, 37]. Simulation-based synthesis uses proxy setups such as world models to perform pre-computations, for instance via safe reinforcement learning, when the environment is large, unknown, or dynamic. These three techniques are distributed across the quadruple (MF , GF , πF , ΣF ): the monitor MF implements specification-based monitoring; the guardrail GF applies CBF projection and simulation-based synthesis, with CBF projection taking precedence when the two conflict at runtime; and when GF cannot maintain safety—either because Πsafe becomes infeasible or ρF (t) < 0—the fallback πF is activated, issuing sF ∈ {nominal, override, MRC} and triggering corrective action or safe-stop. The safety contract ΣF = (AF , ΓF ) closes the contract chain. Given validated plan pO , fused constraints capsfinal and zonesfinal , candidate control udes (t), system state x(t), and system dynamics (f, g), the layer produces a safe actuation command usafe (t), a runtime robustness margin ρF (t), and a safety signal: usafe , ρF , sF = F (pO , udes , x(t), capsfinal , zonesfinal ). Assuming AF —w ∈ WODD and v ≤ vmax , delivered by ΓO , thereby closing the contract chain—the layer guarantees ΓF —no collision occurs, all physical interactions satisfy f ≤ fmax , and any violation triggers corrective action or safe-stop. Feedback flows bottom-up: sF is reported to the Operational Layer, which in turn propagates sO to the User Layer, enabling plan recomputation or user notification when execution cannot be completed safely. Example 4 (Functional Layer). A resident enters the corridor en route to Room 12: MF detects d < dmin and issues a hold. Inside Room 12, a CBF constrains arm force to f ≤ fmax throughout the handover, adapting to the resident’s motion. In Room 18, rearranged furniture blocks the delivery point; the layer escalates to the Operational Layer with an updated obstacle map. Given AF —namely w ∈ WODD and v ≤ vmax , delivered by ΓO —the guarantee ΓF is that no collision occurs and all physical interactions satisfy f ≤ fmax , with violations triggering corrective action or safe-stop. Key open problems include guaranteeing real-time safety under non-stationary dynamics and adapting safety envelopes mid-execution when updated caps or zones arrive from upper layers. A.2

End-to-End Safety Guarantee

Since ΓU ⇒ AO and ΓO ⇒ AF , the three contracts compose sequentially. Let Fi = ¬Γi be the failure event of layer i and pi = P (Γi ) its satisfaction probability. Note that GU produces a deterministic verdict for every input, but determinism does not imply correctness: over the distribution 16

of possible plans, GU may still approve non-compliant plans, so pU < 1 in general. Four bounds of increasing tightness characterise P (system safe) = P (ΓU ∩ ΓO ∩ ΓF ):
P (safe) ≥ max 0, pU + pO + pF − 2 (B1) P P (safe) ≥ pU + pO + pF − 2 + i<j P (Fi ∩ Fj ) (B2) P (safe) = 1 − P (FU ∪ FO ∪ FF ) P (safe) = pU · pO|U · pF |OU

(B3) (B4)

(B1) is the Fréchet–Bonferroni bound [20, 31], derivable from marginals alone and requiring no independence assumption; it is the tightest bound so derivable [31]. (B2) adds pairwise co-failure probabilities P (Fi ∩ Fj ), estimable from paired execution traces; it is strictly tighter than (B1) whenever layers share failure modes [31]. (B3) is exact via inclusion-exclusion; under mutual independence it reduces to pU · pO · pF . (B4) is the sequential chain-rule decomposition, where pO|U = P (ΓO | ΓU ) and pF |OU = P (ΓF | ΓO , ΓU ); it is exact without any independence assumption. When upstream filtering is effective—i.e., when the User Layer removes plans that would cause downstream failures—pO|U ≥ pO and pF |OU ≥ pF , giving pU · pO|U · pF |OU ≥ pU · pO · pF . Whether this holds in a given deployment is an empirical property of the interaction between layers, not a structural guarantee; what the architecture guarantees is that the conditional factors are welldefined and independently certifiable. Any weakening of any single layer degrades all four bounds, motivating independent certification of all three layers. Estimation caveat. Bound (B4) is the natural certification target, but instantiating any of (B1)–(B4) requires estimating quantities—pU , pO , pF , co-failure rates, and the conditionals pO|U , pF |OU —that standard PAC theory [54] cannot supply directly, because LLM agent traces violate the i.i.d. assumption in two ways: each step conditions on prior context (non-stationarity), and all three layers share the same model backbone, coupling their failure events. Several frameworks offer partial remedies. Martingale-based bounds—used by Lotfi et al. [34] to derive token-level generalisation bounds for LLMs—tolerate within-sequence dependence without requiring stationarity. Stability-based bounds for mixing processes [39, 40] give algorithm-specific guarantees for β- and ϕ-mixing sequences, subsuming i.i.d. as a special case, at the cost of characterising the mixing rate of LLM traces. Non-exchangeable conformal prediction [5] provides distribution-free coverage with degradation bounds quantified as a function of total-variation distance from exchangeability. Anytime-valid inference via e-processes [46] yields confidence sequences valid at all stopping times, fitting naturally with the incremental re-certification of Section 4. Most directly, Davidov et al. [14] reframe LLM safety probability estimation as a survival analysis problem, constructing calibrated lower bounds on time-to-unsafe-sampling via conformal prediction with finite-sample coverage guarantees and no distributional assumptions. None of these simultaneously handles non-stationarity, heterogeneous layer composition, and correlated backbone failures; closing that gap is the open problem of Section 4. For (B1) alone, martingale or mixing-process bounds are the most plausible near-term route, as they require only marginal estimates pU , pO , pF under within-sequence dependence. Until the full gap is closed, (B1)–(B4) provide a principled certification target rather than deployable certificates. Together, Examples 2-4 confirm that the composition operates as a live bidirectional assurance loop rather than a one-pass pipeline: forward contract propagation (ΓU ⇒ AO ⇒ ΓO ⇒ AF ⇒ ΓF ) certifies each stage before execution, while bottom-up safety signals (sF → sO → sU ) enable plan recomputation when execution cannot be completed safely within the certified envelope.

B

Why Three Layers Are Structurally Necessary

This appendix formalises the structural argument of Section 2. We do not claim that single-layer certification is logically impossible under all conceivable definitions. We claim something more specific and more useful: any certification architecture that (i) respects the causal timeline of agent execution and (ii) satisfies the three desiderata below must implement the staged information structure of the three-layer design—either explicitly or implicitly. The three layers are therefore not one solution among many but the shape that any adequate solution must take. Formal setup (retained from main text). An LLM agent execution is a triple e = (p, w, x) where p ∈ P is the LLM-generated plan, w ∈ W is the realised world state observed via sensor data ξ at 17

execution time, and x = (x(t), u(t))t≥0 is the state–control trajectory during closed-loop execution. Define three sub-σ-algebras of the underlying probability space (Ω, F , P): IU = σ(y, Puser , role),

IO = σ(IU , ξ, ŵ, WODD ),

IF = σ(IO , x(t), u(t)),

and three Boolean safety predicates: ΦU (e) (semantic safety, IU -measurable), ΦO (e) (operational safety, IO -measurable but not IU -measurable in general), and ΦF (e) (dynamical safety, IF -measurable but not IO -measurable in general). B.1

Information-Set Ordering

Remark 1 (Strict inclusion of information sets). The inclusions IU ̸⊆ IO ̸⊆ IF are strict under any operationally non-degenerate deployment distribution D (Definition 1 below). • (IU ̸⊆ IO ). Two executions (p, w, x) and (p, w′ , x′ ) sharing the same plan, intent, and role metadata are IU -indistinguishable. Setting w ∈ WODD and w′ ∈ / WODD (possible whenever α := PD (w ∈ / WODD ) > 0) yields ΦO (e) = 1 and ΦO (e′ ) = 0, so ΦO is not IU -measurable. • (IO ̸⊆ IF ). Two executions (p, w, x) and (p, w, x′ ) sharing plan and world state are IO -indistinguishable. LLM non-determinism and stochastic actuation can yield trajectories x satisfying all dynamical constraints while x′ incurs a collision at some t > 0 (possible whenever β := PD (ΦF = 0 | w ∈ WODD ) > 0), so ΦF is not IO -measurable. This ordering is a direct consequence of the physical execution timeline and holds regardless of the computational architecture employed. Definition 1 (Operationally non-degenerate distribution). A distribution D over executions e = (p, w, x) is operationally non-degenerate if α > 0 and β > 0 as defined above. Both conditions are mild and hold in any realistic deployment. B.2

Three Causally-Grounded Desiderata

Three desiderata follow directly from the physical and institutional structure of agent deployment rather than from a design choice about contract semantics. Their grounding in the causal structure of action—the distinction between what information is available before an action versus after—is in the tradition of structural-equation models of causality [23]. Desideratum 1 (Preventive semantic gate). Semantic authorisation (ΦU ) must be certified before execution begins, using only information available prior to any world observation. A scheme that permits execution of a semantically unauthorised plan and attempts correction afterward is an incident-response architecture, not a safety architecture. This is not a design preference: actions in the physical world are generically irreversible on the timescale of plan execution, so post-hoc correction cannot restore the pre-execution state. Desideratum 2 (Causal observability of operational safety). Operational safety (ΦO ) depends on the realised world state w, which is physically unavailable until sensor data ξ is collected at execution time. No pre-execution computation can substitute for this observation without either (a) assuming the world state in advance—reintroducing distributional-shift failure modes—or (b) adopting a worst-case bound that blocks all non-trivially constrained execution. Neither substitute constitutes a valid operational safety certificate. Desideratum 3 (Trajectory dependence of dynamical safety). Dynamical safety (ΦF ) depends on the state–control trajectory (x(t), u(t))t≥0 , which does not exist until the control loop is running. Pre-execution or pre-actuation certification of ΦF is either a simulation—inheriting all limitations of the world model—or a worst-case envelope too conservative for practical deployment. Neither substitute constitutes runtime enforcement. These desiderata are not definitional impositions on the A/G framework. D1 reflects the irreversibility of physical action; D2 and D3 reflect the causal precedence of observation over inference: a property that depends on data not yet available cannot be certified before that data arrives [23]. Together, D1–D3 establish the strict temporal ordering τU < τO < τF of certification stages, where τi denotes the earliest time at which Φi can be certified. This ordering coincides exactly with the strict inclusion IU ̸⊆ IO ̸⊆ IF of Remark 1. 18

B.3

The Collapse Argument

Proposition 1 (Collapse argument). Any architecture with fewer than three independently certified stages, each satisfying one of D1–D3, must either (i) violate at least one desideratum, or (ii) implicitly reconstruct the three-stage structure within a nominally single component. Proof. There are three cases, corresponding to the three distinct ways of collapsing two of the three stages. Case A: Collapse IU and IO (single pre-execution stage). A certifier operating only on preexecution information must decide on ΦO before sensor data ξ is available. By D2, no pre-execution computation yields a valid ΦO certificate without either assuming w or reverting to a worst-case bound. Alternatively, if the certifier defers the ΦU decision until after world-state observation is incorporated, D1 is violated: a semantically unauthorised plan may already have initiated actions by the time the joint check completes. In either sub-case, at least one desideratum is violated. The specific failure mode is the one identified in Section 2 for existing operational-layer approaches: ODD membership is assumed rather than verified, or semantic authorisation is not a true pre-execution gate. Case B: Collapse IO and IF (single execution-time stage). A certifier that simultaneously decides ODD membership (a deterministic binary verdict required for downstream composability) and enforces dynamical safety at control-loop frequency entangles two logically and temporally distinct decisions. The ODD verdict—which must be stable and auditable as it constitutes assumption AF for the dynamical controller—is then contingent on real-time trajectory data that changes at every control step. The result is that AF is no longer a well-formed, stable assumption: the dynamical controller cannot rely on a fixed operational certificate, and the contract chain ΓO ⇒ AF is broken. Additionally, collapsing these stages removes the autonomy-envelope governance function of the Operational Layer entirely, since autonomy decisions require world-state estimation without yet having trajectory data. Case C: Collapse IU and IF , leaving IO separate. A certifier must simultaneously check ΦU — which requires only IU and must be completed before any world observation (D1)—and ΦF —which requires IF and is only available during control-loop execution (D3). These two tasks operate at strictly different points in time and over strictly disjoint information increments. Any implementation that performs both necessarily executes them sequentially: first the IU -based check, then the IF -based enforcement. This sequential computation implicitly defines a boundary between IU and IF within the nominally single component—reconstructing the two-stage structure rather than eliminating it, while leaving IO -based certification unaddressed. Combining cases. In Case A, ΦO or ΦU is uncertified; in Case B, the contract chain is broken at ΓO ⇒ AF ; in Case C, ΦF or ΦU is deferred, or the two-stage structure is implicitly reconstructed. All three cases are exhaustive over the possible two-stage designs. None achieves simultaneous satisfaction of D1, D2, and D3 without either violating one or reconstructing the three-stage structure.

B.4

Necessity Corollary and the Reconstruction Remark

Corollary 1 (Three stages are necessary). The three-layer architecture of Section 3 is the minimum number of independently certified stages that can simultaneously satisfy D1, D2, and D3 under any operationally non-degenerate D. Any two-stage design necessarily violates at least one desideratum or implicitly reconstructs the three-stage structure. Additional stages may refine the architecture—for instance by splitting IU into separate cognitive, regulatory, and ethical sub-layers—but cannot reduce the minimum. Remark 2 (On implicit reconstruction). The Collapse Argument shows that a sufficiently powerful single certifier that attempts to handle all three safety dimensions must perform staged computation, implicitly partitioning its input into the three information sets at different points in time. A single component that does this is a three-layer architecture with internal boundaries hidden from external inspection—harder to independently certify, harder to audit, and with no compositional safety bound available between stages, since the intermediate guarantees are not made explicit. Making the 19

boundaries explicit is therefore not a taxonomic preference but a certification requirement: without explicit boundaries, the layer-level probabilities pU , pO , pF and the conditionals pO|U , pF |OU entering bounds (B1)–(B4) cannot be independently estimated or verified. Remark 3 (On the strength of the desiderata). One might ask whether D1–D3 could be weakened: specifically, whether a sufficiently powerful single certifier on IO could perform two-phase computation, first checking ΦU using only its IU -measurable component, then checking ΦO , thereby handling two safety dimensions without explicitly separating the layers. The Collapse Argument shows this does not succeed: such a two-phase certifier reconstructs rather than avoids the layered structure (Case C of Proposition 1 and Remark 2), and leaves ΦF uncertified in any case. The desiderata D1–D3 are grounded in physical causality—the irreversibility of action and the causal precedence of observation over inference [23]—rather than in a design choice about A/G contract semantics, and cannot be dissolved by redefining the certification boundary. B.5

Bound Degradation Under Collapse

The Collapse Argument (Proposition 1) establishes that any two-stage design must violate at least one of the desiderata D1–D3, or implicitly reconstruct the three-stage structure. The following proposition establishes the quantitative counterpart: the conditional factors of (B4) are not merely degraded but structurally unverifiable under any genuine two-stage collapse. Together, Propositions 1 and 2 close the logical gap between the necessity argument of Section 2 and the quantitative framework of Section 3.2: the three-layer architecture is not only the minimal structure satisfying D1–D3, but the unique structure under which (B4) can be independently verified as a product of auditable layer-level quantities. Proposition 2 (Certification Degradation Under Collapse). Let pU · pO|U · pF |OU denote the chainrule decomposition (B4) of the three-layer architecture. Under any two-stage collapse (Cases A, B, C of Proposition 1), at least one factor of (B4) is either undefined as a well-formed conditioning event or unverifiable as an independently certified probability. Consequently, no genuine two-stage design yields (B4) as a product of independently estimable, auditable layer-level quantities. Proof. We treat each collapse case in turn. Case A: Collapse of IU and IO (single pre-execution stage). The combined certifier must issue the ODD-membership verdict (ΦO ) using only IU -measurable information. By Remark 1, ΦO is not IU -measurable: two executions (p, w, x) and (p, w′ , x′ ) sharing the same plan are IU indistinguishable yet can have opposite ODD-membership outcomes (w ∈ WODD , w′ ∈ / WODD ) whenever α := D(w ∈ / WODD ) > 0. A certifier confined to IU therefore cannot label execution traces with ΓO outcomes—which requires observing the world state w—and hence cannot estimate pO|U = Pr(ΓO | ΓU ) from pre-execution information. Any estimate must take one of three forms: (a1) assume w ∈ WODD throughout, yielding an unverified surrogate rather than a certified conditional; (a2) adopt a worst-case bound, blocking most executions and rendering the certificate vacuously conservative; or (a3) integrate over a prior π(w), yielding the marginal pO rather than the conditional pO|U , since the conditioning event ΓU carries no information about w when the certifier cannot observe w. In all three sub-cases, pO|U is not independently estimable as a conditional quantity grounded in observed world states. The factor is unverifiable. Case B: Collapse of IO and IF (single execution-time stage). The factor pF |OU = Pr(ΓF | ΓO , ΓU ) requires ΓO to be a stable, pre-actuation event: the ODD verdict must be issued and fixed before the Functional Layer begins executing, so that it constitutes a well-formed conditioning event for ΓF . In a genuine collapse of IO and IF , however, there is no pre-actuation point at which an ODD verdict can be issued independently of trajectory data: any such verdict would require IF information (the trajectory x(t)) that does not yet exist, or it would require separating the IO -based sub-process from the IF -based sub-process within the nominally single component. If the verdict is issued before x(t) exists by separating the two sub-processes, the architecture has reconstructed the explicit IO /IF boundary (Remark 2) and is not a genuine two-stage design. If the verdict is not separated, ΓO is determined jointly with ΓF as the trajectory unfolds, making the conditioning event in pF |OU trajectory-indexed and therefore not a fixed, pre-actuation event against which ΓF can be independently certified. The factor pF |OU is undefined as a static certification quantity, and (B4) cannot be written as a well-formed product. 20

Case C: Collapse of IU and IF , leaving IO separate. Since the Operational Layer remains independent, ΓO is well-defined and pO|U is estimable. The issue concerns the joint certification of pU and pF |OU . By Proposition 1 Case C, any certifier that simultaneously handles ΦU (pre-execution, IU -measurable) and ΦF (control-loop, IF -measurable) must execute these checks sequentially, implicitly defining an internal boundary. Two sub-cases arise. Sub-case (a): the boundary is hidden. Both ΓU and ΓF are outputs of the same component; their failure events FU and FF are generated by the same internal process. Independent certification of pU and pF |OU —as required by the modular guarantee structure of Appendix A.2, which composes “locally certifiable bounds into a system-level guarantee”—requires each probability to be estimated from an independently auditable sub-system. With a hidden boundary, neither can be audited without auditing the other. In particular, the partialdecoupling strategy of Section 4—deriving ΓF from a CBF certificate whose dynamics model does not share parameters with the LLM reasoning engine that produces ΓU [43]—is unavailable: both verdicts originate in the same component, so the cross-term Pr(FU ∩ FF ) is uncontrolled and cannot be bounded by partial independence. The factors pU and pF |OU in (B4) are not independently verifiable, and (B4) cannot be certified as a product of independently audited quantities. Sub-case (b): the boundary is made explicit. If the component separately certifies its IU -based sub-process and its IF -based sub-process, with each independently auditable, then—combined with the separate Operational Layer—the architecture has reconstructed the full three-stage structure (Remark 2). This is not a genuine two-stage design. Combining cases. In Case A, pO|U is not independently estimable. In Case B, the factor pF |OU is undefined as a well-formed conditioning event. In Case C, either pU and pF |OU are not independently certifiable (sub-case a), or the three-stage structure has been reconstructed (sub-case b). Cases A, B, C are exhaustive over all two-stage designs (Proposition 1). In no case does a genuine twostage architecture yield (B4) as a product of independently estimable and auditable layer-level quantities. Remark 4 (Structural and certification necessity are equivalent). Proposition 1 establishes that three independently certified stages are the minimum satisfying D1–D3 (structural necessity). Proposition 2 establishes that three independently certified stages are necessary and sufficient for (B4) to be a well-formed, independently verifiable certification target (certification necessity). These are not two separate arguments that happen to agree; both follow from the same temporal ordering τU < τO < τF of certifiable information.

C

Numerical Instantiation of the Safety Bounds

This appendix instantiates bounds (B1)–(B4) on the caregiver-robot running example using illustrative but empirically grounded estimates. The purpose is not to certify a real deployment but to show that the bounds produce meaningful numbers, that the conditional structure of (B4) can yield quantifiable gains when upstream filtering is effective, and that the architecture’s sensitivity to individual layer weakening is concrete rather than asymptotic. C.1

Layer-Level Probability Estimates

We assign marginal satisfaction probabilities anchored to published empirical results for comparable deployed components. User Layer (pU = 0.95). ShieldAgent achieves 90.1% rule recall on ShieldAgent-Bench across six web environments and seven risk categories, outperforming prior methods by 11.3% on average [12]. More directly relevant to the embodied deployment context of the running example, AgentSpec eliminates all hazardous actions across every unsafe task category in embodied agent experiments (100% hazard prevention rate) and prevents unsafe executions in over 90% of code-agent cases [57]. We set pU = 0.95, conservative relative to AgentSpec’s embodied result, to account for residual exposure to informal intent, policy ambiguity, and adversarial manipulation [21]. Operational Layer (pO = 0.90). Without a dedicated ODD monitor, frontier LLMs fail to detect 12.24% of direct out-of-domain queries under non-adversarial conditions, rising to 70.72% under adversarial transformations [32]. We set pO = 0.90, assuming a purpose-built deterministic ODD 21

checker (augmented with robust conformal methods [66]) rather than a raw LLM call, consistent with the deterministic-verdict design of Section 3. The gap 1 − pO = 0.10 represents residual failure under non-adversarial ODD boundary conditions. Functional Layer (pF = 0.92). CBF-constrained quadratic programs embedded in LLM planning loops reduce safety violations substantially while maintaining task efficiency in hardware experiments involving heterogeneous robotic agents [30]. Probabilistic CBF extensions now provide explicit bounds on safety-failure probability for learned barriers [37, 53]. We set pF = 0.92, reflecting near-collision-free operation under moderate dynamic uncertainty; the 8% gap captures LLM nondeterminism, stochastic actuation, and scenarios such as the rearranged furniture in Room 18 requiring mid-execution replanning. C.2

Conditional Estimates and Upstream Filtering

The chain-rule decomposition (B4) requires estimating the conditional probabilities pO|U and pF |OU . These conditionals reflect the performance of each layer on inputs that have passed upstream certification. Whether these conditionals exceed the corresponding marginals is an empirical property of the deployment, not a structural guarantee (see footnote in Section 3.2). In the running example, it is plausible that User Layer filtering improves downstream performance: blocking rest-hour entries, camera-routing plans, and plans omitting conditional water-delivery logic removes 4–6% of plans that would require non-trivial ODD adjudication. We therefore consider the illustrative estimates pO|U = 0.94 and pF |OU = 0.96, representing a scenario in which upstream filtering is beneficial. For comparison, we also evaluate (B4) under the conservative assumption pO|U = pO = 0.90 and pF |OU = pF = 0.92 (no filtering benefit), which recovers the naïve product. Estimating these conditionals precisely from non-i.i.d. traces remains the open problem of Section 4, regardless of whether they exceed the marginals. C.3

Bound Evaluation

Table 1 evaluates (B1)–(B4) under (pU , pO , pF ) = (0.95, 0.90, 0.92) and (pO|U , pF |OU ) = (0.94, 0.96). For (B2) we use pairwise co-failure rate ρ = 0.02 for all i < j, reflecting moderate positive correlation from the shared LLM backbone (Section 4); sensitivity to ρ is analysed in Section C.5. Table 1: Safety bounds on the caregiver-robot example. Marginals: (pU , pO , pF ) = (0.95, 0.90, 0.92). Conditionals: pO|U = 0.94, pF |OU = 0.96. Co-failure rate (B2): ρ = 0.02. Triple co-failure (B3): Pr(FU ∩ FO ∩ FF ) = 0.004. All quantities illustrative; see text for empirical grounding. The naïve product assumes independence and no filtering benefit (pO|U = pO , pF |OU = pF ). The gap between (B4) and the naïve product is deployment-dependent; see Section C.4. Bound

Value

Inputs required

max(0, pU + pO + pF − 2) Generalised BF [51] + structural constraints§ P (B1) + i<j Pr(Fi ∩ Fj ) 1 − Pr(FU ∪ FO ∪ FF ) pU · pO|U · pF |OU

0.770 ≥ 0.770 0.830 0.826† 0.857

Marginals only Marginals + layer-independence facts Marginals + pairwise co-failure Full inclusion-exclusion Conditionals; exact, no independence

Naive product (independence assumed, no filtering): pU · pO · pF

0.787

Marginals + independence

(B1) (B1+ ) (B2) (B3) (B4)

Formula / note

† (B3) uses Pr(FU ∩FO ∩FF ) = 0.004. Computed as 1 − (fU + fO + fF − 3ρ + 0.004) = 1 − (0.05 + 0.10 + 0.08 − 0.06 + 0.004) = 1 − 0.174 = 0.826. Sensitive to the triple co-failure rate, which requires full trace data to estimate. §

(B1+ ) improves on (B1) when structural constraints such as bounds on the number of layers that can simultaneously fail are available. The CBF certificate of the Functional Layer is derived from a dynamics model that does not share parameters with the LLM backbone [43], making a bound on Pr(FU ∩ FF ) practically estimable and yielding a tightened floor above 0.770.

22

C.4

Three Observations

(i) When upstream filtering is beneficial, the gain is substantial and compositional. Under the illustrative conditional estimates, the architecture achieves pU · pO|U · pF |OU = 0.95 × 0.94 × 0.96 = 0.857, compared with the naïve product pU ·pO ·pF = 0.95×0.90×0.92 = 0.787 under independence and no filtering benefit. The 7-point gap in this scenario arises from the conditional structure of (B4) and illustrates the potential value of upstream certification. Whether this gap is realised in a given deployment is an empirical question; the architecture’s contribution is that it makes the relevant conditionals well-defined and independently estimable, enabling this question to be answered. (ii) (B1) is the near-term deployable floor; it can be systematically tightened. (B1) = 0.770 requires only marginal estimates, obtainable from execution traces via martingale or mixing-process bounds (Section 4), and provides a valid certification floor while the harder conditional estimates for (B4) are developed. It is however substantially more conservative than (B4) under the illustrative conditional estimates (= 0.857): the 8.7-point gap represents the cost of using marginals-only information in this scenario, and motivates investment in conditional estimation methods. Salako’s generalised Boole–Fréchet dynamic-programming framework [51] shows this floor can be systematically raised by incorporating structural constraints. The CBF certificate’s independence from the LLM backbone [43] provides exactly such a constraint for the FU ∩ FF pair, making (B1+ ) a practically reachable improvement. (iii) A single weak layer degrades all bounds irreversibly; no compensating strengthening suffices. Suppose the Operational Layer degrades to pO = 0.75 under persistent sensor degradation. Under the illustrative conditional estimates, the effect is as follows. (B1): (B1) with pU = pF = 0.99 :

max(0, 0.95 + 0.75 + 0.92 − 2) = 0.62 (was 0.770; loss of 15 pp) max(0, 0.99 + 0.75 + 0.99 − 2) = 0.73 (still 4 pp below baseline)

The (B4) picture is identical in character: with pO|U degrading proportionally to 0.94×(0.75/0.90) = 0.783, pU · pO|U · pF |OU = 0.95 × 0.783 × 0.96 = 0.714

(was 0.857; loss of 14 pp).

Strengthening pU or pF |OU to 0.99 does not recover this loss because the deficit enters multiplicatively through pO|U . This asymmetry motivates independent certification of all three layers rather than investment concentrated in one. C.5

Sensitivity to Co-failure Rate and a Caution on (B2)

Table 2 reports (B2) as ρ varies. (B2) = 0.770 + 3ρ; it remains a valid lower bound for any ρ ≤ 0.077 (the point at which B2 would reach 1). Table 2: Sensitivity of (B2) to pairwise co-failure rate ρ. (B1) = 0.770 throughout. ρ

(B2)

Gap over (B1)

0.005

0.785

+0.015

0.010

0.800

+0.030

0.020

0.830

+0.060

0.030

0.860

+0.090

A critical caution: higher ρ makes (B2) look tighter while true system risk simultaneously increases. A co-failure rate of ρ = 0.030 means all three layers share substantial failure events—the backbone can bring all three down at once—yet (B2) rises to 0.860, near (B4)’s 0.857 under the illustrative conditional estimates. This is precisely the danger identified in Section 4: under positive correlation, (B1) and (B2) are valid but potentially misleading. Until the conditional estimation gap is closed, (B4) should be treated as the certification target while (B1) and (B2) serve as conservative floors, not deployable certificates.

23

NeurIPS Paper Checklist The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit. Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist: • You should answer [Yes], [No], or [N/A]. • [N/A] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available. • Please provide a short (1–2 sentence) justification right after your answer (even for [N/A]). The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will also be asked to include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper. The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While [Yes] is generally preferable to [No], it is perfectly acceptable to answer [No] provided a proper justification is given (e.g., error bars are not reported because it would be too computationally expensive” or “we were unable to find the license for the dataset we used”). In general, answering [No] or [N/A] is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found. IMPORTANT, please: • Delete this instruction block, but keep the section heading “NeurIPS Paper Checklist", • Keep the checklist subsection headings, questions/answers and guidelines below. • Do not modify the questions and only use the provided macros for your answers. 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly state the central claim—that single-layer enforcement is structurally insufficient for LLM agent safety—and accurately scope it as a position paper presenting a formal structural argument (Appendix B), a sketched architecture with probabilistic safety bounds (Section 3), and four open problems (Section 4) rather than empirical results. Aspirational goals such as a deployable standard are clearly flagged as future work. Guidelines: • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 24

2. Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Section 4 explicitly discusses all major limitations: the framework is scoped to single-agent systems, treats the LLM as a fixed black box requiring re-estimation of probability bounds upon model change, and identifies three open problems—bound estimation from non-i.i.d. traces, graceful degradation of contracts under deployment drift, and extension to multi-agent settings—that stand between the architecture and a deployable standard. The two structural assumptions bounding applicability are stated explicitly at the opening of Section 4. Guidelines: • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate “Limitations” section in their paper. • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 3. Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: All theoretical results state their assumptions explicitly. The structural necessity argument is formalised in Appendix B with a complete proof of Proposition 1 (Collapse Argument) covering all three cases, Corollary 1, and Remarks 2–4. The probabilistic bounds (B1)–(B4) are stated with full assumptions in Section 3.2 and elaborated in Appendix A.2, including the estimation caveat and the conditions under which each bound is tighter than the others. Guidelines: • The answer [N/A] means that the paper does not include theoretical results. • All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems. 25

• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 4. Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [N/A] Justification: This is a position paper with no experiments. All empirical figures cited (AgentSafetyBench, Agent Security Bench) are drawn from published third-party benchmarks with full citations, and are used only to motivate the structural argument rather than as primary evidence for the claims. Guidelines: • The answer [N/A] means that the paper does not include experiments. • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [N/A] Justification: This is a position paper. No code or datasets are introduced or used. Guidelines: 26

• The answer [N/A] means that paper does not include experiments requiring code. • Please see the NeurIPS code and data submission guidelines (https://neurips.cc/ public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //neurips.cc/public/guides/CodeSubmissionPolicy) for more details. • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 6. Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results? Answer: [N/A] Justification: This is a position paper with no experiments or training procedures. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 7. Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [N/A] Justification: This is a position paper with no experiments. The probabilistic bounds in Section 3.2 are theoretical guarantees, not empirical measurements. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. 27

• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates). • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [N/A] Justification: This is a position paper with no computational experiments. Guidelines: • The answer [N/A] means that the paper does not include experiments. • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper). 9. Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The research conforms with the NeurIPS Code of Ethics. The paper advocates for safer deployment of LLM agents in safety-critical settings, raises no dual-use concerns, involves no human subjects, and introduces no artefacts with misuse potential. Guidelines: • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics. • If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The paper’s primary contribution is a safety architecture for LLM agents in safety-critical deployments. The positive societal impact is improved runtime assurance for autonomous systems operating near humans. Regarding negative impacts: the framework constrains rather than enables harmful behaviour, introduces no new attack surface, and the open problems identified in Section 4 are prerequisites for deployment rather than risks introduced by the work itself. One potential concern is that premature deployment of the architecture without closing the open problems of Section 4 could create false assurance; this is explicitly cautioned against in Section 4 and the Conclusion. Guidelines: • The answer [N/A] means that there is no societal impact of the work performed. • If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact. 28

• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 11. Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)? Answer: [N/A] Justification: The paper releases no models, datasets, or code and proposes no artefacts with misuse risk. Guidelines: • The answer [N/A] means that the paper poses no such risks. • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: All third-party works cited are properly referenced with full bibliographic entries. The paper uses no datasets or software assets directly; all cited benchmarks and tools are referenced for motivational purposes only, not reused or redistributed. Guidelines: • The answer [N/A] means that the paper does not use existing assets. • The authors should cite the original paper that produced the code package or dataset. • The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset. • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. 29

• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset’s creators. 13. New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [N/A] Justification: The paper introduces no new datasets, models, software, or other artefacts requiring documentation. Guidelines: • The answer [N/A] means that the paper does not release new assets. • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 14. Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [N/A] Justification: The paper involves no crowdsourcing or research with human subjects. Guidelines: • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 15. Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [N/A] Justification: The paper involves no human subjects research and therefore requires no IRB approval or equivalent. Guidelines: • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects. 30

• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 16. Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required. Answer: [N/A] Justification: LLMs are the object of study in this paper, not a component of the research methodology. No LLMs were used as an important, original, or non-standard component of the core methods, proofs, or architectural proposals. Any use of LLMs was limited to writing assistance, which does not require declaration under NeurIPS 2026 policy. Guidelines: • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components. • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.

31