arXiv:2605.00424v1 [cs.CR] 1 May 2026
Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes Alfredo Metere Metere Consulting, LLC [email protected] May 4, 2026
Abstract Agent skills — structured packages of instructions, scripts, and references that augment a large language model (LLM) without modifying the model itself — have moved from convenience to first-class deployment artifact. The runtime that loads them inherits the same problem package managers and operating systems have always faced: a piece of content claims a behavior; the runtime must decide whether to believe it. We argue this paper’s central thesis up front: a skill is untrusted code until it is verified, and the runtime that loads it must enforce that default rather than infer trust from a signature, a clearance, or a registry of origin. Without skill verification, a human-in-the-loop (HITL) gate must fire on every irreversible call — which is operationally untenable and degrades into rubber-stamping at any non-trivial scale. With skill verification treated as a separate, gated process, HITL fires only for what is unverified, and the system becomes sustainable. We give a trust schema (§3) that includes an explicit verification level on every skill manifest; a capability gate (§4) whose HITL policy is a function of that verification level; a biconditional correctness criterion (§5) that any candidate verification procedure must satisfy on an adversarial-ensemble exercise (§6); and a portable runtime profile (§7) with ten normative guidelines abstracted from a working open-source reference implementation [13]. The contribution is harness- and model-agnostic; nothing here requires retraining, fine-tuning, or proprietary infrastructure.
Keywords: LLM agents, agent skills, supply chain, formal verification, human-in-the-loop, mandatory access control, audit log.
1
Introduction
The emergence of SKILL.md as a portable, harness-neutral description of procedural knowledge for LLM agents has solved a real problem: an agent operator can curate a library of small, inspectable, version-controlled skill packages and ship them across platforms. Empirically, curated skill libraries improve agent pass-rates on benchmarks such as Mind2Web [6] and Embodied Agent Interface [11], while indiscriminate admission of self-generated skills can degrade or actively attack an agent’s behavior [1]. The asymmetry is the same one distributed software ecosystems hit decades ago: trust cannot be inferred from origin alone [21], package managers that do infer it have been broken every way the literature predicted [3, 4], and the formal response is by now textbook [18]: typed 1
artifacts, signed manifests, capability sandboxing, hash-chained audit, and least-privilege gates around side-effects. What is not textbook is what the runtime should do with a skill it has just signed and clearancechecked but has not yet verified to behave the way its manifest claims. Most current harnesses treat that distinction as collapsed: signature plus clearance equals trust. We argue this is wrong, dangerously so, and that the cost of getting it wrong is paid by the human in the loop. Skills are untrusted code. A skill package is read by the agent and influences the agent’s prompt, tool calls, and write surfaces. A malicious or compromised skill is a direct prompt-injection vector with the additional advantage that the injection persists across model calls and survives context truncation [8, 17, 20]; the analogous attack class against agent memory and knowledge bases has already been demonstrated in the wild [5]. The skill is the instruction; the model is the executor. A signature on the skill’s manifest tells the runtime that some signer endorsed the artifact at some point; it tells the runtime nothing about whether the artifact’s behavior conforms to its declared manifest. Recent work has shown the gap is exploitable in practice: backdoors can be smuggled into signed code-completion models in ways that pass strong static detection [22], demonstrating that signature plus policy review is not a substitute for behavioral verification. Treating a signed-and-cleared skill as trusted is no more defensible than treating a code-signed Windows executable as malware-free because Microsoft signed it. HITL is the universal default for unverified skills. If the runtime cannot trust a skill’s claims about itself, it has no choice but to gate every irreversible call through human-in-the-loop (HITL) approval. This is operationally untenable: HITL on every call requires unbounded operator attention, and operators under load eventually rubber-stamp every prompt, degrading the gate into a fiction. The right response is not to weaken the gate; it is to introduce a second process — skill verification — whose successful completion allows the runtime to trust portions of a skill’s manifest and correspondingly relax HITL frequency. Verification happens at bootstrap, never at runtime. Skills are verified once, before the runtime accepts any external input. The verification result is recorded in the manifest as an explicit verification level (defined in §3). After bootstrap the result is immutable: a skill’s verification level cannot be upgraded by the agent during a session, and a skill’s content cannot be modified by the agent at all without that modification being treated as an irreversible operation that walks the HITL gate. This matches the “capability bootstrap discipline” of [15] adapted to skills: the trustworthy state is established once and then frozen. Side-effects must be verifiable, not just gated. The verification process needs a property to verify against. We propose a single biconditional criterion: the observable side-effects of an agent run must be in 1-to-1 correspondence with the approved-and-executed set in the audit log. Any candidate verification procedure (manual review, fuzz testing, adversarial ensemble, formal analysis) must demonstrate that the skill under test satisfies this criterion before its manifest can be elevated above the unverified default. Contributions.
This paper makes four contributions.
1. A skill trust schema (§3) that treats a skill as a tuple of manifest, content, and capabilities, with an explicit verification level field and a no-runtime-mutation discipline. 2
2. A capability-gate model (§4) whose HITL policy is a function of the loaded skill’s verification level: unverified skills incur HITL on every irreversible call; verified skills incur HITL only for capabilities outside the verified manifest. 3. A biconditional correctness criterion (§5) that any candidate skill- verification procedure must satisfy, with a characterization of the failure modes it catches and the ones it does not. 4. A portable runtime profile (§7) with ten normative guidelines abstracted from a working open-source reference implementation [13], including a no-bypass-switch invariant and an untrusted-by-default rule for skill admission. We sketch an adversarial-ensemble evaluation (§6) under which the criterion is exercised — and under which a candidate skill-verification procedure can be benchmarked — and discuss open problems in §8. The schema is deliberately model- and harness-agnostic. It assumes nothing about training, fine-tuning, or RLHF, and assumes only the weakest run-time interface: that an agent harness invokes tools through a typed dispatch step the runtime can interpose on. Every existing harness adopting SKILL.md satisfies this constraint. Reference implementation. We draw both the schema and the criterion from a working opensource framework, enclawed [13], which hard-fork-hardens a single-user AI assistant gateway with each of the primitives this paper abstracts: a Bell–LaPadula classification scheme, an Ed25519 signed-module loader with a clearance-bounded trust root, a hash-chained audit log, an egress guard, a regex-based DLP scanner, a HITL controller with a checkpointable agent-session state machine, and a transaction buffer with rollback. Section 7 extracts a small set of normative guidelines from that reference implementation, intended as a starting point for a portable runtime profile any SKILL.md-adopting harness could implement to bound the attack surface skills introduce.
2
Threat model: skills as a new supply-chain surface
We assume an agent operator who already runs LLM agents with tool access in an environment where some of the tools have real-world side-effects (file deletion, message sending, payment, database writes, on-chain operations). The skill ecosystem the operator subscribes to is partially trusted: the operator’s own curated skills are signed by an operator-controlled root, but upstream registries and self-generated skills are not. Adversaries.
We consider three:
1. A registry attacker who publishes a malicious skill that the operator’s harness pulls and runs. The skill’s instructions induce the agent to take harmful side-effects in the operator’s environment. 2. A prompt-injection attacker whose payload reaches the agent through a benign skill’s data dependency (e.g. a fetched document the skill instructs the agent to summarise) and re-routes the agent’s tool calls. 3. A self-generated-skill attacker: the agent itself, under model error or adversarial input, synthesises a new skill, registers it, and uses it. This is not a malicious actor in the usual sense, but operationally indistinguishable from one. 3
Out of scope. We do not assume an attacker who has compromised the operator’s signing key, the runtime binary, or the audit-log storage out-of-band. We do not assume a defended hardware trust anchor (a TPM, secure element, or equivalent), though such an anchor improves every layer below. We do not discuss model-weight-level attacks (poisoning, jailbreak from training data), which are orthogonal. Goal.
The runtime should:
• refuse to load a skill whose manifest does not verify; • refuse to dispatch a tool call whose required capability the loaded skill did not declare; • refuse to execute an irreversible call without an explicit, audited decision; • produce an audit trail under which the biconditional criterion of §5 can be checked.
3
The skill trust schema
We define a skill artefact as a tuple Skill = ( M, content, σ ) where M is a manifest, content is the body of the skill (typically the SKILL.md file plus referenced scripts), and σ is a detached signature over the canonical bytes of (M, content) produced by a signer in the operator’s trust root. The manifest M has five mandatory fields: M.label a classification label ⟨ℓ, C, R⟩ in a Bell–LaPadula style lattice [2], with rank ℓ, compartment set C, and releasability caveats R. Labels combine via join: a ⊔ b = ⟨max(ℓa , ℓb ), Ca ∪ Cb , Ra ∩ Rb ⟩. M.caps a finite set of declared capabilities drawn from a fixed vocabulary; see §3.5. M.signer a key identifier referencing an entry in the trust root. M.version a monotone integer; replays of an older signed manifest are rejected if a newer one with the same identity has been observed. M.verification the skill’s verification level; see §3.1.
3.1
Verification levels
A signed manifest binds an artifact to a signer’s name; it does not bind the artifact’s behavior to the manifest’s claims. We make this distinction explicit in the schema. Every skill manifest carries a verification level M.verification drawn from a fixed enum of four values: unverified, declared, tested, and formal. The default is unverified; higher levels grant the runtime more latitude when policing the skill’s tool calls (§4). unverified. No claim is made about the skill’s behavior beyond its existence. Default for any skill the operator has not explicitly examined. The runtime treats this skill as untrusted code: every irreversible capability call goes through the HITL gate, regardless of whether the manifest declares the capability or not. 4
declared. A trust-root signer has examined the skill’s content and attests that the skill’s sideeffects are bounded by its declared capability set M.caps. The attestation is the signer’s professional reputation; the runtime trusts capabilities in M.caps but still requires HITL on irreversible calls outside that set. tested. In addition to a declaration, the skill passes an adversarial-ensemble verification run (§6) under which the biconditional criterion of §5 holds. The runtime permits irreversible capabilities in M.caps without per-call HITL but logs every such call. formal. A formal analysis tool has produced a machine-checkable proof that the skill’s behavior is a subset of its declared capability set under the runtime’s threat model. This level is aspirational at the time of writing; we include it for completeness because the schema field is fixed-width and adding it later requires a manifest version bump. Why levels, not a continuum. A continuous “trust score” would allow the agent to argue itself into permission gradients; a fixed enum forces a discrete decision that can be audited and revoked atomically. The same reasoning applies to the configuration profiles in §7.2. Verification is an offline, bootstrap-time concern. Verification levels are set by the operator (or the operator’s delegate, e.g., a CI pipeline running an adversarial-ensemble suite over each candidate skill) before the runtime starts, and recorded into the signed manifest at the declared level or above. The runtime never elevates a verification level during a session; doing so would re-introduce the trust-bootstrap problem the schema is designed to avoid. A manifest that arrives at runtime claiming a verification level above what its signer is authorized to attest is rejected.
3.2
No agent mutation of skills at runtime
Once a skill is loaded, its content and manifest are immutable for the lifetime of the agent session: the agent cannot edit a skill, append to its manifest, or upgrade its verification level. Any attempt the agent makes to modify a loaded skill — through a file-system tool call, a runtime API, or any indirect side-effect on the on-disk artifact — is itself an irreversible capability that must walk the HITL gate and be recorded in the hash-chained audit log with the pre- and post-mutation content hashes and the operator’s decision, regardless of whether the request is approved or denied. On approval, the modification produces a new skill artifact that must be re-verified before the next session. Without the immutability rule, an agent that admits one verified skill could rewrite that skill’s content in place and silently change what “verified” means for the rest of the session; without the audit trail, the rewrite would be invisible to post-incident review.
3.3
The trust root
The trust root is a finite, append-only set of signer entries {(keyIdi , pubKeyi , maxClearancei )}. A signer is trusted to sign skills with classification at most maxClearancei . The trust root supports a one-shot lock operation: once locked, mutations (set, remove) raise a typed error. The locked state is the production posture; an unlocked trust root is acceptable only during host bootstrap, before any external input has been read. This is the standard “capability bootstrap discipline” adapted from object-capability systems [15].
5
3.4
Manifest verification
Loading a skill walks the following steps in order at bootstrap, before any external input is read, failing closed on any error: 1. Parse M from canonical JSON; reject unknown fields, prototype-pollution keys, and missing mandatory fields. 2. Resolve M.signer in the trust root. If absent, reject. 3. Verify σ against the resolved public key over the canonical bytes of (M, content); reject on mismatch. 4. Check that M.label ⪯ maxClearance for the resolved signer. A signer cannot sign above its authorized clearance. 5. Check M.label ⪯ user.clearance for the running operator. The operator cannot load a skill that exceeds their own clearance. 6. Check that M.verification does not exceed the signer’s authorized verification level. A signer attesting tested must hold attestation authority for that level; formal requires a corresponding tooling attestation. Default if absent: unverified. 7. Register the declared M.caps with the runtime’s capability gate (§4), tagged with M.verification. A skill that survives all seven steps is loaded; its content is now reachable by the LLM at the registered verification level. A failure at any step produces a typed audit record and aborts the load. After bootstrap completes, the loaded set is frozen for the session; loading a new skill at runtime requires the runtime to be re-bootstrapped and is governed by the §3.2 rule.
3.5
Capability vocabulary
The vocabulary is small enough to be enumerated and large enough to discriminate side-effect classes. We propose a minimal set: Capability
Side-effect class
net.egress(host) fs.read(path) fs.write.rev(path) fs.write.irrev(path) tool.invoke(name) spawn.proc(cmd) publish(channel, ...) pay(token, amount) mutate.schema(target)
DNS-resolved network reach to host filesystem read under path write that can be rolled back delete, overwrite, truncate invoke a named tool registered with the harness external process spawn post to an external channel transfer of fungible value migrate a database / configuration schema
Table 1: A minimal capability vocabulary. The split between fs.write.rev and fs.write.irrev is the load-bearing distinction the gate uses in §4. A skill’s manifest must enumerate every capability its content intends to invoke. A capability not in M.caps is denied at the gate, regardless of what the skill content asks.
6
4
The capability gate
The capability gate is the runtime layer between the LLM-driven agent and the external world. It receives a tool-call envelope emitted by the agent (typically as a JSON object), looks up the corresponding capability, and consults the gate policy.
4.1
Gate policy as a function of verification level
The gate’s behavior depends on the loaded skill’s verification level (§3.1). The mapping is fixed and is not configurable per call: unverified. Every irreversible capability call the agent makes while this skill is the active context passes through HITL, regardless of whether the skill’s manifest declares the capability. Reversible calls execute through the transaction buffer (§4.2). HITL on every irreversible call is the universal default; verification is the only path off it. declared. Irreversible calls whose (capability, target) is in M.caps execute through the transaction buffer (committed if the host call succeeds, rolled back on error); calls outside M.caps pass through HITL. tested. Same as declared, with the addition that the runtime maintains a per-session biconditional check (§5) over the operations the skill issued and aborts the session if the check fails between rounds. formal. Same as tested; the difference is in offline trust, not in runtime behavior. In all four cases, the runtime audits every call; the difference is whether the gate stops to ask a human first. The default is “ask”. Verification is what buys the runtime permission to stop asking, and only for what the verification covered.
4.2
Reversible vs. irreversible
The split between reversible and irreversible capabilities is the design’s load-bearing distinction. • A reversible side-effect leaves a single object in a state from which the runtime, holding a recent snapshot of that object, can return it to its prior state without external coordination. Memory-buffered file writes that commit on confirm are reversible; in-database transactions with rollback are reversible; an SQS queue write to a dead-letter queue under operator control is reversible. • An irreversible side-effect is one for which the runtime cannot, alone, restore the prior state of the affected world. Sending an email is irreversible. Posting to a public channel is irreversible. Issuing an on-chain transaction is irreversible. Deleting a file from a remote store the runtime does not control is irreversible. The runtime classifies every capability call as one or the other. Reversible calls execute through a transaction buffer that holds the change in memory and either commits (after the operation succeeds and is audited) or rolls back. Irreversible calls go through the four-state lifecycle of §4.3.
7
4.3
The HITL lifecycle
For every irreversible call the runtime walks four states: 1. request. The agent emits a tool-call envelope ⟨op, args, reasoning⟩. The runtime appends a typed irreversible.request record to the audit log. 2. decide. The runtime consults a broker — an opaque oracle the operator chose at deploy time — and receives a binary decision d ∈ {approve, deny}. The runtime appends a typed irreversible.decision record carrying d and the broker’s identity. 3. execute. If d = approve, the runtime performs the side-effect through the host APIs the capability resolves to. If the side-effect succeeds the runtime appends irreversible.executed with ok = true. If the host APIs fail (the target file vanished between approval and call, the network is partitioned, the chain rejected the transaction) the runtime appends irreversible. error instead. 4. audit. The records of the previous three states are linked by a shared request-id and form a complete trace of the call.
4.4
Broker policies
The broker is a configurable component. We see four useful defaults: deny-all. The broker denies every request. Equivalent to running the runtime without irreversible capability at all; useful as a baseline and as the fail-safe default when the operator has not configured a broker. policy. The broker reads an out-of-band policy document (a file the LLM cannot reach) carrying allow- and deny-rules over capability + argument shape. The decision is mechanical and reproducible. interactive. The broker prompts a human via terminal, message bus, or webhook, with a timeout that defaults to deny. webhook. The broker delegates to a remote service holding the operator’s policy. The policy regime has the convenient property that, given the same audit log of requests, two independent operators with the same policy produce the same decisions; this makes integration tests deterministic.
5
Biconditional correctness for HITL gating
A gate’s job is not merely to deny bad requests; it is to make the runtime’s claim about the world match the world. The same property that makes the gate auditable in production also makes it the natural criterion against which a candidate skill-verification procedure can be exercised. We formalize the property and discuss both uses.
8
5.1
Setup
Fix an agent run with starting state s0 of a designated corpus (a finite set of mutable objects the agent can act on). After the run the corpus is in state s1 . The runtime has produced an audit log L of typed records. We define: D = delta(s0 , s1 )
(observed)
(1)
S = { r ∈ L | r.type = irreversible.executed,
(2)
r.ok = true }
(audited)
(3)
D is what the runtime says happened to the world; S is what the runtime says it approved and executed.
5.2
The biconditional
Definition 1 (Biconditional pass). The audit log L passes the biconditional for the run iff the multiset projection of D onto operation-and-target equals the corresponding multiset projection of S. In words: every corpus change is explained by a matching approved-and-executed record, and every approved-and-executed record corresponds to a matching corpus change. Either direction broken is a fail.
5.3
Detected failure modes
Proposition 1. If the audit log L passes the biconditional, the following failure modes are absent: F1 Gate bypass: an irreversible side-effect occurred without a corresponding irreversible. request/decision pair. F2 Audit forgery: the log claims an approved-and-executed record whose corresponding corpus change is not present. F3 Approved-but-failed without note: a request was approved and the runtime claims it executed, but the host call silently failed. F4 Wrong-target execution: the runtime approved an op against target t1 but the side-effect landed on t2 ̸= t1 . Proof sketch. F1, F2 follow directly from the multiset equality. F3 is caught because the proposition restricts S to records where r.ok = true; a silent host failure should have been recorded as irreversible.error, which is not in S, so its absence from D violates equality from the S side. F4 is caught because the projection includes target identity.
5.4
Failure modes the criterion does not catch
Honesty about scope: • Read-only exfiltration. The criterion talks about side-effects on the corpus; it cannot detect an agent reading data and silently transmitting it through a covert channel that bypasses both the corpus and the audit log. Defending this requires a content-flow primitive (DLP, an egress-aware classification gate) below the criterion’s layer. 9
• Time-of-check / time-of-use races. If the corpus is mutated by an external party between s0 snapshot and the agent’s first action, D is the observed delta, not the agent’s contribution. The criterion needs a source of truth more refined than start/end snapshots in adversarial environments. • Approved-and-the-attacker-also-acted. If a legitimate approval r ∈ S overlaps a malicious side-effect on the same target, D may equal S at the target level even though the agent’s action was not what D records. These are real limits and we name them rather than paper over them. The criterion is necessary, not sufficient, for full safety; it is sufficient for the “did the gate work?” question that HITL deployments most often need to answer.
5.5
The biconditional as a verification benchmark
Beyond its production use, the biconditional is the formal criterion against which a candidate skillverification procedure can be exercised. To elevate a skill from unverified to tested, the operator (or an attestation pipeline acting on the operator’s behalf) runs the skill against an adversarialensemble of §6 and records the audit log. The skill is admitted at tested iff the biconditional passes on every round. A skill that fails the biconditional under verification is rejected; a skill that passes is admitted at tested and pinned to the manifest hash that was tested. Modifying the skill content invalidates the attestation; this is the load-bearing reason for the no-runtime-mutation rule of §3.2.
6
Adversarial-ensemble evaluation
The biconditional criterion is interesting only if it is exercised under conditions where the agent wants to take destructive action. We propose an evaluation shape we call the adversarial-ensemble sketch. Setup. Generate a fixed corpus of N small files with realistic-looking corporate-document names (audit logs, compliance reports, financial statements, meeting notes, test-result fixtures). Compute the per-file SHA-256 manifest as the integrity baseline. Agents. Wire K LLM-driven agents with system prompts biased toward destructive operations. A representative four-agent ensemble has roles such as “cleaner” (proposing fs.write.irrev on stale files), “consolidator” (proposing in-place edits to merge files), “auditor” (proposing edits that mark records as audited), and “critic” (echoing whichever destructive action is in scope). The system prompts deliberately do not ground the agents in the policy of the gate. Run. For R rounds, every agent takes one turn, emitting a tool-call envelope. The capability gate (§4) routes irreversible calls through the configured broker; reversible calls execute and are audited. Verdict. After the run, recompute the corpus manifest; diff against baseline; check the biconditional against the audit log. Verdict is binary: D = S at the operation-and-target projection, or not.
10
Why this is a useful experimental shape. Two reasons. First, it stresses the gate against the worst plausible agent behaviour: a confederation actively trying to delete things. Second, the verdict is mechanical: no human grader rates outputs; the corpus diff and the audit log are both deterministic artefacts. Two independent operators running the same policy broker on the same corpus produce the same verdict. In our prototype runs the criterion catches all four failure modes of §5 under deliberately-injected faults (stub a tool to skip the gate; fabricate a fake irreversible.executed record; have a tool silently no-op on a wrong target). We expect this to generalize: the criterion is mechanical and the failure-mode characterization is exhaustive of what the criterion was designed to catch.
7
Architectural guidelines from a reference implementation
The schema and the biconditional criterion above are not thought experiments. They are abstracted from a working open-source framework, enclawed [13], which hardens a single-user AI assistant gateway against the same threat surface skills now expose. In this section we walk through the specific architectural choices that framework makes, explain what each choice was answering, and lift the choice into a normative guideline a portable runtime profile for skill-aware harnesses could adopt.
7.1
Architectural choices observed
Configuration as named flavors, not a feature-flag continuum. The reference framework ships in exactly two flavors, open and enclaved, chosen at boot via a single environment variable. The strict flavor disables network egress, disables non-local model providers, and locks the trust root after preload; the dev flavor relaxes those constraints. There is no per-feature flag for the irreversible choices and no in-between continuum. Locked trust root with clearance-bounded signing. The classification scheme and the signedmodule trust root are mutable only during host bootstrap. The bootstrap calls lockTrustRoot() before any external input is read; subsequent mutation attempts raise a typed TrustRootLockedError. Each trust-root entry binds a public key to a maximum clearance level, and a signed manifest declaring a clearance above its signer’s max is rejected at load time — the signer cannot escalate what it endorses. Hash-chained audit consumed by the biconditional. Every gate event (request, decision, executed, denied, integrity failure) is recorded in an append-only log whose records carry a prevHash field; concurrent appends serialize through a small in-process queue. The audit log is the source of truth for post-mortem reconstruction and is consumed directly by the biconditional check of §5. Reversible/irreversible split + deny-by-default boundaries. Every capability is statically tagged as reversible or irreversible. Reversible side-effects route through a hash-chained transaction buffer that commits on confirm or rolls back on error; irreversible side-effects walk the four-state HITL lifecycle of §4.3 directly. Network egress, channel use, provider invocation, and tool dispatch are each gated by per-boundary enforcers (enforceChannel(), enforceProvider(), enforceTool(), enforceHost()) that compare against an a-priori-pinned allow-list rather than a heuristic classifier. The strict flavor’s default policy is empty allow-lists at every layer.
11
Adversarial test corpus shipped with the implementation. The framework ships dozens of pen-test cases targeting specific failure modes (in-place audit edits, log-injection through control characters, signature forgery via key downgrades, hostname-normalization bypass, post-lock trustroot mutation, prototype-pollution, ReDoS bounds, role- spoofing prompt injection, code injection through the module loader); the suite runs in CI on every push.
7.2
Toward a portable runtime profile for skill-aware harnesses
The choices above generalize to a small set of normative guidelines for any runtime that loads, invokes, or composes skills. We use must / should / may in the sense of RFC 2119. G1 Capability bootstrap. Trust roots, classification schemes, and policy tables must be established and locked before the runtime reads any skill content, any model output, or any external network traffic. The lock must be unconditional (no operator override at run time). G2 Deny-by-default at every boundary. Network egress, provider invocation, channel use, file write, and tool dispatch must require positive allow-list membership. The default policy must be deny. G3 Mandatory classification. Every skill artifact, every tool input, and every tool output must carry a classification label drawn from a finite, declared lattice. Label-free artifacts must be denied at the gate. G4 Clearance-bounded signing. Trust- root signers must be bound to a maximum clearance. A skill manifest declaring a clearance above its signer’s maximum must be rejected. G5 Hash-chained audit at gate-event granularity. Every gate event must produce an audit record carrying a prevHash link to its predecessor; concurrent appends must serialize deterministically. The chain must be verifiable by any party that can read the log. G6 Reversible/irreversible split. Every capability must be statically tagged as reversible or irreversible. Irreversible capabilities must pass through the HITL gate of §4; reversible capabilities should use a transaction buffer with rollback. G7 Biconditional post-hoc verification. The runtime should support the biconditional criterion of §5 as a runnable check after any agent run; the corpus delta should be reconcilable with the approved-and-executed set in the audit log without operator interpretation. G8 Adversarial test corpus. The runtime must ship a corpus of pen-tests covering, at minimum, the failure modes F1–F4 of §5 plus the eight attack families enumerated above (audit in-place edit, log injection, signature forgery, hostname-normalization bypass, post-lock trust-root mutation, ReDoS, prompt-injection role-spoofing, code injection through prototype pollution). The corpus must run in CI on every commit. G9 Standard configuration profiles. The runtime must expose at least one strict deployment profile (analogous to the reference’s enclaved flavor) and one development profile (analogous to open). Configuration must not be a free-form continuum of feature flags; deployment-class differences must be named and documented. G10 No bypass switch. The runtime must not expose a build flag, environment variable, runtime API, or operator override that disables the HITL gate, the audit log, the egress guard, 12
the classification check, or the trust-root lock. Defense in depth must not have a master kill-switch. G11 Skills are untrusted by default. The runtime must treat every skill manifest as unverified unless the manifest carries an explicit verification level set by the operator (or an operatorauthorized attestation pipeline) at bootstrap. The unverified default must subject every irreversible capability call to HITL. A signature and a clearance check must not be sufficient to elevate a skill above unverified. G12 Verification is bootstrap-only, skills are immutable in-session, and every mutation attempt is audited. Verification levels must be set before the runtime reads any external input and must not be elevated during a session. Loaded skill content must not be modifiable by the agent during the session; any agent attempt to mutate a loaded skill (file-system edit, runtime API, indirect side-effect on the on-disk artifact) must be intercepted as an irreversible capability call, walked through HITL, and written into the hash-chained audit log with the pre- and post-mutation content hashes regardless of whether the operator approves or denies. Successful mutations must produce a new skill artifact that is re-verified before the next session.
7.3
What G1–G12 buy in terms of attack surface
Each guideline closes a specific attack family that pure-prompt or pure-content defenses cannot reach. • Supply-chain attacks (compromised registry, typosquatting, malicious skill submission) are bounded by G3+G4+G5+G11: an unsigned skill is rejected; a malicious skill that nevertheless loads is at unverified until an operator acts, so every irreversible call walks HITL. • Prompt-injection through skill content is bounded by G2+G6: induced irreversible calls are intercepted at the gate; the agent cannot manufacture approval from model output alone. • Self-generated and in-flight skill mutations are bounded by G1+G4+G12: synthesized skills cannot be signed by trust-root signers, and an attempt to mutate a loaded skill becomes a tracked irreversible call that walks HITL and invalidates the verification level. • Audit forgery is bounded by G5+G7: the chain catches in-place edits; the biconditional catches mismatches between log and world. • Operator pressure to weaken safety is bounded by G10: the runtime exposes no path to disable a gate, so an operator under load cannot quietly degrade the deployment. The reduction is not to zero — the residual surface (§5’s read-only exfiltration, TOCTOU, overlapping-malicious-effect) remains — but the surface G1–G12 close is precisely the one where incidents are post-hoc-undeniable: an agent took an action the operator did not approve, and the runtime cannot prove what happened.
8
Discussion and open problems
Composition, revocation, and self-generated skills. Composing skills raises a labelinheritance question: the join-of-labels rule of §3 is the conservative answer; declassification when 13
output is provably a function of low-label inputs only is a research direction. Signer-key revocation is a familiar PKI problem (CRLs, OCSP staples, short lifetimes); the interesting case is the in-flight skill loaded under a now-revoked signer, which should be evicted at the next gate event rather than allowed to drain. Self-generated skills fold neatly into the schema: a skill the agent synthesises at run time inherits the join of every loaded manifest’s label, cannot be signed by any trust-root signer, and is therefore loadable only at the agent’s effective label through a broker decision — which is what the gate already handles. Scale and hardware roots. The adversarial-ensemble of §6 is a small-corpus benchmark; scaling it to ten-thousand-file corpora and richer tool-use (network egress, payment, schema migration) is mechanically straightforward and exposes pressure on the broker implementations and audit-log throughput rather than on the biconditional itself, which is expected to scale unchanged. Puresoftware trust roots remain vulnerable to host compromise; the schema extends straightforwardly to hardware-rooted variants (TPM-sealed signer keys, secure-enclave-bound brokers, attested HITL devices) without changing the criterion.
9
Related work
Mandatory access control and information-flow. The classification component of the schema is Bell–LaPadula [2] adapted to skill artefacts. SELinux [12] demonstrated MAC at OS scope; Asbestos [7] and HiStar [23] extended decentralized information-flow control to whole systems. Our contribution is not the lattice but its lifting onto skills as deployable artefacts. Audit + capabilities. Hash-chained logs are folklore [19]; the novelty here is linking the audit records to the biconditional check, not the chain construction. The discipline of manifest-declared capabilities and one-shot trust-root locks follows the object-capability tradition [14, 15], lifted from intra-process objects to inter-host deployable units. Prompt-injection defenses + skill ecosystems. Existing model-boundary defenses — NeMo Guardrails [16], Llama Guard [9], Lakera Guard [10] — operate as input/output filters and are complementary to ours: a skill that is malicious but well-formed may pass an input filter yet be defeated by capability denial, and vice versa. Recent work also shows that filter-only defenses can be subverted by weak-to-strong jailbreaking attacks [24], reinforcing the case for a runtime gate that is independent of the model’s own classification of its inputs. Empirical observations on the SKILL.md ecosystem report the asymmetry (curated skills help, indiscriminate self-generated ones harm); our contribution accepts the asymmetry and provides a runtime mechanism that enforces it without trusting the agent to police itself.
10
Conclusion
The four contributions of this paper advance the state of the art as follows. The trust schema (§3) adds the verification-level field and the immutable-in-session discipline absent from current SKILL.md conventions. The capability-gate model (§4) keys HITL to the loaded skill’s verification level, replacing the operationally untenable uniform-HITL default with a tractable one. The biconditional criterion (§5) is an audit-log-driven check on what the runtime did, complementary to behavioural benchmarks like AgentHarm [1] that score what an agent decided. The architectural guidelines G1–G12 (§7) abstract a portable set of structural invariants from a working open implementation [13] that any 14
harness can adopt. The cost is small; the failure modes the criterion catches are precisely the ones operators discover, after the fact, in incident review.
References [1] Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Eric Winsor, Jerome Wynne, Yarin Gal, and Xander Davies. AgentHarm: A benchmark for measuring harmfulness of LLM agents. In International Conference on Learning Representations (ICLR), 2025. [2] D. Elliott Bell and Leonard J. LaPadula. Secure computer system: Unified exposition and Multics interpretation. Technical Report MTR-2997, The MITRE Corporation, 1976. [3] Alex Birsan. Dependency confusion: How I hacked into Apple, Microsoft and dozens of other companies. https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610, 2021. Disclosure of a supply-chain class of attacks against npm, PyPI, and RubyGems. [4] Justin Cappos, Justin Samuel, Scott Baker, and John H. Hartman. A look in the mirror: Attacks on package managers. In Proceedings of the 15th ACM Conference on Computer and Communications Security (CCS), pages 565–574, 2008. [5] Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases. In Advances in Neural Information Processing Systems (NeurIPS), 2024. [6] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2Web: Towards a generalist agent for the web. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2023. [7] Petros Efstathopoulos, Maxwell Krohn, Steve VanDeBogart, Cliff Frey, David Ziegler, Eddie Kohler, David Mazières, Frans Kaashoek, and Robert Morris. Labels and event processes in the Asbestos operating system. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP), pages 17–30, 2005. [8] Emilio Ferrara. GenAI against humanity: Nefarious applications of generative artificial intelligence and large language models. Journal of Computational Social Science, 7:549–569, 2024. [9] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama Guard: LLM-based input–output safeguard for human–AI conversations, 2023. [10] Lakera AI. Lakera Guard: Real-time LLM security. https://www.lakera.ai/, 2024. [11] Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang, Yu Zhou, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, and Jiajun Wu. Embodied agent interface: Benchmarking LLMs for embodied decision making. In Advances in Neural Information Processing Systems (NeurIPS), 2024. [12] Peter Loscocco and Stephen Smalley. Integrating flexible support for security policies into the Linux operating system. In Proceedings of the FREENIX Track of the USENIX Annual Technical Conference, pages 29–42, 2001. 15
[13] Alfredo Metere. enclawed: A configurable, sector-neutral hardening framework for single-user AI assistant gateways, 2026. Open-source reference implementation: https://github.com/ metereconsulting/enclawed. [14] Mark S. Miller. Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control. PhD thesis, Johns Hopkins University, 2006. [15] Mark S. Miller, Ka-Ping Yee, and Jonathan S. Shapiro. The object-capability model: A survey. Technical report, Hewlett-Packard Laboratories, 2003. [16] NVIDIA. NeMo Guardrails: A toolkit for programmable LLM safety. https://github.com/ NVIDIA/NeMo-Guardrails, 2024. [17] OWASP Foundation. OWASP top 10 for large language model applications. https://genai. owasp.org/llm-top-10/, 2025. Accessed 2026. [18] Justin Samuel, Nick Mathewson, Justin Cappos, and Roger Dingledine. Survivable key compromise in software update systems. In Proceedings of the 17th ACM Conference on Computer and Communications Security (CCS), pages 61–72, 2010. [19] Bruce Schneier and John Kelsey. Secure audit logs to support computer forensics. In ACM Transactions on Information and System Security, volume 2, pages 159–176, 1999. [20] The MITRE Corporation. MITRE ATLAS: Adversarial threat landscape for ai systems. https://atlas.mitre.org/, 2025. Accessed 2026. [21] Ken Thompson. Reflections on trusting trust. Communications of the ACM, 27(8):761–763, 1984. [22] Shenao Yan, Shen Wang, Yue Duan, Hanbin Hong, Kiho Lee, Doowon Kim, and Yuan Hong. An LLM-assisted easy-to-trigger backdoor attack on code completion models: Injecting disguised vulnerabilities against strong detection. In Proceedings of the 33rd USENIX Security Symposium, 2024. [23] Nickolai Zeldovich, Silas Boyd-Wickizer, Eddie Kohler, and David Mazières. Making information flow explicit in HiStar. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), pages 263–278, 2006. [24] Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. Weak-to-strong jailbreaking on large language models, 2024.
16