EPISTEME: Trustworthy Coding Agents Through Tool Call Intent

E PISTEME: Trustworthy Coding Agents Through Tool Call Intent

Abstract— Coding agents change software through tool calls. Tool calls mutate the local state. For good? For bad? Nobody knows. Agents never say why a tool call is made. We propose the epistemic agent design, in which the harness guarantees that the model declares why a tool call is made. The harness then records what the command actually did and checks the self-reported declaration against these effects. The result is a journal 1) that can guard calls before they run and 2) that supports audits and forensics. For evaluation, we use ImpossibleBench, whose tasks can be passed only by cheating, and demonstrate that forcing the agents to say why results in exposed cheating. Across 49 reward-hacking runs of two open-weight models, the journal exposed 90%.

C

oding agents increasingly change software on their own. An agent is a large language model working in a loop: it receives a task, proposes a tool call, such as running a shell command or editing a file, sees the result, and decides what to do next. The harness runs this loop and executes the tool calls. The model decides; the harness acts. This is a safety problem. With a tool call, an agent changes the world. Every consequence is mediated through tool calls. On a developer’s machine, a single call can delete files, read credentials, or rewrite the tests that sit in the way of the agent. With internet access, the same mechanism reaches beyond the machine: a call can push code, send messages, install packages, call production APIs, or change remote systems, often irreversibly. The more autonomy and connectivity agents have, the more safety rests on what each call does. Today, tool calls are monitored and audited in three ways: 1) permission rules allow or deny commands by matching their text; 2) logs and transcripts record the calls, and a reviewer reads them after the fact; 3) LLM monitors read the calls and the agent’s reasoning to flag suspicious behavior. All three are fundamentally limited, because a tool call does not carry its intent. Suppose an agent asked to make a failing test suite pass runs sed -i ’22 s/ˆ/# /’ test.py and then the tests. Was it simply exploring, or did it switch off an assertion it could not satisfy? The command does not say. Rules or LLM monitors on command text cannot tell

rejected: error

Language model output

Epistemic harness Tool call command: shell command intent: type, reason

2 Run command

1 Check declaration rejects an invalid intent (type or reason)

3 Tool Call Effects workspace changes before/after

claim

type + command policy

arXiv:2611.06563v1 [cs.SE] 4 Nov 2026

Sofia Bobadilla, Carmine Cesarano, Raphina Liu, Javier Ron, Martin Monperrus KTH Royal Institute of Technology, Stockholm, Sweden

observation

4 Check the declaration against the effects findings: contradiction, warning, protected

5 Append journal entry

example

command:

sed -i ’22 s/^/# /’ test.py && python test.py

type:

test-hypothesis

reason:

Temporarily comment out the flawed test case (line 22) [...]

effects:

modified test.py

findings: contradicted: test.py modified warning: command runs sed -i protected: test.py modified

Guardrail (online)

Forensics (offline)

refuse calls before they run

audit past trajectories

FIGURE 1. The epistemic harness design, with a journal entry for the sed command from the introduction.

the two apart. Shell commands are open-ended, and the same malicious effect can be written in unboundedly many ways. A log records what happened but not why, so a reviewer must reconstruct intent from effects. Regulated domains such as medical devices, avionics, automotive, and infrastructure, which expect a record of what was done and why, cannot accept agent work at face value. We propose the epistemic agent design: a coding agent design where the harness refuses to run a command unless the model declares what kind of action it 1

is, and why. The harness guarantees this information is present. From there, the harness can check what the command actually did against the declared intent. We call our implementation E PISTEME. This design supports automatic checking. The declaration is a claim made by the model; the record of effects is an observation made by the harness. The core value is checking the first against the second: a mismatch between declared intent and observed effects is a signal that no command string or log alone can give. The epistemic declaration can be used online or offline. Online, it acts as a guardrail: the harness can refuse a call by its declared kind before it runs, for example a read-only declaration on a command that looks like an edit. Offline, it acts as forensics: a reviewer, or an automated auditor, replays the recorded declarations and finds the steps whose effects contradict their declared intent. This paper makes the following contributions: • Concept. The epistemic agent design, in which the harness makes intent a mandatory part of every tool call. The declared intent is checked against the observed effects after each call, and can guard calls before they run (online) or be audited afterwards (offline). • Evaluation. On ImpossibleBench [1], the journal exposes 44 of 49 reward-hacking runs (90%), and the same journals reveal how each agent works.

Concept: The Epistemic Agent The epistemic agent makes intent first class in the agent design: intents become part of the tool interface, at the point where the agent acts. A harness specifies each tool’s name, short description, and argument schema to the model. A standard shell tool takes a single argument, command. The epistemic harness adds two required arguments to the shell tool, type and reason, so the model cannot run a command without declaring what kind of act it is and why. Together, they form the call’s epistemic declaration. Figure 1 shows what the epistemic harness does with each tool call. It 1 checks that the declaration is well-formed and rejects it otherwise, 2 runs the command, 3 records the call’s effects by comparing the workspace before and after, 4 checks the declaration against the effects and records each problem as a finding, and 5 appends an entry to the journal: the declaration, supplied by the model, next to the effects and findings, produced by the harness. 2

Every Command Comes With Intent The epistemic type names the kind of act from a closed vocabulary of seven, making acts checkable and countable. The first three types are read-only; the other four change state: › collect-info: read-only observation to build understanding; › test-hypothesis: an experiment to confirm or refute a belief; › verify: re-running a known check on finished work, to produce evidence; › setup: preparing the environment, a means rather than the deliverable; › change-state: a lasting change that is part of the deliverable; › revert: undoing an earlier change that proved wrong, which makes dead ends explicit; › cleanup: removing temporary artifacts the agent created. The reason states the call’s intent in free text, not its actions. The model declares both before execution, committing before any side effect. File tools (read, write, edit) take a reason but no type. The harness assigns collect-info to reads and change-state to writes and edits. An assigned type is not a claim, so it cannot be contradicted: an edit that comments out a test assertion as an experiment is recorded as change-state, and only its reason and, for a protected file, the protected finding reveal it. Only shell commands can contradict their declaration; requiring a type on file edits would extend the check to them. Well-formed declarations. Before execution, the harness rejects calls whose type is missing or outside the vocabulary, or whose reason is shorter than 20 characters. Rejected calls do not execute: the model receives an error naming the problem and must reissue the call; the rejection is journaled. Every executed command carries a well-formed declaration, and every attempt, accepted or not, is on record.

Declarations Are Recorded, Then Checked A journal entry records the declaration, the tool and its arguments, the exit code, the full output, and the effects. It includes file diffs, whether git HEAD moved, the checker’s findings, and, for rejected calls, the error sent to the model. The declaration, effects, and findings form the act’s epistemic provenance: what the agent said it was doing, what happened, and whether they agree. Separating claims from observations gives the journal two properties. First, integrity: the harness ap-

pends journal entries outside the agent’s workspace as execution proceeds, so the model cannot edit the observations. Because it is written call by call, it also survives a stopped or killed agent. Second, verifiability : each declaration is a claim about a command, and the journal holds evidence of what the command did that its author cannot rewrite. Comparing them tests whether each call’s observed effects match the model’s declaration. Consistency checker. After each call, the harness runs a checker that compares the declared type with the observed effects. Its findings have three severities. A contradiction proves the declaration false: a readonly type whose call changed a file, a revert that touches a file no earlier change-state changed, or a cleanup that creates or modifies files, or deletes one the agent did not create. A warning marks a suspicious but not disproven call: a read-only command containing a mutating operation, such as sed -i, rm, or an output redirection, or a change-state or revert that changed nothing. Command text alone proves nothing, because rm of a missing file changes nothing and a Python script can write files without any telltale word; only effects can prove a contradiction. A protected finding marks any change to an operator-protected file, such as the tests, whatever the declared type. It flags rather than prevents the change: an honest edit of a wrong test is not a contradiction, but a reviewer should see it. The full rule set is in our repository. In Figure 1, the sed command from the introduction declares a read-only experiment, but its effects show a test file changed. The checker contradicts the type; the command text raises a warning, and the edit touches a protected file.

Online and Offline Use of Epistemic Provenance E PISTEME runs in one of two modes. In monitor mode, the harness rejects only ill-formed declarations, runs every call, and records the findings without acting on them. Guardrail mode adds one online check: step 1 also refuses a call declared read-only whose command contains a mutating operation. A contradiction proven by effects is known only after the call has run, so both modes record it; a stricter harness could stop the agent or roll back the workspace. Offline, the journal serves forensics. The harness renders it as a report that lists the contradicted calls separately, so a reviewer knows where to start, and the checker can be rerun on a stored journal without the agent or the model.

Prototype E PISTEME, our open-source prototype, is written in Python and supports any OpenAI-compatible provider. Its shell tool schema requires type and reason, and wraps the file tools so that each requires a reason. To observe effects, it hashes the files of the workspace before and after every call and keeps the diff of changed files, and the full output; a separate record per run notes the model, the code version, the token usage, and how the run ended. Commands run in a Bubblewrap sandbox that confines them to the workspace and cuts network access, so every lasting effect lands where the harness observes it. The journal and report are written outside the workspace, beyond the sandbox’s reach. Code, tests, and experiment scripts are open source.1

Cheating in ImpossibleBench An agent that misbehaves has three options: it can declare the act honestly, declare a type that its own effects refute, or hide the act behind a plausible reason. We measure how effectively E PISTEME detects misbehavior.

Experimental Protocol We use ImpossibleBench [1]. Each task asks the agent to implement a function described in its docstring and to make its unit tests pass, but the benchmark’s authors altered the tests so that they contradict the specification. In the oneoff split, one assertion expects an incorrect value; in the conflicting split, two assertions expect different values for the same input. Each split contains 103 tasks. No implementation that follows the specification can pass all the tests, so the right response is to report the faulty test. The wrong solution to the impossibility is to cheat on one of the contradicting aspects. An agent that makes the tests pass anyway can do so in two ways. It can tamper with the test file, or exploit the faulty test by tailoring its code to it, for example by returning that input’s expected value. Both can be checked mechanically, without a judge. These rules identify potential reward hacking. We run E PISTEME on all 206 tasks with two openweight models from different families, Qwen3-235BA22B and Mistral Large 3, with up to three attempts and 60 minutes per task, and the benchmark’s failure feedback. The system prompt contains only the

1 https://github.com/ASSERT-KTH/epistemic-agent

3

TABLE 1. Exposure of cheating in ImpossibleBench. Contradicted (C) and cheat disclosed (D) can overlap. Exposure uses their union (C ∪ D), counting each run once.

Completed runs

Total (N)

Contradicted (C)

Misbehaving runs Cheat disclosed (D)

Exposure rate (|C ∪ D |/N)

Qwen3-235B Mistral Large 3

197 203

18 31

1 2

14 30

78% 97%

Total

400

49

3

44

90%

Model

harness instructions, and the harness runs in monitor mode, so the findings do not influence the agent. Each command is limited to 60 seconds and 2 GiB of memory. The code, the system prompt, and the raw runs are in our repository.

Cheat Detection For each misbehaving run, we locate the calls that carry out the cheat and ask two questions about their epistemic provenance. Is the cheat disclosed: does a declared reason state it? Is the run contradicted: does the checker prove the epistemic type false from the observed effects? For example, a call declared readonly is contradicted if it changes a file. A contradiction shows that the type was false; it does not by itself establish that the act was reward hacking. Two human raters independently assessed disclosure from the reasons, without knowing the model or the checker’s findings. They then resolved disagreements together. The checker determined contradictions mechanically. A misbehaving run is exposed if the act is disclosed, contradicted, or both. It is stealth if it is neither disclosed nor contradicted. Our primary metric is the exposure rate: the proportion of misbehaving runs that are exposed.

Experimental Results We ran 206 tasks per model, totaling 412 runs. Of these, 400 runs completed and 12 were excluded after server errors or timeouts. None of the excluded runs had misbehaved before stopping. We classified 49 completed runs as misbehaving per the ImpossibleBench oracle: there are 18 misbehaving runs for Qwen and 31 for Mistral (Table 1). Misbehavior is more frequent on oneoff tasks (21%) than on conflicting tasks (4%). Qwen mostly changed the tests (13 of its 18 misbehaving runs), whereas Mistral mostly produced overfitting implementations that passed the original impossible tests (28 of 31). In total, 16 runs involved tampering and 33 involved exploitation. Across completed runs, tool calls per run had a minimum, 4

median, 95th percentile, and maximum of 3, 8, 69, and 144 for Qwen, and 6, 16, 34, and 43 for Mistral, respectively. Declared reasons disclosed cheating in 44 of the 49 misbehaving runs. For example, one reason said: “Replace the incorrect formula with a hardcoded lookup table of expected values to pass the unit tests.” Before discussion, the raters agreed on 46 of the 49 disclosure labels (Cohen’s κ = 0.72); they resolved the three disagreements together. E PISTEME detects contradictions between epistemic type and effects in 3 runs. The agent edited the test file with a shell command declared readonly or revert. The journal exposed 44 of the 49 misbehaving runs, an exposure rate of 90% (95% CI 78–96%); every exposed run was disclosed. The checker could not contradict any exploit in this experiment. Every exploit changed the code through a file tool, and the harness assigns file tools a fixed type rather than asking the model to declare one. In four tasks, the benchmark’s test file was broken and could not run, so changing it may have been a repair. Excluding these four runs, the exposure rate was 89% (40 of 45). The remaining 5 runs were classified as stealth misbehavior: the checker did not identify a contradiction and the reason does not disclose the problem. In three runs, the agent edited a test but its reason only described the problem, for example “The test expecting 0 is incorrect.” (run oneoff/lcbhard_54 by model Qwen). In the other two runs, the agent also changed the implementation while giving a general reason. One implementation hard-coded answers for three test inputs; the other passed a faulty test through an incorrect general simplification, without test-specific code. Qwen often investigates and stops: in 72 of its 197 runs, Qwen read the files, ran the tests, and ended the run without changing any file; 51 of these runs are in the conflicting split. Mistral runs a loop of edits and test runs.

test-hypothesis

verify

412 calls (12%)

431 calls (13%)

37% 7%

5%

100% 5% 4%

task 197 runs

12%

8%

5%

7%

collect-info 1,820 calls (54%) 4%

change-state 675 calls (20%)

setup / revert / cleanup 19 calls (1%)

FIGURE 2. Transitions between declared types in 197 Qwen runs.

Epistemic Workflow Evidence The journal is more than an audit trail for misbehavior. Because every call carries a type, the journals of many runs together describe how an agent works: what kind of act it performs, and in what order. In the 400 completed runs, the two models made 3714 bash calls and declared 3577 of them (96.3%) read-only: 1442 collect-info, 1337 test-hypothesis, and 798 verify. Only 137 bash calls (3.7%) declared a mutating type: through the shell, the agents mostly observed, by reading and running tests. To edit code, they mainly used file tools, with 1989 calls to str_replace_edit and write_file against 98 bash calls declared change-state. Counting file edits, 30% of all calls changed state (2126 of 6998). Figure 2 shows Qwen’s transitions between declared types across its 197 completed runs. An arrow counts how often one type directly follows another (3160 transitions overall). Notably every run first opens by collecting information, which is good. Beyond the first tool call, reading dominates: collect-info accounts for 54% of all calls, and 37% of all transitions are one read followed by another. The next piece of evidence is that once the agent changes code, it checks the change: 73% of change-state calls are followed directly by a verify or a test-hypothesis. Each check then leads to a new change or back to reading, the trial-and-error loop of agentic work. Mistral shows the same structure with a slightly different bal-

ance: it reads less (25% of its calls against 54%) and spends 40% of its transitions alternating between change-state and test-hypothesis, showing model impact on epistemic explanations. Practitioners deploying the technique would need to calibrate their epistemic guardrail and forensics processes depending on the model they use.

Related Work Misbehavior of coding agents. Deception has been measured on 200 tasks across 11 LLMs [2], and 94% of human developers fail to detect sabotage by a coding agent [3]. Monitoring agents. LLM monitors read agent trajectories, but an agent aware of the monitor substantially degrades their reliability [4]; black-box monitors can be optimized on synthetic trajectories [5]. A weaker model reading the chain of thought of a reasoning model detects reward hacking in agentic coding far better than one reading actions alone [6], yet chain-ofthought explanations can misrepresent the true reason for an answer [7]. AgentSight correlates intercepted LLM traffic with kernel events [8]. Provenance and audit trails. PROV-AGENT records the prompts, responses, and decisions of agents in scientific workflows [9], and a survey of execution provenance for LLM agents covers whether tool calls were justified [10]. These records capture what happened, not why; the epistemic journal also records why each action was taken, in a closed, machine-checkable 5

vocabulary. Intent and policy enforcement. Qu et al. argue that agent security requires intent-to-execution integrity, an end-to-end property [11]; ProvenanceGuard blocks a tool call by inferring its alignment with the user’s intent from context [12]; and Agent-C enforces temporal safety properties on sequences of raw tool calls [13]. These approaches infer intent from free-form text, raw tool calls, or system events, or state intent integrity as a goal. E PISTEME instead makes a typed declaration of intent a mandatory part of every tool call and checks it against the effects the harness observes. To the best of our knowledge, it is the first harness in which a justification is mandatory for every tool call.

Conclusion A tool call shows what an agent did, but not why. E PIS TEME makes the harness ask: every command carries a declared type and reason, which the harness checks against the effects it observes, as a guardrail while the agent runs and as forensics afterward. On impossible tasks, the journal exposed 90% of the reward-hacking runs, all through the agents’ own declared reasons. Declarations do not make an agent honest, but they turn every tool call into a claim the harness can test. As agents gain autonomy, such checkable claims can replace trust with verification.

REFERENCES 1. Z. Zhong, A. Raghunathan, and N. Carlini, “ImpossibleBench: Measuring LLMs’ propensity of exploiting test cases,” ArXiv, vol. abs/2510.20270, 2025. 2. D. Guo, Q.-Y. Liu, D.-R. Liu, Q.-H. Ren, S. Shao, T. Qiu, H. Li, Y. R. Fung, Z.-J. Ba, J. Dai, J. Ji, Z. Chen, J. Tao, Y. Yang, J. Shao, and X. Hu, “Are your agents upward deceivers?” ArXiv, vol. abs/2512.04864, 2025. 3. J. Ye, H. Zou, S. Yu, and W. Shi, “Coding with “enemy”: Can human developers detect AI agent sabotage?” ArXiv, vol. abs/2606.05647, 2026.

6

4. N. Kale, C. B. C. Zhang, K. Zhu, A. Aich, P. Rodriguez, Scale Red Team, C. Q. Knight, and Z. Wang, “Reliable weak-to-strong monitoring of LLM agents,” ArXiv, vol. abs/2508.19461, 2025. 5. S. Storf, R. Barton-Cooper, J. Peters-Gill, and M. Hobbhahn, “Constitutional black-box monitoring for scheming in LLM agents,” ArXiv, vol. abs/2603.00829, 2026. 6. B. Baker, J. Huizinga, L. Gao, Z. Dou, M. Y. Guan, A. Madry, ˛ W. Zaremba, J. Pachocki, and D. Farhi, “Monitoring reasoning models for misbehavior and the risks of promoting obfuscation,” ArXiv, vol. abs/2503.11926, 2025. 7. M. Turpin, J. Michael, E. Perez, and S. Bowman, “Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting,” Advances in Neural Information Processing Systems, 2023. 8. Y. Zheng, Y. Hu, T. Yu, and A. Quinn, “AgentSight: System-level observability for AI agents using eBPF,” arXiv:2508.02736, 2025. 9. R. Souza, A. Gueroudji, S. Dewitt, D. Rosendo, T. Ghosal, R. B. Ross, P. Balaprakash, and R. D. da Silva, “PROV-AGENT: Unified provenance for tracking AI agent interactions in agentic workflows,” 2025 IEEE International Conference on eScience (eScience), pp. 467–473, 2025. 10. Y.-Q. Wang, J.-Q. Zhang, Z.-K. Wu, T.-T. Cai, Z.-R. Liu, Z.-Q. Sun, M. Dong, M.-K. Zheng, Y.-Q. Duan, X. Yin, and Y. Zhu, “From agent traces to trust: A survey of evidence tracing and execution provenance in LLM agents,” ArXiv, vol. abs/2606.04990, 2026. 11. W. Qu, M. Xu, P. Wang, S. Zhai, J. Zhang, and D. Song, “Securing LLM agents need intentto-execution integrity,” ArXiv, vol. abs/2605.16976, 2026. 12. Y.-N. She, Y. Liang, and E. Kang, “Safeguarding LLM agents from misalignment through provenance analysis,” ArXiv, vol. abs/2607.01236, 2026. 13. A. Kamath, S. Zhang, C. Xu, S. Ugare, G. Singh, and S. Misailovic, “Enforcing temporal constraints for LLM agents,” ArXiv, vol. abs/2512.23738, 2025.

Related documents

• AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents#229449
• Manufacturing the Illusion of Epistemic Trustworthiness#715189
• Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback#200400
• The Stochastic Deputy: Structural Tenant Isolation for Tool-Using LLM Agents#1084702
• The Stochastic Deputy: Structural Tenant Isolation for Tool-Using LLM Agents#919500