Declarative Skills for AI Agents in Knowledge-Grounded Tool-Use Workflows
arXiv:2606.06923v1 [cs.AI] 5 Jun 2026
M. Danish Lim, I. Danial Bin Sharudin, Wen Han Chen, Cedric Lim, Laura Wynter∗ School of Computing and Information Systems Singapore Management University Singapore June 8, 2026 Abstract We study orchestration mechanisms for tool-using AI agents in realistic customer-service workflows over an unstructured knowledge base. We argue that declarative agents—AI agents equipped with natural-language skill files appended to the system prompt—are an effective orchestration paradigm. Concretely, we compare (i) a DeclarativeAgent that reads three domain-specific skill files at inference time and decides its own control flow, (ii) an ImperativeAgent based on a programmatic state machine with explicit phases, and (iii) an unscaffolded baseline agent modeled after the τ -Knowledge benchmark agent. Our ImperativeAgent is motivated by externalised-control inference as in Recursive Language Models and graph-based orchestration frameworks. We formalise the three agents as policy classes within a decentralised partially-observable Markov decision process and analyse their information-theoretic and structural properties; we then test the predicted differences empirically on five language models and two retrieval regimes. Our results show that retrieval quality is a dominant bottleneck for AI agents: when evidence is incomplete or skewed, all agents degrade substantially, and skill files cannot recover lost performance. Under highquality retrieval, however, declarative skills consistently improve accuracy on procedural tasks and reduce orchestration errors, while the imperative state machine’s brittleness does not reliably improve task success or compliance. Code will be provided upon publication
1
Introduction
Tool-using AI agents extend generative language models with the ability to act on external systems, and evaluating how well they perform real-world tasks has become a central research problem as such agents are deployed at scale. We focus on a general and practically important class of agent: customer-service workflows, which combine three distinct competencies: conversational (greeting the user, eliciting and clarifying intent), procedural (determining ordering constraints, verifications required, and tool-call sequences), and reasoning (interpreting retrieved documents to choose the correct tool arguments). These competencies are broadly applicable beyond banking customer service. We make use of τ -Knowledge [1], an extension of τ -Bench [2], which was developed for evaluating AI agents on tasks that require coordinated retrieval over a large unstructured knowledge base (KB). Unlike prior agent benchmarks that supply a fully-specified tool interface up front, τ Knowledge allows the capabilities to be discoverable; for example, many state-changing operations are referenced only in natural-language documentation and must be located via knowledge-base search before they can be invoked. This is framed as a decentralised partially observable Markov ∗
Corresponding author: [email protected]
1
decision process, (Dec-POMDP) [4] over a shared state space S = SDB ×Shistory , where SDB is the state of the database which includes user and system entities, and Shistory is the stored user-AI agent conversation history. The AI agent observes only tool outputs and user messages, and the binary task reward depends on whether the final database state SDB matches a hand-curated target. The problem is a POMDP because the AI agent and the user have asymmetric and incomplete views of the state, S. We propose two paradigms to orchestrate the AI agents. Imperative orchestration is defined as externalised, deterministic control. Deterministic control has broad appeal in combating the non-determinism of LLMs. It can be argued that end-to-end LLM inference for real-world tasks is too risky and that some of the control should be externalised to deterministic code. Recursive Language Models (RLMs) [8] are an aggressive form of this idea that proposes to treat LLM input context as an external environment (e.g. a Python REPL variable). Then, using RLM, the LLM agent programmatically inspects and decomposes the REPL environment, and recursively invokes itself on smaller sub-problems, composing the final answer from its sub-results. The same general idea is found also in LangGraph DFAs [10], ReAct reasoning-action loops [11], and numerous other works. In our instantiation, our ImperativeAgent owns the phase graph and the transition rules (such as verification before write, bounded retries) while the LLM is invoked as a per-phase sub-routine. The desired benefit of the ImperativeAgent is that this deterministic enforcement will reduce hallucination, improve interpretability, and make compliance properties more auditable. The second form of orchestration we examine is called declarative. This paradigm relies on an agent skills approach similar to that proposed by Anthropic[5], who suggest that procedural knowledge should be expressed in natural language and read by the model at run time as needed. An agent skill is a markdown document describing when an action is appropriate, the preconditions and ordering constraints, and the tool-argument requirements. The LLM interprets the agent skill files as part of its system prompt and chooses the workflow and details on-line. The justification for the declarative paradigm is that the LLM’s own attention mechanism can integrate natural language skill content with retrieved evidence in more flexible ways that a fixed state graph. In summary, we define the following two orchestrated agents: • ImperativeAgent that implements a finite-state machine with deterministic transitions, explicit verification gates, and hard-coded retry policies, representing externalised control and programmatic orchestration. • DeclarativeAgent that follows Anthropic-style Agent Skills: the model reads a small set of markdown skill files in its system prompt and is free to choose the workflow, tools, and verification strategy in natural language, with no explicit state machine. Our contribution is to answer the question: does skill-file-based declarative orchestration outperform or underperform programmatic state-machine orchestration for tool-using AI agents in realistic, complex workflows, and what trade-offs do these approaches entail in terms of task success, robustness, compliance, and efficiency? Both our imperative and declarative agentic paradigms have plausible arguments for success; our imperative approach should reduce hallucination and provide reliable results, while our declarative approach should be less brittle. In addition to evaluating our imperative and declarative orchestration paradigms, we evaluate a baseline, unscaffolded LLM agent as used in the τ -knowledge benchmark [1] paper. The paper proceeds as follows. In the next section, we discuss related work. Then, Section 3 casts the three agents as policy classes within a Dec-POMDP and states our three main research questions. Sections 4–5 describe the DeclarativeAgent and ImperativeAgent in detail. Section 6 analyses the three policy classes theoretically. Section 7 provides the experimental design and 2
the main results and Section 8 provides an ablation into the compliance and efficiency of the agents. We conclude with a discussion that ties our findings back to our research questions. The appendix includes additional details.
2
Related Work
The application domain used in τ -Knowledge is τ -Banking, a fintech-customer-service set of tasks. The benchmark environment contains 698 documents (∼195K tokens, 71 topics, 21 product categories), 14 permanent agent tools plus 51 discoverable tools, and 97 evaluation tasks. Each task requires, on average, 18.6 documents and 9.52 tool calls (max 33) to resolve. The benchmark scoring proposed in [1] is passk , and while the paper considers k = 1, 3, we focus only on the more challenging pass1 metric. Documents provided include product specifications, internal procedural policies (e.g. retention protocols, account-closure eligibility), and discoverable-tool signatures with required-argument schemas. Tool names contain random four-digit suffixes (e.g. close bank account 7392) that cannot be guessed. There are two discoverability approaches used in the paper: ”gold” and retrieval. Gold means that the task-critical documents are provided in the system prompt, while retrieval uses an external retriever. The benchmark paper [1] uses both keyword-matching retrieval, via BM25, and embedding-based retrieval. Beyond τ -Knowledge, several recent works benchmark customer-support agents on adherence to business policies, multi-step workflows, and tool-use correctness. [12, 13]. These works assume a fixed orchestration style, comprising a single LLMAgent-like loop with tools, and focus on model or retriever comparisons. Our work proposes to explore the benefits of agent skills via our DeclarativeAgent as well as programmatic approaches to invoking tools via our Imperative Agent under a common benchmark and tool set. Our work also relates to the literature on agentic scaffolding and orchestration. ReAct-style reasoning-and-acting loops [11] interleave natural-language thoughts and tool calls. Graph-based and DFA-style frameworks such as LangGraph [10] expose the agent’s control flow as an explicit state machine or graph, allowing deterministic transitions. Recursive Language Models [8] externalise control further, treating the prompt as an environment variable, and allow the LLM to recursively call itself over decomposed subproblems, demonstrating strong gains on long-context reasoning tasks [9]. Our ImperativeAgent is in this broad family of deterministic approaches to orchestration. A parallel line of work studies retrieval-augmented generation in realistic, messy knowledge bases. τ -Knowledge itself highlights that even frontier models struggle to retrieve, interpret, and act on unstructured documentation [1], and subsequent reports have underscored how retrieval quality often dominates model choice in customer-support agents [1, 14]. There is also increasing interest in replacing brittle tool registries and MCP-style plugins with file-centric agent interfaces, where files serve simultaneously as context, tools, and skills [7]. Our results strengthen these findings: golden retrieval exposes the benefits of skill-file orchestration, while noisy embedding retrieval sharply reduces performance for all agents, illustrating that agent skills and high-capacity LLMs with reasoning cannot compensate for fundamentally incorrect evidence. Related to our declarative orchestration is Anthropic’s Agent Skills specification [5], which proposes reusable SKILL.md files as composable, model-readable procedural knowledge for agents. Agent Skills are loaded via progressive disclosure, with short metadata always in context and full skill bodies read only when needed [5, 6]. Follow-on work has generalised this idea to other ecosystems, arguing that skills should be small, focused markdown files that can be swapped or combined to tailor agent behaviour [7, 6]. Our DeclarativeAgent instantiates this paradigm, using three skill files to encode conversational structure, banking procedures, and knowledge-discovery strategy, and provides, to our knowledge, 3
the first systematic comparison between a skill-file declarative agent and a programmatic statemachine agent on a realistic customer-support benchmark. The authors of [1] identified the main causes of failure when using their benchmark on LLM agents as: (1) complex interdependencies between offerings (∼14.5% of failures) — multi-hop reasoning across documents to find the optimal product combination; (2) failure to respect implicit subtask ordering (∼5%) — e.g. disputes must resolve before credit limit increases; (3) overtrusting user assertions (∼4%) — acting on user-claimed state without verifying via tools; and (4) search inefficiency and unwarranted assumptions (∼23%) — committing to early hypotheses rather than searching the KB. These causes of failure motivate our ImperativeAgent and DeclarativeAgent strategies. While failure type 1 may be assumed to be tied mainly to LLM capacity (and hence model parameter count), we aim to rectify failures 2–3, namely topological task ordering and verification gating using code with our ImperativeAgent. Similarly, we provide explicit KB-search guidance in the agent skills of our DeclarativeAgent, positioning declarative skills as a low-cost capability enhancement for AI agents. The τ -Knowledge paper evaluates five frontier models across the various retrieval configurations. Their main finding is that the benchmark is hard for current LLM agents: their best non-gold configuration was GPT-5.2 (high reasoning) with terminal use at 25.52% pass1 , and even with gold documents provided to the agent in context their best score was Claude-4.5-Opus (high) at 39.69%. Our Table 1 reproduces their benchmark’s main pass1 results using their unscaffolded LLM agent [1, Table 2]. Table 1: τ -Knowledge benchmark’s frontier-model baselines on unscaffolded LLM agents using pass1 (%), reproduced from [1]. Gold provided the minimal document set to the agent in context. Parentheses indicate ∆ vs. Gold setting for each row. Reas. means reasoning level setting. Embeddings Model
Reas.
Gold
text-emb-3-large
Qwen3-emb-8B
BM25
Terminal
GPT-5.2 GPT-5.2 Claude-4.5-Op Claude-4.5-Son Gemini-3-Pro Gemini-3-Flash
High None High High High High
32.73 15.72 39.69 33.76 33.25 36.34
23.45 (−9.3) 8.25 (−7.5) 18.30 (−21.4) 17.53 (−16.2) 12.89 (−20.4) 18.56 (−17.8)
24.74 (−8.0) 12.37 (−3.4) 19.59 (−20.1) 17.78 (−16.0) 12.89 (−20.4) 18.56 (−17.8)
24.48 (−8.2) 9.54 (−6.2) 17.78 (−21.9) 16.75 (−17.0) 13.66 (−19.6) 18.56 (−17.8)
25.52 (−7.2) 11.60 (−4.1) 24.74 (−14.9) 22.42 (−11.3) 15.72 (−17.5) 20.62 (−15.7)
3
Problem Formulation
We model the customer-service interaction as a finite-horizon, two-agent decentralised partiallyobservable Markov decision process (Dec-POMDP) [4]. A simulation, hereafter referred to as a task, is one rollout of this process on a fixed task specification drawn from τ -Knowledge. The world state is S = SDB × Sconv , where SDB is the relational state of the banking database (customer records, accounts, transactions, disputes, cards) and Sconv is the rolling conversation history. Two policies operate over S jointly: the task agent π that we design, and a user-simulator πu whose persona and intent are fixed by the task. Both observe S only through messages and tool outputs; the database is not directly visible to either agent. At each turn t, the task agent emits an action at ∈ A given an information state ht = (o1:t , a1:t−1 ). We partition the action space as A = Asay ∪ Aread ∪ Awrite : Asay are natural-language turns directed at the user; Aread are non-mutating tool calls (KB search, account lookups, tool discovery,
4
identity log verification); Awrite are state-mutating tool calls that change SDB (transaction submissions, account changes, referrals, transfers to human agents, etc.). A task terminates when either party emits a stop token, when an unrecoverable error is raised, or when a horizon Tmax is reached. Reward is binary, n o n o gold final r(τ ) = 1 SDB = SDB · 1 Arequired ⊆ A(τ ) , where Arequired is the set of gold action-checks (canonical write tools with their canonical arguments) supplied by the benchmark and A(τ ) is the multiset of tool calls actually issued. Across K trials, the standard τ -Knowledge metric is passk = Eτ1:K [1{∀i ∈ [k], r(τi ) = 1}]; we report pass1 as our primary metric. We compare three policy classes within this Dec-POMDP. Let θsys denote the baseline system prompt distributed with the benchmark, and let the LLM under evaluation be M . Definition 1 (Baseline policy). The baseline policy is πB (at | ht ; θsys , M ) = M (at | θsys , ht ), i.e. the model conditions only on the static system prompt and the running history. There is no agent-side control flow. Definition 2 (Declarative policy). Let Σ = {s1 , s2 , s3 } be a finite set of natural-language skill files. The declarative policy is πD (at | ht ; θsys , Σ, M ) = M (at | θsys ⊕ Σ, ht ) , where ⊕ denotes prompt concatenation. Structurally, πD differs from πB only in that the system prompt has been enlarged with Σ; there is no phase variable and no restriction on A. Definition 3 (Imperative policy). The imperative policy is a hierarchical pair (πI,ϕ , δ) where ϕ ∈ Φ is a phase, δ : Φ × State → Φ is a deterministic phase-transition function, and each πI,ϕ is a phase-conditional sub-policy that emits actions only in a restricted subset Aϕ ⊆ A together with a phase-specific instruction ιϕ injected into the system prompt: πI (at | ht , ϕt ; θsys , M ) = M (at | θsys ⊕ ιϕt , ht ) · 1{at ∈ Aϕt }. Transitions ϕt+1 = δ(ϕt , statet ) are computed by code, not by the LLM. The three policies share the same model M , the same tools, and the same user-simulator distribution; they differ only in how procedural knowledge is encoded (none, natural-language, or executable code) and in whether the action space is restricted at each turn. This isolates the orchestration choice as the independent variable in the experiments that follow. We define our three main research questions below. RQ1 (skill files as procedural prior). For models M with a procedural-competence gap g(M ) > 0 on this domain — operationalised by pass1 (πB , M ) being substantially below the human or oracle ceiling — our first research question aims to determine whether the declarative policy weakly improves on the baseline: pass1 (πD , M ) ≥ pass1 (πB , M ), and whether the gain shrinks as g(M ) → 0. RQ2 (imperative as compliance enforcer). Does the imperative policy reduce the unauthorizedwrite rate (i.e., writes before a successful log verification) compared to baseline and declarative, by construction of a verification gate V → E? RQ3 (retrieval as bottleneck). Under noisy embedding retrieval, does the advantage of our declarative paradigm over the baseline collapse? Figure 1 contrasts the three policy classes schematically: all three involve conditioning the model M on the history ht , but they differ in what augments the system prompt and whether the action space is restricted at each turn.
5
Baseline (πB )
DeclarativeAgent (πD )
θsys
θsys ⊕ Σ
ImperativeAgent (πI )
ϕt
(skills s1 , s2 , s3 )
LLM M
θsys ⊕ ιϕt (phase-specific)
δ(ϕt , statet )
LLM M
LLM M
at ∈ A
at ∈ Aϕt ⊊ A
at ∈ A
Figure 1: Three policy classes within the same Dec-POMDP. The baseline is conditioned on a fixed system prompt θsys . The DeclarativeAgent enlarges the system prompt with skill files Σ. The ImperativeAgent injects a phase-specific instruction ιϕt , restricts the per-turn action space to Aϕt ⊊ A, and advances the phase via a deterministic transition δ.
4
DeclarativeAgent
Our DeclarativeAgent is a minimal extension of the baseline: it inherits the τ -Bench LLMAgent unchanged and adds a single intervention, that is, natural-language agent skill files appended to the system prompt. This design isolates the orchestration choice as the independent variable, so any observed gap between the baseline and the DeclarativeAgent is attributable to the skill files themselves rather than to differences in tools, prompting style, or control flow. As proposed by the AgentSkill paradigm [5], the LLM decides what to do, when to call which tool, and when to verify, while the skill files describe the workflow in natural language. There is no agent-side control flow, no phase enumeration, and no per-turn instruction template. The declarative system prompt is built from the baseline LLMAgent prompt with an appended <skills> block containing three concatenated Markdown skill files (see Appendix 9.3 for details). The instructions and policy are identical to the baseline agent; skill content is loaded once at agent construction by reading src/skills/*.md alphabetically and concatenating with --separators (see DeclarativeAgent. load skills). The skill set comprises three markdown documents pertaining to the three capabilities required of the AI agent. (src/skills/*.md): banking-procedures.md Maps each banking operation (account information, credit-card operations, replacements, disputes, referrals, account closure, transfers) to its preconditions, the discoverable tools required, ordering constraints between operations (e.g. “credit-limit increase requires no pending disputes”), and the canonical argument set. This skillfile is a policy index intended to keep the model from having to derive potentially erroneous ordering rules from the raw KB. customer-interaction.md Defines a generic four-step conversational structure — Greeting and Understanding, Triage, Verification and Action, Confirmation — with a multi-request inventory at the beginning (“Identify ALL requests in their message before choosing a path”) and guidance to ask exactly one targeted clarifying question. This is the natural language, non-deterministic analog of a state machine: the phases are descriptive, not code, and the model is free to deviate from them. knowledge-discovery.md Specifies the KB-search strategy, specifically: when to search (any operation needing a discoverable tool, any policy edge case), how to construct queries for both embedding-based and golden retrieval, and how to recover if the search fails to 6
ADVISORY KB hit user id
GREETING
TRIAGE
empty
verified
VERIFY
PLANNING
retry
EXECUTION
CONFIRM
COMPLETE
retry
ESCALATE
Figure 2: The state graph of the ImperativeAgent return a result. As the benchmark tool names contain random four-digit suffixes (e.g. close bank account 7392) that cannot be guessed, this skill file emphasises that KB search is a hard precondition for state-changing tools. The behaviour of our DeclarativeAgent proceeds as follows. At each turn. the DeclarativeAgent appends the incoming message to its state, concatenates system messages + messages, and forwards it to generate cached (which is our prompt-caching variant of tau2 benchmark’s generate function). The full tool set is exposed every turn. No phase-conditional tool restrictions are imposed, nor are tool choice overrides or specific per-phase instructions provided. The returned AssistantMessage is then appended to the state and returned to the orchestrator. The control flow is provided in DeclarativeAgent.generate next message.
5
ImperativeAgent
Our ImperativeAgent is defined by a finite-state machine pipeline, as follows: GREETING → TRIAGE → VERIFICATION → PLANNING → EXECUTION → CONFIRMATION → COMPLETE
Additionally, we define an ADVISORY branch from TRIAGE which is called for purely informational requests thus bypassing verification and planning. We also define an ESCALATE terminal phase reachable from VERIFICATION or EXECUTION for the case where a tool exceeds its retry budget. Phase transitions are deterministic functions of agent state; the LLM is invoked once per turn with a phase-specific instruction and the phase’s allowed-tool subset. The state graph is shown in Figure 2. Our AgentState is a Pydantic model in src/agents/state.py which carries six fields that the phase logic reads on every turn: two boolean gates (user identified, verified) that control entry to VERIFICATION and EXECUTION; two list fields (pending tasks, completed tasks) implementing the explicit task queue; a tool retry counts dict for per-tool retry tracking; and a expect violation count counter for response-type-mismatch instrumentation. ESCALATE is the deterministic exit path for tool failures whose retry budget is exhausted. Without it, a model that repeatedly calls a state-changing tool with malformed arguments would loop indefinitely; ESCALATE thus bounds the worst-case trajectory length. We define six structural strategies to implement the deterministic guarantees of our imperative pipeline. Each targets a specific failure identified in the the τ -Knowledge benchmark paper[1]. Details of our strategies are provided in the Appendix. 1. Explicit task queue. PLANNING emits a structured TASKS: . . . END TASKS block; the agent parses it into state.pending tasks, injects the live queue as a <task queue> on every EXECUTION turn, and pops items from the queue as they complete. EXECUTION transitions to CONFIRMATION only when the pending queue is empty. The goal of this strategy is to reduce “forgot the second request” types of failures.
7
2. Topological task ordering. After parsing, the queue is sorted by Kahn’s algorithm [15] over a five-rule keyword precedence table (for example, credit limit ≺ dispute; open ≺ close). This strategy targets ordering failures. 3. State-driven phase transitions. Phase logic reads boolean flags (user identified, verified) instead of inspecting the previous assistant message. Flags are updated in update task state() immediately after the incoming message is appended and before phase determination, so transitions are decoupled from message-stream timing. 4. Verification hard gate. EXECUTION re-checks state.verified and returns to VERIFICATION if false. This way state-changing tools are not used without a verified identity. This strategy targets failures from the LLM agent trusting unverified user assertions. 5. Per-tool retry policy with deterministic escalation. Each retriable tool gets a (max retries, failure phase) policy (e.g. log verification: 3 retries → ESCALATE; KB search: 4 retries → ADVISORY). Retry-limit enforcement runs first in determine phase() so no other logic can override it. This bounds worst-case trajectory length and aims also to improve the cost efficiency of the ImperativeAgent as compared to the baseline LLMagent and the DeclarativeAgent. 6. Strict response-type enforcement. Each phase declares an expected response type (text, tool call, or either); a wrapper around generate() re-prompts requiring tool choice (up to 2 times) on mismatch and records violations in state.expect violation count for post-hoc analysis. Table 2 shows the generate next message execution order. Table 2: Execution steps to generate next message Step 1 2 3 4 5 6 7 8
6
Action Append incoming message update task state() (refresh boolean flags from tool results) determine phase() (reads state flags) Update phase / retries Build tools + instruction (+ queue hint) Tool choice logic enforce expect() wrapping generate() Append assistant message
Theoretical Analysis
We relate our three policy classes to their expected behaviour on the Dec-POMDP. Proposition 1 (Skill-file information advantage). Let A∗ (ht ) denote the set of optimal next actions given information state ht , and let HM (A∗ | ht ) denote the conditional entropy of M ’s action distribution at that turn. For any prompt-side prior Σ that is informative about A∗ , HM (A∗ | ht , θsys , Σ) ≤ HM (A∗ | ht , θsys ), with strict inequality whenever the model has non-trivial procedural-competence gap g(M ) > 0. Regarding the first part of the proposition, we posit that skill files encode procedural knowledge, such as ordering constraints, verification preconditions, search heuristics, and as such should have a non-negative impact on model results on the task. Regarding the second part of the proposition, procedural-competence gap means the ability of the model to correctly follow procedures. We
8
assume that the expected gain from skill files in πD relative to πB will decrease to zero as model capacity increases and thus as g(M ) → 0. Proposition 2 (Imperative restriction is policy-class shrinking). Let ΠB be the set of behaviours expressible by πB over a fixed M and let ΠI be the set expressible by πI with the same M . Because πI allows only at ∈ Aϕt at each turn, and because Aϕt ⊊ A for every non-terminal phase, ΠI ⊊ ΠB . Consequently, in the absence of additional compliance benefits, sup E[r(τ )] ≤ sup E[r(τ )]. π∈ΠI
π∈ΠB
The imperative policy aims to use deterministic gating to increase compliance sufficiently to offset the capacity loss coming from restricting action space. Proposition 3 (Trajectory length governs cost). Let T be the number of LLM calls in a task and let c̄turn be the mean per-call cost for M . Then E[Cost/Task] = E[T ] · c̄turn . The imperative policy’s bounded per-tool retry budget caps E[T ] from above. This should lower expected cost relative to the baseline. Proposition 4 (Retrieval noise as channel degradation). Define the retrieval step as a noisy observation channel Õ = O + η where η injects off-topic chunks into ht . A skill-file prior Σ sharpens the action distribution π(· | ht ) but leaves ht itself unchanged. Therefore the data-processing inequality gives us I(A∗ ; h̃t , Σ) ≤ I(A∗ ; h̃t ) + H(Σ) ≤ I(A∗ ; ht ) + H(Σ), with the first gap growing as η degrades Õ. Beyond a noise threshold, the lift from Σ will be dominated by the information loss in Õ, so the declarative advantage from Proposition 1 will collapse under noisy retrieval. The four propositions thus predict that: (i) πD > πB on weaker models under clean retrieval, (ii) πI increases compliance enough to compensate the reduced capacity, (iii) the imperative cost benefit depends on retry reduction, and (iv) the declarative advantage is conditional on retrieval quality.
7
Experimental Results
We instantiate the three policies on five large language models spanning roughly an order of magnitude in capability: Qwen3.5-Flash, Claude Haiku-4.5, Gemini-3.1-Flash-Lite, DeepSeek-v4Flash, and DeepSeek-v4-Pro. Each model is paired with two retrieval regimes: golden retrieval, in which the task-critical documents are placed in the system prompt directly, and embedding retrieval, using a local all-MiniLM-L6-v2 dense index served through a custom retriever plugged into tau2-bench. We do not report BM25 keyword retrieval because of its poor recall and very high token cost from retry loops. We evaluate on the 97-task suite from τ -Knowledge banking. Details are provided int he Appendix and the companion GitHub repository. Our primary metrics are defined as follows. Pass1 is Eτ [1{r(τ ) = 1}] averaged uniformly over the tasks within a condition; infrastructure errors (i.e., LiteLLM auth retries exhausted) are final = S gold }], the database half of the reward excluded from the average. DB match is Eτ [1{SDB DB without the action-check half, that is, whether the agent reached the right end-state regardless of how it got there. Cost/Task is the per-task mean of the LiteLLM-reported agent cost field, 9
which sums per-message provider-reported usage at published prices and includes cache-read discounts when available. Write-argument accuracy is the fraction of state-mutating tool calls whose argument set matches the gold trajectory, evaluated over all writes issued in a condition. Tables 3 and 4 report the metrics under golden and embedding retrieval, respectively. The golden-retrieval results in Table 3 confirms our first research question RQ1: the DeclarativeAgent improves Pass1 on four of the five models, with parity on Gemini-Flash-Lite. The gain scales roughly with the procedural-competence gap of the underlying model, with the exception of DeepSeek-Pro, that shows a larger gain than its smaller Flash version. The ImperativeAgent underperforms the baseline for every model, consistent with the policy-class-shrinking prediction of Proposition 2. We examine the question of the compliance gain posed in research question RQ2 in the ablations of the next section. Under embedding retrieval as shown in Table 4, Pass1 drops sharply for every model and orchestration strategy. The declarative–baseline advantage collapses with DeepSeek-Pro and Haiku, and provides moderate gains on the medium-capacity Gemini-Flash-Lite and DeepSeekFlash. This confirms Proposition 4, in that skill files cannot compensate enough for the noisy observation channel. Table 5 reports write-argument accuracy on DeepSeek-v4-Pro. The DeclarativeAgent is a strict improvement over the baseline on both retrieval modes and is substantially superior to the ImperativeAgent, which loses roughly twenty percentage points of write accuracy on either retrieval setting. Table 3: Aggregate metrics under golden retrieval. DeclarativeAgent is the best orchestration across the board, with the exception of a tie with the baseline on Gemini-Flash-Lite. The ImperativeAgent underperforms across the board.
8
Model
Agent
N
Pass1
∆Baseline
DB Match
Cost/Task
Haiku-4.5 Haiku-4.5 Haiku-4.5
Baseline Declarative Imperative
95 95 90
0.126 0.179 0.056
— +0.053 −0.071
15.8% 23.2% 10.0%
$0.0644 $0.0755 $0.0895
Qwen3.5-Flash Qwen3.5-Flash Qwen3.5-Flash
Baseline Declarative Imperative
97 97 96
0.082 0.103 0.031
— +0.021 −0.051
9.3% 12.4% 7.2%
$0.0223 $0.0332 $0.0231
Gemini-3.1-Flash-Lite Gemini-3.1-Flash-Lite Gemini-3.1-Flash-Lite
Baseline Declarative Imperative
96 96 95
0.281 0.281 0.147
— 0.000 −0.134
31.2% 29.2% 18.9%
$0.0026 $0.0025 $0.0024
DeepSeek-v4-Flash DeepSeek-v4-Flash DeepSeek-v4-Flash
Baseline Declarative Imperative
95 93 93
0.379 0.387 0.344
— +0.008 −0.035
38.9% 39.8% 39.8%
$0.0039 $0.0042 $0.0084
DeepSeek-v4-Pro DeepSeek-v4-Pro DeepSeek-v4-Pro
Baseline Declarative Imperative
93 93 90
0.462 0.484 0.200
— +0.022 −0.262
46.2% 48.4% 26.7%
$0.0116 $0.0143 $0.0182
Ablations on Compliance and Efficiency
While the task-success rate of the ImperativeAgent is across the board lower, we examine whether it is a safer approach for compliance, in that the state machine should prevent state-changing tool calls outside the EXECUTION phase, with EXECUTION reachable only via VERIFICATION. To test this we replayed the strategies and computed compliance and efficiency metrics on the ImperativeAgent’s tool-call sequence. 10
Table 4: Aggregate metrics under embedding retrieval (all-MiniLM-L6-v2). Pass1 drops sharply on every model under noisy retrieval. The DeclarativeAgent underperforms compared to the baseline on three of the five models, but remains the strongest approach on the medium-sized Gemini-Flash-Lite and DeepSeek-Flash. Model
Agent
N
Pass1
∆Baseline
DB Match
Cost/Task
Haiku-4.5 Haiku-4.5 Haiku-4.5
Baseline Declarative Imperative
95 95 95
0.084 0.032 0.021
— −0.052 −0.063
10.5% 6.3% 6.3%
$0.1988 $0.2090 $0.4313
Qwen3.5-Flash Qwen3.5-Flash Qwen3.5-Flash
Baseline Declarative Imperative
97 97 97
0.041 0.010 0.021
— −0.031 −0.020
3.1% 2.1% 3.1%
$0.0507 $0.0690 $0.0058
Gemini-3.1-Flash-Lite Gemini-3.1-Flash-Lite Gemini-3.1-Flash-Lite
Baseline Declarative Imperative
96 96 96
0.052 0.104 0.062
— +0.052 +0.010
8.3% 13.5% 9.4%
$0.0337 $0.0448 $0.0386
DeepSeek-v4-Flash DeepSeek-v4-Flash DeepSeek-v4-Flash
Baseline Declarative Imperative
95 95 95
0.147 0.189 0.137
— +0.042 −0.010
18.9% 21.1% 18.9%
$0.0089 $0.0091 $0.0256
DeepSeek-v4-Pro DeepSeek-v4-Pro DeepSeek-v4-Pro
Baseline Declarative Imperative
95 95 93
0.211 0.200 0.075
— −0.011 −0.136
23.2% 23.2% 11.8%
$0.0240 $0.0259 $0.0752
Table 5: Write accuracy on DeepSeek-v4-Pro (fraction of write tool calls whose arguments match the gold trajectory) and mean Cost/Task. Write actions are the dominant reward signal in banking knowledge: a single mismatched write usually drops the task reward to 0. Agent
Retrieval
Write Acc.
Avg Cost/Task
Baseline Baseline
Golden Embedding
412 / 571 = 72.2% 409 / 599 = 68.3%
$0.0116 $0.0240
Declarative Declarative
Golden Embedding
455 / 576 = 79.0% 416 / 599 = 69.4%
$0.0143 $0.0259
Imperative Imperative
Golden Embedding
264 / 495 = 53.3% 261 / 542 = 48.2%
$0.0182 $0.0752
Specifically, for every pre-evaluated simulation and the existing results.json files, we walked the message stream and counted: (a) the number of state-mutating tool calls (write tools, defined as the set of apply for credit card, call discoverable agent tool, call discoverable user tool, change user email, give discoverable user tool, request human agent transfer, submit referral, submit transaction; (b) whether each write occurred before the agent had successfully completed a log verification call; and (c) the maximum number of consecutive failed retries of the same tool. From these we derive the three additional metrics: • unauthorized write rate - percentage of tasks with any invalid write, that is, at least 1 unauthorized write was performed before successful verification in the task. • over retry rate - percentage of trials in which the agent re-issued the same tool ≥ 4 times consecutively with failed results. • write pre-verify - percentage of unauthorized writes over all write calls and all tasks. • mean trajectory length - mean number of assistant turns per task.
11
Golden
0.5
Embed.
Baseline Declarative Imperative
Pass1
0.4
0.3
0.2
0.1
0 0.002
0.01
0.05
0.1
0.5
Cost/Task (USD, log scale) Figure 3: Pass1 versus Cost/Task across all 30 (5 models x 3 agents x 2 retrieval types) combinations. Filled symbols are golden retrieval; empty symbols are embedding retrieval. The DeclarativeAgent (squares) are an upper envelope of the golden frontier except for Gemini-FlashLite. Embedding-retrieval (empty shapes) and ImparativeAgent (triangles) are well inside the envelope. Table 6: Compliance and efficiency metrics for the ImperativeAgent. “Writes pre-verify” is the raw share of write tool calls that fired before log verification succeeded. Agent
Retrieval
N
Unauth.
Over-retry
Write pre-verify
Traj. len.
Baseline Baseline Declarative Declarative Imperative Imperative
Golden Embedding Golden Embedding Golden Embedding
93 95 93 95 90 93
4.3% 3.2% 5.4% 1.1% 4.4% 3.2%
1.1% 1.1% 1.1% 0.0% 6.7% 4.3%
6 / 441 = 1.4% 4 / 518 = 0.8% 7 / 492 = 1.4% 2 / 484 = 0.4% 7 / 383 = 1.8% 16 / 432 = 3.7%
15.7 21.6 16.7 22.3 16.4 23.7
From Table 6, we see that the ImperativeAgent’s unauthorized-write rate is in fact not lower than the baseline’s (4.4% vs. 4.3% on golden; constant at 3.2% on embedding). On the granular per-write measure, the ImperativeAgent using embedding retrieval actually has the highest share of pre-verification, meaning unauthorized, writes in the table. In addition, the over-retry rate is 4–7× higher (6.7% golden, 4.3% embedding) for the ImperativeAgent than Baseline or Declarative (≤ 1.1%) and the Trajectory length is not shorter. This implies that the verification gate property of the ImperativeAgent is not effective in practice. Indeed, while TRIAGE→VERIFICATION is gated on a successful identification tool call, a model can go into EXECUTION through a phase mis-classification and still write before verifying. We see this from the fact that the ImperativeAgent over-retry is at 6× the rate of the baseline. This demonstrates the brittleness of code-based actions as opposed to the adaptive LLM-based actions. Detailed inspection of the traces also showed how the DeclarativeAgent was able to better handle complex tasks. The baseline LLM agent, for example, frequently failed to issue a log verification call before state-mutating tools, misordered multi-step requests, and re-asked
12
the user for information already returned by KB search. On the other hand, the DeclarativeAgent was able to handle these types of subtasks correctly through the use of its declarative skill files.
9
Discussion and Conclusion
We show that agent skill files act as a procedural prior whose benefit scales with the model’s procedural-competence gap: the DeclarativeAgent improves on the baseline on four of the five models, with the gain trending downwards on stronger models. Research question RQ3, that retrieval quality dominates the orchestration choice, is clearly demonstrated. On the other hand, the use of imperative verification was shown to not reduce unauthorized writes. This proves the brittleness of the ImperativeAgent approach; phase mis-classification is able to route actions past the deterministic gate, eliminating any potential compliance benefit of the imperative paradigm. Theoretically, agent skill files reduce HM (A∗ | ht ) without modifying ht , so the lift they provide is bounded by the procedural-competence gap of the underlying model and by the quality of the observation channel. The imperative agent’s restricted per-phase action sets shrink the policy class; the resulting capacity loss can only be recovered if the restriction prevents enough costly mistakes to offset the reduction of capacity, which is not the case in practice. From an engineering deployment perspective, skill-augmented system prompts add on a marginal cost per-task LLM cost, while delivering accuracy benefits under high-quality retrieval. Latency is unaffected when using prompt caching since the agent skill block is static. Maintainability of the DeclarativeAgent is straightforward: skill files are markdown editable by domain experts and can be versioned alongside business-policy document. For regulated domains, the empirical refutation of the imperative state-machine compliance guarantee is a useful result, in that deterministic gates do not necessarily improve compliance. We conclude with three main points. First, a declarative agent using natural-language skill files gives a measurable accuracy improvement on tool-using LLM agents whose underlying model has a procedural-competence gap, at a modest cost premium. Second, the imperative state-machine paradigm trades capacity for compliance guarantees that may not be successful in practice. Third, retrieval quality remains a dominant bottleneck for tool-using AI agents: when the observation channel is noisy, no orchestration paradigm can recover the lost information, and advances in retrieval should be prioritised alongside advances in orchestration if agentic systems are to reach their commercial potential. Declaration of generative AI in the manuscript preparation process. During the preparation of this work the authors used Claude Code in compiling the experimental results and producing an initial version of the paper. After using this tool, the authors reviewed and heavily modified all of the content and take full responsibility for the content of the published article. Funding disclosure. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
References [1] Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres. τ Knowledge: Evaluating Conversational Agents over Unstructured Knowledge. Sierra Research / Princeton, arXiv:2603.04370, 2026. https://arxiv.org/abs/2603.04370. [2] Yao, S., Shinn, N., Razavi, P., Narasimhan, K. τ -Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. ICLR, 2025.
13
[3] Sierra Research. τ -Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (Codebase). GitHub repository, 2025. https://github.com/sierra-research/ tau2-bench. [4] Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S. The Complexity of Decentralized Control of Markov Decision Processes. Mathematics of Operations Research, 27(4):819–840, 2002. [5] Anthropic. Agent Skills: Composable, Model-Read Procedural Knowledge for LLM Agents. https://agentskills.io, 2025. [6] Firecrawl. How SKILL.md Files Work and Why They’re Everywhere. Firecrawl Blog, 2026. https://www.firecrawl.dev/blog/agent-skills. [7] LlamaIndex. Files for AI Agents: Context, Search, Skills Guide. LlamaIndex Blog, 2026. https://www.llamaindex.ai/blog/files-are-all-you-need. [8] Anonymous. Recursive Language Models. arXiv:2512.24601, 2025. https://arxiv.org/abs/ 2512.24601. [9] Zhang, A. Recursive Language Models. Blog post, 2025. https://alexzhang13.github.io/ blog/2025/rlm/. [10] Chase, H. et al. LangGraph: Building Stateful, Multi-Actor Applications with LLMs. LangChain, 2024. https://langchain-ai.github.io/langgraph/. [11] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR, 2023. [12] [Redacted for blind review]. Benchmarking Customer Support LLM Agents for BusinessAdherence. EACL Industry Track, 2026. https://aclanthology.org/2026.eacl-industry. 15.pdf. [13] Toloka AI. TAU-bench extension: benchmarking policy-aware agents in realistic settings. Toloka AI Blog, 2026. https://toloka.ai/blog/ tau-bench-extension-benchmarking-policy-aware-agents-in-realistic-settings/. [14] Sierra Research. τ 3 -Bench: Advancing Agent Benchmarking to Knowledge and Voice. Sierra Blog, 2026. https://sierra.ai/blog/ bench-advancing-agent-benchmarking-to-knowledge-and-voice. [15] Kahn, A.B. Topological sorting of large networks. Communications of the ACM, 5(11):558– 562, 1962.
Appendix 9.1
Running the Experiments
A pilot run (5 tasks, 2 conditions) and full experiment (97 tasks, 4 conditions) are available via the project Makefile: 1 2
# 5 - task pilot make pilot
3 4 5
# Full 97 - task experiment make experiment
14
9.2
File Index
src/agents/baseline agent.py BaselineAgent: tau2 LLMAgent + generate cached, no skills, no orchestration. src/agents/declarative agent.py DeclarativeAgent: LLMAgent with <skills>-block injection. src/skills/*.md Skill files (banking-procedures, customer-interaction, knowledge-discovery) consumed by the declarative agent. src/agents/state.py AgentState Pydantic model. src/agents/imperative agent.py ImperativeAgent implementation. src/agents/cached generate.py Drop-in generate cached with Anthropic-/DeepSeek-style prompt caching and DeepSeek reasoning-content passthrough. src/agents/register.py Factories registering baseline agent, declarative agent, imperative agent with the tau2 registry. src/analysis/safety metrics.py Offline compliance/ efficiency metric script.
configs/baseline.yaml, configs/baseline-haiku.yaml, configs/scaling-flash.yaml, configs/scalin Run configs for the evaluations —————————————————–
9.3
DeclarativeAgent system prompt
The skill-file declarative agent uses the following system prompt template. Three Markdown skill files (banking-procedures, customer-interaction, knowledge-discovery) are concatenated with --- separators into the <skills> block at agent construction time. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
< instructions > You are a customer service agent that helps the user according to the < policy > provided below . In each turn you can either : - Send a message to the user . - Make a tool call . You cannot do both at the same time . Try to be helpful and always follow the policy . </ instructions > < policy > { domain_policy } </ policy > < skills > ## SKILL : banking - procedures ... markdown content ... --## SKILL : customer - interaction ... markdown content ... --## SKILL : knowledge - discovery ... markdown content ... </ skills >
9.4
ImperativeAgent Implementation Details
This appendix collects the listings that implement the six deterministic strategies. 15
AgentState fields 1 2 3
# Explicit boolean gates for VERIFICATION and EXECUTION user_identified : bool = False verified : bool = False
4 5 6 7
# Explicit task queue for ordered multi - step execution pending_tasks : list [ str ] = Field ( default_factory = list ) completed_tasks : list [ str ] = Field ( default_factory = list )
8 9 10
# Per - tool retry tracking too l_retr y_coun ts : dict [ str , int ] = Field ( default_factory = dict )
11 12 13
# Response - type violation counter e x p e c t _ v i o l a t i o n _ c o u n t : int = 0
Strategy 1: Explicit task queue PLANNING instructs the model to emit a structured block: 1 2 3 4
TASKS : 1. Request credit limit increase 2. File transaction dispute END_TASKS
The parsed list is stored in state.pending tasks and injected into the EXECUTION phase instruction as a <task queue> hint: 1 2 3
4 5 6
queue_hint = ( " \n < task_queue >\ n " f " Pending : ␣ { ’ , ␣ ’. join ( f ’{ i +1}. ␣ { t } ’ ␣ for ␣i , ␣ t ␣ in ␣ enumerate ( state . pending_tasks ) ) }\ n " f " Completed : ␣ { ’ , ␣ ’. join ( state . completed_tasks ) ␣ or ␣ ’ none ’}\ n " " </ task_queue > " )
After each successful execution tool call, update task state() pops the first item from pending tasks into completed tasks. EXECUTION transitions to CONFIRMATION only when pending tasks is empty. Strategy 2: Topological task ordering (Kahn’s algorithm) 1 2 3 4 5 6 7
M US T_ PR EC ED E_ RU LE S : list [ tuple [ str , (" credit limit " , " dispute ") , (" open " , " clos ") , (" transfer " , " clos ") , (" replacement " , " clos ") , (" balance " , " clos ") , ]
str ]] = [ # credit limit before dispute # open account before closing # transfer funds before closure # resolve replacement before closure # clear balance before closure
8 9 10 11 12 13 14 15
def _ s o r t _ t a s k s _ b y _ d e p e n d e n c i e s ( self , tasks : list [ str ]) -> list [ str ]: n = len ( tasks ) predecessors = [ set () for _ in range ( n ) ] for i , ti in enumerate ( tasks ) : for j , tj in enumerate ( tasks ) : if i == j : continue for kw_a , kw_b in M US T_ PR EC ED E_ RU LE S :
16
16
17 18 19 20 21 22 23 24 25 26 27
if kw_a . lower () in ti . lower () and kw_b . lower () in tj . lower () : predecessors [ j ]. add ( i ) # i must precede j queue = [ i for i in range ( n ) if not predecessors [ i ]] result = [] while queue : idx = queue . pop (0) result . append ( tasks [ idx ]) for j in range ( n ) : predecessors [ j ]. discard ( idx ) if not predecessors [ j ] and tasks [ j ] not in result : queue . append ( j ) return result
Strategy 3 & 4: State-driven transitions and EXECUTION hard gate 1 2
3 4
if current == " TRIAGE " : if state . user_identified : successful ID lookup return " VERIFICATION " ...
# set by _up da te _t as k_ st at e on
5 6 7
8 9
if current == " EXECUTION " : if not state . verified : verification return " VERIFICATION " ...
# hard gate : never execute without
Strategy 5: Tool retry policy with deterministic escalation 1 2 3 4
@dataclass class ToolRetryPolicy : max_retries : int failure_phase : str
# phase to enter after exhausting retries
5 6 7 8 9 10 11
TOO L_RETR Y_POLI CY : dict [ str , ToolRetryPolicy ] = { " log_verification ": ToolRetryPolicy (3 , " c a l l _ d i s c o v e r a b l e _ a g e n t _ t o o l ": ToolRetryPolicy (2 , " u n l o c k _ d i s c o v e r a b l e _ a g e n t _ t o o l ": ToolRetryPolicy (2 , " KB_search ": ToolRetryPolicy (4 , }
" ESCALATE ") , " ESCALATE ") , " ESCALATE ") , " ADVISORY ") ,
The determine phase() method checks retry counts first, before all other logic, so no phase can override the escalation: 1 2 3
failure_phase = self . _ r e t r y _ l i m i t _ e x c e e d e d ( state ) if failure_phase and current not in (" ESCALATE " , " COMPLETE ") : return failure_phase
Strategy 6: Response-type enforcement 1
2 3 4
def _enforce_expect ( self , phase , tools_arg , tool_choice , messages , state ): expect = PHASES [ phase ][" expect "] for attempt in range ( M AX _E XP EC T_ RE TR IE S + 1) : response = generate ( model = self . llm , tools = tools_arg ,
17
5
6 7 8 9 10 11 12 13 14 15
16
tool_choice = tool_choice , messages = messages , ...) got_tool = response . is_tool_call () got_text = response . has_text_content () and not got_tool if expect == " either ": return response if expect == " tool_call " and got_tool : return response if expect == " text " and got_text : return response state . e x p e c t _ v i o l a t i o n _ c o u n t += 1 if expect == " tool_call ": tool_choice = " required " # force tool use on retry elif expect == " text ": messages = [ correction_msg ] + messages [1:] # inject correction return response # best - effort fallback
18