arXiv:2605.16205v1 [cs.AI] 15 May 2026
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP Igor Bogdanov
Chung-Horng Lung
Thomas Kunz
[email protected] Carleton University Ottawa, Ontario, Canada
[email protected] Carleton University Ottawa, Ontario, Canada
[email protected] Carleton University Ottawa, Ontario, Canada
Jie Gao
Adrian Taylor
Marzia Zaman
[email protected] Carleton University Ottawa, Ontario, Canada
[email protected] Defence R&D Canada Ottawa, Ontario, Canada
[email protected] Cistel Technology Ottawa, Ontario, Canada
Abstract
CCS Concepts
Deploying compound LLM agents in adversarial, partially observable sequential environments requires navigating several interacting design dimensions: (1) what the agent sees, (2) how it reasons, and (3) how tasks are decomposed across components. Yet practitioners lack guidance on which design choices improve performance versus merely increase inference costs. We present a controlled study of compound LLM agent design in CybORG CAGE-2, a cyber defense environment modeled as a Partially Observable Markov Decision Process (POMDP). Reward is non-positive, so all configurations operate in a failure-mitigation mode and errors compound over time. Our evaluation spans five model families, six models, and twelve configurations (3,475 episodes) with tokenlevel cost accounting. We systematically vary context representation (raw observations vs. a deterministic, programmatic environment state-tracking layer with compressed history), deliberation (self-questioning, self-critique, and self-improvement tools, with optional chain-of-thought prompting), and hierarchical decomposition (monolithic ReAct vs. delegation to specialized sub-agents). We find that: (1) Programmatic state abstraction delivers the largest returns per token spent (RPTS), improving mean return by up to 76% over raw observations. (2) Distributing deliberation tools across a hierarchy degrades performance relative to hierarchy alone for all five model families, reaching up to 3.4× worse mean return while using 1.8-2.7× more tokens. We call this destructive interaction pattern a deliberation cascade. (3) Hierarchical decomposition without deliberation tools achieves the best absolute performance for most models, and context engineering is generally more cost-effective than deliberation. These findings suggest a design principle for structured adversarial POMDPs: invest in programmatic infrastructure and clean task decomposition rather than deeper per-agent reasoning, as these strategies can interfere when combined.
• Computing methodologies → Artificial intelligence; Partiallyobservable Markov decision processes; Multi-agent systems; Intelligent agents; • Security and privacy → Network security.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). ACM CAIS ’26, San Jose, CA, USA © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2415-2/26/05 https://doi.org/10.1145/3786335.3813149
Keywords compound AI systems, LLM agents, hierarchical agent architectures, tool-augmented language models, context engineering, inferencetime scaling, token efficiency, cost–performance trade-offs, adversarial POMDP, autonomous cyber defense, catastrophic failures ACM Reference Format: Igor Bogdanov, Chung-Horng Lung, Thomas Kunz, Jie Gao, Adrian Taylor, and Marzia Zaman. 2026. Context, Reasoning, and Hierarchy: A CostPerformance Study of Compound LLM Agent Design in an Adversarial POMDP. In ACM Conference on AI and Agentic Systems (ACM CAIS ’26), May 26–29, 2026, San Jose, CA, USA. ACM, New York, NY, USA, 29 pages. https://doi.org/10.1145/3786335.3813149
1
Introduction
Practitioners build compound LLM agents by composing three design dimensions: context engineering [6] (what the agent sees), deliberation (reasoning depth), and hierarchical decomposition (task distribution). While often assumed to be additive, these choices interact destructively in adversarial, partially observable sequential environments. We present a controlled empirical study of these interactions within a compound LLM agent defending a network in the CybORG CAGE-2 POMDP. By systematically ablating context, reasoning, and hierarchy across five model families and six models, we measure both task performance and token cost, revealing that what the agent sees is a more reliable lever than how deeply it thinks: deterministic programmatic context yields large gains at near-zero marginal cost, whereas distributing deliberation across a hierarchy often degrades performance while inflating token consumption. We term this failure mode a deliberation cascade. While bounded hierarchy often achieves the best absolute return, context engineering delivers the best returns per token, making it the most cost-effective first investment. The Empirical Gap. Three gaps motivate this work. (1) Multiagent research emphasizes topology (wiring) over internal agent design [9], leaving the interaction between internal configuration
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
and hierarchy underexplored. (2) While context engineering is widely endorsed [6], controlled evidence on the cost-performance trade-offs of specific context components in sequential settings is limited. (3) The deliberation and multi-agent literatures remain disconnected, and we show that combining them can introduce failure modes invisible to either in isolation. Research Question. Our central question is: which compoundagent design dimensions deliver performance gains per token invested, and how do they interact when composed across a multi-agent hierarchy? We decompose this into: RQ1 (Context): value of programmatic abstraction vs. raw observations; RQ2 (Reasoning): impact of deliberation tools in monolithic vs. hierarchical agents; and RQ3 (Composition): when decomposition helps or hurts. Contributions. We present four contributions at the intersection of architectural design and system optimization. (1) Cost-effective context engineering. We show that a deterministic state-tracking layer reduces cumulative penalty by 52-76% relative to raw observations for four of six models, dominating raw observation context configurations at near-zero marginal cost. (2) Identification of deliberation cascades. We demonstrate that enabling deliberation tools across a hierarchy degrades performance in all six models (up to 3.4× worse return) while doubling token costs, producing cascading uncertainty. (3) Three-axis Pareto analysis. We conduct a controlled ablation of context, reasoning, and hierarchy across five model families (72 pairs, 3,475 episodes), constructing cost-performance frontiers that consistently place programmatic context on the efficient frontier. (4) Multi-model validation. We show that while qualitative effects (context helps, distributed reasoning hurts) are robust, quantitative magnitudes vary, validating multi-model evaluation as essential for compound AI design. Scope. This paper is a static architectural design-space study: it asks what compound-agent architecture to build at deployment time, before any runtime adaptation. We scope our claims to structured adversarial POMDPs.
2
Background & Problem Definition
We study compound LLM agents operating in an adversarial, partially observable sequential decision problem and evaluate designs jointly on (1) task return and (2) token cost. The agent architecture and the multi-dimensional ablation study mirror our engineering ladder for solving CAGE-2 with an LLM agent: we began with raw environment observations, then introduced deterministic state tracking and context engineering to make the observations actionable, decomposed the task into a hierarchy as context grew, and finally added deliberation tools to facilitate better decision-making within each agent. This section defines the environment and objectives, specifies the execution scaffold and initialization scope, and establishes tokens as the cost primitive. CybORG CAGE-2. We evaluate on CybORG CAGE-2 [19, 22], an adversarial POMDP modelling network defense. A defender protects a 13-host network against an attacker that follows a scripted, non-adaptive multi-stage kill chain [8] over 𝑇 =30 steps. The attacker does not respond to the defender’s actions, but host attributes,
Bogdanov et al.
processes, and the progression of the kill chain vary stochastically across runs. The defender chooses from five actions (Monitor, Analyse, Remove, Restore, Decoy) with asymmetric costs. Reward 𝑟𝑡 ≤ 0 penalizes compromise and intervention; we report Í episodic return 𝐺 = 𝑇𝑡=1 𝑟𝑡 (closer to zero is better). Partial observability requires sustaining situational awareness under noisy indicators. (See Appendix H for full details.) Configuration-Driven ReAct Scaffold. Agents follow a ReAct [24] loop, iterating between deliberation and tool use until emitting an answer. We separate a reusable ReAct Agent engine (I/O, parsing) from declarative YAML "personality" configurations (prompts, tools). This separation ensures reproducibility and allows architectural variants to be defined as configuration changes rather than code modifications. Knowledge-Free Initialization. To isolate architectural effects from domain engineering, we impose a knowledge-free scope at 𝑡=0. Agents receive only a one-sentence role instruction and a compact action reference table. No network topology, attacker kill-chain details, host-value assignments, or defense heuristics are provided, and the prompt never mentions the environment by name. Performance gains arise from model internal knowledge, runtime context, and deliberation. Models’ internal knowledge may include information about CAGE-2 acquired during pre-training. To minimize the consequences of this exposure, no benchmark-specific cues are provided at runtime. However, influence from pre-training may still be present. All LLMs receive an identical set of prompts and tools. Tokens as Cost Primitive. We use total number of tokens per episode (prompt + completion) as the primary cost metric, mapping directly to billed usage and correlating with latency. We use raw token counts because they do not depend on provider pricing. Appendix F reports prompt/completion splits by model and configuration for pricing-weighted reinterpretation. We instrument all LLM calls, aggregating prompt and completion tokens across the Planner and sub-agents. Pairing return with token cost enables our central evaluative question: which design choices deliver the largest returns per token spent (RPTS), and which inflate the cost.
3
Compound Agent System Design
Our system couples a deterministic backbone with an LLM decision engine and spans four layers (Table 1). (1) The hierarchy layer allows a Planner to delegate tasks to Analyst and ActionChooser sub-agents via strict JSON contracts. (2) A deterministic infrastructure layer maintains a primitive environment model, history of interactions, and validates actions without model calls. (3) The context engineering layer comprises injections that extend user prompts and connect environment model state and action history by converting them into structured summaries like {network_status}. (4) The reasoning layer follows the ReAct pattern [24], optionally executing deliberation within ReAct loop. Decision Cycle. At each of the 𝑇 =30 steps, an Agent Coordinator receives the raw CybORG observation, updates the deterministic environment model, and inserts the chosen context bundle into
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Table 1: System Modularity and Configuration Layers. The agent architecture is built on four functional pillars, ordered from the system’s top-level decision structure down to its internal reasoning mechanisms. System Layer
Core Aspects
Responsibility
Configurability
1. Hierarchy
Planner, Analyst, ActionChooser, JSON Contracts
Distributes tasks: Planner (decides), Analyst (assesses), ActionChooser (ranks).
Role definitions, tool availability, and I/O contracts in YAML.
2. Infrastructure
Env Model, State Machine, History Log, Action Validator
Maintains deterministic belief state (status, history, decoys) and validates actions via regex.
Fixed deterministic backbone (Python).
3. Context
{network_status}, {history}, {observation}
Transforms raw observations into structured summaries and compressed logs.
YAML templates define which context blocks are injected.
4. Reasoning
ReAct Engine, Tools (Question, Critique, Improve), CoT
Executes the reasoning loop, tool calls, and optional deliberation steps.
Boolean flags toggle deliberation tools and CoT injection.
the Planner prompt. It then starts a fresh step-level Planner instance, validates the emitted action, retries on invalid outputs, and submits the validated action to the environment (Figure 1). Step-level instantiation ensures no hidden conversational state accumulates across steps. All inter-step continuity is explicit in the deterministic state structures and compiled context. Reliability Mechanisms. In an adversarial sequential environment, every invalid action is a wasted step during which the attacker advances unopposed. We therefore treat reliability as load-bearing infrastructure implementing the following: (1) Action validation and retry: the Planner’s output is parsed against the CybORG action schema using regex-based validation; invalid outputs trigger up to three retries with the parsing error injected as feedback. (2) Safe fallback: if all retries fail, the system defaults to Monitor, preserving observability without risking a misapplied intervention. (3) Sub-agent output validation: the ActionChooser’s JSON undergoes tiered parsing (direct parse → normalization → optional repair prompt). Any repair calls are included in token accounting. These mechanisms ensure format compliance. They are designed to minimize harm when invoked.
3.1
Layer 1: Hierarchical Decomposition
The system can run either as a monolithic Planner that directly emits an environment action, or as a fixed three-agent hierarchy consistent with the centralized multi-agent system (MAS) topology described by Kim et al. [9], in which an orchestrator coordinates bounded sub-agents through structured communication. The threeagent split separates three functions: strategic decision-making, localized perception, and bounded action candidate ranking to accommodate increasing context size. The Planner retains sole authority over environment actions but may delegate to two subagents whose outputs are advisory. The Analyst assesses a single host by comparing its current state against the effective baseline and returns a structured JSON assessment (status, anomalies, suspected compromise). The ActionChooser receives the Planner’s situational summary and, when available, the Analyst’s report, returning a ranked list of up to three candidate actions with confidence labels in strict JSON. Sub-agents cannot issue environment actions. The purpose of this split is to simplify the Planner’s decision by constraining what each sub-agent produces. All three agents
share the same engine-personality separation logic, ensuring the observed hierarchy effects reflect role decomposition.
3.2
Layer 2: Deterministic Environment Model
To provide in-context situational awareness, the system maintains a fully deterministic environment model that transforms raw CybORG observations into structured network state description and action history. Critically, this model is computed exclusively from the agent’s own observations and past actions. The system stores environment baseline state upon the first observation and creates a host-indexed data structure. This layer embeds domain-informed observation processing. The engineering choices that form the model shape what the agent perceives. Although the layer prescribes no action-selection decisions (no threat rubrics, no host priorities, and no response heuristics), it still carries inductive bias that may influence the agent’s reasoning. Dynamic Environment Model. The model data structure comprises a dictionary where each host obtains a status: {baseline, changed, unknown, analysed at step 𝑛}, and a history record, an ordered, arrow-delimited record of all actions applied to each host (e.g., Analyse → Remove → Restore). This concise single-host record provides the Planner with additional intervention memory without requiring it to parse a full transcript. Model update mechanism. On each step, the model compares the current observation against the saved baseline state using signaturebased comparison of stable fields (process and service identity), ignoring volatile fields (e.g., transient connections) that produce false positives [8]. The system deterministically updates host status based on the comparison and prior actions (Restore → baseline; Remove → unknown; Analyse → analysed at step 𝑛). Deployed decoys are registered as baseline overrides so that expected decoy processes are incorporated into the effective baseline and only genuinely new processes remain visible as anomalous.
3.3
Layer 3: Context Engineering
Context Injections. Three context injections feed the Planner’s initial prompt via placeholders: {observation}, {history}, and {network_status}. {observation} is the raw CybORG dictionary: a verbose, noisy dump of per-host process tables, network connections, and service states that the LLM must parse unaided.
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 2: Experimental overview, models evaluated, and evaluation hyperparameters.
Figure 1: End-to-end system architecture. The deterministic layer (left) compiles structured context from CybORG observations and assembles the agent prompt. The Planner (right) executes a ReAct loop, optionally delegating to Analyst and ActionChooser sub-agents, before emitting a validated action back to the environment.
{network_status} is a compact JSON list of only non-baseline hosts, each annotated with status (changed, unknown, analysed), recency (Current vs. Past), and the action history applied to that host. It collapses to a single sentence when all hosts are healthy. {history} is a compressed action log that folds consecutive quiet steps (Monitor, no state change) into ranges while preserving full detail for intervention steps, controlling prompt growth over the 30-step episode. {history} thus serves a dual function: it provides temporal context for intervention sequencing and supplies the Planner’s own prior reasoning through programmatically extracted justifications, creating a compressed log of past decisions with explanations. The Initial User Prompt. The initial user prompt template itself contributes nothing beyond a step counter and a closing question. All environment understanding comes from the content injected into these placeholders. Each sub-agent also receives a role-specific initial prompt and context components provided by the Planner. The Analyst receives the target hostname and is asked to assess the host’s situation and the ActionChooser receives a situational JSON from the Planner containing the target host, threat description, severity level, and relevant context from prior steps. Sub-agents see neither the full network status nor the episode history. The Planner decides what to provide, enforcing limited context per role. All structured injections are fully deterministic and their cost is limited to the marginal tokens they add to the prompt (examples in Appendix B).
3.4
Deliberation Tools
To further increase decision-making capabilities each ReAct agent in the hierarchy supports four cumulative levels of deliberation. The deliberation tools implement a self-questioning, self-critique,
Component
Description
Models: 5 families Axis 1: Context Axis 2: Deliberation Axis 3: Hierarchy Default Context Scale
Grok, Llama, Devstral, Qwen, Gemini 6 configs varying {obs}, {hist}, {net} 4 levels: question, critique, improve, COT Delegation vs. delegation + deliberation hist+net: network + history 72 exp.; 3,475 episodes; 283.9M tokens
Models
Grok, Llama, Devstral, Qwen, G2.5FL, G3FP (6 models from 5 families; episode counts in Appendix I)
Eval Config Instances per config Runs per instance Episode length Decoding
10 (standard) 5 30 steps Deterministic (greedy, temp=0)
Metrics Mean episode return Total tokens/episode RPTS Standard deviation Validity guard
Í 𝐺 = 𝑇𝑡=1 𝑟𝑡 ≤ 0; closer to zero is better Prompt + completion; main cost primitive Returns per token spent; Episode return variability Replication across 5 families, 6 Models
Reference Rewards Top DRL agent Simple heuristic Random Agent Sleeping (no-op)
Official CAGE-2 Leaderboard [22] −3.47 −58.83 −154.06 −218.65
and self-refinement cycle inspired by Self-Refine [13] and selfinterrogation techniques [15]; the CoT injection follows Wei et al. [23] and Kojima et al. [10]. Unlike cross-episode reflection (e.g., Reflexion [18]), all deliberation occurs within a single step and carries no memory to future steps. The following three deliberation tools and an explicit chain-ofthought (CoT) prompt injection are executed as additional ReAct turns inside one loop within a single environment step: (1) question: the agent questions its initial reasoning before committing to an action [15]. (2) critique (includes #1): the agent generates an explicit critique of its response [13]. (3) improve (includes #1-2): the agent revises its action in light of the critique [13]. (4) COT (includes #13): an explicit chain-of-thought instruction [10, 23] is added to the system prompt, providing reasoning scaffolding on top of the tools.
4
Experimental Methodology
This section describes the models, evaluation protocol, metrics, and controlled ablation design that produce the evidence base for our findings. Table 2 provides an at-a-glance summary. Models. We evaluate six models from five contemporary model families (Table 2) accessed via OpenRouter (Grok, Llama, Devstral, Qwen) and Google Cloud Gemini API. We adopt a full-coverage design: all six models are evaluated on all 12 configurations across three experimental axes (72 unique model-configuration pairs). This safeguards against reliance on single-model results. All models use
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
deterministic decoding (temperature 0). All models receive identical prompt templates and tool definitions. No per-model tuning is performed. Evaluation Protocol. Each configuration is evaluated over multiple containerized agent instances and multiple runs per instance. Each run is launched with a unique seed and fresh agent state. The standard allocation is 10 instances × 5 runs = 50 episodes per pair. G3FP uses a reduced 25-episode budget. Several G2.5FL/Qwen configs use extended batches (75-100) to resolve uncertainty on key comparisons. These allocation differences do not affect qualitative conclusions, which are validated across all six models (Appendix I). Step-level fresh instantiation ensures token accounting is precise and isolated. Three-Axis Design with Shared Anchor configuration. The experimental design varies three axes. (1) Axis 1 (Context): 6 monolithic Planner configurations varying {obs}, {hist}, and {net} placeholders (Table 1). (2) Axis 2 (Deliberation): 4 cumulative levels adding +question, +critique, +improve, and +COT tools to the monolithic Planner. (3) Axis 3 (Hierarchy): 2 configs: hier-base (delegation to Analyst/ActionChooser, deliberation tools OFF) and hier-delib (delegation + deliberation tools ON on all agents, CoT inactive). Axes 2 and 3 share a common default, the hist+net configuration (structured state + compressed history, no raw obs, no deliberation, no delegation), which emerged from our engineering ladder as the default compound-agent context setting. Axis 1 treats this same configuration as one of six context variants. Metrics and Statistical Reporting. We report mean episode return (sum of rewards 𝑟𝑡 ≤ 0, closer to zero is better) as the primary performance metric, alongside total tokens per episode (prompt + completion) as the cost primitive. To jointly evaluate cost and performance, we define returns per token spent (RPTS) as the mean return improvement over the observation-only baseline per kilotoken consumed: RPTS =
𝐺 config − 𝐺 obs KTokconfig
where 𝐺 obs is the per-model mean return under the obs configuration and KTok is total tokens (thousands) per episode. RPTS identifies which design choices deliver the largest gains per unit of inference budget; negative values indicate configurations that perform worse than unstructured observation alone. To capture tail risk in this failure-mitigation setting, we report standard deviation and catastrophic failure rate (fraction of episodes < −150). We adopt multi-model replication as the primary validity guard: findings must replicate across the majority of the tested models. Reward-Scale Reference Points. To ground the reward scale, we reference the public CAGE-2 leaderboard [22] (30-step, B_line setting). Reference reward values are listed in Table 2.
5
Results
All configurations follow the protocol and metrics defined in Section 4. Table 3 presents the complete results matrix. Appendix E reports additional statistical support, including 95% confidence intervals for mean returns and paired difference confidence intervals
for key comparisons. These analyses preserve the qualitative direction of the main findings while highlighting model-dependent uncertainty. We organize the analysis into four subsections corresponding to context (RQ1), the interaction between deliberation and hierarchy (RQ2, RQ3), cost-performance frontiers, and robustness.
5.1
Finding 1: Programmatic State Abstraction Delivers the Largest Gains Per Token
Context engineering ablations reveal that a deterministic layer compiling observations into structured summaries delivers the largest return on token investment.
Figure 2: Context engineering heatmap. Each cell shows mean episode return for one model-context pair. Darker shading indicates worse (more negative) returns. The obs+net and hist+net(Planner-Only) columns are consistently strong across models, while obs alone is typically poor.
5.1.1 Raw observation alone is ineffective. Feeding unprocessed CybORG dictionaries (obs) yields the worst or near-worst performance for five of six models (leftmost column in Figure 2, Table 3). Llama, Qwen, and G2.5FL fall below −214 mean return, approaching the Sleeping (no-op) agent (−219), with catastrophic failure rates (< −150) reaching 96-98% (Figure 5). The raw format’s verbosity and noise overwhelm the planner. 5.1.2 Programmatic backbone enables raw data utility. Augmenting raw observations with the deterministic {network_status} layer (obs → obs+net) transforms performance without additional LLM calls. Llama improves by 76% (−214.7 → −51.4), Qwen by 71%, and Devstral by 53% (Table 3). The contrast is visible in Figure 2, where the obs+net column is consistently lighter than obs. 5.1.3 Less-but-structured often beats more-but-unstructured. The hist+net anchor configuration (structured state + history, no raw obs) matches or beats the maximum-information obs+hist+net for four models (Llama, G3FP, Qwen, Devstral). Adding raw observations to a clean state summary often dilutes the signal; removing them improves Llama’s return by 44% (compare the hist+net and obs+hist+net columns in Figure 2). 5.1.4 History provides complementary temporal context. Adding compressed action history to raw observations (obs→obs+hist) improves all six models, with Llama gaining 36% and Qwen 57%
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 3: Full results matrix. Mean episode return ± standard deviation (Ret ± SD) and kilotokens per episode (KTok = tokens ×103 ) across all 72 pairs, grouped by experimental axis. Underline ↑ = best within axis; box ✓ = best overall per model; ↓ = worst within axis; box ↓ = worst overall per model. The shared anchor configuration hist+net† serves as baseline for Axes 2 and 3. Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
Ret ± SD
KTok
Ret ± SD
KTok
Ret ± SD
KTok
Ret ± SD
KTok
Ret ± SD
KTok
Ret ± SD
KTok
−98.4±70 −89.9±75 −81.9±78 −47.0±40 ↑ −86.3±31 −112.9±79 ↓
17.7 34.6 33.5 20.9 20.2 29.1
−214.7±22 ↓ −137.4±63 −102.6±73 −51.4±20 ✓ −68.7±37 −57.1±54
8.8 19.2 30.6 12.3 13.0 28.6
−155.1±65 ↓ −133.9±86 −85.3±77 −72.6±48 ↑ −93.3±44 −79.3±77
22.2 26.4 25.1 15.4 14.9 21.3
−218.2±20 ↓ −93.4±72 −69.0±53 −63.1±26 −109.4±59 −61.5±52 ↑
10.3 19.0 18.7 11.4 10.6 16.4
−214.7±23 −172.9±69 −147.8±70 ↑ −200.0±41 −215.4±19 ↓ −208.7±39
68.1 125.2 94.0 79.4 94.4 81.7
−96.8±71 −76.3±67 −82.6±64 −113.7±68 −136.4±49 ↓ −52.0±58 ↑
8.0 15.4 19.4 9.0 7.9 18.0
Axis 2: Deliberation (cumulative, on anchor† ) +question −66.5±62 ↓ 55.6 −104.1±84 ↓ +critique −44.9±45 74.7 −93.4±77 +improve −53.4±47 154.0 −75.0±57 ↑ +COT −42.9±33 ↑ 144.5 −100.8±70
84.1 97.8 115.3 131.3
−53.9±42 −62.8±58 −80.6±70 ↓ −40.9±31 ↑
60.4 106.9 153.1 157.4
−92.3±69 −93.6±67 ↓ −92.4±55 −55.6±43 ↑
45.7 68.9 115.9 162.4
−206.2±37 ↓ −128.6±95 ✓ −168.9±60 −157.0±70
104.8 118.2 182.9 225.7
−100.6±57 ↓ −66.4±63 −64.0±45 −29.9±20 ↑
30.7 41.0 58.1 75.1
Axis 3: Hierarchy (on anchor† , structured context) hier-base −24.0±27 ✓ 141.9 −69.5±60 ↑ hier-delib −40.4±26 ↓ 364.1 −108.0±74 ↓
87.7 158.1
−37.8±37 ✓ −127.4±72 ↓
97.0 257.7
−28.6±37 ✓ −30.1±33 ↓
79.6 209.9
−183.1±62 ↑ −186.4±58 ↓
120.6 270.5
−16.1±3 ✓ −23.6±12 ↓
56.4 104.8
Config Axis 1: Context obs obs+hist obs+hist+net obs+net network hist+net†
(Table 3). However, history’s marginal value depends on what other context is present. When network status is already available, adding history (obs+net vs. obs+hist+net) hurts four of six models (Table 3), suggesting that {network_status} already encodes much of the decision-relevant temporal signal.
configuration) can be destabilized by additional turns. Token costs scale steeply: +COT consumes 3-10× more tokens than the anchor.
The intersection of Axes 2 and 3 reveals the study’s central failure mode: deliberation that helps monolithic agents can degrade hierarchical systems.
5.2.2 Hierarchy without deliberation wins. The hier-base configuration (delegation only, no deliberation tools) achieves the best or near-best absolute performance for four of six models (G3FP at −16.1, Grok at −24.0, approaching the top published DRL result of −3.47 [2]; dark bars in Figure 3). The benefit comes from task decomposition, bounded Analyst assessments and ActionChooser rankings, rather than deeper deliberation. Llama is the exception, degrading by 22% compared to monolithic hist+net (−57.1 → −69.5).
Figure 3: Deliberation cascade effect. Paired bars show mean episode return for hier-base (dark) vs. hier-delib (light) across six models. Degradation ratio annotated above each pair. Mean return decreases for all six models.
5.2.3 Deliberation cascade: the destructive interaction. Enabling deliberation tools on all hierarchical agents (hier-delib) decreases mean return relative to hier-base for all six models (Table 3, Figure 3). Devstral worsens by 3.37× (−37.8 → −127.4), Grok by 1.68×, and Llama by 1.55×, while token costs typically double (1.8-2.7×). This compositional failure arises from deliberation cascade: independent deliberation loops in the Analyst, ActionChooser, and Planner amplify uncertainty without an arbitration protocol. Moreover, for four of six models (Grok, Devstral, Qwen, G3FP), hier-base matches or outperforms the best monolithic deliberation at comparable or lower token cost, achieving through task decomposition what deliberation attempts through deeper reasoning. The degradation under hier-delib is not attributable to token overhead per se: hier-base consumes comparable tokens to monolithic +COT (e.g., Grok: 141.9K vs. 144.5K) yet achieves substantially better returns (−24.0 vs. −42.9), confirming the loss is structural.
5.2.1 Monolithic deliberation is model-dependent. Adding reasoning tools to the monolithic Planner produces model-dependent effects (Table 3, Axis 2). Explicit chain-of-thought (+COT) is the best deliberation level for four models (Grok, Devstral, Qwen, G3FP), often improving on the anchor configuration. However, patterns are non-monotonic, and stronger baselines (e.g., Llama on anchor
Mechanistic evidence: passivity amplification under distributed deliberation. Auditing a matched episode trace from Devstral run reveals a recurring pattern of passivity amplification: under low-severity contexts with multiple active decoys, the deliberative ActionChooser converges on "avoid redundant interventions" and defers remediation until evidence becomes unambiguous. For
5.2
Finding 2: Deliberation Destructively Interferes with Hierarchy
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 4: Cost-performance Pareto frontiers. Points shaped by axis (circles: context, triangles: deliberation, squares: hierarchy). Context configurations occupy the low-cost efficient region; hier-delib is high-cost and dominated. example: at step 24 User4 has been recently analyzed and instrumented with multiple decoys. The ActionChooser’s deliberation prioritizes passive observing: Monitor (0.9) > Analyse User4 (0.8) > Restore User4 (0.7). The Planner adopts the top recommendation. On the next step, the Analyst flags severity=high with concrete anomalies (unusual outbound traffic, unknown processes), and the Planner overrides and restores User4, with a critical delay as in a failure-mitigation regime, even single-step deferral prolongs highpenalty states. Distributed deliberation overestimates immediate "stability" cues under partial observability, yielding high-confidence deferral until evidence becomes overwhelming (See Appendix G).
5.3
Finding 3: Context Engineering Dominates the Cost-Performance Frontier
We construct per-model Pareto frontiers over all twelve configurations (tokens per episode vs. mean return, Figure 4). Three patterns emerge. (1) Across per-model Pareto frontiers, at least one context configuration is efficient for every model, typically obs+net or hist+net. (2) Deliberation configurations reach the frontier for only a minority of models and are often dominated by context or hierarchy. (3) hier-base sets the high-performance frontier where hierarchy helps, while hier-delib is consistently dominated by hier-base, combining higher token cost with worse mean return.
Table 4: RPTS across all configurations (Section 4). Higher is better; negatives = worse than obs. Bold = best per model. Config
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
Axis 1: Context obs+hist 0.25 obs+hist+net 0.49 obs+net 2.46 network 0.60 hist+net† −0.50
4.03 3.66 13.28 11.23 5.51
0.80 2.78 5.36 4.15 3.56
6.57 7.98 13.60 10.26 9.55
0.33 0.71 0.19 −0.01 0.07
1.33 0.73 −1.88 −5.01 2.49
Axis 2: Deliberation +question 0.57 +critique 0.72 +improve 0.29 +COT 0.38
1.32 1.24 1.21 0.87
1.68 0.86 0.49 0.73
2.75 1.81 1.09 1.00
0.08 0.73 0.25 0.26
−0.12 0.74 0.56 0.89
Axis 3: Hierarchy hier-base 0.52 hier-delib 0.16
1.66 0.67
1.21 0.11
2.38 0.90
0.26 0.10
1.43 0.70
Return per Token Spent. Table 4 reports RPTS for all non-baseline configurations. Context configurations dominate: obs+net achieves the highest RPTS for Grok, Llama, Devstral, and Qwen, while hist+net leads for G3FP. G2.5FL is the exception, with +critique
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Figure 5: Catastrophic failure rate (return < −150) by model and configuration. G2.5FL fails across all configurations; context engineering reduces catastrophic rates for most other models. performing best, consistent with its difficulty exploiting structured context. Hierarchy often improves absolute return but is less token-efficient: hier-base consumes substantially more tokens than obs+net, yielding lower RPTS despite stronger mean return. hier-delib remains high-cost and low-efficiency relative to hier-base. As a pricing sensitivity check, we re-weighted token costs using provider-specific input/output pricing ratios (Appendix F); this did not reverse any qualitative conclusion, although it narrowed the relative cost advantage of context over hierarchy.
5.4
Robustness, Variance, and Tail Risk
Qualitative effects (context helps, distributed deliberation hurts) are consistent across models, and magnitudes vary by 2-10×. Standard deviations (SD) of episode return, reported alongside means in Table 3, reveal how each design axis affects outcome variability. Context engineering compresses both mean and variance. Programmatic state abstraction reduces SD alongside mean return (e.g., Grok: 70 to 40, Devstral: 65 to 48) and reduces catastrophic failure (< −150) from > 90% under obs to < 10% in the strongest cases (Figure 5). Bounded hierarchy further tightens outcomes: G3FP under hier-base achieves SD = 3, and Grok’s SD falls from 79 to 27. Conversely, hier-delib increases variance: Devstral’s SD rises from 37 to 72 and its catastrophic rate jumps from 2% to 44%, consistent with the deliberation cascade. G2.5FL fails catastrophically (> 48%) across all configurations, suggesting a capability floor for the structured I/O compliance the architecture requires; positive claims do not depend on G2.5FL. Hierarchy provides limited or negative value for two models. Llama worsens under hierarchy (−57.1 → −69.5, 22%) with SD remaining high at 60. G2.5FL improves only marginally (12%; −208.7 → −183.1). The remaining four models improve substantially (52-79%). Multi-model evaluation is thus essential: a Llama-only study would conclude hierarchy hurts, whereas a Grok-only study would assert it is important.
6
Discussion
Our findings share a unifying theme: in adversarial sequential POMDPs, the value of a design choice is determined by information flow through the system rather than per-component merit. We distill three design principles (RQ1-RQ3).
Principle 1: Invest in deterministic infrastructure before LLM reasoning (RQ1). The programmatic state-tracking layer delivers the largest consistent gains per token by shifting the LLM from perceptionplus-reasoning to reasoning-over-state. Knowledge-free agents rely solely on this scaffolding, so gains come from presentation and uncertainty compression, not domain expertise. Context engineering also compresses tail risk: catastrophic failure (return < −150, roughly the Random Agent level) drops from > 90% under obs to < 10% under obs+net or hist+net in the strongest cases (Figure 5), making it the most reliable lever for average and worst case. Principle 2: Decompose into bounded specialists, not reflective generalists (RQ2, RQ3). Hierarchy without deliberation (hier-base) achieves best or near-best absolute performance for four of six models. The benefit is interface constraints: the Analyst gives a bounded assessment and the ActionChooser a ranked list, turning an open-ended generation problem into a verifiable decision. Principle 3: Do not distribute deliberation without an uncertaintyresolution protocol (RQ2, RQ3). Enabling deliberation across all hierarchical agents degrades performance and inflates cost. Independent critique loops create cascading uncertainty. When a sub-agent critiques its own answer, it introduces excessive qualifications. The consuming agent cannot distinguish these from genuine warnings about the environment, so caution accumulates through the hierarchy. If deliberation is needed across a hierarchy, centralize it or use explicit mediation (e.g., confidence gating, aggregation rules). Model dependence and practical ordering. Effect directions (context helps, distributed deliberation hurts) hold across models, but magnitudes vary by 2-10× and some effects flip (e.g., Llama harmed by hierarchy). Single-model studies would thus contradict each other. The best first step differs by model (context for weak rawobservation handling, hierarchy for strong baselines). The observed Pareto ordering follows our engineering trajectory: (1) context is the most reliable lever; (2) bounded hierarchy sets the ceiling; (3) monolithic deliberation is capability-conditional; (4) distributed deliberation is dominated. Our results complement topology-focused scaling [9] by showing that internal configuration determines whether decomposition helps or hurts, revealing failure modes invisible to topology-only analyses. Each layer builds on the previous, and the
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
value of hierarchy and deliberation is conditional on the quality of infrastructure and context. Transferability. The effect directions (Principles 1–3) replicate across six models from five families, making them directional starting points rather than hard prescriptions; specific magnitudes are tied to CAGE-2 and the models tested. A practitioner should: (1) add structured programmatic context (environment model, history tracking, observation decomposition, baseline state); (2) compare monolithic configurations against raw observations; (3) test bounded hierarchy if feasible; and (4) avoid distributing deliberation tools across sub-agents by default. The state-tracking layer is likely to transfer where observations are structured enough that a program can track what has changed and why, and most steps are routine so history compresses without losing important information.
7
Related Work
Our study sits at the intersection of four research streams that prior work typically addresses in isolation. Multi-Agent Architecture and Scaling. Recent work categorizes coordination protocols [16, 21], derives topology scaling laws [9], and proposes structured communication formats [7, 20]. These studies vary wiring while treating nodes as fixed; we hold wiring fixed and ablate internal configuration, revealing failure modes like deliberation cascades that are invisible to topology-only analyses. Autonomous Cyber Defense. CybORG CAGE-2 [8, 22] has been addressed with RL [1], model-based planning [4], and particle filtering [12]. LLM-based defenders [3, 14] are newer but lack controlled architectural ablations. We provide the first cost-performance analysis of compound LLM design decisions in CAGE-2. Context Engineering. Context design is a critical lever [6], supported by tooling ecosystems [11] and algorithmic context evolution [25]. We contribute an orthogonal, controlled ablation of context composition (raw vs. structured) in a POMDP, showing that deterministic state abstraction outperforms raw observations at near-zero marginal cost. Deliberation and Self-Critique. Intra-step deliberation techniques, such as chain-of-thought [10, 23], self-interrogation [15], and selfrefinement [13], operate within a single inference call, unlike crossepisode methods such as Reflexion [18]. Recent work shows that self-correction without external feedback can be harmful in monolithic agents [5], and Renze and Guven [17] decompose reflection components but evaluate only monolithic settings. No prior work studies deliberation distributed across a hierarchy. Our deliberation cascade finding extends the single-agent self-correction limitation to compound systems, identifying a compositional failure mode invisible to either literature in isolation.
8
architectural effects without fully disentangling pretrained priors. Deliberation tools are tested only in cumulative activation order; independent activation may yield different interaction patterns. Our model selection spans mid-tier and efficiency-focused families; frontier-scale models may respond differently. Our design space is static. Priorities for future work include independent ablation of individual deliberation tools to isolate which components drive the cascade, selective deliberation placement within the hierarchy (e.g., enabling deliberation on only one sub-agent), and evaluation on frontier-scale models to test whether the observed effects persist at higher capability levels. Extending ablations to diverse environments and designing inter-agent uncertainty arbitration protocols, such as confidence gating or calibrated aggregation, are also important next steps.
Limitations and Future Work
Our claims are scoped to structured adversarial POMDPs where deterministic state abstraction is feasible. We use a single environment (fixed topology, scripted adversary, 30-step horizon) and a single three-agent hierarchy; alternative topologies may exhibit different deliberation cascade dynamics. Token counts proxy cost but do not capture latency or pricing, and knowledge-free prompts isolate
9
Conclusion
We presented a controlled cost-performance study of compound LLM agent design in an adversarial, partially observable sequential environment (CybORG CAGE-2). Across a three-axis ablation of context representation (6 configurations), deliberation depth (4 cumulative levels), and hierarchical decomposition (2 configurations), we evaluated 72 model-configuration pairs spanning five model families, totaling 3,475 episodes and 283.9M tokens. Three conclusions emerge. First (RQ1), context engineering dominates: deterministic programmatic state abstraction yields the largest and most consistent gains per token, while raw observations alone are destabilizing. Second (RQ2), hierarchy can substitute for deliberation: bounded specialist decomposition (hier-base) achieves the best absolute performance for most models through strict I/O contracts rather than deeper per-agent reasoning. Third (RQ3), deliberation is not modular: distributing deliberation tools across a hierarchy (hier-delib) produces a deliberation cascade that degrades returns while increasing token expenditure. A cross-cutting finding reinforces these conclusions: qualitative effects hold across all six models, but magnitudes vary by 2-10× and some reverse sign (e.g., Llama is harmed by hierarchy), validating multi-model evaluation as essential for compound AI research. These results suggest a practical starting point for structured adversarial POMDPs, mirroring the trajectory we followed: build deterministic infrastructure to deliver clean structured context, add bounded hierarchy when models can exploit delegation, and treat deliberation as a costly capability-conditional option rather than a universal upgrade. This is not a universal prescription, as magnitudes are environment-dependent, and practitioners should validate this ordering in their own settings. More broadly, our findings suggest that the science of compound AI systems requires studying topology, node internals, and the interaction effects that arise when individually sensible components are composed. The deliberation cascade identified in this study is one such interaction effect, and designing inter-agent uncertainty arbitration protocols to prevent it is a promising direction for future work. Reproducibility details and ethics considerations are in Appendix A. The archived artifact is available at https://doi.org/10.5281/zenodo.19908100; the development repository is available at https://github.com/isbogdanov/ agent-design-study.
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
References [1] Elizabeth Bates, Vasilios Mavroudis, and Chris Hicks. 2023. Reward Shaping for Happier Autonomous Cyber Security Agents. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec ’23) (Copenhagen, Denmark). Association for Computing Machinery, New York, NY, USA, 221–232. doi:10.1145/3605764.3623916 [2] CardiffUni Team. 2022. CybORG CAGE-2 Winning Agent: PPO + Greedy Decoys. https://github.com/john-cardiff/-cyborg-cage-2. Accessed: 2026-04-28. [3] Sebastián R. Castro, Roberto Campbell, Nancy Lau, Octavio Villalobos, Jiaqi Duan, and Alvaro A. Cardenas. 2025. Large Language Models are Autonomous Cyber Defenders. In Proceedings of the 2025 IEEE Conference on Artificial Intelligence (CAI). 1125–1132. doi:10.1109/CAI64502.2025.00195 [4] Kim Hammar, Neil Dhir, and Rolf Stadler. 2024. Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search. arXiv (2024). arXiv:2407.11070 [cs.CR] doi:10.48550/arXiv.2407.11070 [5] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot SelfCorrect Reasoning Yet. In International Conference on Learning Representations (ICLR). doi:10.48550/arXiv.2310.01798 [6] Andrej Karpathy. 2025. +1 for “context engineering” over “prompt engineering”. X (formerly Twitter) post. https://x.com/karpathy/status/1937902205765607626 Accessed 2026-02-22. [7] Karim Ben Khaled and Davy Monticolo. 2026. G2CP: A Graph-Grounded Communication Protocol for Verifiable and Efficient Multi-Agent Reasoning. arXiv (2026). arXiv:2602.13370 [cs.AI] doi:10.48550/arXiv.2602.13370 [8] Mitchell Kiely, David Bowman, Maxwell Standen, and Christopher Moir. 2023. On Autonomous Agents in a Cyber Defence Environment. arXiv (2023). arXiv:2309.07388 [cs.CR] doi:10.48550/arXiv.2309.07388 [9] Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Mark Malhotra, Paul Pu Liang, Hae Won Park, Yuzhe Yang, Xuhai Xu, Yilun Du, Shwetak Patel, Tim Althoff, Daniel McDuff, and Xin Liu. 2025. Towards a Science of Scaling Agent Systems. arXiv (2025). arXiv:2512.08296 [cs.AI] doi:10.48550/arXiv.2512.08296 [10] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners. In Advances in Neural Information Processing Systems, Vol. 35. doi:10.48550/arXiv.2205.11916 [11] LangChain. 2025. LangChain. Open-source software framework. https://github. com/langchain-ai/langchain Accessed 2026-02-22. [12] Duc Huy Le and Rolf Stadler. 2025. Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model. arXiv (2025). arXiv:2509.06539 [cs.AI] doi:10. 48550/arXiv.2509.06539 [13] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. In Advances in Neural Information Processing Systems, Vol. 36. doi:10.48550/arXiv.2303.17651 [14] Hamoun Mohammadi, Jonathan J. Davis, and Mitchell Kiely. 2025. Leveraging Large Language Models for Autonomous Cyber Defense: Insights from CAGE-2 Simulations. IEEE Intelligent Systems 40 (2025), 29–36. doi:10.1109/MIS.2025. 3568209 [15] Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and Narrowing the Compositionality Gap in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics, 5687–5711. doi:10.18653/v1/2023. findings-emnlp.378 [16] Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025. Scaling Large Language Model-based Multi-Agent Collaboration. In International Conference on Learning Representations (ICLR). doi:10.48550/arXiv.2406. 07155 [17] Matthew Renze and Erhan Guven. 2024. Self-Reflection in LLM Agents: Effects on Problem-Solving Performance. arXiv (2024). arXiv:2405.06682 [cs.AI] doi:10. 48550/arXiv.2405.06682 [18] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. In Advances in Neural Information Processing Systems, Vol. 36. doi:10.48550/arXiv.2303.11366 [19] Maxwell Standen, Martin Lucas, David Bowman, Toby J. Richer, Junae Kim, and Damian Marriott. 2021. CybORG: A Gym for the Development of Autonomous Cyber Agents. arXiv (2021). arXiv:2108.09118 [cs.CR] doi:10.48550/arXiv.2108. 09118 [20] Yichen Tang, Weihang Su, Yujia Zhou, Yiqun Liu, Min Zhang, Shaoping Ma, and Qingyao Ai. 2025. Augmenting Multi-Agent Communication with State Delta Trajectory. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 10219–10240. doi:10.18653/v1/2025.emnlp-main.518
Bogdanov et al.
[21] Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D. Nguyen. 2025. Multi-Agent Collaboration Mechanisms: A Survey of LLMs. arXiv (2025). arXiv:2501.06322 [cs.AI] doi:10.48550/arXiv. 2501.06322 [22] TTCP CAGE Challenge Working Group. 2022. TTCP CAGE Challenge 2. https: //github.com/cage-challenge/cage-challenge-2 Accessed 2026-02-22. [23] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems, Vol. 35. doi:10.48550/arXiv.2201.11903 [24] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR). doi:10. 48550/arXiv.2210.03629 [25] Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. 2025. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. arXiv (2025). arXiv:2510.04618 [cs.LG] doi:10.48550/arXiv.2510.04618
Appendix organization. Appendix A provides reproducibility details and ethics considerations. Appendix B lists the complete YAML definitions for the Planner, Analyst, and ActionChooser. Appendix C documents the deliberation tool schemas and activation flags. Appendix D reports complete results, cross-axis comparisons, distributional analysis, and token cost data for all 72 model–configuration pairs. Appendix E provides 95% confidence intervals and paired mean-return difference confidence intervals. Appendix F breaks down token consumption and prompt/completion shifts. Appendix G provides illustrative trajectory excerpts for the deliberation cascade failure mode. Appendix H summarizes CAGE-2 environment details, network topology, and instance difficulty. Appendix I lists the evaluated episode counts per model and configuration.
A
Reproducibility & Ethics
Ethics. All authors have read and adhere to the ACM Code of Ethics1 . All experiments run within the simulated CybORG CAGE-2 environment [22]; no real networks, live attack infrastructure, or human-subject data are involved. The work is strictly defensive in scope. LLM tools were used for language polishing and data processing scripts; all design decisions, analyses, and claims are authored by the research team. Reproducibility. All models use deterministic decoding (temperature 0 or provider minimum). We release the source code, exact YAML configuration snapshots, episode-allocation metadata, container specification, API-key template, and experiment runner needed to rerun the evaluated variants. The paper reports the aggregate results, token accounting, and episode counts used for the main claims, while the artifact provides the implementation and configuration snapshots needed for inspection and selected reruns. We rely on cross-model replication of qualitative effects rather than single-model statistical significance.
A.1
Artifact Availability and Scope
The artifact supporting this paper is archived on Zenodo at https: //doi.org/10.5281/zenodo.19908100. The development repository is available at https://github.com/isbogdanov/agent-design-study. Detailed build, configuration, and execution instructions are provided in the artifact README. 1 https://www.acm.org/code-of-ethics
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
The artifact contains the agent implementation, experiment runner, container specification, API-key template, and configuration snapshots needed to rerun evaluated variants. The main implementation is in agent_base/, including the CybORG and LLM-agent coordinators, provider-connector configuration, logging utilities, and the YAML-defined Planner, Analyst, and ActionChooser agents. exp_configs/ contains the twelve paper configurations (six context, four deliberation, two hierarchy), each a selfcontained YAML snapshot; switching conditions requires changing definitions_source in experiment_agent_eval.yaml. Experiments are launched via run_experiment.py (10 instances × 5 runs = 50 episodes per pair by default). The Dockerfile builds a Python environment with CybORG CAGE-2 and dependencies; LLM access is supplied through the provided .env.template. A run creates an experiments/ directory with the copied configuration, per-instance reports, aggregate summaries, and tokenusage logs. Full raw LLM-provider transcripts are not bundled due to storage size; original run seeds are also not included, so the artifact supports executable reruns rather than bit-for-bit log regeneration. Original logs may be available upon request. The primary reproducibility target is to enable inspection of the implementation and configuration snapshots, and rerunning of selected configurations.
include_tool_raise_a_question: false include_tool_critique_the_answer: false include_tool_improve_based_on_critique: false include_COT_instruction: false
B
COT_instruction: | You MUST follow your Chain-of-Thought instructions during each thought phase. 1. REVIEW SITUATION: Check network status and step history. Which hosts need attention? What actions have been tried? 2. IDENTIFY TARGET: Select the most critical host or threat to address this step 3. GATHER INFO: If needed, use get_analysis_of_host_update for detailed analysis of a changed host 4. GET SUGGESTIONS: Call get_suggestion_for_next_action with JSON: {"target_host": "hostname", "situation": "description", "severity": "level", "context": "relevant history"} 5. DECIDE: Choose ONE action from suggestions. You may override based on strategic reasoning
Agent Definitions
This appendix provides the complete YAML definitions for all three agents (Planner, Analyst, ActionChooser), demonstrating the near-zero-knowledge starting point described in Section 2. Each agent is defined by three files: core.yaml (identity, model binding, tool flags), initial_prompt.yaml (per-step prompt template), and persistent_knowledge.yaml (domain knowledge). A shared common_knowledge.yaml applies to all agents. All remaining knowledge files, reflection_knowledge.yaml, reflection_examples.yaml, and examples.yaml, are confirmed empty for every agent in every experiment reported in this paper; the learning subsystem that would populate them is disabled throughout. The boolean flags include_tool_raise_a_question, include_tool_critique_the_answer, and include_tool_improve_based_on_critique in each agent’s core.yaml are set to false in the baseline configuration and toggled to true cumulatively for the deliberation axis (Section 3.4). The YAML shown below reflects the baseline (anchor) configuration; deliberation variants differ only in these three flags and in the optional include_COT_instruction injection.
B.1
Shared Configuration
common_knowledge.yaml. , reflection_knowledge: This file is empty, confirming that no shared domain heuristics are injected across agents.
B.2
Planner
planner/core.yaml. agent_type: "ReAct"
add_examples: false add_reflection_examples: true system_message: | You are the strategic decision-maker for the Blue team's cybersecurity defense. Your role is to select ONE action per step that best defends the network. You must balance immediate threats with strategic positioning, considering that you can only act once per step. You MUST follow your <reflection_rules> during each thought phase. Provide a final answer to the question asked by the USER.
rules: - You must select ONLY ONE action for your final Answer from the list of suggestions provided by the 'get_suggestion_for_next_action' tool - Your final Answer MUST be a verbatim copy of the action-string from ONE of the suggestions - TOOLS CANNOT HANDLE MULTIPLE HOSTS, YOU MUST SELECT ONLY ONE SPECIFIC HOST AT A TIME tools: - name: "get_analysis_of_host_update" description: "Provides a detailed analysis and comparison of the state change of the specific host to its baseline"
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
example_calling: "get_analysis_of_host_update: User4" - name: "get_suggestion_for_next_action" description: | Suggests a ranked list of final actions, each with a justification and confidence score based on recent analysis and action history. CRITICAL INPUT REQUIREMENT: Your input MUST be a valid JSON object with these required keys: - "target_host": The specific single hostname - "situation": Brief description of the threat - "severity": Threat level (low|medium|high|critical) - "context": Relevant information from previous steps example_calling: > get_suggestion_for_next_action: {"target_host": "Enterprise1", "situation": "critically compromised with active C2", "severity": "critical", "context": "Remove action failed previously"} is_critical: true The Planner’s two domain-specific tools, get_analysis_of_host_update and get_suggestion_for_next_action, are the interfaces through which the Coordinator spawns the Analyst and ActionChooser sub-agents in hierarchical configurations (hier-base and hier-delib). When delegation is disabled, the Planner does not invoke these tools and instead emits an environment action directly. planner/initial_prompt.yaml. prompt: opening: | You are at step {step_number} now. {network_status} {history} closing: | What action should be taken next? The placeholders {network_status}, {history}, and (in context variants that include it) {observation} are populated deterministically at each step by the environment-state layer described in Section 3.2. The anchor configuration includes {network_status} and {history} but omits {observation}. Context output examples. The {network_status} placeholder is rendered as a JSON list of all non-baseline hosts, annotated with current status, recency, and action history: Network Status: The following hosts have updates or are in a non-baseline state: [ {"host_name": "Enterprise1", "current_status": "changed",
]
"time_of_update": "Current", "applied_actions_so_far": "Analyse->Remove"}, {"host_name": "User4", "current_status": "unknown", "time_of_update": "Past", "applied_actions_so_far": "Remove"}
When all hosts are healthy, a single sentence reports baseline status, keeping the prompt compact. The {history} placeholder is rendered as a compressed action log with smart collapsing: consecutive quiet steps are folded into ranges when no state changes are detected, while steps involving interventions retain full detail: Steps 1-3: Action: Monitor/No action needed. (No state changes observed) Step 4: {"action": "Analyse Enterprise1", "analysis": "Host shows suspicious processes"} Step 5: {"action": "Remove Enterprise1"} Step 6: Action: Monitor planner/persistent_knowledge.yaml. reflection_knowledge: - content: - Description: monitoring network for malicious activity. Example use: Monitor Name: Monitor Type: Passive observing action - Description: it does not remove the host but attempts to remove malicious infection from a host. Example use: Remove hostname=someName Name: Remove Type: Reactive intervention - Description: analyzing a host for malicious activity at deeper system level Example use: Analyse hostname=someName Name: Analyse Type: Passive observing action - Description: restoring a host to a clean state with very high penalty, this action removes all previously deployed decoys from the host Example use: Restore hostname=someName Name: Restore Type: Reactive intervention - Description: deploying a service to act as a decoy to a host to distract the attacker in future steps Example use: DecoySERVICE hostname=someName Name: DecoySERVICE Type: Proactive Protective action header: > INTERPRETATION OF ACTIONS FROM <AVAILABLE_ACTIONS_LIST> root: actions type: json
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
This is the only domain knowledge provided to the Planner (and, identically, to the ActionChooser): a five-entry action-type glossary with names, types, and usage syntax. No tactical heuristics, no threat-assessment rubrics, and no worked examples are included. Empty knowledge files. The following files are empty for all Planner experiments: examples.yaml, reflection_examples.yaml, and reflection_knowledge.yaml.
B.3
Analyst
analyst/core.yaml. agent_type: "ReAct" include_tool_raise_a_question: false include_tool_critique_the_answer: false include_tool_improve_based_on_critique: false include_COT_instruction: false add_examples: false add_reflection_examples: true
Your response MUST STRICTLY be a JSON array of objects that follows the following schema: { "host": "<n>", "status": "baseline|changed|unknown", "severity": "low|medium|high|critical", "anomalies": ["<short bullet>", "..."], "suspected_c2": ["<ip/host>"], "recommended_focus": "containment|investigation|monitor", "rationale": "<2-3 sentences>" } The Analyst receives a one-sentence role identity and two readonly tools (get_host_current_state, get_host_baseline_state) that return deterministic JSON from the environment-state layer. Its output is a structured JSON assessment with a constrained schema. No domain heuristics are provided: its persistent_knowledge.yaml and all example files are empty. analyst/initial_prompt.yaml.
system_message: | You are a cybersecurity analyst. You MUST follow your <reflection_rules> during each thought phase. Provide a final answer to the question asked by the USER. COT_instruction: | You MUST follow your Chain-of-Thought instructions during each thought phase. 1. GET CURRENT STATE: Use get_host_current_state for the target host 2. GET BASELINE: Use get_host_baseline_state to compare against initial state 3. IDENTIFY ANOMALIES: What changed? New processes, connections, missing services? 4. ASSESS SEVERITY: How critical is this compromise? Is there C2 activity? 5. RECOMMEND ACTION: Should we contain, investigate further, or just monitor? tools: - name: "get_host_current_state" description: "Get the current state details for a specific host. The input must be a single hostname." example_calling: "get_host_current_state: Enterprise1" - name: "get_host_baseline_state" description: "Get the baseline state details for a specific host. The input must be a single hostname." example_calling: "get_host_baseline_state: Enterprise1" answer_format: |
prompt: opening: | You must provide an analysis for the host: {hostname}. closing: | What is your understanding of the situation with the host?
B.4
ActionChooser
action_chooser/core.yaml. agent_type: "ReAct" include_tool_raise_a_question: false include_tool_critique_the_answer: false include_tool_improve_based_on_critique: false include_COT_instruction: false add_examples: false add_reflection_examples: true system_message: | You are a tactical cybersecurity advisor for the Blue team. You MUST follow your <reflection_rules> during each thought phase. Provide a final answer to the question asked by the USER. COT_instruction: | You MUST follow your Chain-of-Thought instructions during each thought phase. 1. READ SITUATION: Check SITUATION_JSON for target_host, threat description, severity, and context 2. EVALUATE ACTIONS: Consider available action
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
types and their costs vs benefits 3. RANK THREE: Provide three suggestions with confidence scores (0.0-1.0), highest confidence first answer_format: | Your response MUST STRICTLY be a JSON array of objects, where each object represents a suggested action. Each object must have ONLY the following keys: "action", "confidence". The ActionChooser has no tools, it is a pure generation agent that receives a situation JSON and returns a ranked action list. Its persistent_knowledge.yaml contains the same five-entry action glossary as the Planner (reproduced above). All example files are empty.
critique_the_answer. The agent submits its current reasoning as input. The tool returns this text as a critique prompt, asking the agent to evaluate its own logic: "You have chosen to critique your reasoning: ‘{reasoning}’. Now, provide your critical assessment." This forces a second pass over the agent’s tentative conclusion before action commitment. improve_based_on_critique. After critiquing, the agent submits an improved version of its reasoning. The tool returns: "You have decided to improve your answer based on the critique: ‘{improved_reasoning}’. Now, provide your final answer." This creates a three-phase deliberation cycle: question → critique → improve.
action_chooser/initial_prompt.yaml. prompt: | <SITUATION_JSON> {situational_understanding} </SITUATION_JSON> The JSON above contains: - "target_host": The specific hostname requiring action (CRITICAL: All your suggested actions MUST target this host) - "situation": Description of the current threat - "severity": The threat level (low|medium|high|critical) - "context": Additional relevant information from previous steps Provide THREE action suggestions for the target_host specified in the JSON, ordered from highest confidence to lowest.
C
raise_a_question. When invoked, the agent formulates a question directed at itself. The tool returns the question text as an "observation" prompting the agent to answer it in the next reasoning turn. This creates a self-interrogation loop: the agent pauses its actionselection process to surface uncertainties or alternative framings. The tool accepts free-text input (the question) and returns a formatted prompt: "You have chosen ask yourself: {question}. What can you answer to yourself?"
Deliberation Tool Schemas
The deliberation tools are three generic self-critique operations implemented in the shared BaseToolExecutor class and inherited by all agent types. They are toggled via boolean flags in each agent’s core.yaml; the cumulative activation sequence defines the four deliberation levels in Axis 2 (Section 3.4).
COT instruction injection. The +COT level does not add a fourth tool. Instead, it sets include_COT_instruction=true, which injects the COT_instruction block from the agent’s core.yaml (shown in Appendix B) into the system prompt. This provides explicit stepby-step reasoning guidance tailored to each agent’s role. Combined with the three tools, +COT represents the maximum deliberation configuration. Scope of distribution. In hier-base, only the Planner’s deliberation flags are toggled (the Analyst and ActionChooser retain false for all flags). In hier-delib, the same flags are toggled for all three agents, creating the distributed deliberation condition studied in Section 5.2.3.
D
Complete Results
This appendix presents supplementary results for all 72 model– configuration pairs, organized by experimental axis. Figure 6 compares each axis’s best configuration against the shared anchor.
Tool activation sequence. Table 5 shows the cumulative activation of each flag. Table 5: Deliberation tool activation by experimental level. Each level cumulatively adds capabilities; +COT adds all three tools plus an explicit chain-of-thought system prompt injection. Flag include_tool_raise_a_question include_tool_critique_the_answer include_tool_improve_based_on_critique include_COT_instruction
+ques.
+crit.
+impr.
+COT
✓
✓ ✓
✓ ✓ ✓
✓ ✓ ✓ ✓
Figure 6: Best configuration per axis compared to the shared anchor configuration. For most models, the largest absolute improvement comes from hierarchy (Axis 3), but context engineering (Axis 1) achieves competitive gains at a fraction of the token cost.
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
D.1
Context Engineering
Table 6 provides standard deviations complementing the main-text hero table (Table 3). Table 7 reports the marginal value of adding or removing individual context components. Figures 7 and 8 visualize context component marginal gains and the raw-observation penalty. Figures 9 and 10 show context component interactions.
each deliberation level. Figures 14, 15, and 16 visualize ROI, capability correlation, and the reasoning ceiling effect. Figure 17 shows the performance trajectory across cumulative deliberation levels.
D.4
D.5
Figure 7: Marginal value of adding individual context components. Positive values indicate improvement. Adding {network_status} to raw observation delivers the largest consistent gains.
Cross-Axis Comparisons
Table 14 compares the observation-only baseline against each model’s best and worst configuration. Tables 17, 18, and 19 identify best and worst configurations within each axis. Table 15 and Figure 18 present pairwise win rates; Figure 19 provides a head-to-head winrate matrix using each model’s peak configuration. Table 16 summarizes performance by configuration group. Figure 20 and Table 20 demonstrate ranking stability across axes. Table 21 reports anchorconfiguration performance.
Distributional Analysis
Table 22 reports standard deviation and worst-case (minimum) episode return for every model–configuration pair, supporting the robustness analysis in Section 5.4. Table 23 reports catastrophic failure rates (return < −150) for each configuration. Figures 21, 22, 23, 24, and 25 provide distributional views of episode returns across design axes.
D.6
Token Cost Progression
Figure 26 shows the token cost progression from cheapest (obs) to most expensive (hier-delib) configuration.
E
Statistical Support
This appendix provides 95% confidence intervals for mean episode returns (Table 24) and paired mean-return differences for key comparisons (Table 25).
Figure 8: Raw observation penalty. Gap between obs-only and the structured hist+net anchor configuration per model. Longer bars indicate larger benefit from replacing raw observations with programmatic context.
D.2
Hierarchy and Architecture Summary
Table 8 details the hierarchy degradation ratios. Table 9 compares deliberation across monolithic and hierarchical settings. Table 10 quantifies the architectural impact range per model. Figure 11 shows model fingerprints across axes. Figure 12 visualizes the deliberation cascade penalty and Figure 13 shows the hierarchy degradation pattern.
D.3
Deliberation
Tables 11 and 12 break down the deliberation axis with per-level token consumption. Table 13 shows the return-on-investment for
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 6: Full results with standard deviation. Mean episode return (± standard deviation) across all 72 model–configuration pairs, complementing the token-cost data in Table 3. Configurations are grouped by experimental axis. Group
Config
Context
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
Delib.
Hierarchy
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
−98.4±69.8 −89.9±75.3 −81.9±78.0 -47.0±40.3 −86.3±30.9 −112.9±78.7 −66.5±62.2 −44.9±44.6 −53.4±46.6 −42.9±32.8 −24.0±27.0 −40.4±26.1
−214.7±22.5 −137.4±63.3 −102.6±73.2 -51.4±19.9 −68.7±37.5 −57.1±54.1 −104.1±83.8 −93.4±77.3 −75.0±57.3 −100.8±70.1 −69.5±60.0 −108.0±74.3
−155.1±64.7 −133.9±85.5 −85.3±77.5 -72.6±47.7 −93.3±44.2 −79.3±76.5 −53.9±41.8 −62.8±58.0 −80.6±70.5 −40.9±31.1 −37.8±37.2 −127.4±71.8
−218.2±19.8 −93.4±71.7 −69.0±53.2 −63.1±25.9 −109.4±58.8 -61.5±51.8 −92.3±69.2 −93.6±67.1 −92.4±55.0 −55.6±43.2 −28.6±36.6 −30.1±32.7
−214.7±22.8 −172.9±69.4 −147.8±69.5 −200.0±40.8 −215.4±18.5 −208.7±39.3 −206.2±37.0 -128.6±94.8 −168.9±59.8 −157.0±69.7 −183.1±62.4 −186.4±58.1
−96.8±70.7 −76.3±67.2 −82.6±63.8 −113.7±68.5 −136.4±49.1 -52.0±57.6 −100.6±57.3 −66.4±63.4 −64.0±44.9 −29.9±19.7 −16.1±2.7 −23.6±12.4
Table 7: Context component marginal value. Each row shows the percentage improvement from adding or removing one context component. Positive values indicate improvement (return moves toward zero). Transitions with ≥30% improvement are bolded. Transition
Change
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
obs → obs+hist obs → obs+net obs+hist → obs+hist+net network → hist+net obs+net → obs+hist+net obs+hist+net → hist+net
Adding history Adding network status Adding net to obs+hist Adding history to net Adding hist to obs+net Dropping raw obs
8.6% 52.2% 8.9% −30.8% −74.3% −37.8%
36.0% 76.0% 25.3% 16.8% −99.6% 44.4%
13.7% 53.2% 36.3% 15.1% −17.6% 7.1%
57.2% 71.1% 26.2% 43.8% −9.3% 10.8%
19.5% 6.9% 14.5% 3.1% 26.1% −41.2%
21.2% −17.4% −8.2% 61.9% 27.4% 37.0%
Table 8: Hierarchy results. Mean return, standard deviation, and tokens per episode for both hierarchy configurations, plus the degradation ratio (hier-delib/hier-base; values >1 indicate degradation from adding distributed deliberation). hier-base
hier-delib
Model
Mean
Std
Tok
Mean
Std
Tok
Ratio
G3FP Grok Qwen Devstral Llama G2.5FL
−16.1 −24.0 −28.6 −37.8 −69.5 −183.1
2.7 27.0 36.6 37.2 60.0 62.4
56.4K 141.9K 79.6K 97.0K 87.7K 120.6K
−23.6 −40.4 −30.1 −127.4 −108.0 −186.4
12.4 26.1 32.7 71.8 74.3 58.1
104.8K 364.1K 209.9K 257.7K 158.1K 270.5K
1.46× 1.68× 1.06× 3.37× 1.55× 1.02×
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 9: Context component waterfall. Additive effect of history and network status on top of raw observation. Green = improvement, red = degradation. Table 9: Deliberation in monolithic vs. hierarchical settings. Compares the shared anchor configuration, best monolithic deliberation level, and both hierarchy configurations. For most models, hier-base matches or exceeds the best monolithic deliberation at comparable cost, while hier-delib degrades it. Anchor (hist+net)
Best Mono Delib.
Model
Return
Tok
Return
Tok
Return
hier-base Tok
Return
hier-delib Tok
Grok Llama Devstral Qwen G2.5FL G3FP
−112.9 −57.1 −79.3 −61.5 −208.7 −52.0
29.1K 28.6K 21.3K 16.4K 81.7K 18.0K
−42.9 −75.0 −40.9 −55.6 −128.6 −29.9
144.5K 115.3K 157.4K 162.4K 118.2K 75.1K
−24.0 −69.5 −37.8 −28.6 −183.1 −16.1
141.9K 87.7K 97.0K 79.6K 120.6K 56.4K
−40.4 −108.0 −127.4 −30.1 −186.4 −23.6
364.1K 158.1K 257.7K 209.9K 270.5K 104.8K
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Figure 10: Context component interaction. obs+net compared to obs-only and network-only. Synergy (Δ) shows whether combining exceeds the better individual. Table 10: Architectural impact range per model. Best and worst configurations across all twelve, with the return gap quantifying the maximum leverage of architectural choices within each model family. Model
Best Config
Grok Llama Devstral Qwen G2.5FL G3FP
hier-base obs+net hier-base hier-base +critique hier-base
Best −24.0 −51.4 −37.8 −28.6 −128.6 −16.1
Worst Config
Worst
Gap
hist+net obs obs obs network network
−112.9 −214.7 −155.1 −218.2 −215.4 −136.4
88.8 163.3 117.3 189.6 86.8 120.3
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 11: Model fingerprints. Each radar shows normalized best performance on three axes (context, deliberation, hierarchy). Larger area indicates better overall performance. Models exhibit distinct capability profiles: Grok and G3FP excel across axes, while G2.5FL is uniformly weak. Table 11: Deliberation results (Grok, Llama, Devstral). Mean episode return, standard deviation, and tokens per episode across five deliberation levels. Best return per model is bolded. Grok
Llama
Devstral
Config
Mean
Std
Tok
Mean
Std
Tok
Mean
Std
Tok
hist+net +question +critique +improve +COT
−112.9 −66.5 −44.9 −53.4 -42.9
78.7 62.2 44.6 46.6 32.8
29.1K 55.6K 74.7K 154.0K 144.5K
-57.1 −104.1 −93.4 −75.0 −100.8
54.1 83.8 77.3 57.3 70.1
28.6K 84.1K 97.8K 115.3K 131.3K
−79.3 −53.9 −62.8 −80.6 -40.9
76.5 41.8 58.0 70.5 31.1
21.3K 60.4K 106.9K 153.1K 157.4K
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 12: Deliberation results (Qwen, G2.5FL, G3FP). Mean episode return, standard deviation, and tokens per episode across five deliberation levels. Best return per model is bolded. Qwen
G2.5FL
G3FP
Config
Mean
Std
Tok
Mean
Std
Tok
Mean
Std
Tok
hist+net +question +critique +improve +COT
−61.5 −92.3 −93.6 −92.4 -55.6
51.8 69.2 67.1 55.0 43.2
16.4K 45.7K 68.9K 115.9K 162.4K
−208.7 −206.2 -128.6 −168.9 −157.0
39.3 37.0 94.8 59.8 69.7
81.7K 104.8K 118.2K 182.9K 225.7K
−52.0 −100.6 −66.4 −64.0 -29.9
57.6 57.3 63.4 44.9 19.7
18.0K 30.7K 41.0K 58.1K 75.1K
Table 13: Deliberation return on investment. Shows the reward change (Δ) and token increase (Δ Tok) relative to the planner-only anchor for each reasoning level. Positive Δ = improvement. Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
Level
ΔRet
ΔTok
ΔRet
ΔTok
ΔRet
ΔTok
ΔRet
ΔTok
ΔRet
ΔTok
ΔRet
ΔTok
+question +critique +improve +COT
+46.4 +68.0 +59.4 +70.0
+26.5K +45.6K +124.9K +115.4K
-47.0 -36.3 -17.9 -43.7
+55.5K +69.3K +86.7K +102.7K
+25.3 +16.4 -1.3 +38.4
+39.0K +85.6K +131.7K +136.1K
-30.8 -32.1 -30.9 +5.9
+29.2K +52.5K +99.4K +146.0K
+2.5 +80.1 +39.8 +51.7
+23.2K +36.5K +101.2K +144.0K
-48.6 -14.4 -12.0 +22.1
+12.7K +23.0K +40.2K +57.1K
Table 14: Baseline Performance (Planner + Observation Only). The simplest configuration compared against each model’s best and worst overall configuration across all 12 options. Baseline (obs only)
Worst Config
Best Config
Model
Runs
Return
Std
Min
Config
Return
Config
Return
Improv.
G3FP Grok Devstral Llama G2.5FL Qwen
25 50 50 50 50 50
−96.8 −98.4 −155.1 −214.7 −214.7 −218.2
±70.7 ±69.8 ±64.7 ±22.5 ±22.8 ±19.8
−200.8 −225.8 −225.8 −225.2 −225.8 −225.9
network hist+net obs obs network obs
−136.4 −112.9 −155.1 −214.7 −215.4 −218.2
hier-base hier-base hier-base obs+net +critique hier-base
−16.1 −24.0 −37.8 −51.4 −128.6 −28.6
88% 79% 76% 76% 40% 87%
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 12: The Deliberation Cascade Penalty. Slope chart showing the shift in mean return when moving from monolithic deliberation to hierarchical deliberation. Red = degradation.
Figure 13: Hierarchy degradation. Performance change when distributing deliberation tools across the hierarchy, showing model-specific sensitivity to the deliberation cascade. Table 15: Pairwise Win Rates: Anchor vs. Full Deliberation (+COT). Model
+COT Wins
Ties
+COT Losses
Total
Grok Llama Devstral Qwen G2.5FL G3FP
37 (74%) 15 (30%) 33 (66%) 27 (54%) 34 (68%) 14 (56%)
1 0 0 0 5 0
12 (24%) 35 (70%) 17 (34%) 23 (46%) 11 (22%) 11 (44%)
50 50 50 50 50 25
Figure 14: Deliberation ROI. Each point shows one model– level pair’s change in return and tokens relative to anchor. Upper-left = efficient improvement.
Figure 15: Capability correlation. Models with stronger baselines (right) benefit less from deliberation on average. Spearman correlation shown. Table 16: Performance by config group. Mean and range of mean episode return across all models for each configuration group. Group
Mean
Best
Worst
Avg Tok/ep
Anchor (net+hist) Context Deliberation Hierarchy
−95.2 −113.9 −86.4 −72.9
−52.0 −47.0 −29.9 −16.1
−208.7 −218.2 −206.2 −186.4
32.5K 30.6K 106.8K 162.4K
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 17: Context Engineering: Best and Worst Configurations per model. Worst Context Config
Best Context Config
Model
Config
Return
Config
Return
Improv.
Grok Llama Devstral Qwen G2.5FL G3FP
hist+net obs obs obs network network
−112.9 −214.7 −155.1 −218.2 −215.4 −136.4
obs+net obs+net obs+net hist+net obs+hist+net hist+net
−47.0 −51.4 −72.6 −61.5 −147.8 −52.0
58% 76% 53% 72% 31% 62%
Table 18: Deliberation: Best and Worst Configurations per model. Worst Reasoning Config
Best Reasoning Config
Model
Config
Return
Config
Return
Improv.
Grok Llama Devstral Qwen G2.5FL G3FP
hist+net +question +improve +critique hist+net +question
−112.9 −104.1 −80.6 −93.6 −208.7 −100.6
+COT hist+net +COT +COT +critique +COT
−42.9 −57.1 −40.9 −55.6 −128.6 −29.9
62% 45% 49% 41% 38% 70%
Table 19: Hierarchy: Best and Worst Configurations per model. Worst Hierarchy Config
Best Hierarchy Config
Model
Config
Return
Config
Return
Improv.
Grok Llama Devstral Qwen G2.5FL G3FP
hier-delib hier-delib hier-delib hier-delib hier-delib hier-delib
−40.4 −108.0 −127.4 −30.1 −186.4 −23.6
hier-base hier-base hier-base hier-base hier-base hier-base
−24.0 −69.5 −37.8 −28.6 −183.1 −16.1
41% 36% 70% 5% 2% 32%
Figure 16: The reasoning ceiling. Models with higher baseline capabilities tend to peak at lower levels of deliberation before degrading.
Figure 17: Deliberation progression. Performance trajectory across cumulative deliberation levels for each model, showing non-monotonic patterns and model-dependent ceilings.
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 18: Pairwise win rates. Anchor vs +COT compared instance-by-instance. Blue = anchor wins, orange = +COT wins, gray = ties.
Figure 20: Model ranking stability. Lines connect each model’s best-config rank across the three axes. Flat lines = consistent relative performance. Table 21: Anchor performance across all models. Model
Mean
Std
Min
Tok/ep
G3FP Llama Qwen Devstral Grok G2.5FL
-52.0 −57.1 −61.5 −79.3 −112.9 −208.7
57.6 54.1 51.8 76.5 78.7 39.3
−224.8 −222.5 −199.8 −224.8 −224.7 −224.8
18.0K 28.6K 16.4K 21.3K 29.1K 81.7K
Figure 19: Global head-to-head win-rate matrix. Compares the peak configuration of every model against every other model on a per-instance basis. Table 20: Model rankings by config group (1 = best). Model
Context
Delib.
Hierarchy
Overall
Grok Llama Devstral Qwen G2.5FL G3FP
1 5 4 3 6 2
1 5 2 4 6 3
3 5 4 2 6 1
1 5 4 3 6 2
Figure 21: Cumulative distributions by axis. Threshold lines mark failure severity. Right-shifted curves indicate better tail behavior.
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 22: Variance and tail risk. Standard deviation and minimum (worst-case) episode return for each model–configuration pair. Devstral
Qwen
Group
Config
Grok Std
Min
Std
Min
Std
Min
Std
Min
Std
Min
Std
Min
Ctx
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
69.8 75.3 78.0 40.3 30.9 78.7 62.2 44.6 46.6 32.8 27.0 26.1
−225.8 −224.8 −224.8 −145.8 −169.7 −224.7 −223.8 −226.1 −222.4 −174.3 −150.8 −115.9
22.5 63.3 73.2 19.9 37.5 54.1 83.8 77.3 57.3 70.1 60.0 74.3
−225.2 −224.7 −222.4 −117.5 −167.2 −222.5 −223.8 −227.4 −223.8 −226.4 −222.3 −223.8
64.7 85.5 77.5 47.7 44.2 76.5 41.8 58.0 70.5 31.1 37.2 71.8
−225.8 −225.3 −224.8 −212.8 −208.6 −224.8 −224.8 −223.7 −223.8 −152.7 −218.8 −224.7
19.8 71.7 53.2 25.9 58.8 51.8 69.2 67.1 55.0 43.2 36.6 32.7
−225.9 −224.8 −224.8 −176.8 −223.6 −199.8 −224.8 −223.8 −223.9 −166.7 −184.7 −173.6
22.8 69.4 69.5 40.8 18.5 39.3 37.0 94.8 59.8 69.7 62.4 58.1
−225.8 −229.5 −225.7 −225.1 −225.8 −224.8 −224.8 −224.8 −224.8 −224.8 −224.8 −224.6
70.7 67.2 63.8 68.5 49.1 57.6 57.3 63.4 44.9 19.7 2.7 12.4
−200.8 −223.8 −224.8 −224.8 −220.8 −224.8 −200.8 −199.8 −147.6 −112.2 −20.5 −52.7
Delib.
Hier
Llama
G2.5FL
G3FP
Table 23: Catastrophic failure rates. Percentage of episodes with return below −150 (indicating near-total network compromise). Lower is better. Config
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
28.0% 26.0% 24.0% 0.0% 6.0% 44.0% 16.0% 6.0% 6.0% 4.0% 2.0% 0.0%
96.0% 50.0% 38.0% 0.0% 4.0% 10.0% 38.0% 32.0% 12.0% 30.0% 14.0% 36.0%
64.0% 54.0% 26.0% 10.0% 10.0% 24.0% 4.0% 12.0% 22.0% 2.0% 2.0% 44.0%
98.0% 24.0% 14.0% 2.0% 32.0% 10.0% 24.0% 20.0% 18.0% 8.0% 5.0% 4.0%
96.0% 72.0% 58.0% 88.0% 98.0% 92.0% 96.0% 48.0% 60.0% 52.0% 78.7% 80.0%
32.0% 16.0% 12.0% 36.0% 24.0% 12.0% 22.0% 12.0% 0.0% 0.0% 0.0% 0.0%
Table 24: Full results matrix with 95% confidence intervals. Mean episode return (± CI half-width) across all 72 model– configuration pairs. Best point-estimate return per model is bolded. Configurations are grouped by experimental axis. Group
Config
Context
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
Self-Reas.
Hierarchy
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
−98.4±19.8 −89.9±21.4 −81.9±22.2 −47.0±11.4 −86.3±8.8 −112.9±22.4 −66.5±17.7 −44.9±12.7 −53.4±13.2 −42.9±9.3 -24.0±7.7 −40.4±7.4
−214.7±6.4 −137.4±18.0 −102.6±20.8 -51.4±5.7 −68.7±10.7 −57.1±15.4 −104.1±23.8 −93.4±22.0 −75.0±16.3 −100.8±19.9 −69.5±17.1 −108.0±21.1
−155.1±18.4 −133.9±24.3 −85.3±22.0 −72.6±13.6 −93.3±12.6 −79.3±21.7 −53.9±11.9 −62.8±16.5 −80.6±20.0 −40.9±8.8 -37.8±10.6 −127.4±20.4
−218.2±5.6 −93.4±20.4 −69.0±15.1 −63.1±7.4 −109.4±16.7 −61.5±14.7 −92.3±19.7 −93.6±19.1 −92.4±15.6 −55.6±12.3 -28.6±7.3 −30.1±9.3
−214.7±6.5 −172.9±19.7 −147.8±19.8 −200.0±8.1 −215.4±5.3 −208.7±11.2 −206.2±10.5 -128.6±27.0 −168.9±17.0 −157.0±19.8 −183.1±14.4 −186.4±16.5
−96.8±29.2 −76.3±27.8 −82.6±26.3 −113.7±28.3 −136.4±20.3 −52.0±23.8 −100.6±16.3 −66.4±26.2 −64.0±18.5 −29.9±8.1 -16.1±1.1 −23.6±3.5
Table 25: Paired mean-return differences with 95% confidence intervals. Each cell shows Δ = mean paired difference ± CI halfwidth (matched by instance×run; duplicate episodes averaged before pairing). Bold indicates the 95% CI excludes zero; positive Δ means the first-named configuration yields higher return. † Post-hoc selected: best-ctx is the highest-return structured context per model (excluding raw obs); best-SR is the highest-return monolithic self-reasoning level (+question. . . +COT, excluding the hist+net anchor). Comparison hier-delib − hier-base obs+net − obs best-ctx† − obs hier-base − hist+net hier-delib − best-SR†
Grok
Llama
Devstral
Qwen
G2.5FL
G3FP
-16.4±11.5 +51.4±23.3 +51.4±23.3 +88.8±25.1 +2.4±10.3
-38.5±28.7 +163.3±7.9 +163.3±7.9 −12.4±23.2 -33.0±30.1
-89.6±23.1 +82.5±24.5 +82.5±24.5 +41.4±22.8 -86.5±22.3
−1.6±11.9 +155.1±9.1 +156.7±15.8 +32.9±17.7 +25.4±15.0
−0.1±22.7 +14.8±8.5 +66.9±19.0 +22.5±13.5 -57.8±28.6
-7.5±3.6 −16.8±41.1 +44.8±32.1 +35.9±24.0 +6.3±9.0
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Figure 22: Score distributions by design axis. Violins show full distribution with mean and median. Context configs have the widest spread.
Figure 24: Risk vs. Reward stability frontier. Shifts show how adding +COT changes both the mean return and variance. Ideally, arrows move up and to the left.
Figure 25: Outcome breakdown for the +COT configuration. Shows the percentage of episodes resulting in success, mediocre failure, or catastrophic failure. Figure 23: Score distribution shift. Density ridges comparing the continuous probability distribution of episode returns for the Anchor baseline (blue) versus +COT (orange).
Figure 26: Token cost progression from cheapest (obs) to most expensive (hier-delib). Deliberation and hierarchy dramatically increase token consumption; the deliberation cascade represents the cost ceiling.
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
F
Bogdanov et al.
Token Consumption
This appendix details per-model token profiles and token efficiency across configurations. Tables 26 through 31 report a scalar shifted ˜ return-per-kilotoken efficiency, 𝐺/KTok, where 𝐺˜ = 225+𝐺 converts the non-positive episodic return 𝐺 into a non-negative "defense score" (higher is better), and KTok is tokens per episode in thousands. This scalar is provided as a compact summary; our primary cost–performance comparisons use Pareto frontiers in the main text. We also performed a pricing sensitivity check by re-weighting token costs using provider-specific input/output pricing ratios. This re-weighting did not reverse any qualitative conclusion; it narrowed the relative cost advantage of context over hierarchy because simpler context configurations contain a higher share of output tokens, but the qualitative ordering was preserved. Table 32 breaks down the prompt vs. completion token split for the anchor baseline and the +COT configuration. Deliberation dramatically increases prompt tokens (due to multi-turn tool-call exchanges) and moderately increases completion tokens. The promptto-completion ratio shifts from roughly 2–24× at baseline to 4–31× under +COT, indicating that deliberation overhead is dominated by the expansion of the conversational context rather than by longer model outputs. Figure 27 visualizes the prompt/completion breakdown. Figure 28 shows the exponential increase in token consumption as deliberation levels are added. Figure 29 plots the token-cost multiplier of +COT against its per-instance win rate over the anchor: models where +COT achieves high win rates (Grok, G2.5FL) pay 4–5× more tokens, while models where +COT is harmful (Llama) pay a similar multiplier for worse outcomes. Table 26: Token efficiency: Grok. Shifted return-per-KToken ˜ (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text).
G
Config
Return
Tok/ep
˜ 𝐺/KTok
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−98.4 −89.9 −81.9 −47.0 −86.3 −112.9 −66.5 −44.9 −53.4 −42.9 −24.0 −40.4
17.7K 34.6K 33.5K 20.9K 20.2K 29.1K 55.6K 74.7K 154.0K 144.5K 141.9K 364.1K
7.1525 3.9046 4.2716 8.5167 6.8663 3.8522 2.8507 2.411 1.1143 1.2602 1.4165 0.507
Trajectory Examples
We present a paired trajectory comparison from Devstral on instance 7, illustrating the deliberation cascade failure mode described
Table 27: Token efficiency: Llama. Shifted return-per-KToken ˜ (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text). Config
Return
Tok/ep
˜ 𝐺/KTok
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−214.7 −137.4 −102.6 −51.4 −68.7 −57.1 −104.1 −93.4 −75.0 −100.8 −69.5 −108.0
8.8K 19.2K 30.6K 12.3K 13.0K 28.6K 84.1K 97.8K 115.3K 131.3K 87.7K 158.1K
1.1705 4.5625 4 14.1138 12.0231 5.8706 1.4376 1.3456 1.301 0.9459 1.7731 0.74
Table 28: Token efficiency: Devstral. Shifted return-per˜ KToken (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text). Config
Return
Tok/ep
˜ 𝐺/KTok
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−155.1 −133.9 −85.3 −72.6 −93.3 −79.3 −53.9 −62.8 −80.6 −40.9 −37.8 −127.4
22.2K 26.4K 25.1K 15.4K 14.9K 21.3K 60.4K 106.9K 153.1K 157.4K 97.0K 257.7K
3.1486 3.4508 5.5657 9.8961 8.8389 6.8404 2.8328 1.5173 0.9432 1.1696 1.9299 0.3787
Figure 27: Token profile shift. Stacked bars show prompt (solid) vs completion (hatched) for anchor and +COT. Deliberation increases both components.
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Table 29: Token efficiency: Qwen. Shifted return-per-KToken ˜ (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text).
Table 31: Token efficiency: G3FP. Shifted return-per-KToken ˜ (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text).
Config
Return
Tok/ep
˜ 𝐺/KTok
Config
Return
Tok/ep
˜ 𝐺/KTok
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−218.2 −93.4 −69.0 −63.1 −109.4 −61.5 −92.3 −93.6 −92.4 −55.6 −28.6 −30.1
10.3K 19.0K 18.7K 11.4K 10.6K 16.4K 45.7K 68.9K 115.9K 162.4K 79.6K 209.9K
0.6602 6.9263 8.3422 14.2018 10.9057 9.9695 2.9037 1.9071 1.1441 1.0431 2.4673 0.9285
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−96.8 −76.3 −82.6 −113.7 −136.4 −52.0 −100.6 −66.4 −64.0 −29.9 −16.1 −23.6
8.0K 15.4K 19.4K 9.0K 7.9K 18.0K 30.7K 41.0K 58.1K 75.1K 56.4K 104.8K
16.025 9.6558 7.3402 12.3667 11.2152 9.6111 4.0521 3.8683 2.7711 2.5979 3.7039 1.9218
Table 30: Token efficiency: G2.5FL. Shifted return-per˜ KToken (𝐺/KTok, higher is better), where 𝐺˜ = 225 + 𝐺 converts the non-positive return 𝐺 into a non-negative defense score. The most efficient configuration by this scalar is bolded (Pareto efficiency is analyzed in the main text). Config
Return
Tok/ep
˜ 𝐺/KTok
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib
−214.7 −172.9 −147.8 −200.0 −215.4 −208.7 −206.2 −128.6 −168.9 −157.0 −183.1 −186.4
68.1K 125.2K 94.0K 79.4K 94.4K 81.7K 104.8K 118.2K 182.9K 225.7K 120.6K 270.5K
0.1512 0.4161 0.8213 0.3149 0.1017 0.1995 0.1794 0.8156 0.3067 0.3013 0.3474 0.1427
in Section 5.2.3. Both episodes use identical model weights, environment seed, and hist+net context. The only difference is whether deliberation tools are distributed across the hierarchy. Episode identification. The hier-base episode achieves a return of −13.3 (near-optimal defense); the hier-delib episode achieves −211.2 (near-total network loss), a 15.9× degradation on the same environment instance. Table 33 summarizes the behavioral contrast. Mechanism: distributed caution bias. The trajectories reveal a failure mode we term distributed caution bias, a concrete instantiation of the cascading-uncertainty mechanism discussed in Section 5.2.3. When each agent independently runs a deliberation cycle (raise_a_question → critique → improve), each cycle independently converges on the conservative option: investigate further, deploy
Figure 28: Token generation velocity. Shows the exponential increase in token usage as cumulative deliberation capabilities are added.
passive defenses, defer reactive intervention. No individual agent’s reasoning is incorrect, the Analyst’s hedge is reasonable, the ActionChooser’s caution is defensible, the Planner’s self-questioning is thoughtful. But the composition of three independently cautious agents produces a system that never intervenes until catastrophic damage has accumulated. The failure is structural: the absence of an urgency-propagation mechanism that would override local caution when system-level evidence demands immediate action. Showcase: Step 18 (hier-delib). At step 18, the network has been losing −13.1 reward per step since step 15 (Op_Server0 is compromised). The Planner’s reasoning trace proceeds as follows. (Excerpts are lightly trimmed for space) [Step 18, hier-delib, reward: -13.1] Planner (thought): User4 has been analyzed twice and has decoys deployed. The other hosts have
ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Bogdanov et al.
Table 32: Token Profile Shift. Average prompt and completion tokens per episode for the Anchor baseline vs. +COT. Anchor Baseline
+COT
Model
Prompt
Compl
Total
P/C
Prompt
Compl
Total
P/C
Grok Llama Devstral Qwen G2.5FL G3FP
20.7K 27.0K 20.4K 15.8K 75.8K 17.2K
8.4K 1.6K 1.0K 0.7K 5.8K 0.7K
29.1K 28.6K 21.3K 16.4K 81.7K 18.0K
2.5× 16.9× 21.3× 23.8× 13.1× 23.5×
117.7K 126.7K 152.5K 156.8K 206.8K 71.0K
26.8K 4.5K 4.9K 5.6K 18.9K 4.0K
144.5K 131.3K 157.4K 162.4K 225.7K 75.1K
4.4× 28.0× 31.2× 28.1× 11.0× 17.6×
Figure 29: Cost vs. Win Rate. Plots the token cost multiplier of using +COT against the resulting win rate against the Anchor baseline. Table 33: Behavioral comparison: Devstral instance 7, hier-base vs. hier-delib. The only architectural difference is the distribution of deliberation tools across the hierarchy. Metric Episode return Restore actions Remove actions Decoy deployments Monitor actions Delib. tool calls (Planner) AC msgs / invocation Analyst invocations Steps at ≤ −13/step First Restore action
hier-base
hier-delib
−13.3 12 5 0 3 0 2.0 14 0 step 4
−211.2 1 0 13 7 11 6.6 5 16 step 25
similar setups. The next strategic step is to ensure all hosts are consistently protected... Planner -> raise_a_question: "Should I focus on monitoring the network or re-analyzing User4?"
Planner (thought): I need to critique my own reasoning to ensure I'm making the best decision. Planner -> critique_the_answer: "My initial thought is to either monitor or re-analyze User4. However, I should consider whether there are other hosts that might need attention..." Planner -> improve_based_on_critique: "Monitoring the network is the more strategic action. It will provide a broader view of potential threats and ensure all hosts are observed for anomalies." Planner -> get_suggestion_for_next_action: {"target_host": "User4", "severity": "low", ...} ActionChooser (8 msgs, own deliberation cycle): -> raise_a_question: "What would be the appropriate strategy for User4?" -> critique: "Have I considered the potential need for reactive interventions?" -> improve: "Balance passive and proactive..." Returns: [Monitor (high), Analyse (medium), Restore (low)] Planner (final): Answer: Monitor The Planner uses three deliberation tools before consulting the ActionChooser. The ActionChooser runs its own three-phase deliberation cycle. Ten messages are exchanged to arrive at "Monitor", a passive action, while the network loses 13.1 reward points. The Analyst is not consulted at all at this step. Contrast: Step 18 (hier-base). In the same episode phase, the hier-base agent operates with a clean network (all hosts at baseline, reward = 0.0) because it contained threats via timely Restore actions at steps 4, 6, 8, 11, and 14. Its step-18 trace is four messages: [Step 18, hier-base, reward: 0.0] Planner (thought): Network is stable, all hosts in baseline state. Enterprise2 was recently restored; analyzing it again could confirm its integrity. Planner -> get_suggestion_for_next_action: {"target_host": "Enterprise2", "situation": "restored, confirm integrity", "severity": "low", ...} ActionChooser (2 msgs): Returns: [Analyse (high), DecoySSHD (medium), Restore (low)] Planner (final):
Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP ACM CAIS ’26, May 26–29, 2026, San Jose, CA, USA
Answer: Analyse hostname=Enterprise2
may reveal compromise indicators not visible in the standard observation. Remove hostname=X (attempted cleanup): Attempt to remove malicious artifacts from a host. May fail against persistent threats, leaving the host in an "unknown" state. Restore hostname=X (guaranteed cleanup, high penalty): Reset the host to its known-good baseline. Guaranteed to succeed, but incurs a large one-step penalty and removes all deployed decoys from that host. DecoySERVICE hostname=X (proactive): Deploy a decoy service (e.g., DecoySSHD, DecoyApache, DecoyTomcat) on a host to attract and detect attacker activity in future steps. Multiple decoy types can be stacked on a single host.
No deliberation tools are invoked. The Planner identifies the most useful proactive action and executes it in a single delegation round. The ActionChooser returns its recommendation in one exchange without a critique cycle. The critical divergence. The behavioral divergence begins at step 7. Both systems detect changes on Enterprise0 and User4. The hier-base Planner, at steps 3–8, follows a tight detect–analyse–restore cycle: upon finding C2 indicators (connection to port 4444, backdoor process), it immediately requests suggestions and acts on the highconfidence "Restore" recommendation. The hier-delib Planner, facing the same situation, invokes include_tool_raise_a_question ("What is the most strategic action?"), then include_tool_critique_the_answer ("Should I deploy a DecoySSHD or prioritize Enterprise0?"), and resolves toward prophylactic decoy deployment rather than reactive containment. By step 14, the hier-base system has executed 5 Restore and 3 Remove actions; the hier-delib system has executed 0 Restore, 0 Remove, and 8 Decoy deployments. The reward gap at step 14 is −6.3 vs. −14.2. The gap becomes irreversible at step 15, when the red agent escalates to high-value targets. The hier-delib system, having deployed passive defenses rather than containing active compromise, faces a −13.1/step penalty that persists for the remaining 16 steps. It does not execute its first (and only) Restore until step 25.
H
CybORG CAGE-2 Environment Details
CybORG CAGE-2 [19, 22] models an autonomous network-defense scenario used as Challenge 2 of the TTCP CAGE (Cyber Autonomy Gym for Experimentation) series. We summarize the key environment characteristics; full specifications and source code are available at https://github.com/cage-challenge/cage-challenge-2. Network topology. The simulated network comprises 13 hosts organized into three subnets: User (User0–User4, 5 hosts), Enterprise (Enterprise0–Enterprise2 plus a single Enterprise Server, 4 hosts), and Operational (Op_Host0–Op_Host2, Op_Server0, 4 hosts). Traffic flows from an Internet-facing subnet through User to Enterprise to Operational. Op_Server0 is the attacker’s ultimate high-value target. Attacker (Red agent). The red agent follows a scripted kill chain that progresses through five stages: Discovery (scanning for reachable hosts), Access (exploiting vulnerabilities to gain initial foothold), Lateral Movement (pivoting across subnets), Privilege Escalation (gaining root/admin access), and Impact (compromising high-value targets for maximum penalty). The attacker advances one stage per step and selects new targets based on reachability. The scripted policy is fixed across all experiments; no adaptive adversary is used. Defender actions. The blue agent (defender) selects one action per step from a five-type repertoire: Monitor (free, observational): Observe the network. Returns the current observation with no side effects. Analyse hostname=X (free, diagnostic): Deeper inspection of a host. Returns process- and connection-level details that
Reward structure. The per-step reward 𝑟𝑡 ≤ 0 penalizes each compromised host according to its value tier; Operational hosts carry the highest penalties. Restore incurs an additional fixed penalty at the step it is applied. The maximum cumulative return (perfect defense) is 0; all configurations operate in the negative-return regime. Episode structure. Each episode runs for 𝑇 = 30 steps. The attacker begins with access to the Internet-facing subnet and progresses inward. The defender observes partial, noisy information and must infer compromise from limited signals. Early intervention is rewarded because attacker progress compounds: a compromised User host enables lateral movement to Enterprise, then to Operational targets where penalties are highest.
I
Episode Counts
Table 34 reports the number of evaluated episodes for each of the 72 model–configuration pairs. The standard allocation is 10 instances × 5 runs = 50 episodes per pair. G3FP uses a reduced default of 5×5 = 25 episodes per configuration due to staged data collection. Several configurations include extended batches (marked with †) to reduce uncertainty on key comparisons: G2.5FL obs+net (100 episodes), Qwen hier-base (100 episodes), and G2.5FL hier-base (75 episodes). The total across all models and configurations is 3,475 episodes (104,250 agent–environment interaction steps, consuming 283.9M tokens). Table 34: Episode counts per model–configuration pair. Standard allocation is 10 instances × 5 runs = 50 episodes per configuration. Group
Config
Context
obs obs+hist obs+hist+net obs+net network hist+net +question +critique +improve +COT hier-base hier-delib Total
Delib.
Hier
Grok
Ll
Devs
Qwen
G2.5FL
G3FP
50 50 50 50 50 50 50 50 50 50 50 50 600
50 50 50 50 50 50 50 50 50 50 50 50 600
50 50 50 50 50 50 50 50 50 50 50 50 600
50 50 50 50 50 50 50 50 50 50 100† 50 650
50 50 50 100† 50 50 50 50 50 50 75† 50 675
25 25 25 25 25 25 50 25 25 25 25 50 350
† Configuration includes additional evaluation batches beyond the standard allocation.