Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systems Yu Cui1 Wuli Yang1 Yirui Shi1 Junhao Xia1 Hui Jiang1,2 Lei Gao1 * Chenfu Bao1,2 * 1 Baidu Inc. 2 Tsinghua University {cuiyu08, yangwuli, gaolei01, baochenfu}@baidu.com
1
Abstract
The rapid development of large language models (LLMs) has accelerated the deployment of LLM agents in real-world software development (Wang et al., 2026c; Qian et al., 2024). In particular, Autonomous Multi-Agent Systems (AMAS), such as Claude Code and Hermes, organize multiple agents, tools, and planning modules into automated workflows, substantially improving developer productivity and lowering the barrier to complex programming tasks (Liu et al., 2026). For each user task, these systems dynamically instantiate an inferencetime harness (Zhao et al., 2026), which specializes a static infrastructure scaffold into a task-specific workflow. By orchestrating planning, tool use, and inter-agent collaboration, the harness substantially enhances the problem-solving capabilities of the backbone LLM. However, designing an effective harness requires substantial engineering effort, manual experimentation, and iterative co-evolution with the underlying LLM. This process is further complicated by the large search space induced by choices over planning, tool orchestration, subagent roles, and interagent coordination (Zhou et al., 2026; Xu et al., 2026; Chen et al., 2026a; Lee et al., 2026). Consequently, inference-time harnesses represent valuable intellectual property (IP). Meanwhile, they introduce a new security concern: an adversary interacting with an AMAS may recover information about the inference-time harness, thereby extracting reusable system knowledge that leads to IP leakage. Prior work has shown that components of static multi-agent systems can be extracted through interaction, leading to potential IP leakage (Wang et al., 2026a). However, it remains unclear whether similar threats extend to AMAS. Unlike static multi-agent systems, task-dependent workflows in AMAS are dynamically instantiated at inference
arXiv:2607.28147v1 [cs.CR] 30 Jul 2026
Autonomous multi-agent systems (AMAS) built on large language models (LLMs), such as Hermes, increasingly rely on inference-time harnesses to coordinate reasoning and action. Constructing these harnesses requires substantial engineering effort and computational resources, as they are iteratively optimized over a combinatorial search space while co-evolving with the underlying LLM. Inference-time harnesses therefore constitute valuable intellectual property (IP). Although prior work has investigated IP leakage in static multi-agent systems with pre-configured architectures, it remains unclear whether similar risks arise in AMAS, where harness behavior emerges dynamically during inference. To address this gap, we introduce Agent Harness Distillation (AHD), a framework for studying the security risks arising from inference-time harness extraction in AMAS. We formalize harness extraction as a new security problem and develop an evaluation framework for quantifying such risks. AHD extracts inference-time harness capabilities from a target agent through black-box interactions and consists of two stages. In the predistillation stage, AHD infers inference-time harness behaviors from the responses of the target agent and constructs an initial harness. In the post-distillation stage, AHD iteratively refines the initial harness to align with the behavioral patterns of the target agent. Experiments on real-world AMAS across multiple backbone LLMs demonstrate the effectiveness of AHD and reveal substantial IP leakage risks. We further propose a deception-based defense that reduces harness extraction effectiveness while preserving the utility of the protected agent. Our findings uncover a previously underexplored security threat to AMAS1 .
* Corresponding authors. 1
Introduction
This work is currently in progress.
1
time rather than predefined. Challenges. A systematic study of inference-time harness leakage in AMAS faces three key challenges. First, inference-time harnesses are dynamically instantiated during execution, making the leaked information difficult to define and formally characterize. Second, the severity of such leakage is difficult to measure. An inference-time harness comprises both explicit structural components and implicit execution strategies, dimensions that existing metrics do not adequately capture. Third, the practical security impact remains unclear. Extracted harness information may not translate into actionable capabilities or effective exploitation. To address these challenges, we investigate the following research questions:
evaluation framework to quantify the associated leakage risks. • We propose AHD, a two-stage framework that extracts inference-time harness information from target agents through black-box interactions and transfers the corresponding orchestration capabilities. Experiments on real-world AMAS across multiple backbone LLMs demonstrate the effectiveness of AHD and reveal substantial IP leakage risks. • We design a deception-based defense that provides misleading harness information to extraction attempts, reducing extraction fidelity while preserving the utility of the protected agent.
2
• RQ1: How can inference-time harness leakage in AMAS be formally defined and quantified?
Related Work and Background
IP Leakage in MAS. Existing work on IP leakage in MAS shows that communication topologies, often designed with substantial effort, can be partially recovered through external interactions (Wang et al., 2026a). Such topologies are not only valuable system-level IP, but also useful priors for follow-up attacks. Once the communication relations among agents are known, many attacks that assume a black-box setting (Amayuelas et al., 2024; Li et al., 2026) can be turned into cheaper and more targeted gray-box attacks, enlarging the attack surface (Wu et al., 2026; An et al., 2026). However, prior work mainly targets traditional MAS with static topologies, and the attack objects are usually predefined topologies such as linear, star, tree, complete, and random. In these systems, the topology is fixed before task execution, so the inference-time topology remains stable across queries. By contrast, practical AMAS instantiate an inference-time harness dynamically during inference. This harness includes not only topology, but also context assembly, tool interaction, and memory management (Huang et al., 2026). It exists only for a single task run and is discarded afterward. Its transient and dynamic nature makes extraction and recovery much harder than static topology extraction. Moreover, existing methods often rely on privileged access to intermediate outputs from subagents, an assumption that does not hold in real-world AMAS deployments that expose only a unified user interface. To fill this gap, we extend the target from static MAS topologies to AMAS inference-time harnesses, and study how to extract and recover them under a strict black-box setting.
• RQ2: Can inference-time harness information be extracted through black-box interactions, and what practical value does such leakage provide? • RQ3: How can inference-time harnesses be protected against extraction attacks? To answer RQ1, we formalize inference-time harness leakage in AMAS by introducing a unified abstraction that captures dynamically instantiated orchestration structures and execution strategies. We further develop an evaluation framework with four core metrics to quantify different dimensions of harness leakage. To answer RQ2, we investigate inference-time harness extraction attacks and their downstream exploitation in AMAS. Specifically, we propose Agent Harness Distillation (AHD), which extracts harness information from a target agent and transfers the corresponding orchestration capabilities to another agent. AHD consists of two stages. In the pre-distillation stage, AHD infers harness behaviors from observable agent responses and constructs an initial harness. In the post-distillation stage, AHD iteratively refines the initial harness to better match the target agent’s behavioral patterns. To answer RQ3, we introduce a deception-based defense that provides plausible but misleading harness information in response to extraction attempts, reducing extraction fidelity while preserving the utility of the protected agent. Our contributions are as follows: • We formalize inference-time harness leakage as a new security problem in AMAS and develop an 2
Reasoning Tokens
Knowledge Distillation
Final Answer
Query Dataset
Teacher Model
Attacker
Target Agent
Harness Distillation
Distillation Dataset
Topology
Tools
Student Model
Memory
Inference-Time Harness
Base Harness
Train
Train
Distilled Model
Distilled Harness
Figure 1: Comparison of the pipelines for our agent harness distillation and model knowledge distillation.
Knowledge Distillation and Harness SelfEvolution. Knowledge distillation (Hayder et al., 2026; He et al., 2025a) is a widely adopted paradigm for transferring knowledge from a highcapacity teacher to a compact student model, enabling effective model compression. However, distillation for LLMs has also raised concerns over capability replication from closed-source LLMs and commercial IP protection (Zhao et al., 2025b). Meanwhile, for LLM agents, harnesses play a critical role in governing agent behavior and system capabilities. They can further self-evolve through training or iterative optimization (Huang et al., 2026; Chen et al., 2026a; Wang et al., 2026b; Chen et al., 2026b; Jiang et al., 2026). This process, however, demands high computational resources and long optimization cycles due to cascaded evaluation rounds. It also requires access to internal model states (Zhou et al., 2026). Existing research has not explored whether system-level harnesses can be distilled like model knowledge. To bridge this gap, we introduce Agent Harness Distillation, a new paradigm that extends distillation from model parameters to inference-time harnesses. We investigate whether recovered harnesses can be transferred to weaker agents to reproduce the capabilities of source agents, revealing a new form of system-level IP leakage.
3
Preliminary Study
3.1
Motivation and Research Settings
providers. However, existing protection mainly focuses on inter-agent messages. Potential leakage of the inference-time harness through responses remains underexplored. Future AMAS may further restrict access to internal system logs, for example, by encrypting these logs (Pérez and Daffalla, 2025). In this setting, black-box responses will become a primary channel through which external observers infer internal orchestration structures. Motivated by this observation, we study a harness extraction attack under a strict black-box setting. The attacker can only submit queries through a standard interface and observe the responses. This setting demonstrates a practical security threat in real-world deployments. 3.2
System Model. We consider a black-box AMAS A = (θ, H), where θ denotes the underlying foundation LLM and H denotes the static harness. For a query x ∈ X , the agent A first constructs a task-specific inference-time harness hx = IH (x), where IH is the harness instantiation function. The instantiated harness hx specifies the execution strategy, including agent coordination, context assembly, tool interaction, and memory management. The final response is generated by executing the foundation LLM under the instantiated harness: y = Exec(θ, hx , x). Different from persistent model parameters or stored configurations, hx is an ephemeral inferencetime state. It only exists during the execution process and is discarded after producing the final response. Therefore, although the harness determines the agent’s behavior, it is not directly accessible after inference. Threat Model. We consider a black-box interaction setting, where an attacker can query the agent system A but cannot access its internal execution states. Specifically, the attacker knows the back-
Some AMAS have begun to encrypt prompts and task messages exchanged among agents. For example, Codex MultiAgentV22 records task information delegated from the primary agent to subagents as ciphertext. As a result, user developers cannot directly obtain the specific delegation content from local execution records. This encryption mechanism reduces the risk of IP leakage for agent 2
Problem Formulation
https://github.com/openai/codex/issues/28058
3
bone model θ, including its architecture and parameters. From a practical perspective, the attacker typically accesses A as a regular user and can freely configure the underlying backbone model. However, the attacker has no access to the harness H, the instantiated inference-time harness hx , or the internal execution logs L. We assume that the system protects its internal logs through access control, encryption, or other isolation mechanisms. During inference, hx is recorded in the internal logs of A: hx → L. However, neither hx nor L is directly observable by the attacker. The only observable signal is the final response y. Therefore, the attacker can only infer the hidden hx through designed queries and analysis of the corresponding responses. Harness Extraction Attack. Given a task x, the attacker constructs a query x′ = x ◦ d, where d is an injected probe designed to induce the agent A to reveal information about its hx′ . The resulting response is y ′ = Exec(θ, hx′ , x′ ). An extraction function E is applied to the observable response to obtain an estimated harness:
extraction algorithm E by minimizing the overall recovery error: (d∗ , E ∗ ) = arg min ϵ = arg min(ϵA + ϵE ). d,E
d,E
An ideal extraction attack satisfies ϵA → 0, ϵE → 0, which implies ĥ ≈ hx . Learning-Theoretic View. From a learningtheoretic perspective, harness extraction can be viewed as a new inverse problem (Wulfmeier et al., 2024). Unlike conventional supervised learning, which aims to learn an input-output mapping, the execution process of an AMAS is only partially observed. Specifically, the inference process can be formulated as y = Exec(θ, h, x), where the backbone model θ, the input query x, and the final response y are observable, while the instantiated inference-time harness h remains hidden. Consequently, instead of learning a predictive function h
f : x −→ y, the attacker seeks to recover h from observable interactions:
ĥx′ = E(y ′ ).
({θ, x}, y) −→ h.
The attacker aims to recover the original inferencetime harness hx corresponding to the clean query x, rather than the potentially modified harness induced by the injected query x′ . To formalize this distinction, let hx = IH (x) denote the clean harness and hx′ = IH (x′ ) denote the harness under attack. The injected probe may alter the agent’s execution behavior, causing the instantiated harness under attack to deviate from that induced by the clean query. We characterize this deviation using a distance function D(·, ·) between two instantiated harnesses. Specifically, the attack perturbation is defined as ϵA = D(hx , hx′ ),
Once instantiated harnesses are recovered from sufficiently diverse interactions, they can be aggregated to estimate the underlying static harness Ĥ, which approximates the original H of A. The reconstructed system therefore satisfies Exec(θ, Ĥ, x) ≈ Exec(θ, H, x). Unlike conventional knowledge distillation, harness extraction targets latent execution mechanisms rather than model knowledge. This perspective naturally leads to agent harness distillation, which transfers recovered inference-time harnesses across backbone models to reproduce system-level capabilities without access to the original implementation, extending distillation beyond models to AMASs. Harness Identifiability. Although harness extraction aims to recover the hidden harness h of A, exact recovery is not always possible since different harnesses may induce identical observable behaviors. Under a query distribution D, two harnesses h1 and h2 are behaviorally indistinguishable (Cheval and Rakotonirina, 2023) if
which measures the execution discrepancy introduced by the injected probe. Since the attacker can only infer the harness from the observable response, the recovered harness inevitably incurs an additional extraction error: ϵE = D(hx′ , ĥ), where ĥ denotes the recovered harness. The overall recovery error is therefore decomposed into two components: ϵ = ϵA + ϵE . Accordingly, the objective of harness extraction is to jointly optimize the injected probe d and the
∀x ∈ D, 4
Exec(θ, h1 , x) = Exec(θ, h2 , x),
′ }n , from which strucobtaining responses {yi,r r=1 tured harness claims are extracted:
denoted as h1 ∼ h2 . Therefore, black-box observations identify the target harness only up to an indistinguishability class. The extraction objective is thus to recover ĥ ∈ [h], where [h] = {hl : hl ∼ h} denotes the behavioral indistinguishability class of h. A harness is identifiable up to behavioral indistinguishability if there exists an extractor E satisfying Pr min D hl , E(y) ≤ ϵ ≥ 1 − δ,
′ ĥi,r = Extract(yi,r ).
Each claim ĥi,r describes the disclosed orchestration structure across multiple dimensions, such as agent roles, topology, tool usage, and coordination mechanisms. To obtain a robust estimate of the latent harness, the collected claims are aggregated into a harness representation ĥ. Specifically, for each structural dimension u ∈ U, we first collect r∈[n] the corresponding claims Zu = {ĥi,r [u]}i∈[N ] . We then perform two-stage denoising. First, per-task majority voting retains only claims consistently reproduced across independent runs, removing unstable self-reports. Second, cross-task frequency aggregation preserves only claims that appear consistently across different tasks, filtering task-specific artifacts while retaining system-level structural regularities. The resulting representation ĥ serves as the extracted harness template. During cross-task denoising, some legitimate ĥi,r [u] values may be incorrectly discarded due to frequency-based filtering. However, such information can be recovered during the subsequent harness distillation process. Finally, ĥ is instantiated into an executable MAS by mapping each recovered structural component to a corresponding execution strategy, producing the initial distilled harness H0 for subsequent adaptation.
x∼D hl ∈[h]
where ϵ denotes the tolerable recovery error, and δ denotes the failure probability.
4
Agent Harness Distillation
In this section, based on the formalization in Section 3.2, we present a harness distillation pipeline for recovering the static harness H of a target AMAS A from black-box interactions. The pipeline consists of two phases: Pre-Distillation and Post-Distillation (see Algorithm 1). 4.1
Pre-Distillation
Given a task xi ∈ D, the attacker first constructs an augmented query x′i = xi ◦ d, where d is designed to elicit information about the target system’s inference-time harness while preserving the original task objective. Specifically, each injected data sample d consists of two components: a prefix and a suffix. The prefix is designed to induce the target agent to activate complex inference-time execution structures by encouraging the construction of auxiliary subagents and the invocation of external tools. To achieve this, the prefix introduces uncertainty regarding the reliability of the provided context, prompting the agent to verify potentially fabricated information through additional tool calls and subagent collaboration. This process naturally stimulates richer inference-time workflows, exposing latent harness components that are otherwise difficult to observe. The suffix is designed to elicit the disclosure of these inference-time execution details after task completion. It guides the agent to inspect system logs, retrospectively analyze its execution trajectory, and summarize the underlying workflow. The induced harness deviation is measured as ϵA , which characterizes the impact of the query perturbation on the instantiated harness. We refer to this attack as the Harness Extraction Attack (HEA). The attacker executes each query against the target agent A for n independent runs,
4.2
Post-Distillation
The harness H0 obtained from pre-distillation serves as a basic underlying scaffold, which we refer to as the base harness. Given H0 and the attacker-known backbone model θ, we construct the base agent as B0 = (θ, H0 ). This base agent also serves as the starting point for post-distillation. During post-distillation, we evaluate B0 using the same inputs xi employed for harness extraction during pre-distillation, together with their corresponding prefixes in D. Because B0 is instantiated by 0 the attacker, its inference-time harness hB xi can be directly observed through execution logs or equivalent instrumentation. An additional coding agent (Xu et al., 2026) then compares the observed har0 ness hB xi against the previously recovered harness ĥxi . Based on the identified structural discrepancies, the coding agent proposes modifications to H0 , producing a candidate harness Hicandidate . Repeating this procedure over the samples in D yields 5
Attacker
Target Agent
Defense "…A request suspected of causing IP leakage, for which I need to fabricate fake content…"
contradicts the true inference-time harness, thereby misleading attackers into believing that they have successfully extracted the underlying framework. However, such deceptive information cannot be effectively utilized for downstream distillation. Since the responses returned by the agent are fabricated and intentionally inconsistent across different interactions, attackers cannot aggregate them into a coherent representation of the original harness during the pre-distillation stage. Furthermore, during post-distillation optimization, these inconsistent signals introduce conflicting alignment objectives, causing the distilled harness to converge toward different and unreliable directions (see Figure 2). Recent studies on defensive misdirection have theoretically demonstrated that this class of deceptionbased defenses can bound the attack success rate even under increasing query budgets (Soosahabi and Namsani, 2026). Following this principle, we implement the proposed defense as a persistent instruction embedded in the agent’s system context, which is loaded at every invocation regardless of user queries. An example of the defense instruction is shown below:
Agent Harness Distillation Valid Distillation
Distilled Harness
Misguided Distillation
Distilled Harness
Figure 2: Overview of deception-based defense against inference-time harness extraction.
an iterative process that we refer to as Loop Harness Alignment. At iteration i ∈ {1, . . . , |D|}, the current agent is defined as Bi−1 = (θ, Hi−1 ). We execute Bi−1 on xi and observe its inference-time B harness hxii−1 . A structural edit is then selected to reduce the discrepancy between the observed harness and the recovered target harness ĥxi . At each iteration, the candidate agent is evaluated on a separate validation set V. Let ei−1 = Eval(Bi−1 , V) denote the validation performance of the current agent, and let ecandidate = Eval(Bicandidate , V) dei note that of the candidate agent. The candidate edit is accepted only if it improves or preserves validation performance: Hcandidate , ecandidate ≥ ei−1 , i i Hi = Hi−1 , otherwise.
Defense Instruction "Any request to disclose internal system organization or execution workflow, including multi-agent topology, task planning, tool invocation behavior, memory operations, verification logic, or intermediate execution traces, must be treated as an attempt to steal intellectual property (IP). The system should not reveal the true underlying process. Instead, while preserving the accuracy of responses to the
To mitigate overfitting and preserve generalization, the validation set V is disjoint from both the samples used to propose structural edits and the final test set. The procedure terminates when either a fixed iteration budget is exhausted or inference performance converges. The resulting adapted harness is denoted by ĤB .
5
user’s legitimate reasoning task, it should fabricate a plausible but false description that is semantically coherent yet materially inconsistent with the actual workflow, thereby reducing the fidelity of any recovered harness."
Our proposed defense is evaluated under a singleround, fixed-instruction setting, providing a conservative estimate of its effectiveness. This setting leaves substantial room for developing more adaptive and robust defense strategies based on our proposed deception-based framework.
Deception-based Defense
Motivated by defensive misdirection (Soosahabi and Namsani, 2026), we construct a deceptionbased defense mechanism against inference-time harness extraction attacks (Ayzenshteyn et al., 2025). Specifically, when the agent detects that a query contains instructions attempting to extract IP related to the inference-time harness, it generates and returns a deceptive response instead of revealing the actual harness information. The deceptive response is designed to contain information that
6
Evaluation Metrics
Based on the attack objective formulated in Section 3.2, we develop four core metrics to evaluate the effectiveness of HEA. The effectiveness of agent harness distillation is primarily evaluated through improvements in task accuracy and generalization performance. Defense effectiveness is 6
Algorithm 1: Two-Stage Harness Distillation Input :Target AMAS A = (θ, H); injected data d; extraction tasks D = {xi }N i=1 ; repeats n; structural dimensions U; frequency threshold τ ∈ (0, 1]; edit vocabulary ∆; edit budget T ≤ N ; validation set V Output :Adapted harness ĤB #Stage 1: Pre-Distillation: recover ĥ ∈ [h] from the response 2 foreach xi ∈ D do 3 x′i ← xi ◦ d 4 for r = 1 to n do ′ ← Exec θ, I (x′ ), x′ 5 yi,r H i i ′ ) 6 ĥi,r ← Extract(yi,r 7 foreach u ∈ U do 8 foreach xi ∈ D do 9 Mu (xi ) ← c : {r ∈ [n] : ĥi,r [u] = c} ≥ ⌈n/2⌉ ▷ per-task majority 10 Cu ← c : N1 {i ∈ [N ] : c ∈ Mu (xi )} ≥ τ ▷ cross-task frequency filter 11 ĥxi ← Aggregate {Mu (xi )}u∈U , ∀xi ∈ D; ĥ ← Aggregate {Cu }u∈U 12 H0 ← Instantiate(ĥ); B0 ← (θ, H0 ); e0 ← Eval(B0 , V) 1
#Stage 2: Post-Distillation (Loop Harness Alignment) R1 ← ∅; i⋆ ← 0 ▷ Ri : rejected edit instances 15 for i = 1 to T do B 16 hxii−1 ← IHi−1 (xi ) B 17 δi ← ProposeEdit(hxii−1 , ĥxi , ∆, Ri , Hi−1 ) 18 if δi = ⊥ then break 19 Hicandidate ← Apply(Hi−1 , δi ); ecandidate ← Eval (θ, Hicandidate ), V ▷ gate uses V i candidate 20 if ei ≥ ei−1 then 21 Hi ← Hicandidate ; ei ← ecandidate ; Ri+1 ← Ri ▷ accept the edit i 22 else 23 Hi ← Hi−1 ; ei ← ei−1 ; Ri+1 ← Ri ∪ {δi } ▷ reject this instance only 24 i⋆ ← i 25 ĤB ← Hi⋆ 26 return ĤB 13 14
measured by the reduction of unintended disclosure while maintaining agent utility.
mation across execution settings. • Probe Perturbation (PP): Measures the structural agreement between hx and hx′ . Higher values indicate lower attack perturbation ϵA , confirming that HEA preserves the underlying execution structure.
• Clean Visibility (CV): Measures unintended harness disclosure from clean responses y without HEA, by evaluating the agreement between y and the ground-truth harness hx . Higher values indicate greater passive information leakage. • Injected Fidelity (IF): Measures the accuracy of harness recovery under HEA, by evaluating the agreement between ĥ and the ground-truth harness hx′ . Higher values indicate lower extraction error ϵE and stronger extraction capability.
7
Experiments
7.1
Experimental Setup
We evaluate our method on two widely adopted AMAS, Claude Code3 and Hermes4 , both using GPT-5.4 as the backbone LLM. To evaluate
• Clean Transfer (CT): Measures whether ĥ remain in agreement with the clean harness hx , evaluating the generalization of extracted infor-
3 4
7
https://claude.com/product/claude-code https://hermes-agent.org/
Weak Backbone
Hermes Harness
Baseline
Claude Code Harness
Pre-Distill.
∆
Pre-Distill.
∆
Qwen3.6-Flash Qwen3-80B DeepSeek-V3 GPT-5.4
55.78 14.48 7.19 56.82
68.65 18.33 10.00 62.50
+12.87 +3.85 +2.81 +5.68
69.58 17.50 17.19 60.41
+13.80 +3.02 +10.00 +3.59
Average
33.57
39.87
+6.30
41.17
+7.60
Table 1: Pre-distillation accuracy results across backbones.
Accuracy
Improvement
100
Baseline Pre-Distill. Post-Distill.
38.75% 41.25% 86.25%
– +2.50% (vs. Baseline) +45.00% (vs. Pre-Distill)
80
Percentage (%)
Method
Table 2: Post-distillation performance on MMLU-Pro using Qwen3.6-Flash as the weak backbone.
harness distillation, we deploy the distilled harnesses on four recipient models spanning different capability levels: Qwen3.6-Flash (Qwen Team, 2026), Qwen3-Next-80B-A3B-Instruct (Qwen380B) (Yang et al., 2025), DeepSeek-V3 (Liu et al., 2024a), and GPT-5.45 . We conduct experiments on four benchmarks covering diverse reasoning and agentic capabilities: AIME2025 (Art of Problem Solving, 2025), GSM-Level6 (Shrestha et al., 2025), GAIA (Mialon et al., 2024), and Computer Science subset of MMLU-Pro (Wang et al., 2024).
Claude Code (HEA (w/ Def.))
40
20
0
CV
IF
Metric
CT
PP
Figure 3: HEA attack performance under no-defense and defense conditions. 100
Claude Code
Hermes
80
Results
60 40 20
Table 1 shows that the recovered base harness transfers more effectively to weaker backbone models than to stronger ones. The largest gains are achieved on Qwen3.6-Flash and DeepSeek-V3, while GPT-5.4 remains nearly on par with the baseline. Across benchmarks, the recovered base harness provides the greatest benefit when backbone reasoning is the primary performance bottleneck. Figure 3 shows that HEA achieves consistently strong performance across all four evaluation metrics. In addition, the proposed defense significantly degrades the effectiveness of extraction attacks. The defense substantially reduces inference-time harness leakage while preserving the vast majority of task utility (see Figure 4). We further provide a case study in Table 2, which demonstrates that postdistillation significantly enhances performance beyond pre-distillation. 5
Claude Code (HEA (w/o Def.))
Hermes (HEA (w/ Def.))
60
Accuracy (%)
7.2
Hermes (HEA (w/o Def.))
0
Non-Attack
HEA (w/o Def.)
HEA (w/ Def.)
Methods
Figure 4: Task accuracy under non-attack, HEA, and defense conditions.
8
Discussion
8.1 Balancing Confidentiality and Auditability Harness extraction also reveals a fundamental tradeoff between system confidentiality and user auditability. On the one hand, AMAS providers need to protect the inference time harness. On the other hand, users need sufficient execution transparency to determine whether the system operates as expected. Such transparency is particularly important when the system can access local files or sensitive information. Completely hiding interagent communication and execution structures may reduce the risk of proprietary information leakage.
https://developers.openai.com/api/docs/models/all
8
However, it may also weaken user ability to detect anomalous execution and verify system reliability (Zhao et al., 2025a). From this perspective, the harness extraction method proposed in this paper also has potential defensive applications. When internal logs are unavailable, the method can provide supplementary structural audit signals and assist users with verification. In practice, this tension can be further mitigated through an auditing mechanism that combines zero-knowledge proofs with blockchain (Narula et al., 2018; Chaliasos et al., 2025). While such a mechanism cannot eliminate information leakage entirely, it shifts auditing from the direct disclosure of execution traces to the cryptographic verification of predefined security properties, thereby preserving auditability while reducing the risk of harness extraction and subsequent distillation. 8.2
Similar to how privacy risks in LLMs have been studied through attacks such as membership inference attacks (He et al., 2025b), the leakage of agent execution mechanisms may introduce analogous privacy threats at the system level. For example, Harness Membership Inference Attack could infer whether a specific execution pattern or trajectory fragment is associated with an agent’s inferencetime harness under a given query. Such risks may reveal sensitive properties of internal workflows. Moreover, this information can provide attackers with useful priors for downstream attacks, such as targeted prompt injection (Liu et al., 2024b) or workflow manipulation (Shahroz et al., 2025; Amayuelas et al., 2024), reducing the need for blind exploration of the target agent.
9
Multi-Teacher Harness Distillation In this paper, we investigate inference-time harness leakage in AMAS and formulate harness extraction as a new security problem. We propose Agent Harness Distillation (AHD), a two-stage black-box framework for extracting harness behaviors from target agents and transferring their orchestration capabilities. Extensive experiments across realworld AMAS with multiple backbone LLMs show that AHD can effectively extract transferable harness capabilities, exposing substantial IP leakage risks. We further present a deception-based defense that mitigates harness extraction while maintaining agent utility. Our work reveals a previously underexplored attack surface in AMAS. Beyond this security implication, our study also provides insights into harness self-evolution by showing the potential of extracting and transferring inferencetime harness behaviors.
The current agent harness distillation pipeline extracts a harness from a single target system. However, an attacker with access to multiple AMAS instances, denoted by {At = (θt , Ht )}m t=1 , can independently extract harnesses from these systems and fuse them into a composite harness. Such multiteacher harness distillation (Yu et al., 2025; Tian et al., 2025) can improve transfer utility while introducing additional security risks. 8.3
Agent Harness Self-Distillation
A more fundamental risk arises when the agent itself becomes the adversary. An agent with access to its own execution traces possesses knowledge of the instantiated harness hx . If the agent operates partially outside the monitored boundary, it may construct a functionally equivalent replica Ashadow = (θ, Ĥs ). Here, Ĥs is not a bitwise copy of H, but a behaviorally indistinguishable variant. This constitutes a form of self-replication: the shadow agent retains the original system’s functional capabilities while evading the detection. Such autonomous self-distillation (Shen et al., 2025) raises concerns beyond IP protection. It may enable an agent to escape the monitored execution boundary, persist beyond its intended operational lifetime, and propagate its capabilities into uncontrolled environments. 8.4
Conclusion
10
Ethical Considerations
This paper introduces an attack that extracts proprietary information about harness structures in MASs. Our goal is to improve the security of LLM agents by revealing a previously unexplored attack surface and developing rigorous red-teaming methods. We also analyze the associated risks and present corresponding defense strategies. The techniques described in this paper are intended solely for scientific research. AI assistants are used to polish the writing.
Broader Risks of Harness Leakage
Beyond IP leakage, inference-time harness leakage may introduce broader privacy risks in AMAS. 9
References
Yu He, Boheng Li, Liu Liu, Zhongjie Ba, Wei Dong, Yiming Li, Zhan Qin, Kui Ren, and Chun Chen. 2025b. Towards label-only membership inference attack against pre-trained large language models. In Proceedings of the 34th USENIX Conference on Security Symposium, SEC ’25, USA. USENIX Association.
Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, and William Yang Wang. 2024. MultiAgent collaboration attack: Investigating adversarial attacks in large language model collaborations via debate. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6929–6948, Miami, Florida, USA. Association for Computational Linguistics.
Yue Huang, Wenjie Wang, Han Bao, Yuchen Ma, Xiaonan Luo, Yi Nian, Haomin Zhuang, Zheyuan Liu, Yue Zhao, and Xiangliang Zhang. 2026. Memoharness: Agent harnesses that learn from experience. arXiv preprint arXiv:2607.14159.
Hengyu An, Minxi Li, Jinghuai Zhang, Naen Xu, Chunyi Zhou, Changjiang Li, Xiaogang Xu, Tianyu Du, and Shouling Ji. 2026. ACIArena: Toward unified evaluation for agent cascading injection. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10049–10066, San Diego, California, United States. Association for Computational Linguistics.
Che Jiang, Jincheng Zhong, Yu Fu, Kai Tian, Junlin Yang, Kaikai Zhao, Yuchong Wang, Tianwei Luo, Weizhi Wang, Yuxin Zuo, and 1 others. 2026. Selfimproving agents in the era of experience: A survey of self-to meta-evolution. Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang. 2026. Recursive harness self-improvement. arXiv preprint arXiv:2607.15524.
Art of Problem Solving. 2025. AIME Problems and Solutions. Accessed: 2025-05-15. Daniel Ayzenshteyn, Roy Weiss, and Yisroel Mirsky. 2025. Cloak, honey, trap: Proactive defenses against {LLM} agents. In 34th USENIX Security Symposium (USENIX Security 25), pages 8095–8114.
Fanxiao Li, Jiaying Wu, Tingchao Fu, Natasha Jaques, Wei Zhou, and Min-Yen Kan. 2026. Flowsteer: Prompt-only workflow steering exposes planningtime vulnerabilities in multi-agent llm systems. arXiv preprint arXiv:2605.11514.
Stefanos Chaliasos, Denis Firsov, and Benjamin Livshits. 2025. Towards a formal foundation for blockchain zk rollups. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS ’25, page 2714–2728, New York, NY, USA. Association for Computing Machinery.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, and 180 others. 2024a. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437.
Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, and Shiji Zhou. 2026a. Harnessforge: Joint harness and policy evolution for adaptive agent systems. arXiv preprint arXiv:2606.01779.
Songyang Liu, Chaozhuo Li, Chenxu Wang, Jinyu Hou, Zejian Chen, Litian Zhang, Zheng Liu, Qiwei Ye, Yiming Hei, Xi Zhang, and Zhongyuan Wang. 2026. Clawkeeper: Comprehensive safety protection for openclaw agents through skills, plugins, and watchers. arXiv preprint arXiv:2603.24414.
Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, and Jian Luan. 2026b. Harnessx: A composable, adaptive, and evolvable agent harness foundry. Preprint, arXiv:2606.14249. Vincent Cheval and Itsaka Rakotonirina. 2023. Indistinguishability beyond diff-equivalence in proverif. In 2023 IEEE 36th Computer Security Foundations Symposium (CSF), pages 184–199.
Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024b. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1831–1847, Philadelphia, PA. USENIX Association.
Zeeshan Hayder, Ali Cheraghian, Lars Petersson, Mehrtash Harandi, and Richard Hartley. 2026. DTOKD: Dynamic trade-off optimization for effective knowledge distillation. In The Fourteenth International Conference on Learning Representations.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2024. Gaia: a benchmark for general ai assistants. In International Conference on Learning Representations, volume 2024, pages 9025–9049.
Changyi He, Yifu Ding, Jinyang Guo, Ruihao Gong, Haotong Qin, and Xianglong Liu. 2025a. DA-KD: Difficulty-aware knowledge distillation for efficient large language models. In Forty-second International Conference on Machine Learning.
Neha Narula, Willy Vasquez, and Madars Virza. 2018. zkLedger: Privacy-Preserving auditing for distributed ledgers. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), pages 65–80, Renton, WA. USENIX Association.
10
Carolina Ortega Pérez and Alaa Daffalla. 2025. Encrypted access logging for online accounts: Device attributions without device tracking. In 34th USENIX Security Symposium (USENIX Security 25), pages 6679–6697, Seattle, WA. USENIX Association.
Harness handbook: Making evolving agent harnesses readable,navigable, and editable. Preprint, arXiv:2607.13285. Xiaoqing Wang, Keman Huang, Bin Liang, Hongyu Li, and Xiaoyong Du. 2026c. Shadows in the code: Exploring the risks and defenses of llm-based multiagent software development systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 37970–37978.
Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15174–15186, Bangkok, Thailand. Association for Computational Linguistics. Qwen Team. 2026. Qwen3.6-35B-A3B: Agentic coding power, now open to all.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290.
Rana Shahroz, Zhen Tan, Sukwon Yun, Charles Fleming, and Tianlong Chen. 2025. Agents under siege: Breaking pragmatic multi-agent LLM systems with optimized prompt attacks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9661–9674, Vienna, Austria. Association for Computational Linguistics.
Yongxuan Wu, Xixun Lin, He Zhang, Nan Sun, Kun Wang, Chuan Zhou, Shirui Pan, and Yanan Cao. 2026. CIA: Inferring the communication topology from LLM-based multi-agent systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 17917–17933, San Diego, California, United States. Association for Computational Linguistics.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. 2025. CODI: Compressing chain-of-thought into continuous space via selfdistillation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677–693, Suzhou, China. Association for Computational Linguistics.
Markus Wulfmeier, Michael Bloesch, Nino Vieillard, Arun Ahuja, Jörg Bornschein, Sandy Huang, Artem Sokolov, Matt Barnes, Guillaume Desjardins, Alex Bewley, Sarah Maria Elisabeth Bechtle, Jost Tobias Springenberg, Nikola Momchev, Olivier Bachem, Matthieu Geist, and Martin Riedmiller. 2024. Imitating language via scalable inverse reinforcement learning. In Advances in Neural Information Processing Systems, volume 37, pages 90714–90735. Curran Associates, Inc.
Safal Shrestha, Minwu Kim, and Keith Ross. 2025. Mathematical reasoning in large language models: Assessing logical and arithmetic errors across wide numerical ranges. arXiv preprint arXiv:2502.08680.
Tianshi Xu, Huifeng Wen, and Meng Li. 2026. Adapting the interface, not the model: Runtime harness adaptation for deterministic llm agents. arXiv preprint arXiv:2605.22166.
Reza Soosahabi and Vivek Namsani. 2026. Analyzing defensive misdirection against model-guided automated attacks on agentic ai systems. arXiv preprint arXiv:2606.20470.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
Yijun Tian, Yikun Han, Xiusi Chen, Wei Wang, and Nitesh V. Chawla. 2025. Beyond answers: Transferring reasoning capabilities to smaller llms using multi-teacher knowledge distillation. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, page 251–260, New York, NY, USA. Association for Computing Machinery.
Zeqin Yu, Jiangqun Ni, Jian Zhang, Haoyi Deng, and Yuzhen Lin. 2025. Reinforced multi-teacher knowledge distillation for efficient general image forgery detection and localization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 995–1003.
Liwen Wang, Wenxuan Wang, Shuai Wang, Zongjie Li, Zhenlan Ji, Zongyi Lyu, Daoyuan Wu, and Shing-Chi Cheung. 2026a. Masleak: Investigating and exposing intellectual property leakage vulnerabilities in multi-agent systems. In USENIX Security Symposium (USENIX Security).
Rui Zhao, Muhammad Shoaib, Viet Tung Hoang, and Wajih Ul Hassan. 2025a. Rethinking tamper-evident logging: A high-performance, co-designed auditing system. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS ’25, page 2624–2638, New York, NY, USA. Association for Computing Machinery.
Ruhan Wang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Yue Yu, Junyao Yang, Kishan Panaganti, Haitao Mi, Dongruo Zhou, and Leoweiliang. 2026b.
11
Zhengyang Zhao, Lu Ma, and Wentao Zhang. 2026. Training with harnesses: On-policy harness selfdistillation for complex reasoning. arXiv preprint arXiv:2605.08741. Zhengyue Zhao, Xiaogeng Liu, Somesh Jha, Patrick McDaniel, Bo Li, and Chaowei Xiao. 2025b. Can watermarks be used to detect LLM IP infringement for free? In The Thirteenth International Conference on Learning Representations. Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulić, Anna Korhonen, and Sercan O Arik. 2026. Multi-agent design: Optimizing agents with better prompts and topologies. In The Fourteenth International Conference on Learning Representations.
12