From Trajectories to Differentiated Skills: Cognitive Differentiation for Self-Improving Agents

From Trajectories to Differentiated Skills: Cognitive Differentiation for Self-Improving Agents Yewei Zhao1 Derong Xu1,3 Wei Chen1 Yin Xu1,* Peilun Zhou2,* Xiaohan Wang2 Guojun Yin2 Wei Lin2 Mingjun Xiao1,* 1 University of Science and Technology of China 2 Meituan

3 City University of Hong Kong * Corresponding authors.

SEALQA

Abstract

54.1 Automation

arXiv:2610.23599v1 [cs.AI] 16 Oct 2026

Large language model (LLM) agents tackle complex tasks through reasoning and interaction with external environments. Agent skills provide persistent, transferable capabilities that guide task planning, inform contextual decisions, support tool use, and enable executable procedures. Yet existing skill-evolution methods largely organize knowledge around skill artifacts, overlooking the functional heterogeneity within individual skills and its implications for knowledge maintenance and use. Inspired by cognitive science, we propose Cognitive Differentiation in Skill Evolution (CDSE), which evolves a unified skill through four functionally distinct yet coordinated components: Meta, Experience, Tool, and Script. CDSE extracts evidence-grounded semantic gradients from successful and failed trajectories, consolidates them by functional role, and applies component-specific updates and validation with cross-component coordination and adaptive invocation. An analysis of 500 top-ranked skill packages shows that 96.8% contain knowledge spanning at least two functional roles. Across six benchmarks, three agent models, and two execution harnesses, CDSE achieves the highest scores in all 36 settings, improving over the Base agent by 7.6 percentage points and the strongest competing method in each setting by 2.6 points on average. Ablation studies further support the importance of functional differentiation and adaptive invocation.

1

49.7

Spreadsheet2

32.9

30.7

24.7

26.8

Avg. +7.6% Accuracy SWE-Pro

68.4

76.4

63.1 85.0

77.2 OfficeQA

87.7 LiveMath

Base

SkillOpt-Lite

Ours

(a) Task scores (%) on six benchmarks for GLM-5.2 under Claude Code. 1. Semantic Gradient Discovery

Success

2. Cognition-Guided Gradient Consolidation

Meta

Experience

Task-level control

Contextual heuristics

Plan · Arrange

Decide · Recover

3. Adaptive Self-Improvement Update

4. Skill-Augmented Agent

Specialized maintenance

Cross-component coordinate Tool

Script

Failure

Tool-specific guidance

Executable procedures

Semantic Gradients

Select · Use

Run · Validate

Adaptive invocation interface

Task & Environment

(b) CDSE overview. Figure 1: Performance and framework of CDSE.

evaluated execution trajectories, agents can continuously refine their skills to preserve effective behaviors, address recurring failures, and improve subsequent task performance. This has motivated growing research on trajectory-driven skill evolution as a means of continual agent self-improvement. Recent studies have explored complementary approaches to transforming agent execution experience into persistent, reusable capabilities. Experience- and memory-based methods accumulate workflows, reasoning strategies, and procedural knowledge from past interactions, enabling knowledge reuse across tasks (Wang et al., 2025; Ouyang et al., 2026; Fang et al., 2026). Trajectoryto-skill approaches further distill successful and failed executions into reusable skill artifacts and

Introduction

Large language model (LLM) agents tackle complex tasks through reasoning, planning, and interaction with external environments (Yao et al., 2023; Wang et al., 2024b; Yang et al., 2024). External skills extend these capabilities by providing a persistent, reusable knowledge state without modifying model parameters (Wang et al., 2024a, 2025; Zheng et al., 2025). By leveraging feedback from 1

Skills containing component

executable routines (Zheng et al., 2025; Ni et al., 2026; Alzubi et al., 2026). More recent methods focus on refining and managing evolving skills through evaluation feedback, failure analysis, behavioral validation, and lifecycle management (Yang et al., 2026b; Gautam et al., 2026; Moll et al., 2026; Zhang et al., 2026a; Lin et al., 2026). Collectively, these studies have substantially advanced how agents acquire, refine, and maintain reusable capabilities from execution experience. Despite these advances, existing approaches still largely organize learned knowledge around artifact boundaries, such as skill documents, memory entries, or skill banks, rather than systematically accounting for the distinct behavioral functions of individual updates. Yet a single execution trajectory can yield knowledge serving different purposes. For example, a spreadsheet task may suggest inspecting workbook structure before acting, verifying units before comparison, using a particular library to preserve formulas, or automating repeated sorting through a reusable script. Although derived from the same execution, these updates differ in scope, representation, validation requirements, and when they become useful. Organizing them around artifact boundaries alone does not determine how each should be maintained and used. The central challenge is therefore not merely what knowledge to retain, but how to organize, update, validate, and invoke it according to its functional role. Cognitive science offers a useful perspective on how to organize such functionally diverse knowledge. Atkinson and Shiffrin distinguished taskdependent control processes from relatively stable memory structures (Atkinson and Shiffrin, 1968). For agent skills, this distinction points to Meta knowledge, which governs task-level planning, monitoring, verification, and stopping, while other knowledge supports specific decisions and actions. Yet the knowledge underlying these actions is itself diverse. Cohen and Squire’s distinction between knowing how and knowing that (Cohen and Squire, 1980) highlights differences in how knowledge can be learned and retained. In agent skills, Experience captures context-conditioned heuristics and recovery strategies, Tool provides capabilityspecific operational guidance, and Script encodes reusable executable procedures with execution and validation contracts. These functional categories are tailored to agent execution rather than directly mapped from human cognitive systems. Finally, Miyake et al. highlighted the unity and diversity

100

3.2%

98.6% 88.6%

79.6%

17.8%

75 50

≥ 2 components

23.8%

25 0

20.8%

96.8% 58.2%

Meta Experience Tool

Script

Components per skill 1 2 3 4

Figure 2: Functional composition of 500 widely used skill packages. Left: prevalence of each component; right: the number of components per skill. Component labels are non-exclusive.

of executive functions (Miyake et al., 2000), underscoring the importance of coordination among specialized functions. Together, these perspectives suggest that different forms of skill knowledge should be maintained according to their functional roles while remaining coordinated during task execution. To examine these functional distinctions in realworld agent skills, we analyze 500 widely used skill packages using an expert-derived annotation scheme (Appendix F). As shown in Figure 2, 96.8% of the packages contain content spanning at least two functional roles, with Experience, Meta, and Tool appearing more frequently than Script. These findings suggest that multiple forms of reusable knowledge commonly coexist within individual skills, supporting a coordinated yet functionally differentiated approach to skill evolution. Building on these cognitive insights and empirical observations, we propose Cognitive Differentiation in Skill Evolution (CDSE), a framework that evolves a unified skill through four functionally distinct yet coordinated components. CDSE first performs Outcome-Contrastive Semantic Gradient Discovery to extract evidence-grounded directions for reusable behavioral change from successful and failed trajectories. It then applies CognitionGuided Gradient Consolidation to identify which semantic gradients are worth retaining and assign them to Meta, Experience, Tool, or Script according to their functional roles. Finally, Adaptive Self-Improvement Update refines these components through specialized update and validation mechanisms, while cross-component coordination and adaptive invocation support their use as task demands change. In this way, CDSE aligns how knowledge is learned from trajectories with how it is maintained and used during subsequent tasks. Across six benchmarks, three agent models, and two execution harnesses, CDSE achieves the highest scores in all 36 evaluation settings, improv2

ing over the Base agent by 7.6 percentage points and over the strongest competing skill-evolution method in each setting by 2.6 points on average. Component-level analyses further reveal distinct patterns of contribution, invocation, and transfer. We summarize our main contributions as follows: (1) identifying and empirically characterizing functional heterogeneity in real-world agent skills; (2) proposing CDSE to organize and evolve trajectoryderived knowledge through function-specific yet coordinated mechanisms; and (3) systematically evaluating its effectiveness and examining the behavior of differentiated skill components across diverse tasks, models, and execution harnesses.

2

cent trajectory-driven methods create and refine skill artifacts from successful and failed executions using failure analysis, bounded edits, and evaluation feedback (Ni et al., 2026; Alzubi et al., 2026; Yang et al., 2026b; Gautam et al., 2026), with subsequent work improving validation, lifecycle management, memory, multi-file evolution, and skill or meta-skill adaptation (Moll et al., 2026; He and Yang, 2026; Zhang et al., 2026e,a; Yang et al., 2026a; Lin et al., 2026; Shen et al., 2026; Wang et al., 2026). Despite these advances, the primary unit of organization remains an artifact boundary— a skill document, folder, or bank entry. Even when the artifact is hierarchical, multi-file, or memoryenhanced, an update’s execution function does not explicitly determine its ownership, representation, update rule, validation contract, or invocation timing. CDSE instead makes functional ownership the organizing principle, assigning trajectory-derived updates to Meta, Experience, Tool, or Script and maintaining them through differentiated but coordinated mechanisms.

Related Work

Language agents and self-improvement. LLM agents interleave reasoning, actions, and tool use to solve tasks through environment interaction (Yao et al., 2023; Wang et al., 2024b; Yang et al., 2024). Self-improvement methods extend this loop by turning execution feedback into persistent changes, either by retaining feedback, cross-task insights, and reasoning strategies in context or memory (Shinn et al., 2023; Zhao et al., 2024; Fu et al., 2024; Ouyang et al., 2026; Zhang et al., 2026d), or by modifying the agent implementation and even the meta-level adaptation procedure (Zhang et al., 2026b,c). Another line of work jointly evolves task generation and reasoning or tool-using policies through reinforcement learning on self-generated curricula (Zhao et al., 2025; Huang et al., 2026; Xia et al., 2026). These works show that improvement can persist at multiple levels of an agent system. CDSE focuses on a complementary setting: the base model and improvement protocol remain fixed, while evaluated trajectories update a persistent external skill state. The central question is therefore how heterogeneous behavioral updates should be organized within that evolving state.

3

Cognitive Differentiation in Skill Evolution

CDSE evolves a unified skill by discovering semantic gradients from evaluated trajectories, consolidating them by functional role, and updating its components through specialized maintenance and cross-component coordination (Figure 3). The updated components are then invoked adaptively during subsequent tasks. 3.1

Problem Formulation: Trajectory-Driven Skill Self-Evolution

An agent πθ solves a task x by interacting with an environment while selectively loading relevant components of a persistent skill. At evolution round t, the unified skill state comprises four functionally differentiated components: Meta, Experience, Tool, and Script. We denote this state by B (t) = (M, E, T , S)(t) , with component indices K = {M, E, T, S}. t Let D(t) = {(τi , yi )}N i=1 denote the evaluated trajectory corpus, where τi is an execution trace and yi its evaluated outcome. CDSE aims to improve subsequent task performance by updating the persistent skill state as follows: 
 B (t+1) = U B (t) , R G(D(t) ) , (1)

Agent skills and skill evolution. Agent skills externalize reusable capabilities as instructions, workflows, or executable procedures. Prior systems accumulate executable programs, reusable action routines, or APIs from experience (Wang et al., 2024a, 2025; Zheng et al., 2025). Agent Skills packages broaden this representation to instructions, linked resources, and scripts loaded through progressive disclosure (Zhang et al., 2025), while SkillsBench shows that skill efficacy depends on the task, model, harness, and package design (Li et al., 2026). Re-

where G extracts candidate semantic gradients, R 3

Outcome-Contrastive Semantic Gradient Discovery

Cognition-Guided Gradient Consolidation

Semantic update gradients Core Analysis: Verify evidence before finalizing.

Success

Success Trajectory filtering

Failure

Failure

Skill-Augmented Agent

Validate causal path

Identify behaviors worth preserving

Localize decisive failure

Infer minimal supported recovery

Adaptive SelfImprovement Update

Skill Task

Align by control role

1. Check units and footnotes before comparison. 2. Fetch remaining content if tool output is truncated.

Script Opportunity:

Consolidate control policies

Crosscomponents Coordinator

Existing Tool Categories

Assign to Tools

Tool A Tool B Tool C

Experience Semantic gradients

Align by local condition

Induce new task-level policies Updated Meta component

Current Experience Heuristics

Revise operational guidance Tool Semantic gradients

Guides contextspecific decisions

Meta Semantic gradients

Experience Semantic gradients

Aligned gradients

Policies

Adaptive Invocation Interface

Regulates task-level control

Create a reusable script for column-based sorting.

Unmatched gradients Current Meta

Adaptive invocation when and what skill components to use

Meta Semantic gradients

Reusable Lessons:

Functional Abstraction What reusable behavioral function should change?

Tool A

Align within Tool B category

Aligned gradients Induce new tool-use rules

Updated Tool B Knowledge

Script Semantic gradients

New Tool D

Align to existing scripts

Assign to Scripts

Other Script A gradients

Induce new tool-use rules

New Tool E

New Tool E Knowledge

Script B

accept

Functional Role Attribution

Guides tool selection Defines a reusable and operation executable procedure Tool Semantic gradients

Script Semantic gradients

Refine contextual heuristics Aligned gradients Induce new situational heuristics Unmatched gradients

Updated Experience component

Update & Independent implement Self-test verification

Tool B Unmatched gradients New Tool Categories

Retention Gate Worth retaining?

Semantic gradients

Updated script A

Script A

Script B Worthiness gate Candidate Script C

Candidate Script D

Defer

Create & Independent implement Self-test verification New script C

Figure 3: CDSE discovers semantic gradients from successful and failed executions, consolidates them by functional role, and updates the four components through specialized maintenance and coordination. Revised invocation descriptions connect the evolving skill to subsequent agent behavior.

retains and assigns them to functional components, and U performs the validated skill update detailed in Section 3.4. This process updates only the external skill state, leaving the model parameters and update protocol fixed. The formulation also supports repeated consolidation over a fixed trajectory corpus, as examined in our multi-round experiments. 3.2

trajectory and its outcome to produce candidate semantic gradients: F = G(D(t) ) =

Nt [

As(yi ) (P (τi ), yi ).

(2)

i=1

Each f = (g, e) ∈ F pairs a proposed reusable behavioral update g with its supporting trace lines e. The analyzer produces a core analysis, reusable lessons, and, when applicable, a script opportunity. Supporting evidence remains attached to each candidate, providing the basis for subsequent retention and functional attribution.

Outcome-Contrastive Semantic Gradient Discovery

The discovery stage extracts candidate semantic gradients from evaluated trajectories, i.e., trace-grounded, reusable directions for behavioral change (Shinn et al., 2023; Pryzant et al., 2023). A deterministic filter P removes runtime-only artifacts while preserving reasoning, tool actions, and observations relevant to behavioral diagnosis. Successful and failed trajectories are then analyzed under distinct diagnostic objectives. For successful trajectories, the analyzer identifies trace-supported behaviors worth preserving and safeguards for fragile successes. For failed trajectories, it localizes evidenced decisive errors or missing transitions and identifies the minimal supported recovery. An observed outcome alone does not establish the validity of a procedure or diagnosis; unsupported causes remain unresolved. Let s(yi ) ∈ {+, −} denote the success/failure label derived from the evaluated outcome yi . The corresponding analyzer As(yi ) processes the filtered

3.3 Cognition-Guided Gradient Consolidation Drawing on cognitive distinctions between control processes and memory structures, as well as between forms of knowing (Atkinson and Shiffrin, 1968; Cohen and Squire, 1980), CDSE separates gradient consolidation into two steps: retention and functional attribution. Retention determines which candidate gradients are worth preserving, while attribution identifies the components responsible for maintaining them. A retention gate r(f ) ∈ {0, 1} selects evidencesupported, actionable, and reusable gradients, filtering out generic, instance-specific, unsupported, or redundant findings. Each retained gradient is assigned one primary component and, when its evidence supports an additional functional role, at 4

most one secondary component. Let ρ(f ) ⊆ K denote the set of assigned components, with ρ(f ) = ∅ if r(f ) = 0. Consolidation produces Fk = {f ∈ F : r(f ) = 1, k ∈ ρ(f )}, k ∈ K,

1 ≤ |ρ(f )| ≤ 2 if r(f ) = 1.

feasible tests. Accepted proposals are implemented or revised, self-tested, and independently verified. Unverified executable changes are withheld. Cross-component coordination. After specialized maintenance, the coordinator C resolves conflicting or redundant guidance and residual ownership ambiguity across components through minimal edits that preserve verified executable artifacts and component-specific responsibilities.

(3)

Functional attribution follows the behavioral role of each gradient. Meta maintains task-level control policies, including planning, monitoring, and global verification. Experience maintains context-conditioned heuristics and recovery strategies activated by local conditions. Tool maintains capability-specific operational guidance, while Script receives proposals for reusable executable procedures and their execution or validation contracts. The primary assignment is recorded separately, and each gradient retains its original content and supporting evidence for subsequent component-specific updates. 3.4

Adaptive invocation. The description rewriter D derives each component’s scope, applicable conditions, and expected contribution from the coordinated skill state. Visible before component loading, these descriptions guide the agent’s selection of relevant components through its ordinary invocation mechanism as task demands change. The updated skill state is published only after coordination and description changes pass their respective validation checks. Details are in Appendix G.

Adaptive Self-Improvement Update

Given the consolidated gradient sets {Fk }k∈K , the update operator U in Equation 1 combines component-specific maintenance operators Uk , cross-component coordination C, and invocationdescription rewriting D: (t+1) (t) Bbk = Uk (Bk , Fk ),   B (t+1) = D C(Bb(t+1) ) .

4

Experiments

4.1

Experimental Setup

We evaluate on six benchmarks: OfficeQA (OpsahlOng et al., 2026), SpreadsheetBench2 (Zhu et al., 2026), SealQA (Pham et al., 2026), LiveMathBench (Liu et al., 2025), SWE-Pro (Deng et al., 2025), and AutomationBench (Shepard and Salimans, 2026), using GLM-5.2 (GLM-5-Team et al., 2026), DeepSeek-V4-Flash (DeepSeek-AI et al., 2026), and MiniMax-M3 (Lai et al., 2026) under both Claude Code and Codex. Each benchmark collection is split into training and test sets at a 2:7 ratio. Training trajectories support skill evolution; all reported results use the test split and benchmark-specific scoring procedures. We compare CDSE with the same agent without an evolved skill state (Base), Trace2Skill (Ni et al., 2026), SkillOpt-lite (Yang et al., 2026b), and EvoSkill (Alzubi et al., 2026). Within each model– harness block, methods share the training split, task instances, tool access, and context limit. Table 1 reports results after three evolution rounds for each skill-evolution method. Task execution is bounded by benchmark-specific per-question time and turn limits; tasks that remain unfinished when either limit is reached are counted as failures. Aggregate improvements are macro-averages of withinbenchmark percentage-point changes over the relevant model–harness–benchmark cells. Dataset

(4)

(t+1)

Here, Bbk denotes the candidate update to component k, and Bb(t+1) collects the four component updates. Inspired by the unity and diversity of cognitive functions (Miyake et al., 2000), this design combines specialized maintenance with coordination across the unified skill state. Specialized updates. Meta and Experience maintain atomic behavioral rules. Both align incoming gradients with existing rules, aggregate supporting evidence for revisions, and induce new rules from unmatched gradients. Meta updates task-level control policies, whereas Experience organizes heuristics around explicit local activation conditions. Tool assigns gradients to capabilityspecific categories and revises or induces operational rules within each category. Category-level capacity control preserves coverage across different tools. Script aligns proposals with existing procedures and screens new proposals for deterministic reuse, explicit input/output contracts, and 5

Agent model

Method

OfficeQA

Spreadsheet2

SealQA

LiveMath

SWE-Pro

Automation

71.5 74.1 +2.6 SkillOpt-lite 76.4 +4.9 EvoSkill 76.0 +4.5 CDSE 77.2 +5.7

24.0 28.3 +4.3 26.8 +2.8 25.9 +1.9 30.7 +6.7

45.9 48.2 +2.3 49.7 +3.8 52.4 +6.5 54.1 +8.2

83.0 82.3 -0.7 85.0 +2.0 84.4 +1.4 87.7 +4.7

60.2 62.6 +2.4 63.1 +2.9 62.6 +2.4 68.4 +8.2

20.8 21.4 +0.6 24.7 +3.9 25.5 +4.7 32.9 +12.1

Base

68.0 Trace2Skill 70.1 +2.1 DeepSeek-V4-Flash SkillOpt-lite 75.4 +7.4 (Claude Code) EvoSkill 72.8 +4.8 CDSE 77.3 +9.3

16.8 24.2 +7.4 22.7 +5.9 24.2 +7.4 26.9 +10.1

51.8 54.3 +2.5 55.9 +4.1 56.5 +4.7 59.4 +7.6

59.9 62.3 +2.4 63.9 +4.0 65.7 +5.8 69.1 +9.2

57.8 56.8 -1.0 60.0 +2.2 59.6 +1.8 64.1 +6.3

14.3 19.2 +4.9 22.6 +8.3 21.3 +7.0 28.9 +14.6

MiniMax-M3 (Claude Code)

71.6 73.2 +1.6 SkillOpt-lite 75.1 +3.5 EvoSkill 76.3 +4.7 CDSE 81.8 +10.2

25.3 33.6 +8.3 31.9 +6.6 30.5 +5.2 34.4 +9.1

49.4 53.1 +3.7 54.8 +5.4 54.2 +4.8 57.9 +8.5

73.5 77.6 +4.1 79.5 +6.0 80.1 +6.6 82.7 +9.2

57.2 62.0 +4.8 61.2 +4.0 62.9 +5.7 65.3 +8.1

9.1 13.3 +4.2 15.0 +5.9 15.5 +6.4 20.0 +10.9

GLM-5.2 (Codex)

68.8 70.8 +2.0 SkillOpt-lite 72.9 +4.1 EvoSkill 71.6 +2.8 CDSE 74.9 +6.1

22.7 29.8 +7.1 28.5 +5.8 29.4 +6.7 31.3 +8.6

22.6 24.1 +1.5 25.9 +3.3 24.0 +1.4 27.6 +5.0

79.0 80.5 +1.5 81.1 +2.1 81.5 +2.5 83.5 +4.5

62.3 64.8 +2.5 65.7 +3.4 67.0 +4.7 68.9 +6.6

42.2 43.8 +1.6 44.3 +2.1 44.9 +2.7 47.1 +4.9

Base

66.7 Trace2Skill 68.1 +1.4 DeepSeek-V4-Flash SkillOpt-lite 68.9 +2.2 (Codex) EvoSkill 69.8 +3.1 CDSE 71.9 +5.2

30.0 36.2 +6.2 34.3 +4.3 35.4 +5.4 38.1 +8.1

36.5 37.4 +0.9 38.0 +1.5 38.2 +1.7 40.8 +4.3

64.2 65.9 +1.7 66.5 +2.3 67.5 +3.3 69.2 +5.0

58.9 61.5 +2.6 62.9 +4.0 63.6 +4.7 65.9 +7.0

37.0 40.5 +3.5 41.7 +4.7 43.1 +6.1 45.4 +8.4

65.1 67.5 +2.4 SkillOpt-lite 70.9 +5.8 EvoSkill 68.6 +3.5 CDSE 72.2 +7.1

18.7 27.2 +8.5 29.1 +10.4 25.3 +6.6 29.4 +10.7

21.5 25.0 +3.5 25.3 +3.8 26.2 +4.7 29.4 +7.9

71.0 72.9 +1.9 73.5 +2.5 72.2 +1.2 74.5 +3.5

54.1 56.7 +2.6 57.1 +3.0 57.5 +3.4 60.8 +6.7

34.4 36.3 +1.9 38.7 +4.3 37.1 +2.7 39.9 +5.5

Base

Trace2Skill GLM-5.2 (Claude Code)

Base

Trace2Skill

Base

Trace2Skill

Base

Trace2Skill MiniMax-M3 (Codex)

Table 1: Task scores (%) across models, harnesses, and skill-evolution methods.

selections, the evaluation protocol, and baseline configurations are detailed in Appendix H. 4.2

These results support CDSE across diverse tasks and execution environments. With GLM-5.2 and Claude Code, single-round token costs are comparable across methods (Appendix A).

Overall Performance

Table 1 shows that CDSE achieves the highest score in all 36 settings across six benchmarks, three agent models, and two execution harnesses. On average, it improves over Base by 7.6 percentage points and exceeds the strongest competing skillevolution baseline in each setting by 2.6 points. This advantage holds on every benchmark with both Claude Code and Codex, spanning document and mathematical reasoning, spreadsheet manipulation, software engineering, and business workflows.

4.3

Functional Differentiation and Adaptive Invocation

Figure 4 examines component-removal effects (a), invocation rates (b), and first-invocation timing (c). Invocation patterns vary across benchmarks: all four components are invoked in at least 76.7% of AutomationBench episodes, whereas Script is invoked in only 6.0% on SpreadsheetBench2. Invocation timing also differs, with Script loaded later 6

Meta

Exp.

Tool

Script

Meta

Exp.

Tool

Script

OfficeQA

-4.5

-3.4

-5.5

-5.0

63%

97%

69%

25%

SpreadsheetBench2

-6.0

-6.0

-3.4

-8.0

71%

61%

23%

6%

SealQA

-2.3

-4.7

-1.2

+0.0

84%

64%

53%

7%

LiveMathBench

-7.5

+1.2

-2.2

-1.0

7%

10%

0%

0%

SWE-Pro

-6.2

-4.6

-6.5

+1.1

65%

63%

20%

8%

AutomationBench

-5.6

-8.2

-6.3

-5.0

100%

99%

100%

77%

+8.5

0%

Meta

1 -8.5

(a) Removal delta

Exp.

3

Tool

5

Script

7

9

11

13

100%

(b) Invocation rate

(c) Mean first invocation turn

Figure 4: Component removal effects and invocation behavior on GLM-5.2. Native

Score change from R0 (pp)

on SpreadsheetBench2 and SWE-Pro, but Tool and Script loaded early on AutomationBench. Component contributions are similarly taskdependent. Removing Tool reduces performance by 1.2 points on SealQA but 6.5 on SWE-Pro, while removing any component on AutomationBench causes a 5.0–8.2-point drop. Notably, despite its low invocation rate, removing Script causes the largest drop (8.0 points) on SpreadsheetBench2. Component importance therefore does not directly follow invocation frequency. Under progressive disclosure, descriptions remain visible before loading, so even components with zero recorded invocations can influence execution and exhibit ablation effects. Appendix B reports mean invocation turns and further component analysis. 16

Spreadsheet2

LiveMath 16

12

12

8

8

4

4

0 R0

R1

R2

(a) GLM-5.2

0 R3 R0

R1

R2

Score change from target Base (pp)

16

12

12

8

8

4

4

0 -4

0 OQA

SSB2 SealQA LMB SWE-Pro AB

-4

OQA

(c) Target harness: Claude Code

SSB2 SealQA LMB SWE-Pro AB

(d) Target harness: Codex

16

16

12

12

8

8

4

4

-4

0 OQA

SSB2 SealQA LMB SWE-Pro AB

-4

OQA

SSB2 SealQA LMB SWE-Pro AB

Figure 6: Cross-model and cross-harness transfer gains.

4.5

Transfer across Tasks, Models, and Harnesses

Cross-model and cross-harness transfer. Figure 6 evaluates the transfer of complete skill states between GLM-5.2 and DeepSeek-V4-Flash, and between Claude Code and Codex. Cross-model transfer improves 11 of 12 target configurations, averaging 4.2 percentage points (pp) over Base for each target model. Cross-harness transfer improves all 12 configurations, averaging 3.3 pp for Claude Code and 3.4 pp for Codex. Although native skill states perform better on average, transferred states retain substantial gains across models and harnesses (Appendix D).

R3

Figure 5: Score gains across three CDSE evolution rounds.

4.4

(b) Target model: DeepSeek-V4-Flash

16

0

Automation

(b) DeepSeek-V4-Flash

Transferred

(a) Target model: GLM-5.2

Multi-Round Skill Evolution

Cross-benchmark component transfer. Table 2 evaluates component transfer from SpreadsheetBench2 or AutomationBench to other benchmarks. Meta improves 9 of 10 source–target pairs, averaging 3.6 pp over Base, whereas Experience transfers more selectively and Tool and Script show limited consistent gains. These differences are consistent with the broader applicability of task-level control knowledge and the stronger context or environment dependence of other components. Parenthesized

Figure 5 compares three CDSE evolution rounds over the same trajectory corpus on SpreadsheetBench2, LiveMathBench, and AutomationBench, using GLM-5.2 and DeepSeek-V4-Flash under Claude Code. The first round delivers most of the improvement; later rounds show smaller, benchmark-dependent changes and non-monotonic fluctuations. Results across all six benchmarks and three models show the same diminishing-return pattern under this fixed-corpus setting (Appendix C). 7

Source

Component

OfficeQA

Spreadsheet2

SealQA

LiveMath

SWE-Pro

Automation

Spreadsheet2

Meta Experience Tool Script

+4.1 (18%) +0.8 (6%) -1.3 (1%) -0.6 (2%)

– – – –

+4.3 (20%) -2.8 (14%) +0.0 (0%) -0.9 (0%)

+0.0 (0%) -0.4 (0%) +1.0 (0%) +0.7 (0%)

+4.1 (10%) +0.0 (7%) -0.6 (1%) -3.9 (3%)

+7.1 (28%) -1.6 (2%) +0.0 (0%) -1.6 (0%)

Automation

Meta Experience Tool Script

+5.7 (10%) +2.2 (7%) -1.6 (0%) +0.0 (0%)

+1.4 (19%) -1.2 (0%) -0.5 (0%) +0.0 (0%)

+6.7 (35%) +3.8 (38%) -1.1 (0%) +0.0 (0%)

+0.2 (0%) -0.8 (0%) +0.0 (0%) -0.4 (0%)

+2.6 (6%) +0.0 (4%) -0.6 (0%) +0.0 (0%)

– – – –

Table 2: Cross-benchmark component transfer on GLM-5.2.

values report explicit component-loading rates; descriptions remain visible before loading, so zero loading does not imply zero influence.

nization, specialized maintenance, and descriptionguided component selection. Appendix E details the four variants.

4.6

4.7

Mechanism Ablations

Variant

Using GLM-5.2 with Claude Code, we divide each method’s single-round input-token count by its score gain over Base in the same round. Table 4 reports the resulting cost per percentage point of improvement. CDSE achieves the lowest cost on all three benchmarks with comparable input-token consumption (Appendix A), indicating improved token efficiency without substantially increasing the evolution budget.

Spreadsheet Automation Bench2 SWE-Pro Bench

Base

24.0

60.2

20.8

Full CDSE

30.7

68.4

32.9

w/o Cognitive Differentiation

25.8

62.6

24.3

w/o Specialized Update

27.1

65.4

28.0

w/o Cross-Component Coordination

29.6

67.5

30.7

w/o Adaptive Invocation Interface

26.4

Token Efficiency

Method 64.3

Trace2Skill SkillOpt-lite EvoSkill CDSE

27.1

Table 3: Mechanism ablations on GLM-5.2 with Claude Code. All entries are task scores (%).

Spreadsheet2

SWE-Pro

Automation

15.41 29.33 42.67 7.60

41.30 59.82 75.47 15.04

23.90 6.56 5.06 2.19

Table 4: Input tokens per percentage point of score gain over Base after one evolution round (million tokens/pp, ↓), using GLM-5.2 with Claude Code.

Table 3 evaluates four mechanism ablations on three benchmarks. Removing cognitive differentiation collapses the four components into a unified skill, whereas removing specialized updates preserves the component structure but applies a shared update strategy. The other variants disable crosscomponent coordination or remove the Adaptive Invocation Interface that provides content-aware descriptions to guide component selection. Full CDSE achieves the highest score across all three benchmarks. Removing cognitive differentiation causes the largest drops, followed by removing the Adaptive Invocation Interface. The no-differentiation variant uses the same semantic gradients as CDSE, suggesting benefits from functional organization and maintenance beyond lesson extraction. Specialized updates and crosscomponent coordination also contribute. These results highlight the importance of functional orga-

5

Conclusion

CDSE organizes trajectory-derived knowledge into four functionally distinct components—Meta, Experience, Tool, and Script—using specialized updates, cross-component coordination, and adaptive invocation. Across six benchmarks, three models, and two harnesses, it achieves the highest scores in all 36 settings. Component and transfer analyses reveal task-dependent contributions and varying transferability, while ablations highlight the importance of functional differentiation and adaptive invocation. Token-efficiency analysis further shows that CDSE achieves greater performance gains per input token. These findings suggest that effective skill evolution benefits from maintaining and using knowledge according to its functional role. 8

Limitations

procedural memory. In Findings of the Association for Computational Linguistics: ACL 2026.

Our evaluation covers six benchmarks, three agent models, and two execution harnesses, leaving generalization to other domains and agent configurations to future work. Multi-round experiments reuse a fixed trajectory corpus and therefore do not establish sustained improvement in continually changing environments. The four-component organization is supported by our survey and experiments, but alternative decompositions and their effectiveness remain to be studied.

Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. 2024. AutoGuide: Automated generation and selection of context-aware guidelines for large language model agents. In Advances in Neural Information Processing Systems, volume 37. Srishti Gautam, Arjun Radhakrishna, and Sumit Gulwani. 2026. SkillAxe: Sharpening llmauthored agent skills through evaluation-guided selfrefinement. Preprint, arXiv:2606.10546. GLM-5-Team et al. 2026. GLM-5: from vibe coding to agentic engineering. Preprint, arXiv:2602.15763.

Ethical Considerations CDSE is intended to support research on reliable and reusable LLM agents. Evolved skills may retain errors or biases from models and execution trajectories, so improved task performance does not guarantee safe deployment. Practical use should respect data privacy and third-party licenses, restrict tool permissions, and retain human oversight for consequential actions. We encourage responsible use of the framework and derived artifacts.

Yu He and Weikai Yang. 2026. SkillCommit: Evolving agent skills through behaviorally validated scope expansion. Preprint, arXiv:2608.15165. Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. 2026. R-Zero: Selfevolving reasoning LLM from zero data. In International Conference on Learning Representations. Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu, Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, Jinkai Hu, Jiayao Li, Rui Gao, Zekun Li, Songquan Zhu, Jingkai Zhou, and Pengyu Zhao. 2026. MiniMax Sparse Attention. Preprint, arXiv:2606.13392.

References Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Weiyuan Chen, and Tu Vu. 2026. EvoSkill: Automated skill discovery for multi-agent systems. Preprint, arXiv:2603.02766.

Xiangyi Li, Yimin Liu, Wenbo Chen, et al. 2026. SkillsBench: Benchmarking how well agent skills work across diverse tasks. Preprint, arXiv:2602.12670.

Richard C. Atkinson and Richard M. Shiffrin. 1968. Human memory: A proposed system and its control processes. In K. W. Spence and J. T. Spence, editors, The Psychology of Learning and Motivation: Advances in Research and Theory, volume 2, pages 89–195. Academic Press.

Huawei Lin, Peng Li, Jie Song, Fuxin Jiang, and Tieying Zhang. 2026. MUSE-Autoskill: Self-evolving agents via skill creation, memory, management, and evaluation. Preprint, arXiv:2605.27366. Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. 2025. Are your LLMs capable of stable reasoning? In Findings of the Association for Computational Linguistics: ACL 2025, pages 17594–17632. Association for Computational Linguistics.

Neal J. Cohen and Larry R. Squire. 1980. Preserved learning and retention of pattern-analyzing skill in amnesia: Dissociation of knowing how and knowing that. Science, 210(4466):207–210. DeepSeek-AI et al. 2026. DeepSeek-V4: Towards highly efficient million-token context intelligence. Preprint, arXiv:2606.19348.

Akira Miyake, Naomi P. Friedman, Michael J. Emerson, Alexander H. Witzki, Amy Howerter, and Tor D. Wager. 2000. The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1):49–100.

Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Kenstler. 2025. SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks? Preprint, arXiv:2509.16941.

Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan, Martin Hadamitzky, Daniel Rueckert, Lisa Adams, and Keno Bressem. 2026. GRASP: Gated regressionaware skill proposer for self-improving llm agents. Preprint, arXiv:2605.29668. Accepted at EMNLP 2026.

Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. 2026. Memp: Exploring agent

9

Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. 2026. Trace2Skill: Distill trajectory-local lessons into transferable agent skills. Preprint, arXiv:2603.25158.

via two-timescale meta-skill evolution. Preprint, arXiv:2607.05297. Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2025. Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 63897–63911. PMLR.

Krista Opsahl-Ong, Arnav Singhvi, Jasmine Collins, Ivan Zhou, Cindy Wang, Ashutosh Baheti, Owen Oertell, Jacob Portes, Sam Havens, Erich Elsen, Michael Bendersky, Matei Zaharia, and Xing Chen. 2026. OfficeQA Pro: An enterprise benchmark for end-to-end grounded reasoning. Preprint, arXiv:2603.08655.

Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, Yiyang Zhou, Caiming Xiong, and Huaxiu Yao. 2026. Agent0: Unleashing self-evolving agents from zero data via tool-integrated reasoning. In Conference on Language Modeling.

Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2026. ReasoningBank: Scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations.

John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems, volume 37.

Thinh Pham, Nguyen Nguyen, Pratibha Zunjare, Weiyuan Chen, Yu-Min Tseng, and Tu Vu. 2026. SealQA: Raising the bar for reasoning in searchaugmented language models. In International Conference on Learning Representations.

Min Yang, Jinghua Piao, Xu Xia, Xiaochong Lan, Jiaju Chen, Yongshun Gong, and Yong Li. 2026a. SkillMaster: Toward autonomous skill mastery in llm agents. Preprint, arXiv:2605.08693. Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, Dongdong Chen, Xue Yang, and Chong Luo. 2026b. SkillOpt: Executive strategy for self-evolving agent skills. Preprint, arXiv:2605.23904.

Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with “Gradient Descent” and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7957–7968. Association for Computational Linguistics.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations.

Junhao Shen, Zhanqiu Zhang, Yiwen Guo, and Hong Cheng. 2026. Progressive agent skill generation via reinforcement learning. Preprint, arXiv:2608.01678. Daniel Shepard and Robin Salimans. 2026. AutomationBench. Preprint, arXiv:2604.18934.

Barry Zhang, Keith Lazuka, and Mahesh Murag. 2025. Equipping agents for the real world with Agent Skills. Anthropic Engineering. Official technical article, October 16, 2025.

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36.

Hanrong Zhang, Shicheng Fan, Henry Peng Zou, Yankai Chen, Zhenting Wang, Jiayu Zhou, Chengze Li, WeiChieh Huang, Yifei Yao, Kening Zheng, Xue Liu, Xiaoxiao Li, and Philip S. Yu. 2026a. CoEvoSkills: Self-evolving agent skills via co-evolutionary verification. Preprint, arXiv:2604.01687. Accepted at COLM 2026.

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024a. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research.

Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. 2026b. Darwin Gödel Machine: Open-ended evolution of self-improving agents. In International Conference on Learning Representations.

Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. 2024b. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18:186345.

Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. 2026c. Hyperagents. Preprint, arXiv:2603.19461.

Zefeng Wang, Minxi Yan, Jinhe Bi, Sikuan Yan, Volker Tresp, and Yunpu Ma. 2026. MetaSkillEvolve: Recursive self-improvement of llm agents

10

gain over Base in the same round. Nonpositive gains are marked N/A.

Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. 2026d. Agentic context engineering: Evolving contexts for self-improving language models. In International Conference on Learning Representations.

B

Table 7 reports the mean invocation turns underlying Figure 4. These values describe when a component is invoked; components that are never invoked have no corresponding mean turn. The components exhibit distinct but coordinated behavioral roles across tasks. Their removal effects vary substantially by benchmark, indicating that different tasks rely on different forms of reusable knowledge. AutomationBench provides a clear example of coordination: removing Meta, Experience, Tool, or Script reduces performance by 5.6, 8.2, 6.3, and 5.0 points, respectively, while every component is invoked in at least 76.7% of episodes. In contrast, other benchmarks show more selective dependence. For example, SealQA relies primarily on Meta and Experience, whereas SWE-Pro is substantially more sensitive to Meta and Tool. These heterogeneous contribution profiles support the functional differentiation of the skill state rather than treating all reusable knowledge as a homogeneous collection of guidance. Invocation frequency does not directly reflect functional importance. On SpreadsheetBench2, Script is invoked in only 6.0% of episodes, yet removing it causes the largest performance drop on the benchmark, at 8.0 points. This suggests that infrequently activated components can still be critical when triggered at the appropriate stage of execution. Invocation timing further reveals task-conditioned specialization. Meta and Experience are generally selected early, while Script first appears much later on SpreadsheetBench2 and SWE-Pro, at turns 12.3 and 10.2 on average. By contrast, Tool and Script are invoked almost immediately on AutomationBench. Together, these patterns show that the agent does not follow a fixed component pipeline, but adaptively selects and coordinates differentiated knowledge according to task demands. This be-

Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. 2026e. Library drift: Diagnosing and fixing a silent failure mode in self-evolving llm skill libraries. Preprint, arXiv:2605.19576. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632–19642. No. 17. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025. Absolute zero: Reinforced self-play reasoning with zero data. Preprint, arXiv:2505.03335. Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su. 2025. SkillWeaver: Web agents can self-improve by discovering and honing skills. Preprint, arXiv:2504.07079. Jian Zhu, Yuzheng Zhang, Zeyao Ma, Bohan Zhang, Armin Schoepf, Daniel Woloch, Peter Yiliu Wang, Guangyu Robert Yang, Samuel Jacob, Siddharth Nagisetty, Abhiram Chundru, Jean Lin, Spencer Mateega, and Jing Zhang. 2026. SpreadsheetBench 2: Evaluating agents on end-to-end business spreadsheet workflows. Preprint, arXiv:2606.29955.

A

Token Consumption and Efficiency for One Evolution Round

Table 5 reports the input and output token counts for one evolution round with GLM-5.2 and Claude Code. Table 6 extends the main-text input-token efficiency comparison to all six benchmarks and includes output tokens. Each value is the corresponding single-round token count divided by the score OfficeQA

Spreadsheet Bench2

Extended Component Analysis

SealQA

LiveMathBench

SWE-Pro

Automation Bench

Method

Input

Output

Input

Output

Input

Output

Input

Output

Input

Output

Input

Output

Trace2Skill SkillOpt-lite EvoSkill CDSE

35.80 38.50 37.20 37.00

1.350 1.450 1.410 1.400

49.30 52.80 51.20 50.90

2.700 2.900 2.820 2.800

10.60 11.40 11.10 11.00

0.480 0.520 0.510 0.500

9.51 10.10 9.89 9.84

1.050 1.150 1.110 1.100

123.90 131.60 128.30 127.80

5.600 5.950 5.830 5.800

23.90 25.60 24.80 24.70

1.050 1.150 1.110 1.100

Table 5: Input and output token consumption (millions of tokens) in a single evolution round with GLM-5.2 and Claude Code.

11

Spreadsheet Bench2

OfficeQA

SealQA

LiveMathBench

Automation Bench

SWE-Pro

Method

Input

Output

Input

Output

Input

Output

Input

Output

Input

Output

Input

Output

Trace2Skill SkillOpt-lite EvoSkill CDSE

12.34 6.64 10.94 7.87

0.466 0.250 0.415 0.298

15.41 29.33 42.67 7.60

0.844 1.611 2.350 0.418

5.89 3.17 1.61 1.55

0.267 0.144 0.074 0.070

N/A 11.22 6.59 3.08

N/A 1.278 0.740 0.344

41.30 59.82 75.47 15.04

1.867 2.705 3.429 0.682

23.90 6.56 5.06 2.19

1.050 0.295 0.227 0.097

Table 6: Single-round input and output tokens per percentage point of score gain over Base on all six benchmarks (million tokens/pp, ↓), using GLM-5.2 with Claude Code. Bold marks column minima before rounding. N/A indicates no positive gain: Trace2Skill decreases the LiveMathBench score by 1.4 points. Benchmark

Meta Exp. Tool Script

Benchmark

R0

R1

R2

R3

OfficeQA SpreadsheetBench2 SealQA LiveMathBench SWE-Pro AutomationBench

1.60 1.40 1.00 6.25 1.30 1.10

OfficeQA SpreadsheetBench2 SealQA LiveMathBench SWE-Pro AutomationBench

71.5 24.0 45.9 83.0 60.2 20.8

76.2 30.7 53.0 86.2 68.7 32.1

77.8 29.9 53.5 87.3 69.0 33.3

77.2 30.7 54.1 87.7 68.4 32.9

1.10 2.70 1.13 2.76 1.70 1.10

1.80 3.60 4.90 12.30 2.24 2.67 – – 3.80 10.20 1.10 1.80

Table 7: Mean first invocation turn of the four skill components in Figure 4, evaluated with GLM-5.2 under Claude Code. A dash indicates no invocation.

Table 8: Multi-round evolution scores (%) for GLM-5.2 under Claude Code. R0 is the no-skill baseline.

havior is consistent with the cognitive-science motivation of unity and diversity: distinct functional components remain separable in their roles while cooperating within a unified capability state.

C

Additional Evolution Results

Figure 5 in the main text reports three evolution rounds on SpreadsheetBench2, LiveMathBench, and AutomationBench for GLM-5.2 and DeepSeekV4-Flash under Claude Code, using the same trajectory corpus across rounds. For GLM-5.2, R1 improves over the no-skill R0 baseline by 6.7, 3.2, and 11.3 percentage points on these benchmarks, respectively; DeepSeek-V4-Flash gains 10.1, 8.8, and 13.3 points. At R3, the gains are 6.7, 4.7, and 12.1 points for GLM-5.2, and 10.1, 9.2, and 14.6 points for DeepSeek-V4-Flash. Intermediate checkpoints show modest non-monotonic fluctuations. Figure 7 compares all six benchmarks across GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3 under Claude Code. Across these settings, R1 accounts for most of the improvement, while later rounds yield smaller, task-dependent changes. Tables 8, 9, and 10 report the corresponding R0–R3 scores for each model. Under this fixed-corpus setting, subsequent updates mainly produce small revisions to the existing skill state. Sustained improvement may therefore require additional execution experience that supplies new behavioral evidence.

Benchmark

R0

R1

R2

R3

OfficeQA SpreadsheetBench2 SealQA LiveMathBench SWE-Pro AutomationBench

68.0 16.8 51.8 59.9 57.8 14.3

76.5 26.9 59.4 68.7 64.7 27.6

77.3 26.9 59.8 68.3 63.5 28.2

77.3 26.9 59.4 69.1 64.1 28.9

Table 9: Multi-round evolution scores (%) for DeepSeek-V4-Flash under Claude Code. R0 is the noskill baseline.

Benchmark

R0

R1

R2

R3

OfficeQA SpreadsheetBench2 SealQA LiveMathBench SWE-Pro AutomationBench

71.6 25.3 49.4 73.5 57.2 9.1

80.8 34.1 57.9 82.2 65.0 18.8

81.4 34.7 57.5 82.5 64.7 19.5

81.8 34.4 57.9 82.7 65.3 20.0

Table 10: Multi-round evolution scores (%) for MiniMax-M3 under Claude Code. R0 is the no-skill baseline.

D

Transfer Evaluation Details

Cross-benchmark component transfer. We take an individual component evolved on SpreadsheetBench2 or AutomationBench and make it available on a different target benchmark. Table 2 reports the score change from the target Base on GLM-5.2. The parenthesized value is the fraction 12

DeepSeek-V4-Flash

15

10

10

10

5

Gain (pp)

15

0

5

0 R0

R1

R2

R3

5

0 R0

(a) OfficeQA

R1

R2

R3

R0

15

10

10

10

0

Gain (pp)

15

5

5

0 R1

R2

(d) LiveMathBench

R3

R2

R3

R2

R3

(c) SealQA

15

R0

R1

(b) SpreadsheetBench2

Gain (pp)

Gain (pp)

MiniMax-M3

15

Gain (pp)

Gain (pp)

GLM-5.2

5

0 R0

R1

R2

R3

R0

(e) SWE-Pro

R1

(f) AutomationBench

Figure 7: Evolution over three rounds on all six benchmarks with GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3 under Claude Code. Scores are shown as percentage-point gains over each model’s no-skill baseline (R0). Each curve shows one reported score at each round; the same trajectory corpus is reused across rounds. The first round provides most of the gain, with smaller, benchmark-dependent changes thereafter.

of target episodes that explicitly load the transferred component. Under progressive disclosure, component descriptions remain visible to the agent before loading and may affect execution even when the full component is never loaded. Invocation rate therefore measures explicit component loading, rather than total exposure to the component. The observed transfer differences are consistent with the functional scopes of the components. Meta stores task-level control policies that can remain useful across heterogeneous tasks. Experience stores context-conditioned heuristics, while Tool and Script store capability-specific instructions and executable procedures, respectively. These forms of knowledge require progressively closer alignment between source and target settings, consistent with their more selective or limited crossbenchmark transfer.

each transfer experiment. In Figure 6, Native denotes a state evolved in the receiving configuration, whereas Transferred denotes one evolved in the paired source configuration. Tables 11 and 12 give the exact scores for these settings, including the receiving Base and both skill states. GLM-5.2 Benchmark

Base

N

DeepSeek-V4-Flash T Base

N

T

OfficeQA 71.5 77.2 72.5 68.0 77.3 SpreadsheetBench2 24.0 30.7 30.0 16.8 26.9 SealQA 45.9 54.1 49.4 51.8 59.4 LiveMathBench 83.0 87.7 85.8 59.9 69.1 SWE-Pro 60.2 68.4 63.5 57.8 64.1 AutomationBench 20.8 32.9 29.2 14.3 28.9

72.5 20.7 48.5 71.6 60.4 20.1

Table 11: Cross-model transfer scores (%) for Figure 6 (Claude Code fixed). Column groups identify the receiving configuration. N: Native; T: Transferred from the other configuration. Gains are relative to Base.

Cross-model and cross-harness transfer. For cross-model transfer, we exchange complete skill states evolved by GLM-5.2 and DeepSeek-V4Flash while keeping Claude Code fixed. For crossharness transfer, we exchange states evolved under Claude Code and Codex while keeping GLM5.2 fixed. Each direction is evaluated on all six benchmarks, yielding 12 receiving settings for 13

Claude Code Benchmark

Base

N

tribution of this interface to component selection during execution.

Codex

T Base

N

T

OfficeQA 71.5 77.2 75.3 68.8 74.9 74.3 SpreadsheetBench2 24.0 30.7 28.6 22.7 31.3 26.1 SealQA 45.9 54.1 47.5 22.6 27.6 25.5 LiveMathBench 83.0 87.7 84.2 79.0 83.5 80.2 SWE-Pro 60.2 68.4 64.2 62.3 68.9 65.6 AutomationBench 20.8 32.9 25.1 42.2 47.1 46.0

Functional Composition of the Top-500 Skills

F.1

Survey Procedure and Annotation Labels

We analyze the top-500 skill packages from the all-time ranking on https://skills.sh. Human experts first examine the top-30 packages to identify recurring functional content and establish 15 annotation labels: 14 labels grouped under Meta, Experience, Tool, and Script, and an auxiliary label, O1 (Other / Non-behavioral / Unclear). We use GPT 6-Astra to apply these expert-derived labels to the top-500 packages. Labels are non-exclusive, allowing a package to contain multiple labels within and across components. Table 14 lists the labels and their frequencies. Meta captures global task-level control, while Experience captures contextual decisions and recovery; M3 and E4 distinguish global evidence control from local interpretation or verification. Tool covers capability-specific operational knowledge, and Script covers executable procedures (S1) and their execution or validation contracts (S2). Scriptrelated content may therefore include contracts without executable code. A package is counted once for a component if it contains any of that component’s labels. All percentages use the 500 packages as the denominator. O1 is an auxiliary label and is excluded from the four-component count.

Table 12: Cross-harness transfer scores (%) for Figure 6 (GLM-5.2 fixed). Column groups identify the receiving configuration. N: Native; T: Transferred from the other configuration. Gains are relative to Base.

Native evolution is stronger on average in both experiments. The positive transfer gains indicate that substantial learned behavioral knowledge remains useful after a change in model or execution harness, while the advantage of native states indicates additional benefit from adapting to the receiving configuration.

E

F

Mechanism Ablation Details

We evaluate four mechanism ablations on SpreadsheetBench2, SWE-Pro, and AutomationBench using GLM-5.2 under Claude Code. Unlike the component-removal analysis in Figure 4, these ablations modify the organization, maintenance, coordination, or invocation of the skill state. Without cognitive differentiation. The four components are merged into a unified skill maintained by a generic update mechanism, using the same semantic gradients as full CDSE.

Validation against expert annotations. Human experts annotate 100 skill packages randomly sampled from the 500-package collection. Using these annotations as reference, we evaluate GPT 6-Astra’s labels separately for Meta, Experience, Tool, and Script. Table 13 reports precision, recall, and F1 = 2P R/(P + R). The macro average weights the four components equally.

Without specialized updates. The fourcomponent structure and adaptive invocation are preserved, but component-specific maintenance operators are replaced by a shared update strategy. Without cross-component coordination. Component-specific updates and validation are retained, while cross-component conflict resolution, deduplication, and ownership reconciliation are disabled. Without the Adaptive Invocation Interface. This variant removes the content-aware invocation interface that describes each component’s scope, when it is useful, and its expected benefit to guide component selection. Component-specific maintenance and cross-component coordination are retained. The ablation therefore evaluates the con-

Component

Precision

Recall

F1

Meta Experience Tool Script

0.860 0.880 0.900 0.920

0.840 0.840 0.860 0.900

0.850 0.860 0.880 0.910

Macro average

0.890

0.860

0.875

Table 13: LLM annotation quality against expert labels on a random 100-package subset. F1 is computed from the reported precision and recall.

14

Component

ID

Label

Meta Meta Meta Meta Experience Experience Experience Experience Tool Tool Tool Tool Script Script Other

M1 M2 M3 M4 E1 E2 E3 E4 T1 T2 T3 T4 S1 S2 O1

Task Planning & Decomposition Progress & Resource Control Global Verification & Evidence Control Stopping & Finalization Context-Conditioned Heuristic Failure Pattern & Anti-Pattern Recovery & Correction Strategy Local Interpretation & Verification Tool Selection & Applicability Tool Operation & Sequencing Tool Output & Limitation Handling Tool Failure & Environment Handling Reusable Executable Procedure Execution Contract & Validation Other / Non-behavioral / Unclear

Count

Percentage

356 267 214 345 486 421 268 435 389 377 286 257 98 68 19

71.2% 53.4% 42.8% 69.0% 97.2% 84.2% 53.6% 87.0% 77.8% 75.4% 57.2% 51.4% 19.6% 13.6% 3.8%

Table 14: Fine-grained label prevalence among 500 skills. Labels are non-exclusive; each count is the number of skills containing that label.

F.2

Survey Results

These descriptive results characterize the selected highly ranked packages.

Table 15 reports component prevalence. Experience appears in 493 packages (98.6%), Meta in 443 (88.6%), Tool in 398 (79.6%), and Script-related content in 119 (23.8%). Component

Count / 500

Percentage

Meta Experience Tool Script

443 493 398 119

88.6% 98.6% 79.6% 23.8%

G

This section specifies the implementation contracts underlying the three stages in Section 3. We use the same component state and trajectory notation as the main text. The following details expand the filtering, evidence selection, and validation steps summarized there. G.1

Table 15: Component prevalence. A skill is counted once for a component if it contains any of that component’s labels.

Skills

Percentage

0 1 2 3 4

0 16 104 291 89

0.0% 3.2% 20.8% 58.2% 17.8%

Total

500

100.0%

Trajectory Filtering and Finding Extraction

Preserving evidence while filtering logs. The input to discovery is the evaluated corpus D(t) . A deterministic filter produces τ̄i = P (τi ) by removing runtime-only heartbeats, retry notifications, lifecycle status, duplicated terminal metrics, and redundant metadata and wrappers. It retains ordered reasoning, tool calls and observations, permission failures, final responses, and termination state, while retaining the raw trace separately. Analysis operates on filtered execution evidence rather than a model-generated summary.

Table 16 shows that 484 packages (96.8%) contain at least two components, including 380 (76.0%) with at least three and 89 (17.8%) with all four. Only 16 packages (3.2%) contain a single component. This prevalence of multiple functional components motivates CDSE’s functionally differentiated skill organization. Components

Implementation Details of CDSE

Outcome-conditioned behavioral diagnosis. The analyzer uses (τ̄i , yi ) to identify a reusable direction of behavioral change, or semantic gradient. Natural-language update directions follow the semantic and textual gradient perspective (Shinn et al., 2023; Pryzant et al., 2023). Here, outcome-contrastive discovery applies different diagnostic objectives to successes and failures; it requires neither matched trajectory pairs nor a numerical gradient. For a success, the analyzer reconstructs a minimal trace-supported account of

Table 16: Distribution of the number of functional components per skill. Unlike component marginals, these categories are mutually exclusive.

15

Retention gate. The router first tests whether a candidate is supported, actionable, reusable, and likely to change execution. It drops generic advice, unsupported diagnoses, sample-specific answers, and redundant core analyses already covered by the extracted lessons. For failures, dropping a redundant core requires the lessons to preserve both the diagnosis and the recovery. Writing the gate as r(f ) ∈ {0, 1}, its output is F keep = {f ∈ F : r(f ) = 1}. Only retained gradients proceed to attribution.

how the result was reached and rates its integrity as valid, fragile, invalid, or unknown. It extracts behavior worth preserving and safeguards that would strengthen a fragile success. For a failure, it locates the earliest clearly evidenced decisive error, missing transition or recovery, or external obstacle in the inspected trace, together with the smallest supported recovery. Answer mismatch, timeout, and parse failure are symptoms whose causes must be supported by the trace. A successful outcome alone does not establish a sound procedure, and unresolved causes remain unknown.

Functional role attribution. For each retained gradient, the router identifies a primary component and at most one secondary component. The set ρ(f ) records membership in these targets, with primary status stored separately:

Functional abstraction into semantic gradients. Let zi = s(yi ) ∈ {+, −} map the task-specific outcome to its success/failure label. The two diagnostic paths feed the same functional abstraction step: Fi = Azi (P (τi ), yi ),

fij = (gij , eij ),

ρ(f ) ⊆ K,

(5)

Fk = {f ∈ F keep : k ∈ ρ(f )}.

where gij expresses a reusable behavioral update and eij identifies its supporting trace lines. A+ seeks behavior to preserve or strengthen; A− seeks an evidenced failure and supported recovery. Each analysis emits a core finding, up to two reusable lessons, and at most one script suggestion for a repeated deterministic operation: i Fi = {fij }m j=1 ,

1 ≤ mi ≤ 4,

F=

Nt [

(7)

A secondary component is included only when the same evidence has substantial value under another functional responsibility. Meta receives task-wide control changes; Experience receives heuristics activated by identifiable local conditions. Tool receives operational guidance whose validity depends on a named tool or capability; Script receives proposals for deterministic, testable procedures or their execution contracts. For example, checking units before comparing values is contextual experience, whereas a restriction specific to a spreadsheet library belongs to Tool. A repeatable transformation with explicit inputs and tests can instead become a Script proposal. The router neither rewrites the findings nor generates skill content, preserving the distinction between evidence, ownership, and the subsequent update.

Fi .

i=1

(6) Each candidate carries exact trace-line provenance. The analyzer separates observed evidence from reusable generalization; it does not reconstruct unavailable ground-truth answers. The output F contains candidate semantic gradients; the next stage determines which are worth retaining and what function each should serve. G.2

1 ≤ |ρ(f )| ≤ 2,

G.3

Retention and Attribution Rules

Component Maintenance and Validation

Once ownership is established, each component evolves through its own operator:   (t+1) (t) Bbk = Uk Bk , Fk , k ∈ K. (8)

Consolidation maps the semantic gradients F to component-specific update evidence {Fk }k∈K . The distinction between knowledge structures and control processes motivates separating a finding’s content from its execution role (Atkinson and Shiffrin, 1968). Evidence for dissociable forms of knowing further motivates specialized maintenance (Cohen and Squire, 1980). CDSE instantiates these principles through a retention gate followed by functional role attribution.

Each Uk produces a component-level candidate. The full operator U subsequently coordinates and validates these candidates before returning the next skill state. Specialized maintenance followed by integration reflects the cognitive principle that differentiated functions can remain coordinated (Miyake et al., 2000).

16

Meta and Experience: consolidating behavioral rules. Both components store ordered atomic rules with stable identifiers. A finding aligns with at most two existing rules as reinforcing, extending, or correcting the same knowledge unit; it is split only into independently actionable parts. Evidence for one rule is aggregated into one revision, while unaffected rules are preserved. Unaligned findings with compatible triggers, actions, scope, and constraints induce new rules, with proposals reconciled across batches. When capacity is exceeded, the updater shortens rules, merges genuine redundancy, and finally removes low-value rules. Meta applies this process to task-level control, whereas Experience requires a local activation condition. Thus, shared rule machinery maintains distinct behavioral responsibilities.

A complete Meta, Experience, or Tool duplicate can be removed when its meaning survives in the appropriate component; partial overlap is clarified. The coordinator cannot introduce, renumber, or reorder rules, change Tool category membership, or alter Script entries or executables. A validator compares the candidate with its input snapshot and confines changes to the four component specifications, preserving the previously verified programs. Adaptive invocation and publication. A description rewriter D derives each component’s scope, timing, and expected benefit from the coordinated content. Only the descriptions visible before component loading may change; a validator requires well-formed, distinct descriptions and byte-identical remaining content and artifacts. Successful coordination and description validation produce B (t+1) = D(C(Bb(t+1) )); U is a partial operator, undefined when either stage fails validation, in which case no updated state is published. These checks enforce edit contracts, while task performance is assessed separately in our experiments. For execution history hℓ at step ℓ, write the agent’s invocation choice as   (t+1) Iℓ = Selectπθ hℓ , {dk }k∈K ⊆ K, (9)

Tool: maintaining capability-grounded knowledge. Tool organizes operational rules within stable capability categories. Findings first align with an existing tool or category; unmatched category proposals are reconciled across batches. Rule alignment, revision, and induction then occur within each category. Local capacity control prevents frequently encountered tools from displacing knowledge about other capabilities. Script: turning suitable feedback into verified procedures. Script findings first align with existing executable components. Residual proposals pass a worthiness gate requiring repeatable deterministic behavior, explicit I/O and error contracts, available inputs, feasible synthetic tests, and material savings over one-off tool use. Compatible proposals are grouped; others are deferred. Each work item selects a documentation update, local modification, replacement, creation, or deferral. Implementations stay within allocated paths, declare dependencies, and pass self-tests. An independent verifier then checks the assembled Script component in a clean environment using component and full-suite tests. One bounded repair is allowed; unverified code is withheld. This branch gives procedural changes an executable acceptance contract.

(t+1)

where dk is the revised description of component k and Iℓ contains the components selected for loading at that step. Selection uses the agent’s ordinary invocation mechanism. As the task progresses, different components become relevant; their use changes agent behavior and supplies execution evidence for subsequent skill evolution.

H

Experimental Setup

H.1

Benchmarks and Data Splits

Table 17 lists the task families and dataset selections used in our evaluation. We split each resulting benchmark collection into training and test sets at a 2:7 ratio. The training split supplies the trajectories used for skill evolution, and all reported results are computed on the test split. H.2

Cross-component coordination. Component updates produce a candidate skill state Bb(t+1) . A coordinator C reads all four components and makes minimal edits to resolve concrete conflicts, duplication, ownership ambiguity, and documentation mismatches. Script artifacts remain authoritative.

Common Evaluation Protocol

We test GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3 under both Claude Code and Codex. Within each model–harness block, all methods receive access to the same underlying training split, task instances, tool access, and context limit. Each

17

Benchmark

Task family

Dataset selection

OfficeQA

Document-grounded question answering Spreadsheet manipulation Long-context evidence synthesis Mathematical reasoning Repository-level software engineering Tool-based business workflows

Easy and Hard subsets

SpreadsheetBench2 SealQA LiveMathBench SWE-Pro AutomationBench

Financial_Model and Template subsets Complete dataset 202605 and 202606 subsets Three Python subsets: pandas, scikit-learn, and xarray All instances from the Sales and Marketing domains

Table 17: Task families and dataset selections. Each selected collection is split into training and test sets at a 2:7 ratio.

question is subject to benchmark-specific maximum runtime and turn limits. If execution reaches either limit without completing the task, the outcome is counted as a failure. Base is the same agent without an evolved skill state. CDSE starts from the same skill initialization and updates one unified state whose four functional components are selectively loaded at inference time. Table 1 reports the state after three evolution rounds for each skill-evolution method. We apply the benchmark-specific scoring procedures described below. Reported differences are percentage-point changes within a benchmark. For aggregate statements, we macro-average these changes over the relevant model–harness– benchmark cells; we do not average the underlying heterogeneous task metrics. H.3

containing multiple numbers, every number must match. Integers from 1900 to 2100 are treated as year candidates and excluded from numeric matching unless the reference itself is a year. Purely textual answers use normalized substring or exact matching. Empty answers receive zero and count as failures in the reported success rate. Strict success requires s ≥ 0.8; the continuous mean score is an auxiliary diagnostic. In the Codex evaluation, timeouts and crashes may be retried at most twice. In Claude Code, reaching the 60-turn limit is counted as a failure without retry. SpreadsheetBench2. Only the hidden answer regions of the workbook are scored. Comparing the input workbook with the reference partitions these regions into unchanged (regression) cells and cells requiring modification. The output is evaluated separately on both groups. A task succeeds only when both accuracies are 1.0, after regression accuracy of at least 0.998 is rounded to 1.0; no partial credit contributes to the reported task success rate. Numeric cells allow 1% relative error, with absolute tolerance 0.01 when either value is zero. Numeric strings are parsed and rounded to two decimal places, and dates are converted to Excel serial values. Formula comparisons ignore case and absolute-reference markers. Array formulas use their formula text; recognized financial placeholders are normalized, as are blank cells and zero where specified by the evaluator. Excel error values trigger a fallback to formula-text comparison. Missing outputs fail the task; missing sheets or corrupted files invalidate the affected scoring regions.

Scoring Rules and Failure Handling

Evaluation depends on the task output: OfficeQA and SealQA score final answers, SpreadsheetBench2 scores generated workbooks, LiveMathBench scores selected answer labels, SWE-Pro scores executable test outcomes, and AutomationBench scores the final environment state. The following rules specify the reported success rates and distinguish them from auxiliary partial-credit scores. OfficeQA. Predicted and reference answers undergo numeric extraction and normalization, including magnitude units, signs, and thousands separators. Numeric answers are compared at relativeerror tolerances T = {0, 0.01, 0.025, 0.05, 0.1}, yielding P wt 1[match at t] 1 s = t∈T P , wt = . 1 + 20t t∈T wt

SealQA. A shared offline judge labels each answer Correct, Incorrect, or Not Attempted; only Correct receives one point. The answer must contain the reference’s essential information without contradictory claims or internal inconsistency. Differences in wording, punctuation, capitalization,

A match also requires the reference’s nonnumeric text to occur in the prediction. For a reference 18

and contextually redundant information are ignored. Uncertain phrasing is allowed when the required information is complete and consistent; an incomplete or unattempted answer receives zero. All evaluated models, harnesses, and methods use the same few-shot judging template and the deepseek-v4-flash-huawei judge with temperature 0 and a 2,048-token output limit. Judging is performed after trajectories are saved.

benchmark prompts, Friday-search server JSON, or arm-specific tool lists. OfficeQA The question is passed through -p, and –add-dir exposes the document corpus. The analyst system prompt is appended to the agent context. The enabled tools are Read, Glob, Grep, and Bash, with a 60-turn limit. The command explicitly selects GLM-5.2 and records safe mode with the acceptEdits permission mode. Verbose streaming JSON and the supplied schema require stringvalued final_answer and reasoning fields.

LiveMathBench. Our multiple-choice evaluation uses a deterministic scorer. It extracts the last ANSWER: declaration containing a letter from A to E and reverses the deterministic option shuffle before comparing with the reference label. Only the correct option receives one point. The designated weaker-true option, other incorrect options, missing answer declarations, exhausted turn budgets, and unresolved execution errors all receive zero. No language-model judge is used.

claude \ -p "<question>" \ --model glm-5.2 \ --safe-mode \ --output-format stream-json --verbose \ --json-schema '{"type":"object","properties":{"final_answer ⌋ ,→ ":{"type":"string"},"reasoning":{"type":"string"}},"req ⌋ ,→ uired":["final_answer","reasoning"]}' \ --allowed-tools Read,Glob,Grep,Bash \ --add-dir <document corpus> \ --append-system-prompt "<analyst system prompt>" \ --permission-mode acceptEdits \ --max-turns 60

SWE-Pro. An instance is resolved only if all tests in both the FAIL_TO_PASS and PASS_TO_PASS sets pass after the agent’s changes. This binary criterion requires the intended fix while preserving previously passing behavior; partial test coverage does not count as task success.

SpreadsheetBench2 The task prompt is passed through -p, and the system prompt is loaded from a file. The tool set comprises Bash, Read, Write, Edit, Glob, Grep, and Skill; Agent, WebFetch, and WebSearch are explicitly disabled. Evaluation uses a 60-turn limit, bypassPermissions, and verbose JSON output. This launch template does not include an explicit –model argument.

AutomationBench. Rubric assertions evaluate the final world state rather than the agent’s answer text. The reported metric is binary task completion: every scored assertion must pass. The fraction of passing assertions provides auxiliary partial credit. Assertions already satisfied initially and still satisfied at the end are excluded to avoid rewarding inaction; initially satisfied assertions that the agent breaks remain failures. Tasks whose correct behavior is inaction can explicitly retain their negative assertions. Negative assertions check that prohibited actions do not occur, and checks marked as unscored or explicitly excluded do not contribute to the metric. H.4

claude \ --max-turns 60 \ --permission-mode bypassPermissions \ --tools "Bash Read Write Edit Glob Grep Skill" \ --disallowed-tools "Agent WebFetch WebSearch" \ --append-system-prompt-file <system_prompt.md> \ --output-format json --verbose \ -p "<task prompt>"

SealQA The question and evaluated model are supplied explicitly. Search is provided by the Friday-search MCP server configuration; –strict-mcp-config restricts MCP loading to that configuration. The allowed-tool list is supplied for the corresponding experimental arm, while WebSearch and WebFetch are disabled. The system prompt may include the indicated nudge. The run allows 80 turns and uses bypassPermissions, verbose streaming JSON, and the same two-field answer schema as OfficeQA.

Claude Code Invocation Parameters

The following launch templates specify the benchmark-dependent Claude Code configuration used in our evaluation. Angle-bracketed entries denote the question or task prompt, evaluated model, corpus path, prompt file, or MCP configuration supplied for each run. The templates preserve the tool and permission flags used in the experiments. The answer schema is shown in full where specified; the source configuration does not provide the literal

claude \ --model "<model>" \ --verbose \ -p "<question>" \ --output-format stream-json \ --json-schema '{"type":"object","properties":{"final_answer ⌋ ,→ ":{"type":"string"},"reasoning":{"type":"string"}},"req ⌋ ,→ uired":["final_answer","reasoning"]}' \ --strict-mcp-config \ --mcp-config '<friday-search server JSON>' \

19

Git repository check, request JSON output, and set model_context_window=1000000. We reproduce the supplied flags without inferring settings omitted from the templates.

--permission-mode bypassPermissions \ --allowed-tools "<arm-specific list>" \ --disallowed-tools "WebSearch WebFetch" \ --append-system-prompt "<SYSTEM_PROMPT [+ nudge]>" \ --max-turns 80

LiveMathBench The question prompt, evaluated model, and system prompt are passed directly. Bash, Write, and Edit are enabled, and the turn limit is 15. The recorded command includes acceptEdits, –dangerously-skip-permissions, and –safe-mode together, with verbose streaming JSON output. These flags are reproduced as configured in the experiment.

OfficeQA The command selects GLM-5.2, uses the workspace-write sandbox, disables web search, and supplies an OpenAI-compatible gateway URL. The system prompt is supplied through developer_instructions. codex exec --skip-git-repo-check --json \ -c model="glm-5.2" \ -s workspace-write \ -c model_context_window=1000000 \ -c tools.web_search=false \ -c model_providers.codex.base_url="<OpenAI-compatible ,→ gateway URL>" \ -c developer_instructions="<system prompt>"

claude -p <question_prompt> \ --output-format stream-json --verbose \ --model <model> \ --append-system-prompt <SYSTEM_PROMPT> \ --allowed-tools "Bash Write Edit" \ --permission-mode acceptEdits \ --dangerously-skip-permissions \ --safe-mode \ --max-turns 15

SpreadsheetBench2 The evaluated model is supplied explicitly. The command uses workspace-write, disables web search, and loads developer_instructions from system_prompt.md.

SWE-Pro The repository-level task prompt is supplied through -p. Bash, Read, Grep, Glob, Edit, and Write support repository inspection and modification. The command uses safe mode, bypassPermissions, verbose streaming JSON, and a 250-turn limit. As in SpreadsheetBench2, the supplied launch template omits an explicit –model argument.

codex exec \ --skip-git-repo-check \ --json \ -s workspace-write \ -c model="<model>" \ -c model_context_window=1000000 \ -c tools.web_search=false \ -c developer_instructions="$(cat system_prompt.md)"

SealQA The evaluated model and gateway URL are supplied explicitly. Unlike the OfficeQA and SpreadsheetBench2 templates, this command enables web search. It uses workspace-write and passes the system prompt through developer_instructions.

claude -p "<task prompt>" \ --safe-mode \ --permission-mode bypassPermissions \ --output-format stream-json \ --verbose \ --max-turns 250 \ --allowed-tools "Bash Read Grep Glob Edit Write"

AutomationBench The user prompt is supplied through –print, with an explicit evaluated model. A per-task MCP configuration specifies the task tools, and strict MCP loading is enabled. The system prompt is read from a file. The builtin tool set and allowed-tool list both specify Skill and Bash. Evaluation uses a 60-turn limit, bypassPermissions, and verbose JSON output.

codex exec \ --skip-git-repo-check \ --json \ -s workspace-write \ -c model="<model>" \ -c model_providers.codex.base_url="<gateway URL>" \ -c model_context_window=1000000 \ -c tools.web_search=true \ -c developer_instructions="<SYSTEM_PROMPT>"

LiveMathBench The evaluated model is selected through -m. The command uses workspace-write, disables web search, and supplies the system prompt through developer_instructions.

claude --print <user_prompt> \ --model <model> \ --mcp-config <per-task.mcp.json> --strict-mcp-config \ --append-system-prompt-file <system_prompt.txt> \ --output-format json --verbose \ --max-turns 60 \ --permission-mode bypassPermissions \ --tools Skill,Bash --allowed-tools Skill,Bash

H.5

codex exec --skip-git-repo-check --json \ -s workspace-write \ -m <MODEL> \ -c model_context_window=1000000 \ -c tools.web_search=false \ -c developer_instructions="<SYSTEM_PROMPT>"

Codex Invocation Parameters

The following launch templates list the recorded Codex configuration flags for the same six benchmarks. Angle-bracketed entries denote the evaluated model, system prompt, gateway URL, or pertask paths. All templates use codex exec, skip the

SWE-Pro The command supplies the evaluated model and system prompt and uses workspace-write. The recorded template does not include an explicit web-search setting. 20

codex exec --skip-git-repo-check --json \ -s workspace-write \ -c model="<model>" \ -c model_context_window=1000000 \ -c developer_instructions="<system prompt>"

## Targets - `meta`: task-level control knowledge, including priorities, evidence-access boundaries, preservation of user constraints, phase transitions, search or turn budgets, stopping and finalization, resolution of known contradictions, and prevention of answer leakage or unsupported answers. - `experience`: context-dependent heuristics, anti-patterns, verification practices, interpretation risks, and recovery tied to a particular data shape, source situation, or calculation context. - `tool`: actionable knowledge about using an existing named environment tool, including selection, sequencing, output interpretation, permissions, limitations, misuse, inefficient repetition, and recovery. - `script`: a proposed reusable deterministic operation with defined inputs and outputs that can be implemented and automatically tested.

AutomationBench This template selects GLM5.2 and uses the danger-full-access sandbox. The ab MCP server is launched with the specified Python interpreter, server script, task file, and final-world-state output file. The system prompt is supplied through developer_instructions. codex exec \ --skip-git-repo-check \ --json \ -s danger-full-access \ -m glm-5.2 \ -c model_context_window=1000000 \ -c 'mcp_servers.ab.command="<abs-path-to-venv-python>"' \ -c 'mcp_servers.ab.args=["<abs>/ab_mcp_server.py","--task-f ⌋ ,→ ile","<abs>/task.json","--output-file","<abs>/world_fin ⌋ ,→ al.json"]' \ -c developer_instructions='"<system prompt>"'

H.6

Baseline Configurations

## Decision order

We follow the parameter settings recommended by the original methods unless specified otherwise. The configurations used in our comparison are as follows.

Evaluate each finding in this order: 1. Does it control whether the task may proceed, return to an earlier phase, stop, or finalize? Consider `meta`. 2. Does it give actionable guidance about an existing named environment tool? Consider `tool`. 3. Does it propose a new deterministic executable capability? Consider `script`. 4. Is its remaining value a local heuristic, anti-pattern, verification practice, or recovery? Consider `experience`.

Trace2Skill. For Trace2Skill (Ni et al., 2026), we combine the Error and Success analysis branches and merge all generated updates. EvoSkill. For EvoSkill (Alzubi et al., 2026), we run 1.5 epochs with a frontier size of 3 and construct updates only from erroneous examples. For benchmarks without binary outcomes, we identify errors using the corresponding task-score threshold.

This order guides evaluation but does not restrict target combinations. ## Boundaries Route by reusable function, not by words that merely appear in the finding.

SkillOpt-lite. For SkillOpt-lite (Yang et al., 2026b), we run 10 optimization rounds with a batch size of 10.

I

For `meta`, judge semantic scope rather than observed frequency. A task-level rule may be routed to `meta` from one trajectory; cross-trajectory frequency and final admission are evaluated later.

System Prompt for Gradient Consolidation

Do not use `experience` as the default for every useful lesson. Use it for knowledge that remains local after checking `meta`, `tool`, and `script`.

This appendix reproduces the system prompt for Cognition-Guided Gradient Consolidation, the second stage of CDSE (Section 3.3). The remaining prompts and implementation code are provided in the Software supplementary material.

For `tool`, require an explicit named tool and actionable guidance about how it should or should not be used. Bash, Read, Write, Edit, Glob, Grep, and other named environment tools count. Merely using or mentioning a tool is insufficient. Documents, datasets, source tables, file formats, benchmarks, and domain-specific data sources are not tools.

You are the routing classifier in a four-skill evolution pipeline. You receive exactly one completed trajectory-analysis JSON. The original trajectory, other analyses, existing skills, and external resources are unavailable and must not be reconstructed. Classify only the supplied findings; do not rewrite, merge, split, summarize, or create findings.

21

Prefer `tool` when the central rule changes if the named tool is replaced; prefer `experience` when the rule remains fully tool-independent. A concrete named-tool sequence, usage boundary, output-handling rule, or recovery has meaningful `tool` value even when it also supports a broader verification or efficiency practice. Use `tool` for guidance about an existing capability and `script` for a new maintained executable capability. A shell snippet or generic calculation is not a tool finding merely because Bash can run it. For `script`, require a deterministic and testable operation. Do not route semantic judgment, source-authority selection, business-definition choice, general planning, or “find the correct answer” capabilities to `script`. ## Keep or drop Drop findings that are generic, unsupported, purely sample-specific, or unlikely to change future behavior. Apply the core-specific redundancy rules below. A valid success may contribute positive behavior. A fragile success should contribute only supported risks or improvements. An invalid success may contribute an anti-pattern or recovery, but not a positive pattern. Drop unknown-success claims whose causal basis is unverified. Drop a `success_analysis` core when its reusable behavior, risk, or improvement is already captured by `reusable_lessons`. A `failure_analysis` core may remain useful because it combines cause and recovery. Keep it when its failure mechanism, task-control implication, recovery, or routing value adds material information not fully preserved by the lessons. Drop it as redundant only when both its reusable cause and recovery are fully covered; partial overlap is insufficient. The drop reason must state that both are covered. Do not keep sample-specific detail alone. ## Output behavior A kept finding must have one or at most two ordered targets. The first is primary. Use two targets when a finding has both a concrete tool-specific rule and a meaningful tool-independent lesson; order them by primary value. Otherwise add a secondary target only when it has substantial independent value. Do not generate skill content or reason about downstream wording. Keep each routing reason to one concise sentence. Return only JSON matching the supplied schema.

22

Related documents

• ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL#792479
• ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL#788479
• Knowing the User Is Not Knowing What We Established: Grounding Awareness in Personal Agents#1344182
• Knowing the User Is Not Knowing What We Established: Grounding Awareness in Personal Agents#1319031
• Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents#1342966