Preprint.
SoftSkill: Behavioral Compression for Contextual Adaptation Xijia Tao♠,*
Yihua Teng♡,*
Yuzhi Zhao♢
Xinyu Fu♡,*
Suiyun Zhang♡
♠ The University of Hong Kong
Ziru Liu♡
Rui Liu♡,†
♡ Huawei Research
Kecheng Chen♣
Lingpeng Kong♠,† ♣ City University of Hong Kong
♢ Huazhong University of Science and Technology * Equal contribution
† Corresponding authors
arXiv:2606.20333v1 [cs.AI] 18 Jun 2026
Abstract Agent skills are commonly deployed as natural-language Markdown files that encode answer policies, evidence-use habits, and task procedures. These files are readable and portable, but they are consumed indirectly: for each task instance, a frozen language model must translate a long textual artifact into generation-time behavior. This paper asks whether a naturallanguage skill can instead initialize a compact continuous context object, refined by a trainable soft delta while the base model remains frozen. We propose S OFT S KILL, a frozen-backbone method that tunes such soft skills with next-token prediction and deploys them as latent behavioral priors at inference time. In our main single-round setting, a length-32 S OFT S KILL prefix on Qwen3.5–4B improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA. Relative to SkillOpt, S OFT S KILL improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath, while replacing hundreds to thousands of Markdown skill tokens with a few virtual tokens. We further study agentic execution as a harder boundary case, where sparse trajectory imitation provides useful signal but does not yet robustly compress long-horizon procedural behavior. Code: https://github.com/xijia-tao/SoftSkill
1
Introduction
Large language models increasingly act through execution harnesses that expose files, tools, verifiers, browser state, and long-horizon interaction traces. In these settings, adaptation is not only a matter of asking the model the right question. A useful agent must know when to inspect evidence, how to call tools, how to verify intermediate outputs, and which domain-specific failure modes to avoid. Recent agent-learning and “skill” systems make this adaptation explicit by packaging reusable procedural knowledge into natural-language memories, executable skills, or Markdown files that are loaded at inference time (Shinn et al., 2023; Zhao et al., 2024; Wang et al., 2023; Zhang et al., 2025; Yang et al., 2026). Textual skills have an important advantage: they are readable, editable, and easy to route through progressive disclosure (Zhang et al., 2025). They also have an important limitation. The text is not the behavior itself. At inference time, the model must repeatedly internalize the skill, decide which parts matter for the current instance, and translate those instructions into hidden-state dynamics that shape token generation. A skill may be semantically correct yet inefficient, poorly aligned with the model’s internal control directions, or verbose enough to consume context and output budget. This paper frames skill adaptation as compact contextual adaptation, or behavioral compression in the limited sense that a learned prefix can replace a longer skill artifact at inference time. Rather than optimizing only the external Markdown artifact, we ask whether tasksuccessful answer behavior can be distilled into a compact continuous prefix that remains in the model context. In the main QA setting, the object being compressed is answer style, evidence reliance, and direct final-answer behavior; in the agentic stress test, we additionally probe whether trajectory patterns from successful tool-use examples can be internalized. A 1
Preprint.
1 A
Skill Representations
No Skill
NTP Training / Selection Loop
2
1
Task prompt
Frozen LM (frozen)
Collect successful trajectories
2
Skill document
3
Initialize prefix p₀ with NL skill embedding
SkillOpt (hard text)
## Skill • Extract evidence • Verify against context • Plan before acting • ...
4
Task prompt
Minimize next-token prediction loss ℒₙₜₚ = -Σ log Pθ(xₜ | x<t, p₀+Δp)
Frozen LM (frozen)
Optimize soft delta only p = p₀ + Δp on successful trajectories
Interpretable and editable, but long (300–2000+ tokens) C
+ Δp (learned)
SoftSkill-NTP (main) Skill document (Markdown)
tune + p = p₀ + Δp
Inference-time Usage
Replace skill section with soft skill SkillOpt (hard text)
SoftSkill (ours)
System prompt
No prior behavior B
3
System prompt replace with
## Skill •…
< soft prefix p >
Task request
Task request
Frozen LM (acts as agent)
Frozen LM (acts as agent)
Example: ALFWorld init
Task prompt
Backbone frozen
p₀
Obs₀
Pass -> Continue
Validation gate (optional)
Fail -> Revert to previous best checkpoint
Frozen LM (frozen)
Readable skill initializes metadata; compact tuned prefix is used at inference
5
▧ Export tuned soft skill with best validation accuracy
You are in the kitchen. A mug is on the counter.
Thought₁
I should pick up the mug.
Action₁
go to counter
Obs₁
You are at the counter.
Thought₂
Now I can take the mug.
Action₂
take mug
Obs (env)
Thought (LM)
Action (LM)
Figure 1: S OFT S KILL initializes a compact soft prefix from skill text, tunes only the soft delta with next-token prediction, and selects the deployed checkpoint by held-out task validation.
soft skill is therefore not a reward model in the usual inference-time sense–it does not score candidate outputs–but a latent behavioral prior that biases generation toward actions and answers that previously received supervision. SkillOpt (Yang et al., 2026) is the closest text-space point of departure. It treats a skill file as an external trainable state for a frozen agent: a separate optimizer model observes scored rollouts, proposes bounded textual edits, and accepts an edited skill only when it improves held-out validation performance. We use SkillOpt as a strong baseline and as a source of skill metadata or initialization in the agentic setting. However, S OFT S KILL is not primarily a continuous version of SkillOpt. SkillOpt optimizes what the agent reads; S OFT S KILL optimizes a small part of the conditioning state through which the frozen model enacts behavior. This distinction matters because a natural-language skill that works well for one model may not be equally effective for another model with different scale, pretraining data, tokenizer, or post-training behavior. The training step is therefore not only a cost of the method: it is the mechanism by which a readable skill is internalized into a model-specific behavioral prior. We propose S OFT S KILL, a training procedure for learning a soft prefix embedding for contextual agent adaptation. Given a natural-language skill document, we first map the skill into an initial embedding prefix, then tune only a soft delta to this prefix from ground-truth answers or logged successful trajectories while keeping the target model weights frozen. Our main empirical setting is single-round QA, where the target behavior is answer style and direct final-answer generation. We additionally study agentic execution as a harder regime in which the prefix must encode procedural tool-use behavior. We report these regimes separately because their supervision, failure modes, and empirical strength differ. The central empirical questions are: 1. Can a short learned prefix replace a long textual skill while preserving or improving held-out success? 2. How much of the gain comes from the learned delta rather than the text initialization alone? 3. Where is S OFT S KILL useful compared with hard skills, mean-pooled prefixes, and LoRA? 2
Preprint.
Our contribution is a study of compact latent behavioral priors for frozen-backbone QA. The evidence is strongest on SearchQA and LiveMath, where a 32-token S OFT S KILL prefix exceeds SkillOpt with Qwen3.5–4B while replacing hundreds or thousands of Markdown skill tokens. DocVQA provides a supported vision-QA prompt-start result that remains close to SkillOpt, but we do not claim a DocVQA placement ablation because the vision implementation does not insert prefixes at the skill-section slot. LoRA remains a strong baseline, especially on SearchQA, so the claim is not that soft skills uniformly dominate adapter tuning; rather, they provide a compact contextual deployment path when no adapter-serving path is desired. Agentic tasks are treated as a separate stress test for trajectory source, prefix length, and validation-selected prefix checkpoints rather than as the main headline result. The method preserves some operational advantages of skill systems– reusable artifacts, validation-guided selection, and optional human-readable metadata– while testing whether low-dimensional continuous context can distill refinements that are costly to express, parse, or regenerate as Markdown.
2
Method
2.1
Problem Setup
Let f be a frozen language model used inside an execution harness H. A task instance x ∈ D can produce an interaction trajectory τ = H( f , x, s, p) and a scalar task score R(τ ), which we use for validation and test evaluation. The hard skill s is a Markdown document inserted into the agent context. The soft skill is a sequence of m virtual token embeddings with hidden dimension d, decomposed as p = p0 + ∆p, where p0 is a text-derived initialization and only the soft delta ∆p is trainable. In the main experiments, training is supervised next-token prediction over answers or successful trajectories: " # T
∆p⋆ = arg min E( x,y)∼Dtrain − ∑ log p f (yt | y<t , x, s, p0 + ∆p) , ∆p
(1)
t =1
subject to frozen model weights. Held-out task reward is not optimized online in the main method; it is used to select the deployed checkpoint and to report validation and test performance. 2.2
Soft Skills as Behavioral Compression
The prefix is trained to summarize behavior that would otherwise be expressed through long instructions, demonstrations, or generated rationales. In the main QA setting, a naturallanguage skill document initializes the prefix and next-token prediction tunes a soft delta using direct answer targets. In the agentic stress test, the same NTP objective is applied to successful trajectories. Reward-based refinement is a possible extension for agentic tasks, but it is not required for the core claim. The learned object is a small contextual control vector: it remains present during generation and biases the frozen model toward behaviors that were useful during supervised training. This view separates S OFT S KILL from an inference-time reward model. A reward model evaluates candidate behavior; a soft skill preconditions behavior before tokens are sampled. It also separates S OFT S KILL from ordinary prompt tuning. The prefix is attached to a reusable skill or task family, selected by held-out execution performance, and evaluated through task accuracy and deployment-cost diagnostics. 2.3
Initialization and Skill Metadata
The first design choice is initialization. An unstructured non-text prefix may require many trajectories before it expresses useful procedural knowledge. We therefore initialize pθ from 3
Preprint.
a textual skill in the main setting. Given a tokenizer and embedding matrix E, we encode the skill document and derive m initial vectors by one of the following mechanisms: 1. Mean-pooled initialization: initialize the virtual tokens from a mean embedding derived from the model vocabulary or from spans of the skill document. This condition distinguishes non-text starting points from natural-language or SkillOptartifact initialization. 2. Natural-language initialization: initialize from a human-written or optimized Markdown skill by embedding its token sequence and using the resulting vectors, or pooled spans of those vectors, as the starting prefix. This initialization may be directly useful when the text already matches the target model’s instructionfollowing behavior, but it can also serve mainly as a structured starting point for the learned soft delta. Natural-language metadata remains useful even when the deployed behavior is continuous. In a progressive-disclosure skill system, the readable metadata can be used for retrieval, routing, and audit, while the retrieved skill’s learned prefix is injected into the model context. This preserves the operational framing of skills: S OFT S KILL does not require every skill to be visible as full Markdown at inference time, but it can still keep human-readable descriptions for selection and governance. 2.4
Training Objectives
For tasks with supervised targets or successful trajectories, we train the soft delta by nexttoken prediction while keeping the backbone language model frozen. Given an input instance x, an optional hard skill s, and a target sequence y1:T , the loss is T
LNTP (∆p) = − ∑ log p f (yt | y<t , x, s, p0 + ∆p).
(2)
t =1
The target sequence depends on the regime. For single-round QA tasks, y1:T is the groundtruth answer sequence, and the learned prefix mainly captures answer-formatting and task-specific response behavior. For agentic execution tasks, y1:T can be extracted from successful trajectories and may include actions, tool calls, rationales, or final answers, depending on the harness. This objective is simple and stable, and it is the training objective used in the main experiments. However, lower next-token loss does not necessarily imply higher task success: the prefix may imitate surface patterns, overfit scarce trajectories, or learn formatting behavior that does not improve execution. We therefore use task-level validation for checkpoint selection. 2.5
Validation-Selected Prefix Checkpoints
We select the deployed prefix checkpoint using held-out task performance rather than training loss. During prefix training, saved checkpoints are evaluated on a validation split with the task-level metric used for model selection, and the checkpoint with the best validation score is exported for test evaluation. This differs from SkillOpt’s validation gate, which accepts or rejects each proposed textual edit online. In the NTP setting, validation does not rewind training after an unsuccessful epoch; it only selects the final prefix checkpoint. A stricter rewind-based gate is a possible extension, but it is not part of the main experiments.
3
Experiments
3.1
Benchmarks and Models
We evaluate six low-data benchmarks from the SkillOpt suite (Yang et al., 2026), covering search-style QA, math problem solving, document question answering, local document 4
Preprint.
retrieval, spreadsheet manipulation, and embodied decision making. Comparisons with SkillOpt reuse its split discipline, released harnesses, and held-out test reporting. The primary model setting uses open-weight Qwen-family models (Yang et al., 2025) where embedding prefixes can be optimized while all backbone weights remain frozen. We do not evaluate proprietary target models. For comparison with SkillOpt, we report the openmodel rows from Yang et al. (2026) and use the same task names: SearchQA (Dunn et al., 2017), LiveMath (He et al., 2026), DocVQA (Mathew et al., 2021), OfficeQA (Databricks, 2025), SpreadsheetBench (Ma et al., 2024), and ALFWorld (Shridhar et al., 2021). We divide the suite into two regimes. SearchQA, LiveMath, and DocVQA are single-round QA or answer-generation tasks. For these tasks, the NTP response is the ground-truth answer without chain-of-thought, and S OFT S KILL is evaluated as a compact replacement for the skill context under a tight generation budget. SearchQA and LiveMath support both prompt start and skill section insertion. DocVQA uses a vision-language path that currently supports only prompt start; the implementation caveat is detailed in Appendix B.1. OfficeQA, Spreadsheet, and ALFWorld are agentic execution tasks with tool calls, environment interaction, or multi-step policy structure. For these tasks, S OFT S KILL is initialized from a SkillOpt artifact and trained on successful trajectories. The default agentic setting uses GPT-5.5 as a trajectory generator, not as the target model: we run the SkillOpt codebase to collect candidate rollouts, keep only trajectories that satisfy the task harness success criterion, remove CoT from the supervised target when constructing the NTP data, and train/evaluate the soft prefix on the open-weight Qwen target model. The target-model rollout setting, where available, instead uses the target model’s own CoT trajectories. Because rollouts do not always succeed, usable training trajectories are smaller than the nominal train splits: 31/50 for OfficeQA, 61/80 for Spreadsheet, and 31/39 for ALFWorld. These benchmarks are intentionally scarce in training signal: across all six tasks there are only 711 nominal training instances, four tasks have at most 107 training instances, and ALFWorld and LiveMath have fewer than 40. This scarcity makes validation-selected lowdata adaptation central to the experimental design rather than an incidental constraint. Full split sizes and supervision details are in Appendix Tables 8 and 9. 3.2
Metrics
Primary metrics are task accuracy, validation-selected test performance, and improvement over the relevant no-skill or SkillOpt hard-skill baseline. We also report context tokens, average generated tokens, prefix length, train-data scaling, checkpoint-selection epochs, and loss-versus-validation behavior. Robustness under perturbation, transfer gaps, tool-call statistics, and trajectory-level error categories are important future diagnostics, but they are not used as main-paper evidence here. 3.3
Baselines
The full baseline taxonomy is in Appendix Table 7. In the main results, we emphasize the baselines that determine the claim boundary: no skill tests whether any task-specific context is useful, SkillOpt tests whether a long optimized Markdown artifact is still stronger than a compact prefix, and LoRA tests whether conventional parameter-efficient tuning is a better use of the same supervised signal. For initialization ablations, “NL skill” means the prefix is initialized from a human-written natural-language skill, “SkillOpt artifact” means it is initialized from the optimized Markdown artifact, and “Mean-pooled” denotes the non-text initialization described in Section 2.
4
Main Results
Single-round tasks. Table 1 reports the primary single-round comparison on SearchQA, LiveMath, and DocVQA using Qwen3.5–4B. The main S OFT S KILL setting initializes a soft 5
Preprint.
Table 1: Main single-round QA comparison on Qwen3.5–4B. The main S OFT S KILL rows report mean ± standard deviation over three seeds. Method
SearchQA
LiveMath
DocVQA
No skill Human skill LLM skill Trace2Skill (Ni et al., 2026) TextGrad (Yuksekgonul et al., 2024) GEPA (Agrawal et al., 2025) SkillOpt (Yang et al., 2026)
68.1 (+0.0) 66.3 (-1.8) 65.0 (-3.1) 68.5 (+0.4) 60.7 (-7.4) 68.6 (+0.5) 71.2 (+3.1)
22.4 (+0.0) 18.4 (-4.0) 28.8 (+6.4) 27.2 (+4.8) 10.6 (-11.8) 28.8 (+6.4) 52.0 (+29.6)
86.9 (+0.0) 87.8 (+0.9) 88.0 (+1.1) 88.0 (+1.1) 85.6 (-1.3) 85.1 (-1.8) 89.0 (+2.1)
78.6 (+10.5) 76.4 ± 0.0 (+8.3) 76.4 ± 0.0 (+8.3)
58.9 (+36.5) 50.0 ± 9.1 (+27.6) 64.5 ± 2.4 (+42.1)
87.7 (+0.8) 88.2 ± 1.4 (+1.3) –
70.5 (+2.4) 72.6 (+4.5) 72.6 (+4.5)
0.8 (-21.6) 59.7 (+37.3) 42.7 (+20.3)
0.0 (-86.9) 89.6 (+2.7) –
LoRA (Hu et al., 2021) S OFT S KILL (prompt start) S OFT S KILL (skill section) S OFT S KILL-mean-init-only (prompt start) S OFT S KILL-mean (prompt start) S OFT S KILL-mean (skill section)
prefix from a natural-language skill document and then tunes only a soft delta with nexttoken prediction, keeping the base model frozen. We evaluate two insertion placements where supported: prompt start, which places the tuned soft skill before the system prompt, and skill section, which inserts it in the slot normally occupied by the SkillOpt Markdown skill. For DocVQA, the vision-language path currently supports only prompt start; skill section cells are therefore omitted rather than treated as failed comparisons. The SkillOpt-related text-skill baselines are transcribed from the open-model setting of Yang et al. (2026). Across the three single-round tasks, S OFT S KILL improves over the no-skill baseline and is competitive with, or better than, hard skill optimization. On SearchQA, both S OFT S KILL placements reach 76.4, an 8.3 point gain over no skill and a 5.2 point gain over the reported SkillOpt result. This also approaches the LoRA result of 78.6, despite updating only a short input-side prefix rather than model weights. On LiveMath, placement matters more: S OFT S KILL with prompt start reaches 50.0 ± 9.1, close to the reported SkillOpt result of 52.0, while skill section reaches 64.5 ± 2.4, outperforming both SkillOpt and LoRA. On DocVQA, S OFT S KILL with prompt start reaches 88.2 ± 1.4, improving over no skill but slightly below the reported SkillOpt score of 89.0. The mean-pooling variant gives the strongest DocVQA number, 89.6, but we treat the tuned soft-delta setting as the main method because it is the most direct compression of the natural-language skill into a trainable prefix. These results suggest that soft skill tuning is most effective when the task benefit can be captured by compact answer-format, retrieval, or reasoning preferences. The gains are not uniform across all initialization and placement choices: mean initialization alone is weak on LiveMath and collapses on DocVQA, showing that the learned update is essential rather than a trivial consequence of inserting averaged skill embeddings. Overall, the single-round QA setting provides the clearest evidence that a short optimized soft skill can replace long natural-language skill context while preserving or improving task accuracy. Compression diagnostics. Figure 2 evaluates whether the accuracy gains in Table 1 are accompanied by lower deployment overhead. The primary comparison is skill-context length. In the reproduced SkillOpt setting, the final skill artifacts contain 407 tokens for DocVQA, 671 tokens for LiveMath, and 2035 tokens for SearchQA. S OFT S KILL replaces each of these long text skills with 32 virtual tokens, giving roughly 12.7×, 21.0×, and 63.6× reductions in skill-context length, respectively. The same diagnostic also reports average generated tokens. These numbers should be interpreted cautiously because S OFT S KILL is trained with direct-answer, no-CoT NTP 6
Preprint.
SkillOpt (reproduced)
Skill/context tokens (log scale) SearchQA
LiveMath
DocVQA
2035 32
SoftSkill
Average generated tokens (log scale)
Accuracy
98.4
72.0
8.7
76.4
671
4422.0
32
37.1
6.0
60.5
407
29.6
32
90.1
11.1
10
2
10
3
10
1
89.3
10
2
10
3
0
20
40
60
80
100
Figure 2: Compression diagnostics for the single-round QA setting on Qwen3.5–4B. The primary deployment comparison is skill-context length: S OFT S KILL replaces long SkillOpt text with 32 virtual tokens. Average output-token reductions are reported as a secondary consequence of direct-answer, no-CoT NTP supervision, not as independent evidence of emergent reasoning compression. SkillOpt (reproduced) results are from our inference run using the released final SkillOpt artifacts with the available Qwen3.5–4B harness; these reproduced accuracies, especially LiveMath, can differ from the SkillOpt paper’s reported numbers because exact inference settings are not fully pinned. Token axes use log scale.
targets, so shorter generations are partly induced by the supervision format. Still, the reductions are large: average generated tokens decrease from 98.4 to 8.7 on SearchQA, from 4422.0 to 6.0 on LiveMath, and from 29.6 to 11.1 on DocVQA. Thus, the strongest claim from Figure 2 is context compression, while output-length reduction is best viewed as a secondary deployment consequence of the training setup rather than independent evidence of reasoning compression. The SkillOpt token and accuracy measurements in Figure 2 come from our reproduced inference run using the released final SkillOpt artifacts and the available Qwen3.5–4B harness. These reproduced accuracies differ from the paper-reported SkillOpt numbers, especially on LiveMath. We therefore use the reproduced SkillOpt run only for the compression diagnostic, where token length, generation length, and accuracy are measured under the same local inference harness. Agentic execution: useful signal, but not robust refinement. We next evaluate agentic execution as a stress test rather than as the main success case. Unlike single-round QA, these tasks require the prefix to encode multi-step procedural behavior, including tool calls, file or environment interaction, and long-horizon action dependencies. This setting is therefore a harder boundary case for sparse trajectory imitation. Table 2 reports the default agentic comparison. The setting differs from the single-round QA experiments: it uses Qwen3.6–35B-A3B, reproduced SkillOpt artifacts, and GPT-5.5generated successful trajectories as NTP supervision. “Val-selected” denotes the checkpoint selected by held-out validation, while “Oracle grid best” is a post-hoc diagnostic from the appendix grid and should not be interpreted as a selectable method. The results are mixed. On OfficeQA, validation-selected S OFT S KILL reaches 33.8, improving over no skill, the initial hard artifact, and the final hard artifact; the oracle grid reaches 36.0. On Spreadsheet, however, the final hard artifact remains strongest at 52.5, while validation-selected S OFT S KILL drops to 28.2 and even the oracle grid remains below the no-skill baseline. On ALFWorld, S OFT S KILL improves over no skill, 71.6 versus 57.5, but does not match the final hard artifact at 79.1. These results indicate that the prefix can absorb some useful behavioral signal, but it does not yet robustly compress or refine long-horizon agentic behavior. We therefore separate agentic execution from the main QA evidence. The single-round tasks show a clearer compression–accuracy tradeoff, while the agentic tasks expose a limitation of 7
Preprint.
Table 2: Agentic-task comparison under the default setting. Task
No skill
Hard initial
Hard final
Val-selected S OFT S KILL
Oracle grid best
OfficeQA Spreadsheet ALFWorld
25.6 (+0.0) 39.6 (+0.0) 57.5 (+0.0)
26.2 (+0.6) 33.6 (-6.0) 64.9 (+7.4)
26.7 (+1.1) 52.5 (+12.9) 79.1 (+21.6)
len=32: 33.8 (+8.2) len=8: 28.2 (-11.4) len=auto: 71.6 (+14.1)
len=8: 36.0 (+10.4) len=auto: 35.7 (-3.9) len=auto: 71.6 (+14.1)
the present NTP-based soft-skill training recipe. In particular, long-horizon settings may require stronger validation selection, richer trajectory supervision, reward-based updates, or architectures that preserve more procedural structure from the original hard skill. These agentic results should be read as reproduced-harness comparisons rather than exact reproductions of the SkillOpt paper. In particular, our OfficeQA reproduction uses the released offline setting, whereas the reported SkillOpt setting used a web-search configuration; the released materials also do not fully specify Qwen-family inference parameters such as stopping behavior and decoding budgets.1 These differences mean that Qwen reproduction gaps may reflect both environment differences and model-serving choices.
5
Analysis
The main results show that S OFT S KILL can replace long text skills with a short learned prefix in single-round tasks. We now ask what explains this behavior. The analysis separates four questions: whether the gains come from prefix capacity, whether they depend on model scale or initialization, how much supervision is needed, and whether the learned prefix captures task behavior beyond output formatting. 5.1
Capacity, Scale, and Data
Prefix length. Table 3 reports a prefix-length sweep on Qwen3.5–4B. Accuracy is not strongly monotonic in prefix length. On SearchQA, performance varies only from 75.6 to 76.9 across lengths 8, 32, 256, and auto-length prefixes. On DocVQA, all supported lengths remain close, between 87.4 and 89.3. LiveMath is more sensitive, but the main variation comes from placement rather than raw capacity: at length 32, skill section reaches 66.9, compared with 60.5 for prompt start; at length 256, the corresponding scores are 63.7 and 58.1. These results make length 32 a practical default rather than a carefully tuned optimum. It is short enough to give a large deployment compression ratio, yet it already matches or exceeds the longer prefixes on most settings. The auto-length rows are especially useful: even when the soft prefix is expanded to match the tokenized hard-skill length, accuracy does not consistently improve. This weakens a simple capacity explanation for the singleround gains. Instead, the results suggest that many answer-generation behaviors can be captured by a compact conditioning vector, while longer or more structured signals may be needed for procedural tasks. Model scale. Table 4 evaluates length-32 prefixes across Qwen-family models. Scaling generally improves SearchQA and DocVQA, but the trend is not uniform across tasks. On SearchQA, S OFT S KILL improves from 76.4 on Qwen3.5–4B to 80.9 on Qwen3.5–9B and 82.3 on Qwen3.6–35B-A3B. On DocVQA, prompt start similarly rises from 89.3 to 92.5 and 93.3. LiveMath is less stable: Qwen3.5–9B reaches 66.9, but Qwen3.6–35B-A3B drops to 50.0 under prompt start and 45.2 under skill section. This non-monotonicity suggests that soft-skill behavior depends on more than backbone strength. Placement, decoding settings, model architecture, and task-specific answer style 1 https://github.com/microsoft/SkillOpt/issues/51
8
Preprint.
Table 3: Prefix-length sweep with Qwen3.5–4B, reported as seed-1 test-hard accuracy.
Placement
Prefix length
SearchQA
LiveMath
DocVQA
No Skill SkillOpt
– –
68.1 (+0.0) 71.2 (+3.1)
22.4 (+0.0) 52.0 (+29.6)
86.9 (+0.0) 89.0 (+2.1)
prompt start skill section prompt start skill section prompt start skill section prompt start skill section
8 8 32 32 256 256 auto (2035/671/407) auto (2035/671/407)
75.6 (+7.5) 75.6 (+7.5) 76.4 (+8.3) 76.4 (+8.3) 76.9 (+8.8) 76.9 (+8.8) 75.6 (+7.5) 75.6 (+7.5)
56.5 (+34.1) 43.6 (+21.2) 60.5 (+38.1) 66.9 (+44.5) 58.1 (+35.7) 63.7 (+41.3) 55.6 (+33.2) 59.7 (+37.3)
87.4 (+0.5) – 89.3 (+2.4) – 89.3 (+2.4) – 89.3 (+2.4) –
Table 4: Model-size sweep for length-32 S OFT S KILL, reported as seed-1 test-hard accuracy.
Model
Placement
SearchQA
LiveMath
DocVQA
Qwen3.5–4B Qwen3.5–4B Qwen3.5–4B Qwen3.5–4B
No Skill SkillOpt prompt start skill section
68.1 (+0.0) 71.2 (+3.1) 76.4 (+8.3) 76.4 (+8.3)
22.4 (+0.0) 52.0 (+29.6) 60.5 (+38.1) 66.9 (+44.5)
86.9 (+0.0) 89.0 (+2.1) 89.3 (+2.4) –
Qwen3.5–9B Qwen3.5–9B
prompt start skill section
80.9 78.6
66.9 64.5
92.5 –
Qwen3.6–35B-A3B Qwen3.6–35B-A3B Qwen3.6–35B-A3B Qwen3.6–35B-A3B
No Skill SkillOpt prompt start skill section
72.7 (+0.0) 80.3 (+7.6) 82.3 (+9.6) 80.6 (+7.9)
31.2 (+0.0) 41.6 (+10.4) 50.0 (+18.8) 45.2 (+14.0)
87.6 (+0.0) 91.4 (+3.8) 93.3 (+5.7) –
can all interact with the learned prefix. The Qwen3.6–35B-A3B result is particularly cautionary because it is a mixture-of-experts model: it improves SearchQA and DocVQA relative to Qwen3.5–4B, but transfers less cleanly on LiveMath. We therefore interpret the model-size sweep as evidence that S OFT S KILL can scale, not as evidence of a smooth model-size law. 5.2
Initialization and Data
Initialization. Table 5 compares trained prefixes initialized from natural-language skill text, mean-pooled embeddings, and SkillOpt artifacts. All prefix rows use the same NTP training recipe, so the table asks whether the starting point still matters after optimization. The answer is mixed. Natural-language initialization is strongest on SearchQA, reaching 76.4 under both placements. Mean-pooled initialization is weaker on SearchQA but competitive on DocVQA, where it reaches 89.6. SkillOpt-artifact initialization gives the best LiveMath prompt start result, 66.1, but does not consistently dominate natural-language initialization. These patterns suggest that NTP often supplies the main improvement, while initialization acts as a task-dependent bias rather than a guaranteed source of better behavior. This is also why we avoid making a strong semantic-compression claim. If the learned prefix simply preserved the natural-language skill in compressed form, we would expect text-derived initialization to dominate more consistently. Instead, the results are better explained as learned contextual adaptation: initialization provides a useful starting point, but the final behavior is shaped substantially by the trajectory targets and the insertion interface. 9
Preprint.
Table 5: Sensitivity to initialization after training.
Initialization
Placement
SearchQA
LiveMath
DocVQA
– –
No Skill SkillOpt
68.1 (+0.0) 71.2 (+3.1)
22.4 (+0.0) 52.0 (+29.6)
86.9 (+0.0) 89.0 (+2.1)
NL skill NL skill Mean-pooled Mean-pooled SkillOpt artifact SkillOpt artifact
prompt start skill section prompt start skill section prompt start skill section
76.4 (+8.3) 76.4 (+8.3) 72.6 (+4.5) 72.6 (+4.5) 76.4 (+8.3) 76.4 (+8.3)
50.0 (+27.6) 64.5 (+42.1) 59.7 (+37.3) 42.7 (+20.3) 66.1 (+43.7) 63.7 (+41.3)
88.2 (+1.3) – 89.6 (+2.7) – 88.5 (+1.6) –
Table 6: Train-data scaling study. Task
Training fraction
SearchQA SearchQA SearchQA SearchQA SearchQA
0% (No Skill) 100% (SkillOpt) 10% 50% 100%
LiveMath LiveMath LiveMath LiveMath LiveMath
0% (No Skill) 100% (SkillOpt) 10% 50% 100%
DocVQA DocVQA DocVQA DocVQA DocVQA
0% (No Skill) 100% (SkillOpt) 10% 50% 100%
LoRA
S OFT S KILL prompt start
S OFT S KILL skill section
68.1 (+0.0) 71.2 (+3.1) 72.4 (+4.3) 77.4 (+9.3) 78.6 (+10.5)
71.1 (+3.0) 74.0 (+5.9) 76.4 (+8.3)
71.1 (+3.0) 74.0 (+5.9) 76.4 (+8.3)
22.4 (+0.0) 52.0 (+29.6) 33.1 (+10.7) 62.1 (+39.7) 58.9 (+36.5)
37.1 (+14.7) 61.3 (+38.9) 50.0 (+27.6)
30.6 (+8.2) 64.5 (+42.1) 64.5 (+42.1)
86.9 (+0.0) 89.0 (+2.1) 86.9 (+0.0) 88.8 (+1.9) 87.7 (+0.8)
88.0 (+1.1) 89.6 (+2.7) 88.2 (+1.3)
– – –
Data scaling. Table 6 compares S OFT S KILL with LoRA under different training-data fractions. The trends vary by task. On SearchQA, LoRA benefits steadily from more data and remains ahead at 50% and 100%, reaching 78.6 compared with 76.4 for S OFT S KILL. On LiveMath, S OFT S KILL is more competitive: it outperforms LoRA at 10% data under prompt start, and skill section is best at both 50% and 100%. On DocVQA, S OFT S KILL remains close to LoRA while updating only the prefix, with the best supported setting reaching 89.6 at 50% data. The data-scaling study clarifies the target regime. S OFT S KILL is not a replacement for LoRA when full parameter-efficient training is available and the goal is maximum accuracy. Its advantage is narrower: it provides compact, input-side adaptation that works with a frozen backbone and can be competitive in low- or moderate-data settings. 5.3
Behavioral Compression
Context and generation budget. The compression diagnostic in Figure 2 gives the clearest deployment-side evidence. On SearchQA, S OFT S KILL replaces a 2035-token SkillOpt artifact with a 32-token soft prefix, reduces average output length from 98.4 to 8.7 tokens, and improves reproduced accuracy from 72.0 to 76.4. On LiveMath, 671 skill tokens become 32 virtual tokens, average output length drops from 4422.0 to 6.0 tokens, and reproduced accuracy rises from 37.1 to 60.5. On DocVQA, the comparison is closer: S OFT S KILL compresses 10
Preprint.
Figure 3: Training loss versus held-out validation task accuracy for single-round runs (left) and agentic runs (right). Orange points mark the checkpoint selected by validation task accuracy within each run; blue points are other checkpoints. Lower loss is not sufficient to identify the best task checkpoint, especially in the agentic setting. 407 skill tokens to 32 and shortens outputs from 29.6 to 11.1 tokens, while accuracy changes from 90.1 to 89.3. The robust conclusion is therefore context compression with preserved or improved task accuracy on most single-round settings. The generated-token reduction is useful for deployment, but it should not be overinterpreted: because the NTP targets are direct answers without chain-of-thought, the prefix is partly trained to induce short responses. In other words, the result shows efficient behavior under the chosen supervision format, not independent evidence that the model has discovered a compressed reasoning procedure. Loss versus task success. Figure 3 compares NTP loss with held-out task accuracy across checkpoints. The relationship is informative but not sufficient for model selection. Very high loss usually corresponds to poor behavior, but among lower-loss checkpoints the correlation with task success is weak. The validation-selected checkpoint is often not the minimum-loss checkpoint. This supports a conservative view of the training objective. NTP can teach answer style, response length, and trajectory conventions without guaranteeing better task execution. This issue is mild in single-round tasks and more pronounced in agentic runs, where lower loss may reflect imitation of local action patterns rather than successful long-horizon behavior. Held-out task validation is therefore essential rather than just a reporting convenience. Prefix geometry. Although the learned prefix is continuous, embedding-space probes can still diagnose how training changes it. For each learned prefix vector pi ∈ Rd and vocabulary embedding e j ∈ Rd , we compute cos( pi , e j ) =
pi⊤ e j . ∥ p i ∥2 ∥ e j ∥2
(3)
Nearest-token probes rank vocabulary items by this score for each prefix position, and the corresponding top-1 score gives a hard-decoding cosine diagnostic. For text-initialized prefixes, we also compute cos( pi , pinit i ), averaged over prefix positions, to measure how far training moves the prefix from its initialization. These probes are diagnostic only: they can reveal geometric drift or clustering, but they do not prove that the continuous prefix preserves the semantics of the initializing skill. Figure 4 shows task-dependent prefix geometry. SearchQA produces more heterogeneous learned prefixes, while LiveMath and DocVQA show more consistent structure across runs. Mean-pooled initialization is visibly different from text-based initialization, producing less structured and more dispersed embeddings. Together with the initialization table, this 11
Preprint.
Figure 4: Prefix-level cosine similarity heatmaps for Qwen3.5–4B on SearchQA (left), LiveMath (middle), and DocVQA (right). Each matrix visualizes pairwise cosine similarity between learned prefix embeddings from different runs. Dark stripes indicate mean-pooled initialized prefixes, which exhibit lower and less structured similarity. Yellow indicates higher cosine similarity. Overall, task-dependent clustering is visible, with stronger run-torun variability on SearchQA and more consistent structure on LiveMath and DocVQA. suggests that the learned prefix is not merely a shortened copy of the original skill text; it is an optimized conditioning object whose geometry depends on task, initialization, and placement. Validation-selected checkpoints. Validation often selects a checkpoint before the final epoch. Figure 5 summarizes this effect. In single-round runs, the best checkpoint occurs at epoch 1, 2, and 3 in 28.2%, 41.0%, and 30.8% of runs, respectively. In agentic runs, selection is more front-loaded: 50.0% of runs peak at epoch 1, compared with 28.6% at epoch 2 and 21.4% at epoch 3. This pattern indicates that prefix tuning can overfit even though only a small number of virtual tokens is trained. Later epochs may improve trajectory imitation while hurting held-out task behavior, especially when the supervision contains sparse successful rollouts. Validation selection is therefore part of the method: it prevents the final checkpoint from being treated as automatically best and exposes when NTP training stops improving actual task success. 5.4
Limitations and Future Directions
Serving and access. The deployment beneValidation-selected checkpoint epochs fit of S OFT S KILL assumes a serving stack that can inject learned prefix embeddings. This is realistic for open or self-hosted models, but it makes the method less black-box than Markdown skill optimization. Training and serving require access to the target model’s embedding interface, and the learned prefix may be tied to a specific tokenizer, prompt template, decoding setup, or inference harness. Best validation checkpoint epoch The safest deployment claim is therefore not that S OFT S KILL learns a universal skill, but Figure 5: Validation-selected checkpoint that it provides compact contextual adapta- epochs for S OFT S KILL training runs. Singleround runs peak across all three epochs, tion under a frozen-backbone serving path. with epoch 2 selected most often; agentic runs more often peak at epoch 1. Auditability. A learned prefix is harder to inspect than a natural-language skill. One practical compromise is to keep the MarkRuns selecting checkpoint (%)
60
40 30
16 41.0%
7 50.0%
12 30.8%
11 28.2%
4 28.6%
20
3 21.4%
10
0
12
Single-round QA (n=39) Agentic tasks (n=14)
50
Epoch 1
Epoch 2
Epoch 3
Preprint.
down skill as metadata for retrieval, provenance, and human audit, while injecting the learned prefix at inference time. In this design, the text skill remains useful, but it should not be treated as a complete explanation of the continuous behavior. Our experiments also use one skill per task and do not evaluate retrieval among many skills, so progressive-disclosure deployment remains an implication rather than a directly tested result. Procedure versus formatting. The experiments do not fully distinguish robust procedure learning from answer-format learning. This matters because the strongest single-round results may partly come from inducing direct answers, short generations, or benchmarkspecific extraction formats. Agentic tasks expose this limitation more sharply: OfficeQA, Spreadsheet, and ALFWorld require multi-step behavior rather than a single final response. Future perturbation tests should change prompt templates, tool names, observation formats, stopping rules, and answer extractors to measure whether the learned prefix captures task procedure or mainly harness conventions. Transfer and composition. Unlike Markdown skills, continuous prefixes are not directly portable across models. A prefix trained for one embedding space may not transfer to another, even within the same model family. Vocabulary-projection transfer is one possible diagnostic: map each source prefix vector to a sparse distribution over source tokens, then reconstruct a target prefix using aligned token strings or shared vocabulary IDs. Poor transfer would not invalidate S OFT S KILL, but it would clarify that the learned object is mostly model-specific adaptation rather than a model-independent skill. The present experiments also train one soft skill per task. They do not show that soft skills compose, route, or transfer across related tasks. A stronger version of S OFT S KILL would need to support multiple skills through concatenation, interpolation, routing, or learned mixtures, and it would need to preserve behavior when task format changes. This is especially important for agentic settings, where useful behavior may be distributed across retrieval, planning, tool use, verification, and final-answer formatting. Takeaway. The analysis supports a focused interpretation. S OFT S KILL is effective as compact contextual adaptation for single-round tasks: it substantially reduces skill-context length, often preserves or improves accuracy, and can be competitive with LoRA in limiteddata settings. The evidence does not yet support a stronger claim that the prefix universally compresses procedural agent skills. Agentic execution, cross-model transfer, and compositional use remain the main open directions.
6
Related Work
Several lines of work study how language-model behavior can be adapted without full model retraining. Textual skill and prompt-optimization methods modify the artifact that the model reads; soft-prompt and prefix-tuning methods modify continuous conditioning vectors; adapter-based methods modify additional model parameters; and distillation or feedback-based methods transfer behavior from demonstrations, rollouts, or preferences into a deployable policy or control signal. S OFT S KILL lies at the intersection of these directions: it starts from reusable natural-language procedural knowledge, learns a small continuous object while keeping the backbone frozen, and compresses successful answers or trajectories into a cheaper deployable form. We therefore review related work according to the representation being optimized and the deployment interface it induces. Reusable skills and agent experience. Skill systems package reusable agent behavior as natural-language files that can be selected, inspected, and loaded at inference time (Zhang et al., 2025). SkillOpt (Yang et al., 2026) formulates skill learning as optimization over a single external natural-language state: rollout evidence and held-out validation guide bounded edits to a Markdown skill artifact. Related agent-learning systems store reusable experience in other explicit forms. Voyager builds a library of executable skills for an embodied agent (Wang et al., 2023), while Reflexion and ExpeL convert trial-and-error experience into verbal memory or lessons that can guide later attempts without updating 13
Preprint.
model weights (Shinn et al., 2023; Zhao et al., 2024). These systems share the premise that useful behavior can be externalized as text, code, or memory. S OFT S KILL asks a complementary question: whether some of that reusable behavior can be compressed into a learned continuous context object consumed directly by the target model. Text-space prompt and instruction optimization. Prompt and instruction optimization methods search over textual artifacts rather than model weights. AutoPrompt uses gradient guidance to discover discrete trigger tokens (Shin et al., 2020); APE and OPRO use language models to propose and refine natural-language instructions (Zhou et al., 2023; Yang et al., 2024); DSPy treats prompts and pipelines as optimizable program components (Khattab et al., 2023). InstructZero, TextGrad, GEPA, and SkillOpt similarly optimize language artifacts with black-box or feedback-driven search (Chen et al., 2023; Yuksekgonul et al., 2024; Agrawal et al., 2025; Yang et al., 2026). These approaches retain the portability and auditability of text, but the target model must still reinterpret the optimized artifact at inference time. S OFT S KILL instead optimizes the embedding object that conditions the target model, requiring white-box access during training while keeping the backbone frozen. Soft prompts, prefix tuning, and PEFT. Prompt tuning, prefix tuning, and P-tuning variants show that a small number of continuous virtual tokens can adapt a frozen language model (Lester et al., 2021; Li & Liang, 2021; Liu et al., 2022). Subsequent work studies sensitivity to initialization, robustness, prompt length, and reparameterized or residual prompt updates (Wu et al., 2023; Razdaibiedina et al., 2023). LoRA and related low-rank adaptation methods provide a stronger parameter-efficient tuning baseline when updating model parameters is allowed (Hu et al., 2021). S OFT S KILL does not claim that continuous prompts are new. Its distinction is to use a natural-language skill artifact as the source of a compact, model-specific behavioral prior and to evaluate that prior against skill-system baselines rather than only against generic prompt tuning. Behavioral compression from supervision. Knowledge distillation compresses behavior from a larger or more expensive source into a smaller deployable object (Hinton et al., 2015). Behavior cloning and imitation learning similarly train a policy from demonstrated or successful trajectories (Ross et al., 2011). Our implemented objective is closer to this supervised view than to reinforcement learning: next-token prediction on answers or successful trajectories trains the prefix to bias a frozen decoder toward observed successful behavior. Reward models in RLHF pipelines score candidate outputs or trajectories and can then guide policy optimization (Christiano et al., 2017; Ouyang et al., 2022). By contrast, a S OFT S KILL prefix does not score candidates at inference time; it directly conditions generation. Reward-based refinement is a natural extension, but it is not required for the core setting studied here. Continuous-prompt diagnostics and steering. Continuous prompts and activation-level steering methods raise the question of what remains inspectable when behavior is encoded in latent vectors (Hambardzumyan et al., 2021; Turner et al., 2024). Later sections use nearest-token and cosine-similarity probes only as diagnostics: they test how far a learned prefix moves from its text-derived initialization, not whether the prefix is semantically equivalent to the original skill. This supports a limited audit claim–the text skill remains useful for initialization, routing, and provenance–without treating the learned continuous prefix itself as human-interpretable.
7
Conclusion
S OFT S KILL asks whether reusable skill behavior can be compressed from a long naturallanguage artifact into a compact continuous prefix for a frozen target model. Our results show that short soft skills can preserve or improve direct-answer QA behavior while replacing substantially longer Markdown skills at inference time. The learned delta adds value beyond text-based initialization, but validation-selected checkpointing is important 14
Preprint.
because language-modeling loss is not a reliable proxy for task success. Overall, S OFT S KILL is best viewed as compact contextual adaptation for frozen-backbone serving, not as a universal substitute for textual skills, adapters, or reward optimization. More procedural agentic settings remain a harder boundary case and call for stronger transfer, robustness, and sparse-trajectory learning methods.
References Saurabh Agrawal, Aman Madaan, Sameer Singh, and Graham Neubig. Gepa: Reflective prompt evolution can outperform reinforcement learning. arXiv preprint arXiv:2507.19457, 2025. Lichang Chen, Matei Zaharia, and James Zou. Instructzero: Efficient instruction optimization for black-box large language models. In International Conference on Machine Learning, 2023. Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, 2017. Databricks. Officeqa: A grounded reasoning benchmark, 2025. Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. Searchqa: A new q&a dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179, 2017. URL https://arxiv.org/abs/1704.05179. Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. Warp: Word-level adversarial reprogramming, 2021. URL https://arxiv.org/abs/2101.00121. Linyang He, Qiyao Yu, Hanze Dong, Baohao Liao, Xinxing Xu, Micah Goldblum, Jiang Bian, and Nima Mesgarani. Livemathematicianbench: A live benchmark for mathematicianlevel reasoning with proof sketches, 2026. URL https://arxiv.org/abs/2604.01754. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into self-improving pipelines, 2023. URL https://arxiv.org/abs/ 2310.03714. Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021. Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021. Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2022. Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation, 2024. URL https://arxiv.org/abs/2406.14991. 15
Preprint.
Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021. Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. Trace2skill: Distill trajectory-local lessons into transferable agent skills, 2026. URL https://arxiv.org/abs/2603.25158. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022. Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Jimmy Ba, and Amjad Almahairi. Residual prompt tuning: Improving prompt tuning with residual reparameterization, 2023. URL https://arxiv.org/abs/2305.03937. Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning, 2011. URL https://arxiv.org/ abs/1011.0686. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222–4235, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.346. URL https://aclanthology.org/2020.emnlp-main. 346/. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URL https://arxiv.org/abs/2303.11366. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=0IOX0YcCdTn. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, 2024. URL https://arxiv.org/abs/2308.10248. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URL https://arxiv.org/abs/2305.16291. Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, and Ricardo Henao. Infoprompt: Information-theoretic soft prompt tuning for natural language understanding, 2023. URL https://arxiv.org/abs/2306.04933. An Yang, Anfeng Li, Baosong Yang, Beichen Zheng, Binyuan Hui, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. URL https://arxiv.org/abs/2505.09388. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers, 2024. URL https://arxiv.org/abs/ 2309.03409. Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, Dongdong Chen, Xue Yang, and Chong Luo. Skillopt: Executive strategy for self-evolving agent skills. arXiv preprint arXiv:2605.23904, 2026. URL https://arxiv.org/abs/2605.23904. 16
Preprint.
Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. Textgrad: Automatic differentiation via text. arXiv preprint arXiv:2406.07496, 2024. Barry Zhang, Keith Lazuka, and Mahesh Murag. Equipping agents for the real world with agent skills. https://www.anthropic.com/engineering/ equipping-agents-for-the-real-world-with-agent-skills, October 2025. Anthropic engineering blog. Accessed: 2026-06-15. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners, 2024. URL https://arxiv.org/abs/2308. 10144. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers, 2023. URL https://arxiv.org/abs/2211.01910.
17
Preprint.
Table 7: Comparison of skill optimization methods and related adaptation baselines. Method
Optimized object
Training signal
No skill
None
None
Rollouts? Inference-time deployment No
Task prompt only; frozen model Long text skill in context Optimized text skill in context
Human writNo ing/editing SkillOpt hard skill Markdown skill Scored rollouts Yes and validation feedback Prompt optimization Prompt text Black-box scores, Usually Optimized text examples, or reprompt in context flections Supervised trajec- Optional Adapter-aware LoRA / adapter Adapter weights tories or RL feedserving; extra back weights loaded Prefix / P-tuning Virtual tokens Supervised trajec- Optional Frozen model plus tories or RL feedlearned continuback ous prefix S OFT S KILL Text-initialized soft skill Gradient signal Frozen model plus No from answers or compact learned successful trajectoskill prefix ries Hard skill + soft delta Text skill + prefix Answers, trajec- Optional Text skill remains; tories, or reward prefix provides feedback compact correction Manual hard skill
A
Markdown skill
Experiment Setting
The first appendix tables summarize the method taxonomy, data splits, and benchmark regimes used by the main experiments. Table 8: Dataset split sizes and experiment configuration. Main-text experiments summarize the total nominal train size and the smaller usable trajectory counts for the agentic tasks. Task DocVQA SearchQA LiveMath ALFWorld OfficeQA Spreadsheet
B
Train
Val
Test
Total
Usable train
107 400 35 39 50 80
53 200 18 18 24 40
374 1,400 124 134 172 280
534 2,000 177 191 246 400
107 400 35 31 31 61
Additional Results
The reproduced SkillOpt baselines use the public artifacts and the available evaluation harness, but they are not exact reproductions of the paper’s OfficeQA environment. The authors clarified that their OfficeQA experiments used a Google-backed web-search tool in which candidate files served as hints rather than as the full accessible evidence set. By contrast, the public offline mode uses local document tools, and the public custom-search path requires a compatible authenticated search service. The public release also does not fully pin Qwen inference parameters, so differences in stopping criteria, decoding budget, and serving configuration may change reproduced Qwen results. 18
Preprint.
Table 9: Benchmark regimes and training supervision. Agentic tasks are reported separately because they test SkillOpt-artifact refinement rather than pure answer-behavior compression. Task
Regime
NTP target
Expected relation to SkillOpt
SearchQA LiveMath DocVQA OfficeQA Spreadsheet ALFWorld
Single-round QA Single-round QA Single-round QA Agentic execution Agentic execution Agentic execution
Ground-truth answer, no CoT Ground-truth answer, no CoT Ground-truth answer, no CoT Successful trajectory Successful trajectory Successful trajectory
Answer/reasoning compression Answer/reasoning compression Answer/reasoning compression SkillOpt-artifact refinement SkillOpt-artifact refinement SkillOpt-artifact refinement
In the auxiliary QA tables, “hard” and “soft” refer to the evaluator used to score a generated answer, not to whether the inference input contains a hard or soft skill. “Hard” accuracy requires the exact task-level criterion used in the main results. “Soft” accuracy uses the more permissive answer-equivalence evaluator used for auxiliary diagnostics. Validation rows score the held-out selection split; test rows score the final held-out test split. For DocVQA, skill section cells are omitted because the vision-language implementation prepends the soft prefix regardless of the logged placement label. B.1
DocVQA Placement Support
Supporting true DocVQA skill section insertion would require carrying an insertion index, then splicing the prefix consistently into sequence-shaped tensors such as input embeddings, attention masks, labels, multimodal token-type metadata when present, and position inputs used by the vision-language model. Table 10: Compression diagnostics for the single-round QA setting. S OFT S KILL results are from the main seed-1 Qwen3.5–4B run; SkillOpt (reproduced) rows are from our inference run using the released final SkillOpt artifacts with the available Qwen3.5–4B harness. Reported SkillOpt accuracies are the Qwen3.5–4B numbers from Yang et al. (2026). Task
Method
SearchQA SearchQA LiveMath LiveMath DocVQA DocVQA
SkillOpt (reproduced) S OFT S KILL SkillOpt (reproduced) S OFT S KILL SkillOpt (reproduced) S OFT S KILL
Skill tokens
Average output tokens
Reproduced accuracy
Reported SkillOpt accuracy
2035 32 671 32 407 32
98.4 8.67 4422 5.99 29.6 11.1
72.0 76.4 37.1 60.5 90.1 89.3
71.2 – 52.0 – 89.0 –
19
Preprint.
Table 11: Auxiliary main-setting aggregate metrics for Qwen3.5–4B, length 32. Each cell reports mean ± standard deviation over three seeds. DocVQA skill section cells are omitted because that placement is unsupported in the vision-language path.
Metric
Placement
SearchQA
LiveMath
DocVQA
Test hard Test hard Test soft Test soft Validation hard Validation hard Validation soft Validation soft
prompt start skill section prompt start skill section prompt start skill section prompt start skill section
76.4 ± 0.0 76.4 ± 0.0 84.1 ± 0.0 84.1 ± 0.0 78.1 ± 0.0 78.1 ± 0.0 82.8 ± 0.0 82.8 ± 0.0
50.0 ± 9.1 64.5 ± 2.4 50.0 ± 9.1 64.5 ± 2.4 63.0 ± 3.2 72.2 ± 5.6 63.0 ± 3.2 72.2 ± 5.6
88.2 ± 1.4 – 93.1 ± 0.6 – 91.7 ± 3.6 – 95.7 ± 0.2 –
Table 12: Per-seed test-hard accuracy for the Qwen3.5–4B length-32 main setting. DocVQA skill section cells are omitted because that placement is unsupported in the visionlanguage path. Placement
Seed
prompt start prompt start prompt start skill section skill section skill section
1 2 3 1 2 3
SearchQA
LiveMath
DocVQA
76.4 76.4 76.4 76.4 76.4 76.4
60.5 45.2 44.4 66.9 62.1 64.5
89.3 88.8 86.6 – – –
Table 13: Auxiliary prefix-length metrics for Qwen3.5–4B, reported as seed-1 accuracy. DocVQA skill section cells are omitted because that placement is unsupported in the vision-language path.
Metric
Placement
Prefix length
Test soft Test soft Test soft Test soft Test soft Test soft Validation hard Validation hard Validation hard Validation hard Validation hard Validation hard Validation soft Validation soft Validation soft Validation soft Validation soft Validation soft
prompt start skill section prompt start skill section prompt start skill section prompt start skill section prompt start skill section prompt start skill section prompt start skill section prompt start skill section prompt start skill section
8 8 32 32 256 256 8 8 32 32 256 256 8 8 32 32 256 256
20
SearchQA
LiveMath
DocVQA
83.6 83.6 84.1 84.1 84.7 84.7 79.7 79.7 78.1 78.1 76.6 76.6 85.3 85.3 82.8 82.8 80.6 80.6
56.5 43.6 60.5 66.9 58.1 63.7 66.7 50.0 66.7 77.8 61.1 72.2 66.7 50.0 66.7 77.8 61.1 72.2
93.1 – 93.6 – 93.6 – 96.9 – 93.8 – 96.9 – 96.9 – 95.8 – 96.9 –
Preprint.
Table 14: Auxiliary model-sweep metrics for length-32 S OFT S KILL-NTP, reported as seed-1 accuracy. DocVQA skill section cells are omitted because that placement is unsupported in the vision-language path. Metric / placement
Model
Test soft, prompt start Test soft, skill section Test soft, prompt start Test soft, skill section Test soft, prompt start Test soft, skill section Validation hard, prompt start Validation hard, skill section Validation hard, prompt start Validation hard, skill section Validation hard, prompt start Validation hard, skill section Validation soft, prompt start Validation soft, skill section Validation soft, prompt start Validation soft, skill section Validation soft, prompt start Validation soft, skill section
Qwen3.5–4B Qwen3.5–4B Qwen3.5–9B Qwen3.5–9B Qwen3.6–35B-A3B Qwen3.6–35B-A3B Qwen3.5–4B Qwen3.5–4B Qwen3.5–9B Qwen3.5–9B Qwen3.6–35B-A3B Qwen3.6–35B-A3B Qwen3.5–4B Qwen3.5–4B Qwen3.5–9B Qwen3.5–9B Qwen3.6–35B-A3B Qwen3.6–35B-A3B
SearchQA
LiveMath
DocVQA
84.1 84.1 87.5 86.1 89.5 87.4 78.1 78.1 79.7 79.7 82.8 79.7 82.8 82.8 83.3 83.2 87.3 82.2
60.5 66.9 66.9 64.5 50.0 45.2 66.7 77.8 77.8 77.8 94.4 94.4 66.7 77.8 77.8 77.8 94.4 94.4
93.6 – 96.0 – 95.8 – 93.8 – 100.0 – 96.9 – 95.8 – 100.0 – 99.0 –
Table 15: Validation-selected checkpoint epochs. Each run evaluates checkpoints after epochs 1–3 and exports the checkpoint with the best validation score. Counts report how often each epoch is selected. Task
Runs
Epoch 1
Epoch 2
Epoch 3
SearchQA LiveMath DocVQA OfficeQA Spreadsheet ALFWorld
13 13 13 5 3 6
2 2 7 3 1 3
6 5 5 2 1 1
5 6 1 0 1 2
Single-round QA total Agentic total
39 14
11 7
16 4
12 3
Table 16: Prefix-interpretability diagnostics. Prefix cosine measures geometric similarity to the initialization, while greedy-decoded content match is a qualitative probe rather than evidence of semantic equivalence. Task
Initialization
Prefix cosine similarity
Greedy-decoded content match
SearchQA LiveMath DocVQA OfficeQA Spreadsheet ALFWorld
NL skill NL skill NL skill SkillOpt artifact SkillOpt artifact SkillOpt artifact
0.760 0.934 0.892 0.907 0.754 0.820
Yes Yes Yes Yes No Yes
21
Preprint.
Table 17: Full agentic grid under the default setting: Qwen3.6–35B-A3B, reproduced SkillOpt artifacts, and GPT-5.5-generated successful trajectories used as NTP supervision. For truncated-prefix rows, the corresponding untrained truncated-prefix baseline was not evaluated, so init-only scores and deltas are left blank rather than computed against a different source. Task
Source artifact
Prefix construction
Prefix length
Init-only val
Init-only test
Trained val
Trained test
∆ val
∆ test
OfficeQA OfficeQA OfficeQA OfficeQA OfficeQA
initial.md final.md final.md final.md final.md
full artifact first n tokens first n tokens first n tokens full artifact
auto 8 32 256 auto
25.0 – – – 33.3
26.2 – – – 26.7
16.7 33.3 41.7 37.5 25.0
11.4 36.0 33.8 33.7 25.6
-8.3 – – – -8.3
-14.8 – – – -1.1
Spreadsheet Spreadsheet Spreadsheet Spreadsheet Spreadsheet
initial.md final.md final.md final.md final.md
full artifact first n tokens first n tokens first n tokens full artifact
auto 8 32 256 auto
56.3 – – – 18.8
33.6 – – – 52.5
25.0 50.0 37.5 37.5 37.5
23.2 28.2 30.7 30.0 35.7
-31.3 – – – +18.7
-10.4 – – – -16.8
ALFWorld ALFWorld ALFWorld ALFWorld ALFWorld
initial.md final.md final.md final.md final.md
full artifact first n tokens first n tokens first n tokens full artifact
auto 8 32 256 auto
77.8 – – – 83.3
64.9 – – – 79.1
72.2 66.7 72.2 66.7 83.3
66.4 62.7 57.5 61.9 71.6
-5.6 – – – 0.0
+1.5 – – – -7.5
Figure 6: Per-position cosine similarity between each learned Qwen3.5–4B soft prefix and its initialization, ordered as SearchQA, LiveMath, and DocVQA. Higher values indicate a smaller learned displacement from the initial embedding sequence. For DocVQA, runs logged under skill section are effective prompt-start insertions in the vision-language path. SearchQA shows larger divergence from initialization than the other QA tasks, and mean-pooled initialization requires the largest learned displacement.
22