arXiv CSopen access

Latent Action Reparameterization for Efficient Agent Inference

Latent Action Reparameterization for Efficient Agent Inference

arXiv:2605.18597v1 [cs.AI] 18 May 2026

Wenhao Huang1∗, Qingwen Zeng2∗, Qiyue Chen2 , Zijie Guo3 , Yu Sun4 , Cheng Yang5 , Siru Ouyang6 , Jiri Gesi7 , Fang Wu8 , Jiayi Zhang5,9 , Huaming Chen2 , Bang Liu1 , Xiangru Tang4†, Chenglin Wu5† 1

4

Université de Montréal 2 The University of Sydney 3 Fudan University Yale University 5 DeepWisdom 6 University of Illinois Urbana-Champaign 7 Amazon Science 8 Stanford University 9 The Hong Kong University of Science and Technology (Guangzhou) 10 Mila - Quebec AI Institute

Abstract Large language model (LLM) agents often rely on long sequences of low-level textual actions, resulting in large effective decision horizons and high inference cost. While prior work has focused on improving inference efficiency through system-level optimizations or prompt engineering, we argue that a key bottleneck lies in the representation of the action space itself. We propose Latent Action Reparameterization (LAR), a framework that learns a compact latent action space in which each latent action corresponds to a multi-step semantic behavior. By reparameterizing agent actions into latent units, LAR enables decision making over a shorter effective horizon while preserving the expressiveness of the original action space. Unlike hand-crafted macros or hierarchical controllers, latent actions are learned from agent trajectories and integrated directly into the model, allowing both planning and execution to operate over abstract action representations. Across a range of LLM-based agent benchmarks, LAR significantly reduces the effective action horizon and improves inference efficiency under fixed compute budgets. As a consequence, our approach achieves substantial reductions in action tokens and corresponding wall-clock inference time, while maintaining or improving task success rates. These results suggest that action representation learning is a critical and underexplored factor in scaling efficient LLM agent inference, complementary to advances in model architecture and hardware. Source code is attached here 3 .

1

Introduction

Large language model (LLM) agents have emerged as a powerful paradigm for solving tasks involving multi-step reasoning, tool use, and interaction with external environments [34, 46]. By repeatedly generating actions conditioned on intermediate observations, LLM agents can perform search, planning, and decision making across diverse domains [24, 14, 40, 45, 47]. However, as these agents are applied to increasingly complex tasks, inference efficiency has become a critical bottleneck [23]. Agent execution often requires long sequences of decisions, leading to inference latency and prohibitive computational cost, which in turn limits scalability, deployment, and real-time interaction [15]. ∗ Equal contribution. † Correspondence to: [email protected], [email protected]. 3 https://github.com/EZ-hwh/LAR

Preprint.

Prior work has primarily addressed agent inference efficiency through improvements in model architecture, hardware acceleration, system-level optimizations, or prompt engineering [5, 7, 10, 33]. These approaches reduce the cost of individual inference steps or improve throughput, but they largely operate orthogonally to the structure of the agent’s decision process itself [51]. In particular, while per-token generation may become faster [13, 5], the number of decision steps required to complete a task often remains unchanged. As a result, the overall inference cost continues to scale poorly with task horizon, especially in settings that require multi-step reasoning or search [9]. In this work, we argue that inference efficiency in LLM agents is fundamentally constrained by the representation of the action space, particularly in sequential decision-making settings. In current agent systems, actions are typically realized as low-level textual outputs, where each generated token constitutes an explicit decision that conditions subsequent computation, planning, or interaction with the environment [19]. Such token-level action representations induce excessively fine-grained decision making, resulting in unnecessarily large effective decision horizons even for semantically simple behaviors [44, 6]. Consequently, inference scaling is dominated not by model size alone, but by the granularity at which agent actions are represented and composed over time [49, 41]. We therefore posit that action representation should be treated as a first-class modeling choice in LLM-based agents, on par with model architecture and system-level design. Motivated by this observation, we propose Latent Action Reparameterization (LAR), a framework that learns a compact latent action space for LLM agents. In LAR, each latent action corresponds to a multi-step semantic behavior that would otherwise be realized through a sequence of low-level textual actions. By reparameterizing agent decisions into these latent units, planning and execution can operate over a substantially shorter effective horizon while preserving the expressiveness of the original action space. Unlike hand-crafted macros or hierarchical controllers [1, 2, 4], latent actions in LAR are learned directly from agent trajectories and integrated into the model, enabling end-to-end decision making over abstract yet executable action representations. A challenge in action abstraction for LLM agents lies in balancing representational abstraction with action executability [42, 27]. Fully implicit latent representations are effective for internal reasoning and credit assignment, but they are insufficient for agent systems that must interact with external tools or environments through explicit, protocol-constrained interfaces [27, 16]. In such settings, actions must remain decodable, interpretable, and executable by downstream systems. LAR addresses this challenge by explicitly modeling the latent–explicit boundary: latent actions provide higher-level abstraction while remaining directly realizable as concrete, executable action sequences. Specifically, our framework compresses low-entropy, structurally recurring patterns, including system prompts, tool invocation syntax, and recurring configurations into latent units, while strictly preserving highentropy, parameter-rich inputs (e.g., specific search queries or numerical entities) in the explicit output space. This design reflects a broader principle in agent modeling: increased abstraction is not always beneficial, as executability fundamentally constrains useful action representations [42]. Our contributions are fourfold. First, Significant Efficiency Gains: We demonstrate that LAR significantly reduces the effective action horizon, leading to substantial reductions in action tokens and corresponding system-level gains in token throughput and peak GPU memory across diverse LLM agent benchmarks (Section 4.2, Table 8). Second, Preserved Task Performance: Despite operating over a compressed latent action space, our approach maintains or improves task success rates compared to baselines using raw textual actions (Table 1) and transfers to held-out benchmarks without retraining (Section 4.3), proving that efficiency need not come at the cost of performance. Third, Analysis of Abstraction Limits: We identify a distinct performance collapse threshold, empirically delineating the boundary between compressible structural redundancy and essential semantic content (Section 5.3). Fourth, New Perspective on Scaling: Our results highlight action representation learning as a critical and underexplored factor in scaling efficient LLM agent inference, validated across model scales up to 32B (Appendix A.10) and industrial agent runtimes (Appendix A.14), offering a complementary path to advances in model architecture and hardware.

2

Related Work

A large body of prior work improves the efficiency of LLM-based agents by modifying stages of the agent pipeline, including how inputs are conditioned, how tokens are generated, and how interaction histories are maintained. Prompting and Input-Level Control: Prompting and input-level methods improve efficiency by shaping the conditioning signal before or during inference [10]. Techniques 2

Env

Agent

Ot

at​=(xt,1​… xt,∣at​∣​)

At

Rt

Ot

Raw Action Trajectory

Rt

xt,1 xt,2 xt,3 . . . xt,∣at​-1∣ xt,∣at​∣ Reparameterized Compressed Trajectory Action Trajectory

At

xt,1 zt,1 . . . xt,∣at-1​∣ xt,∣at​∣ Fewer Tokens!

Latent Action Space

Figure 1: Overview of Latent Action Reparameterization (LAR). LAR reformulates agent decision making by collapsing transition-equivalent action segments into executable latent actions, thereby reducing the effective decision horizon. Low-entropy structural components are abstracted into latent actions, while high-entropy, parameter-binding content remains explicit to preserve executability. such as Chain-of-Thought elicit intermediate reasoning that improves accuracy but often increases generation length and latency [36, 35]. Subsequent prompt engineering constrains reasoning formats to reduce verbosity while preserving answer quality [50, 22]. Token-Level Generation Control and Inference-Time Interventions: Another line of work regulates token emission during inference to reduce redundant generation [21, 20, 30]. Representative approaches include in-generation guidance that encourages shorter reasoning traces (e.g., ConciseHint-style methods [31]) and token scoring or pruning mechanisms that skip low-utility tokens (e.g., TokenSkip [37]). These methods optimize efficiency within the original token-level generation process. Context and Memory Optimization for Agents: For interactive and tool-using agents, efficiency bottlenecks often stem from long interaction histories carried as context [29, 26, 25, 48, 43]. Context and memory optimization methods reduce conditioning costs by compressing or summarizing histories. ACON is the representative optimizing the agent’s memory representation via history and observation compression [18]. Collectively, the above approaches improve efficiency while leaving the decision interface fundamentally unchanged: the agent still reasons and acts at the level of token emissions, and efficiency gains arise from modifying inputs, regulating token generation, or compressing memory [33]. As a result, the effective decision horizon remains dictated by token-level granularity. In contrast, our work challenges this assumption and targets inefficiency at its source by redefining the unit of decision-making itself. LAR reparameterizes the action space by collapsing multi-step action segments that induce transition-equivalent behaviors into single latent actions, thereby directly reducing the effective decision horizon. Crucially, reparameterization is constrained by executability: parameter-binding actions that determine environment-facing semantics are preserved explicitly, while only stable, context-invariant scaffolds are abstracted. This reframes efficiency not as a byproduct of shorter text, but as a consequence of operating over a more appropriate decision representation.

3

Methodology

3.1

Action Definition and Problem Setup

We consider an LLM-based agent in a sequential decision-making setting. An agent–environment interaction is represented as a trajectory τ = (o1 , a1 , o2 , a2 , . . . , oT , aT ), where ot denotes the observation at step t and at denotes the action produced by the agent at that step. Observations may include textual context, intermediate reasoning states, tool outputs, or environment feedback. In contemporary LLM agents, actions are instantiated as explicit textual outputs. Each action at is a sequence of generated tokens at = (xt,1 , . . . , xt,|at | ), with xt,i drawn from the model vocabulary. We treat all generated tokens that condition subsequent computation or interaction as action decisions, including system-level configurations and interaction scaffolds. PT We define the effective action horizon of a trajectory as Heff (τ ) = t=1 |at |, which measures the number of generation decisions and directly determines inference cost. Our objective is to reduce this horizon by altering action representation, without modifying agent behavior or executability. 3

3.2

Latent Action Reparameterization

We propose Latent Action Reparameterization (LAR), which reformulates agent decision making over a compact action space as shown in Fig. 1. Instead of operating over token-level action primitives, LAR enables the agent to reason over higher-level action units that correspond to multi-step semantic behaviors. Let Z denote the latent action space, and let zt ∈ Z denote the latent action instantiated at step t. Each latent action represents a semantic decision unit that subsumes a sequence of low-level P actions. Under this representation, the effective horizon becomes Hlat (τ ) = t |zt |, where each latent action is treated as a single decision step. LAR aims to preserve the functional behavior induced by original trajectories while eliminating redundant decision steps caused by overly fine-grained action representations, a property we empirically verify in Section 5.1. Planning and execution, therefore operate directly over latent actions rather than token-level primitives. 3.3

Learning Latent Actions from Trajectories

Latent actions are learned directly from agent trajectories by identifying recurrent multi-step behaviors that function as stable decision units. Rather than treating each action at as atomic, we consider action segments, which may span multiple decision steps or structured sub-sequences within a single action. Such segments capture extended behaviors that recur across trajectories. We characterize recurrence using transition equivalence. Let T denote the transition dynamics induced by the agent and environment. Two segments a and a′ are said to be transition-equivalent if, for any preceding history h, the induced transitions satisfy T (h ◦ a) ≈ T (h ◦ a′ ), where ◦ denotes sequence concatenation and ≈ denotes equivalence up to task-relevant outcomes. In practice, this equivalence is not enforced universally but is approximated on the empirical trajectory distribution induced by the agent, with the approximation procedure given in Section 3.5. A latent action z ∈ Z is defined as an equivalence class of segments under this relation. To integrate latent actions into the agent, each z is parameterized as a vocabulary symbol (Section 3.5), so that planning and execution operate over latent actions as over ordinary tokens. 3.4

Executable Latent Actions

Not all latent actions are executable. In our framework, executability subsumes both syntactic validity under external interfaces and semantic correctness with respect to agent–environment interaction. Specifically, while actions must remain decodable and interpretable by downstream systems, true executability further requires that replacing concrete trajectory segments with a latent action does not alter the induced transition behavior. Formally, a latent action z is executable if all segments belonging to z are transition-equivalent: for any two realizations a, a′ ∈ z and any preceding history h, T (h ◦ a) ≈ T (h ◦ a′ ). This defines a semantic constraint on the latent action space. Latent actions satisfying this constraint correspond to behaviors whose effects are invariant across contexts; in contrast, segments whose effects depend on task-specific parameters or bindings violate transition equivalence and cannot be safely abstracted. The implementation of this constraint, via an entropy-based filter, is described in Section 3.5. 3.5

Implementation

LAR is realized as a four-stage pipeline that operationalizes the formal concepts introduced above: (1) identifying transition-equivalent action segments from agent trajectories, (2) constructing a latent action vocabulary, (3) preparing dual-format training data, and (4) aligning the model’s predictive behavior via trajectory-level distillation. Identifying transition-equivalent segments. Direct verification of the transition equivalence condition T (h ◦ a) ≈ T (h ◦ a′ ) from Section 3.3 over histories h is intractable. P We approximate it through the next-token entropy of a candidate segment s, defined as H(s) = − w∈Vs p(w | s) log2 p(w | s), where Vs is the set of tokens observed after s and p(w | s) is the empirical probability of w following s. A low H(s) indicates that the continuation behavior of s is predictable regardless of preceding history, precisely the condition required by transition equivalence. Conversely, high-entropy segments such as specific search queries or task-specific arguments exhibit context-dependent continuations 4

and would violate executability if abstracted. Next-token entropy thus serves as a tractable empirical surrogate for transition equivalence. The identification procedure (Algorithm 1, Appendix A.2) extracts word-level n-grams within boundaries, filters them by frequency (freq(s) ≥ fmin ) and entropy (H(s) ≤ Hmax ), ranks candidates by score(s) = freq(s)/(H(s) + 1), deduplicates the result, and retains the top-K segments as the latent action set Z. Per-task thresholds and vocabulary sizes are reported in Appendix A.4. Vocabulary and training data. Each segment in Z is assigned a dedicated vocabulary symbol. Training data is prepared in a dual-trajectory format: each original trajectory τ is paired with its reparameterized counterpart τ̂ , in which segments matching Z are replaced by the corresponding latent action symbols via longest-first matching. The original trajectory serves as the teacher signal, while the reparameterized trajectory is the student input. Trajectory-level distillation. A frozen copy of the original LLM (the teacher) processes τ , while a student, the same base model augmented with a LoRA adapter (rank r = 8, α = 16, applied to q/k/v/o projections) and new latent action embeddings, processes τ̂ . Only the LoRA weights and new embeddings are trainable, amounting to 0.1% of total parameters; all pretrained weights remain distillation over shared content positions, LKL = P frozen. The trainingTobjective is pure KL 1 S /τ )), where M is the set of token positions whose /τ ) ∥ softmax(z D (softmax(z KL i i i∈M |M | textual content is identical in both teacher and student sequences (excluding latent action symbols), ziT and ziS are the teacher and student logits, and τ = 2.0 is the distillation temperature. Restricting the loss to M is the mechanism by which latent action embeddings acquire their semantic content: the student must reproduce the teacher’s predictive distribution on non-compressed content despite receiving compressed input, forcing the new embeddings to encode the full semantics of the segments they replace. Detailed training hyperparameters are reported in Appendix A.3. Inference. In inference, latent action symbols are processed identically to ordinary vocabulary tokens through the same embedding lookup and transformer forward pass; no expansion or post-processing is required. Latent action decoding therefore introduces zero additional computational overhead, and the token-level compression achieved by reparameterization translates directly into proportional savings in prefill computation, KV-cache memory, and end-to-end inference latency (Table 8). PN PN Reparameterization rate. We quantify the degree of compression as r = i=1 |τ̂i | / i=1 |τi |, where |τi | and |τ̂i | are the token counts of the i-th original and reparameterized trajectories. A smaller r corresponds to higher compression. Because the identification procedure ranks candidates by score(s), segments with the strongest evidence of transition invariance are abstracted first, inducing a natural priority ordering exploited in the progressive abstraction ablation (Section 5.3). 3.6

Applicability and Failure Modes

LAR is effective when the agent’s action space contains a substantial subset of executable latent actions. Tasks with rich structural scaffolding, such as web interaction with protocol-constrained tool invocations or code generation with recurring syntactic patterns, admit larger compressible subsets, whereas reasoning-intensive tasks with diverse free-form content admit smaller ones. Failure mode and concrete instance. Failure arises when abstraction merges segments that are not transition-equivalent: the latent action no longer represents a single behavior, and replacing concrete actions alters the induced transitions. As a concrete example, consider a TriviaQA trajectory where the agent must convey the query “Next British Prime Minister after Arthur Balfour” to a search tool (the same trajectory analyzed in Section 5.4 and Figure 4). Under default thresholds, the query is preserved explicitly because its high next-token entropy exceeds Hmax , but if Hmax is raised aggressively, the entropy filter may admit query-adjacent patterns and replace the query itself with a latent action. The tool interaction then breaks entirely, producing an abrupt rather than gradual performance degradation: a categorical breakdown of environment-facing transitions, characterized empirically as the Phase III collapse in our progressive abstraction ablation (Section 5.3). Prevention by design. LAR addresses this failure mode through prevention at the identification stage rather than runtime fallback. The entropy filter introduced in Section 3.5 excludes high-entropy, parameter-binding segments before they enter the latent action vocabulary, while the frequency filter freq(s) ≥ fmin excludes long-tail patterns lacking sufficient statistical support. Once an incorrect latent action is trained into the model, runtime rollback is difficult, so filtering at the identification stage eliminates the need for runtime mitigation. This makes LAR conservative by default: it improves 5

Table 1: Main results on three representative LLM agent benchmarks. We compare LAR with general prompting baselines and efficiency-oriented methods operating at the token or context level, across two backbone models. Numbers report task performance, with parentheses indicating the relative change in action tokens. Best performance for each backbone and benchmark is highlighted in bold. Backbone

Qwen3-8B

Llama-3.1 8B-Instruct

Method

TriviaQA

KodCode

Mind2Web

Vanilla COT ReAct

67.40 69.43 77.84

34.44 35.10 53.64

36.73 34.15 -

TokenSkip ACON ConciseHint

57.02 (-28.7%) 55.33 (-27.9%) 68.69 (-12.7%)

29.80 (-28.4%) 28.67 (-22.5%) 28.47 (-12.5%)

31.13 (-16.6%) 30.77 (-17.7%) 35.33 (+11.9%)

LAR

80.09 (-27.1%)

54.30 (-9.2%)

39.84 (-2.9%)

Vanilla COT ReAct

73.63 75.50 59.88

31.13 22.52 33.11

24.40 13.85 -

TokenSkip ACON ConciseHint

68.87 (-12.0%) 57.14 (-25.4%) 67.32 (-13.5%)

26.03 (-5.8%) 24.67 (-23.1%) 25.83 (-12.0%)

14.27 (-1.9%) 15.63 (-16.7%) 17.33 (-13.0%)

LAR

72.46 (-23.3%)

35.10 (-9.8%)

28.30 (-20.8%)

efficiency on the common, structurally regular portions of the action space while introducing no risk on the rare, irregular portions. The method therefore does not seek maximal compression but identifies the largest subset of latent actions that preserve transition behavior; the empirical boundary of this subset is characterized in Section 5.3.

4

Main experiment

4.1

Experimental Setup

Backbone Models: We evaluate LAR on two widely used instruction-tuned LLMs: Meta-Llama-3.1-8B-Instruct [12] and Qwen3-8B [39]. These models allow us to assess whether action space reparameterization generalizes across model families. For each model, latent actions are learned exclusively from its own rollout trajectories. Benchmarks: We consider a diverse set of LLM agent benchmarks covering different interaction patterns and action structures. TriviaQA [17] represents multi-step reasoning tasks; KodCode [38] represents code-generation tasks with highly structured action patterns; and Mind2Web [11] represents web-based, tool-using tasks with rich interaction scaffolds. For code benchmarks, latent actions are learned jointly across datasets to evaluate cross-task generalization within a shared action style. Baselines: We choose vanilla LLM agents and ReAct-style agents, as well as representative efficiencyoriented methods operating at different stages of the agent pipeline, including token-level generation control (TokenSkip [37], ConciseHint [31]) and context/memory optimization (ACON [18]). Evaluation Metrics: We report task-specific performance metrics (e.g., accuracy or success rate) and the relative reduction in action token counts compared to the original (Vanilla) trajectories. All methods are evaluated using identical decoding settings and hardware configurations. More experimental information is detailed in Appendix A. 4.2

Main Results: Performance and Efficiency Analysis

Table 1 reports task performance alongside the relative reduction in action tokens (in parentheses, compared to the original Vanilla trajectories). We analyze the results from three perspectives: the accuracy-efficiency trade-off, robustness across interaction regimes and backbones, and overall comparison with baselines. Accuracy-Efficiency Trade-off under Action Reparameterization: As shown in Table 1, LAR consistently achieves favorable accuracy-efficiency tradeoffs across backbones and benchmarks. In most settings, LAR reduces the effective decision horizon, reflected in fewer action tokens, while 6

preserving or improving task success rates. These results suggest that LAR primarily eliminates redundant decision steps rather than semantically critical ones. Unlike TokenSkip, ACON, and ConciseHint, which intervene at the token generation or conditioning stage and may destabilize decision semantics, LAR reparameterizes the action space itself into executable latent units, preserving task-relevant transition behavior. The results also reveal abstraction boundaries. On TriviaQA with Llama-3.1-8B-Instruct, LAR achieves a 23.3% token reduction but incurs a slight accuracy drop versus Vanilla and CoT, suggesting that when structurally redundant action segments are limited, further abstraction approaches the boundary of semantic decision-making rather than indicating instability. Overall, LAR occupies a more favorable region in accuracyefficiency space, supporting the conclusion that the effective decision horizon, rather than token count alone, is the dominant factor governing LLM agent efficiency. Robustness across Heterogeneous Interaction Regimes and Backbone Behaviors: Table 1 further shows that LAR generalizes robustly across benchmarks with distinct interaction structures, reasoningintensive retrieval (TriviaQA), structured code generation (KodCode), and tool-using web interaction (Mind2Web), where the degree of improvement is jointly determined by task-level structural regularity and backbone-specific generation behavior. On KodCode, LAR matches or closely approaches the strongest baselines while reducing the decision horizon by approximately 9 to 10%, suggesting that code-generation tasks contain substantial executable structural redundancy amenable to abstraction. On Mind2Web, performance gains are more strongly modulated by backbone behavior: Qwen3-8B achieves the highest accuracy with modest token reduction, while Llama-3.1-8B-Instruct yields a substantial accuracy improvement and a larger reduction in decision horizon, reflecting differences in the volume of intermediate text generated and, thus, the fraction of action content that can be safely abstracted. Overall, these results indicate that action representation learning serves as a mechanism complementary to model- and system-level optimizations for efficient LLM agent inference. Overall Performance Comparison: Across all benchmarks and backbones, LAR consistently achieves the best or near-best task performance among efficiency-oriented methods, while substantially outperforming existing agentic baselines, which reduce inference cost through token pruning or context compression but frequently incur severe performance degradation on structured tasks like KodCode and Mind2Web. In contrast, LAR maintains strong task performance by operating over a reparameterized latent action space, rather than applying token- or context-level compression, thereby preserving environment-facing transition semantics while eliminating structural redundancy in agent behavior. These results support our central claim that enforcing executability constraints during reparameterization is critical for achieving favorable efficiency-performance trade-offs without sacrificing task success. 4.3

Held-out Benchmark Generalization of Latent Actions

This experiment evaluates whether latent ac- Table 2: Held-out benchmark generalization result. tions learned by LAR capture reusable decision LAR is compared against ReAct under identical structure or merely encode dataset-specific arti- backbone settings. facts. We first introduce another three datasets: Musique [32] for QA tasks, MBPP [3] and Hu- Backbone Method Musique HumanEval MBPP manEval [8] for coding tasks. We regard them ReAct 27.61 89.63 74.17 as a held-out benchmark because we do not use Qwen3-8B LAR 26.57 91.46 75.50 them for any retraining or adaptation. CorreLlama-3.1 ReAct 20.93 56.71 48.40 sponding to this, we regard TriviaQA and Kod- 8B-Instruct LAR 22.35 60.37 46.60 Code as held-in datasets. Specifically, we train latent actions using trajectories collected on held-in benchmarks and directly apply the resulting action reparameterization to other held-out benchmarks. This setup directly tests the core assumption underlying LAR’s design. If latent actions correspond to transition-equivalent and executable decision units (Section 3), they should generalize across benchmarks that share a common action domain even when surface distributions differ. Conversely, if they merely overfit to dataset-specific patterns, their effectiveness should deteriorate when transferred to unseen benchmarks. As shown in Table 2, LAR demonstrates strong held-out benchmark generalization across both backbones. When trained on a held-in dataset and evaluated on a held-out benchmark, LAR consistently matches or outperforms ReAct without task-specific retraining or prompt engineering. This behavior 7

indicates that LAR learns domain-level action abstractions for tasks, rather than benchmark-specific heuristics. In particular, the latent actions appear to encode stable structural behaviors such as code scaffolding and formatting patterns that remain valid across datasets.

5

Mechanism Analysis and Case Study

5.1

Action Equivalence Analysis on Latent Action Reparameterization

LAR achieves performance gains with efficient action reparameterization. However, two questions remain: whether the reparameterized latent action is functionally equivalent to the original action sequence, and whether the observed performance improvements can be mechanistically attributed to the abstraction induced by reparameterization. To eliminate the confounding effect of sequence length reduction at inference, we manually append padding tokens following the reparameterized sequence to match the length of the original action sequence. Table 3 shows the result. LAR-PT per- Table 3: Action equivalence of LAR. LAR-PT denotes forms better than ReAct across all settings, LAR inference with padding tokens. demonstrating that the language model derives performance gains uniquely from acBackbone Method TriviaQA KodCode Mind2Web tion abstraction, independent of sequence ReAct 77.84 53.64 34.15 length reduction. Nevertheless, LAR still Qwen3-8B LAR 80.09 54.30 39.84 marginally outperforms LAR-PT, particuLAR-PT 79.64 53.64 34.40 larly on Mind2Web, indicating that inferReAct 59.88 33.11 24.40 Llama-3.1 ence efficiency from sequence compresLAR 72.46 35.10 28.30 8B-Instruct sion provides an additional, complemenLAR-PT 74.25 34.44 27.15 tary source of performance improvement. 5.2

Learning Stability of Latent Action Reparameterization

Score

Score

To validate the hypothesis that action 85 70 reparameterization induces more con80 60 sistent and stable rollout trajectories, which in turn leads to improved learn50 75 ing stability during policy optimizaLAR LAR tion, we apply GRPO under both the Vanilla Vanilla 40 70 vanilla backbone model and the post0 2k 4k 6k 8k 10k 0 5k 10k 15k 20k 25k 30k Training Data Training Data LAR training model and compare the convergence behavior of the training (a) TriviaQA with Llama (b) TriviaQA with Qwen3 score curves. The detailed experimental settings are in Appendix A.3. Figure 2: These two curves indicate two different characterFigure 2 shows the score curves on istics of LAR on further policy optimization: (a) converges these two models. LAR generally con- faster; (b) consistent learning stability. verges faster and exhibits nearly the same stability during the GRPO training process. It demonstrates that reduced rollout variance induced by reparameterization leads to more homogeneous trajectory distributions, thereby providing a more stable learning signal and enabling faster convergence of the policy optimization process. 5.3

Progressive Abstraction Ablation and the Boundary of Executable Latent Actions

This experiment investigates the question: where the abstraction boundary lies for latent action reparameterization, through a structured ablation over abstraction strength. While previous results show that moderate abstraction can improve efficiency without harming performance, we conduct a progressive abstraction ablation to characterize both the beneficial regime and the failure mode of LAR by gradually increasing the reparameterization rate and observing when task performance begins to degrade. This analysis directly corresponds to the failure modes discussed in Section 3.6. Experimental Design (Progressive Abstraction Ablation): We perform a controlled ablation by progressively increasing the reparameterization rate, defined as the proportion of action segments replaced by latent actions, while keeping the backbone model and decoding settings fixed. This ablation systematically varies the degree of abstraction applied to the agent’s action space, allowing us to trace how performance evolves as abstraction moves from low-entropy structural components toward high-entropy parameterized components. 8

Phase I: Moderate Abstraction (Ablation Regime): In the low-to-moderate abstraction regime of this ablation, performance consistently improves alongside inference efficiency. This regime corresponds to the selective abstraction of low-entropy structural components, such as recurring scaffolds, protocol formats, and stable interaction patterns. Removing these redundant decision steps shortens the effective decision horizon and reduces inference cost, while preserving task-relevant semantics. The observed performance gains in this ablation regime indicate that moderate abstraction removes redundant or noisy action fragments rather than eliminating essential decisions. This trend is visually illustrated in Fig. 3, which shows a consistent performance increase as low-entropy structural components are progressively abstracted. Phase II: Abstraction Boundary: As the reparameterization rate continues to increase, performance peaks, marking the abstraction boundary. At this point, most transition-invariant, executable components have been safely abstracted and further abstraction yields no additional benefit. This peak empirically delin(a) TriviaQA (Qwen3-8B) (b) Mind2Web (Llama-3.1-8B) eates the maximal scope of abstraction that preserves transition equiv- Figure 3: Progressive abstraction ablation results. Task performance as a function of the reparameterization rate for two alence and executability. representative settings. Moderate abstraction improves perforPhase III: Performance Collapse: mance by eliminating low-entropy structural redundancy, while Beyond the abstraction bound- excessive abstraction leads to a sharp performance collapse once ary, performance degrades sharply. high-entropy, parameterized components are abstracted. This collapse occurs when the ablation begins to include high-entropy parameterized components, such as queries, entity references, or task-specific arguments. Abstracting these components violates the executability condition, as latent actions no longer correspond to transition-equivalent behaviors across contexts. Notably, the degradation is abrupt rather than gradual, reflecting a semantic failure mode caused by broken environment-facing transitions rather than insufficient modeling capacity. Cross-Task Consistency: The three-phase behavior along the abstraction ablation axis—performance improvement, peak, and collapse—appears consistently across both Mind2Web and TriviaQA. While the exact location of the abstraction boundary varies with task and backbone, the qualitative trend is shared across domains, suggesting that the abstraction boundary is a structural property of action reparameterization rather than an artifact of a specific task. Method-Level Takeaway: This ablation provides direct empirical support for LAR’s design choice to restrict abstraction to executable latent actions. Rather than aiming for maximal compression, LAR identifies the largest subset of low-entropy, transition-invariant action segments that can be safely abstracted. The observed performance collapse beyond the abstraction boundary validates this restriction and highlights why executability fundamentally constrains useful action representations. 5.4

Case study

To better reveal the characteristics of LAR, we conduct a case analysis on a multi-step reasoning instance from TriviaQA to illustrate how LAR reshapes the agent’s action structure. As shown in Fig. 4, the vanilla agent generates a long sequence of fine-grained textual actions for information retrieval and answer construction, many of which correspond to recurrent structural scaffolds such as reasoning templates and tool invocation formats, rather than task-critical semantic decisions. Under LAR, this trajectory is reformulated into two classes of actions: executable latent actions and explicit high-entropy parameterized components. Low-entropy, recurrent structural patterns are abstracted into latent actions representing transition-equivalent behaviors, while parameter-binding elements that determine task semantics, including the concrete search query and final answer, remain explicit to ensure executability. This case study highlights that LAR shortens the effective decision horizon by abstracting transitioninvariant structure, while preserving correct interaction with external tools by retaining parameter-rich 9

components. Together, these effects illustrate how selective action reparameterization improves inference efficiency without compromising semantic correctness or executability.

6

Conclusion

We introduced Latent Action Reparameterization (LAR), a principled framework that improves LLM-agent efficiency by treating action representation as a first-class modeling choice. By learning executable latent actions that compress recurrent low-entropy structures while preserving highentropy, parameter-binding content, LAR redefines the unit of decision making and directly addresses the inefficiency caused by overly fine-grained action interfaces, achieving favorable performance– efficiency trade-offs across diverse tasks and backbone models.

References [1] Mostafa Al-Emran. Hierarchical reinforcement learning: a survey. International journal of computing and digital systems, 4(02), 2015. [2] Christopher Amato, George Konidaris, Leslie P Kaelbling, and Jonathan P How. Modeling and planning with macro-actions in decentralized pomdps. Journal of Artificial Intelligence Research, 64:817–859, 2019. [3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. [4] Pierre-Luc Bacon, Jean Harb, and Doina Precup. The option-critic architecture. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017. [5] Yuxuan Cai, Xiaozhuan Liang, Xinghua Wang, Jin Ma, Haijin Liang, Jinwen Luo, Xinyu Zuo, Lisheng Duan, Yuyang Yin, and Xi Chen. Fastmtp: Accelerating llm inference with enhanced multi-token prediction. arXiv preprint arXiv:2509.18362, 2025. [6] Dingyang Chen, Qi Zhang, and Yinglun Zhu. Efficient sequential decision making with large language models. arXiv preprint arXiv:2406.12125, 2024. [7] Hao Mark Chen, Wayne Luk, Ka Fai Cedric Yiu, Rui Li, Konstantin Mishchenko, Stylianos I Venieris, and Hongxiang Fan. Hardware-aware parallel prompt decoding for memory-efficient acceleration of llm inference. arXiv preprint arXiv:2405.18628, 2024. [8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. [9] Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025. [10] Tonmoy Debnath, Md Nurul Absar Siddiky, Muhammad Enayetur Rahman, Prosenjit Das, Antu Kumar Guha, Muhammad Rezaur Rahman, and HM Kabir. A comprehensive survey of prompt engineering techniques in large language models. TechRxiv, 2025. [11] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36:28091–28114, 2023. [12] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407, 2024. [13] In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference. Proceedings of Machine Learning and Systems, 6:325–338, 2024. 10

[14] Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito, Kiril Gashteovski, David Friede, Roberto Bifulco, and Carolin Lawrence. Agentquest: A modular benchmark framework to measure progress and improve llm agents. arXiv preprint arXiv:2404.06411, 2024. [15] Gonzalo Gonzalez-Pumariega, Leong Su Yean, Neha Sunkara, and Sanjiban Choudhury. Robotouille: An asynchronous planning benchmark for llm agents. arXiv preprint arXiv:2502.05227, 2025. [16] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. [17] Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017. [18] Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan. Acon: Optimizing context compression for long-horizon llm agents. arXiv preprint arXiv:2510.00615, 2025. [19] Jeonghye Kim, Sojeong Rhee, Minbeom Kim, Dohyung Kim, Sangmook Lee, Youngchul Sung, and Kyomin Jung. Reflact: World-grounded decision making in llm agents via goal-state reflection. arXiv preprint arXiv:2505.15182, 2025. [20] Minbeom Kim, Hwanhee Lee, Kang Min Yoo, Joonsuk Park, Hwaran Lee, and Kyomin Jung. Critic-guided decoding for controlled text generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4598–4612, 2023. [21] Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR, 2023. [22] Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36:62630–62656, 2023. [23] Jiayu Liu, Cheng Qian, Zhaochen Su, Qing Zong, Shijue Huang, Bingxiang He, and Yi R Fung. Costbench: Evaluating multi-turn cost-optimal planning and adaptation in dynamic environments for llm tool-use agents. arXiv preprint arXiv:2511.02734, 2025. [24] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688, 2023. [25] Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. Memgpt: Towards llms as operating systems. 2023. [26] Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023. [27] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36: 68539–68551, 2023. [28] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024. [29] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023. 11

[30] Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. Distilling reasoning capabilities into smaller language models. Findings of the Association for Computational Linguistics: ACL 2023, pages 7059–7073, 2023. [31] Siao Tang, Xinyin Ma, Gongfan Fang, and Xinchao Wang. Concisehint: Boosting efficient reasoning via continuous concise hints during generation. arXiv preprint arXiv:2506.18810, 2025. [32] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554, 2022. [33] Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, Jiachen Liu, Zhongnan Qu, Shen Yan, Yi Zhu, Quanlu Zhang, et al. Efficient large language models: A survey. arXiv preprint arXiv:2312.03863, 2023. [34] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024. [35] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. [36] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. [37] Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067, 2025. [38] Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode: A diverse, challenging, and verifiable synthetic dataset for coding. arXiv preprint arXiv:2503.02951, 2025. [39] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [40] Cheng Yang, Jiaxuan Lu, Haiyuan Wan, Junchi Yu, and Feiwei Qin. From what to why: A multi-agent system for evidence-based chemical reaction condition reasoning. arXiv preprint arXiv:2509.23768, 2025. [41] Ruihan Yang, Yikai Zhang, Aili Chen, Xintao Wang, Siyu Yuan, Jiangjie Chen, Deqing Yang, and Yanghua Xiao. Aria: Training language agents with intention-driven reward aggregation. arXiv preprint arXiv:2506.00539, 2025. [42] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022. [43] Xinlei Yu, Chengming Xu, Zhangquan Chen, Bo Yin, Cheng Yang, Yongbo He, Yihao Hu, Jiangning Zhang, Cheng Tan, Xiaobin Hu, et al. Dual latent memory for visual multi-agent system. arXiv preprint arXiv:2602.00471, 2026. [44] Yuanzhao Zhai, Tingkai Yang, Kele Xu, Dawei Feng, Cheng Yang, Bo Ding, and Huaimin Wang. Enhancing decision-making for llm agents via step-level q-value models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27161–27169, 2025. [45] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, et al. Aflow: Automating agentic workflow generation. arXiv preprint arXiv:2410.10762, 2024. 12

[46] Jiayi Zhang, Yiran Peng, Fanqi Kong, Yang Cheng, Yifan Wu, Zhaoyang Yu, Jinyu Xiang, Jianhao Ruan, Jinlin Wang, Maojia Song, et al. Autoenv: Automated environments for measuring cross-environment agent learning. arXiv preprint arXiv:2511.19304, 2025. [47] Jiayi Zhang, Yongfeng Gu, Jianhao Ruan, Maojia Song, Yiran Peng, Zhiguang Han, Jinyu Xiang, Zhitao Wang, Caiyin Yang, Yixi Ouyang, Bang Liu, Chenglin Wu, and Yuyu Luo. Harnessing agentic evolution. arXiv preprint arXiv:2605.13821, 2026. [48] Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems, 43(6):1–47, 2025. [49] Ruijie Zheng, Ching-An Cheng, Hal Daumé III, Furong Huang, and Andrey Kolobov. Prise: Llm-style sequence compression for learning temporal action abstractions in control. arXiv preprint arXiv:2402.10450, 2024. [50] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625, 2022. [51] Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, et al. A survey on efficient inference for large language models, 2024. URL https://arxiv. org/abs/2404.14294, 2024.

A

Detailed Experimental Setup and Design Rationale

A.1

Agent Models and Training Protocol

We conduct experiments using Meta-Llama-3.1-8B-Instruct [12] and Qwen3-8B [39] to ensure that observed efficiency gains are not specific to a single model family. No architectural modifications are applied to the base models: the attention mechanisms, layer counts, and hidden dimensions remain unchanged. To integrate latent action symbols into the model, we employ parameter-efficient adaptation via LoRA (rank r = 8, scaling factor α = 16, applied a to the query, key, value, and output projection matrices) together with newly added embedding and output-head entries for the latent action vocabulary Z. Only the LoRA adapter weights and the new latent action embeddings are trainable, amounting to approximately 0.1% of total model parameters; full optimization details are summarized in Appendix A.3. All original pretrained weights, including all pre-existing token embeddings, remain frozen throughout training. This is parameter-efficient adaptation rather than full fine-tuning, and the original base model is preserved without modification. Latent action vocabularies are constructed separately for each model using trajectories generated by the same model. This ensures that reparameterization does not rely on cross-model transfer of action representations and that the learned latent actions reflect the generation behavior of the specific backbone. Concrete configurations and the resulting vocabulary sizes for each (model, benchmark) pair are reported in Appendix A.4. A.2

Latent Action Identification Algorithm

Algorithm 1 provides the full segment identification procedure summarized in the main text. The algorithm extracts word-level n-grams within sentence boundaries, applies frequency and entropy filters that operationalize the transition equivalence condition, ranks candidates by a composite score that balances coverage and behavioral stability, and removes redundant entries so that each retained latent action is maximally informative. We use the overlap threshold ρ = 0.7 across all benchmarks. Per-benchmark settings of fmin , Hmax , [nlo , nhi ], and K are reported in Appendix A.4. A.3

Reproducibility and Implementation Details

Hardware and decoding. All models and baselines are trained and evaluated on servers equipped with 8×H200 140GB GPUs. We use vLLM for inference with temperature T = 0 for deterministic decoding. All experiments are conducted with fixed random seeds. 13

Algorithm 1 Latent Action Identification Require: Trajectory dataset D = {τ1 , . . . , τN }; minimum frequency fmin ; maximum entropy Hmax ; n-gram size range [nlo , nhi ]; capacity K; overlap threshold ρ Ensure: Latent action set Z 1: C ← ∅ ▷ candidate segments 2: for each trajectory τ ∈ D do 3: Extract all word-level n-grams of size n ∈ [nlo , nhi ] within sentence boundaries 4: C ← C ∪ {extracted n-grams} 5: end for 6: Cfreq ← {s ∈ C | freq(s) ≥ fmin } ▷ frequency filter 7: for each s ∈ Cfreq do 8: Compute the next-token entropy H(s) as defined in the main text 9: end for 10: Cent ← {s ∈ Cfreq | H(s) ≤ Hmax } ▷ entropy filter 11: Rank Cent by score(s) = freq(s)/(H(s) + 1) in descending order 12: Z ← ∅ 13: for s in ranked Cent do 14: if s is not a substring of any s′ ∈ Z and overlap(s, s′ ) < ρ for all s′ ∈ Z then 15: Z ← Z ∪ {s} 16: end if 17: if |Z| = K then 18: break 19: end if 20: end for 21: return Z

LAR Training configuration. LAR training follows the trajectory-level distillation procedure described in Section 3.5, with the parameter-efficient adaptation setup (LoRA plus newly added latent action embeddings, all original weights frozen) summarized in Appendix A.1. The full optimization configuration is summarized in Table 4. We use a pure KL distillation objective (λ = 1.0) computed over shared content positions M , with distillation temperature τ = 2.0.

Table 4: LAR training configuration. Component

Value

Optimizer Effective batch size Learning rate LR schedule Warmup steps Weight decay Epochs Precision Distillation temperature τ KL weight λ LoRA rank r LoRA scaling α LoRA target modules Hardware

AdamW 16 (via gradient accumulation) 1 × 10−4 Cosine 100 0.01 3 fp16 2.0 1.0 8 16 q, k, v, o projections 8× H200 140GB GPUs

Trajectory data. Latent actions are identified from agent rollout trajectories generated by each backbone on the corresponding benchmark training split. The amount of trajectory data used for identification and distillation is benchmark-dependent; for example, on KodCode, we sample 20K trajectories from the full 447K dataset for post-training, balancing identification quality against compute cost. Per-benchmark data sizes and corresponding latent action vocabulary configurations are reported in Appendix A.4. 14

Table 5: Main training hyperparameters for GRPO on TriviaQA and KodCode. Parameter

TriviaQA

KodCode

Train batch size Validation batch size Learning rate LR warmup ratio Max prompt length Max response length Max observation length Rollout agents Max turns KL loss coef

256 128 1 × 10−6 0.285 4096 1024 1024 4 4 0.001

512 128 1 × 10−6 0.285 4096 8192 2048 8 10 0.001

GRPO Training Setting for Learning Stability Analysis. The code for GRPO training is adapted from Search-R1 4 , which is established on VeRL framework [28]. We conduct experiments on TriviaQA and KodCode to support the learning stability analysis in Section 5.2. The detailed training settings are in Table 5. Source code. Complete source code, including the segment identification pipeline, training scripts, and evaluation harness, is publicly released at the anonymous repository linked in the abstract. A.4

Per-Task LAR Configuration

The latent action identification pipeline (Algorithm 1) is configured per benchmark to reflect differences in action structure: tasks with rich, recurring interaction scaffolds admit smaller frequency thresholds and larger compressible fractions, whereas reasoning-intensive tasks with diverse free-form content require stricter frequency thresholds to ensure that only stable patterns are abstracted. Table 6 summarizes the configuration used for each benchmark in our experiments. Table 6: Per-benchmark LAR configuration. fmin is the minimum frequency threshold; Hmax is the maximum next-token entropy threshold; [nlo , nhi ] denotes the n-gram size range; K is the latent action capacity. “#Latent actions” reports the resulting vocabulary size after frequency, entropy, and redundancy filters; “Avg. words/action” reports the average segment length across the retained latent actions. Benchmark

n-gram range

fmin

Hmax

# Latent actions

Avg. words/action

TriviaQA KodCode Mind2Web

[3, 5] [2, 6] [2, 6]

2000 10 1000

10.0 10.0 10.0

200–1000 ∼100 ∼100

3–5 2–6 2–6

Configuration rationale. The per-benchmark differences reflect characteristics of the action structure observed in each domain. TriviaQA combines free-form multi-step reasoning with retrieval calls, producing many recurring reasoning templates and tool-invocation scaffolds; we therefore use a wider n-gram range (3–5) and a high frequency threshold (fmin = 2000) to retain only stable, broadly applicable patterns. KodCode contains highly structured code-generation trajectories with consistent syntactic and protocol scaffolds shared across diverse problems; the large pool of stable patterns allows a lower frequency threshold (fmin = 10) while still ensuring statistical reliability, and the resulting ∼100 latent actions transfer directly to HumanEval and MBPP without further identification, as reported in Section 4.3. Mind2Web involves protocol-constrained tool invocations and substantial HTML scaffolding; we additionally extract recurring HTML tag sequences alongside textual n-grams to capture the protocol-level repetitions characteristic of web interaction. The redundancy removal threshold ρ = 0.7 is held constant across benchmarks, as is the entropy threshold Hmax = 10.0. The latter is set conservatively to ensure that high-entropy parameter-binding content (search queries, entity names, code identifiers) is reliably preserved in the explicit output space. 4 https://github.com/PeterGriffinJin/Search-R1

15

Trajectory data sizes. For latent action identification (Appendix A.2) and distillation (Appendix A.3), we sample agent rollout trajectories from each benchmark’s training split. Concretely, we use 20K trajectories sampled from the 447K full KodCode training set, and analogous sampling for TriviaQA and Mind2Web (sized to provide reliable frequency and entropy estimates while keeping post-training cost modest). HumanEval and MBPP are evaluated in a strict zero-shot transfer setting using the latent action vocabulary learned exclusively from KodCode. A.5

Benchmark Selection and Task Categorization

Benchmarks are selected to span distinct agent behaviors and action structures. QA benchmarks (TriviaQA, Musique) emphasizes multi-step reasoning with relatively low structural repetition. Code benchmarks (HumanEval, MBPP, KodCode) exhibit strong syntactic regularities and recurring generation patterns, making them suitable for studying reusable semantic action units. Mind2Web involves complex tool interactions with extensive protocol-level scaffolds and repeated system-level configurations, providing a setting where executable action abstraction is particularly impactful. A.6

Latent Action Learning for Held-out Benchmarks: Source and Transfer Setting

We evaluate the generalization of LAR on two domains of tasks. For the QA domain, latent actions are identified and trained from TriviaQA trajectories alone, while for the coding domain, latent actions are identified and trained from KodCode trajectories, following the procedure in Appendix A.2 and Appendix A.3. The resulting vocabulary and the trained embeddings are then directly applied to Musique (HumanEval and MBPP for code) without any additional retraining, adaptation, or prompt engineering. This corresponds to a held-out benchmark transfer setting, in which the latent actions are never exposed to the evaluated benchmark. This setup is designed to test whether latent actions capture domain-level structural regularities, including function scaffolds, formatting conventions, and invocation patterns shared across different tasks, rather than dataset-specific artifacts. The corresponding empirical results are reported in our held-out benchmark generalization experiments. A.7

Baseline Methods

Baselines are chosen to represent distinct efficiency paradigms: • Vanilla LLM agents, which operate directly over token-level actions; • ReAct-style agents, which interleave reasoning and acting; • Token-level efficiency methods, which regulate generation dynamics (TokenSkip, ConciseHint); • Context and memory optimization methods, which compress interaction histories (ACON). All baselines preserve the original decision interface and do not alter the action space representation. A.8

Baseline Evaluation • Vanilla LLM agents, which operate directly over token-level actions; • ReAct-style agents, which interleave reasoning and acting; • TokenSkip is adapted to the COT prompt template of ours, and the cutoff lengths for LoRA adapter training are set to 4096 for TriviaQA and Mind2Web, and 8192 for KodCode; the compression ratio is set to 0.7, while the other settings are left unchanged. • ACON was modified to use Qwen3-8B as both the compressor and generator, where we set the maximum generated tokens to 8192, and we tested n cases for the benchmarks. • Context and memory optimization methods, which compress interaction histories (ACON).

16

Table 7: The generalization experiment on LAR. LAR-U denotes the unified model trained on the dataset including trajectories from three domains. Numbers report task performance, with parentheses indicating the relative change in action tokens.

A.9

Backbone

Method

TriviaQA

KodCode

Mind2Web

Qwen3-8B

LAR LAR-U

80.09 (-27.3%) 76.05 (-12.3%)

58.28 (-9.2%) 62.91 (-4.1%)

39.84 (-2.9%) 31.18 (+11.5%)

Llama-3.1 8B-Instruct

LAR LAR-U

72.46 (-23.3%) 76.05 (-24.3%)

35.10 (-9.8%) 38.41 (-3.8%)

28.30 (-20.8%) 26.89 (-18.1%)

Generalization across different domains

To demonstrate LAR’s generalizability across different domains, we merge all trajectories collected from TriviaQA, KodCode, and Mind2Web into a unified training corpus and fine-tune a single model jointly across all three domains without any domain-specific adaptation. The unified model is then evaluated on each benchmark independently to assess whether a single set of reparameterization tokens can effectively compress and represent action sequences across heterogeneous task distributions. Table 7 shows that our method maintains comparable performance across different tasks and domains. This result can be explained from two perspectives. On one hand, combining trajectories from diverse domains exposes the model to a richer set of reasoning patterns, thereby enhancing its general problem-solving capability. This effect is particularly pronounced for the Llama model, which benefits more substantially from the increased data diversity to strengthen its underlying reasoning ability. On the other hand, certain tasks, such as Mind2Web exhibit a moderate performance degradation under the unified setting, which can be attributed to the fact that the reparameterization tokens, when trained across heterogeneous domains, may fail to capture domain-specific structural patterns, leading to a reduced compression effectiveness and a less compact latent action representation for domain-specialized tasks. Besides, we notice that the compression rate in five of the six settings decreases, which suggests that the unified training setting generally leads to a less aggressive compression compared to domainspecific LAR models. This is expected, as the reparameterization tokens must now encode a more heterogeneous action space spanning across QA, coding, and web navigation tasks. When trained on a single domain, the reparameterization tokens can specialize in capturing the recurring structural patterns and action primitives specific to that domain, enabling a more compact and efficient latent representation. In contrast, the unified model must learn a shared token space that accommodates the diverse action vocabularies of all three domains simultaneously, which inevitably dilutes the domain-specific compression signal and results in a more conservative encoding strategy.

A.10

Scalability of Latent Action Reparameterization

To further demonstrate the scalability of our proposed LAR, so that it can easily be adopted in a larger model, we transfer the whole pipeline to Qwen3-32B [39] with TriviaQA. Compared with the original LAR training setting in Table 4, we set LoRA rank r = 16, LoRA scaling α = 32, learning rate lr = 5 × 10−5 , and training epoch E = 2. On testing, Qwen3-32B with ReAct framework has an accuracy of 73.20%, while Qwen3-32B with LAR achieves an accuracy of 75.26%. The experimental results demonstrate that LAR scales effectively to larger models, yielding a consistent performance improvement over the ReAct baseline. This gain is comparable in magnitude to those observed in 7B-model settings, suggesting that the benefit of action reparameterization is robust across model scales rather than being confined to a particular parameter regime. These findings indicate that LAR constitutes a general and modelagnostic framework, whose advantages persist as model capacity increases, underscoring its practical applicability in large-scale deployment scenarios. 17

Table 8: System-level efficiency measurements on three benchmarks. TT: Token Throughput (tokens/s); AGU: Avg GPU Utilization (%), PG: Peak GPU Memory (GB). Model

TriviaQA

Method

KodCode

Mind2Web

TT

AGU

PG

TT

AGU

PG

TT

AGU

PG

Qwen

ReAct LAR

127.8 150.2

71.1 58.5

75.3 144.5 73.1 146.1

92.9 92.5

69.6 152.4 69.9 153.9

83.6 80.2

51.2 47.7

Llama

ReAct LAR

151.6 152.2

56.0 54.3

42.5 140.7 42.4 151.3

70.7 69.9

70.1 119.0 70.7 121.4

16.7 11.9

69.8 69.4

Table 9: The experimental results for the three Mind2Web sub-test sets.

A.11

Backbone

Method Cross-Task Cross-Website Cross-Domain

Qwen3-8B

Vanilla LAR

39.56 45.05

17.58 26.37

30.69 35.64

Llama-3.1 Vanilla 8B-Instruct LAR

22.76 27.64

8.13 11.38

22.44 26.69

Metrics and Evaluation Protocol

In addition to task performance, we measure efficiency at both the decision and system levels. The effective decision horizon is defined as the total number of explicit generation decisions, aligning with the formal definition in Section 3. Wall-clock inference time is measured under identical hardware, decoding parameters, and maximum context lengths for all methods. Token reduction is reported as an outcome of action reparameterization rather than an explicit optimization objective. Table 8 shows the three measurements related to inference cost in the hardware aspect. It shows that LAR not only accelerates the token throughput but also slightly reduces the GPU memory usage. This is because latent action decoding introduces zero additional computational overhead at inference time: latent action symbols are standard vocabulary tokens processed through the same embedding lookup and transformer forward pass as any other token. A.12

Detailed experimental result of Mind2Web

Mind2Web provides a unique opportunity to evaluate generalizability at three different levels: cross domains, cross websites, and cross tasks. The experimental result is shown in Table 9, which shows that LAR can effectively compress redundant HTML context and help LLM-based agents generate more precise actions. A.13

Detailed Case Analysis and Trajectory Example

Figure 4 displays a complete trajectory on a TriviaQA task. The task requires the agent to identify a famous person based on a description. The Vanilla agent follows a standard ReAct-style approach, generating a sequence of thoughts, tool actions, and a final answer. This process involves generating a large number of tokens, many of which are structural and serve only to format the interaction. LAR reparameterizes this trajectory by identifying and abstracting these recurrent structural patterns into latent actions. As shown in the figure, the lengthy sequence of tokens corresponding to the search action is compressed into a single latent token. Crucially, the high-entropy content, the search query “Next British Prime Minister after Arthur Balfour”, is preserved explicitly to maintain executability. This transformation results in a significantly shorter effective decision horizon. The Vanilla trajectory requires many steps to express the intent, whereas the LAR trajectory expresses the same high-level decisions in far fewer steps. By operating over these latent actions, the agent can plan and execute at a higher level of abstraction, reducing computational cost while preserving the integrity of the interaction with the environment and the final output. This example highlights how LAR selectively compresses structural redundancy while maintaining the necessary granularity for effective task performance. 18

System prompt [compress] assistant. TOOL SPECIFICATION: To use [compress] tags: <search> query </search> ... OUTPUT FORMAT: [compress]: 1. <think> [reasoning about [compress] </think> 2. ... Wait for <observation> ... CONSTRAINTS: [compress]...

Question Who was the next British Prime Minister after Arthur Balfour? <think> [compress] who the next British Prime Minister was after Arthur Balfour. Let [compress] British Prime Ministers..., [compress]. Wait, maybe it was someone else. Let me check. Wait, after Balfour, the next PM was Henry Campbell-Bannerman, right? He was from the Liberal Party. But I should verify the dates. Balfour left in 1905, and Campbell-Bannerman took over... So the immediate successor to Balfour would be Campbell-Bannerman. [compress] this. Let me [compress] make sure. </think> <search>Next British Prime Minister after Arthur Balfour</search>

Doc 1(Title: "Arthur Balfour") Freedom of the City/Freedom of the Borough of Arthur Balfour Arthur James Balfour, ... Doc 2(Title: "Arthur Balfour") Arthur Balfour Arthur James Balfour... Doc 3(Title: "David Lloyd George") David Lloyd George David Lloyd George...

… <think> Okay, let‘s see. [compress] was the next British Prime Minister after Arthur Balfour. From the previous search results... [compress] that his Conservative government fell in December 1905, and then the Liberals under Henry Campbell-Bannerman formed a government... [compress] any conflicting info, but the third document clearly states that the Liberals under Campbell-Bannerman formed the government after Balfour's fall. Therefore, the next PM is Henry Campbell-Bannerman. </think> <answer>Henry Campbell-Bannerman</answer>

Figure 4: Case analysis of LAR on a TriviaQA example. LAR abstracts low-entropy structural components into executable latent actions while preserving high-entropy parameterized content (e.g., the search query), reducing the effective decision horizon without altering task execution.

A.14

Transferability to Industrial-Grade Agent Frameworks: A Case Study on OpenClaw

Our main benchmarks (TriviaQA, KodCode, Mind2Web) are designed for controlled scientific evaluation, but real-world LLM agents are typically deployed through industrial-grade frameworks such as OpenClaw, LangChain, and Claude Code. These frameworks embed extensive static scaffolding (tool specifications, output-format constraints, role descriptions, recurring protocol templates) into their system prompts, exhibiting exactly the structural profile LAR is designed to compress: high frequency, low next-token entropy, and weak coupling to task-specific parameters. We therefore test whether LAR transfers to such deployment-grade environments without modifying the agent framework itself. We instantiate this evaluation on OpenClaw, an open-source autonomous agent runtime. The LAR training pipeline is identical to that used in our main experiments (Section 3): latent actions are identified from TriviaQA rollout trajectories generated under OpenClaw, and a LoRA adapter together with new latent action embeddings is trained via trajectory-level distillation. At deployment, the learned latent tokens replace stable spans of OpenClaw’s static system prompt; OpenClaw’s runtime logic, tool interfaces, and ReAct loop remain unchanged. No framework-level code modification is required. We evaluate on TriviaQA under the OpenClaw runtime and report two quantities. Compression Rate is the proportion of OpenClaw’s static system-prompt tokens replaced by latent action tokens. Exact Match (EM) is the standard TriviaQA accuracy metric, measuring the fraction of agent answers matching the reference. The five settings differ only in how much of the static prompt is reparameterized: Vanilla preserves the original prompt; Short, Medium, and Long replace progressively larger contiguous spans of the static scaffolding (boilerplate, tool-format descriptions, and constraint blocks, in increasing order of semantic load); AllStatic replaces the entire static portion. The “vs. Vanilla” column reports the absolute and relative EM improvement over the uncompressed baseline. Table 10 shows that even the most conservative Short setting, compressing only 6.7% of the static prompt, raises EM from 0.4218 to 0.5358 (a 27.0% relative improvement). The gain is obtained purely through prompt-level reparameterization, without altering OpenClaw’s tool interface, ReAct loop, or runtime, demonstrating that LAR functions as a plug-in optimization layer decoupled from the underlying agent framework. The OpenClaw Vanilla EM (0.4218) is also markedly lower than the TriviaQA accuracy in Table 1 for the same backbone under a benchmark-native prompt, reflecting how deployment-oriented frameworks dilute task-relevant signal with structural overhead. 19

Table 10: TriviaQA exact-match (EM) results of LAR applied to the OpenClaw industrial agent runtime. Compression Rate is the fraction of OpenClaw’s static system-prompt tokens replaced by latent action tokens. Settings range from Vanilla (no compression) to AllStatic (full static-prompt replacement). LAR yields its largest gains under conservative compression (Short) and saturates as compression encroaches on parameter-binding content, mirroring the abstraction boundary in Section 5.3. Setting

Compression Rate

EM

vs. Vanilla

Vanilla Short Medium Long AllStatic

0.0% 6.7% 15.2% 24.7% 45.3%

0.4218 0.5358 0.4672 0.4668 0.4308

– +0.1140 (+27.0%) +0.0454 (+10.8%) +0.0450 (+10.7%) +0.0090 (+2.1%)

LAR’s improvement on OpenClaw is correspondingly larger, supporting the design hypothesis from Section 3: the more low-entropy structural redundancy a prompt contains, the more LAR can recover by reparameterizing it into executable latent tokens. As compression deepens, gains diminish. Medium and Long retain +10.8% and +10.7% over vanilla, but their marginal benefit shrinks as more semantically loaded content is absorbed, and AllStatic at 45.3% compression retains only a 2.1% gain. This trajectory aligns with the Phase III collapse characterized in Section 5.3: once compression encroaches on segments carrying task-relevant binding information, executability is violated, and gains erode. The OpenClaw experiment therefore demonstrates that LAR can be deployed to industrial agent runtimes as a drop-in prompt-level replacement without modifying the framework itself, and also empirically reproduces the abstraction boundary predicted by our framework in a different deployment regime.

20