TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies Dong Jing13∗ , Jingchen Nie2∗ , Tianqi Zhang3∗ , Jiaqi Liu3 , Huaxiu Yao3 , Zhiwu Lu1† , Mingyu Ding3† 1 RUC, 2 FDU, 3 UNC {jingdong98, luzhiwu}@ruc.edu.cn, [email protected]
arXiv:2606.06491v1 [cs.RO] 4 Jun 2026
∗
Equal contribution † Corresponding authors
(a) Variable-Speed Trajectory Augmentation
(b) Speed-Controllable VLA Policy
(c) 10-s Motion Trails Across Six Speeds
Slow (s = 0.5×) observations
split
Normal (s = 1×)
task
VLA Policy speed s
actions action magnitude controlled by s
merge
Fast (s = 2×)
s = 0.5×
s = 0.75×
s = 1×
s = 1.25×
s = 1.5×
s = 2×
Low-level Controller execution
0.5×
0.75×
1×
1.25×
1.5×
Figure 1: TempoVLA: a speed-controllable VLA framework. (a) VSTA re-times any demonstration to a target speed by selectively merging consecutive actions to speed up or splitting them to slow down, while preserving motion semantics. (b) The policy takes a scalar speed s as an explicit conditioning input that scales the magnitude of its predicted actions, with the low-level controller left unchanged. (c) For a fixed task, the rollout motion trails of one TempoVLA policy at six commanded speeds tighten under slow commands and stretch under fast ones.
Abstract: Robot manipulation alternates between low-risk transit phases that call for fast execution and high-risk contact stages that demand slow, precise motion. Yet existing Vision-Language-Action models (VLAs) only inherit a single fixed speed from training demonstrations. Prior efforts to accelerate VLAs through model compression, KV-cache reuse, or reinforcement learning only shift the policy from one fixed speed to another, and leave deceleration almost unexplored. We observe that the magnitude of each predicted action already governs how fast the robot moves, opening a direct route to controllable execution speed. We turn this observation into TempoVLA, a single VLA whose execution speed is controlled by an explicit condition. TempoVLA combines two coupled components. (1) A data-side Variable-Speed Trajectory Augmentation (VSTA) that re-times demonstration to any target speed by merging or splitting actions while preserving its motion semantics. (2) A model-side conditioning mechanism that feeds the speed to the policy. Statistics show that VSTA reaches the requested speed with negligible motion error. Experiments in simulation and on real-world tasks demonstrate that TempoVLA achieves flexible speed control in both directions, while VSTA additionally boosts the default 1× performance via better data utilization. Furthermore, by cooperating with a large multimodal model, TempoVLA realizes dynamic speed control, accelerating through low-risk phases and decelerating for high-risk ones. Keywords: Vision-Language-Action Model, Robot Manipulation, Speed Control, Data Augmentation
1
Introduction
Vision-Language-Action models (VLAs) have emerged as a mainstream paradigm for generalpurpose robot manipulation [1, 2, 3, 4, 5, 6]. By training large vision-language backbones on robot demonstrations, VLAs follow language instructions and act across diverse embodied platforms, from robot arms to quadrupeds and humanoids. A core but under-controlled dimension in deploying these policies is the execution speed. Real manipulation alternates between low-risk transit phases that should run fast and high-risk contact phases that should slow down for precision. Yet today’s VLAs silently inherit a single fixed execution speed from their training demonstrations. Existing efforts to alter this speed sit at the inference or controller side, accelerating policies through model compression, KV-cache reuse, asynchronous action chunking, or Reinforcement-Learning (RL) rollouts [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. However, these methods merely shift the policy from one fixed speed to another rather than offering explicit, on-demand speed control. They also focus exclusively on acceleration, while deceleration, which remains essential for precision insertion, fragile handover, and other contact-rich behaviors, receives little attention. The open challenge is therefore to give a single VLA explicit, bidirectional speed control without retraining its base architecture from scratch. We observe that the magnitude of each predicted action already governs how fast the robot moves in the embodied setting, which opens a direct route to controllable execution speed. As shown in Figure 1, building on this insight, we approach the problem from two coupled sides while leaving the low-level controller untouched. On the data side, we introduce Variable-Speed Trajectory Augmentation (VSTA), an online strategy that re-times any existing demonstration to any target speed by merging consecutive actions into fewer, larger ones to speed up, or splitting actions into more, smaller ones to slow down, while preserving the underlying motion semantics (Figure 1 (a)). On the model side, we feed the speed s to the policy as an explicit conditioning input that scales its predicted action magnitude through three different injection schemes (Figure 1 (b)). Both our data-side and model-side designs are lightweight and applicable to all existing VLAs. Figure 1 (c) previews the resulting behavior: a single policy trained with our method can execute the task at multiple different commanded speeds, with the motion trail tightening at slow speeds and stretching at fast ones. Experiments on LIBERO and on real-world tasks confirm that this control extends in both directions, and that VSTA additionally acts as useful data augmentation that improves default 1× performance. We further show that pairing the speed-conditioned policy with a Vision-Language Model (VLM) enables automated dynamic speed scheduling and boosts better performance, where the system accelerates through low-risk phases and decelerates for high-risk ones without human intervention. In summary, our contribution is threefold. 1. We propose VSTA together with speed conditioning, a lightweight data-and-model pair that equips existing VLAs with bidirectional speed control without new data collection. 2. We find that with properly re-timed data, speed control is easy to implant and largely independent of the conditioning mechanism, and that variable-speed training acts as an effective augmentation that consistently lifts the default 1x success rate in simulation and the real world. 3. We demonstrate that this design extends to VLM-driven dynamic speed scheduling, turning execution speed into a new control channel for higher-level reasoners.
2
Related Work
Vision-Language-Action Models. Vision-Language-Action models (VLAs) map visual observations and language instructions to executable robot actions [20, 1, 2, 3, 4, 5, 6, 21, 22, 8, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36]. They are trained at scale by imitating large collections of teleoperated demonstrations, supported by a growing ecosystem of embodied datasets and manipulation benchmarks [37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]. Across this landscape, the action decoder broadly falls into three families: regression heads that emit continuous actions directly, as 2
in ACT [48]; diffusion or flow-matching heads that model the action distribution generatively, as in Diffusion Policy [49], ALOHA Unleashed [50], and π0 [3]; and discrete token heads that autoregressively decode action tokens, as in RT-2 [1] and OpenVLA [2]. Yet regardless of decoder family, the execution speed of a trained VLA is silently inherited from its demonstration data, which becomes a bottleneck when a single task contains phases that call for different motion paces. Model-based VLA Acceleration. A first line of work makes VLAs faster by intervening inside the policy itself. Model compression shrinks the policy footprint to reduce per-step inference cost, as in TinyVLA [7], SmolVLA [8], and EfficientVLA [9]. Token and KV-cache compression accelerates the language backbone or the action head, such as the FAST action tokenizer for π0 [10] and the parallel-decoding fine-tune of OpenVLA-OFT [11]. Asynchronous chunked execution hides inference latency by overlapping prediction with execution, as in Real-Time Chunking [12], while related work trains flow policies to produce smoother chunk-boundary continuations that remove the discontinuities at chunk transitions [13]. Reinforcement-learning fine-tuning retrains the policy with task rewards to encourage faster, more decisive behavior [14]. A complementary line manipulates the demonstration tempo itself, including DemoSpeedup [51], SpeedAug [52], ESPADA [53], and SAIL [54]. However, none of these methods expose execution speed as an explicit, on-demand control; they at best shift the policy from one fixed speed to another. Deceleration, in particular, is left almost entirely unaddressed. Model-free VLA Acceleration. A complementary line operates strictly downstream of the policy, tuning the robot-side execution stack for stable and rapid motion without touching policy weights. Classical and GPU-accelerated motion planners such as CHOMP [55], Riemannian motion policies [56], and cuRoBo [57] produce smoother, faster trajectory tracking. Recent work further shows that low-level controller gains themselves substantially shape how a learned policy executes its predictions [58]. These approaches are orthogonal to ours and can be stacked on top of a speedconditioned VLA, but operating strictly downstream they can only rescale or smooth whatever actions the policy emits. They cannot recover from upstream pathologies such as imprecise predictions or hesitation stalls inherited from teleoperation data.
3
Methodology
3.1
Problem Formulation
VLA for robot manipulation. A Vision-Language-Action (VLA) policy πθ is a sequential decision model for end-to-end robot manipulation. At each step t, it consumes the observation ot = (vt , ℓt , ρt ) comprising the visual input vt , the language instruction ℓt , and an optional proprioceptive state ρt . From this observation, the policy predicts an action chunk At = (at , . . . , at+H−1 ) of horizon H. The policy is trained by imitation on a demonstration set D = {(ot , At )}: θ⋆ = arg min E(ot ,At )∼D L πθ (ot ), At , θ
(1)
where L is the imitation objective (e.g., regression or flow-matching). Goal of TempoVLA. TempoVLA aims to produce a single policy whose execution speed is controllable through an explicit scalar input. We split this goal into two coupled sub-objectives. On the data side, we want to online re-time any demonstration in D to an arbitrary target speed s ∈ R+ e On the model side, without losing motion semantics, yielding a multi-speed augmented dataset D. e that scales its predicted action magniwe want a speed-conditioned policy πθ (ot , s) trained on D tudes according to s, where s > 1 speeds up, s < 1 slows down, and s = 1 recovers the default speed. The downstream low-level controller is left untouched throughout. 3.2
Variable-Speed Trajectory Augmentation
Variable-Speed Trajectory Augmentation (VSTA) realizes the data-side objective of TempoVLA by re-timing any demonstration to a target speed s online during training, as illustrated in Figure 2 (a). 3
(a)
Variable-Speed Trajectory Augmentation (VSTA)
(b)
Speed-Conditioned VLA Training
🎬 Demo 𝜏
Vision-Language Model
Original Demonstration Segment
MotionConsistent Segmentation
Action Chunk
③ visual observations
①
②
Modulation
Text Prefix
Soft Prompt
𝑠
“stack cups”
→
↻
||
⇆
Translate
Rotate
Still
Gripper
→↻
“Perform the task at 1.5x speed”
task
Translate & Rotate
→↻⇆…↻
MLP
Controller Time Embed
P ∈ ℝ!×#×$
Action Expert
Norm Layer
Execute
speed 𝑠
✂
✂
✂
✂
Per-segment Speed Transform
{𝑎! , … , 𝑎𝒒 }
accumulate & re-split
{𝑎)! , … , 𝑎) 𝒑 } , 𝑠 = 𝑞/𝑝
(c)
Dynamic Speed Control with VLM
Re-timed Outputs at Variable Speeds 𝑠 = 0.5×
…
…
You are real-time execution-speed controller ...
…
visual observation
𝑠 = 1× 𝑠 = 2×
↻ online process w/ random speed s and chunk-start offset r
prompt
Vision-Language Model VLM schedules current speed
Approach s=1.25×
Align s=0.75×
Grasp s=0.5×
Lift & Move s=1.5×
fast
slow carefully
very slow precise
fast
many steps
many steps
few steps
few steps
VLA executes at command pace
Figure 2: Framework of TempoVLA. (a) VSTA re-splits each motion-consistent segment from q to p actions to realize s = q/p, with s and a chunk-start offset r resampled online. (b) The speed s enters the policy via a text prefix, a soft prompt, or an MLP-driven modulation. (c) At deployment, a VLM scheduler observes the scene and dispatches per-chunk speeds for TempoVLA to execute. The procedure has three steps and is detailed in Algorithm 1 (Appendix B): motion-consistent segmentation, chunk-level speed transform, and online chunk-start sampling. Motion-consistent segmentation. We first cut each demonstration into segments whose motion is internally consistent. Every frame is labeled with one of four motion modes (still, translate, rotate, or translate-and-rotate) according to whether its translation and rotation magnitudes exceed a small threshold, and a boundary is placed at every mode change. Within a single mode, we further split whenever the motion direction reverses, i.e., the cosine similarity between consecutive translation or rotation directions falls below τdir . Gripper open/close events are kept as hard boundaries so that a discrete state switch is never blurred by resampling. Chunk-level speed transform. Inside each segment we realize the target speed s by re-allocating actions between source and output frames. We write s = q/p with coprime integers q, p, so that q source frames are mapped to p output frames (q > p speeds up, q < p slows down). We partition the segment into non-overlapping chunks of q consecutive frames, leaving anyP trailing remainder shorter q than q unchanged. For each chunk, we accumulate its total motion ∆ = i=1 ai , and then re-split ∆ into p equal-magnitude steps by linearly interpolating the cumulative motion. By construction, the p new actions sum back to ∆ exactly, so the integrated motion of the chunk is preserved and only the within-chunk shape is altered. The accumulate-then-split operation is valid only when adding actions equals composing them. This holds for Cartesian translation in R3 , joint velocities, and rotational increments written as axis-angle vectors in so(3) (whose axis the segmentation keeps approximately constant within a segment). Representations that are not closed under addition, such as unit quaternions, rotation matrices, or Euler angles, must first be mapped to so(3) or interpolated on the manifold (e.g., SLERP [59]) before VSTA can be applied. The gripper command is copied discretely, and gripper switches serve as anchors so they are never averaged across. Online chunk-start sampling. Once a chunk is sped up, only the observation at the chunk start corresponds to an emitted action, and the other q − 1 observations would have to be dropped from training. To avoid permanently discarding them, following [51], we randomize where the chunks begin: for each segment we sample an offset r ∼ U {0, . . . , q − 1}, so the first r frames pass through verbatim and the chunks start at r, r + q, r + 2q, . . . Because VSTA runs online during training, a fresh offset is drawn every time the demonstration is sampled. Over the course of training, every source frame eventually becomes a chunk start and contributes a valid training observation. 3.3
Speed Conditioning in TempoVLA
e produced by VSTA, we train TempoVLA as a speed-conditioned On top of the multi-speed dataset D policy πθ (ot , s) through one of three lightweight schemes that inject s into a VLA (Figure 2 (b)). 4
Textual prefix. We prepend a short phrase such as “Perform the task at ⟨s⟩x speed.” to the original instruction ℓ, leaving the architecture entirely unchanged. Speed-modulated RMSNorm. A small two-layer MLP ϕmod (s) ∈ Rdmod embeds the scalar speed, and we add its output to the flow-matching timestep embedding σts that already conditions every transformer block of the action expert. The summed signal drives RMSNorm of each expert layer, adaRMSNorm x; σts + ϕmod (s)
= γ σts + ϕmod (s) ⊙
x , ∥x∥RMS
(2)
so that s rescales the feature statistics throughout the expert. Soft prompt with speed anchors. We maintain a learnable tensor P ∈ RK×P ×demb that stores P soft-prompt tokens for each of K training-speed anchors sk ∈ S. During training, the P tokens for the current speed are inserted between the image and language tokens at the encoder input. At inference, we pick the anchor nearest the requested speed, k ⋆ = arg mink |s−sk |, and use its tokens. 3.4
Dynamic Speed Control with a VLM Scheduler
Beyond fixed speed commands, TempoVLA supports automated dynamic speed scheduling when paired with a high-level Vision-Language Model (VLM), as illustrated in Figure 2 (c). At deployment, the VLM takes the current observations and a prompt as input, and predicts the speed st for the next few action chunks. TempoVLA then executes those chunks at the dispatched speed st . The behavior accelerates through low-risk transit phases such as free-space approach and slows down for high-risk contact phases such as grasping or insertion. Because the VLM and TempoVLA communicate only through the scalar s, the planner can be upgraded without retraining the policy.
4
Simulation Experiments
Simulation Setup. We evaluate TempoVLA on LIBERO [40], which provides four manipulation task suites (Spatial, Object, Goal, Long), each containing 10 tasks and 500 human-teleoperated demonstrations. Its demonstrations are smooth and free of abrupt speed changes, which makes it a clean testbed for speed control. Each action is a 7-dim end-effector (EEF) command comprising a translation (∆x, ∆y, ∆z), an axis-angle rotation increment, and a gripper signal. The translation and rotation parts lie in the linearly composable space of Section 3.2, so VSTA applies through its accumulate-then-split operation, while the gripper is handled discretely. Base Model and Implementation Details. Our base model is π0.5 [4], a flow-matching VLA built on PaliGemma [60] and pre-trained on large-scale embodied datasets. We feed the target speed s with the textual prefix as the default unless stated otherwise. All models are trained for 30k iterations with batch size 512 on 32 NVIDIA H20 GPUs under a fixed random seed for fair comparison. 4.1
Feasibility of Variable-Speed Trajectory Augmentation
We first verify that VSTA produces executable demonstrations at each target speed. For each s ∈ {0.5, 0.75, 1, 1.25, 1.5, 2}, we apply VSTA to the LIBERO demonstrations and replay the re-timed actions in the simulator. The segmentation stage is speed-independent and divides each demonstration into 5.96 segments of mean length 41 steps on average. The default 1× replays the original actions and serves as the baseline, so its motion error is reported as “–”. As Table 1 shows, the realized Data Ratio closely tracks the target speed, with a small rounding gap appearing at higher acceleration ratios because the per-segment chunk count must be integer-valued. For replay success rate (SR), speeds close to the baseline stay highly reliable, with 0.75× and 1.25× reaching 92.9% and 92.4% versus 97.6% at 1×. The SR then degrades monotonically as the target moves further from 1× in either direction. The Motion Err., the absolute deviation in integrated end-effector displacement caused by re-timing, grows with the speed factor s but stays below 5 × 10−8 throughout, which is negligible compared to controller tolerances. Overall, VSTA is a reliable data-processing primitive for producing variable-speed demonstrations to train TempoVLA. 5
Table 1: Feasibility of VSTA on LIBERO. Re- Table 2: Ablation of speed-integration timed demonstrations replay at each target speed scheme. SR: average success rate (%); Steps: s. Blue subscripts give the Data Ratio gap to s. average rollout length on successes. Target s Data Ratio Succ. (#) SR (%) Motion Err. 0.5 0.75 1 1.25 1.5 2
4.2
0.5 0.76+0.01 1 1.20−0.05 1.43−0.07 1.90−0.10
664 743 781 739 653 540
83.0 92.9 97.6 92.4 81.6 67.5
2.8E-10 4.4E-9 – 1.1E-8 2.2E-8 4.8E-8
Speed
Text
Modulation Soft Prompt-8
SR ↑ Steps SR ↑ Steps SR ↑
Steps
0.75× 1.0× 1.25× 1.5×
96.5 96.9 97.0 96.8
96.4 95.8 96.8 96.9
200 153 129 111
Avg.
96.8 146.6 96.8 147.6 96.5
148.2
197 151 126 111
95.8 97.0 97.3 97.2
198 152 129 111
Ablation on the Speed-Integration Scheme
We next study how the speed signal should be injected into the VLA. We compare the three schemes of Section 3.3: a textual prefix (Text), an action-expert modulation (Modulation), and a soft prompt with P =8 anchor tokens (Soft Prompt-8). All three are trained and evaluated on LIBERO with the same speed set {0.75, 1, 1.25, 1.5}×, and we report per-speed success rate alongside the average length of successful rollouts. As Table 2 shows, the three schemes are essentially tied, with overall SRs of 96.8 / 96.8 / 96.5 within 0.3% of each other and comparable rollout lengths at each commanded speed. This indicates that speed control can be injected into a VLA with little engineering effort, largely independent of the specific mechanism. Among the three, Text ties for the highest overall SR while requiring no architectural change or pre-defined anchor set, which makes it the simplest and most flexible to deploy. We therefore adopt the textual prefix as the default speed-integration scheme of TempoVLA in all subsequent experiments. 4.3
Effect of the Training Speed Range
We now study how the set of training speeds affects a speed-controllable policy. Starting from the single-speed baseline, we train three policies on progressively designed speed ranges: a narrow range {0.75, 1, 1.25, 1.5}×, a wider range with a larger stride {0.5, 1, 1.5, 2}×, and a wide range with a refined stride {0.5, 0.75, 1, 1.25, 1.5, 1.75, 2}×. Each policy is evaluated at every speed it was trained on, and the results are summarized in Table 3. Comparison with the baseline. Across all three ranges, training with VSTA preserves or improves the 1× success rate over the single-speed baseline (96.7). A per-suite breakdown shows that the gain is concentrated in Object and Goal, both rising by +2.0 to +2.6 over the baseline. We attribute this to the speed-conditioned training itself: when the same observation must produce different action magnitudes under different commanded speeds, the policy can no longer memorize a single observation-to-magnitude mapping, and is forced to extract finer object- and goal-aware features that also transfer to the 1× regime. Peak performance shifts away from 1×. More strikingly, the peak success rate of every speedconditioned policy occurs not at 1× but at 1.25× or 1.5×: 97.0, 97.2, and 97.4 for the narrow, fourspeed, and seven-speed ranges respectively, each exceeding its 1× counterpart. We attribute this to natural pacing slack in teleoperation data: even on the clean LIBERO benchmark, demonstrations contain rhythm padding and ambiguous transition frames that VSTA’s merge operation compresses out at moderate speedups. Trained under this compression, the policy executes more decisively at 1.25× and 1.5×, which reduces the ambiguity-induced stalls that occasionally appear at the original 1× rate. A practical implication is that the default deployment speed of TempoVLA is best set slightly above 1× rather than at the demonstration rate. Effect of the speed range. Comparing the ranges reveals two consistent trends. First, a finer speed granularity helps: over the shared speeds {0.5, 1, 1.5, 2}×, refining the stride from 0.5 to 0.25 raises the success rate at every speed (e.g., 94.1 → 95.0 at 0.5× and 88.4 → 89.4 at 2×). Second, including 6
Table 3: Effect of the training speed range on LIBERO. Each block trains one policy on the indicated speed range and evaluates it at every speed in that range. Avg.: success rate averaged over the four suites. Steps: average steps of successful rollouts. Model Ratio: speed ratio realized by the policy at rollout, measured as Steps1× /Stepss . Data Ratio: the data-level ratio achieved by VSTA (Section 4.1). Both ratios ideally equal the commanded speed s. Red ↑ marks the 1× gain over the baseline; blue subscripts give the gap (Model Ratio − Data Ratio). Speed
Spatial
Object
Goal
Long
Avg.
Steps
Model Ratio
Data Ratio
Baseline (single-speed) 1× 99.4 95.6
96.0
95.8
96.7
152
1
1
Range {0.75, 1, 1.25, 1.5}× 0.75× 99.4 95.6 1.0× 99.6 97.6 1.25× 99.4 98.0 1.5× 99.4 97.6
97.2 97.4 96.6 96.2
93.6 92.8 94.0 94.0
96.5 96.9↑0.2 97.0↑0.3 96.8↑0.1
197 151 127 111
0.77+0.01 1 1.19−0.01 1.36−0.07
0.76 1 1.20 1.43
Range {0.5, 1, 1.5, 2}× 0.5× 98.8 88.0 1.0× 99.2 95.2 1.5× 99.2 98.4 2.0× 78.6 96.0
96.4 97.8 96.8 90.4
93.3 94.8 94.4 88.4
94.1 96.8↑0.1 97.2↑0.5 88.4
295 153 111 98
0.52+0.02 1 1.38−0.05 1.56−0.34
0.50 1 1.43 1.90
Range {0.5, 0.75, 1, 1.25, 1.5, 1.75, 2}× 0.5× 97.6 94.4 96.0 0.75× 98.4 95.4 97.0 1.0× 99.2 98.2 98.4 1.25× 99.0 96.0 98.8 1.5× 98.6 98.0 96.8 1.75× 93.6 98.0 97.0 2.0× 78.6 97.0 92.4
92.1 94.4 91.8 95.6 95.8 93.6 89.6
95.0 96.3 96.9↑0.2 97.4↑0.7 97.3↑0.6 95.6 89.4
296 201 153 129 112 105 97
0.52+0.02 0.76±0 1 1.19−0.01 1.37−0.06 1.46−0.08 1.58−0.32
0.50 0.76 1 1.20 1.43 1.54 1.90
the extreme 0.5× and 2× broadens the augmentation enough to lift the moderate speeds, where the seven-speed range matches or exceeds the narrow {0.75–1.5}× range at 1, 1.25, and 1.5×. The refined seven-speed range thus offers the best overall trade-off between coverage and granularity. Realized versus data speed ratio. Finally, we compare the Model Ratio (the speed actually realized at rollout) with the Data Ratio (the speed achievable on the augmented data). The model broadly hits the target but under-shoots at high speedups (e.g., 1.56× realized at the 2× command versus a 1.90× data ratio). The gap arises from two factors: corrective steps after imperfect first attempts inflate the rollout length, and the low-level controller cannot accurately track the large action magnitudes.
5
Real-world Experiments
We deploy TempoVLA on a 7-DoF Franka arm with a 1-DoF parallel gripper, observed by a primary camera and a wrist-mounted camera (Figure 3). We evaluate on five tasks covering four pickand-place behaviors and one deformable-object task. For each task we collect 50 teleoperated trajectories for training and run 10 rollouts per commanded speed for evaluation. At inference, the policy re-queries after executing the first 10 steps of each predicted action chunk. Each action is 8dimensional, consisting of a 7-dim joint velocity and a 1-dim gripper value, both lying in a linearly composable space so that VSTA applies directly. We train one TempoVLA policy on the speed set {0.75, 1, 1.25, 1.5}× alongside a single-speed baseline at 1× for comparison. 5.1
Results
VSTA boosts the 1× success rate, mirroring the simulation finding. On the Franka platform, VSTA boosts the default 1× success rate from 80.0 (single-speed baseline) to 88.0, an 8-point gain that mirrors the implicit-augmentation effect observed in simulation. The 1.25× speed also outperforms the baseline (84.0 versus 80.0), showing that TempoVLA delivers consistent gains at and above the demonstration speed. 7
7-DoF Franka Arm
(b) Success Rate across speed settings Overall Average 100
80%
Primary Camera
66%
100 90 90
80
80
70
60
Wrist Camera
T1: move tide to the front of the book
PMC
88% 100
90
90 90
70
60
50
40
SC
MT
84% 100
80 80
90 90
GC
FT
70%
96% 100
100100100 80 60
70
80
80
80
70
50
30
20
1-DoF Gripper
Success Rate (%)
Baseline 1x
0.75x
1.0x
1.25x
1.5x
Dynamic
Various Speed
(c) Average Steps of successful rollouts (a) Platform Setup
T2: pick up middle cup
Overall Average
228
200
197
PMC
MT
153
GC
FT
Ratio
133
163
519
400 300
313
SC
367 255 193174 150
100
302 235 189
1.50
343
323
0.63
1.00 192 122 108
0.75x
1.0x
204
1.29 176
1.48
263
139 90 100
212 156 76 83
128
1.25 256 1.21 1.00 140 100101 0.75
207
0.50
0 Avg. Steps
T3: stack the cups
T4: grasp the coke
Baseline 1x
1.25x
1.5x
Dynamic
Real Model Ratio
Various Speed
T5: fold towel diagonally
Figure 3: Real-world Setup and Results. (a) Four pick-and-place behaviors and one deformableobject task with Franka. (b) TempoVLA improves 1× success rate from 80% to 88% over singlespeed baseline, and the GPT-4o-scheduled variant reaches the highest overall success rate of 96%. (c) The realized Model Ratio closely tracks the commanded ratio for s = (1.29, 1.48). Realized speedup closely matches the commanded ratio. The Model Ratio realized at rollout tracks the commanded speed across the trained range, with 0.63×, 1.29×, and 1.48× realized at commanded 0.75×, 1.25×, and 1.5× respectively. This confirms that TempoVLA’s speed conditioning translates faithfully into execution-speed control on real hardware, not only at the policyprediction level but also through the unchanged low-level controller. 5.2
Dynamic Speed Control with a VLM Scheduler
We further test whether TempoVLA, paired with a high-level VLM, can schedule its own speed at runtime (Figure 2 (c)). We adopt GPT-4o [61] as the scheduler, querying it once every two action chunks to dispatch the speed for the next segment. TempoVLA with dynamic scheduling reaches 96% average success rate, 8 points above the best fixed-speed configuration (88% at 1×), while still completing tasks at an average realized speedup of 1.21× over the 1× baseline. In our prompt (full text in Appendix F) we explicitly encourage GPT-4o to favor aggressive speedups during low-risk phases. Yet the actual schedule remains conservative, with the vast majority of decisions falling on the 1× or 1.25× tier and 1.5× rarely dispatched. Despite this conservatism, GPT-4o reads the execution state of the real robot with high reliability, correctly anticipating freespace transit, fine alignment, and contact phases. The resulting schedule realizes the phase-aware variable-speed behavior we expect, just biased toward the safer end of the speed range.
6
Conclusion
Existing Vision-Language-Action models inherit a single fixed execution speed from training data. We propose TempoVLA, a single speed-controllable VLA framework that pairs a data-side Variable-Speed Trajectory Augmentation with a lightweight model-side conditioning mechanism, both lightweight and applicable to existing VLAs. Experiments in simulation and real world show that TempoVLA delivers flexible bidirectional speed control ability, improves default 1× performance, and achieves dynamic speed control with an external VLM. Limitation and Future Work. At the high end of the speed range, the realized speedup gradually saturates because the policy’s per-step targets begin to exceed the fixed low-level controller’s tracking bandwidth (Appendix D), and co-tuning the controller alongside TempoVLA is a natural extension (Appendix H).
8
References [1] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. [2] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. [3] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. π0 : A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. [4] K. Pertsch, K. Black, N. Brown, D. Driess, C. Finn, J. Mahler, O. Mees, D. Sadigh, et al. π0.5 : A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. [5] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. [6] S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. RDT-1B: A diffusion foundation model for bimanual manipulation. In International Conference on Learning Representations, 2025. [7] J. Wen, Y. Zhu, J. Li, M. Zhu, K. Wu, Z. Xu, R. Cheng, C. Shen, Y. Peng, F. Feng, and J. Tang. TinyVLA: Towards fast, data-efficient vision-language-action models for robotic manipulation. arXiv preprint arXiv:2409.12514, 2024. [8] M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844, 2025. [9] Y. Yang, Y. Wang, Z. Wen, Z. Luo, C. Zou, Z. Zhang, C. Wen, and L. Zhang. EfficientVLA: Training-free acceleration and compression for vision-language-action models. arXiv preprint arXiv:2506.10100, 2025. [10] K. Pertsch, K. Black, N. Brown, M. Y. Galliker, D. Driess, S. Nair, and S. Levine. FAST: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. [11] M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025. [12] K. Black, M. Y. Galliker, and S. Levine. Real-time execution of action chunking flow policies. arXiv preprint arXiv:2506.07339, 2025. [13] Y. Liu, H. Yu, J. Zhao, B. Li, D. Zhang, M. Li, W. Wu, Y. Hu, J. Xie, J. Guo, et al. Learning native continuation for action chunking flow policies. arXiv preprint arXiv:2602.12978, 2026. [14] D. D. Yuan, T. Z. Zhao, K. Burns, and C. Finn. SpeedTuning: Speeding up policy execution with lightweight reinforcement learning. In IEEE International Conference on Robotics and Automation, pages 1184–1192, 2025. doi:10.1109/ICRA55743.2025.11128753. [15] Y. Ma, Y. Zhou, Y. Yang, T. Wang, and H. Fan. Running vlas at real-time speed. arXiv preprint arXiv:2510.26742, 2025. 9
[16] C. Yang, Y. Hu, Y. Ma, Y. Yang, J. Tan, and H. Fan. Realtime-VLA V2: Learning to run vlas fast, smooth, and accurate. arXiv preprint arXiv:2603.26360, 2026. [17] W. Song, J. Chen, S. Chen, J. Wang, P. Ding, H. Zhao, Y. Qin, X. Zheng, D. Wang, Y. Wang, and H. Li. Fast-dVLA: Accelerating discrete diffusion vla to real-time performance. arXiv preprint arXiv:2603.25661, 2026. [18] H. Park, D. Lim, S. Kim, and S. Park. Proleptic temporal ensemble for improving the speed of robot tasks generated by imitation learning. arXiv preprint arXiv:2410.16981, 2024. [19] Z. Wu, J. Ye, Z. Zhang, Y. Sun, H. Lin, J. Luo, H. Ren, L. Yuan, and Y. Yu. Speedup patch: Learning a plug-and-play policy to accelerate embodied manipulation. arXiv preprint arXiv:2603.20658, 2026. [20] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. RT-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. [21] D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al. PaLM-E: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. [22] Y. Jiang, A. Gupta, Z. Zhang, G. Wang, Y. Dou, Y. Chen, L. Fei-Fei, A. Anandkumar, Y. Zhu, and L. Fan. VIMA: General robot manipulation with multimodal prompts. arXiv preprint arXiv:2210.03094, 2022. [23] S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J. T. Springenberg, et al. A generalist agent. arXiv preprint arXiv:2205.06175, 2022. [24] S. Haldar, Z. Peng, and L. Pinto. BAKU: An efficient transformer for multi-task policy learning. arXiv preprint arXiv:2406.07539, 2024. [25] K. Bousmalis, G. Vezzani, D. Rao, C. Devin, A. X. Lee, M. Bauza, T. Davchev, Y. Zhou, A. Gupta, A. Raju, et al. RoboCat: A self-improving generalist agent for robotic manipulation. arXiv preprint arXiv:2306.11706, 2023. [26] D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al. SpatialVLA: Exploring spatial representations for visual-language-action model. arXiv preprint arXiv:2501.15830, 2025. [27] Q. Li, Y. Liang, Z. Wang, L. Luo, X. Chen, M. Liao, F. Wei, Y. Deng, S. Xu, Y. Zhang, et al. CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024. [28] S. Deng, M. Yan, S. Wei, H. Ma, Y. Yang, J. Chen, Z. Zhang, T. Yang, X. Zhang, W. Zhang, H. Cui, Z. Zhang, and H. Wang. GraspVLA: a grasping foundation model pre-trained on billion-scale synthetic action data. arXiv preprint arXiv:2505.03233, 2025. [29] W. Zhang, H. Liu, Z. Qi, Y. Wang, X. Yu, J. Zhang, R. Dong, J. He, H. Wang, Z. Zhang, L. Yi, W. Zeng, and X. Jin. DreamVLA: A vision-language-action model dreamed with comprehensive world knowledge. arXiv preprint arXiv:2507.04447, 2025. [30] C. Cheang, S. Chen, Z. Cui, Y. Hu, L. Huang, T. Kong, H. Li, Y. Li, Y. Liu, X. Ma, et al. GR-3 technical report. arXiv preprint arXiv:2507.15493, 2025. [31] J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, et al. GR00T N1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025. 10
[32] Q. Bu, Y. Yang, J. Cai, S. Gao, G. Ren, M. Yao, P. Luo, and H. Li. UniVLA: Learning to act anywhere with task-centric latent actions. arXiv preprint arXiv:2505.06111, 2025. [33] J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y. Feng, Y. Zheng, J. Zou, Y. Chen, J. Zeng, et al. XVLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274, 2025. [34] Y. Mu, Q. Zhang, M. Hu, W. Wang, M. Ding, J. Jin, B. Wang, J. Dai, Y. Qiao, and P. Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 36, 2024. [35] StarVLA-Community. StarVLA: A lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014, 2026. [36] D. Jing, G. Wang, J. Liu, W. Tang, Z. Sun, Y. Yao, Z. Wei, Y. Liu, Z. Lu, and M. Ding. Mixture of horizons in action chunking. arXiv preprint arXiv:2511.19433, 2025. [37] A. Padalkar, A. Pollet, A. Jain, et al. Open x-embodiment: Robotic learning datasets and RT-X models. arXiv preprint arXiv:2310.08864, 2023. [38] A. Khazatsky, K. Pertsch, et al. DROID: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024. [39] H. Walke, K. Black, A. Lee, M. J. Kim, M. Du, C. Zheng, T. Zhao, P. Hansen-Estruch, Q. Vuong, A. He, et al. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning, 2023. [40] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems Datasets and Benchmarks, 2023. [41] A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martin-Martin. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108.03298, 2021. [42] T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-World: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning, 2020. [43] A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox. MimicGen: A data generation system for scalable robot learning using human demonstrations. In Conference on Robot Learning, 2023. [44] S. Dasari, F. Ebert, S. Tian, S. Nair, B. Bucher, K. Schmeckpeper, S. Singh, S. Levine, and C. Finn. RoboNet: Large-scale multi-robot learning. In Conference on Robot Learning, 2019. [45] S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. RLBench: The robot learning benchmark and learning environment. arXiv preprint arXiv:1909.12271, 2019. [46] S. Nasiriany, S. Nasiriany, A. Maddukuri, and Y. Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots. In International Conference on Learning Representations (ICLR), 2026. [47] Q. Bu, J. Cai, L. Chen, X. Cui, Y. Ding, S. Feng, S. Gao, X. He, X. Hu, X. Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025. [48] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems, 2023. 11
[49] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems, 2023. [50] T. Z. Zhao, J. Tompson, D. Driess, P. Florence, K. Ghasemipour, C. Finn, and A. Wahid. ALOHA unleashed: A simple recipe for robot dexterity. arXiv preprint arXiv:2410.13126, 2024. [51] L. Guo, Z. Xue, Z. Xu, and H. Xu. DemoSpeedup: Accelerating visuomotor policies via entropy-guided demonstration acceleration. arXiv preprint arXiv:2506.05064, 2025. [52] T. Nam and S. J. Hwang. SpeedAug: Policy acceleration via tempo-enriched policy and RL fine-tuning. arXiv preprint arXiv:2512.00062, 2025. [53] B. Kim, J. Pahk, C. Lee, J. Kim, J. Lee, T. T. Kim, K. Shim, J. K. Lee, and B.-T. Zhang. ESPADA: Execution speedup via semantics aware demonstration data downsampling for imitation learning. arXiv preprint arXiv:2512.07371, 2025. [54] N. R. Arachchige, Z. Chen, W. Jung, W. C. Shin, R. Bansal, P. Barroso, Y. H. He, Y. C. Lin, B. Joffe, S. Kousik, and D. Xu. SAIL: Faster-than-demonstration execution of imitation learning policies. arXiv preprint arXiv:2506.11948, 2025. [55] N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa. Chomp: Gradient optimization techniques for efficient motion planning. In 2009 IEEE international conference on robotics and automation, pages 489–494. IEEE, 2009. [56] N. D. Ratliff, J. Issac, D. Kappler, S. Birchfield, and D. Fox. Riemannian motion policies. arXiv preprint arXiv:1801.02854, 2018. [57] B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V. Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos, et al. Curobo: Parallelized collision-free robot motion generation. In IEEE International Conference on Robotics and Automation (ICRA), pages 8112–8119. IEEE, 2023. [58] A. Bronars, Y. Park, and P. Agrawal. Tune to learn: How controller gains shape robot policy learning. arXiv preprint arXiv:2604.02523, 2026. [59] K. Shoemake. Animating rotation with quaternion curves. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pages 245–254, 1985. [60] L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello, et al. PaliGemma: A versatile 3B VLM for transfer. arXiv preprint arXiv:2407.07726, 2024. [61] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
12
A
Hyperparameters Table 5: Training hyperparameters of π0.5 on real-world tasks.
Table 4: Training hyperparameters of π0.5 on LIBERO. Hyperparameter
Value
Hyperparameter
Value
GPUs Total Batch Size Optimizer Scheduler Learning Rate Iterations Warmup Step Minimum LR Max Gradient Norm
32 × H20 512 AdamW Cosine Decay 5e-5 30k 1k 5e-7 1.0
GPUs Total Batch Size Optimizer Scheduler Learning Rate Iterations Warmup Step Minimum LR Max Gradient Norm
16 × H20 128 AdamW Cosine Decay 5e-5 10k 1k 5e-7 1.0
Training hyperparameters. Please refer to Table 4 and Table 5. When evaluating at slow commanded speeds, we proportionally raise the maximum rollout step budget so that the policy is given enough time to complete the task at the reduced per-step magnitude. VSTA segmentation hyperparameters. For VSTA segmentation on the real-robot data, we mark a frame as a direction change when consecutive action directions diverge by more than 60◦ , while on the LIBERO, the threshold is 90◦ . For both environments, the gripper event are labeled when the absolute gripper state changes by more than 0.5. This partitions each demonstration into 5.2/6.0 segments on average with a mean length of 45/41 steps in real-world tasks/LIBERO.
B
Pseudocode of VSTA
Algorithm 1 gives the full pseudocode of Variable-Speed Trajectory Augmentation, applied online to a single demonstration during training. Algorithm 1 Online Variable-Speed Trajectory Augmentation (one demonstration) Require: demonstration τ = {(ot , at )}Tt=1 , target speed s 1: write s = q/p with coprime integers q, p ▷ q source → p output frames 2: {Sk } ← S EGMENT(τ ) ▷ motion mode + direction split; gripper events as anchors 3: T ← ∅ ▷ re-timed (action, validity) stream, in output order 4: for each segment Sk do 5: sample chunk-start offset r ∼ U {0, . . . , q − 1} ▷ drawn online, per segment 6: append frames [0, r) to T , all marked valid ▷ leading passthrough 7: for each non-overlapping chunk of q frames starting at r do P 8: ∆ ← i ai over the q frames ▷ accumulate motion 9: re-split ∆ into p steps by interpolating the cumulative motion 10: append the p steps to T , marking only the chunk-start observation valid 11: end for 12: append the trailing < q leftover frames to T , all marked valid ▷ trailing passthrough 13: end for 14: return re-timed trajectory T with its validity mask
C
More Ablation Study
C.1
The Effect of Soft Prompt Length
As Table 6 shows, the average success rate is essentially flat across P ∈ {4, 8, 16}, with the three averages all within 0.3 points of each other. Performance only starts to slip at P = 32 (96.3 on 13
average), suggesting that a few anchor tokens per speed are sufficient and that over-long prompts mildly hurt optimization. We therefore adopt P = 8 as the default in our experiments. Table 6: Effect of soft prompt length P on LIBERO. All variants use the speed set {0.75, 1, 1.25, 1.5}×. SR: average success rate (%); Steps: average rollout length on successes. P =4
Speed
D
P =8
P = 16
P = 32
SR ↑
Steps
SR ↑
Steps
SR ↑
Steps
SR ↑
Steps
0.75× 1.0× 1.25× 1.5×
96.4 96.8 96.7 96.7
199 153 128 112
96.4 95.8 96.8 96.9
200 153 129 111
96.9 96.9 96.5 97.0
199 152 127 111
95.8 96.7 97.1 95.7
199 152 128 111
Avg.
96.6
147
96.5
148
96.8
147
96.3
147
Stress Test at Extreme Speeds
To probe the limits of TempoVLA, we train a single policy on the wide, fine-grained range {0.25, 0.5, 0.75, 1, 1.5, 2, 2.5, 3, 4}× and evaluate it at every training speed. Table 7 reports the per-speed success rate together with the realized Model Ratio, the Data Ratio, and the controller’s tracking gap. Table 7: Performance under an extreme speed range (LIBERO). A single TempoVLA policy trained on the wide range {0.25, 0.5, 0.75, 1, 1.5, 2, 2.5, 3, 4}× and evaluated across speeds. Controller Gap: tracking error between the per-step EEF motion requested of the controller and the motion actually realized in one simulation step. Blue subscripts give Model Ratio − Data Ratio. Speed 0.25× 0.5× 1× 1.5× 2× 2.5× 3× 4×
Avg. 75.8 92.8 96.6 96.6 88.3 72.8 56.0 34.3
Steps 536 288 152 112 95 96 99 93
Model Ratio 0.28+0.03 0.53+0.03 1.00 1.36−0.07 1.60−0.30 1.58−0.51 1.54−1.05 1.63−1.43
Data Ratio 0.25 0.50 1.00 1.43 1.90 2.09 2.59 3.06
Controller Gap Pos. (m)
Rot. (rad)
0.009 0.019 0.038 0.057 0.075 0.095 0.112 0.146
0.015 0.031 0.069 0.100 0.129 0.151 0.189 0.243
Speed control degrades gracefully within 0.5× to 1.5× and breaks beyond it. Inside this regime, SR stays above 92 and the realized Model Ratio closely tracks the target. Outside, performance degrades on both ends. At 0.25×, SR drops to 75.8 because the per-step magnitudes shrink to nearly zero and the policy becomes sensitive to ambiguous observations. At the high end, SR collapses from 96.6 at 1.5× to 34.3 at 4×, and the realized Model Ratio saturates around 1.6, far below the commanded speed. The acceleration bottleneck is the controller, not TempoVLA. The Controller Gap columns measure the discrepancy between the per-step end-effector target sent to the controller and the motion actually realized after one simulation step. This gap grows steeply with speed, from 0.038 m / 0.069 rad at 1× to 0.146 m / 0.243 rad at 4×, indicating that the per-step target becomes too large for the operational-space controller and the robot dynamics to realize within one control interval. Action clipping stays near zero throughout, ruling out the controller’s input range as the limiting factor. The Model Ratio saturating around 1.6 for s ≥ 2× is a direct consequence: no matter what TempoVLA predicts, the robot cannot move faster than the controller can track. Summary. TempoVLA itself extends gracefully across 0.5× to 1.5×, while headroom beyond this regime is governed by the low-level controller rather than the policy. Reaching higher real speeds therefore requires joint tuning of the controller alongside TempoVLA, consistent with the orthogonality view in Section G.1. 14
Speed-Dependent Failure Modes Analysis (b) Too Slow: Stalled Progress
(a) Too Slow: Hesitation ×
×
1× baseline
×
×
stalled failure
Extended rollout still stalls
Moves a lot, but no effective Grasp
(c) Too Fast: Tracking Error
( Requested > Executed )
(d) Too Fast: Overshoot
" = 4.0×
× approaching frame 93/519
closest frame 218/519
overshoot frame 301/519
final failure frame 477/519
Overshoot → irreversible failure slow
fast
normal baseline (1×)
start
goal
× end
Figure 4: Failure Mode Analysis.
E
Qualitative Comparison
E.1
Failure Mode Analysis
Qualitative failure mode analysis. Figure 4 provides qualitative examples for the degradation patterns observed in the extreme-speed stress test. This analysis is not intended to suggest that TempoVLA is unreliable within its practical operating range. As shown in Table 7, the policy remains robust for moderate commands between 0.5× and 1.5×, while performance degrades primarily near the two ends of the evaluated speed spectrum. The examples below therefore characterize the boundary cases of speed-conditioned execution. Slow-speed failures. At very low speeds, the dominant failure mode is insufficient task progress. In Figure 4(a), the end effector produces repeated local motions around the target object, but these motions do not accumulate into an effective grasp. This is distinct from a static failure: the policy remains active, yet the reduced per-step displacement is too small to reliably drive the system across key manipulation transitions, such as approach-to-contact and contact-to-grasp. We refer to this behavior as hesitation. It is consistent with the quantitative result at 0.25×, where action magnitudes approach zero and the policy becomes more sensitive to visually ambiguous states. Figure 4(b) shows a related stalled-progress failure. Even with an extended rollout horizon, the robot remains in a similar local behavior pattern and does not complete the task. This indicates that the failure is not merely due to an insufficient number of control steps. Rather, each step contributes too little effective progress, allowing the policy to remain trapped near a phase boundary instead of transitioning to the next manipulation stage. Thus, slow execution is beneficial only when the reduced action magnitude still preserves enough progress to complete the required phase transition. Fast-speed failures. At high speeds, failures arise from a different mechanism. Figure 4(c) shows a mismatch between requested and realized motion: the policy issues larger per-step targets, but the low-level controller cannot faithfully execute them within one control interval. This observation matches the controller tracking-gap measurements in Table 7, where the realized Model Ratio saturates around 1.6× even as the commanded speed continues to climb. This suggests the high-speed limit is primarily imposed by execution-side tracking rather than by errors introduced by VSTA. Figure 4(d) illustrates a downstream consequence of this tracking mismatch. The end effector approaches the target region too aggressively, passes the valid interaction window, and fails before the policy can correct its motion. We refer to this failure mode as overshoot. Such failures are particularly damaging in contact-rich stages, where successful interaction often depends on a narrow spatial and temporal tolerance. Once the gripper moves past the object or perturbs it into an out-of-distribution state, the remaining rollout can become unrecoverable. 15
Implication. These qualitative examples clarify the usable speed envelope of TempoVLA. Slow commands can lead to hesitation or stalled progress because each action contributes too little effective task progress. Fast commands can lead to tracking error or overshoot because the requested per-step motion exceeds the controller’s tracking capability. TempoVLA should therefore not be used by assigning an extreme fixed speed to the entire rollout. Instead, speed should be selected according to the current manipulation phase: faster during low-risk free-space motion and slower near contact-rich phases that require precise interaction. This observation is consistent with our dynamic speed scheduling results, where phase-aware speed selection outperforms fixed-speed execution. E.2
Demonstration
In Figures 5 and 6, we present demonstration rollouts at different speeds and tasks for illustration.
F
Prompt to GPT4o for Dynamic Speed Control
Figure 7 reproduces the prompt we send to GPT-4o for dispatching the per-segment speed during dynamic speed control.
G
Discussion
G.1
Relationship with Controller-focused Works
A natural alternative to changing the execution speed of a robot is to act on the low-level controller, for example by scaling target velocities or stretching step periods after the policy has produced its outputs. We view such controller-side methods as orthogonal to TempoVLA rather than competing with it: controllers sit downstream of the policy and can only rescale or retime what the policy has already predicted, while variable-speed training changes the content of those predictions at the policy level itself. The two approaches therefore intervene at different layers of the control stack and can be composed directly, so a TempoVLA policy can still be paired with any controller-side modulation when finer execution-side tuning is desired. G.2
The difference between effect of VSTA on EEF and Joint Action Space
For manipulation tasks, success is defined by the relative pose between the end effector and the manipulated objects, so the end-effector trajectory is the quantity we ultimately care about when evaluating how faithfully a re-timed demonstration preserves the original task semantics. Both EEF and joint commands satisfy the linear-composability requirement of Section 3.2, so VSTA’s accumulate-then-split operation is mathematically well-defined in either case. The relevant question is therefore not whether VSTA is valid on joints, but how large the resulting end-effector deviation becomes after re-timing, and three structural reasons make this deviation larger in joint space than in EEF space. Geometric amplification at single joints. A small re-timing error on a proximal joint is geometrically magnified at the end effector in proportion to its lever arm, a structural penalty that joint-space re-timing cannot avoid. Non-linear coupling across the kinematic chain. Forward kinematics maps joint angles to endeffector pose through a non-linear function, so linearly interpolated joint increments translate into end-effector motions that no longer interpolate linearly, while re-timing directly on EEF translations and axis-angle increments keeps the linear interpolation aligned with the quantity that defines task success. Controller realizability under speed change. For operational-space controllers commonly used on modern arms, joint-space re-timing commits the policy to a specific joint trajectory, whereas EEF-space re-timing lets the controller use its inverse-kinematics resolution to find a joint trajectory dynamically feasible at the new speed, helping absorb effects such as inertia, friction, and torque saturation. 16
T1: Fold Towel Diagonally, T2: Stack Cups 0.75×
1.00×
1.25×
1.50×
0.75×
1.00×
1.25×
1.50× 4s
8s
12s
16s
20s
24s
28s
32s
36s
40s
44s
48s
52s
36s
40s
44s
48s
52s
rollout time (Δt = 4s per frame)
Task: libero_10 t07 0.50×
1.00×
1.50×
2.00×
2.50× 4s
8s
12s
16s
20s
24s
28s
32s
rollout time (Δt = 4s per frame)
Figure 5: Demonstration rollouts of TempoVLA at various speed (1/2).
17
Task: libero_goal t04 0.25× 0.50× 1.00× 1.50× 2.00× 2.50× 2s
4s
6s
8s
10s
12s
14s
16s
18s
20s
22s
24s
26s
28s
30s
32s
34s
36s
38s
24s
26s
28s
30s
32s
34s
36s
38s
24s
26s
28s
30s
32s
34s
36s
38s
rollout time (Δt = 2s per frame)
Task: libero_10 t05 0.50× 1.00× 1.50× 2.00× 2.50× 3.00× 2s
4s
6s
8s
10s
12s
14s
16s
18s
20s
22s
rollout time (Δt = 2s per frame)
Task: libero_object t07 0.50× 1.00× 1.50× 2.00× 2.50× 2s
4s
6s
8s
10s
12s
14s
16s
18s
20s
22s
rollout time (Δt = 2s per frame)
Figure 6: Demonstration rollouts of TempoVLA at various speed (2/2).
18
Prompt to GPT4o for Dynamic Speed Control You are a real-time execution-speed controller for a Franka robot arm running a VisionLanguage-Action (VLA) policy on a real-world manipulation task. The VLA policy executes the SAME task, but it can run at one of a few discrete speed multipliers. Your ONLY job is to look at the current scene and decide which speed multiplier the robot should use for the next short segment of motion (about 2 seconds). Speed options (multiplier on the base execution speed), ordered from your most to least preferred: - 1.5 -> very fast (THIS IS YOUR DEFAULT; use it most of the time) - 1.25 -> fast (near objects or mid-approach, not yet touching) - 1.0 -> normal (only for alignment / grasping / insertion / precise placing) - 0.75 -> very slow (rare; only for exceptionally delicate or fragile manipulation) Decision principle (be AGGRESSIVE: go fast by default, slow down only when truly necessary): - Pick 1.5 for the vast majority of the task: any free-space motion, reaching or approaching, transporting an already-grasped object, retracting, or returning home. If no precise contact is happening right now, default to 1.5. - Pick 1.25 when the gripper is getting close to a target or the scene is mildly cluttered, but the precise contact action has not started yet (e.g. the mid approach before the final few centimeters). - Drop to 1.0 ONLY for the precise, contact-critical moments: the final fine alignment immediately before contact, grasping or picking an object, inserting or plugging, and placing precisely. Return to a high speed as soon as that precise action is finished. - Reserve 0.75 for rare, exceptionally delicate situations only; avoid it unless 1.0 is clearly too fast for an extremely fragile or tight manipulation. - When unsure, prefer the FASTER option. Do NOT be conservative: most segments of the task should run at 1.5 or 1.25, and only the actual alignment/grasp/insertion moments should slow to 1.0. You are given two camera views: 1. "exterior" - a third-person view of the whole workspace. 2. "wrist" - a close-up view from the camera mounted on the gripper, useful for judging proximity to objects and fine manipulation. First infer which PHASE of the task the robot is currently in, then pick the most appropriate speed, defaulting to a high speed (1.5, or 1.25 when close to a target) unless a precise contact action is happening right now. Respond with STRICT JSON only, no markdown, no extra text, in EXACTLY this schema: {"phase": "<one short phrase describing the current motion phase>", "speed": <one of 0.75, 1.0, 1.25, 1.5>, "reasoning": "<one short sentence>"}
Figure 7: Prompt sent to GPT-4o for dynamic speed control.
19
Taken together, these reasons explain why we prefer EEF actions for VSTA when both representations are available, although VSTA can still be applied directly on platforms that expose only joint commands without harming task-relevant performance in our setup.
H
Future Directions
Extending VSTA to non-composable action spaces. VSTA’s current implementation assumes the action space is closed under linear composition, which excludes representations such as unit quaternions, rotation matrices, and Euler angles. A one-time mapping to a tangent-space representation or an on-manifold interpolation scheme such as SLERP [59] extends VSTA to these representations without changing its core algorithm, broadening the plug-and-play scope of TempoVLA across more platforms. Co-tuning TempoVLA with the low-level controller. At the high end of the speed range, the realized speedup is bounded by the fixed low-level controller rather than by TempoVLA itself, since we deliberately leave the controller unchanged throughout this work to isolate the contribution of policy-level speed control. Pairing TempoVLA with controller-side adjustments such as a higher control frequency, a wider admissible action range, or finer sub-step action decomposition would push this ceiling further and let the policy’s full speed-control envelope translate into even larger executed speedups, consistent with the orthogonality view in Section G.1. Reducing the VLM scheduling latency. In our current dynamic speed control implementation, the GPT-4o scheduler is invoked synchronously between action chunks, which adds wall-clock overhead to the rollout. This cost can be hidden by feeding the scheduler a longer observation history and running it asynchronously in parallel with TempoVLA, so that scheduling decisions arrive in time for the next chunk without blocking inference. A thorough exploration of this engineering optimization is left to future work. Default speed regularization. TempoVLA currently treats the original demonstration speed as 1×, which implicitly assumes that the per-action granularity within a dataset is uniform. In practice, however, action granularity varies considerably across demonstrations and even across segments within the same demonstration, since human operators rarely move at a strictly constant pace. A cleaner formulation would first apply a VSTA-style normalization to flatten this within-dataset speed variability before defining the 1× reference, so that the speed scalar s would condition the policy on a deviation from a well-calibrated mean rather than from an inconsistent demonstrator pace. We expect this to sharpen the correspondence between the commanded speed and the realized execution speed, and leave a principled implementation of this normalization step to future work.
20