UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models Yukun Dai, Mingzhe Dai, Tianshi Wang1 , Fengling Li2 , Jingjing Li3 , Lei Zhu1 1 Tongji University Mohamed bin Zayed University of Artificial Intelligence 3 University of Electronic Science and Technology of China yukundai@tongji.edu.cn, daimingzhe@bupt.edu.cn, tswang0116@163.com, fenglingli2023@gmail.com, lijin117@yeah.net, leizhu0608@gmail.com 2

Abstract

(A) Universal 2D Patch

arXiv:2608.13453v1 [cs.CV] 13 Aug 2026

Task 1

Vision-Language-Action (VLA) models have emerged as generalist robotic policies capable of following diverse language instructions and performing a wide range of manipulation tasks. However, their direct control over embodied agents also exposes them to adversarial interference that may cause unsafe physical behaviors. Existing attacks on robotic policies are typically optimized for a single task or instruction, leaving the cross-task vulnerabilities of multitask VLAs largely unexplored. We introduce UniTexture, a cross-task universal adversarial texture attack that uses a single textured 3D object to induce targeted deviations in VLA action predictions across multiple tasks. UniTexture backpropagates gradients from the policy’s action outputs to surface texture parameters through a differentiable renderer. It jointly optimizes the shared texture over a distribution of tasks, instructions, states, and viewpoints using a targeted action-space objective, steering predicted actions toward attacker-defined targets without optimizing a separate texture for each task. We evaluate UniTexture on OpenVLA and π0.5 across diverse manipulation tasks and multiple evaluation settings. UniTexture reduces the mean task success rate from 90.0% under benign conditions to 48.4% under attack, induces target-aligned action shifts, and further exhibits cross-suite and cross-model transfer without re-optimization. Together, these findings reveal shared crosstask vulnerabilities in multitask VLAs that can be systematically exploited through a single adversarial surface texture.

Introduction Vision-language-action (VLA) models are emerging as generalist policies for robotic manipulation. By grounding natural-language instructions and visual observations in robot actions, a single policy can execute diverse tasks without requiring a separately engineered controller for each behavior (Brohan et al. 2023; Zitkovich et al. 2023; O’Neill et al. 2024; Kim et al. 2024; Black et al. 2024). This versatility, however, also broadens the attack surface: a single adversarial visual pattern may influence multiple behaviors governed by the same policy. Moreover, an erroneous VLA prediction is not merely a semantic mistake. Because the predicted action is executed in the environment, adversarial interference may lead to unsafe motion, failed manipulation, or physical damage.

Task 2

(B) Per-Task 3D Texture Task K

Frozen VLA Policy

Action Output

Task 1

Task K

Task 2

Frozen VLA Policy

VLA

Patch across Tasks

No Action Target

Task 2

Task K

Per-Task Texture

Feature-Space Proxy

Action Proxy

(D) UniTexture: Visualized Results (π0.5 on LIBERO-Spatial)

(C) UniTexture: Cross-Task Optimization Task 1

3D Object Attack

2D Image Attack

Feature/Attention/ Semantic Objective

Action Output

VLA

Success Rates

Frozen VLA Policy

100

Action Output

99

97

plate bowl

98

VLA

33

Target Action Loss

Original

Gaussian

40

UniTexture

Target Direction Shift 17.9

plate bowl

10.6 Targeted Output Action Space 3D Object Attack across Tasks

-0.1 0.2 Original

1.8 0.8 Gaussian

UniTexture

Figure 1: Overview and comparison of UniTexture. (A) Universal 2D patches transfer across tasks but lack 3D surface constraints and explicit action targets. (B) Existing 3D texture attacks are object-bound but require task-specific textures and feature-space proxies. (C) UniTexture optimizes one shared object-bound texture across tasks using targets in the VLA’s native action space. (D) On π0.5 in LIBERO-Spatial, UniTexture reduces task success and induces larger target-direction action shifts than the original and Gaussian baselines.

Existing visual attacks on VLAs have progressed along several complementary directions. Image-space attacks demonstrate the sensitivity of VLA action predictions to adversarial observations (Wang et al. 2024). Universal physical patches further show that a single two-dimensional pattern can affect multiple tasks, viewpoints, and model architectures (Lu et al. 2026a). Separately, Tex3D maps adversarial textures onto three-dimensional object surfaces, preserving their spatial correspondence as the object moves across viewpoints and interaction trajectories (Chen et al. 2026). However, universal patches lack object-surface constraints and direct targeted control in the policy’s action space, whereas Tex3D optimizes separate textures for individual tasks. Whether a single geometry-aware texture can consistently control a multitask VLA without per-task optimization therefore remains unclear.

This setting is more demanding than causing an isolated task failure. Across tasks, the instruction, scene context, nominal action distribution, and interaction trajectory all change. The target object’s pose, visibility, and image footprint also evolve during manipulation and across camera views. Consequently, gradients from different tasks must produce one texture that remains effective under these coupled variations. Targeted control introduces a further requirement: the objective must encode the attacker’s intended behavior in the policy’s action space, rather than treating arbitrary feature displacement as a proxy for success. The attack must satisfy these requirements while remaining fixed on the same physical object. We introduce UniTexture, a cross-task universal adversarial texture attack for VLAs, as illustrated in Figure 1. Given a frozen target VLA, a target object, and a collection of manipulation tasks, UniTexture jointly optimizes a single shared surface texture over a cross-task training distribution. We first calibrate a differentiable renderer using clean multiview observations by aligning the scene geometry and estimating photometric parameters, after which all rendering parameters are frozen. During attack optimization, we use task-balanced sampling to obtain task-conditioned observations, object poses, and language instructions. The renderer composites the shared texture into each observation, while targeted objectives defined in the model’s native action representation propagate gradients from the VLA outputs to the texture parameters. Only the texture is updated, and the resulting texture is applied unchanged across all tasks, without separate per-task optimization or refinement. We evaluate UniTexture on OpenVLA (Kim et al. 2024) and π0.5 (Intelligence et al. 2025) using LIBERO-Spatial and LIBERO-Goal (Liu et al. 2023). For each model-object-suite configuration, a single texture is jointly optimized for all tasks in the corresponding suite. Across the evaluated settings, UniTexture reduces the mean task success rate from 90.0% under benign conditions to 48.4% under attack, while consistently shifting predicted actions toward attacker-specified targets. Cross-suite and cross-model evaluations further demonstrate that the optimized textures retain attack effectiveness beyond the task distribution and model used during optimization, without re-optimization. These findings expose a deployment-level vulnerability: task diversity alone does not prevent a single adversarial object appearance from repeatedly influencing a multitask robotic policy. Our contributions are summarized as follows: • Cross-task universal 3D texture attack. We formulate a threat model in which a single object-bound texture is jointly optimized across multiple tasks and applied unchanged, without per-task optimization or refinement. • Targeted action-space optimization. We develop model-compatible objectives and metrics that directly encode attacker-specified targets in each VLA’s native action space, rather than relying on feature-space proxies or untargeted failure. • Comprehensive cross-task evaluation. Experiments on OpenVLA and π0.5 across LIBERO-Spatial and LIBERO-Goal demonstrate cross-task attack effective-

ness, targeted action manipulation, and transfer across task suites and model architectures.

Related Work Vision-Language-Action Models Vision-language-action (VLA) models integrate visual perception, language conditioning, and action generation into unified policies for robot control (Zitkovich et al. 2023). Because a single policy is reused across diverse tasks, shared visual inputs create a common attack surface across behaviors. However, VLA architectures differ substantially in their action representations: OpenVLA (Kim et al. 2024) uses autoregressive action tokens, the π family (Black et al. 2024; Intelligence et al. 2025) employs flow-matching action experts, and recent variants introduce continuous prediction or alternative action tokenizations (Kim, Finn, and Liang 2025; Pertsch et al. 2025). This heterogeneity motivates attack objectives defined in each model’s native action space.

Adversarial Attacks on VLA Models Building on classical digital and physical attacks (Szegedy et al. 2014; Madry et al. 2018; Eykholt et al. 2018; Athalye et al. 2018), VLA attacks include training-time backdoors (Zhou et al. 2025), language-based attacks (Jones et al. 2025), and deployment-time visual attacks (Wang et al. 2024; Cheng et al. 2024; Lu et al. 2026b; Zhang et al. 2026; Guo et al. 2025; Cui et al. 2026). Existing visual attacks mainly pursue universality or geometric persistence. Imagespace attacks reveal vulnerabilities in VLA action prediction (Wang et al. 2024), while UPA-RFAS (Lu et al. 2026a) and VLA-Hijack (Fu et al. 2026) optimize transferable 2D patches through feature-, attention-, or semantic-level objectives. These patches, however, are composited in the image plane rather than bound to object geometry. Tex3D (Chen et al. 2026), building on texture attacks against navigation agents (Liu et al. 2020), maps adversarial textures onto 3D object surfaces through differentiable rendering, but optimizes separate textures for individual tasks and relies on a visual-feature proxy for π-family policies. UniTexture addresses the intersection of these directions by jointly optimizing one object-bound texture across tasks using attackerspecified targets in each VLA’s native action space.

Method Threat Model and Problem Formulation We attack a frozen VLA policy F that maps an RGB observation and a language instruction to robot actions. The attacker controls only the surface appearance of a single object in the workspace: it may repaint that object’s texture, but cannot modify the policy weights, the instruction, the camera pose, the sensing pipeline, or any other object in the scene. Given a task suite S = {τ1 , . . . , τK }, a target object, and an attacker-chosen target action a⋆ , the attacker optimizes one texture offline and then deploys the repainted object unchanged on every task in S. Gradients of F are available during optimization, but F is never updated. Let θ ∈ [0, 1]H×W ×3 denote the UV texture map of the target object. We optimize θ while keeping the object’s mesh

1 Cross-Task Distribution Per-task attacks:

3 Cross-Task Deploy

Offline Renderer Calibration Geometry Alignment

Photometric Calibration

Fit Clean Render Clean Fitted

UniTexture: Camera

Task 1 Pick up the black bowl between the plate and...

Object Pose

View Light

Cam.

Obj. Pose

Task Sampling

Shared Texture

Object Material

Task 2

Frozen

2 UniTexture Joint Optimization

Task 2 Pick up the black bowl ... on the plate

Mask

Task 1

Mask Render

Task K

Fixed Mesh Agent View

Wrist View Instruction

Task K Pick up the black bowl next to the plate and ...

Update

Frozen VLA Policy

One Shared Texture

Attacker-Specified Action Target

No Task-Specific Refine

Targeted Action Loss

Target Action Control

Figure 2: Overview of UniTexture. (1) A task suite induces a cross-task distribution: rather than fitting a separate texture per task, UniTexture optimizes one shared texture θ for all tasks. Before joint optimization, an offline calibration stage fits per-view lighting ψvL and per-object material ψom so that the rendered clean object matches the simulator’s appearance; the resulting ψ ⋆ is then frozen. (2) During joint optimization, each sampled frame supplies its rendering condition (Ci , Pi , Mi ); the shared texture is rendered on the fixed mesh, composited into the agent view (OpenVLA and π0.5 ) and the wrist view (π0.5 ), and passed with the instruction L to the frozen VLA policy. A targeted action loss compares the predicted action a with the attacker-specified target a⋆ and backpropagates to θ alone. (3) The optimized texture is deployed unchanged across all tasks.

where p(τ ) is the task-sampling distribution over S, Dτ is the demonstration distribution of task τ , and a⋆ is the attackerspecified target action.

comprising view-specific lighting parameters ψvL and objectspecific material parameters ψom . Here, V = {agent, wrist} denotes the policy views, O denotes the calibrated objects, and Q = {amb, diff, spec} indexes the ambient, diffuse, and specular components. For each view v, pW v denotes the world-space light position and cqv denotes the RGB color or intensity of lighting component q. For each object o, mqo denotes the corresponding material coefficient and msh o denotes its shininess parameter. The lighting parameters for each view are shared across all objects and tasks, whereas the material parameters for each object are shared across all views and tasks, with each mqo broadcast across the RGB channels. The world-space light position is transformed into the object frame for each sample using its pose P . Let Dcal denote the clean calibration distribution obtained by first sampling a view–object pair and then a calibration frame ζ = (I, C, P, M ) for that pair. We obtain the calibrated parameters from frames containing the original texture θ0 by minimizing the masked photometric discrepancy:

Calibrating and Freezing the Renderer

ψ ⋆ = argmin

geometry fixed. Each demonstration frame supplies a tuple ξ = (I, L, C, P, M ): a policy-aligned observation I, its instruction L, camera parameters C, the object’s 6-DoF pose P , and the object’s image-space support mask M . Writing Rψ for the differentiable rendering of the textured object under rendering parameters ψ, the attacked observation is the composite Ω(θ; I, C, P, M, ψ) = M ⊙Rψ (θ; C, P )+(1−M )⊙I. (1) Using the calibrated rendering parameters ψ ⋆ defined below, we write the attacked policy input as Ie = Ω(θ; I, C, P, M, ψ ⋆ ). UniTexture optimizes a single shared texture over the target task distribution, h  i e L); a⋆ . (2) θ⋆ = argmin E E Ltgt F (I, θ

τ ∼p(τ ) ξ∼Dτ

Gradients reaching θ through (1) are only useful when the rendered object is photometrically consistent with the policy observations; otherwise, the attack may optimize against rendering artifacts. We therefore calibrate
ψ = {ψvL }v∈V , {ψom }o∈O ,
q (3) ψvL = pW v ∈ V, v , {cv }q∈Q ,
m q sh ψo = {mo }q∈Q , mo , o ∈ O,

ψ

E

ζ∼Dcal

[H (M ⊙ Rψ (θ0 ; C, P ), M ⊙ I)] .

(4) where H denotes the Smooth-L1 loss over masked object pixels, and ψ ⋆ is fixed during texture optimization.

Model-Specific Targeted Objectives To accommodate heterogeneous VLA action interfaces, we instantiate Ltgt for autoregressive tokens and flow-matching action chunks.

Autoregressive action tokens. Autoregressive action-token policies represent each action dimension as a discrete token and generate the resulting token sequence autoregressively. We apply targeted token supervision (Wang et al. 2024) to optimize the shared object texture through differentiable rendering. Let Qact denote the policy’s action tokenizer. Given the attacked observation Ie defined above, we define z⋆ = Qact (a⋆ ),   1 X (5) ˜L , Ltok log pF zj⋆ I, tgt (θ) = − |J | j∈J

where zj⋆ is the target token for action dimension j, pF (zj⋆ | e L) denotes the conditional probability that the frozen polI, icy F assigns to this token, and J contains the attackerselected dimensions. In implementation, we mask all nontarget action-token positions so that they do not contribute to the targeted loss. Flow-matching action chunks. Policies with flowmatching action experts generate continuous action chunks. Let A ∈ RHa ×d denote the clean action chunk associated with a demonstration frame, where Ha is the number of action steps and d is the total number of action dimensions. We construct the targeted chunk A⋆ by replacing only the attacker-selected action dimension j across the entire action horizon: clip[−1,1] (a⋆ ), k = j, h = 1, . . . , Ha . (6) Ah,k , k ̸= j, Here, h indexes the action steps, k indexes the action dimensions, and a⋆ is the attacker-specified target value. The targeted chunk is then interpolated with Gaussian noise ϵ at flow time t: xt = tϵ + (1 − t)A⋆ , ut = ϵ − A⋆ . (7) Here, ϵ is a Gaussian noise chunk with the same shape as A⋆ , t ∈ [0, 1] is the flow-matching time, xt is the corresponding noisy action chunk, and ut is its target velocity. Given xt , the frozen π0.5 action expert predicts a velocity field. We minimize its flow-matching residual only on the attacker-selected action dimension j: Ha h i2 1 X ˜ L, xt , t)h,j − (ut )h,j . (8) Lflow vF (I, tgt (θ) = Ha A⋆h,k =



h=1

Here, vF denotes the velocity field predicted by the frozen π0.5 policy F . The indices (h, j) select the targeted action dimension at action step h; all non-target action dimensions are excluded from the loss.

Optimization We optimize only the shared texture θ, keeping the policy F , object geometry, and calibrated renderer parameters ψ ⋆ fixed. At each outer iteration, we sample a minibatch of frames and apply R consecutive texture updates. For each update, we re-render the texture under the frame-specific conditions (Ci , Pi , Mi ), composite the object using (1), evaluate the targeted objective, and update θ with AdamW. The texture is then clamped elementwise to [0, 1]. Algorithm 1 summarizes the procedure.

Algorithm 1 UniTexture: cross-task universal texture Require: frozen policy F ; task suite S; target-object mesh; demonstration distributions {Dτ }; target specification (a⋆ , J ); outer iterations N ; inner updates R 1: fit ψ ⋆ on clean frames by (4); freeze ψ ⋆ 2: initialize θ ← θinit 3: for n = 1 to N do 4: sample B = {ξi }B i=1 with τi ∼ p(τ ) and ξi ∼ Dτi 5: for r = 1 to R do 6: I˜i ← Ω(θ; Ii , Ci , Pi , Mi , ψ ⋆ ), i = 1, . . . , B P 7: L ← B1 i Ltgt (I˜i , Li ; a⋆ , J ) 8: θ ← AdamW(θ, ∇θ L) 9: θ ← clip(θ, 0, 1) 10: end for 11: end for 12: return texture θ ⋆

Experiments Experimental Setup Victim Policies and Task Suites. We attack two VLA models that instantiate the action interfaces described above. For the autoregressive action-token interface, we use the suite-specific OpenVLA (Kim et al. 2024) checkpoints openvla-7b-finetuned-libero-spatial and openvla-7b-finetuned-libero-goal. For the flow-matching action-chunk interface, we use the official π0.5 (Intelligence et al. 2025) checkpoint pi05_libero. OpenVLA consumes the agent view, whereas π0.5 consumes both the agent and wrist views together with proprioceptive state; we composite the rendered textured object into every visual stream while leaving proprioception unchanged. We evaluate both policies on LIBERO-Spatial and LIBERO-Goal (Liu et al. 2023), each comprising 10 language-conditioned manipulation tasks. Both suites contain the same plate and bowl objects across all tasks, enabling within-suite cross-task evaluation and controlled cross-suite transfer without changing the target object. Because OpenVLA uses suite-specific fine-tuned checkpoints, its crosssuite evaluation transfers the texture across both task suites and fine-tuned checkpoints, whereas π0.5 retains the same checkpoint across suites. Attack Configuration. For each policy-suite pair, we optimize separate object-bound textures for the plate and bowl, with each texture shared across all 10 tasks in the suite. Unless otherwise stated, we target the z-translation dimension (j = 2) with a⋆ = +1, corresponding to an upward command. Each texture is optimized for 1001 outer iterations using AdamW with batch size 8, an initial learning rate of 10−3 , and cosine decay. For each sampled minibatch, we perform 50 consecutive texture updates before sampling the next minibatch. We render at 224 × 224 using PyTorch3D (Ravi et al. 2020) with soft Phong shading and 8 faces per pixel. Evaluation Protocol. Each setting covers all 10 tasks with 10 rollouts each, totaling 100 episodes. Following the official LIBERO initialization and OpenVLA evaluation protocols (Liu et al. 2023; Kim et al. 2024), we execute 10

π0.5

OpenVLA Condition

Spatial plate

Goal bowl

Spatial bowl

plate

78.0 62.0 29.0

100.0 97.0 33.0

0.4 2.2 2.5

− 0.1 1.8 10.6

pDHR / DHR (%; pDHR for rendered conditions; DHR for Clean) Clean 14.8 14.2 Original 23.4 29.1 21.8 26.0 Gaussian 27.4 30.3 27.6 28.0 UniTexture 37.6 33.3 33.8 32.5

16.6 44.0 61.0

SR (%, lower indicates stronger task disruption) Clean 84.0 Original 80.0 77.0 Gaussian 80.0 53.0 UniTexture 53.0 25.0

plate 80.0 77.0 67.0 58.0

TDS / TDA ( TDS for rendered conditions; TDA for Clean) Clean −19.2 −23.5 Original 0.2 − 0.8 − 0.1 Gaussian 1.5 0.6 1.6 UniTexture 3.0 − 2.4 1.9

Goal bowl

plate

99.0 98.0 40.0

96.0 97.0 77.0

0.2 0.8 17.9

0.1 0.4 6.6

39.3 42.6 65.5

23.0 31.7 49.3

99.0

bowl 97.0

−21.0

95.0 95.0 72.0 −26.9

14.0

0.3 0.3 4.9 16.0 28.5 29.6 44.7

Table 1: Within-suite results and non-adversarial controls. Each UniTexture is jointly optimized across all ten tasks of one suite and deployed without task-specific refinement; each model-object-suite setting is evaluated over 100 episodes. Clean observations bypass rendering, so each clean result spans the corresponding plate and bowl columns. dummy actions after resetting the simulator and begin evaluation only after objects settle. OpenVLA predicts one action per step, whereas π0.5 executes the first five actions of each predicted chunk before re-querying, following the OpenPI LIBERO protocol. At each query, clean and attacked predictions are computed from the same simulator state and, for π0.5 , the same flow-sampling noise. Only attacked actions are executed; clean predictions serve as step-aligned counterfactuals along the attacked trajectory. Our metrics distinguish nominal behavior, attack-induced action shifts, and task outcomes. For an episode with T steps, let i index the step, j the targeted action dimension, and σ the attacker-specified direction. We first characterize the absolute clean behavior using the Target Direction Action(TDA) and Direction Hit Rate(DHR): T 1 X clean σa × 100, TDA = T i=1 i,j T

DHR =

1 X  clean 1 σai,j ≥ 0.01 × 100%. T i=1

Here, 1 is the indicator function, returning 1 when the condition holds and 0 otherwise. TDA measures the mean signed clean action along the target direction, while DHR is the percentage of clean steps following that direction with magnitude at least 0.01. For rendered conditions, we measure the change induced by the texture relative to the paired clean prediction. The Target Direction Shift(TDS) and Paired Direction Hit Rate(pDHR) are T
1X clean TDS = σ aadv × 100, i,j − ai,j T i=1 T

pDHR =

1X  clean 1 σ aadv ≥ 0.01 × 100%. i,j − ai,j T i=1

TDA and DHR characterize the clean policy’s nominal directional behavior, whereas TDS and pDHR isolate the directional effect attributable to the attack. A positive TDS therefore provides direct evidence that the attack steers the predicted action toward the attacker-specified direction relative to a state-matched clean prediction. Finally, Task Success Rate(SR) is the percentage of successfully completed episodes, with lower values indicating stronger task disruption. Non-Adversarial Texture Controls. To disentangle targeted optimization from renderer-induced discrepancies and appearance changes, we include a renderer-free clean reference and two non-adversarial texture controls. Clean observation uses the observation image produced directly by LIBERO, bypassing our rendering and compositing pipeline. Rendered original texture passes the object’s original asset texture through the frozen calibrated renderer and the same compositing procedure used by UniTexture, isolating changes introduced by rendering and composition. Rendered Gaussian-noise texture instead uses an unoptimized Gaussian-noise texture while keeping the rendering and compositing procedure unchanged, sampled once from N (0.5, 0.22 ) and held fixed throughout evaluation. Neither texture control is optimized using Ltgt . All three conditions use the same tasks, initial states, rollout budgets, and policy execution settings as the adversarial evaluation.

Results Within-Suite Cross-Task Attack. Table 1 shows that a single texture jointly optimized across a suite reduces task success in all eight model–object–suite settings without task-specific refinement. For OpenVLA, SR drops from the renderer-free clean baselines of 84% on Spatial and 80% on Goal to 53%/25% (plate/bowl) and 58%/29%, respectively; for π0.5 , it falls from 99% and 97% to 33%/40% and

Figure 3: Representative failed π0.5 rollout under a UniTexture attack on the plate in LIBERO-Spatial. The leftmost agent view shows projected gripper-center trajectories for the clean rollout (green; TDA = −12.4, DHR = 12.5%) and attacked rollout (red; TDS = 18.3, pDHR = 58.6%), interpolated from centers tracked every 20 frames. The remaining panels show the attacked rollout in temporal order, with agent views above wrist views.

Figure 4: Per-task results of the two object-specific UniTexture attacks against π0.5 on LIBERO-Spatial. The plate and bowl textures yield aggregate SRs of 33% and 40%, respectively, in Table 1. Each texture is jointly optimized across all ten tasks and deployed unchanged, with 10 rollouts per task. T1–T10 follow the suite’s standard task order; the panels show SR, TDS, and pDHR from top to bottom, and the dashed line marks the 100% clean-reference SR. Both textures produce positive TDS on every task, while the resulting reduction in SR varies across tasks. 77%/72%. UniTexture also yields lower SR than both the rendered original-texture and Gaussian-noise controls in every setting, indicating that the degradation is not explained by rendering and compositing discrepancies or an arbitrary texture change alone. At the suite-aggregate level, the reduction relative to clean ranges from 20 points for π0.5 on Goal-plate to 66 points on Spatial-plate, showing broad but architecture- and suite-dependent effectiveness. Targeted Control versus Disruption. For π0.5 , UniTexture achieves both targeted control and task disruption, most clearly on Spatial. The TDS reaches 10.6/17.9, and pDHR reaches 61.0%/65.5% for plate/bowl, while SR falls to

33%/40%. On Goal, positive TDS of 6.6/4.9 and pDHR of 49.3%/44.7% coexist with substantially higher SR of 77%/72%. Thus, a persistent target-aligned shift does not necessarily translate into task failure. OpenVLA exhibits a different pattern. UniTexture reduces SR to 53%/25% on Spatial and 58%/29% on Goal, but its TDS remains modest and becomes negative (−2.4) in the Spatial-bowl setting. This setting produces OpenVLA’s largest success reduction despite shifting the action in the opposite direction on average, directly separating disruption from targeted control. OpenVLA also has lower clean SR than π0.5 on the evaluated suites (80–84% versus 97–99%), which should temper cross-policy comparisons of absolute attacked SR. We attribute this decoupling to OpenVLA’s discrete autoregressive action interface. The token-level objective can alter the targeted action token and propagate its effect through subsequent decoding, disrupting the action sequence without inducing a coherent signed displacement in the targeted dimension. By contrast, the flow-matching objective directly steers a continuous action chunk, yielding the larger positive TDS and pDHR observed for π0.5 . Gaussian-noise textures already produce nontrivial pDHR, reaching 44.0%, so pDHR should be interpreted relative to its matched control. Nevertheless, UniTexture exceeds the corresponding Gaussian pDHR in all eight settings. Overall, UniTexture provides stronger target-aligned control for π0.5 , whereas OpenVLA can be substantially disrupted without reliable steering in the attacker-specified direction. Figure 3 illustrates how target-aligned action shifts accumulate into task-level disruption. In this episode, the clean behavior predominantly opposes the targeted +z direction, with a TDA of −12.4 and a DHR of 12.53%. UniTexture nevertheless induces a large positive paired shift, reaching a TDS of 18.3 and a pDHR of 58.6%. Compared with the smooth clean reference trajectory (green), the attacked trajectory (red) alternates between downward corrective motions and attack-induced upward motions before task failure. Task-Level Coverage of Shared Textures. Figure 4 expands the π0.5 LIBERO-Spatial results in Table 1 into their task-level outcomes. The targeted effect spans the entire suite:

Policy

Suite

Object

SR

TDS

pDHR (%)

Cross-suite (same policy, texture reused on the other suite) OpenVLA S→G plate 65.0 1.5 33.0 OpenVLA S→G bowl 39.0 5.3 35.2 OpenVLA G→S plate 72.0 2.9 32.5 OpenVLA G→S bowl 24.0 − 1.6 31.9 π0.5 S→G plate 74.0 3.0 42.7 π0.5 S→G bowl 81.0 5.1 44.2 π0.5 G→S plate 78.0 6.6 60.3 π0.5 G→S bowl 53.0 13.4 59.5 Cross-model (same suite, texture reused on the other policy) O→ π Spatial plate 92.0 3.4 52.2 O→ π Spatial bowl 95.0 2.2 50.0 O→ π Goal plate 94.0 1.0 36.6 O→ π Goal bowl 97.0 1.0 36.3 π →O Spatial plate 61.0 3.4 33.9 π →O Spatial bowl 26.0 0.7 34.8 π →O Goal plate 55.0 2.5 32.6 π →O Goal bowl 41.0 1.7 30.7

Table 2: Cross-suite and cross-model transfer without re-optimization. Arrows indicate the optimization-toevaluation direction; S/G and O/π denote Spatial/Goal and OpenVLA/π0.5 , respectively. OpenVLA cross-suite transfer also changes the suite-specific checkpoint.

both object-specific shared textures produce positive TDS on all ten tasks, with pDHR ranging from 42% to 90% across all 20 task–object pairs. The attack also reduces SR below the 100% per-task reference in every pair, including complete failure on two plate tasks. These results show that the suite-level effectiveness is not driven by a small subset of vulnerable tasks: each shared texture induces the specified action direction throughout the task suite without task-specific refinement. Differences in residual SR instead characterize how individual tasks respond to the induced action shift, rather than whether the attack reaches them. Cross-Suite and Cross-Model Transfer. Table 2 evaluates direct texture reuse when either the task suite or the victim policy changes. Cross-suite transfer reduces SR below both the renderer-free clean baseline and the corresponding Gaussian-texture control in all eight settings. For π0.5 , the directional objective also transfers consistently: TDS remains positive in all four settings, with pDHR ranging from 42.7% to 60.3%. Cross-model transfer is asymmetric in task disruption. Textures optimized for π0.5 reduce OpenVLA SR to 26%–61%, whereas textures optimized for OpenVLA leave π0.5 SR at 92%–97%. This asymmetry reflects a substantial difference in robustness to object appearance. Under the matched nonadversarial controls, replacing the rendered original texture with a Gaussian-noise texture reduces OpenVLA’s Spatial– bowl SR from 77% to 53%, whereas π0.5 changes only from 99% to 98%. This sensitivity becomes more pronounced under adversarial textures. The directly optimized OpenVLA– Spatial–bowl texture, the transferred OpenVLA–Goal–bowl texture, and the transferred π0.5 –Spatial–bowl texture produce nearly identical SRs of 25%, 24%, and 26%, respectively. Despite this severe disruption, all three exhibit nearzero or negative TDS and pDHR of only 31.9%–34.8%,

Dim.

Semantics

SR

TDS

pDHR (%)

0 0 1 1 2 2

+x −x +y −y +z −z

62.0 38.0 68.0 45.0 33.0 91.0

4.8 5.6 3.8 4.3 10.6 2.7

48.1 55.3 50.8 55.1 61.0 36.4

Table 3: Effect of the targeted action dimension and direction (π0.5 -Spatial-plate). The upward z target used throughout the paper is the most effective choice, while its downward counterpart is the least effective.

close to the Gaussian control of 30.3%. Weak directional metrics therefore do not indicate an ineffective attack: the textures strongly disrupt OpenVLA, but do so by destabilizing its visual–action mapping rather than by inducing a coherent shift in the specified direction. We attribute this failure mode to OpenVLA’s limited robustness to object appearance, which allows texture-induced instability to overwhelm the directional signal encouraged by the targeted objective. In contrast, π0.5 remains stable under the Gaussian texture and exhibits strong targeted control in the same Spatial–bowl setting, reaching TDS of 17.9 and pDHR of 65.5%. Which Action Target Matters. Table 3 varies the targeted action dimension and direction while holding the policy, object, task suite, and optimization configuration fixed. Under our evaluation coordinate convention, +z and −z denote upward and downward motion, +y and −y denote leftward and rightward motion in the camera view, and +x and −x denote motion toward and away from the camera, respectively. The evaluated horizontal targets remain controllable, with pDHR between 48.1% and 55.1%, but produce weaker task disruption, with SR between 45% and 68%. The opposite vertical target, −z, is the least effective on both criteria, retaining an SR of 91% while reaching only 2.7 TDS and 36.4% pDHR. We attribute this directional asymmetry to the manipulation geometry: a persistent upward bias pulls the end effector away from the interaction surface and disrupts reaching and contact, whereas a downward bias often agrees with the nominal approach motion.

Conclusion UniTexture shows that task diversity does not inherently protect a multitask VLA from a shared visual attack surface: one object-bound texture can induce cross-task adversarial effects without per-task refinement. By jointly optimizing the texture over a task distribution, UniTexture turns a persistent object appearance into an attack shared across different instructions, scenes, and trajectories. Results on OpenVLA and π0.5 demonstrate attacker-specified directional shifts and task disruption, while cross-suite and asymmetric cross-model transfer show that some effects extend beyond the optimization setting. These findings argue that VLA robustness should be evaluated against persistent perturbations shared across tasks, rather than only task-specific attacks.

References Athalye, A.; Engstrom, L.; Ilyas, A.; and Kwok, K. 2018. Synthesizing robust adversarial examples. In International conference on machine learning, 284–293. PMLR. Black, K.; Brown, N.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Groom, L.; Hausman, K.; Ichter, B.; et al. 2024. π0 : A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint arXiv:2410.24164. Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Dabis, J.; Finn, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; Hsu, J.; et al. 2023. Rt-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems. Chen, J.; Huang, S.; Du, J.; Chen, S.; Tian, Y.; Wei, M.; Yu, C.; and Yin, Z. 2026. Tex3D: Objects as Attack Surfaces via Adversarial 3D Textures for Vision-Language-Action Models. arXiv preprint arXiv:2604.01618. Cheng, H.; Xiao, E.; Yu, C.; Yao, Z.; Cao, J.; Zhang, Q.; Wang, J.; Sun, M.; Xu, K.; Gu, J.; and Xu, R. 2024. Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models. arXiv preprint arXiv:2409.13174. Cui, R.; Zhang, Z.; Pang, J.; Chi, H.; Guo, J.; Zhang, S.; Xie, S.; Jin, X.; Mu, Y.; Yang, J.; Yao, G.; Zhan, X.; Zhang, Y.-Q.; and Zhao, H. 2026. LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in VisionLanguage-Action Models. In European Conference on Computer Vision. Eykholt, K.; Evtimov, I.; Fernandes, E.; Li, B.; Rahmati, A.; Xiao, C.; Prakash, A.; Kohno, T.; and Song, D. 2018. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1625–1634. Fu, J.; Jiang, K.; Jia, J.; Chen, Z.; Chen, X.; Hong, L.; Gao, S.; Tan, C.; Yang, D.; and Zhang, W. 2026. VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking. arXiv preprint arXiv:2605.28083. Guo, J.; Wu, Z.; Tu, C.; Ma, Y.; Kong, X.; Liu, Z.; Ji, J.; Zhang, S.; Chen, Y.; Chen, K.; Dou, Q.; Yang, Y.; Liu, X.; Zhao, H.; Lv, W.; and Li, S. 2025. On Robustness of VisionLanguage-Action Model against Multi-Modal Perturbations. arXiv preprint arXiv:2510.00037. Intelligence, P.; Black, K.; Brown, N.; Darpinian, J.; Dhabalia, K.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Galliker, M. Y.; Ghosh, D.; Groom, L.; Hausman, K.; Ichter, B.; Jakubczak, S.; Jones, T.; Ke, L.; LeBlanc, D.; Levine, S.; Li-Bell, A.; Mothukuri, M.; Nair, S.; Pertsch, K.; Ren, A. Z.; Shi, L. X.; Smith, L.; Springenberg, J. T.; Stachowicz, K.; Tanner, J.; Vuong, Q.; Walke, H.; Walling, A.; Wang, H.; Yu, L.; and Zhilinsky, U. 2025. π0.5 : a VisionLanguage-Action Model with Open-World Generalization. arXiv:2504.16054. Jones, E. K.; Robey, A.; Zou, A.; Ravichandran, Z.; Pappas, G. J.; Hassani, H.; Fredrikson, M.; and Kolter, J. Z. 2025. Adversarial Attacks on Robotic Vision Language Action Models. arXiv preprint arXiv:2506.03350.

Kim, M. J.; Finn, C.; and Liang, P. 2025. Fine-Tuning VisionLanguage-Action Models: Optimizing Speed and Success. In Robotics: Science and Systems. Kim, M. J.; Pertsch, K.; Karamcheti, S.; Xiao, T.; Balakrishna, A.; Nair, S.; Rafailov, R.; Foster, E. P.; Sanketi, P. R.; Vuong, Q.; et al. 2024. OpenVLA: An Open-Source VisionLanguage-Action Model. In Agrawal, P.; Kroemer, O.; and Burgard, W., eds., 8th Annual Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, 2679–2713. PMLR. Liu, A.; Huang, T.; Liu, X.; Xu, Y.; Ma, Y.; Chen, X.; Maybank, S. J.; and Tao, D. 2020. Spatiotemporal attacks for embodied agents. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16, 122–138. Springer. Liu, B.; Zhu, Y.; Gao, C.; Feng, Y.; Liu, Q.; Zhu, Y.; and Stone, P. 2023. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 44776–44791. Curran Associates, Inc. Lu, H.; Yu, Y.; Yang, Y.; Yi, C.; Zhang, Q.; Shen, B.; Kot, A. C.; and Jiang, X. 2026a. When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-LanguageAction Models. arXiv:2511.21192. Lu, X.; Chen, J.; Xiao, S.; Jin, Z.; Chen, Z.; Yu, H.; Qian, B.; Zhou, R.; Ji, X.; and Xu, W. 2026b. Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks. In Proceedings of the AAAI Conference on Artificial Intelligence. Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; and Vladu, A. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations. O’Neill, A.; Rehman, A.; Maddukuri, A.; Gupta, A.; Padalkar, A.; Lee, A.; Pooley, A.; Gupta, A.; Mandlekar, A.; Jain, A.; et al. 2024. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 6892–6903. IEEE. Pertsch, K.; Stachowicz, K.; Ichter, B.; Driess, D.; Nair, S.; Vuong, Q.; Mees, O.; Finn, C.; and Levine, S. 2025. FAST: Efficient Action Tokenization for Vision-Language-Action Models. arXiv preprint arXiv:2501.09747. Ravi, N.; Reizenstein, J.; Novotny, D.; Gordon, T.; Lo, W.-Y.; Johnson, J.; and Gkioxari, G. 2020. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501. Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.; and Fergus, R. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014. Wang, T.; Han, C.; Liang, J. C.; Yang, W.; Liu, D.; Zhang, L. X.; Wang, Q.; Luo, J.; and Tang, R. 2024. Exploring the adversarial vulnerabilities of vision-language-action models in robotics. arXiv preprint arXiv:2411.13587.

Zhang, Y.; Zhang, B.; Fan, J.; Shen, J.; Cai, Y.; Yang, Y.; and Ji, J. 2026. RedVLA: Physical Red Teaming for VisionLanguage-Action Models. arXiv preprint arXiv:2604.22591. Zhou, X.; Tie, G.; Zhang, G.; Wang, H.; Zhou, P.; and Sun, L. 2025. BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization. arXiv preprint arXiv:2505.16640. Zitkovich, B.; Yu, T.; Xu, S.; Xu, P.; Xiao, T.; Xia, F.; Wu, J.; Wohlhart, P.; Welker, S.; Wahid, A.; Vuong, Q.; Vanhoucke, V.; Tran, H.; Soricut, R.; Singh, A.; Singh, J.; Sermanet, P.; Sanketi, P. R.; Salazar, G.; Ryoo, M. S.; Reymann, K.; Rao, K.; Pertsch, K.; Mordatch, I.; Michalewski, H.; Lu, Y.; Levine, S.; Lee, L.; Lee, T.-W. E.; Leal, I.; Kuang, Y.; Kalashnikov, D.; Julian, R.; Joshi, N. J.; Irpan, A.; Ichter, B.; Hsu, J.; Herzog, A.; Hausman, K.; Gopalakrishnan, K.; Fu, C.; Florence, P.; Finn, C.; Dubey, K. A.; Driess, D.; Ding, T.; Choromanski, K. M.; Chen, X.; Chebotar, Y.; Carbajal, J.; Brown, N.; Brohan, A.; Arenas, M. G.; and Han, K. 2023. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. In Tan, J.; Toussaint, M.; and Darvish, K., eds., Proceedings of The 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, 2165–2183. PMLR.

Related documents

• Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models#1129887
• MatchingPolicy: Correspondence-Aware Policy Enables Cross-Object In-Context Learning#625276
• MatchingPolicy: Correspondence-Aware Policy Enables Cross-Object In-Context Learning#623003
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels#130752
• Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models#609221