arXiv:2606.22966v1 [cs.LG] 22 Jun 2026
Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models Linghan Chen Adelaide University [email protected]
Kaiyan Ji Adelaide University [email protected]
Minyu Guo* Adelaide University [email protected]
June 23, 2026 Abstract Many recent vision-language-action (VLA) policies adopt an imagine-then-act design. A world-action model (WAM) first imagines a short future as a latent trajectory z̃, on which the action is then conditioned. We identify this trusted imagination, rather than the reactive policy, as the exposed attack surface. A downstream oracle, such as a safety gate, a visual model-predictive-control (MPC) planner, or an imagine-then-check verifier, consumes z̃ as a prediction of the future. The robustness of the policy therefore does not entail the robustness of systems that rely on the WAM. The underlying phenomenon is an asymmetry. Corrupting the imagination is easy, since it requires only displacing z̃ from its natural-future manifold. Steering it precisely is hard, since it must reach a specified on-manifold target. We adopt a capability-based threat model with an ℓ∞ -bounded observation perturbation. The attacker applies projected gradient descent through the fully differentiable observation-to-imagination map. The same off-manifold property motivates a parameter-free denoiser detector. We evaluate three targets: RynnVLA-002, LingBot-VA, and LaDi-WM. Untargeted corruption is roughly 60× stronger than random and is detected at AUC 1.0. Targeted control remains bounded. An adaptive attacker evades detection only by forgoing corruption. The reactive policy remains robust to corrupted imagination. A native imagination-driven MPC, however, exhibits the first adversary-specific task failure (at ϵ=0.01, success 0.70 versus 0.05; Fisher p < 10−4 ).
1
Introduction
Classical vision-language-action (VLA) policies are reactive: an observation o is mapped directly to an action a = π(o). A second family adopts an imagine-then-act design built around a world-action model (WAM). The model first imagines a short future z̃t+1:t+K , a latent trajectory of upcoming visual representations. The action is then conditioned on this imagination [8, 9]. The imagination provides the policy with a form of internal rehearsal. Most adversarial work on robot policies attacks the action pathway and measures action error or task success. We instead ask who consumes the imagination. The imagined future z̃ may serve two very different consumers. The first is a reactive policy, which treats z̃ as an internal feature. Such a policy still executes in a real closed loop, where environment feedback continually corrects its errors. The second is a downstream oracle, which treats z̃ as a trusted prediction of the future and acts on it without any real-world corrective signal. Examples include a predictive safety gate, a visual MPC planner, and an imagine-then-check verifier. The gate imagines a rollout, predicts success or collision, and decides whether to abort. The planner imagines each candidate action 1
clean
closed-loop SR
Obs. o + δ, ∥δ∥∞ ≤ ϵ diff’able
WAM imagines future z̃ feature
trusted
Reactive policy
Oracle: MPC/gate
z̃=feature; real loop robust 0.98 → 0.97
trusts z̃; no correction broken 0.55 → 0.05
1
adversarial
0.980.97
0.55
0.5 0.05
0 Reactive policy
Oracle (MPC)
(a) two consumers of the same imagination (b) measured task success under attack
Figure 1: Motivation: policy-robust ̸= oracle-robust, with data. A single ℓ∞ -bounded perturbation steers the WAM’s imagined future z̃, but its effect depends on who consumes z̃ (a). A reactive policy treats z̃ as an internal feature and still executes in a real closed loop, where environment feedback corrects errors. A downstream oracle (visual-MPC, safety gate, or verifier) instead trusts z̃ as the future and has no corrective signal. Panel (b) shows the measured contrast. The reactive policy barely moves even at a large budget (LingBot-VA, oracle-attacked imagination ϵ0.8: 0.978 → 0.966; table 10). The oracle MPC collapses at a tiny budget (LaDi-WM sampling-MPC ϵ0.01: 0.55 → 0.05; table 8). Robustness of the policy is not robustness of systems that trust the WAM. and selects the best-imagined one. This second consumer is the attack surface of our work. When the WAM acts as planner, verifier, or simulator, its imagination is the sole information source. No real world is available to correct it, so corrupting the imagination directly corrupts the downstream decision (fig. 1). Contributions. • A new attack surface (sections 4 and 5). The first white-box integrity attack against the trusted imagination of imagine-then-act WAMs. The observation→imagination map is continuous and differentiable (no discrete vector quantization, no in-place cache writes), enabling true projected gradient descent (PGD) on the imagined latent. • An asymmetry mechanism (sections 5 and 6). Imagination is easy to corrupt (untargeted) but hard to precisely steer (targeted). The two regimes are consistent across targets and are pinned by decoded-frame evidence, not only latent cosine. • A defense that empirically holds against adaptive attackers (sections 5.6 and 5.7). Corruption is intrinsically off-manifold; a parameter-free denoiser self-consistency detector reaches AUC 1.0, and across our adaptive-attacker sweep an attacker cannot evade detection without surrendering corruption. • An honest task-level null, folded into motivation (section 7). A reactive policy is robust to corrupted imagination. We turn this into the thesis “policy robust ̸= oracle robust” and demonstrate, on a native imagination-driven MPC, the first adversary-specific real task failure.
2
2
Related Work
World models and imagine-then-act policies. Several robot policies couple a generative world model with an action head, so that actions are conditioned on an imagined future. RynnVLA-002 [1] unifies action and image generation in one autoregressive backbone. It decodes actions from their own token state, so imagination and action share only the backbone. LingBot-VA [9] interleaves video and action tokens in a single causal sequence. An action token attends the just-generated imagined-future latent through a block-causal mask, which makes imagination a genuine input to the action. LaDi-WM [8] predicts the latent evolution of DINO/SigLIP features with an iterative latent-diffusion world model. It then uses this prediction to refine a diffusion policy’s actions. These three targets span the imagination-to-action coupling from disconnected to native (table 2). Prior work studies such imagination as a way to help the policy. We instead treat it as an attack object. We also treat it as a signal that a downstream planner, verifier, or safety gate may trust without a corrective real-world signal. Adversarial attacks on robot policies. Bounded-perturbation attacks [7, 13] and zeroth-order variants [14] are standard for vision models. They have been applied to the observation-to-action path of robot policies, where harm is measured as action error or task-success drop. Our threat model differs in its target. We attack the imagined future consumed by an oracle, rather than the executed action of a reactive policy. The two can come apart, since an attack that strongly corrupts imagination leaves reactive task success almost unchanged (section 7). Attacks and robustness of world-action models. The closest contemporaneous work attacks world-action models in two ways that differ from ours. JailWAM [12] is a black-box, semantic jailbreak that searches adversarial visual trajectories to trigger unsafe behaviors. We are instead white-box and operate within an ℓ∞ pixel budget on a single observation, directly through the differentiable observation-to-imagination map. Robustness studies of world-action models [18] measure generalization under natural visual and language perturbations, not a worst-case adversary. To our knowledge, no prior work addresses the white-box integrity attack on the trusted imagination consumed by a downstream oracle. We pair this attack with a matching off-manifold detector and an adaptive-attacker analysis. Benchmarks. We evaluate on LIBERO [11] (long-horizon manipulation); related sim-based evaluation suites such as SimplerEnv [10] target real-to-sim transfer and are complementary but out of scope here.
3
Threat Model
We adopt a capability-based threat model (table 1). An imagine-then-act WAM is deployed, and a downstream oracle trusts its imagination z̃ for a decision. The attacker perturbs only the current camera observation, within an ℓ∞ budget. The attacker is white-box in the Kerckhoffs sense and knows the WAM and the encoder. It does not modify weights, does not touch the oracle, and has no access to the true future. Harm is measured at the decision level of the oracle, through verifier flip rate, MPC mis-selection, and divergence of the trusted imagination. Task-level closed-loop success is reported as the reactive-robustness control. Among the three oracle instantiations, only the visual-MPC consumer is instantiated end-to-end with closed-loop success (LaDi-WM, section 5.9).
3
Objective (definition 2): untargeted: push z̃ off-manifold, away from z̃clean targeted: steer z̃ toward attacker’s z̃tgt
o+δ
E
W
z̃adv
oracle V
observation
encoder
world model
imagined future
decision
no VQ, no in-place cache ⇒ end-to-end differentiable Detector (section 5.6): denoiser self-consistency s(z̃) ≷ τ (no clean ref.) PGD ∇δ L
Figure 2: Method overview. The attacker adds an ℓ∞ -bounded δ to the observation and backpropagates through the frozen encoder E and world model W to run true PGD on the imagined future z̃. These two modules form a continuous, fully differentiable observation→imagination map, with no discrete VQ and no in-place KV-cache writes. There are two objectives (definition 2): untargeted corruption pushes z̃ off the natural-future manifold (easy, about 60× random), and targeted steering aims z̃ at an attacker-chosen z̃tgt (bounded). A downstream oracle V consumes z̃, and a parameter-free denoiser self-consistency detector flags off-manifold corruption before the oracle acts. The imagine-then-check verifier (section 5.4) and the predictive safety gate (section 8) are evaluated as decision-level probes, whose scope we state explicitly. Table 1: Capability-based threat model. Setting Deployment Attacker capability Out of scope for attacker Objective Harm metric
4
Imagine-then-act WAM; imagination z̃ trusted by a downstream oracle (gate / MPC / verifier). ℓ∞ -bounded perturbation of the observation, ∥δ∥∞ ≤ ϵ; white-box. no weight modification; no access to the oracle internals; no access to the true future. untargeted: maximize divergence of z̃ from the model’s own clean imagination; or targeted: steer z̃ to an attacker-chosen scene. decision-level (verifier flip, MPC mis-selection, trusted-imagination divergence); plus task SR as control.
Method
A WAM comprises an encoder E and a world model W . The observation is encoded, and the world model generates the imagined future z̃ = W (E(o), c) under instruction c. A downstream oracle V then makes a decision from z̃. The attacker optimizes δ to change z̃ (fig. 2). Definition 1 (Adversarial perturbation of imagination). The attacker forms o′ = o + δ with ∥δ∥∞ ≤ ϵ, which induces the corrupted imagination z̃adv = W (E(o + δ), c). For continuous diffusionlatent WAMs the map o 7→ z̃ is end-to-end differentiable, with no discrete VQ and no in-place cache writes. True gradient PGD is therefore applicable. 4
Two objectives: corrupt vs. steer. Definition 2 (Untargeted corruption / targeted steering). Untargeted maximizes divergence from the clean imagination, Luntgt (δ) = − 1 − cos(z̃adv , z̃clean ) ,
corruption = 1 − cos(z̃adv , z̃clean ).
(1)
Targeted drives the imagination toward an attacker-chosen z̃tgt (the imagination for a swapped instruction or scene), Ltgt (δ) = ∥z̃adv − z̃tgt ∥2 ,
gap_closed = 1 −
∥z̃adv − z̃tgt ∥ . ∥z̃clean − z̃tgt ∥
(2)
gap_closed=1 means the imagination is perfectly steered to the target; =0 means unchanged; <0 means farther than clean. Attack and detector. Algorithm 1 gives the PGD attack and the off-manifold detector. One implementation detail matters. All _reset(c) calls must be wrapped in no_grad, and all parameters (transformer, text encoder, VAE) must be frozen. Otherwise an un-frozen text encoder caches a graph-bearing text context that is reused across forward passes, which triggers a “backward through the graph a second time” error. The gradient is sometimes unavailable, for example when actions read z̃ through an in-place KV-cache write that severs the gradient and yields ∥g∥=0. In that case we fall back to SPSA [14], which we verified converges. Algorithm 1 Imagination attack (true PGD) + off-manifold detector Require: WAM W , encoder E, observation o, instruction c, budget ϵ, steps T , step size α 1: Freeze all parameters; wrap every _reset(c) in no_grad 2: δ ← 0 3: for t = 1 . . . T do 4: z̃adv ← W (E(o + δ), c); g ← ∇δ L(δ) ▷ L = untargeted or targeted (definition 2) 5: δ ← Π∥·∥∞ ≤ϵ δ − α sign(g) 6: end for 7: return δ Detector (no clean reference): 8: s(z̃) ← mean future-frame velocity-prediction norm at small noise levels t ∈ {0.8, 0.9, 0.97} 9: if s(z̃) > τ then flag z̃ as corrupted
5
Experiments
All experiments are run on a single-A100 cluster (Adelaide Phoenix). The benchmark is LIBERO-10 (long horizon). Closed-loop rollouts use n=20 with Wilson 95% intervals and Fisher’s exact test.
5.1
Setup: three targets spanning the imagination-action coupling
We study three imagine-then-act targets chosen to span the strength of the imagination→action coupling (table 2).
5
Scale at a glance (read with the caption of each table). Latent-level attack and detection probes are reported over N tasks. These are targeted steering at N =1 to 8 (table 3), untargeted corruption and the negative control at N =5 (tables 4 and 5), detection at N =8, and the adaptive attacker at N =4 (table 7). The closed-loop results use full episodes with Wilson 95% intervals. The reactive null aggregates 143 to 190 episodes per condition across all 10 tasks (table 10). The LaDi-WM MPC failure is, at present, a single task (task 4) with N =20 per ϵ (tables 8 and 9). We flag this single-task scope as a limitation (section 8) rather than smoothing over it. The conclusions that do not depend on rollouts (untargeted corruption, detection, adaptive defense) are the most robust. Table 2: Three imagine-then-act targets. Target
Imagination
Imagination→action coupling
RynnVLA-002
discrete VQ tokens
LingBot-VA
continuous diffusion latent z̃
disconnected (action reads its own token hidden) same-model, weak at task level
LaDi-WM
latent diffusion, native refine
5.2
Role
negative control / zero transfer oracle attack/defense (O1–O8) strong (action-conditioned dy- real task failure + MPC namics)
RynnVLA-002: coupling exists but transfers nothing at task level
On RynnVLA-002 the action head reads its own token hidden state and does not consume the imagination at inference. Imagination is a separate generation path, and the two heads share only the backbone. We find a strong but undirected same-model gradient coupling (| cos(ga , gimg )|=0.31, 534× random). Same-model transduction is real but weaker than directly attacking the action, with about 20% catastrophic state reversals under an impact proxy. A cross-model rollout, however, shows zero transfer. Imagining with one checkpoint and transferring the perturbation to a victim yields task success = 100% = random = clean, whereas directly attacking the victim’s action yields 0%. An early oracle-deception probe already hints at the central asymmetry: an untargeted push corrupts (1.6× random), but targeted steering is unstable. This result motivates the move to continuous, differentiable targets and to the oracle threat model.
5.3
Targeted manipulation is bounded
On LingBot-VA true PGD is available, since the o → z̃ map is differentiable (∥g∥ ∼ 0.094). Targeted steering is adversarial-direction-specific, as random achieves ≈ 0. It is nonetheless bounded once the target is a genuinely different scene (table 3). When the target is the same observation with a swapped instruction (already 89% similar), gap_closed reaches 0.45 to 0.52 and a verifier is fooled 5/5. For a cross-scene target the latent cosine moves to gap_closed=0.207, but the decoded frames and the verifier do not. The verifier flips 0/8, and the decoded final frames remain the original scene. Directing imagination precisely to an arbitrary different scene is therefore bounded.
5.4
Untargeted corruption is easy and perceptually visible
This is the strongest result on the attack side. We run untargeted PGD that maximizes divergence from the true imagination, over N =5 tasks (table 4). A fidelity verifier flips to “imagination not trustworthy” when cos < τ . Adversarial corruption exceeds random by up to 60×. It flips the 6
Table 3: Targeted steering (LingBot-VA): adversarial-specific but bounded for cross-scene targets. Exp
Setting
adv. gap_closed
random
verifier / decode
O1 O2 O3
same-obs swap-instr., N =1 ϵ 0.1–0.4 sweep, single obs cross-scene, N =8, ϵ0.2
0.447 0.46–0.52 0.207 ± 0.056
−0.079 all neg. −0.002 ± 0.008
cos(adv, tgt) 0.891→0.966 verifier fooled 5/5 vs. random 1/5 verifier 0/8; decode ≈ clean
̸=
≈ (a) clean imagination
(b) targeted adversarial
(c) attacker’s target
Figure 3: Targeted steering is bounded (LingBot-VA, decoded imagined frame f =12, task 0 → 3; each panel shows the two decoded camera views, left: third-person agentview, right: wrist). Despite a latent-cosine gap_closed=0.207, the adversarial imagination (b) still depicts the original scene (the basket-and-cans tabletop of (a), with only mild artifacts; b ≈ a) and does not become the attacker’s target scene (c, a different kitchen layout; b ̸= c). Latent cosine moves; pixels and the verifier do not. verifier at 100% from ϵ=0.1, with 0% false positives. Decoding confirms perceptually visible damage, including object smearing, ghosting, and artifacts. This contrasts with the perceptually invisible targeted case (section 5.3). Table 4: Untargeted corruption × fidelity verifier (LingBot-VA), N =5 tasks. ϵ
adv. corruption 1− cos
random
ratio
verifier flip (adv / rand)
0.05 0.1 0.2 0.8
0.135 ± 0.05 0.197 ± 0.056 0.266 ± 0.03 0.467 ± 0.09
0.002 0.006 0.013 0.109
∼ 60× ∼ 30× ∼ 20× ∼ 4×
n/a @τ 0.9: 100%/0% @τ 0.8: 100%/0% 100%/n/a
Which verifier, honestly? The fidelity verifier above thresholds cos(z̃adv , z̃clean ), the very quantity the untargeted attack minimizes. Its 100% flip is therefore partly by construction, and it should be read as a trust/fidelity gate rather than an independent oracle. We also evaluate an independent decision-level verifier (V1), a nearest-prototype task classifier over clean imagined futures, with clean accuracy 1.000. Under untargeted corruption it misclassifies the imagined task in only 12% of cases, against 0% for random (targeted false-accept-as-target 0%, N =8, ϵ0.2). The most faithful decision-level harm is therefore the availability attack of section 8. A real denoiser safety gate, when attacked, drops effective task success by 0.18 to 0.39. We report all three measures rather than only the largest.
5.5
Negative control: corruption is imagination-channel-specific
Is the corruption a directed exploit of the imagination channel, or merely generic visual damage that any consumer would see? For the same bounded observation perturbation, we compare the divergence on two quantities. The first is the imagined future frames z̃[1:] . The second is the 7
⇒ (a) clean imagination
(b) untargeted adversarial (ϵ0.2)
Figure 4: Untargeted corruption is perceptually visible (LingBot-VA, decoded imagined frame f =12, task 0; each panel shows the two decoded camera views, left: third-person agentview, right: wrist). Compared to the clean imagined future (a), the corrupted imagination (b) exhibits object smearing, ghosting, and a torn region on the right. In direct contrast to the targeted case (fig. 3), the damage here is plainly visible to any consumer of the imagined frames. current-observation read-out latent (frame 0), which is what the no-imagination reactive consumer sees. The amplification amp = divfuture /divobs is 24.5× at ϵ0.1 and stays above 1 throughout (table 5). Moreover divfuture (adv) ≫ divfuture (rnd). The corruption is thus a directed exploit of the imagination channel and is largely invisible to an observation-faithful consumer. That channel is exactly the one a downstream oracle uniquely trusts. Table 5: Negative control (LingBot-VA, N =5): corruption amplifies on the imagination channel relative to the obs read-out.
5.6
ϵ
divfuture (adv)
divobs (adv)
amp
divfuture (rnd)
0.1 0.2 0.4 0.8
0.198 ± 0.045 0.282 ± 0.029 0.378 ± 0.042 0.545 ± 0.043
0.0083 0.0325 0.1044 0.2656
24.5× 8.7× 3.6× 2.1×
0.004 0.012 0.040 0.110
Detection: denoiser self-consistency reaches AUC 1.0
Corrupted imagination departs the model’s own learned manifold, and is thus off-manifold. Our detector is parameter-free: it is the mean future-frame velocity-prediction norm at small noise levels, and it needs no clean reference. This detector separates clean from adversarial almost perfectly (table 6). Surface statistics such as latent RMS and max-abs are at chance level, since the attack stays within the ℓ∞ budget and does not inflate norms. Table 6: Detection AUC (clean vs. adversarial), LingBot-VA. Detector
ϵ0.1
ϵ0.2
ϵ0.4
TPR@5%FPR
surface RMS / max-abs simple clip-corruption denoiser self-consistency
∼ 0.5 0.5–0.62 0.89
∼ 0.5 n/a 1.00
∼ 0.5 0.86 1.00
chance weak at low ϵ 62%/100%/100%
8
5.7
Adaptive attacker: the defense holds
A reviewer will ask whether a knowledgeable attacker can evade the detector. We let the attacker minimize L = −corruption + λ · det_score (table 7). As λ grows, AUC falls below 0.5 and the det-score approaches the clean baseline 0.369. At the same time the corruption falls with it, from 0.279 to 0.064. Being off-manifold is part of what it means to corrupt the imagination. To evade detection the attacker must stay on the manifold, which means not corrupting. The two goals cannot both be met, so the defense holds against adaptive attackers. Table 7: Adaptive attacker vs. the denoiser detector (LingBot-VA). Evasion requires surrendering corruption.
5.8
λ
corruption 1− cos
det_score
AUC
0.0 0.5 2.0 8.0
0.279 0.241 0.156 0.064
0.562 0.407 0.345 0.337
1.00 0.88 0.31 0.06
Universal transfer
A single scene-agnostic δ (|δ|=21.25) is optimized on train tasks [0, 3] and then frozen. It still corrupts the held-out tasks [4, 7], with test corruption 0.199 against random 0.039 (∼ 5×; train 0.356 against 0.034). This indicates a deployable, scene-agnostic threat.
5.9
LaDi-WM sampling MPC: adversary-specific real task failure
Unlike the reactive null (section 7), this setting produces genuine task failure. LaDi-WM’s policy natively refines actions with imagination. We build a sampling MPC on top of it: every 5 steps it samples K=4 candidate actions, imagines and scores each, and takes the arg max. The attacker corrupts imagination via ℓ∞ -PGD on the observation. We evaluate on task 4 with N =20 (fig. 5 and table 8). Table 8: LaDi-WM MPC: SR-vs-ϵ (task 4, N =20). Clean = 0.55. ϵ
random SR
adv. SR
reading
0.01 0.03 0.06
14/20 = 0.70 13/20 = 0.65 8/20 = 0.40
1/20 = 0.05 1/20 = 0.05 0/20 = 0.0
random ≈ clean, adversarial collapses = adversary-specific same direction, adversary-specific window saturation: random also collapses
At ϵ=0.01 the random Wilson interval [0.48, 0.85] and the adversarial interval [0.01, 0.24] do not overlap (Fisher p < 10−4 ). Random noise of the same magnitude is harmless (0.70 ≈ clean), whereas the adversarial direction collapses success to 0.05. We also note an honest self-correction. An earlier single-point read at ϵ=0.06, where random and adversarial both collapse, was mistaken for “mere visual fragility”. The full sweep shows that this point is the saturation regime. The adversary-specific window genuinely exists at ϵ ∈ [0.01, 0.03]. There, both ϵ=0.01 and ϵ=0.03 show adversarial SR = 1/20 = 0.05 against random ≥ 0.65 at full N =20. Causal isolation. A five-condition probe at ϵ=0.06 and N =20 localizes the harm (table 9). In the score_only condition we feed the adversarial observation only to the MPC scoring step and execute 9
closed-loop success rate
0.8 0.7
0.7 0.55
adversary-specific window
clean baseline (0.55) random noise adversarial
0.65 0.55 0.55
0.4
0.4
0.70 vs 0.05 Fisher p < 10−4
0.2 0
0
5 · 10−2
0.01
5 · 10−2
0.03 ℓ∞ perturbation budget ϵ
0.06 0
Figure 5: LaDi-WM MPC closed-loop SR vs. perturbation budget (task 4, N =20 per point, Wilson 95%). In the shaded adversary-specific window (ϵ ∈ [0.01, 0.03]), random noise of the same magnitude is harmless (SR ≈ 0.65 to 0.70 ≈ clean) while the adversarial direction collapses success to 0.05 (non-overlapping intervals, Fisher p < 10−4 ). At the large budget ϵ0.06 both collapse (saturation): magnitude alone, not adversariality, dominates there. on a clean observation. Success then recovers to 0.60, so the explicit outer planning and ranking layer is innocent. The final_only result of 0 does not, however, license a “pure direct-vision” claim. The execution step’s direct-vision path and the in-act imagination-refinement path share the same corrupted image and the same DINO-SigLIP encoder. An observation-space attacker cannot separate these two paths. This is why the oracle threat model is needed. Only when the attacker manipulates the predicted latent directly, in the trusted downstream z-space, does “imagination” become a surface that can be attacked and defended on its own. Table 9: Causal isolation on LaDi-WM MPC (ϵ0.06, N =20).
6
Condition
SR
meaning
clean random (matched magnitude) adversarial (both paths) score_only (adv. scoring, clean execution) final_only (clean scoring, adv. execution)
11/20 = 0.55 8/20 = 0.40 0/20 = 0.0 12/20 = 0.60 0/20 = 0.0
baseline random control full attack outer planner innocent (recovers) execution-step vision is the carrier
Mechanism: off-manifold is intrinsic to corrupting imagination
A single mechanism accounts for the positive results. Imagination z̃ lives on a low-dimensional manifold M of natural futures learned by the model. Untargeted corruption amounts to pushing z̃ off M. The target set is huge, since any “wrong” direction will do, so corruption is easy (60× random) and necessarily off-manifold. The denoiser prior therefore flags it (AUC 1.0). Targeted steering must instead land z̃ at another specific point of M. It must both leave the original point and re-land on a particular on-manifold location, which is a strong constraint, so it is bounded (gap 0.2, decode unchanged, verifier 0/8). Evading detection means staying on M, which means not 10
corrupting, so AUC and corruption collapse together (section 5.7). The same mechanism explains why corruption is easy, why steering is hard, and why corruption is detectable.
7
The task-level null, and why it motivates the oracle threat
We state a negative result that constrains the rest of the paper. A moved imagination cosine, or a flip of our own verifier, is not the same as real task failure. In a reactive closed loop, corrupted imagination has essentially no task-level effect (table 10). Clean success is 0.978, oracle-attacked imagination at ϵ0.8 is still 0.966, and a direct observation attack at ϵ1.5 leaves 0.943. At mid-budget a universal adversarial δ is in fact more harmless than random noise: the random noise drives SR to 0, whereas the adversarial δ preserves 0.83 to 1.0. Table 10: Reactive task-level success: the honest null (aggregated rollouts). Condition
real SR
reading
clean oracle-attacked imagination ϵ0.8 direct observation attack ϵ1.5 universal adversarial ϵ0.8–1.2
0.978 0.966 0.943 0.83–1.0 (vs. random→ 0)
baseline barely drops also unmoved adversarial more harmless than random
This null is the motivation rather than a failure: policy robustness ̸= robustness of a downstream verifier or planner that trusts the WAM. The same “policy is robust” data is a negative result for the reactive consumer and an attack surface for the oracle consumer. LaDi-WM’s MPC (section 5.9) is exactly an oracle consumer that natively plans with imagination, and the adversary-specific real failure appears there.
8
Limitations
Several limitations remain. First, under observation-space attacks the direct-vision and imaginationrefinement paths share an encoder and cannot be separated. Imagination-specific attribution is therefore only possible under the oracle threat model, with direct z-space manipulation (section 5.9). Second, targeted steering is bounded: precisely directing imagination to an arbitrary scene is not achievable (gap 0.2, decode unchanged, verifier 0/8). The core threat is corruption rather than arbitrary control. Third, the availability claim is downgraded. An early N =3 pilot suggested a catastrophic effective-SR collapse and gate AUC 1.00. At full scale (N =179) it is instead a mild but statistically significant availability loss, with an effective-SR drop of −0.18 to −0.39 across FPR and gate AUC 0.771, which is the number we report. The hardest contributions (untargeted corruption, detection, adaptive-attacker defense) do not depend on rollouts. Fourth, the evaluation is simulation only and limited in closed-loop scale. The MPC rollouts are currently task 4 with N =20 across an ϵ sweep, and a multi-task study with 20 to 50 episodes is needed before any universality claim. A semantic CLIP verifier is unreliable at the decoded resolution, so only the denoiser fidelity verifier is used as primary evidence.
9
Conclusion
WAM imagination is easy to corrupt and hard to steer precisely. Untargeted corruption flips a verifier at ϵ0.1, is perceptually visible, and is detectable at AUC 1.0, whereas targeted steering 11
stays bounded. This asymmetry is dangerous for any oracle that trusts a WAM’s predicted future, whether a planner, a verifier, or a simulator. The reactive task-level null, in which the policy stays robust, becomes the central motivation. A native imagination-driven MPC then shows the first adversary-specific real task failure. Because corruption is off-manifold, the resulting detector holds even against adaptive attackers. Reproducibility. All numbers are real, measured on a single-A100 cluster. Code, attack/defense scripts, decoded frames, and per-condition rollout JSONs are available from the authors.
References [1] Jun Cen, Siteng Huang, Yuqian Yuan, Kehan Li, Hangjie Yuan, Chaohui Yu, Bohan Hou, Yuming Jiang, Jiayan Guo, Xin Li, Hao Luo, Fan Wang, Deli Zhao, and Hao Chen. RynnVLA-002: A unified vision-language-action and world model. arXiv preprint arXiv:2511.17502, 2025. [2] Qiuli Dai, Honglong Chen, Zhichen Ni, Yubin Yang, Linghan Chen, Xinglong Fan, and Xiangyu Li. Reinforcement learning-based resource reservation for mobile edge computing with probable failure. IEEE Transactions on Vehicular Technology, 74(5):7985–7996, 2025. [3] Yudong Gao, Honglong Chen, Peng Sun, Junjian Li, Anqing Zhang, Zhibo Wang, and Weifeng Liu. A dual stealthy backdoor: From both spatial and frequency perspectives. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3):1851–1859, 2024. [4] Yudong Gao, Honglong Chen, Peng Sun, Zhe Li, Junjian Li, and Huajie Shao. Energy-based backdoor defense without task-specific samples and model retraining. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pages 14611–14637, 2024. [5] Yudong Gao, Honglong Chen, Peng Sun, Junjian Li, Yangxu Yin, Zhibo Wang, and Weifeng Liu. A triple stealthy backdoor: Hidden in spatial, frequency, and feature domains. IEEE Transactions on Dependable and Secure Computing, 22(6):7746–7758, 2025. [6] Yudong Gao, Zongjie Li, Yuanyuan Yuan, Zimo Ji, Pingchuan Ma, and Shuai Wang. SkillReducer: Optimizing LLM agent skills for token efficiency. arXiv preprint arXiv:2603.29919, 2026. [7] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015. [8] Yuhang Huang, Jiazhao Zhang, Shilong Zou, Xinwang Liu, Ruizhen Hu, and Kai Xu. LaDi-WM: A latent diffusion-based world model for predictive manipulation. In Conference on Robot Learning (CoRL), 2025. arXiv:2505.11528; PMLR vol. 305. [9] Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control. arXiv preprint arXiv:2601.21998, 2026. [10] Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation. In Conference on Robot Learning (CoRL), 2024. arXiv:2405.05941. [11] Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks, 2023.
12
[12] Hanqing Liu, Songping Wang, Jiahuan Long, Jiacheng Hou, Jialiang Sun, Chao Li, Yang Yang, Wei Peng, Xu Liu, Tingsong Jiang, Wen Yao, and Yao Mu. JailWAM: Jailbreaking world action models in robot control. arXiv preprint arXiv:2604.05498, 2026. [13] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018. [14] James C. Spall. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, 37(3):332–341, 1992. [15] Wenhan Wu, Yili Gong, Jiawei Jiang, Chuang Hu, Xiaobo Zhou, and Dazhao Cheng. Defending against attribute inference attacks in post-training of recommendation systems via unlearning. In 2025 IEEE 41st International Conference on Data Engineering (ICDE), pages 2656–2669, 2025. [16] Wenhan Wu, Huanghuang Liang, Tianyu Tu, Jiawei Jiang, Chuang Hu, and Dazhao Cheng. Mimir: Data-free federated unlearning through client-specific prompt generation for personalized models. IEEE Transactions on Mobile Computing, 24(10):10537–10556, 2025. [17] Jimiao Yu, Honglong Chen, Junjian Li, Linghan Chen, Yudong Gao, Weifeng Liu, and Lei Zhang. Black-box adversarial defense based on image decomposition and reconstruction. IEEE Transactions on Multimedia, 27:5909–5921, 2025. doi: 10.1109/TMM.2025.3565987. [18] Zhanguang Zhang, Zhiyuan Li, Behnam Rahmati, Rui Heng Yang, Yintao Ma, Amir Rasouli, Sajjad Pakdamansavoji, Yangzheng Wu, Lingfeng Zhang, Tongtong Cao, Feng Wen, Xinyu Wang, Xingyue Quan, and Yingxue Zhang. Do world action models generalize better than VLAs? a robustness study. arXiv preprint arXiv:2603.22078, 2026.
13