arXiv CSopen access

WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving

WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving Xuerun Yan1,2† , Zhexi Lian1† , Nuoheng Zhang1 , Shiyu Fang1 , Haoran Wang1 , Chen Lv2 , Jia Hu1⊠ , and Binyang Song2⊠ 2

Tongji University, China Nanyang Technological University, Singapore

†

Equal contribution

arXiv:2607.08375v1 [cs.CV] 9 Jul 2026

1

⊠

Corresponding author

Abstract. Vision-Language-Action (VLA) models have advanced endto-end autonomous driving. However, existing methods either lack comprehensive world cognition or suffer from fragmented world foresight, inherently confining these models to reactive driving. To address this limitation, we propose WCog-VLA, a novel dual-level World-Cognitive VLA framework that successfully bridges semantic world forecasting with generative world evolution to achieve proactive autonomous driving. At the semantic level, WCog-VLA unifies world cognition and reasoning by incorporating 3D spatial perception and injecting agent tokens to capture the world dynamics, while concurrently enabling Game-theoretic Chain-of-Thought (Game-CoT) reasoning. At the generative level, we introduce the Aligned Decoupled Diffusion Transformer (ADDT) as a powerful generative world model that synthesizes physically-plausible joint multi-agent trajectories. Through scene representation alignment, ADDT reduces the number of denoising steps required and thus significantly accelerates inference. To facilitate strategic reasoning, we further construct a large-scale dataset featuring 85k Game-CoT annotations. Extensive experiments on the NAVSIM benchmark demonstrate that WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9. Keywords: End-to-end autonomous driving · Vision-Language-Action · World cognition

1

Introduction

End-to-end (E2E) autonomous driving has emerged as a dominant paradigm by directly mapping raw sensory inputs to planned trajectories [5, 19, 20, 26] within a unified and differentiable framework. Although these E2E models show remarkable performance in common scenarios, they often struggle in complex or long-tail situations [4, 63]. This fragility arises from insufficient causal reasoning and world knowledge, leaving the models unable to fully understand and reason about the surrounding environments.

2

X. Yan et al. )*)+,-,.ÿ0+1).

2334ÿ56789:;<=

012ÿ456ÿ17ÿ898ÿ6  1 ÿ  17  47511  6   1

02ÿ45ÿÿ(1"  ÿ  ÿ   1 ÿ  17  47 511 51 ÿ  6 ÿ4  ÿ   8 %1& '

0$2ÿ45ÿ17ÿ898ÿ6  1 ÿ 47 511 6

02ÿ45ÿÿ15  ÿ  ÿ ÿ0 72ÿ  17  1 ÿ ÿ ! "15  ÿ 47 51 ÿ  511 6 #  1 5 

 17   8 ÿ4     %1& '

  

6  %1& '

Fig. 1: Four paradigms of leveraging VLM in E2E autonomous driving. Our method (d) advances existing frameworks to enable proactive driving by establishing a dual-level world cognition with the integration of semantic forecasting and generative evolution. >To 5?@1,,,A1,8@;address @B8CDEF98G,4HI,JBJ@8,>5these =K8@,2334ÿ56789:;long-tail <= 0,0 2] challenges, Vision-Language-Models (VLMs) [1,

have been increasingly integrated into E2E autonomous driving frameworks [25, 47, 53]. Equipped with extensive world knowledge and strong reasoning capabilities, VLMs significantly advance scene comprehension in complex driving scenarios [27,60]. Building upon this foundation, Vision-Language-Action (VLA) models further extend VLM capabilities to action generation. Existing VLA approaches typically operate in two ways. The first formulates action outputs as autoregressive sequence generation (Fig. 1(a)), producing either discrete text tokens [37, 39, 42] or learned action codes [64]. Alternatively, the second utilizes VLMs as cognitive encoders and attaches dedicated action decoders (Fig. 1(b)) to generate continuous trajectories [13, 52], e.g., diffusion models [23, 32]. Despite these advancements, existing VLA models still face several challenges: (1) Lack of 3D spatial awareness. Relying primarily on 2D image features, current models lack structured 3D spatial representations of surrounding road participants [38, 63], which are essential for accurate spatial reasoning and precise ego planning. (2) Insufficient world cognition. Existing methods struggle to adequately represent world states and forecast future dynamics [46], such as the intentions of surrounding agents. This confines current methods to reactive rather than proactive driving (Fig. 1(a),(b)). While some works incorporate world cognition via VLM hidden states [28, 54], they treat world modeling as an auxiliary semantic task while neglecting generative-level world evolution. This fragmented world foresight (Fig. 1(c)) overlooks the reciprocal interplay between the ego and surrounding agents, failing to synthesize joint interactive trajectories from a world-generative perspective. (3) Absence of strategic social reasoning. Current reasoning mechanisms mainly focus on static scene descriptions [7, 39], lacking the game-theoretic ‘if-what’ imagination required for proactive social interaction among traffic participants. Motivated by these limitations, we seek to answer a key question: How can we endow VLA models with com-

WCog-VLA

3

prehensive world cognition across both semantic-level forecasting and generative-level evolution, thereby enabling proactive driving? To address these challenges, we propose WCog-VLA, a novel VLA framework featuring dual-level World Cognition (Fig. 1(d)). WCog-VLA achieves comprehensive world cognition by bridging the gap between semantic-level forecasting and generative-level physical evolution. At the semantic level, WCogVLA first embeds 3D spatial priors into the VLM, establishing a cognitive foundation for structured environment understanding. To operationalize the semantic world understanding, we decouple the VLM’s output hidden states into two functional roles. The cognition role encapsulates the understanding of current world states and future world dynamics. Meanwhile, the reasoning role drives a four-step game-theoretic reasoning process. This strategic reasoning mechanism transforms the ego vehicle from a passive observer to a proactive negotiator in social driving scenarios. At the generative level, we introduce the Aligned Decoupled Diffusion Transformer (ADDT) as our generative world model. Conditioned on the VLM’s hidden states, ADDT employs a decoupled encoder-decoder architecture to generate multi-agent trajectories. Specifically, the condition encoder explicitly aligns its latent space with dynamic scene representation. Guided by this alignment, the generation decoder efficiently synthesizes joint multi-agent trajectories, firmly grounding ego-planning within predicted interactive future dynamics. In summary, our main contributions are as follows: 1. Dual-level world cognition framework. We propose WCog-VLA that bridges semantic-level forecasting with generative-level evolution, enabling proactive autonomous driving. 2. Generative world model. We introduce ADDT as a generative world model, which synthesizes physically-plausible joint multi-agent trajectories with efficient inference time. 3. Game-theoretic reasoning dataset. We construct Game-CoT, a Gametheoretic Chain-of-Thought reasoning dataset with 85k samples that fills the gap in game-theoretic reasoning supervision for social driving. 4. State-of-the-art (SOTA) performance. WCog-VLA achieves a SOTA PDMS score of 92.9 on the NAVSIM [10] benchmark.

2

Related Work

End-to-End Autonomous Driving. E2E autonomous driving directly maps sensory inputs to trajectories within unified frameworks [8, 19, 33]. Pioneering works like UniAD [20] and VAD [26] integrate perception and planning using dense Bird’s-Eye-View (BEV) representations, while SparseAD [45] improves efficiency via sparse queries. Recent paradigms, e.g., GenAD [61], DiffusionDrive [35], and VADv2 [5], introduce generative and probabilistic models for multimodal trajectory planning [62]. However, current E2E models are limited by training data coverage, frequently failing in long-tail scenarios. This fragility primarily stems from an inherent lack of semantic reasoning and environment understanding capabilities, motivating the integration of VLMs into E2E driving.

4

X. Yan et al.

VLA for Autonomous Driving. VLMs are initially applied as high-level semantic interpreters for scenario understanding, such as DriveGPT4 [55] and DriveLM [47]. Building upon this, unified VLA models have been proposed to directly map multi-modal inputs to driving actions [25], exemplified by EMMA [22, 53], SimLingo [44], and DriveMoE [56]. While early VLA models driving actions directly as text [37, 39], later work such as AutoVLA [64] introduces autoregressive generation of action tokens. More recently, studies integrate VLM with generative planners [29, 49] to mitigate modal collapse between text and actions, e.g., VAE-based ORION [13] and diffusion-based ReCogDrive [32]. However, current VLA methods predominantly remain reactive observers. Lacking explicit world cognition and future forecasts, they fail to anticipate dynamic changes in complex social scenarios, motivating the integration of comprehensive world cognition into the VLA framework. World Cognition Building for VLA-based Autonomous Driving. The latest VLA studies [24,36,51,54] have sought to incorporate world cognition. One branch of research focuses on enhancing spatial awareness of VLMs [9, 16], e.g., DrivePI [38] introduces spatial-aware world cognition, and SGDrive [28] builds world features around a scene-agent-goal hierarchy. Another branch leverages generative forecasting for future foresight, employing future image generation as an auxiliary objective, e.g., UniDrive-WM [54] and DriveVLA-W0 [30]. However, these methods exhibit fragmented world foresight. Whether relying on semantic or image forecasting, they treat world evolution as a supervised perceptual task and lack joint multi-agent planning, failing to model interactive physical behaviors from a generative world perspective. To bridge this gap, WCog-VLA introduces a unified, dual-level world cognition framework. At the semantic level, the explicit agent tokens encapsulate the cognition of world dynamics. At the generative level, our ADDT synthesizes joint multi-agent trajectories, manifesting explicit world cognition within the generative execution stage.

3

WCog-VLA

As shown in Fig. 2, our WCog-VLA consists of two tightly coupled components that bridge semantic-world understanding and reasoning with generative trajectory synthesis. First, a VLM Backbone is constructed on a multi-modal architecture to jointly process multi-view camera inputs and textual instructions. These heterogeneous tokens are fused to facilitate a game-theoretic Game-CoT reasoning process, enabling structured reasoning over scene context and agent interactions. To explicitly model spatial structure and agent dynamics, the VLM backbone further incorporates agent tokens from a 3D perception module. A specialized world head then decodes the aggregated agent tokens to current 3D perception and future trajectory predictions of surrounding agents, enabling explicit semantic-level world cognition. Second, to seamlessly bridge semantic intent with physically feasible motion, we employ the ADDT as a generative world model. By intrinsically aligning its latent space with implicit scene dynamics through a

WCog-VLA 45467879ÿ45;58

728ÿ 81386 -9 6 .

2 28ÿ 

ÿ9ÿ 167

<==>?@ABCDEFGH

96  3681 96 168361

3ÿ 8386 9ÿ

136

 8ÿ8368ÿ1( 6 ÿ012 (61781 ÿ8  "6 8ÿ36

96 2 28 68

38 8 8 316

132ÿ2)83  2$

278 2$61( 082 6  &226 118 ÿ 81386

/61 #82

9ÿ0648 8 8126 ÿ83681 0 ! !!  0

6 6 ÿ 3681 6ÿ# 61$ 0 ! !!  0 !!!  0 6ÿ28 ÿ 8   0ÿ38 8 38188ÿ123688 "6 8ÿ36 1 IJKD;777L;7ADFDBAMN?OPAQ7>RS7TBTDA7IJHUAD7<==>?@ABCDEFGH

2780'81683ÿ082 6

9  3681 8  168361 01234561781 -8 .

%6ÿ218ÿ2 ÿ26 676 1& ÿ$ 87!ÿ%61 2 4ÿ ÿ6ÿ183 08+ &8'38ÿ(18ÿ128361$ 0648 ,81 )2 8ÿ6 ÿ7&8* 327812ÿ728 !!! 1318ÿ$61ÿ182 6  6 -08+. 28 8)$8!!!

5

518ÿ 1836

8 ÿ0128361$

6 116  V7V

Fig. 2: Overview of WCog-VLA. Our framework achieves dual-level world cognition by tightly coupling a multi-modal VLM backbone with a generative world model ADDT. The VLM integrates vision, text, and agent tokens to perform Game-CoT reasoning and semantic world forecasting, while the ADDT translates these cognitive representations to generate physically-plausible, joint multi-agent trajectories.

structured conditioning mechanism, ADDT generates high-fidelity, joint multiagent trajectories. Collectively, the proposed framework consistently translates explicit semantic world cognition into coherent and dynamically plausible trajectory generation. Detailed model architectures are provided in this section. 3.1

VLM Backbone

Model Inputs and Base VLM Model. The VLM takes multi-view camera images, instructions, and ego-vehicle states as inputs. The visual input I = {I i }6i=1 comprises six surround-view images. The instruction lins provides navigation commands (e.g., ‘turn right’). The ego state S = {v, a, Thist } encapsulates the current velocity v, acceleration a, and a 2-second historical trajectory Thist sampled at 2 Hz. To process these multi-modal inputs, we adopt InternVL3-2B [65] as our VLM backbone, utilizing a 300M-parameter InternViT vision encoder and a Qwen2.5 Large Language Model (LLM). 3D Spatial Perception. We extend 2D VLM perception into the 3D domain to enable explicit 3D spatial perception. Specifically, the multi-view camera features extracted by the vision encoder are lifted into a BEV representation FBEV via an off-the-shelf BEV encoder from BEVFormer [34]. To extract structured object representations, we employ a TrackFormer [20] that maps dense BEV features

6

X. Yan et al.

:;:<=>=:ÿ:@A>B

1 *# ÿ! *+ 8+#+$* 6&+%#$#   ÿ

)" 7 #

2 +$# ÿ*+

'9 3-%*  '%  ÿ )#* + )# 1 *# ÿ#& ,-#.$" # 1 *# ÿ 3$#-+ +$/#+0 3$#-+ 45,

'(

76 

76 32

45,

76

012345678 9226 23

76 32

3-%* '% )# !"#& #$#%

! *+

CDDEFGHI

Fig. 3: Illustration of the ADDT. The ADDT features a decoupled architecture comprising a condition encoder with representation alignment and a generation decoder.

to sparse agent-centric tokens. By performing cross-attention between learnable agent queries Qagent and FBEV , the module extracts a set of Na agent tokens a Tagent = {tjagent }N j=1 , where Na denotes the number of detected agents. These explicit agent tokens capture spatial locations and geometric features, providing structured inputs for subsequent multi-modal reasoning. Unified World Cognition and Reasoning. To achieve comprehensive scene understanding, the vision (Tvision ), text (Ttext ), and agent tokens (Tagent ) are concatenated along the sequence dimension and fed into the LLM to model JKLmulti-modal MA===DA=NMGMONPQRSTNU=VWC=XOXMN=JKIYinteraction NM=CDDEFGHI and fusion. The resulting hidden states are defined as:@=@ O_{\text {vision}}, O_{\text {text}}, O_{\text {agent}} = \text {LLM}([\mathcal {T}_{\text {vision}}, \mathcal {T}_{\text {text}}, \mathcal {T}_{\text {agent}}]) \label {eq:llm_output}

(1)

We decouple these output tokens of hidden states to support two distinct downstream tasks. Specifically, Oagent encapsulates semantic-level world cognition, which is routed to a specialized world head for current 3D perception and future trajectory prediction of surrounding agents. Meanwhile, Ovision and Otext are processed by the language modeling head to generate textual responses. Trained via our Game-CoT reasoning paradigm, the model can output explicit game-theoretic reasoning processes in its textual responses. 3.2

Aligned Decoupled Diffusion Transformer

While diffusion transformers exhibit remarkable generation quality, they suffer from an optimization dilemma in single-network architectures [41]: the encoding of low-frequency abstract semantics conflicts with the decoding of high-frequency continuous details [48]. In autonomous driving, this manifests as a tension between modeling complex multi-agent interactions and generating precise trajectories. To resolve this challenge and bridge VLM semantic cognition with physical actions, we propose ADDT (Fig. 3), which features a decoupled architecture comprising a specialized condition encoder and a dedicated generation decoder. Condition Encoder. The condition encoder focuses on extracting structural and interactive semantics, decoupled from the burden of precise trajectory recovery. Let xt ∈ RNm ×H×3 denote the joint multi-agent action noises at diffusion timestep t, where Nm is the maximum number of agents and H is the planning

WCog-VLA

7

horizon. To inject temporal and cognitive context, we construct a fused noises representation Fat by concatenating embedded noise actions, historical ego actions τhis , and average-pooled VLM output tokens F̄VLM . As shown in Fig. 3, Fat serves as the primary input for N1 Diffusion Transformer (DiT) blocks. Within these blocks, the diffusion timestep t and ego states S are injected via AdaLN modulation to provide physical kinematics guidance. Concurrently, the full-sequence VLM output tokens FVLM = [Ovision , Otext , Oagent ] are injected into cross-attention layers, providing high-level semantic cognition priors. Finally, the encoder outputs a semantic self-condition feature zt , formulated as: z_t = \text {Encoder}(F_{at}, t, S, F_{\text {VLM}}),\text { }F_{at} = \text {concat}(E_{act}(x_t), E_{his}(\tau _{\text {his}}), \bar {F}_{\text {VLM}}) \label {eq:encoder}

(2)

The resulting feature zt captures semantic scene dynamics, which guides the subsequent trajectory generation process. Representation Alignment. To ensure zt adheres strictly to real-world dynamics, we introduce a representation alignment mechanism. Specifically, the intermediate feature hi from the i-th DiT block of the condition encoder is aligned with a latent scene representation r∗ extracted from a pre-trained VAE encoder, thereby reducing the ‘semantic’ gap between condition encoder output and latent scene space. Following GenAD [61], this VAE is pre-trained to reconstruct multi-agent trajectories via an MLP encoder and GRU decoder. The resulting latent space after VAE encoder captures both global traffic patterns and individual characteristics of each agent. We enforce this alignment using a cosine similarity constraint [57] with a learnable projection MLP hϕ : \mathcal {L}_{\text {align}} = 1 - \cos (r_*, h_\phi (h_i)) \label {eq:align}

(3)

Crucially, this explicit alignment acts as a regularization technique, maintaining the local consistency of zt across adjacent denoising timesteps. It ensures that the generated trajectories are grounded in feasible scene dynamics while stabilizes semantic features throughout the progressive denoising steps. Generation Decoder. The generation decoder, comprising N2 DiT blocks, shares the condition encoder’s architecture but focuses exclusively on recovering high-frequency geometric details. Guided by the self-condition feature in zt , it processes the fused action noises Fat and VLM output tokens FVLM to estimate the denoised multi-agent trajectories. Unlike the condition encoder, the generation decoder injects both the timestep t and the self-condition feature zt via AdaLN modulation, enabling semantically aligned denoising. The decoding process is formulated as: x_{t-1} = \text {Decoder}(F_{at}, t, z_t, F_{\text {VLM}}) \label {eq:decoder} 3.3

(4)

Game-CoT Reasoning Annotation

Existing reasoning datasets often lack social interaction logic and game-theoretic analysis. To bridge this gap, we propose an automated annotation pipeline powered by advanced Qwen3-VL-Plus to generate structured reasoning across four

8

X. Yan et al. /0/12324ÿ/06/1

84ÿ, 01ÿ34564789 ÿ354 599

59 4

3464789



  6 45 56  545

01 3464789

1484689 4

 #" 01ÿ789" $

11 ÿ!4"

84ÿ0 11 ÿ74594ÿ94  9

59 4

01  11 3464789 3464789  +47544889 894 8 ÿ 8 4  5)468 5* 5)468 5* " 9   11

789:;<=79>?@=A>

84ÿ- ÿ74594ÿ94  9 4 3464789

01 3464789



'#  ' 5"ÿ#  989  (45 4 8ÿ

11 1594ÿ%

&74  564ÿ% 3" 9ÿ% 4#&

#554 8ÿ3464789 854ÿ3549689

84ÿ. +49$ 5644 8ÿ94  9 4 3464789

01 3464789



11

11

+3&

+4(5  4"

ÿ8 4 

Fig. 4: Illustration of four-stage training paradigm of WCog-VLA, including threestage supervised fine-tuning and one-stage reinforcement fine-tuning. B:C?6222D628?=?98EF<GH8I2JKL2M9M?82B:>N8?2789:;<=79>?@=A>

O2O

sequential steps: (1) scene description, (2) critical object analysis, (3) gametheoretic reasoning, and (4) payoff evaluation. The final output includes the optimal ego action and the inferred responses of surrounding agents. Specifically, the game-theoretic reasoning step formulates traffic interactions as a Stackelberg game [17], where the ego vehicle acts as the leader and surrounding agents serve as followers. Adopting a ‘if-what’ imagination, the model enumerates candidate ego actions and infers the corresponding reactions of followers. The payoff evaluation step then assesses the safety and efficiency of these hypothetical outcomes to determine the optimal strategy. To minimize hallucinatory outputs and ensure logical consistency, we incorporate Ground-Truth (GT) actions as guiding hints. This compels the VLM to reconstruct explicit causal chains linking observed scene contexts to final GT actions. Ultimately, we construct a Game-CoT dataset comprising 85k high-quality annotations on the NAVSIM benchmark. More detailed Game-CoT annotation process is illustrated in the Supplemental Material.

3.4

WCog-VLA Training

Fig. 4 shows our four-stage training paradigm, including Supervised Fine-Tuning (SFT) in Stages 1–3 and Reinforcement Fine-Tuning (RFT) in Stage 4. 3D Perception Pre-Training. This stage optimizes the BEV encoder and TrackFormer. Following the perception training method in UniAD [20], we utilize a detection head for class labeling and 3D box regression. The overall detection loss combines a focal loss for classification and an L1 loss for 3D box localization: Ls1 = λfocal Lfocal + λL1 LL1 with weighting coefficients λfocal and λL1 .

WCog-VLA

9

VLM Supervised Fine-Tuning. This stage optimizes the VLM for both Visual Question Answering (VQA) capability and world cognition. The language modeling is trained on a mixture of public driving VQA datasets (e.g., DriveLM), trajectory-specific VQA, and our Game-CoT dataset. The learned semantic world cognition are routed through a world head into current perception and future prediction of surrounding agents, which are supervised by groundtruth 3D bounding boxes, and future trajectories. The overall training objective combines the text generation loss LLM and the world cognition loss Lworld : \mathcal {L}_{\text {s2}} = \mathcal {L}_{\text {LM}} + \lambda _{\text {world}} \mathcal {L}_{\text {world}} \label {eq:stage2_loss}

(5)

\mathcal {L}_{\text {world}} = \frac {1}{N_a} \sum _{i=1}^{N_a} \lambda _{\text {box}} \mathcal {L}_{\text {L1}}(b_i) + \frac {1}{N_a H} \sum _{i=1}^{N_a} \sum _{t=1}^{H} \lambda _{\text {traj}} \mathcal {L}_{\text {L1}}(\tau _{i,t}) \label {eq:loss_world}

(6)

where LLM is the standard cross-entropy loss for language modeling, LL1 (bi ) and LL1 (τi,t ) denote the L1 losses for 3D box localization and trajectory prediction of agent i at timestep t. The λ terms denote the corresponding weights. ADDT Supervised Fine-Tuning. In Stage 3, we freeze the pre-trained VLM \resizebox {0.93\linewidth }{!}{$ L_{\text {s4}} = \underbrace {-\frac {1}{GT} \sum _{i=1}^{G} \sum _{t=1}^{T} \gamma ^{t-1} \log \pi _\theta (x_{t-1}^{(i)} \mid x_t^{(i)}) \hat {A}_i}_{L_{\text {RL}}} - \underbrace {\lambda _{bc} \frac {1}{GT} \sum _{i=1}^{G} \sum _{t=1}^{T} \log \pi _\theta (\tilde {x}_{t-1}^{(i)} \mid \tilde {x}_t^{(i)})}_{L_{\text {BC}}} $} \label {eq:rl_loss} to serve as a semantic-level world model and train the ADDT for joint multiagent trajectory generation. The training objective combines the standard L2 denoising loss Ldiff and the representation alignment loss Lalign : \mathcal {L}_{\text {s3}} = \mathcal {L}_{\text {diff}} + \lambda _{\text {align}} \mathcal {L}_{\text {align}} \label {eq:loss_gil}

\mathcal {L}_{\text {diff}} = \mathbb {E}_{z_{t}, F_{at}, \epsilon \sim \mathcal {N}(0,I)} \left [ \| \mathbf {W} \odot \big ( \epsilon - \epsilon _\theta (F_{at}, t, z_t, F_{\text {VLM}}) \big ) \|_2^2 \right ] \label {eq:loss_diff}

(7) (8)

where Ldiff optimizes the generation decoder ϵθ to predict the added noise ϵ, and ⊙ is the Hadamard product. W is an agent-specific weight mask applying distinct penalties (αego and αsurr ) to prioritize the generative accuracy of the ego vehicle over surrounding agents and λalign is the weighting coefficient. Reinforcement Fine-Tuning. To enable driving exploration beyond imitation [21], we adopt DiffGRPO [14, 32], a diffusion-specific GRPO algorithm. The DiffGRPO loss includes an RL policy optimization loss and a Behavior Cloning (BC) loss to prevent policy collapse during exploration: (9)

where G and T denote the sampled group size and total denoising steps, respectively. γ is the discount coefficient mitigating instability in early denoising steps, (i) (i) πθ (xt−1 | xt ) is each step’s conditional probability. Âi is the group-relative ad(i) (i) vantage, λbc is the weight of the BC loss. x̃t−1 and x̃t are sampled from the reference policy πref (e.g., the model after SFT).

10

X. Yan et al.

We design a joint reward function decoupling the ego vehicle and surrounding agents. Ego driving quality is evaluated via the NAVSIM Predictive Driving Model Score (PDMS), whereas surrounding agents are optimized for accurate motion forecasting via a negative L1 displacement penalty. The overall reward is formulated as ri = rPDMS − λsurr LL1 (τsurr ) where λsurr balances ego planning performance with consistent surrounding agents’ trajectory prediction.

4

Experiments

4.1

Experimental setup

Dataset. We evaluate WCog-VLA on the large-scale, real-world simulation benchmarks NAVSIMv1 and NAVSIMv2 [10]. NAVSIM is a planning-oriented dataset comprising challenging scenarios. The data is partitioned into 1,192 training (navtrain) and 136 test (navtest) scenes. To establish the foundational driving cognition of the VLM, we compile a comprehensive training mixture comprising over 158k samples from open-source driving VQA datasets, including DriveLM [47], CODA-LM [3], LingoQA [40], nuScenes-QA [43], NuInstruct [11], and DriveGPT4 [55]. This corpus is further augmented with 170K NAVSIMtailored samples, including 85k trajectory-specific VQA and 85k Game-CoT reasoning samples. Implementation Details. Our training pipeline consists of four sequential stages. First, the 3D perception module is trained on NAVSIM for 1 epoch. Second, the VLM is pre-trained for 1 epoch on the 158k VQA samples, followed by 3 epochs of joint fine-tuning with the world heads using the 170K NAVSIMtailored samples. Third, with the VLM frozen, ADDT is trained via DDPM [18] for 200 epochs on NAVSIM. The ADDT features a symmetric 16-block DiT architecture (8 blocks each for the condition encoder and generation decoder). Representation alignment is extracted from the 6th DiT encoder block. Finally, ADDT is refined via GRPO on NAVSIM for 10 epochs using 6 group sizes. All training stages are conducted on 4 NVIDIA A100 40GB GPUs. Additional implementation details are provided in the Supplemental Material. 4.2

Main Results

Results on NAVSIM v1. Tab. 1 presents the closed-loop evaluation of our WCog-VLA on the NAVSIM v1. Our method achieves a SOTA PDMS of 92.9, outperforming all listed standard end-to-end and VLM-based methods. Notably, despite using only camera inputs, WCog-VLA surpasses multi-modal baselines that leverage both camera and lidar inputs, yielding an improvement of 4.6 PDMS over WoTE. Furthermore, WCog-VLA demonstrates clear advantages over VLM-based methods. It outperforms two massive generalist models, QwenVL2.5 and InternVL3, by 9.6 PDMS, validating the effectiveness of our tailored architecture and driving knowledge injection. Crucially, our compact

WCog-VLA

11

Table 1: Performance comparison on NAVSIM v1 navtest. Our WCog-VLA is evaluated after the complete four-stage training. Metrics include NC (no at-fault collision), DAC (drivable area compliance), TTC (time-to-collision), Comf. (comfort), EP (ego progress), and PDMS (predictive driver model score). † indicates models fine-tuned on the NAVSIM trajectory-specific dataset. Method

Image Lidar NC↑ DAC↑ TTC↑ Comf. ↑ EP↑

Constant Velocity Ego Status MLP

PDMS↑

68.0 93.0

57.8 77.3

50.0 83.6

100 100

19.4 62.8

20.6 65.6

97.2 98.9 97.8 97.7 97.7 97.9 98.0 98.3 98.2 98.5 98.6

89.1 90.7 91.9 92.5 92.8 92.4 93.1 96.0 96.2 96.8 98.3

91.6 94.9 92.9 92.9 92.8 93.0 94.8 94.6 94.7 94.9 94.9

100 95.6 100 100 100 99.8 100 100 100 99.9 100

76.0 79.7 78.8 78.7 79.2 79.3 80.1 78.7 82.2 81.9 88.0

80.9 82.4 83.4 83.8 84.0 84.0 85.5 86.5 88.1 88.3 91.7

VADv2-V8192 [5] DrivingGPT [6] UniAD [20] BevDrive [58] TransFuser [8] PARA-Drive [50] DRAMA [59] Hydra-MDP-V8192 -W-EP [33] DiffusionDrive [35] WoTE [31] iPad [15]

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

VLMs-based Methods QwenVL2.5-8B [2]† InternVL3-8B [65]† ReCogDrive-2B [32] AutoVLA-3B [64] LatentVLA-3B [52]

✓ ✓ ✓ ✓ ✓

97.8 97.0 97.9 99.1 98.9

92.1 92.4 97.3 97.1 98.2

92.8 91.8 94.9 97.1 95.2

100 100 100 100 100

78.3 78.9 87.3 87.6 88.2

83.3 83.3 90.8 92.1 92.4

WCog-VLA-2B(ours)

✓

99.4

98.8

98.5

100

87.1

92.9

✓ ✓ ✓ ✓ ✓ ✓

2B model surpasses the RL-refined VLM-based methods, ReCogDrive and AutoVLA, by at least 0.8 PDMS and outperforms the 3B-parameter LatentVLA by 0.5 PDMS. These results underscore that enhancing VLA with world cognition and world-dynamics forecasting enables superior planning performance. Beyond overall planning performance, WCog-VLA excels in safety metrics, achieving a remarkable 99.4 in NC and 98.5 in TTC. This is because our model anticipates the future intents of surrounding agents, enabling proactive safety measures and collision avoidance in complex scenarios. Results on NAVSIM v2. Tab. 2 presents the evaluation on the NAVSIM v2 benchmark, with WCog-VLA deployed after three-stage SFT process. Our WCog-VLA achieves a SOTA Extended PDMS (EPDMS) of 85.9, outperforming DiffusionDrive by 1.6 EPDMS. Besides, WCog-VLA attains the highest safety scores in both NC and TTC, while maintaining highly competitive performance across all other metrics. These findings further confirm the effectiveness and robust generalization capability of WCog-VLA in extended driving evaluations. 4.3

Ablation Study

Effect of the Four-Stage Training. Tab. 3 ablates our four-stage training paradigm. Using only Stage 2 yields a baseline PDMS of 84.4, whereas incorporating 3D perception pre-training improves 1.1 PDMS, reflecting enhanced

12

X. Yan et al.

Table 2: Performance comparison on NAVSIM v2 navtest with extended metrics. Our WCog-VLA is evaluated after three-stage SFT. Newly introduced metrics include DDC (driving direction compliance), TLC (traffic light compliance), LK (lane keeping), HC (history comfort), EC (extended comfort), and EPDMS (extended PDMS). Method

NC ↑ DAC ↑ DDC ↑ TLC ↑ EP ↑ TTC ↑ LK ↑ HC ↑ EC ↑ EPDMS ↑

VADv2 [5] TransFuser [8] HydraMDP++ [33] ARTEMIS [12] ReCogDrive-8B [32] WoTE [31] DiffusionDrive [35]

97.3 97.7 97.9 98.3 98.3 98.5 98.0

91.7 92.8 96.5 95.1 95.2 96.8 96.0

98.2 98.3 98.9 98.6 99.5 98.8 99.5

99.9 99.9 100 99.8 99.8 99.8 99.8

77.6 79.2 79.2 81.5 87.1 86.1 87.7

92.7 92.8 93.4 97.4 97.5 97.9 97.1

66.0 67.6 67.2 96.5 96.6 95.5 97.2

100 100 100 100 98.3 98.3 98.3

97.4 95.3 97.7 98.3 86.5 82.9 87.6

76.6 77.8 80.6 83.1 83.6 84.2 84.3

WCog-VLA-2B(ours) 98.8

96.6

99.3

99.8

85.8

98.2

96.4

98.3

86.3

85.9

Table 3: Ablation on the four-stage Table 4: Ablation on dual-level world cognitraining process. Trajectories are gen- tion. Cur and Fut denote current perception erated as textual tokens via the VLM and future prediction supervision. Generative in IDs 1 and 2, and as continuous ac- enables joint multi-agent trajectory synthesis; tions through ADDT in IDs 3 and 4. otherwise, only ego-trajectory generated. ID Stage 1 Stage 2 Stage 3 Stage 4 PDMS ↑

ID

1

1 2 3 4 5 6

✓

84.4

2

✓

✓

85.5

3

✓

✓

✓

4

✓

✓

✓

89.3 ✓

92.9

Semantic Cur Fut ✓ ✓

✓ ✓

✓

✓

Generative

PDMS ↑

✓ ✓

86.5 87.0 87.2 88.1 87.4 89.3

spatial understanding. Introducing ADDT in Stage 3 fundamentally shifts the paradigm from discrete textual output to continuous trajectory generation, resulting in a 3.8 PDMS improvement. Finally, Stage 4 RFT further optimizes the driving policy, improving 3.6 PDMS and achieving the SOTA 92.9. These consistent gains confirm that every training stage is indispensable. Effect of Dual-Level World Cognition. Tab. 4 evaluates the effect of duallevel world cognition, where all variants are trained with the three-stage SFT. The baseline without either cognitive level achieves 86.5 PDMS. Integrating semantic current perception or future prediction improves the score to 87.0 and 87.2, respectively, while combining both yields 88.1. Enabling only generativelevel multi-agent synthesis without semantic cognition achieves 87.4. Ultimately, unifying both semantic with generative cognition triggers a synergistic leap to 89.3 PDMS. These findings demonstrate that coupling semantic forecasting with generative evolution is essential for robust planning. Effect of ADDT. Tab. 5 evaluates our ADDT design in terms of PDMS and inference time. We compare pure VLM text generation, including direct answer (VLMwo/r ) and Game-CoT reasoning (VLMw/r ), against a Standard Diffusion Transformer (SDT) [41] and several ADDT variants: without alignment (DDT), without the decoupled architecture (ADT), and the full ADDT. All diffusion

WCog-VLA Table 5: Effect of ADDT. wo/r means VLM text output without reasoning, w/r is with reasoning. Method

VLM+DDT VLM+ADT VLM+ADDT

Effect

Denoise step

PDMS ↑

Infer Time (s) ↓

ID

Traj

Drive

1.131 9.896 0.105 0.388 0.108 0.381 0.103 0.392 0.106 0.383

1 2 3 4

✓ ✓ ✓ ✓

✓

5 20 5 20 5 20 5 20

85.0 85.5 87.4 88.5 87.9 88.7 88.6 89.1 89.3 89.6

VLMwo/r VLMw/r VLM+SDT [41]

Table 6: dataset.

✓

of

13 VQA

CoT

PDMS ↑

✓ ✓

86.7 88.2 87.5 89.3

Table 7: 3D perception effect. 3D perception × ✓

PDMS ↑ 86.0 89.3

models utilize an identical DiT backbone and are trained with three-stage SFT. Results show that ADDT with 5 denoising steps achieves a 10.7× speedup over direct VLM text generation. Crucially, ADDT attains superior performance with fewer denoising steps. Compared to the 20-step SDT, our 5-step ADDT improves PDMS by 0.8 while accelerating inference by 3.7×. Besides, ADDT exhibits low sensitivity to the number of denoising steps: increasing the steps from 5 to 20 yields a marginal 0.3 PDMS gain. This robustness stems from the explicit alignment mechanism, which maintains consistent encoder latent features across different denoising steps and reduces the need for costly iterative refinement. Effect of VQA Dataset. Tab. 6 shows that training solely on NAVSIM trajectory-specific VQA yields a baseline PDMS of 86.7. Adding open-source driving VQA (Drive) or Game-CoT reasoning data (CoT) improves PDMS to 88.2 and 87.5. Combining all three data sources achieves the highest PDMS of 89.3, confirming that incorporating diverse VQA data can improve performance. Effect of 3D Perception. Tab. 7 validates the contribution of the 3D perception module. Without explicit 3D perception, ADDT relies solely on generic VLM vision tokens, which limits spatial precision and yields a PDMS of 86.0. Incorporating the dedicated 3D perception boosts performance to 89.3. More ablation study results (e.g., effect of alignment layer position) are shown in the Supplemental Material. 4.4

Qualitative Results

Compare with Previous SOTA Method. Fig. 5 compares WCog-VLA with ReCogDrive [32] in complex urban scenarios. ReCogDrive acts overly conservatively, remaining trapped in the current slow lane. Conversely, WCog-VLA identifies the slow leading bus in the current lane and changes the lane to improve efficiency, closely matching the human ground truth. This visualization demonstrates our WCog-VLA enables efficient and human-aligned driving behaviors. Proactive Driving via Generative-Level World Cognition. Fig. 6 highlights the advantages of generative world cognition. In the intersection scenario,

14

X. Yan et al. Ground truth

Front concat view

Ego planning trajectory

ReCogDrive

WCog-VLA

Fig. 5: Comparison with previous SOTA method on Navtest. Ground truth

Ego planning trajectory

Front concat view

Surrounding predicted trajectory

Without generative world cognition

With generative world cognition

Fig. 6: Visualization of proactive driving via generative-level world cognition. Future prediction

Predicted boxes

Ground truth boxes

Compare with previous method 忽略了对旁边大卡车体型的 Ground truth 感知，导致想插进去

Prediction

Fig. 7: Visualization of our semantic-level world cognition.

the baseline lacks interactive foresight of the oncoming vehicle and generates an ego-only trajectory, resulting in passive deceleration to blindly avoid spurious conflicts. Conversely, our model synthesizes joint multi-agent trajectories that explicitly forecast the oncoming vehicle’s straight trajectory. This foresight enables ego vehicle to confidently execute a left turn. The results confirm that generative world cognition empowers the model to perform proactive maneuvers. Explicit Semantic-Level World Cognition Representation. Fig. 7 shows the semantic world cognition decoded via the world head. Both 3D perception and future trajectory prediction align closely with the ground truth, demonstrating the model’s cognition of current world states and future world dynamics. More qualitative results are shown in the Supplemental Material.

5

Conclusion

In this work, we propose WCog-VLA, a novel VLA framework with explicit dual-level World Cognition for end-to-end autonomous driving. To bridge the

WCog-VLA

15

gap between semantic-level forecasting and generative-level evolution, WCogVLA tightly couples a multi-modal VLM backbone with a generative Aligned Decoupled Diffusion Transformer (ADDT). At the semantic level, our model unifies the world cognition and reasoning, enabling comprehensive world understanding and interactive game-theoretic reasoning. At the generative level, ADDT acts as a generative world model, translates the VLM’s cognitive representations into physically-plausible joint multi-agent trajectories. Extensive experiments on the NAVSIM v1 and v2 benchmarks demonstrate SOTA performance of WCog-VLA. However, the current semantic cognition focuses on agents and omits the future evolution of road geometry and map topology. Future work will incorporate these dynamics to build a more comprehensive world model.

16

X. Yan et al.

References 1. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023) 2. Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025) 3. Chen, K., Li, Y., Zhang, W., Liu, Y., Li, P., Gao, R., Hong, L., Tian, M., Zhao, X., Li, Z., et al.: Automated evaluation of large vision-language models on self-driving corner cases. In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 7817–7826. IEEE (2025) 4. Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., Li, H.: End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 46(12), 10164–10183 (2024) 5. Chen, S., Jiang, B., Gao, H., Liao, B., Xu, Q., Zhang, Q., Huang, C., Liu, W., Wang, X.: Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243 (2024) 6. Chen, Y., Wang, Y., Zhang, Z.: Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 26890–26900 (2025) 7. Chi, H., Gao, H.a., Liu, Z., Liu, J., Liu, C., Li, J., Yang, K., Yu, Y., Wang, Z., Li, W., et al.: Impromptu vla: Open weights and open data for driving visionlanguage-action models. arXiv preprint arXiv:2505.23757 (2025) 8. Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., Geiger, A.: Transfuser: Imitation with transformer-based sensor fusion for autonomous driving. IEEE transactions on pattern analysis and machine intelligence 45(11), 12878–12895 (2022) 9. Dang, C., Wang, J., Li, G., Hou, Z., You, Z., Ye, H., Ma, J., Chen, L., Wang, Y.: Sparseoccvla: Bridging occupancy and vision-language models via sparse queries for unified 4d scene understanding and planning. arXiv preprint arXiv:2601.06474 (2026) 10. Dauner, D., Hallgarten, M., Li, T., Weng, X., Huang, Z., Yang, Z., Li, H., Gilitschenski, I., Ivanovic, B., Pavone, M., et al.: Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems 37, 28706–28719 (2024) 11. Ding, X., Han, J., Xu, H., Liang, X., Zhang, W., Li, X.: Holistic autonomous driving understanding by bird’s-eye-view injected multi-modal large models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13668–13677 (2024) 12. Feng, R., Xi, N., Chu, D., Wang, R., Deng, Z., Wang, A., Lu, L., Wang, J., Huang, Y.: Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving. IEEE Robotics and Automation Letters 11(1), 226–233 (2025) 13. Fu, H., Zhang, D., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., Bai, X.: Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 24823–24834 (2025) 14. Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

WCog-VLA

17

15. Guo, K., Liu, H., Wu, X., Pan, J., Lv, C.: ipad: Iterative proposal-centric end-toend autonomous driving. arXiv preprint arXiv:2505.15111 (2025) 16. Han, J., Tian, M., Zhu, J., He, F., Zhang, H., Guo, S., Zhu, D., Tang, H., Xu, P., Guo, Y., et al.: Percept-wam: Perception-enhanced world-awareness-action model for robust end-to-end autonomous driving. arXiv preprint arXiv:2511.19221 (2025) 17. Hang, P., Lv, C., Xing, Y., Huang, C., Hu, Z.: Human-like decision making for autonomous driving: A noncooperative game theoretic approach. IEEE Transactions on Intelligent Transportation Systems 22(4), 2076–2087 (2020) 18. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851 (2020) 19. Hu, S., Chen, L., Wu, P., Li, H., Yan, J., Tao, D.: St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. In: European Conference on Computer Vision. pp. 533–549. Springer (2022) 20. Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al.: Planning-oriented autonomous driving. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 17853– 17862 (2023) 21. Huang, Z., Sheng, Z., Qu, Y., You, J., Chen, S.: Vlm-rl: A unified vision language models and reinforcement learning framework for safe autonomous driving. Transportation Research Part C: Emerging Technologies 180, 105321 (2025) 22. Hwang, J.J., Xu, R., Lin, H., Hung, W.C., Ji, J., Choi, K., Huang, D., He, T., Covington, P., Sapp, B., et al.: Emma: End-to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262 (2024) 23. Jiang, A., Gao, Y., Sun, Z., Wang, Y., Wang, J., Chai, J., Cao, Q., Heng, Y., Jiang, H., Dong, Y., et al.: Diffvla: Vision-language guided diffusion planning for autonomous driving. arXiv preprint arXiv:2505.19381 (2025) 24. Jiang, A., Gao, Y., Wang, Y., Sun, Z., Wang, S., Heng, Y., Sun, H., Tang, S., Zhu, L., Chai, J., et al.: Irl-vla: Training an vision-language-action policy via reward world model. arXiv preprint arXiv:2508.06571 (2025) 25. Jiang, B., Chen, S., Liao, B., Zhang, X., Yin, W., Zhang, Q., Huang, C., Liu, W., Wang, X.: Senna: Bridging large vision-language models and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313 (2024) 26. Jiang, B., Chen, S., Xu, Q., Liao, B., Chen, J., Zhou, H., Zhang, Q., Liu, W., Huang, C., Wang, X.: Vad: Vectorized scene representation for efficient autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8340–8350 (2023) 27. Jiang, S., Huang, Z., Qian, K., Luo, Z., Zhu, T., Zhong, Y., Tang, Y., Kong, M., Wang, Y., Jiao, S., et al.: A survey on vision-language-action models for autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4524–4536 (2025) 28. Li, J., Wu, J., Hu, D., Huang, X., Sun, B., Hao, Z., Lang, X., Zhu, X., Zhang, L.: Sgdrive: Scene-to-goal hierarchical world cognition for autonomous driving. arXiv preprint arXiv:2601.05640 (2026) 29. Li, P., Zheng, Y., Wang, Y., Wang, H., Zhao, H., Liu, J., Zhan, X., Zhan, K., Lang, X.: Discrete diffusion for reflective vision-language-action models in autonomous driving. arXiv preprint arXiv:2509.20109 (2025) 30. Li, Y., Shang, S., Liu, W., Zhan, B., Wang, H., Wang, Y., Chen, Y., Wang, X., An, Y., Tang, C., et al.: Drivevla-w0: World models amplify data scaling law in autonomous driving. arXiv preprint arXiv:2510.12796 (2025)

18

X. Yan et al.

31. Li, Y., Wang, Y., Liu, Y., He, J., Fan, L., Zhang, Z.: End-to-end driving with online trajectory evaluation via bev world model. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 27137–27146 (2025) 32. Li, Y., Xiong, K., Guo, X., Li, F., Yan, S., Xu, G., Zhou, L., Chen, L., Sun, H., Wang, B., et al.: Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving. arXiv preprint arXiv:2506.08052 (2025) 33. Li, Z., Li, K., Wang, S., Lan, S., Yu, Z., Ji, Y., Li, Z., Zhu, Z., Kautz, J., Wu, Z., et al.: Hydra-mdp: End-to-end multimodal planning with multi-target hydradistillation. arXiv preprint arXiv:2406.06978 (2024) 34. Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., Dai, J.: Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence 47(3), 2020–2036 (2024) 35. Liao, B., Chen, S., Yin, H., Jiang, B., Wang, C., Yan, S., Zhang, X., Li, X., Zhang, Y., Zhang, Q., et al.: Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 12037–12047 (2025) 36. Liu, L., Song, Z., Jia, C., Ye, H., Hao, X., Chen, L., et al.: Driveworld-vla: Unified latent-space world modeling with vision-language-action for autonomous driving. arXiv preprint arXiv:2602.06521 (2026) 37. Liu, P., Ning, Q., Lu, X., Liu, H., Ma, W., She, D., Jia, P., Lang, X., Ma, J.: Omnireason: A temporal-guided vision-language-action framework for autonomous driving. arXiv preprint arXiv:2509.00789 (2025) 38. Liu, Z., Huang, R., Yang, R., Yan, S., Wang, Z., Hou, L., Lin, D., Bai, X., Zhao, H.: Drivepi: Spatial-aware 4d mllm for unified autonomous driving understanding, perception, prediction and planning. arXiv preprint arXiv:2512.12799 (2025) 39. Luo, Y., Li, F., Xu, S., Lai, Z., Yang, L., Chen, Q., Luo, Z., Xie, Z., Jiang, S., Liu, J., et al.: Adathinkdrive: Adaptive thinking via reinforcement learning for autonomous driving. arXiv preprint arXiv:2509.13769 (2025) 40. Marcu, A.M., Chen, L., Hünermann, J., Karnsund, A., Hanotte, B., Chidananda, P., Nair, S., Badrinarayanan, V., Kendall, A., Shotton, J., et al.: Lingoqa: Visual question answering for autonomous driving. In: European Conference on Computer Vision. pp. 252–269. Springer (2024) 41. Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4195–4205 (2023) 42. Qian, K., Jiang, S., Zhong, Y., Luo, Z., Huang, Z., Zhu, T., Jiang, K., Yang, M., Fu, Z., Miao, J., et al.: Agentthink: A unified framework for tool-augmented chainof-thought reasoning in vision-language models for autonomous driving. arXiv preprint arXiv:2505.15298 1(2), 3 (2025) 43. Qian, T., Chen, J., Zhuo, L., Jiao, Y., Jiang, Y.G.: Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 4542–4550 (2024) 44. Renz, K., Chen, L., Arani, E., Sinavski, O.: Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 11993–12003 (2025) 45. Sun, W., Lin, X., Shi, Y., Zhang, C., Wu, H., Zheng, S.: Sparsedrive: End-to-end autonomous driving via sparse scene representation. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 8795–8801. IEEE (2025)

WCog-VLA

19

46. Team, R., Gao, Z., Wang, Q., Zeng, Y., Zhu, J., Cheng, K.L., Li, Y., Wang, H., Xu, Y., Ma, S., et al.: Advancing open-source world models. arXiv preprint arXiv:2601.20540 (2026) 47. Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., Zhan, K., Jia, P., Lang, X., Zhao, H.: Drivevlm: The convergence of autonomous driving and large visionlanguage models. arXiv preprint arXiv:2402.12289 (2024) 48. Wang, S., Tian, Z., Huang, W., Wang, L.: Ddt: Decoupled diffusion transformer. arXiv preprint arXiv:2504.05741 (2025) 49. Wang, Y., Luo, W., Bai, J., Cao, Y., Che, T., Chen, K., Chen, Y., Diamond, J., Ding, Y., Ding, W., et al.: Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail. arXiv preprint arXiv:2511.00088 (2025) 50. Weng, X., Ivanovic, B., Wang, Y., Wang, Y., Pavone, M.: Para-drive: Parallelized architecture for real-time autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15449–15458 (2024) 51. Xiao, J., Yang, Y., Chang, X., Chen, R., Xiong, F., Xu, M., Zheng, W.S., Zhang, Q.: World-env: Leveraging world model as a virtual environment for vla post-training. arXiv preprint arXiv:2509.24948 (2025) 52. Xie, C., Sun, B., Li, T., Wu, J., Hao, Z., Lang, X., Li, H.: Latentvla: Efficient vision-language models for autonomous driving via latent action prediction. arXiv preprint arXiv:2601.05611 (2026) 53. Xing, S., Qian, C., Wang, Y., Hua, H., Tian, K., Zhou, Y., Tu, Z.: Openemma: Open-source multimodal model for end-to-end autonomous driving. In: Proceedings of the Winter Conference on Applications of Computer Vision. pp. 1001–1009 (2025) 54. Xiong, Z., Ye, X., Yaman, B., Cheng, S., Lu, Y., Luo, J., Jacobs, N., Ren, L.: Unidrive-wm: Unified understanding, planning and generation world model for autonomous driving. arXiv preprint arXiv:2601.04453 (2026) 55. Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.Y.K., Li, Z., Zhao, H.: Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters 9(10), 8186–8193 (2024) 56. Yang, Z., Chai, Y., Jia, X., Li, Q., Shao, Y., Zhu, X., Su, H., Yan, J.: Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving. arXiv preprint arXiv:2505.16278 (2025) 57. Yu, S., Kwak, S., Jang, H., Jeong, J., Huang, J., Shin, J., Xie, S.: Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940 (2024) 58. Yu, Z., Li, J., Wei, Y., Lyu, Y., Tan, X.: Combining camera–lidar fusion and motion planning using bird’s-eye view representation for end-to-end autonomous driving. Drones 9(4), 281 (2025) 59. Yuan, C., Zhang, Z., Sun, J., Sun, S., Huang, Z., Lee, C.D.W., Li, D., Han, Y., Wong, A., Tee, K.P., et al.: Drama: An efficient end-to-end motion planner for autonomous driving with mamba. arXiv preprint arXiv:2408.03601 (2024) 60. Zhang, D., Sun, J., Hu, C., Wu, X., Yuan, Z., Zhou, R., Shen, F., Zhou, Q.: Pure vision language action (vla) models: A comprehensive survey. arXiv preprint arXiv:2509.19012 (2025) 61. Zheng, W., Song, R., Guo, X., Zhang, C., Chen, L.: Genad: Generative end-to-end autonomous driving. In: European Conference on Computer Vision. pp. 87–104. Springer (2024)

20

X. Yan et al.

62. Zheng, Y., Liang, R., Zheng, K., Zheng, J., Mao, L., Li, J., Gu, W., Ai, R., Li, S.E., Zhan, X., et al.: Diffusion-based planning for autonomous driving with flexible guidance. arXiv preprint arXiv:2501.15564 (2025) 63. Zhou, X., Han, X., Yang, F., Ma, Y., Tresp, V., Knoll, A.: Opendrivevla: Towards end-to-end autonomous driving with large vision language action model. arXiv preprint arXiv:2503.23463 (2025) 64. Zhou, Z., Cai, T., Zhao, S.Z., Zhang, Y., Huang, Z., Zhou, B., Ma, J.: Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506.13757 (2025) 65. Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025)