TANGO : Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model Anqi Li1,2,∗ , Yuxin Chen1,∗,‡ , Zhaobo Li1,∗ , Zhuo Cao1,3,∗ , Junli Ren1,4 , Masayoshi Tomizuka1 , Dhruv Shah5,† University of California, Berkeley, 2 Peking University, 3 Tsinghua University, 4 The University of Hong Kong, 5 Princeton University ∗ Equal contribution., ‡ Project lead., † Corresponding author.
arXiv:2609.09158v1 [cs.RO] 8 Sep 2026
1
We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first wholebody vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data. Keywords: Vision-Language Navigation, Vision-Language-Action Model, Whole-Body Control Website: tango-vla.github.io
1
Introduction
Language-guided traversal in complex 3D environments is a fundamental capability for domestic humanoid robots expected to assist with everyday tasks. [1] Unlike wheeled or mobile-base robots, humanoids navigate with high-dimensional articulated bodies whose geometry changes continuously during motion, making the robot’s body configuration an inherent part of the navigation problem. As illustrated in Figure 1, traversal feasibility in cluttered indoor spaces depends not only on the intended route but also on whether the robot can physically move through the surrounding scene geometry without collisions involving the arms, torso, or legs. This creates a tight coupling between navigation
1
Long-Horizon Navigation
Stride
Squat
Sidestep
Figure 1 | Humanoid whole-body navigation in cluttered environments. We propose TANGO, a whole-body foundation model for cluttered indoor scene navigation. TANGO demonstrates strong scene understanding and robust traversal capability, navigating a 30-meter route zero-shot in a real-world cluttered office scene, and traversing through cluttered scenes with overhead and floor obstacles, as well as narrow passages.
decisions and whole-body feasibility: An action that appears valid at the planning level may still be infeasible for the embodied humanoid to execute. Despite rapid progress in humanoid control [2, 3, 4, 5], vision-language navigation (VLN) [6, 7, 8, 9, 10, 11], and collision-aware traversal [10, 12, 13], effectively integrating these capabilities remains largely unexplored. Existing VLN methods typically formulate navigation as high-level decision making, where an agent predicts 2D waypoints or discrete actions from visual observations and language instructions [14, 15, 16]. While effective for mobile platforms and simplified embodied agents, their low-dimensional action spaces cannot explicitly represent the relationship between navigation decisions and wholebody feasibility, limiting their ability to handle spatially constrained traversal scenarios. Recent humanoid foundation models and whole-body VLA systems [17, 18, 19, 20] have demonstrated impressive whole-body control capabilities. Nevertheless, navigation in these systems is typically represented through high-level locomotion commands and delegated to downstream controllers, preventing explicit reasoning about whole-body traversability during navigation. A complementary line of work explores collision-aware humanoid traversal through reinforcement learning [12, 21]. Although effective in specific traversal scenarios, these approaches often rely on task-specific priors or training distributions, limiting their scalability to long-horizon language-guided navigation in diverse cluttered environments. Consequently, existing approaches remain unable to jointly reason about navigation intent and whole-body traversability, motivating the need for a unified framework for 2
language-guided whole-body navigation. To address these challenges, we present Traversability-Aware Vision-Language Navigation (TANGO), a unified VLA framework for humanoid navigation in cluttered environments. Given a language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for whole-body humanoid control, avoiding the need for separate navigation and control modules. A key challenge is obtaining large-scale training data that captures both semantic task diversity and physically plausible whole-body traversal behaviors. To this end, we develop a scalable simulation pipeline that automatically synthesizes collision-free humanoid traversal trajectories and provides dynamically feasible supervision for training the VLA model. At deployment, the learned policy is executed through a robust motion tracker with real-time action chunking [2], enabling reliable execution and zero-shot transfer to real humanoid hardware. We evaluate TANGO against state-ofthe-art VLN and humanoid spatial traversal baselines in both simulation and real-world environments. Across long-horizon navigation tasks requiring obstacle negotiation and geometry-aware wholebody adaptation, TANGO consistently achieves stronger collision-free traversal performance than existing methods. We will open-source the data pipeline, generated dataset, VLA framework, model checkpoint, and deployment system to facilitate reproducibility.
2
Related Works
Large Models for Vision-Language Navigation. Recent large multi-modality models (LMMs) emerge with strong scene understanding and physical awareness, leading to extensive zero-shot navigation works that leverage off-the-shelf large models [22, 23, 24, 25]. Moreover, recent efforts have explored fine-tuning such models on simulated and real-world navigation samples, resulting in strong VLA models for navigating in diversified environments [7, 8, 10, 11, 16, 26]. Nevertheless, these methods take visual navigation as a pure planar trajectory planning task, omitting the physical gap [27] when deployed in real physical environments. In contrast, TANGO is trained with inherent physical awareness, mitigating the embodied gap while enabling explicit reasoning about whole-body traversability in cluttered environments. Recent efforts have also explored dual-system design for VLA models to achieve continuous, real-time navigation [10, 28]. In this work, we equip TANGO with a flow-matching-based action expert as system-1, trained with real-time chunking [29], to achieve continuous and responsive humanoid control. Cluttered Environment Traversal. Traversal in cluttered scenes is critical for deploying embodied agents in complex real-world scenarios. Recent humanoid parkour works have demonstrated impressive traversal capabilities over challenging terrains and obstacles [21, 30, 31]. However, these methods mainly focus on short-horizon interactions with scene objects. In contrast, TANGO performs long-horizon navigation with collision avoidance, which requires excellence in physical and semantic understanding capabilities. HumanoidPF [12] introduces RL-based collision-free indoor traversal for humanoids and achieves high success rates in most cases, but remains difficult to scale, especially to long-horizon navigation and complex obstacle compositions. In this work, we synthesize collisionfree and dynamically feasible humanoid motions through a scalable pipeline, collecting low-cost, high-quality datasets for 3D traversal policy training. Some VLN works [13, 32] also study traversal in cluttered scenes, but are fundamentally limited by their 2D problem formulation and primarily consider bypassing behaviors. In contrast, TANGO learns humanoid whole-body motions, enabling richer capabilities when facing complex obstacles. Humanoid Whole-Body Control through Large-Scale Learning. Recent advances in humanoid motion tracking [2, 3, 33] have enabled large-scale learning of humanoid control policies. Represen-
3
tative works such as GR00T-N1.6 [18], Ψ0 [17], and WholeBodyVLA [19] adopt a decoupled design, predicting upper-body motions while issuing high-level commands to a lower-body tracker. While this significantly simplifies loco-manipulation learning, it limits whole-body coordination required for tasks such as cluttered-scene traversal. In contrast, TANGO directly learns end-to-end whole-body motions and uses them as reference trajectories for a low-level tracker. Non-decoupled approaches, including LeVERB [34], HumanoidVLA [20], and PhysiFlow [35], learn latent motion representations decoded by specialized controllers. Instead, TANGO directly predicts executable whole-body actions and relies on a pre-trained general-purpose tracker for execution, avoiding controller co-training and task-specific motion decoders while enabling scalable deployment across diverse humanoid platforms and traversal tasks.
3
Method
We present TANGO, a whole-body VLA system for cluttered indoor vision-language navigation (Figure 2). In this section, we first provide a formal definition of whole-body vision-language navigation (Section 3.1). Next, we introduce an automated data-generation pipeline for building a large-scale whole-body navigation dataset through scalable scene augmentation and motion editing in simulation environments (Section 3.2). We then describe how TANGO generates whole-body motions from language instructions and RGB observations (Section 3.3). Lastly, we describe how to deploy TANGO both in simulated environments and on a real humanoid robot (Section 3.4).
3.1
Problem Formulation
Existing VLN works are fundamentally limited by their planar action spaces, failing to represent complex traversing motions in real-world environments. In this work, we study the problem of whole-body vision-language navigation. Given a natural language instruction ℓ, current observation ,dn o𝑡 containing a temporal sequence of RGB images from front and downward cameras Ifr 1:𝑡 and whole-body joint-angle proprioceptive state q𝑡 , our model learns to predict a whole-body action chunk A𝑡 = {a1 , · · · , a 𝐻 } over an action horizon 𝐻 , where a𝑖 = {qd,𝑖 , rb,𝑖 }, 𝑖 ∈ {1, · · · , 𝐻 }, with qd,𝑖 ∈ ℝ29 and rb,𝑖 ∈ ℝ6 denoting the desired whole-body joint angles and the base 6D rotation representation at the 𝑖-th step, respectively. The predicted action chunk is then streamed to a low-level motion tracker for physically grounded humanoid navigation.
3.2
Simulation Data Generation
Environment Augmentation. Existing navigation datasets [6, 36, 37] primarily capture standard room layouts , while real-world environments are characterized by randomly placed objects that create complex spatial constraints for robot traversal. To create more challenging scenarios, we augment indoor scenes from VLNVerse [36] and SAGE-3D [37], which originally contain 263 and 1,000 scenes, respectively. After filtering low-quality scenes using Gemini 2.5 Flash, we retain 205 VLNVerse scenes and 373 SAGE-3D scenes, totaling 578 source scenes for augmentation. Following HumanoidPF [12], we introduce three obstacle categories: lateral obstacles to construct narrow passages, ground-level obstacles to necessitate stepping, and overhead obstacles to enforce upper-body clearance. To ensure visual and semantic consistency, we curate category-specific templates from existing assets within the source datasets and automatically instantiate these semantically matched objects via SO ( 3) transformations. Please refer to Appendix C.1 for further details. Collision-Free Motion Generation. Humanoid motion collection pipelines often rely on laborintensive human motion capture followed by retargeting [38, 39, 40], which introduces motion 4
Figure 2 | TANGO Architecture and Data Pipeline. Top: the proposed Plan–Edit–Track (PET) pipeline automatically synthesizes collision-free whole-body traversal data. Bottom: TANGO combines a vision-language backbone, a diffusion-based action expert, and a low-level tracker for language-guided whole-body humanoid control.
degradation and embodiment mismatch. To generate scalable, human-like traversal data, we propose Plan, Edit, Track (PET), an automatic pipeline that synthesizes collision-free, whole-body motions for humanoid cluttered indoor navigation, in an offline manner. Given limited space, full details are discussed in Appendix C.2. Given a start and goal location in an augmented scene, PET first plans a collision-aware planar reference path using A* with an obstacle-aware path cost that biases the search toward safer regions. A heading adjustment module detects narrow passages and inserts 90-degree heading changes, inducing sideways walking when frontal traversal is spatially constrained. The resulting trajectory is converted into velocity commands for the SONIC motion planner [2] to produce natural humanoid walking motions along the planned 2D trajectories. We then replay these trajectories in IsaacSim [41] to render egocentric RGB observations at 2 Hz, which is then fed into Gemini 2.5 Flash [42] to generate formatted VLN instructions, following previous work [36]. PET subsequently edits the synthesized motions to incorporate whole-body obstacle interactions. We apply sampled guidance forces from humanoid potential field [12] conducted with obstacle and ground truth trajectory priors to key body links through SoftMimic-style pseudo-forces [43]. To negotiate ground-level obstacles, our gait-adaptation module retargets foot landing positions beyond each obstacle and adjusts swing-foot clearance while preserving the reference gait phase and timing. This stage yields reference motions with explicit whole-body avoidance behaviors, such as arm clearance, stepping over, and crouching. Finally, PET employs a SONIC tracker as an executability filter. The edited motions are tracked in simulation to verify physical feasibility and collision freedom. Failing or colliding trajectories are discarded. Crucially, instead of using the tracked trajectories as training supervision, which might degrade the human-like quality, we utilize the collision-free reference motions from the planning and editing stages as the action supervision signal. This preserves human-like structures while ensuring physical executability. Based on the verified trajectories, we regenerate the RGB observations and
5
update the language instructions accordingly. The resulting dataset contains 64,633 trajectories; PET and rendering require 86 and 125 RTX PRO 6000 GPU-hours, respectively, for a total of 211 GPU-hours (Table 6).
3.3
TANGO Architecture and Training
Whole-body VLA navigation in complex environments demands physical world understanding, continuous action prediction, and robust action execution. To address these requirements, TANGO adopts a triple-system architecture [17, 44, 45] integrating a Vision-Language (VL) backbone (system-2), a multi-modal diffusion transformer (MM-DiT) action expert with real-time chunking [29] (system-1), and an off-the-shelf motion tracker (system-0), as shown in Figure 2. We jointly train the VL backbone and the action expert. During deployment, the predicted action chunks are streamed to the low-level tracker to generate continuous, high-frequency humanoid control signals. System-2: Vision-Language Perception. We instantiate system-2 using Qwen2.5VL-7B [46], warmstarted with InternVLA-N1 [10] weights to inherit strong navigation priors. We choose InternVLA-N1 for its open-source availability and navigation pretraining, while other VLA-based VLN backbones dn could also be adapted to this framework. At timestep 𝑡 , front and downward camera views (Ifr 𝑡 , I𝑡 ) are vertically stacked into a single frame I𝑡 . To manage long-horizon video history I1:𝑇 within a given memory capacity, we apply Budget-Aware Token Sampling (BATS) [7]. In the time step 𝑇 , all history frames are sampled independently into the navigation context according to the probability function 𝑃 ( 𝑡 ) = ( 1 − 𝜖) 𝑒𝑘 ( 𝑡 −𝑇 )/𝑇 + 𝜖, 𝑡 ∈ [ 1, 𝑇 ] , where 𝜖 and 𝑘 regulate temporal intensity. The visual features 𝑣𝑖 = VisionEncoder ( I𝑖 ) ∈ ℝ𝑛 × 𝑝 × 𝑐 are further spatial-grid pooled via 𝑣˜𝑖 = GridPool ( 𝑣𝑖 , 𝑔𝑖 ) ∈ ℝ𝑔𝑖 × 𝑐 [47], allocating finer grids to recent observations and coarser grids to history. Finally, the sampled visual tokens 𝑣˜sampled and language instruction ℓ are fed into the VLM to produce a latent context token 𝑧 = VLM ( 𝑣˜sampled , ℓ) . System-1 & System-0: Action Prediction and Execution. Conditioned on the latent 𝑧 and current proprioception q𝑑,𝑡 , the system-1 action expert predicts a future whole-body reference chunk. While standard actions are defined as joint angles and base poses a𝑖 = {q𝑑,𝑖 , r𝑏,𝑖 }, we formulate a stabilized training target to facilitate regression ã𝑖 = {q𝑑,𝑖 , r̃𝑏,𝑖 , Δ𝑥 𝑖 , Δ 𝑦𝑖 , Δ𝜓𝑖 }, where the base yaw in r̃𝑏,𝑖 is parameterized relative to the first frame of the chunk, and the auxiliary deltas (Δ𝑥 𝑖 , Δ 𝑦𝑖 , Δ𝜓𝑖 ) explicitly encode chunk-level planar displacement and heading changes. We implement system-1 using a flowbased MM-DiT [48] trained via flow-matching to generate the horizon Ã𝑡:𝑡+ 𝐻 . To align offline training with online streaming execution, we apply training-time RTC [29], conditioning the model on a randomized committed prefix of 𝑑 actions to inpaint the remaining horizon. The generated chunk is subsequently recovered to A𝑡:𝑡+ 𝐻 and streamed to the SONIC tracker (system-0), which tracks the reference against robot proprioception to provide high-frequency joint commands. Joint Training Objectives. Following dual-branch designs for VL decoding [7, 49], we append a textdecoding branch to System-2 and co-tune navigation tasks alongside VideoQA samples [50] to preserve generalized world knowledge. The joint optimization objective is defined as L = LCE + 𝑤FM · LFM , where LCE is the cross-entropy loss for VideoQA, LFM denotes the flow-matching loss, and 𝑤FM = 20. TANGO is trained end-to-end for a single epoch with a learning rate of 1 × 10 − 5 .
3.4
Deployment
We design the deployment system of TANGO on a Unitree G1 in both simulation and real world for evaluating the effectiveness and robustness of our system in both scenarios.
6
Client
Simulation Deployment. The task of wholebody navigation requires both high rendering quality and physical authenticity. To achieve both, we employ a digital twin teleportation system in simulation, where we leverage MuJoCo [51] for low-level tracker deployment and physical simulation, and teleport a humanoid digital twin in IsaacSim [41] to acquire photorealistic visual observation from designated camera pose solved from a given humanoid robot pose using forward kinematics (FK), and use it as VLA input.
Server
Front Cam
“Walk forward across the open lounge area...Turn right...Bend down to avoid ...Stop by the blue bin."
Downward Cam
VLA 2Hz Inference Action Chunk 30Hz Upsample
SONIC 200Hz
Action Chunk 50Hz
Real-World Deployment. We design a robust, Figure 3 | TANGO real-world deployment system. plug-and-play real-world deployment system for We adopt a server-client design to separately run lowTANGO. As shown in Figure 3, TANGO adopts frequency VLA model and high-frequency whole-body a cloud-edge deployment architecture that sepatracker. rates the compute-intensive VLA module from the high-frequency WBC module. The VLA (system-2 and system-1) runs on a cluster server equipped with an RTX PRO 6000, while the SONIC tracker (system-0) runs on an onboard Jetson Orin NX. The two systems are connected through standard IP networking. The humanoid captures front- and downward-facing RGB observations using RealSense D455 and D435i cameras and streams them, together with proprioceptive states, to the server with approximately 20ms latency. The server serves as a global clock and performs VLA inference every 0.5s, matching an execution horizon of 𝑠 = 15 actions at 30Hz. Predicted motion chunks are streamed back to the robot, resampled to 50Hz, and executed by the SONIC tracker, which closes the low-level control loop at approximately 200Hz. Additionally, we design a webpage-based control panel with a user-friendly interface for sending navigation instructions, monitoring VLA output and robot observation, and sending control signals to the VLA and WBC systems. Please refer to Appendix B for more details.
4
Experiments
To evaluate the effectiveness of our method, we conduct extensive experiments and ablation studies to answer three key questions: 1) Can TANGO perform well on VLN tasks compared to state-of-the-art baselines? 2) Can TANGO effectively learn to traverse through cluttered indoor environment without collision? and 3) Is the key design of our method effective?
4.1
VLN Performance
We evaluate TANGO on VLNVerse, a newly established VLN benchmark with photorealistic indoor scenes in IsaacSim. We compare against strong baselines in the benchmark, including discrete action models CMA and Seq2Seq [52], continuous action model RDP [27], neural implicit representation method HNR [53], and state-of-the-art VLA models InternVLA-N1 [10] and Uni-NaVid [47]. We evaluate all methods on the fine-grained validation splits. We report Success Rate (SR), Success weighted by Path Length (SPL), Navigation Error (NE), and Oracle Success Rate (OSR). To ensure fair comparison, all methods are trained on 3963 trajectories from VLNVerse-train and evaluated on 423 and 825 trajectories from VLNVerse-seen and VLNVerse-unseen, respectively. For metric calculation details, please refer to Appendix D. We fine-tune InternVLA-N1 and Uni-NaVid on VLNVerse-train for 7
Methods CMA [52] RDP [27] Seq2Seq [52] HNR [53] InternVLA-N1 [10] Uni-NaVid [47] Ours
Low-level Control ✗ ✗ ✗ ✗ ✗ ✗ ✓
Val Seen
Val Unseen
NE↓
OSR↑
SR↑
SPL↑
NE↓
OSR↑
SR↑
SPL↑
5.36 4.02 4.78 – 3.91 4.31 3.90
59.81 68.09 44.68 – 62.50 60.62 70.31
37.35 47.28 32.62 36.34 51.56 51.88 54.69
33.36 41.69 30.39 32.10 34.37 39.72 40.18
5.16 3.75 4.36 – 4.09 3.97 3.72
62.79 71.93 49.58 – 64.13 62.50 71.07
31.15 48.60 35.03 32.95 45.56 45.00 52.89
27.92 42.72 33.37 29.56 34.98 39.42 40.18
Table 1 | VLNVerse benchmark result. We evaluate TANGO on VLNVerse benchmark against strong baselines with different action representations. Among all methods, only TANGO is enabled with low-level physical control, while others are evaluated in a teleportation setting. A dash denotes an unreported metric. Bold and underlined values mark the best and second-best results, respectively.
five epochs following their original training settings. Results for CMA, Seq2Seq, RDP, and HNR are provided by the VLNVerse team. Since all baselines lack low-level control modules, we evaluate them in a teleportation setting following the original VLNVerse protocol. In contrast, TANGO is the only method equipped with low-level control and operates under realistic physical constraints. Nevertheless, as shown in Table 1, TANGO achieves the highest SR and the lowest reported NE on both VLNVerse-seen and VLNVerse-unseen, while achieving SPL and OSR comparable to the state-of-the-art RDP baseline. These results highlight the potential of whole-body navigation methods in cluttered indoor environment traversal. The relatively lower SPL suggests reduced navigation efficiency, likely due to the conservative behavior of the low-level tracker, which may favor safer but less direct trajectories around obstacles.
4.2
Cluttered Environment Traversal Performance
We evaluate our method on augmented VLNVerse-unseen scenes (Section 3.2) to verify its obstacleavoidance and spatial understanding capability in 3D environments. We compare our method with InternVLA-N1 in both zero-shot and fine-tuned settings, with two types of low-level executors: Unitree official RL controller [54] with a model predictive control (MPC) module, which executes basic movement according to planar velocity command; and HumanoidPF generalist policy [12] which performs obstacle-avoidance motions in complex 3D scenes. For the fine-tuned setting, we fine-tune InternVLAN1 in augmented VLNVerse-train scenes, with visual input collected from either Unitree-controller or HumanoidPF motions (denoted as “+Unitree WBC” and “+HumanoidPF”). During inference, Unitreecontroller takes in 2D velocity commands interpreted from predicted trajectory; while HumanoidPF tracks waypoints sampled 1.2m ahead on the trajectory, both at 50Hz. Note that while HumanoidPF leverages LiDAR as additional input for 3D scene information, our method uses pure RGB input. To better quantify navigation safety in cluttered environments, we introduce Collision Rate (CR) [36], defined as the percentage of evaluation episodes with at least one collision (Appendix D). As shown in Table 2, TANGO achieves the highest SR and SPL while maintaining the lowest CR across all methods. In detail, TANGO reduces CR from 15.81% to 9.90% compared to the strongest baseline, despite relying solely on RGB observations, whereas the HumanoidPF-tracked baseline additionally has access to LiDAR-based geometric perception. Meanwhile, TANGO improves SR by 1.87 percentage points and SPL by 2.34 points against the fine-tuned InternVLA-N1 + HumanoidPF baseline. These results support the benefit of end-to-end whole-body action generation over the evaluated modular approaches.
8
Methods InternVLA-N1 (zero-shot)† InternVLA-N1 (zero-shot)‡
Augmented Val Unseen NE↓ SR↑
SPL↑
CR↓
5.34 26.67 11.92 19.03 3.19 35.29 24.35 16.11
Method
4.01 43.75 31.83 9.90
Ours
† Equipped with Unitree low-level controller and MPC. ‡ Tracked by HumanoidPF generalist policy
Cluttered 3D Env.
SR↑ Coll.↓ SR↑ Coll.↓ SR↑ Coll.↓ InternVLA-N1† 11/15 Ours
InternVLA-N1 + Unitree WBC 4.57 35.00 17.38 17.49 InternVLA-N1 + HumanoidPF 4.04 41.88 29.49 15.81
Short-horizon Long-horizon 2D Env. 2D Env.
1.40
6/15
3.47
12/15 0.40
8/15
1.07 10/15 0.73
6/15
1.93
† Equipped with Unitree WBC.
Table 3 | Quantitative real-world navigation results. Three scenes with five trials each give 15 trials per method and setting. SR: successful trials out of 15; Coll.: mean collisions per trial.
Table 2 | Cluttered environment evaluation. We report performance comparison of TANGO on augmented VLNVerse-unseen against strong modular baselines.
4.3
Real-World Experiment
We conduct real-world experiments to evaluate whether TANGO can transfer from simulation to a physical humanoid platform without any real-world training. In particular, we focus on scenarios that require simultaneous language-guided navigation, obstacle avoidance, and whole-body motion adaptation, which jointly test the key capabilities targeted by our approach. Qualitative Experiments. We evaluate TANGO in four representative real-world scenarios: longhorizon navigation, side-stepping through a narrow pathway, bending down to avoid overhead obstacles, and stepping over obstacles on the ground. As shown in Figure 4, TANGO demonstrates robust scene understanding and spatial traversal capability in all cases. The robot executes continuous whole-body motions while following natural-language instructions, adapts its body configuration to negotiate obstacles, and maintains progress toward the navigation goal, demonstrating zero-shot sim-to-real transfer in cluttered physical environments. Quantitative Experiments. We compare TANGO with the fine-tuned InternVLA-N1 + Unitree WBC baseline in three settings: short-horizon 2D navigation (one turn, approximately 10 m), long-horizon 2D navigation (two or three turns, approximately 30 m), and cluttered 3D navigation with one challenging obstacle. Each setting contains three scenes with five trials per scene, giving 15 trials per method per setting. We report successful trials and mean collisions per trial in Table 3; the latter is a collision count, distinct from the episode-level CR used in simulation. The cluttered-scene instructions explicitly specify the required traversal behavior, such as stepping over, side-stepping, or bending down. As shown in the table, TANGO scores the highest SR across the three tested settings, with fewer collisions than the baseline. These results support improved task completion and safer traversal across the three tested settings.
4.4
Ablation Studies
Whole-body action representation. We study the impact of action representation and low-level execution on the original VLNVerse-unseen benchmark to validate our 29-DoF whole-body action space. We compare against InternVLA-N1 in a zero-shot setting and a planar variant of our method, denoted as Ours-2D. Since planar trajectory prediction is an auxiliary objective in our formulation (Section 3.3), Ours-2D isolates the effect of whole-body action generation. For both baselines, we evaluate under two execution settings: teleportation and physical execution using the Unitree low-level controller with MPC.
9
Long-Horizon Navigation
Prompt: "Walk forward through the open area, passing planters and benches on your left, and a sofa and tables on your right. Turn right when you reach the sofa on your right, passing an enclosed booth and working desks on your left, until you reach a wall with two doorways. With a grey sofa on your left, turn right into the open doorway. Walk straight down this next hallway, passing planters along windows on your left and cubicles with desks on your right. Stop next to the cabinet on your right."
Sidestep
Prompt: "Move to the blue chair, then turn left to face the chair. Side-step to your right towards the end of the pathway. Stop at the end of the pathway."
Sidestep
Prompt: "Turn right, then side-step to your left. At the end of the hallway, turn left. Walk towards the glass door, then stop."
Squat
Prompt: "Walk forward and duck under the black obstacle. Continue forward through the passage. Walk forward and turn left at the end of the passage with a white desk in front of you. Stop by the glass door."
Stride
Prompt: "Walk forward in the kitchen area, and step over the floor obstacle in your path. Turn left, and stop in front of the orange chair."
Figure 4 | Real-world deployment of TANGO. We show real-world qualitative results of TANGO in longhorizon navigation, side-stepping through narrow pathways, bending down to avoid overhead obstacles, and stepping over obstacles on the ground, demonstrating zero-shot sim-to-real transfer without any real-world training.
10
Method
Low-level control
Val Unseen SR↑
SPL↑
InternVLA-N1 Ours-2D
✗ ✗
43.69 45.74
35.74 35.44
InternVLA-N1† Ours-2D† Ours
✓ ✓ ✓
42.37 26.67 52.89
22.50 8.34 40.18
† Equipped with Unitree low-level controller and
MPC. Table 4 | Ablation studies on action space. We ablate the effect of action space and low-level control in VLNVerse-unseen.
Method
Augmented Val Unseen SR↑
SPL↑
CR↓
w/o RTC w/o Motion Editing SONIC → ScaleBFM
10.94 36.25 40.94
10.94 30.36 29.65
14.60 20.60 9.10
Ours
43.75
31.83
9.90
Table 5 | Ablation studies on key components. We evaluate RTC, motion editing, and the low-level WBC module on augmented VLNVerse-unseen. The default configuration uses SONIC; the tracker variant uses ScaleBFM. SR and CR are percentages; SPL is scaled by 100.
As shown in Table 4, both planar policies experience a substantial performance drop when moving from teleportation to physical execution, highlighting the difficulty of transferring conventional navigation policies to embodied settings. In contrast, TANGO consistently outperforms all baselines and remains robust under low-level control constraints. The comparison between Ours-2D and TANGO supports the benefit of whole-body action prediction. The zero-shot InternVLA-N1 rows provide additional reference points, but do not isolate action representation from training differences. Other Key Components. We separately evaluate the design choice of motion editing, RTC, and low-level tracker on augmented VLNVerse-unseen (Table 5). Removing RTC reduces SR from 43.75% to 10.94%, a drop of 32.81 percentage points, and increases CR from 9.90% to 14.60%. This result supports the importance of maintaining motion continuity across action chunks during execution. Removing motion editing reduces SR to 36.25% and increases CR to 20.60%, indicating that obstacleaware motion supervision contributes to collision avoidance. Replacing SONIC with ScaleBFM [55] yields 40.94% SR, 29.65 SPL, and 9.10% CR, with all three metrics within three points of the default configuration. This comparison suggests compatibility with another general whole-body controller.
5
Conclusion
This work introduces TANGO, to our best knowledge, the first whole-body vision-language navigation framework that directly predicts 29-DoF joint-space actions. We build up a diverse large-scale dataset covering not only indoor navigation patterns but also whole-body collision avoidance prior. Given language instruction, image observations and robot proprioception, we train a Qwen2.5VL-7B [46] based backbone with flow matching action expert. To match real-time execution on humanoid robot, training-time RTC [29] and high-frequency general tracker [2] are leveraged. Our experiment results indicate both state-of-the-art vision-language navigation capability and decent traversal performance in cluttered environments. Our ablations support the benefits of whole-body action prediction, obstacle-aware motion editing, and RTC, and show compatibility with an alternative low-level tracker. Overall, TANGO represents a meaningful step toward practical whole-body large planning models by demonstrating the feasibility of directly predicting 29-DoF actions. Building on TANGO as a foundation model, future work can further adapt and scale this framework toward more general loco-manipulation foundation models capable of tackling challenging tasks that require coordinated and active use of the robot’s entire body.
11
Limitation. Despite the promising results, the capability of low-level tracker shows up as a key constraint for further deployment in more complex environments, e.g. walking up stairs. Another limitation lies in our vision input, which relies solely on RGB images. This may restrict the model’s ability to fully understand complex scenes, particularly in cluttered, visually ambiguous, or low-light environments, where depth camera and LiDAR are expected to help. We leave these for future work to explore.
12
References [1] Zhaoyuan Gu, Junheng Li, Wenlan Shen, Wenhao Yu, Zhaoming Xie, Stephen McCrory, Xianyi Cheng, Abdulaziz Shamsah, Robert Griffin, C. Karen Liu, Abderrahmane Kheddar, Xue Bin Peng, Yuke Zhu, Guanya Shi, Quan Nguyen, Gordon Cheng, Huijun Gao, and Ye Zhao. Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning. IEEE/ASME Transactions on Mechatronics, 2026. [2] Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Sirui Chen, Fernando Castañeda, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, Xingye Da, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Zi Wang, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi "Jim" Fan, and Yuke Zhu. Sonic: Supersizing motion tracking for natural humanoid whole-body control. arXiv preprint arXiv:2511.07820, 2025. [3] Yanjie Ze, Siheng Zhao, Weizhuo Wang, Angjoo Kanazawa, Rocky Duan, Pieter Abbeel, Guanya Shi, Jiajun Wu, and C. Karen Liu. Twist2: Scalable, portable, and holistic humanoid data collection system. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [4] Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Yuman Gao, Guy Tevet, Koushil Sreenath, and C. Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion. arXiv preprint arXiv:2508.08241, 2025. [5] Arthur Allshire, Hongsuk Choi, Junyi Zhang, David McAllister, Anthony Zhang, Chung Min Kim, Trevor Darrell, Pieter Abbeel, Jitendra Malik, and Angjoo Kanazawa. Visual imitation enables contextual humanoid control. In Proceedings of the Conference on Robot Learning (CoRL), 2025. [6] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. [7] Jiazhao Zhang, Anqi Li, Yunpeng Qi, Minghan Li, Jiahang Liu, Shaoan Wang, Haoran Liu, Gengze Zhou, Yuze Wu, Xingxing Li, et al. Embodied navigation foundation model. In The 14th International Conference on Learning Representations, 2026. [8] Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and-language navigation. Robotics: Science and Systems, 2024. [9] Noriaki Hirose, Catherine Glossop, Ajay Sridhar, Dhruv Shah, Oier Mees, and Sergey Levine. Lelan: Learning a language-conditioned navigation policy from in-the-wild video. In Conference on Robot Learning, 2024. [10] Meng Wei, Chenyang Wan, Jiaqi Peng, Xiqian Yu, Yuqiang Yang, Delin Feng, Wenzhe Cai, Chenming Zhu, Tai Wang, Jiangmiao Pang, and Xihui Liu. Ground slow, move fast: A dual-system foundation model for generalizable vision-language navigation. In The 14th International Conference on Learning Representations, 2026. [11] An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıyık, Hongxu Yin, Sifei Liu, and Xiaolong Wang. Navila: Legged robot vision-language-action model for navigation. In Robotics: Science and Systems, 2025. [12] Han Xue, Sikai Liang, Zhikai Zhang, Zicheng Zeng, Yun Liu, Yunrui Lian, Jilong Wang, Qingtao Liu, Xuesong Shi, and Yi Li. Collision-free humanoid traversal in cluttered indoor scenes. arXiv preprint arXiv:2601.16035, 2026. [13] Tianyu Xu, Jiawei Chen, Jiazhao Zhang, Wenyao Zhang, Zekun Qi, Minghan Li, Zhizheng Zhang, and He Wang. Mm-nav: Multi-view vla model for robust visual navigation via multi-expert learning. arXiv preprint arXiv:2510.03142, 2025.
13
[14] Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation. In Conference on Robot Learning (CoRL), 2023. [15] Dhruv Shah, Blazej Osinski, Brian Ichter, and Sergey Levine. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In Conference on Robot Learning (CoRL), 2023. [16] Noriaki Hirose, Catherine Glossop, Dhruv Shah, and Sergey Levine. Omnivla: An omni-modal visionlanguage-action model for robot navigation. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [17] Songlin Wei, Hongyi Jing, Boqian Li, Zhenyu Zhao, Jiageng Mao, Zhenhao Ni, Sicheng He, Jie Liu, Xiawei Liu, Kaidi Kang, Sheng Zang, Weiduo Yuan, Marco Pavone, Di Huang, and Yue Wang. 𝜓0 : An open foundation model towards universal humanoid loco-manipulation. Robotics: Science and Systems, 2026. [18] GEAR Team. GR00T N1.6: An improved open foundation model for generalist humanoid robots, December 2025. NVIDIA Research Blog. [19] Haoran Jiang, Jin Chen, Qingwen Bu, Li Chen, Modi Shi, Yanjie Zhang, Delong Li, Chuanzhe Suo, Chuang Wang, Zhihui Peng, and Hongyang Li. Wholebodyvla: Towards unified latent vla for whole-body loco-manipulation control. In The 14th International Conference on Learning Representations, 2026. [20] Pengxiang Ding, Jianfei Ma, Xinyang Tong, Binghong Zou, Xinxin Luo, Yiguo Fan, Ting Wang, Hongchao Lu, Panzhong Mo, Jinxin Liu, Yuefan Wang, Huaicheng Zhou, Wenshuo Feng, Jiacheng Liu, Siteng Huang, and Donglin Wang. Humanoid-vla: Towards universal humanoid control with visual integration. arXiv preprint arXiv:2502.14795, 2025. [21] Zhen Wu, Xiaoyu Huang, Lujie Yang, Yuanhang Zhang, Xi Chen, Pieter Abbeel, Rocky Duan, Angjoo Kanazawa, Carmelo Sferrazza, Guanya Shi, and C. Karen Liu. Perceptive humanoid parkour: Chaining dynamic human skills via motion matching. Robotics: Science and Systems, 2026. [22] Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. In The AAAI Conference on Artificial Intelligence, 2024. [23] Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, and Qi Wu. Navgpt-2: Unleashing navigational reasoning capability for large vision-language models. In European Conference on Computer Vision (ECCV), 2024. [24] Bhargav Chandaka, Gloria X. Wang, Haozhe Chen, Henry Che, Albert J. Zhai, and Shenlong Wang. Human-like navigation in a world built for humans. In Conference on Robot Learning (CoRL), 2025. [25] Navid Rajabi and Jana Kosecka. Travel: Training-free retrieval and alignment for vision-and-language navigation. arXiv preprint arXiv:2502.07306, 2025. [26] Anqi Li, Zhiyong Wang, Jiazhao Zhang, Minghan Li, Yunpeng Qi, Zhibo Chen, Zhizheng Zhang, and He Wang. Urbanvla: A vision-language-action model for urban micromobility. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [27] Liuyi Wang, Xinyuan Xia, Hui Zhao, Hanqing Wang, Tai Wang, Yilun Chen, Chengju Liu, Qijun Chen, and Jiangmiao Pang. Rethinking the embodied gap in vision-and-language navigation: A holistic study of physical and visual disparities. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025. [28] Noriaki Hirose, Catherine Glossop, Dhruv Shah, and Sergey Levine. Asyncvla: An asynchronous vla for fast and robust navigation on the edge. arXiv preprint arXiv:2602.13476, 2026. [29] Kevin Black, Allen Z. Ren, Michael Equi, and Sergey Levine. Training-time action conditioning for efficient real-time chunking. arXiv preprint arXiv:2512.05964, 2025. [30] Ziwen Zhuang, Shenzhe Yao, and Hang Zhao. Humanoid parkour learning. In Conference on Robot Learning (CoRL), 2024.
14
[31] Yeke Chen, Ji Ma, Zeren Luo, Yimin Han, Yinzhao Dong, Bowen Xu, and Peng Lu. Learning autonomous and safe quadruped traversal of complex terrains using multi-layer elevation maps. IEEE Robotics and Automation Letters, 10(10):9606–9613, 2025. [32] Jiahang Liu, Tianyu Xu, Jiawei Chen, Lu Yue, Jiazhao Zhang, Zhiyong Wang, Minghan Li, Qisheng Zhao, Anqi Li, Qi Su, Zhizheng Zhang, and He Wang. Span-nav: Generalized spatial awareness for versatile vision-language navigation. arXiv preprint arXiv:2603.09163, 2026. [33] Jialong Li, Xuxin Cheng, Tianshu Huang, Shiqi Yang, Ri-Zhao Qiu, and Xiaolong Wang. Amo: Adaptive motion optimization for hyper-dexterous humanoid whole-body control. Robotics: Science and Systems, 2025. [34] Haoru Xue, Xiaoyu Huang, Dantong Niu, Qiayuan Liao, Thomas Kragerud, Jan Tommy Gravdahl, Xue Bin Peng, Guanya Shi, Trevor Darrell, Koushil Sreenath, and Shankar Sastry. Leverb: Humanoid whole-body control with latent vision-language instruction. arXiv preprint arXiv:2506.13751, 2025. [35] Weikai Qin, Sichen Wu, Ci Chen, Mengfan Liu, Linxi Feng, Xinru Cui, Haoqi Han, and Hesheng Wang. Physiflow: Physics-aware humanoid whole-body vla via multi-brain latent flow matching and robust tracking. arXiv preprint arXiv:2603.05410, 2026. [36] Sihao Lin, Zerui Li, Xunyi Zhao, Gengze Zhou, Liuyi Wang, Rong Wei, Rui Tang, Juncheng Li, Hanqing Wang, Jiangmiao Pang, Anton van den Hengel, Jiajun Liu, and Qi Wu. Vlnverse: A benchmark for vision-language navigation with versatile, embodied, realistic simulation and evaluation. arXiv preprint arXiv:2512.19021, 2025. [37] Bingchen Miao, Rong Wei, Zhiqi Ge, Xiaoquan sun, Shiqi Gao, Jingzhe Zhu, Renhan Wang, Siliang Tang, Jun Xiao, Rui Tang, and Juncheng Li. Towards physically executable 3d gaussian for embodied navigation. In The 14th International Conference on Learning Representations, 2026. [38] Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning human-to-humanoid real-time whole-body teleoperation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8944–8951, 2024. [39] Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. HumanPlus: Humanoid shadowing and imitation from humans. In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard, editors, Proceedings of the 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, pages 2828–2844. PMLR, 2025. [40] Yanjie Ze, Zixuan Chen, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu, and Karen Liu. TWIST: Teleoperated whole-body imitation system. In Joseph Lim, Shuran Song, and Hae-Won Park, editors, Proceedings of the 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pages 2143–2154. PMLR, 2025. [41] NVIDIA. Isaac sim, 2025. [42] Gheorghe Comanici et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [43] Gabriel B. Margolis, Michelle Wang, Nolan Fey, and Pulkit Agrawal. Softmimic: Learning compliant whole-body control from examples. arXiv preprint arXiv:2510.17792, 2025. [44] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. 𝜋0.5 : a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. [45] NVIDIA, :, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil
15
Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jiannan Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025. [46] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. [47] Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang, Zhizheng Zhang, and He Wang. Uni-navid: A video-based vision-language-action model for unifying embodied navigation tasks. Robotics: Science and Systems, 2025. [48] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206, 2024. [49] Shaoan Wang, Jiazhao Zhang, Minghan Li, Jiahang Liu, Anqi Li, Kui Wu, Fangwei Zhong, Junzhi Yu, Zhizheng Zhang, and He Wang. Trackvla: Embodied visual tracking in the wild. In Conference on Robot Learning (CoRL), 2025. [50] Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed Elhoseiny, and Vikas Chandra. Longvu: Spatiotemporal adaptive compression for long video-language understanding. In The 42nd International Conference on Machine Learning, 2025. [51] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033, 2012. [52] Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-language navigation in continuous environments. arXiv preprint arXiv:2004.02857, 2020. [53] Zihan Wang, Xiangyang Li, Jiahao Yang, Yeqi Liu, Junjie Hu, Ming Jiang, and Shuqiang Jiang. Lookahead exploration with neural radiance representation for continuous vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13753–13762, June 2024. [54] Unitree Robotics. Unitree RL Gym: Reinforcement learning implementation for unitree robots. https://github.com/unitreerobotics/unitree_rl_gym, 2025. Accessed: 2026-05-29; commit 276801e46c5d433564f24658bac64f254b7d2d4b. [55] Weishuai Zeng, Kangning Yin, Xiaojie Niu, Shunlin Lu, Weixiang Zhong, Jiahe Chen, Feiyu Jia, Xiao Chen, Zirui Wang, Furui Xu, Ming Zhou, Kailin Li, Weinan Zhang, He Wang, Li Yi, Dahua Lin, Jiangmiao Pang, and Jingbo Wang. Scaling behavior foundation model for humanoid robots. arXiv preprint arXiv:2607.15163, 2026.
16
Contents 1 Introduction
1
2 Related Works
3
3 Method 3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Simulation Data Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 TANGO Architecture and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 4 4 6 6
4 Experiments 4.1 VLN Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Cluttered Environment Traversal Performance . . . . . . . . . . . . . . . . . . . . . 4.3 Real-World Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Ablation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7 7 8 9 9
5 Conclusion
11
A Training Details
18
B Deployment Details
18
C Environment Augmentation and Motion Generation Details 18 C.1 Environment Augmentation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 C.2 Motion Generation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 D Experiment Metrics
23
17
Appendix A
Training Details
For our training procedure, we emphasize rare whole-body behaviors through both data-level rebalancing and loss-level weighting. After converting rollouts into action chunks, we rebalance the training set by upsampling behaviorally important rows, including large turns, sideways motion, squat, stride, and mixed squat-stride segments; and especially, stride and sideways motions are boosted most strongly to compensate for their lower frequency and weaker action signal. We further apply motion-conditioned per-dimension loss weights on the action vector, assigning higher weights to the joint-position dimensions most associated with each behavior, such as hip, spine, and knee joints for squat, spine and shoulder joints for stride, their union for mixed motions, and shoulder plus body-frame motion dimensions for sideways motion. The model is trained with full-parameter tuning on 16 nodes of 8 × NVIDIA A100 GPUs for approximately 7 hours, totaling 896 A100 GPU hours.
B
Deployment Details
The real-world system runs the VLA on an RTX PRO 6000 server and the SONIC tracker on the onboard Jetson Orin NX. Front and downward RGB observations from the RealSense D455 and D435i, together with proprioception, are transmitted to the server with approximately 20 ms latency. The server performs VLA inference every 0.5 s. Each execution segment contains 15 actions at 30 Hz, which are resampled to 50 Hz before tracking; the low-level control loop runs at approximately 200 Hz. A web-based control panel provides navigation-instruction entry, observation and predicted-motion monitoring, and control signals to the VLA and WBC systems.
C
Environment Augmentation and Motion Generation Details
C.1
Environment Augmentation Details
To better evaluate whole-body planning, we augment VLNVerse and SAGE-3D scenes with trajectoryconditioned obstacles rather than randomly sampled clutter. Given an original scene, its A* navigation trajectories, and the corresponding occupancy map extracted from the scene, we insert obstacles along the paths at the locations where whole-body behaviors are likely required. We consider three targeted interaction types: stride, where a low obstacle is placed on the path to encourage stepping over; sidle, where a pair of side obstacles forms a narrow passage; and squat, where an overhead or torso-height obstacle encourages ducking or lowering the body. For each scene, candidate placements are sampled along valid A* trajectories while avoiding the start and goal regions. Each candidate is aligned with the local path direction so that the inserted obstacle naturally interacts with the robot’s intended route. The obstacle assets are selected according to the target behavior: low objects such as rugs, cushions, or stools for stride; chairs, plants, shelves, or similar objects for sidle; and ceiling lights, lamps, curtains, or wall-mounted objects for squat. This design makes the augmented scenes semantically plausible while explicitly inducing whole-body navigation challenges. To preserve scene validity, each placement is checked against the occupancy map and all available trajectories in the same scene. In particular, obstacles are prevented from blocking unrelated paths or appearing too close to trajectory endpoints, while the source trajectory is allowed to be affected by
18
Stride Sidle Squat Figure 5 | Examples of targeted environment augmentation. Examples of augmented VLNVerse scenes with three obstacle types: low obstacles for stepping over, side obstacles for sidling through narrow passages, and overhead obstacles for squatting.
the inserted obstacle. After placement, the augmented obstacles are also written into the occupancy representation so that downstream planners observe the same geometry as the policy. From the original 263 VLNVerse and 1,000 SAGE-3D scenes, Gemini 2.5 Flash filtering removes low-quality scenes, such as those with uneven ground or incomplete geometry. We retain 205 VLNVerse and 373 SAGE-3D scenes. Across the augmented scenes, the proportions of stride, sidle, and squat obstacles are 44%, 15%, and 41%, respectively. Obstacle poses are randomly sampled within traversable ranges rather than fixed for each category. Please refer to Figure 5 for a visualization of the augmented scenes. To better align instructions and action labels to facilitate training, we further augment the language prompts from VLNVerse with the knowledge of added obstacles and their corresponding target behavior types via Gemini 2.5 Flash [42]. In practice, including collision-avoidance prompts helps convergence and the learning of obstacle-avoidance behaviors. In the RGB-only real-world setting, these prompts provide complementary cues to visible obstacles. This is an empirical observation, rather than a controlled language ablation, and the reported cluttered real-world trials use behavior-explicit instructions.
19
Dataset
GPU-hours
Trajectories PET
Rendering
Total
VLNVerse (original) VLNVerse (augmented) SAGE-3D (original) SAGE-3D (augmented)
3,478 3,784 28,678 28,693
3 6 20 57
8 17 45 55
11 23 65 112
Total
64,633
86
125
211
Table 6 | Dataset scale and generation cost. Costs are measured in RTX PRO 6000 GPU-hours and cover PET motion generation and rendering for the original and augmented scenes.
Dataset scale and generation cost. Table 6 reports the trajectory counts and GPU-hours for the original and augmented versions of both datasets. Generating 64,633 trajectories requires 86 GPUhours for PET and 125 GPU-hours for rendering, totaling 211 RTX PRO 6000 GPU-hours. These costs cover trajectory generation and rendering, not VLA training.
C.2
Motion Generation Details
This section describes the Plan, Edit, Track (PET) pipeline used to generate the trajectories that supervise TANGO. Given an augmented indoor scene and a start–goal pair, PET produces a 29-DoF whole-body reference motion, then re-renders egocentric observations and regenerates language instructions. The pipeline consists of three stages: Plan, Edit (which contains the motion editing deferred from the main text), and Track. Plan: Path Planning and Reference-Gait Synthesis Given the start 𝑠, the goal 𝑔 , and the scene occupancy map, which already includes the augmented obstacles , we plan a planar path with A★ on the 2D floor grid. To favor safer routes with whole-body clearance, we add an obstacle-aware term to the step cost. This term is computed from the 2D signed distance to occupied cells, Φ2D ( x) : 𝑐 ( x) = 𝑐step + 𝜆 exp − Φ2D ( x)/𝑑0 ,
(1)
so cells near obstacles receive a soft penalty that decays with clearance 𝑑0 , while the admissible Euclidean-to-goal heuristic preserves optimality under this cost. The output is a planar polyline P = {( 𝑥 𝑘 , 𝑦𝑘 )} 𝑘𝐾=1 with a tangent heading 𝜓𝑘 = atan2 ( 𝑦𝑘+1 − 𝑦𝑘 , 𝑥 𝑘+1 − 𝑥 𝑘 ) . To enable side walking through narrow passages, we used a rule-based yaw rewriter. Along P , we detect narrow passages by probing the lateral free width 𝑤⊥ ( 𝑠) on both sides of the path at arc length 𝑠. When 𝑤⊥ ( 𝑠) falls below a threshold 𝑤min , the body heading is rewritten to be 90◦ offset from the body travel tangent, 𝜓𝑘 = 𝜓𝑘 ± 𝜋2 , with the sign chosen so that the leading shoulder remains on the wider ◦ side. Smooth ±90 transitions into and out of the passage produce a crab-walking, or side-stepping, gait in which the direction of translation and the facing direction are decoupled. body
The rewritten planar trajectory {( 𝑥 𝑘 , 𝑦𝑘 , 𝜓𝑘 )} is converted into a stream of base velocity commands u𝑘 = ( 𝑣𝑥 , 𝑣 𝑦 , 𝜔 ) 𝑘 , where the forward and lateral linear velocities and yaw rates are obtained by finite-differencing the path under the rewritten heading. These commands are fed to SONIC [2], a motion-tracking model trained on ∼800 hours of high-quality motion-capture data, which produces a natural whole-body gait. A PD controller corrects drift between the realized base pose and P , 𝑇 ref 29 yielding a dynamically feasible reference walk M ref = {qref joint angles plus the 𝑡 } 𝑡 =1 with q𝑡 ∈ ℝ floating-base pose. This gait is smooth, but above the floor it remains obstacle-agnostic: it follows the planar route and side-walks through narrow passages, but it does not yet step over, duck under, 20
or otherwise adapt the body to 3D obstacle geometry. These adaptations are introduced in the Edit stage. Edit: Whole-Body Obstacle-Avoidance Editing The Edit stage transforms the reference motion M ref into a whole-body obstacle-avoidance motion. The reference gait is kept as a locomotion prior: its timing, gait phase, and motion style are preserved, and edits are applied locally only where the path corridor contains obstacles. At each frame, we solve a whole-body inverse-kinematics (IK) problem with four objectives: reference posture tracking, foot contact and landing targets, a center-of-mass (CoM) target, and potential-field (PF) link forces. These objectives cover the three behavior families introduced above. A potential-field crouch controller handles squat obstacles, a gait-adaptation module handles stride obstacles, and crab-walking from the Plan stage, together with lateral PF guidance, handles sidle passages. The scene voxel occupancy is converted into a 3D signed distance field Φ ( p) using the fast marching method, which provides distance values and gradients ∇ Φ throughout the workspace. Thin structural obstacles, such as bars, are also represented as exact oriented boxes with an analytic SDF, so that clearance queries remain continuous. Editing is restricted to a spatial corridor, defined as a tube of half-width 𝑤corr (≈ 0.5 m) around the planned path. Every field sample whose horizontal position lies outside the tube is treated as free space,
( Φ̃ ( p) =
Φ ( p) ,
dist𝑥 𝑦 ( p, P) ≤ 𝑤corr ,
+∞,
otherwise,
(2)
and the guidance vector outside the tube is zeroed. As a result, the editor ignores clutter that A★ has already routed around, and only on-path obstacles that the robot must negotiate affect the edit. Following HumanoidPF [12], we use a sampled repulsive field for avoidance and pass this guidance to the IK through SoftMimic-style pseudo-forces [43]. Instead of imposing hard collision constraints, each force is converted into a bounded soft displacement target that the whole-body IK balances against posture and stability terms. We compute a guidance field g ( p) by combining the path tangent ˆ , which encourages forward progress, with the SDF repulsion ∇ Φ̃, which encourages obstacle 𝝉 avoidance. The weights are chosen so that repulsion dominates near obstacles: ˆ ( p) + 𝛽 𝑒 − Φ̃ ( p )/𝜎 g ( p) = 𝝉
∇ Φ̃ ( p) . ∥∇Φ̃ ( p) ∥
(3)
We apply forces to a set of key body links L , including the shoulders, elbows, wrists, torso, and optionally the knees and pelvis. Feet and ankles are excluded so that the contact pattern is still determined by the reference gait and the gait-adaptation module. For each link ℓ ∈ L at world position pℓ , we sample a repulsive force fℓ = 𝐹max g ( pℓ ) and convert it into a displacement under a link stiffness 𝜅, clipped to a per-link cap 𝛿max :
Δpℓ = clip 𝜅1 fℓ , 𝛿max ,
f̃ℓ = 𝜅 Δpℓ .
(4)
The bounded pseudo-forces {f̃ℓ } are added as soft tasks to the whole-body IK and are temporally low-pass filtered to remove discontinuities caused by field changes at gait transitions. For squat obstacles, we keep only the vertical component of each force, so the field provides a body-lowering cue for ducking rather than a lateral push. For sidle passages, the horizontal components push the trunk and arms away from the side walls in coordination with the crab-walking heading. The per-link forces can also be aggregated into a common CoM offset, with only the residual deviation applied to 21
each link, so that the body moves away from obstacles as a coherent whole rather than having each limb react independently. Two additional terms make ducking under squat obstacles start early and remain stable. The lookahead term addresses the fact that the raw field can stay near zero until the body is already under a ceiling. We probe Φ̃ ahead of the upper body at several head and shoulder heights and at several forward distances. If the minimum probe lies within a margin 𝜂 of the geometry, a downward force proportional to the violation is added, so the body begins to lower before reaching the obstacle. The virtual head barrier protects a point phead = ptorso + ℎˆ𝑧 with clearance 𝑏 = Φ̃ ( phead ) − 𝑟 . When 𝑏 < 𝑏𝑚 , it adds a strictly downward force 𝑘𝑠 ( 𝑏𝑚 − 𝑏) + 𝑘ℎ max ( 0, −𝑏) , corresponding to soft and hard barrier components, clipped to a maximum trunk displacement. The resulting downward displacement of the upper body is converted into a coordinated squat by coupling the crouch activation 𝛼 = clip ( Δ𝑧 /𝛿max , 0, 1) to posture: a forward waist-pitch target 𝜃waist ← 𝜃waist + 𝛼 𝜃0 , a proportional hip-pitch bias, and a downward shift of the CoM target Δ𝑧CoM = 𝛼 𝑐 𝑧 . This CoM shift lets the legs lower the body rather than letting the CoM task pull the body back upright. The implementation supports three coordinated presets: waist-only, which bends at the waist while the tracker controls the legs; com-drop, which lowers the body vertically; and full-squat, which combines waist motion, pelvis tilt, hip flexion, knee bending, ankle dorsiflexion, and CoM lowering. During an active crouch, the upper-body reference-tracking terms are relaxed so that the field can reshape the trunk instead of competing with the upright reference. For stride obstacles, upper-body guidance is not enough because the swing feet must step over the object. The gait-adaptation module retargets footsteps online while preserving the phase and timing of the reference gait. At each foot lift-off, the base is projected onto the planned path, a short arc-length look-ahead is queried, and the landing target is specified in the path-yaw frame. This target is clamped to a maximum displacement from the reference touchdown, which prevents the IK solver from pursuing unreachable foot placements. A footprint-aware SDF scan detects a low obstacle ahead of the swing foot and returns the interval [ 𝑑near , 𝑑far ] spanned by the obstacle, together with its top height 𝑧top . We classify obstacles by height: only 𝑧top ≤ 𝑧bar , corresponding to a steppable bar, triggers a step-over motion. Taller geometry is treated as a wall or ceiling and is handled by the lateral and upper-body layers, so the robot does not attempt to step over a wall. For a steppable bar, the landing target is pushed past the far edge by a margin, 𝑑 land = 𝑑far + 𝑚. The landing objective is evaluated with respect to this target rather than the reference touchdown on the bar. This both clears the obstacle and compensates for the forward-reach undershoot of the downstream tracker, keeping the realized landing past the bar. The swing trajectory is formed by warping the reference foot trajectory from motion capture or RL data onto the retargeted endpoints and adding a vertical clearance arc. This keeps the natural swing shape and leaves ordinary steps unchanged. With swing phase 𝑢 ∈ [ 0, 1] , after smoothstep interpolation to 𝑢¯, and reference foot path pref ( 𝑢) , pswing ( 𝑢) = pref ( 𝑢) + ( 1 − 𝑢¯) Δplift + 𝑢¯ Δpland + ℎarc 𝜌 ( 𝑢) ˆ𝑧 ,
(5)
where Δplift , Δpland are the offsets between the retargeted and reference endpoints, and 𝜌 ( 𝑢) is an asymmetric clearance profile with a fast quarter-sine rise to an early peak followed by a quarter-cosine fall. The asymmetric profile keeps the foot high early in the swing, which is needed for the trailing leg that crosses the bar shortly after lift-off. A symmetric sin ( 𝜋𝑢) arc would be too low at this stage. The peak height ℎarc is solved for each step so that the full foot footprint clears the obstacle box at every sampled phase, 𝑧ground + ℎarc 𝜌 ( 𝑢) ≥ 𝑧top ( 𝑢) + 𝜖,
∀ 𝑢 : footprint ( 𝑢) ∩ obstacle ≠ ∅, 22
(6)
using the same profile 𝜌 in both the solver and the executed arc, so that the planned and executed motions remain consistent. A swing-knee bend bias encourages knee flexion rather than straight-leg extension for obstacle clearance. When several low obstacles appear in sequence, a short foothold sequence places natural-length steps through the gaps instead of requiring a single long step. Because the foot retargeter moves the support polygon while the reference pelvis still follows the original gait, the body can lag behind the planned support. During a forward step-over, this mismatch can make the body lean in place rather than translate forward. Two small offsets, smoothed with EMA, correct this effect: root redirection shifts the pelvis reference toward the planned support, and CoM redirection shifts the IK CoM target accordingly. During a bar crossing, these offsets switch to a low-latency update rate so that the base translates with the crossing leg. This produces a forward step-over rather than an in-place lunge, with a ramped transition to avoid abrupt motion. Each frame is solved in two passes. The first pass performs a lower-body projection that edits the reference to match the foot, support, and CoM targets. The second pass performs full compliant whole-body IK, which tracks the edited reference while applying the PF link forces. Strong foot-anchor costs are used so that the feet behave as contacts and posture edits do not induce foot sliding. The output is the edited 29-DoF whole-body trajectory M edit . Track: Feasibility Restoration and Collision Filtering The edited trajectory M edit is kinematic and may violate dynamic feasibility, for example through overly fast swings or aggressive crouches. We restore feasibility by passing it through the SONIC tracker [2] in closed loop. The edited wholebody reference, including joint positions, joint velocities, and base orientation, is streamed frame by frame to the RL tracking policy, which tracks it under physics in MuJoCo and outputs a dynamically feasible motion M track . The same tracker is used at deployment (system-0), making the synthesized supervision consistent with execution. The known forward undershoot of the tracker is also why the Edit stage uses generous step-over landing margins.
D
Experiment Metrics
We evaluate navigation performance with standard vision-language navigation (VLN) metrics, including Success Rate (SR), Success weighted by Path Length (SPL), Navigation Error (NE), and Oracle Success Rate (OSR). For cluttered-scene traversal, we additionally report Collision Rate (CR) to quantify physical safety. Let E denote the evaluation set and |E | its number of episodes. For each episode 𝑒 ∈ E , the policy receives a language instruction ℓ𝑒 and sequential observations as defined in Section 3.1, and executes whole-body action chunks through the low-level tracker. The resulting robot-base trajectory is denoted as Γ𝑒 = ( x𝑒,0 , x𝑒,1 , . . . , x𝑒,𝐾𝑒 ) , (7) where x𝑒,0 is the start position, x𝑒,𝐾𝑒 is the final stopping position, and 𝐾𝑒 is the number of sampled trajectory points in episode 𝑒. Let g𝑒 denote the target goal position, and let 𝑑 G (·, ·) denote geodesic distance in the navigation environment. We use 𝛿 as the success threshold, set to 3 meters following standard VLN evaluation. We further denote the executed path length as 𝑃𝑒 =
𝐾𝑒 ∑︁
x𝑒,𝑘 − x𝑒,𝑘 − 1 2 ,
(8)
𝑘=1
and the shortest-path distance from the start position to the goal as 𝐿𝑒 = 𝑑 G ( x𝑒,0 , g𝑒 ) .
23
(9)
Navigation Error (NE). Navigation Error measures the average geodesic distance between the final stopping position and the target goal: NE =
1 ∑︁
|E | 𝑒 ∈ E
𝑑 G ( x𝑒,𝐾𝑒 , g𝑒 ) .
(10)
Lower NE indicates that the agent stops closer to the target goal. Success Rate (SR). An episode is considered successful if the final stopping position is within the success threshold 𝛿 of the target goal. The success indicator is
𝑆𝑒 = 1 𝑑 G ( x𝑒,𝐾𝑒 , g𝑒 ) ≤ 𝛿 ,
(11)
and SR is computed as SR =
1 ∑︁
|E | 𝑒 ∈ E
𝑆𝑒 .
(12)
In simulation tables, SR is reported as a percentage. In Table 3, SR is shown as successful trials out of 15. Oracle Success Rate (OSR). Following common VLN evaluation, Oracle Success Rate measures whether the executed trajectory ever enters the goal region, regardless of the final stopping position. The oracle success indicator is min 𝑑 G ( x𝑒,𝑘 , g𝑒 ) ≤ 𝛿 ,
𝑂𝑒 = 1
0 ≤ 𝑘 ≤ 𝐾𝑒
(13)
and OSR is computed as OSR =
1 ∑︁
|E | 𝑒 ∈ E
𝑂𝑒 .
(14)
OSR therefore evaluates whether the trajectory reaches the goal neighborhood at least once, rather than whether the agent stops there. Success weighted by Path Length (SPL).
SPL jointly measures task completion and path efficiency:
SPL =
1 ∑︁
|E | 𝑒 ∈ E
𝑆𝑒
𝐿𝑒 . max ( 𝑃𝑒 , 𝐿𝑒 )
(15)
Failed episodes contribute zero to SPL, while successful but unnecessarily long trajectories are penalized by the path-length ratio. Collision Rate (CR). For cluttered-scene traversal, we report Collision Rate as an episode-level safety metric. Let 𝐶 𝑒 = 1 [ Γ𝑒 results in at least one collision] . (16) CR is then defined as CR =
1 ∑︁
|E | 𝑒 ∈ E
𝐶𝑒 .
(17)
Lower CR indicates safer whole-body traversal in cluttered environments. In our tables, CR is reported as a percentage. Mean collisions in real-world trials. For each real-world setting, let 𝑛𝑒 be the number of collisions 1 Í15 recorded in trial 𝑒. We report Coll. = 15 𝑒=1 𝑛𝑒 . Unlike CR, this metric counts multiple collisions within a trial and is not a percentage. 24