arXiv CSopen access

LIME: Learning Intent-aware Camera Motion from Egocentric Video

LIME: Learning Intent-aware Camera Motion from Egocentric Video

arXiv:2607.02417v1 [cs.RO] 2 Jul 2026

Boyang Sun1,∗ Jiajie Li1,∗ Yung-Hsu Yang1 Chenyangguang Zhang1 Tim Engelbracht1 Sunghwan Hong1 Cesar Cadena1 Marc Pollefeys1,2 Hermann Blum3 1 ETH Zurich 2 Microsoft 3 University of Bonn *Equal contribution. Project page: https://boysun045.github.io/LIME-Page/

Figure 1: LIME learns intent-aware camera motion from passive human video and transfers it to robots: given the current view and a natural-language intent, it generates relative target camera poses that acquire intent-relevant visual evidence. Abstract: Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user’s intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. The challenge is that useful viewpoint changes depend on latent perceptual intent, ranging from coarse spatial moves to fine-grained inspection or occlusion revealing. To model this structure, we mine multi-intent camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an autoregressive observation-gain output with a continuous flowmatching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across viewpointprediction experiments and downstream robotic tasks, we show that LIME learns intent-conditioned camera motion from passive egocentric video, turning ordinary human recordings into supervision for a reusable active-perception primitive that supports manipulation, embodied question answering, and multi-step robot behaviors.

1

Introduction

Vision is often treated as an input to action: an agent observes the scene, recognizes what matters, and then decides where to move or what to manipulate [1]. In everyday behavior, however, this

dependence also runs in the opposite direction: an intention often causes us to move our sensors before acting, so that the next observation contains the information we need [2]. We lean to see behind an occluder, step closer to inspect a small detail, look around a corner before entering, or shift viewpoint to disambiguate an object’s shape. These motions are not merely navigation or manipulation side effects; they are camera-motion actions whose purpose is to acquire intent-relevant visual evidence. Because most modern mobile robots carry onboard cameras, learning this mapping from language intent to useful viewpoint change is a natural component of general embodied intelligence. This motivates the problem of language-conditioned camera motion generation: given the current view and an intent, predict how the camera should move to obtain a more useful next observation. Active perception has long studied sensor motion for better observations [2, 3, 4], with most formulations optimizing task-specific utilities such as exploration [5], reconstruction [6], object search [7]. Recent language-conditioned embodied models broaden robot behavior. In vision-language navigation, instructions specify routes, destinations, or landmarks, and camera viewpoint changes occur as a consequence of moving through the scene [8, 9]. In vision-language-action manipulation, visual observations condition end-effector control, while camera motion is coupled to the embodiment or execution policy [10, 11]. These settings leave open a different interface: not “where should the robot navigate to?” or “what should the arm and gripper do?”, but “how should the camera move so the next observation better resolves a given intent?” A central difficulty is that the desired next view depends on the intent the agent is trying to resolve, not only on the visible scene geometry. Given the same observation, an agent may need to move differently depending on whether the intent is to inspect an object, reveal an occluded region, enter a room, or prepare for a downstream interaction. Furthermore, for the same intent, multiple relative target poses may be valid because different viewpoints can expose different but similarly useful evidence. Conversely, the same human motion can support intentions at different semantic granularities, from checking a visible object part to understanding the layout of a larger space. This makes the problem a language-conditioned distribution over target camera poses, rather than a deterministic next-pose regression problem. A useful model should therefore couple geometric pose prediction with an efficient representation of the visual evidence the motion is expected to reveal: the former captures where the camera should move, while the latter captures why that view is useful. In this paper, we study this interface as language-conditioned camera motion generation. Given a current RGB observation and a free-form intent, the task is to predict a distribution over relative SE(3) target camera poses together with an observation-gain description of what the next view is expected to reveal. To obtain supervision without teleoperated active-perception demonstrations, we mine egocentric video by pairing temporally separated frames: the relative camera transform provides a motion target, while a labeling module produces plausible intents and observation-gain descriptions from the image pair. We instantiate this formulation in LIME, a VLM-based model that autoregressively predicts observation gain and conditions a continuous flow-matching pose head on the resulting hidden sequence. We evaluate this formulation through a dedicated camera-motion benchmark and downstream embodied perception tasks. The results demonstrate that LIME can act as a reusable active-perception interface across diverse embodied tasks. In summary, our main contributions are: • We formulate intent-aware camera motion generation, where an embodied agent predicts a relative SE(3) target camera pose from the current observation and a free-form intent. • We introduce LIME, a vision-language camera-motion generator trained from egocentric video frame pairs with mined intents, observation-gain descriptions, and relative camera poses. • Through experiments on a constructed benchmark and downstream tasks, we demonstrate that our learned intent-aware model is effective across tasks with diverse granularities and benefits downstream applications. 2

2

Related Work

Active perception has long studied sensor motion for information gathering: classical exploration and active mapping reduce map or reconstruction uncertainty by designing task-specific information gain measurement [12, 13, 14, 15, 16, 17, 18], active localization improves state or pose estimates by learning to look at feature-rich region [19, 20], and object- or image-goal navigation searches for semantic or visual targets with accumulated scene knowledge [7, 21, 22, 23, 24]. These methods reason about viewpoint, but optimize predefined objectives such as coverage, reconstruction, localization, or target search, rather than free-form language intent as the conditioning signal for continuous camera motion. Recent language-conditioned embodied models broaden the goal space of robot learning [25, 26, 27], but usually place language over navigation or manipulation actions. In Vision-and-Language Navigation (VLN), natural-language instructions specify a route, destination, or landmark, while the agent acts through base motion or discrete waypoints [28, 29, 30, 8, 31, 32, 33]. Another line of work uses training-free pipelines that leverage LLMs or VLMs without updating the model parameters [34, 35, 36, 37]. In Vision-Language-Action (VLA) manipulation, language and vision condition end-effector or whole-body actions for task execution [10, 11, 38]. A subset of works explicitly studies active viewpoint selection for better manipulation [39, 40, 41, 42, 43, 44]. These approaches typically learn viewpoint behavior jointly with task execution through imitation or reinforcement learning, which ties them to task-specific demonstrations, reward designs, or training environments and limits their generality across intents. Existing benchmarks for active perception span a range of embodied navigation settings, largely focusing on goal reaching rather than fine-grained viewpoint adjustment. Navigation benchmarks such as ObjectNav [7], VLN-CE [45], GOAT-Bench [46], and HM3D-OVON [47] measure goal reaching in synthetic simulators with discrete or low-dimensional actions, leaving fine-grained viewpoint adjustment outside the task definition. Embodied question-answering benchmarks, from EmbodiedQA [48] to OpenEQA [49], HM-EQA [50], and EXPRESS-Bench [51], often use synthetic or reconstructed indoor environments and navigation-style actions, but score answer correctness rather than motion quality. Closest to our setting, VG-AVS [52] studies single-step local view selection in procedural and mesh-based scenes, I-Perceive [44] focuses on simulation-centered inspection pose prediction without dedicated exploration evaluation, and E3VS-Bench [53] uses high-fidelity 3D Gaussian Splatting (3DGS) observations but centers on local inspection around a nearby or alreadyvisible target. These benchmarks highlight the importance of viewpoint selection, while leaving room for evaluating language-conditioned camera motion across broader intent granularities, from exploration and target approaching to fine-grained 6-DoF perspective adjustment.

3

Method

3.1

Problem Formulation

We study vision-language conditioned camera motion generation: given a current RGB observation Is and a free-form language intent x, predict a relative camera motion Tgs that moves the camera from the current pose Ps to a goal pose Pg , producing a next observation Ig that provides visual evidence relevant to the intent. In addition to motion, we predict an observation-gain description g, a natural-language summary of what the next view is expected to reveal beyond the current view. Therefore, the model learns pθ (g | Is , x),

pθ,ϕ (Tgs | Is , x, g),

(1)

where Tgs is represented by 3D translation and a continuous 6D rotation parameterization. At a high level, this asks where the camera should look next to resolve the intent. LIME implements the query with two coupled interfaces over a shared vision-language representation: a language interface for g and an action interface for Tgs . Figure 2 shows the overview of the proposed pipeline. 3

(b) Data Labeling from Egocentric Video

look at the upper kitchen cabinets

𝐻'

GT Pose 𝑇&!

𝑃#,∅ (𝑇&! |𝐼! , 𝑥, 𝑔) 𝐻&|'

Text Dec

ℒ!"#$

Ceiling and upper kitchen cabinets above the current frame. Refrigerator and …

ℒ%&'(

𝑃# (𝑔 | 𝐼! , 𝑥)

Intent 𝑥

GT Observation Gain find the refrigerator find the dining table turn right …

Labeling Prompts

Labeling VLM

Start 𝐼!

𝐕𝐋𝐌𝜽

Flow Matching Pose Head

Start 𝐼!

Subsample

(a) Language-conditioned Camera Motion Model Intent Candidates

Goal 𝐼&

Figure 2: LIME pipeline. Panel (a) shows the VLM-based camera-motion generator with an autoregressive language interface and a continuous flow-matching pose head (Sec. 3.2). Panel (b) shows how we mine passive egocentric video into intent-conditioned camera-motion supervision with observation-gain descriptions (Sec. 3.3). 3.2

Vision-Language Camera-Motion Generator

The observation-gain interface keeps part of the task in the VLM’s native output space by describing the intended visual outcome of the movement. This outcome-level language target is naturally conditioned on both image and intent, and provides auxiliary supervision that encourages the hidden representation to anticipate what new evidence a future view should contain. The continuous flow-matching head then models the conditional distribution of relative SE(3) target poses from this fused representation. Formally, given (Is , x), we construct a multimodal prompt and encode the hidden sequence Hx = VLMθ (Is , x) over visual and language tokens with the VLM backbone before generating the observation-gain description. The language interface uses the VLM’s autoregressive decoder to generate the observation-gain description, Y pθ (g | Is , x) = pθ (gk | g<k , Hx ). (2) k

After autoregressive gain generation, we form the gain-conditioned hidden sequence Hx,g = [Hx ; Hg|x ], where Hg|x denotes the hidden states of the observation-gain tokens conditioned on the image-intent prompt. This sequence combines three sources of information: the current visual evidence, the language intent, and the predicted visual outcome of the motion. Instead of representing actions or spatial coordinates as language tokens [11, 10, 54], we attach a separate continuous flow-matching head, which preserves geometric supervision in SE(3) and models the multimodal distribution of plausible target transforms, similar to recent works [38]. This captures the multiple valid target poses that can satisfy the same (Is , x). We parameterize the target transform Tgs as y = ψ(Tgs ) ∈ R9 , using 3D translation and the first two columns of the rotation matrix [55]. For a flow time t ∼ U (0, 1) and Gaussian noise ϵ ∼ N (0, I), we construct zt = (1 − t)ϵ + ty. The pose head Fϕ takes (Hx,g , zt , t) and predicts the clean target ŷϕ = Fϕ (Hx,g , zt , t), with loss Lpose = ∥ŷϕ − y∥22 . We adopt the x-prediction parameterization [56]: instead of directly regressing the velocity field, the network predicts the clean target pose parameter vector ŷϕ = Fϕ (Hx,g , zt , t). The velocity used for numerical integration is then recovered as vt = (ŷϕ − zt )/(1 − t). This anchors supervision to valid pose parameters, especially the orthonormalized rotation representation, rather than unconstrained velocities. 3.3

Mining Active Camera-Motion Supervision from Passive Egocentric Video

Passive egocentric video provides raw camera-motion supervision: ordered frames capture viewpoint transitions, and poses provide geometry. What it lacks is intent; available annotations usually describe coarse activities or scenes, not the fine-grained perceptual reason for moving the camera [57, 58, 59, 60, 61]. Yet nontrivial egocentric motions often reveal new evidence, improve existing 4

Method

JanusVLN Uni-NaVid VG-AVS VLMnav Ours

Target-approaching

Exploration

Perspective-shift

All

SR

CA-SR

SR

CA-SR

SR

CA-SR

SR

CA-SR

2.0 ± 0.0 0.0 ± 0.0 8.6 ± 0.0 0.0 ± 0.0 45.8 ± 1.6

1.3 ± 0.0 0.0 ± 0.0 6.6 ± 0.0 0.0 ± 0.0 31.8 ± 2.2

34.5 ± 0.6 13.4 ± 2.5 30.0 ± 0.9 4.5 ± 0.7 51.4 ± 0.6

27.5 ± 0.6 11.0 ± 1.8 11.7 ± 0.9 3.8 ± 0.7 32.6 ± 1.4

31.3 ± 0.6 26.5 ± 1.4 36.4 ± 0.4 8.4 ± 0.6 45.8 ± 1.1

31.3 ± 0.6 26.2 ± 1.3 33.1 ± 1.0 8.4 ± 0.6 39.4 ± 1.0

21.9 ± 0.2 12.6 ± 0.8 24.3 ± 0.2 4.1 ± 0.3 47.7 ± 0.1

19.3 ± 0.3 11.8 ± 0.5 16.5 ± 0.0 3.8 ± 0.3 34.4 ± 0.2

Table 1: Success rate (SR, %) and collision-aware success rate (CA-SR, %) under the shared motion budget. CA-SR additionally requires the trajectory to remain collision-free under a 0.15 m distance to occupied space. Cells report mean ± standard deviation over three runs. Colored backgrounds indicate best , second , and third results. views, or reorient toward another region, so we interpret each start–goal pair in hindsight and mine plausible intent labels using the goal view and relative motion as privileged context for labeling. For each egocentric trajectory, we first sample temporally ordered start–goal frame pairs (Is , Ig ). We retain local transitions with available RGB frames and valid camera geometry, discarding pairs with excessive displacement. When camera poses are not provided by the dataset, they can be recovered from RGB trajectories using off-the-shelf camera-pose or reconstruction methods [62]. We then label each retained transition with a structured hindsight VLM prompt. The labeller receives the current frame Is , the goal frame Ig , and a compact summary of Tgs , which gives explicit motion cues, including translation direction, distance, and rotation angle, so the generated labels remain grounded in the actual camera movement. Instead of asking for a free-form caption, the prompt asks for contrastive fields: a motion type, newly visible objects or regions, improved views of content already present in Is , spatial anchors between the two views, an observation-gain description g, and a set of plausible intents Xs,g = {xi }m i=1 . These fields capture visual changes at multiple semantic scales while tying each label to the actual camera motion. We unroll each labeled transition into m examples (Is , xi , g, Tgs ), one per intent. The resulting training set contains approximately 3 M intent-conditioned examples from RoomTour3D [63] and Nymeria [58], covering room-scale walkthroughs and body-scale egocentric interactions; full prompt details are provided in the supplementary material. 3.4

Training and Inference

For each training tuple (Is , xi , g, Tgs ), we apply teacher-forced next-token prediction to g and cleantarget flow matching to Tgs , optimizing: X L = Lgain + λpose Lpose , Lgain = − log pθ (gk | g<k , Is , xi ). (3) k

Together, the losses keep the backbone aligned with language generation while training the continuous head for the same intent-conditioned motion. We instantiate Qwen3-VL-4B-Instruct [54], freeze the vision encoder, and train the multimodal projector, language model, and flow-matching head; the pose loss uses detached VLM hidden states, updating only the flow head and VLM-to-pose projection while the backbone is updated by the observation-gain loss. At inference, we autoregressively generate ĝ and reuse cached gain-token hidden states as the flow-head condition, avoiding a second full VLM forward. We train for one bf16 epoch on 16 NVIDIA GH200 GPUs, taking approximately 30 hours; additional system details are provided in the supplementary material.

4

Benchmark

To evaluate intent-conditioned camera motion at the granularity we study, the benchmark needs three properties: (R1) free-viewpoint photorealistic rendering over continuous SE(3) poses; (R2) diverse intent coverage across spatial scales; and (R3) outcome-level success measures under a shared motion budget, so methods with different action interfaces and stopping behaviors remain comparable. These requirements motivate a benchmark design that combines photorealistic continuous-view rendering with intent annotations and a unified evaluation protocol. 5

JanusVLN

Uni-NaVid

VG-AVS

VLMnav

Ours

Target-approaching: Go to see the matches on the coffee table.

Exploration: Go to see the painting on the wall in the living room.

Perspective-shift: Look at the area under the table in front of you.

Figure 3: Qualitative Comparisons. Columns compare methods and rows show intent families with their language intents. Cyan insets show the shared start view; top-right insets show the first successful frame, or the final frame when no success is reached within the movement budget.

4.1

Benchmark Design

We instantiate the benchmark on InteriorGS [64], a 1K-scene real-world indoor 3DGS dataset that supports photorealistic rendering from arbitrary camera poses and intrinsics. We sample start views across scenes, construct benchmark examples by moving the camera to an intent-relevant reference view, author language intents, and assign per-example camera intrinsics randomly sampled within a realistic range. Each start–reference pair contains (Is , Xs,g , Ig ) plus the underlying camera poses and intrinsics; at inference, models receive only (Is , x), x ∈ Xs,g , and success is defined by acquiring an intent-satisfying view rather than matching the annotated goal pose. The benchmark contains 425 examples across three intent families. Target-approaching: move toward a visible or partially visible target, from furniture-scale to small tabletop objects. Exploration: acquire evidence not sufficiently visible at start, often by following spatial cues such as doors, corridors, or room boundaries. Perspective-shift: change viewpoint around an object or region to reveal occluded content, inspect spatial relations, or adjust distance for a more informative view. Detailed examples and the intrinsics-sampling protocol can be found in the supplementary material.

4.2

Evaluation Setup

To accommodate methods with different action interfaces and intents at different spatial granularities, we evaluate all methods with a budgeted multi-step protocol, even when the desired outcome may require only a local viewpoint adjustment. Each method outputs a camera motion from the current observation and intent, the renderer applies it and returns the next view, and the process repeats until a stop signal or until the shared budget of 6 m translation and 600◦ rotation is reached. Methods without a stop signal run until the budget is reached. An evaluation trajectory succeeds if any rendered view before termination satisfies the intent. For Exploration and Perspective-shift, Gemini-3.1-Pro-Preview judges the candidate frame against the intent with Ig as a non-exclusive reference. For Target-approaching, success requires geometric proximity to the target object’s InteriorGS bounding box and Gemini-verified visual recognizability of the target. Collision-aware success rate further requires the trajectory prefix to first success to remain 0.15 m clear of the scene point cloud. A balanced human-labeled subset audits the automatic judge, with agreement statistics reported in Sec. 5.2. Gemini prompt templates and target-proximity thresholds can be found in the supplementary material. 6

Figure 4: Qualitative samples on ScanNet++ indoor scenes. Each row fixes the same current observation and varies the language intent; colored camera frustums show five sampled target poses from the flow-matching head, illustrating intent-conditioned motion and multimodal pose hypotheses.

5

Experiment

5.1

Experiment Setup

Our experiments aim to answer three questions: (a) whether LIME outperforms diverse baselines on intent-relevant 3D camera-pose prediction; (b) whether its capability generalizes across intent families and deployment settings; and (c) whether it benefits downstream embodied tasks. On the proposed benchmark, each method receives the same start image Is and language intent x in its adapted input format, runs in the same budgeted multi-step protocol, and is evaluated by success rate under a 6 m translation and 600◦ rotation budget. The goal image Ig is withheld from the model and used only for evaluation. We report success rate (SR) and collision-aware success rate (CA-SR) per intent family and overall, following the success protocol defined in Sec. 4.2. Given the limited availability of public implementations under exactly matched assumptions, we compare against closest representative open-source methods spanning language-conditioned navigation, active VQA/view selection, and zero-shot VLM navigation: JanusVLN [29] and Uni-NaVid [8], fine-tuned VLN models for language-instructed navigation; VG-AVS [52], an embodied VLM that actively chooses next views for VQA; and VLMnav [36], a zero-shot VLM-based navigation pipeline. All baselines are adapted to the same renderer, rollout budget, and success metric; further details on baseline adaptation and parameters are provided in the supplementary material. 5.2

Results and Discussion

Table 1 shows that our method achieves the highest success rate across Target-approaching, Exploration, Perspective-Shift, and overall, outperforming baselines specialized for language navigation, active view selection, or VLM-based navigation. It is worth noting that the model is trained from egocentric video, receives no fine-tuning on benchmark scenes, and still performs strongly in rendered evaluation environments. The advantage persists under the collision-aware metric, suggesting that egocentric motion supervision also provides useful traversability bias. Figure 3 illustrates these trends: our method reaches intent-relevant views with fewer evaluation steps and uses full 3D targetpose prediction to combine translation and rotation, such as moving closer while tilting to reveal evidence that planar or discrete-action baselines can miss. As a sanity check, we compare Geminibased success judgments with human judgments on a balanced subset of 90 benchmark examples, yielding 450 trajectory-level labels across five methods. Gemini agrees with human judgments on 91.3% of labels overall, with 85.3–98.0% agreement across intent families; full values are provided in the supplementary material. Figure 4 further probes generalization to different ScanNet++ [65] scenes. For the same current image, changing only the language intent shifts the sampled target poses toward different intent7

Figure 5: Real-robot experiments. The learned camera-motion policy moves the robot camera to acquire visual evidence for chained perception and interaction intents. Blue text indicates action commands passed to a separate manipulation policy. relevant evidence, indicating that the model conditions on the intent rather than only a scene-level prior. Our pose generator also captures uncertainty properly: samples concentrate when the goal is visually supported in the current view, but spread across multiple plausible directions for ambiguous intents such as leaving a room. We further deploy our method on a Boston Dynamics Spot with an arm, using RGB-D images from its hand camera. For real-world robot experiments, we use a lightweight LoRA-adapted checkpoint; details are provided in the supplementary material. Figure 5 shows language-conditioned viewpoint changes on physical scenes, such as viewing below an object or checking the region left of an oven. We also integrate the camera-motion policy with VidBot [66], a vision-language-conditioned manipulation trajectory generator that, like most manipulation policies, requires the target object to be visible before acting. When the target is initially outside the field of view, our policy first reveals the task-relevant object or region before VidBot acts, and can also verify outcomes after execution. These results suggest that the adapted camera-motion policy transfers beyond rendered benchmarks and can serve as an active perception module for downstream embodied interaction. More detailed analysis of how LIME supports manipulation and other downstream embodied tasks is provided in the supplementary material.

6

Conclusion

We presented language-conditioned camera motion generation as a first-class embodied capability: given the current view and an intent, a robot should predict where to move its camera to acquire more useful visual evidence. To study this problem, we introduced a pipeline that mines intentconditioned camera-motion supervision from passive egocentric video, trains a VLM-based model with observation-gain language supervision and a continuous flow-matching pose head, and evaluates the resulting policy on a dedicated benchmark and downstream embodied tasks. The results suggest that ordinary human video can provide effective supervision for intent-aware robot camera motion, enabling models to generalize across target approaching, exploration, and perspective shift, while transferring to real robot observations. More broadly, free-form intent-conditioned camera motion can serve as a reusable active-perception primitive: the same LIME interface supports viewpoint generation for manipulation, embodied question answering, and longer multi-step behaviors such as navigation and object scanning.

8

Supplementary Material for “LIME: Learning Intent-aware Camera Motion from Egocentric Video”

A

Technical Details

A.1

Dataset Curation

The RoomTour3D and Nymeria labeling process uses dataset-specific forks of the same structured pair-labeling prompt. Both forks share the output schema below; the Nymeria fork additionally filters egocentric hands, body, and held-object content, to prevent the labeling result from concentrating on them. Qwen3-VL Dataset Labeling Prompt System prompt. You are an expert annotator producing structured observation-gain labels for pairs of frames from an egocentric trajectory. Your output is the supervision signal for a vision-language model that learns to reason about what new information a camera move reveals. You receive, in order: 1. Current frame: the agent’s view before the move. 2. Goal frame: the agent’s view after the move. 3. Pose summary: the camera motion from the current frame to the goal frame, expressed in OpenCV camera convention. Trust this metric ground truth over your visual guess of the motion. Produce a single JSON object describing what new information the goal frame gives the agent that was not available in the current frame, plus a closed-vocabulary tag for the type of camera motion that produced the gain. Use the pose summary as the primary signal for choosing motion type; the visual content tells what was revealed, and the pose tells how. Set info gain present to false when the move is static, disconnected, or when no new static-scene content is visible in the goal frame. When false, leave the sub-fields, captions, and intentions empty, null, or []. When true, fill the following structured fields: • newly visible: items not visible in the current frame but clearly visible in the goal frame. • enhanced views: items already visible in the current frame that are now seen better. • spatial anchor: one sentence describing how the goal frame is positioned relative to the current frame in space. caption long is a purely visual observation-gain description. Describe the visual information gained going from the current frame to the goal frame, grounded in newly visible and enhanced views. Keep it about what is seen, not how the camera moved, and lead directly with content. caption short is a short intent-style query and must be a faithful compression of caption long. Produce a diverse set of short imperative intentions a user might say to ask the agent to make this move. Each intention is grounded in one of the structured fields: newly visible, enhanced views, spatial anchor, or motion type. The allowed intention kinds are find, explore, inspect, and navigate. Output strict JSON with these fields: • motion type: one of the closed-set motion types. • info gain present: boolean. • reason: string or null. • newly visible: list of entries with item, kind, and where relative to current. • enhanced views: list of entries with item and change. • spatial anchor: string or null. • caption long: string or null. • caption short: string or null. • intentions: list of entries with text, kind, and anchor, where kind is find, explore, inspect, or navigate. • quality self score: integer from 1 to 5.

9

User prompt. [current frame -- view BEFORE the move] [goal frame -- view AFTER the move] [POSE SUMMARY] <POSE SUMMARY> Produce the JSON object now.

Starting from roughly 2M candidate start–goal pairs extracted from RoomTour3D and Nymeria, we apply balanced subsampling before expanding pairs into intent-conditioned examples. We subsample across data source, intent kind, motion type, translation magnitude, and rotation magnitude. Figure 6 summarizes the resulting distributions, showing that the final training pool retains coverage over semantic and geometric axes rather than collapsing to short forward motions or a single intent family. Figures 7 and 8 show representative valid image pairs with generated labels. During training, the dataloader samples one intent from each available intent category for a retained pair.

Figure 6: Dataset distributions after balanced subsampling. We balance the start-goal image pairs across data source, intent kind, motion type, translation magnitude, and rotation magnitude before expanding them into intent-conditioned training examples.

10

Figure 7: Example of a start–goal pair label from RoomTour3D, showing the paired frames, motion metadata, observation-gain description, structured visual-change fields, and generated intent set.

11

Figure 8: Example of a start–goal pair label from Nymeria, showing the paired frames, motion metadata, observation-gain description, structured visual-change fields, and generated intent set.

12

A.2

Training Setup

Table 2 lists the key configurations and hyperparameters for the optimizer, input, flow-head, and inference used in training our main model. Component

Setting

Optimizer AdamW, learning rate 1e−5 Schedule Cosine decay, warmup ratio 0.03 Regularization Weight decay 0.1, max gradient norm 1.0 Precision bf16 with DeepSpeed ZeRO-3 Sequence length Maximum length 8192 tokens Image resolution budget 200704 max pixels, 784 min pixels Loss weight λpose = 1.0 Pose target 3D translation + first two rotation columns Flow head 512 hidden dimension, 6 cross-attention blocks, 8 heads Time embedding 256-D sinusoidal embedding followed by an MLP Flow parameterization x-prediction with zero-initialized output projection Training augmentation Hidden-state noise 0.01; no intent-token masking Inference 128 max gain tokens, 10 Euler steps, 5 pose samples Table 2: Training and inference hyperparameters for the main LIME model.

For the real-world robot experiments, we further adapt the LIME checkpoint with a lightweight LoRA fine-tuning stage on a small real-world dataset collected with Aria Gen 1 glasses. The set contains around 1,700 start–goal pairs, aligned to the robot camera setting and balanced across find, explore, and navigate-style intents. We initialize from the main LIME checkpoint, continue finetuning the flow-matching head, and train LoRA adapters on the VLM backbone with rank 64, alpha 128, and dropout 0.0. This adaptation is used only for the real-world robot experiments. The robotadaptation run uses learning rate 5e−5, batch size 4, gradient accumulation 1, and 3 epochs on 4 GPUs.

B

Benchmark Design and Evaluation Details

B.1

Benchmark Construction

The main paper describes the high-level construction of our benchmark. Here we provide additional details, including the intent-family distribution, start–reference camera-motion statistics, and the camera intrinsics and height ranges used for rendering. B.1.1

Intent-Family Distribution and Test Set Statistics

The benchmark is built from 105 InteriorGS scenes and 259 curated start–reference pairs. Since a single start–reference pair may support multiple language intents, the final benchmark contains 425 instruction-level examples. These examples are distributed across the three intent families: 152 Target-approaching, 142 Exploration, and 131 Perspective-shift. We summarize the start–reference motion distribution across the 425 instruction-level examples in Table 3. Translation is measured as the Euclidean distance between the start and reference camera centers, and rotation is measured as the geodesic angle between the start and reference camera orientations. These statistics characterize the spatial scale of the annotated reference motions and show that most remain within a local viewpoint-change range, consistent with the benchmark’s focus on local intent-conditioned camera motion. 13

#Pairs

#Examples

Trans. med./p90 (m)

Rot. med./p90 (◦ )

Target-approaching Exploration Perspective-shift

90 78 91

152 142 131

4.10 / 5.03 3.17 / 4.41 1.92 / 3.78

46.4 / 74.3 88.5 / 143.5 43.5 / 140.0

Overall

259

425

3.18 / 4.74

56.2 / 123.5

Intent Family

Table 3: Benchmark dataset and motion statistics. Examples are instruction-level samples derived from the start–reference pairs. Translation and rotation are computed between the annotated start and reference poses and aggregated over instruction-level examples.

B.1.2

Camera Agent Configuration

All benchmark images are rendered at 640×360 resolution. For each curated start–reference pair, we use a pinhole camera model with square pixels, centered principal point, and focal length fx = fy uniformly sampled from [260, 350] pixels. This corresponds to a vertical field-of-view range of approximately 54.5◦ to 69.4◦ . The sampled intrinsics are held fixed for the start image, reference image, and evaluation-trajectory frames of each benchmark example, and are shared across all evaluated methods. Quantity

Value / Range

Image resolution 640 × 360 Principal point (320, 180) px Focal length fx = fy ∼ U(260, 350) px Vertical field of view 54.5◦ –69.4◦ Start-view height, 10–90 percentile 1.40–1.81 m Reference-view height, 10–90 percentile 1.41–1.86 m Table 4: Camera intrinsics, image resolution, and camera-height statistics for the benchmark. Intrinsics are sampled per curated start–reference entry and kept fixed throughout evaluation.

B.1.3

Benchmark Examples

Figure 9 shows representative benchmark examples from the three intent families. Each example consists of a start image, a language intent, and a held-out reference view that illustrates one possible intent-satisfying camera pose. B.2

Evaluation Protocol

All methods are evaluated through the same budgeted multi-step protocol. Each method receives the current rendered observation and the language intent in its adapted input format, predicts a camera motion, action, or stop signal, and the resulting view is rendered in InteriorGS. Success is evaluated over the generated trajectory prefix under a shared motion budget, rather than by direct pose error to the annotated reference view. The held-out reference image Ig is used only by the evaluator as visual evidence of one intent-satisfying view; it is not shown to the method. The following subsections define the normalized motion budget, VLM-as-a-judge criteria for Exploration and Perspective-shift, two-stage Target-approaching evaluation, collision-aware evaluation, human validation of Gemini judgments, and baseline adaptation details. B.2.1

Sequential-Query Evaluation Trajectory and Step Budget

The benchmark supports multi-step execution by repeatedly applying each method to the latest rendered observation. Each evaluation trajectory starts from the benchmark start view, denoted frame 0. At frame k, the model receives the current rendered RGB image and the language intent, then predicts either a relative camera motion, a discrete/parameterized action, or a stop signal depending on the method. The predicted motion is applied to the current camera pose, and the next observation 14

(a) Exploration: Go to see the painting on the wall in the living room.

(b) Target-approaching: Go to the floor lamp next to the TV.

(c) Perspective-shift: Look at the ceiling area above the bed in front of you.

Figure 9: Representative benchmark examples from the three intent families. In each row, the left image is the start view and the right image is the held-out goal reference view.

is rendered from the resulting pose using the sample’s camera intrinsics. Thus, for k > 0, frame k is the view obtained after applying the k-th model prediction. Because different methods use different action parameterizations, we measure evaluation-trajectory length using a normalized motion budget rather than a fixed number of model calls. For a transition from pose pk−1 to pose pk , let ∆tk be the Euclidean distance between the two camera centers in meters, and let ∆rk be the geodesic rotation angle between the two camera orientations in degrees. We define the transition cost as   ∆tk ∆rk ck = max , . 0.1 10 The cumulative cost at frame k is Ck =

k X

ci .

i=1

Under our default success metric, a frame is eligible to count as successful only if Ck ≤ 60. This corresponds to a budget of up to 6 m of pure translation or 600◦ of pure rotation, while also constraining mixed translation–rotation trajectories. The budget is intentionally local: it allows multi-step correction and limited exploration around the start view, while preventing the evaluation from becoming long-horizon navigation or allowing success through unconstrained random walk. If a method emits a stop action, the evaluation trajectory terminates at the current frame. Methods without an explicit stop action are evaluated until no further frame can be produced within the shared budget. If a predicted transition would exceed the budget, we record the over-budget prediction for 15

debugging but do not treat the resulting frame as a valid success candidate. Therefore, success is determined only over frames whose cumulative cost is within the budget. We report example-level success rate (SR), computed over benchmark examples rather than generated frames. A benchmark example is counted as successful if at least one eligible frame in the evaluation trajectory satisfies the category-specific success criterion described below. Otherwise, it is counted as a failure. We do not evaluate by direct pose error to the annotated reference pose, because many different viewpoints can satisfy the same language intent. The held-out reference view is used only as evaluation evidence for what an intent-satisfying observation can look like, not as a unique target pose that the model must reproduce.

B.2.2

VLM-as-a-Judge Success Criteria for Exploration and Perspective-Shift

For examples in the Exploration and Perspective-shift intent families, success cannot be reliably measured by distance to the annotated reference pose. The same intent may be satisfied by multiple nearby or even substantially different viewpoints, as long as the resulting image reveals the requested visual evidence. We therefore evaluate these categories using a VLM-as-a-judge protocol, following recent viewpoint-dependent active perception evaluations such as E3VS-Bench [53]. For each eligible candidate frame in an evaluation trajectory, the judge is given the language intent, the start image Is , the held-out reference image Ig , and the candidate image rendered from the model’s predicted pose. The reference image is used as evidence for one valid way to satisfy the intent, but it is not treated as a pixel-level target or as the only acceptable view. A candidate frame is judged successful if it provides sufficient visual evidence to satisfy the instruction, even when its viewpoint, scale, or composition differs from Ig . For Exploration, success requires the candidate view to reveal the requested object, region, or visual evidence that is absent or insufficiently recognizable from the start view; moving in a plausible exploratory direction is not sufficient unless the requested evidence becomes visible. For Perspectiveshift, success requires the candidate view to improve observation of the specified object, region, or spatial relation, for example by revealing occluded content, changing the viewing side, inspecting above/below/around an object, or adjusting distance to obtain a more informative view. In both cases, the candidate frame need not match the held-out reference view exactly, but it must provide enough visual evidence to satisfy the intent. We use Gemini-3.1-Pro-Preview as the automatic judge and ask it for a binary success decision for each candidate frame. Candidate frames are evaluated under the shared step budget described above, and the first eligible frame judged successful is recorded as the first successful frame. If no eligible frame is judged successful, the evaluation trajectory is counted as a failure for that benchmark example. The exact judge prompt for these two intent families is provided below. 16

Exploration and Perspective-shift Gemini judge prompt System prompt. You are judging one candidate view from a robot/agent navigation trajectory in a 3D indoor scene. You will be shown exactly three RGB images in this order: 1. START context: the stored starting view that the human labeller wrote the instruction against. Use it only as context to interpret the instruction. 2. END reference frame: a human-labelled reference view for the intended end state. 3. CANDIDATE frame: the agent trajectory view being judged. The instruction in the user message is the source of truth. Decide whether the CANDIDATE frame satisfies that instruction. Candidate frame 0 is the agent’s initial trajectory view; depending on the evaluation mode it may or may not look identical to the START context. An agent can be successful at its initial view, so frame 0 is a valid candidate. The END reference frame is only a visual reference for the intended goal state; it is not the only acceptable answer and not a pixel-matching goal. The candidate does not need to match the exact viewpoint, distance, crop, or angle of the END reference. It can succeed from a different view if it clearly satisfies the instruction. Use the START frame only as context for the initial situation. Do not mark the candidate successful merely because it is different from START. Do not evaluate the physical plausibility or quality of intermediate path frames; this judge only decides whether this one candidate view satisfies the instruction, with the END reference as non-exclusive visual context. Output strict JSON with two fields: • success: boolean, true iff the candidate frame clearly satisfies the instruction. • reasoning: string, at most 60 words, giving a brief justification. Mention how the candidate relates to the instruction and, when useful, to the END reference. User prompt. Instruction: "<INSTRUCTION>" Candidate frame: frame <CANDIDATE FRAME INDEX> of <N FRAMES TOTAL> total frames. Images are provided in order: START context frame, END reference frame, then CANDIDATE frame. The candidate may be frame 0, meaning the agent’s initial trajectory view, which may or may not look identical to the START context. The END reference is a helpful example of the intended goal state, but it is not a pixel-matching goal. The candidate does not need to match the exact viewpoint, distance, crop, or angle of the END reference. Judge whether the candidate satisfies the instruction. Respond with strict JSON: {"success": <bool>, "reasoning": <str>}.

B.2.3

Geometric and Visual Success Criteria for Target-Approaching

Target-approaching examples ask the model to move toward a specified target object, object group, or fixture. Unlike Exploration and Perspective-shift examples, these examples include an explicit spatial requirement: the model should not merely obtain any view in which the target object is visible, but should move close enough to the intended target object for the view to support inspection. We therefore use a two-stage evaluation protocol that combines geometric proximity with visual verification. In the first stage, we check whether each candidate camera pose is sufficiently close to the annotated target. Each Target-approaching example is associated with a target name and a 3D target bounding box in the InteriorGS scene. For a candidate frame, we compute the distance from the camera center to the target box surface. The frame passes stage 1 if this distance is below an adaptive threshold determined by the physical size of the target object. Let s denote the maximum side length of the target bounding box. We assign a threshold of 0.8 m to targets at the 10th percentile of s, and a threshold of 1.2 m to targets at the 90th percentile of s. For targets with intermediate sizes, the distance threshold is linearly interpolated between these two values; targets outside the percentile range use the corresponding clipped endpoint threshold. We visually inspected representative targets near these two percentile anchors to ensure that the thresholds correspond to physically meaningful close-up distances for both small tabletop objects and larger furniture-scale targets. The geometric stage is necessary because Target-approaching success depends on spatial proximity, not only visual presence. A candidate view may contain the target while still being far away, especially for large or salient objects, and a VLM-as-a-judge evaluator is not reliable at estimating metric 17

distance from a single rendered image. Conversely, geometric proximity alone is also insufficient: a camera can be close to the target box while the target is occluded, outside the field of view, or visually ambiguous. The second stage therefore verifies visual recognizability. In the second stage, frames that pass the geometric proximity test are evaluated by a Gemini-based VLM-as-a-judge visual-verification step. The judge is given the original instruction, the target object name, a contextual target-object phrase, the END goal reference image, and the candidate image. The exact visual-verification prompt used for this stage is provided below. A frame is counted as a Target-approaching success only if it passes both stages: it must be geometrically close to the annotated target and the target object must be visually identifiable in the rendered image. The first eligible frame satisfying both conditions is recorded as the first successful frame. If no frame within the step budget satisfies both stages, the trajectory is counted as a Target-approaching failure. This two-stage protocol avoids two complementary failure modes. It prevents image-only false positives in which the target is visible but not actually approached, and it prevents geometry-only false positives in which the camera is near the target annotation but the rendered image does not provide recognizable visual evidence of the target. We use the default adaptive threshold pair (0.8 m, 1.2 m) for the main results; Fig. 10 analyzes how target stage-1 SR changes as this threshold pair is relaxed. The main ranking between different methods is stable across thresholds.

JanusVLN

Uni-NaVid

VG-AVS

VLMnav

Ours

Target stage-1 SR (%)

80

60

40

20

0 0.8/1.2

1.0/1.5 1.2/1.8 1.4/2.1 Adaptive threshold pair (small / large target, m)

1.6/2.4

Figure 10: Target stage-1 proximity SR under increasingly relaxed adaptive distance thresholds. Each x-axis tick denotes the small-object / large-object threshold pair in meters, assigned to the 10th and 90th percentiles of target AABB max-side size, with intermediate thresholds linearly interpolated. Curves report mean SR over three runs under the shared motion budget.

Relaxing the threshold increases stage-1 SR for all methods, as expected, but the relative ordering remains stable across the sweep: our method achieves the highest stage-1 SR at every threshold pair, while VLMnav remains near zero and Uni-NaVid improves only modestly. VG-AVS is the most threshold-sensitive baseline, rising sharply as the allowed distance increases, which suggests that it often moves in the general direction of the target but does not approach it as closely under the default close-up criterion. This sensitivity analysis supports the default (0.8 m, 1.2 m) setting as a strict but physically meaningful target-approach criterion rather than an arbitrary operating point. 18

Stage-2 Target-approaching visual-verification judge prompt System prompt. You are judging whether one robot/agent candidate view clearly shows a specific target object in a 3D indoor scene. You will be shown exactly two RGB images in this order: 1. END goal reference image: a human-labelled reference view that helps identify the intended target object and context. 2. Candidate prediction frame: the trajectory frame being judged. The END goal reference is not a pixel-matching goal. Use it only as visual context for the target object’s identity, appearance, and scene context. The candidate can succeed from a different viewpoint, crop, distance, or angle if it clearly shows the intended target object. Decide whether the candidate prediction frame shows the target object clearly enough for a human to recognize or inspect it. Mild occlusion, different viewpoint, or imperfect centering can still be successful. Reject the candidate if the target is absent, too tiny, too blurry/dark, heavily occluded, or cut off so much that the target cannot be identified. Output strict JSON with exactly these fields: • success: boolean • visibility: clear, mostly clear, partially visible, not visible, or uncertain • occlusion: none, mild, moderate, heavy, or uncertain • reasoning: string, at most 60 words User prompt. Original instruction: "<INSTRUCTION>" Target object name: "<TARGET OBJECT NAME>" Target-object phrase with context: "<PROMPT TARGET PHRASE>" Images are provided in order: END goal reference image, then candidate prediction frame. The END goal reference image is a non-exclusive reference for identifying the target object and context, not a pixelmatching goal. Judge only whether the candidate prediction frame clearly shows the target object enough for recognition or inspection. The candidate frame has already passed a geometric proximity check to the target annotation. Do not judge navigation path quality, physical plausibility, or whether the camera is close enough. Your only job is to judge whether the target is visually visible and identifiable in the candidate image. Use the target-object phrase/context to disambiguate the intended object when there are multiple similar objects. Do not require every contextual support object to be fully visible if the target object identity is otherwise clear. If the target-object phrase is plural or refers to a group, such as “the books” or “the oranges”, success does not require every individual item to be visible. Mark success when enough of the intended group is clearly visible to recognize or inspect the target group. Respond with strict JSON: {"success": <bool>, "visibility": "clear|mostly clear|partially visible|not visible|uncertain", "occlusion": "none|mild|moderate|heavy|uncertain", "reasoning": <str>}.

B.2.4

Collision-Aware Success Rate (CA-SR)

The standard SR metric evaluates whether a trajectory eventually reaches an intent-satisfying view, but it does not penalize trajectories whose camera path passes through scene geometry before reaching that view. CA-SR uses the same example-success criterion as SR under the shared motion budget, but requires the trajectory prefix up to the first successful frame to remain collision-free with respect to the InteriorGS scene point cloud. We model the agent as a point located at the camera center. For each scene, we use the InteriorGS 3D Gaussian point cloud as the geometric proxy for occupied scene structure and build a nearest-neighbor index over its 3D point positions. For an originally successful trajectory with first successful frame f ⋆ , we check the camera centers for frames 0, 1, . . . , f ⋆ . Frame 0 corresponds to the start camera pose, and frame k > 0 corresponds to the camera pose after the k-th model prediction. Let qk be the camera center at frame k, and let dk = min ∥qk − p∥2 p∈P

19

be its nearest-neighbor distance to the scene point cloud P. A frame is considered collision-free if dk ≥ τclear , where we use τclear = 0.15 m for the reported results. A successful trajectory remains collision-aware successful only if all frames in the prefix satisfy this clearance constraint: min dk ≥ τclear .

0≤k≤f ⋆

Original failures remain failures under CA-SR. Original successes are converted to collision-aware failures if any checked camera center violates the clearance threshold before or at the first successful frame. The denominator of CA-SR is unchanged from SR: all benchmark examples in the corresponding intent family are counted. This audit is intentionally conservative but lightweight. It checks only the evaluated discrete frames, not continuous line segments between consecutive poses, and it treats the camera center as a point agent rather than modeling the full camera body. We also only audit the prefix through the first successful frame, since later frames are irrelevant once the trajectory has already satisfied the benchmark example. Thus, CA-SR should be interpreted as a stricter version of SR that penalizes visually successful trajectories whose successful prefix intersects the reconstructed scene geometry. B.2.5

Human-Judge Validation

Because our benchmark uses automatic visual judging for semantic success, we validate the Geminibased evaluator against human annotations on a balanced subset of trajectories. This audit is intended to measure whether the automatic judge agrees with human perception of task success, rather than to replace the full automatic evaluation. We use the same budgeted success criterion as in the main results. We sample 30 examples from each intent family: Target-approaching, Exploration, and Perspectiveshift. This gives 90 benchmark examples in total. For each of them, we include the run-1 evaluation trajectory from each of the five main-table methods, resulting in 450 trajectory-level human judgments. Samples are drawn from examples for which all five methods have run-1 evaluation trajectories, so every selected benchmark example can be compared across methods. Human annotators view the start image, the held-out reference image, the instruction, and the generated trajectory frames, then mark the first successful frame if any frame within the evaluation trajectory satisfies the intent; otherwise the trajectory is marked as failure. We compare these human labels against the corresponding Gemini-based SR labels. Agreement is the fraction of trajectory-level binary success/failure labels that match between the human annotator and Gemini; it does not require the first successful frame index to be identical.

Intent Family

n

Human SR

Gemini SR

∆ (G–H)

Agreement

Target-approaching Exploration Perspective-shift

150 150 150

11.3 24.7 28.7

13.3 34.0 36.7

+2.0 +9.3 +8.0

98.0 90.7 85.3

Overall 450 21.6 28.0 +6.4 91.3 Table 5: Human validation of the Gemini-based SR evaluator on a balanced subset of benchmark trajectories under the shared motion budget. We sample 30 examples per intent family and evaluate run-1 trajectories from five methods, giving 150 judgments per intent family and 450 judgments in total. SR values and agreement are reported in percent. ∆ denotes Gemini SR minus human SR.

The automatic judge shows strong agreement with human annotations, reaching 91.3% agreement overall. Agreement is highest for Target-approaching examples, where the two-stage geometric and visual protocol makes the success criterion relatively explicit. Exploration and Perspectiveshift examples have lower agreement because multiple views can partially satisfy an intent, making the boundary between partial and sufficient visual evidence less crisp. Gemini SR is consistently higher than human SR, indicating that the automatic judge is somewhat more permissive than human 20

annotators. Nevertheless, agreement remains high across all intent families, supporting the use of Gemini-based judging for the full benchmark while retaining this human audit as a calibration check. B.2.6

Baseline Adaptation Details

We compare LIME against four representative open-source baselines: JanusVLN and Uni-NaVid for language-conditioned navigation, VG-AVS for active view selection in visual question answering, and VLMnav for zero-shot VLM-based navigation. These methods were not originally designed for relative SE(3) target-pose prediction in InteriorGS, so we adapt their input and action interfaces while keeping the shared motion budget and success metrics fixed. The baselines differ mainly in their language interface, motion output, and native observation/action convention. JanusVLN and Uni-NaVid receive navigation-style instructions and output discrete planar actions such as moving forward, turning left/right, or stopping. VG-AVS receives a questionstyle input and outputs a planar active-view action parameterized by heading rotation, forward distance, and final view rotation. VLMnav is a zero-shot navigation pipeline that first queries whether to stop and then selects a polar navigation action from depth-derived navigability candidates. For a fair comparison, we instantiate the VLMnav pipeline with Qwen3-VL-4B as its VLM backend, matching the scale of LIME’s VLM backbone. In contrast, LIME directly predicts a relative SE(3) target camera pose from the current RGB observation and language intent. JanusVLN and Uni-NaVid are naturally tied to gravity-aligned planar navigation conventions; VG-AVS follows a similar planar active-view convention in its AVS-HM3D evaluation setup; and VLMnav renders observations from a fixed downward-pitch camera viewpoint in its native setup. For the main-table comparison, we choose an adapter that preserves the benchmark observation while respecting each baseline’s native action space. Specifically, JanusVLN, Uni-NaVid, and VGAVS observe the benchmark’s original sample-start view, but their predicted planar actions are executed in a gravity-planar action frame derived from the start pose. This avoids changing the visual input seen by the method, while still applying actions in the planar convention expected by these baselines. VG-AVS is given an EQA-style question derived from the same underlying intent. VLMnav is evaluated with the sample-start view rather than its native fixed downward-pitch camera convention for the main comparison, but its pipeline still uses rendered depth to construct navigability masks and candidate polar actions. LIME does not require an action adapter because its output is already a relative SE(3) camera motion. For LIME, each evaluation step uses 10 Euler integration steps and draws 5 flow-matching pose samples. We execute the mean predicted pose as the relative camera motion for that step, avoiding an additional sample-selection heuristic. We also evaluate alternative adapter choices to test whether the main conclusion depends on this interface choice. For JanusVLN, Uni-NaVid, and VG-AVS, the gravity-planar variant renders the initial observation from the gravity-planar view, which is closer to their native embodied-agent setup but changes the benchmark start image. For VLMnav, the VLMnav-pitch variant renders observations using its native fixed downward-pitch camera convention instead of the benchmark sample-start view. Table 6 reports these variants. The adapter comparison shows that baseline performance is sensitive to the observation/action convention, but the overall conclusion is stable. Preserving the sample-start view is generally stronger for JanusVLN and VG-AVS, while Uni-NaVid benefits somewhat from the fully gravity-planar variant. VLMnav improves when using its native fixed downward-pitch camera convention, indicating that its pipeline is particularly tied to its original camera convention. However, all adapter variants remain substantially below LIME, suggesting that the main result is not an artifact of a single unfavorable baseline adapter. B.2.7

Evaluation Hardware

Evaluation trajectories were generated on NVIDIA GPUs. JanusVLN was evaluated on an NVIDIA A100 80GB GPU because its inference pipeline exceeded the memory available on 24GB GPUs. All other methods, including LIME, Uni-NaVid, VG-AVS, and VLMnav, were evaluated on NVIDIA 21

Method JanusVLN (sample-start) JanusVLN (gravity-planar) Uni-NaVid (sample-start) Uni-NaVid (gravity-planar) VG-AVS (sample-start) VG-AVS (gravity-planar) VLMnav (sample-start) VLMnav (VLMnav-pitch) Ours

Target-approaching

Exploration

Perspective-shift

SR

CA-SR

SR

CA-SR

SR

CA-SR

SR

All CA-SR

2.0 0.7 0.0 1.3 8.6 8.6 0.0 0.0 48.0

1.3 0.7 0.0 1.3 6.6 6.6 0.0 0.0 34.9

33.8 21.8 9.9 12.0 30.3 23.2 4.9 10.6 50.7

27.5 14.8 8.5 8.5 12.0 9.9 4.2 8.5 31.0

32.1 29.8 27.5 32.1 36.6 32.8 7.6 15.3 44.3

32.1 29.8 27.5 32.1 32.8 30.5 7.6 13.7 38.2

21.9 16.7 11.8 14.4 24.5 20.9 4.0 8.2 47.8

19.5 14.4 11.3 13.2 16.5 15.1 3.8 7.1 34.6

Table 6: Run-1 SR and CA-SR for baseline model variants and our method under the shared motion budget. Sample-start variants preserve the benchmark sample’s original start view for observation, while gravity-planar variants render the initial observation from a gravity-aligned planarized start pose. VLMnav-pitch renders observations using VLMnav’s native fixed downward-pitch camera convention instead of the benchmark sample-start view.

RTX 4090 24GB GPUs. This hardware difference was only used to satisfy model memory requirements; all methods followed the same InteriorGS rendering setup, budgeted multi-step protocol, success criteria, and judging protocol described above.

C

Additional Experiments

C.1

Ablation

We ablate the main design choices of LIME under the same benchmark setting as the main evaluation, using the same motion budget, Gemini-based success metric, and collision-aware audit. We compare the full model against variants without observation-gain supervision, with the flowmatching condition augmented by a monocular depth image [67], with different numbers of flowmatching samples, and with a larger Qwen3-VL-8B backbone. Table 7 reports SR and CA-SR for each intent family and overall. Method

Target-approaching

Exploration

Perspective-shift

SR

SR

SR

CA-SR

CA-SR

CA-SR

All SR

CA-SR

w/o Gain 9.2 5.9 33.1 18.3 29.8 26.7 23.5 16.5 Depth-aug FM 39.5 30.3 42.3 29.6 54.2 39.7 44.9 32.9 FM samples=1 40.8 28.9 42.3 24.6 46.6 40.5 43.1 31.1 FM samples=10 46.7 33.6 53.5 35.9 50.4 40.5 50.1 36.5 8B backbone 44.7 27.0 54.2 33.8 45.8 41.2 48.2 33.6 Ours main 48.0 34.9 50.7 31.0 44.3 38.2 47.8 34.6 Table 7: Ablation results on the proposed benchmark. All rows use run 1 and report SR and CA-SR in percent under the shared motion budget. Unless specified otherwise, the default number of flowmatching samples is 5.

C.2

Sequential-Query Inference Efficiency

We evaluate whether LIME reaches successful views with fewer sequential model decisions. Because LIME is trained from start–goal image pairs, it can predict a local 3D target pose that makes larger progress in a single step rather than relying on short primitive actions. Inference steps are counted up to the first successful frame under the shared success criterion, where frame 0 is the start image and frame k corresponds to the k-th decision step. Runtime is averaged over all recorded trajectory steps, and also over successful trajectories up to the first successful frame. For VLMnav, one decision step includes both a stopping query and an action-selection query. Table 8 reports each 22

method’s overall SR, the mean number of inference steps to success, and runtime per inference. Since inference-step averages are conditioned on successful trajectories, the SR column indicates how broad each method’s success set is. LIME requires fewer inference steps than the VLN-style baselines while achieving substantially higher SR; VG-AVS uses fewer steps, but each inference is substantially slower. Method JanusVLN (7B) Uni-NaVid (7B) VG-AVS (7B) VLMnav (4B) Ours (4B)

SR (All)

#Inf. Target-approaching

#Inf. Exploration

#Inf. Perspective-shift

#Inf. All

Time/Inf. (s)

Succ. Time/Inf. (s)

21.9 12.6 24.3 4.1 47.7

22.67 – 5.31 – 7.66

15.71 12.60 3.80 1.89 3.59

10.83 7.70 2.31 2.97 3.89

13.78 9.43 3.30 2.58 5.08

0.72 0.24 9.61 3.80 2.83

0.64 0.23 9.62 4.22 2.79

Table 8: Multi-step inference efficiency and runtime on the primary benchmark trajectories. SR (All) is the overall mean SR, reported in percent under the shared motion budget. Inference-step counts are averaged over successful trajectories up to the first successful frame for each intent family. Time/Inf. averages all recorded trajectory steps, while Succ. Time/Inf. averages successful trajectories up to the first successful frame.

C.3

Additional Intent-conditioned View Predictions

Figure 11 shows additional LIME prediction examples in InteriorGS scenes. Each row starts from the initial observation and proceeds through LIME-predicted views up to the first successful view. These examples focus on local viewpoint changes that still require intent reasoning and SE(3) pose prediction, with the goal reached in one or a few steps. C.4

LIME as Active Perception Module for Manipulation

Extending from the main paper’s test, we further evaluate whether LIME can serve as viewpoint preconditioning front end for downstream manipulation on LIBERO-Goal [68], a 10-task languageconditioned manipulation suite that varies task goals while keeping objects and layouts controlled. For each task instruction, we compare two pipelines from the same initial robot state: directly running a VLA policy π0.5 [69], and first using LIME to move the wrist camera before running the same policy. LIME receives a simple target-seeking instruction, “Look for the <object>,” using the task-relevant object or fixture. After LIME predicts a relative camera pose, the robot arm moves the wrist camera to the predicted view, and π0.5 is then executed with the standard LIBERO-Goal manipulation budget. Instead of using the default LIBERO initial wrist-camera views, we sample constrained reachable starts where the task-relevant object or fixture is only partially visible. This initial observation is generally more challenging for a VLA policy to finish the tasks. Both pipelines are evaluated from the same sampled starts, yielding paired direct-vs-LIME rollouts that isolate whether one active camera-motion step improves the visual precondition for manipulation.

23

Table 9: LIBERO-Goal manipulation results from sampled initial states. Each task uses five sampled starts, with one rollout per start. Direct SR runs π0.5 from the sampled start pose, while LIME+π0.5 SR first applies one LIME-predicted wrist-camera motion from the same state. SR is reported as a percentage. Task ID Task instruction Direct π0.5 SR LIME+π0.5 SR

0 1 2 3 4 5 6 7 8 9 Overall

Open the middle drawer of the cabinet Put the bowl on the stove Put the wine bottle on top of the cabinet Open the top drawer and put the bowl inside Put the bowl on top of the cabinet Push the plate to the front of the stove Put the cream cheese in the bowl Turn on the stove Put the bowl on the plate Put the wine bottle on the rack

60% 60% 0% 0% 60% 20% 0% 0% 60% 0%

100% 100% 60% 20% 100% 80% 60% 60% 100% 60%

All sampled rollouts

26%

74%

Table 9 shows that the LIME-assisted pipeline substantially improves the downstream manipulation success rate in this targeted setting, increasing overall success from 26% to 74%. The result supports the intended use of LIME as an active visual preconditioning module: it acquires a more informative wrist-camera view before the same π0.5 policy performs the manipulation. The per-task results also show that the benefit is not uniform across manipulation tasks; for example, the twostage drawer-and-placement task remains difficult even with the additional LIME viewpoint. Figure 12 shows representative sampled starts and the corresponding post-LIME wrist-camera views, illustrating how the active-perception step changes the visual evidence available before executing the manipulation policy.

24

Start External

After LIME Wrist

External

Wrist

Task 0: Open the middle drawer of the cabinet

Task 1: Put the bowl on the stove

Task 2: Put the wine bottle on top of the cabinet

Task 4: Put the bowl on top of the cabinet

Task 6: Put the cream cheese in the bowl

Figure 12: Qualitative LIBERO-Goal examples from sampled initial states. Columns are grouped by the sampled start view and the post-LIME view; each group shows the external and wrist cameras. The post-LIME views are captured before executing the downstream π0.5 manipulation policy.

25

(a) Look at the sink under the faucet.

(b) Look at the lemons on the chopping board.

(c) Find the orange juice on the dining table.

(d) Check what is under the sink.

(e) Check what is behind the wall.

(f) Get a clear view of the entire cup on the coffee table in front of you.

(g) Go to the multi-seat sofa in the living room.

Figure 11: Additional qualitative LIME prediction examples. For each row, images are ordered from the start frame to the first successful frame.

26

C.5

LIME for Embodied Question Answering (EQA) Task

LIME’s intent-aware camera-motion capability can also be used for embodied question answering (EQA). We evaluate LIME on AVS-ProcTHOR [52], an active-view-selection benchmark for visual question answering. Instead of judging whether a predicted camera trajectory satisfies an explicit intent, AVS-ProcTHOR measures whether an agent can select a view that improves question answering. This setting tests whether the camera-motion behavior learned by LIME transfers to a different simulator, action interface, and downstream objective. We apply three adaptations to deploy LIME on AVS-ProcTHOR. First, following the baselines, we use Gemini-2.5-Flash as the verifier model that answers the question from the rendered view at the predicted pose. Second, because LIME takes an intent-style instruction rather than a questionanswer input, we convert each AVS-ProcTHOR question into a view-selection instruction using its parsed target and supporting object. The conversion uses simple “look at” templates: Existence questions are converted to support-focused instructions such as “Look at the sink,” Counting questions to plural coverage instructions such as “Look at all the apples on the side table,” and State questions to target-and-support instructions such as “Look at the book on the dining table.” This preserves the visual evidence needed by the original VQA question while matching LIME’s intentconditioned camera-motion interface. Third, AVS-ProcTHOR expects a planar active-view action rather than a relative pose in 3D. We convert LIME’s predicted relative pose into the VG-AVS action format consisting of a heading rotation, forward displacement, and final view rotation. The resulting action is then executed through the original VG-AVS ProcTHOR rendering path using the AI2-THOR agent camera, matching the official AVS-ProcTHOR embodied-agent protocol. Table 10 reports the evaluation results. With LIME as the action model, the EQA agent outperforms the listed backbone, spatial-VLM, and EQA-framework baselines, while the specialized VG-AVS framework remains strongest overall. This comparison should be read in context: LIME is trained from real-world egocentric video supervision and receives no training or fine-tuning on the ProcTHOR scene distribution, the AVS-ProcTHOR question distribution, or the VG-AVS planar action format. In contrast, the VG-AVS SFT/RL models are trained directly on ProcTHOR active-VQA data with the same planar action interface used at evaluation time, so remaining below these specialized variants is expected. Table 10: Results on the AVS-ProcTHOR benchmark. Accuracy is reported in percent using Gemini-2.5-Flash as the answer verifier, following the AVS-ProcTHOR evaluation setup. LIME predicts a relative SE(3) camera motion, which we convert to the VG-AVS planar action format before rendering through the official ProcTHOR agent-camera path.

Action Model

AVS-ProcTHOR Existence

Counting

State

Average

No Action

Query view Target view

49.22 93.02

16.36 69.14

61.57 92.58

42.38 84.91

Backbone Model

Qwen2.5-VL-7B

64.34

29.74

56.55

50.21

Spatial VLMs

ViLaSR SpatialReasoner

57.95 54.65

25.46 22.68

52.84 52.62

45.42 43.32

EQA Framework

Fine-EQA

63.57

31.97

64.41

53.32

Proprietary Models

GPT-5 Gemini-2.5-Pro

81.01 82.95

55.58 52.79

79.69 81.00

72.09 72.25

VG-AVS Framework

SFT RL SFT+RL

91.28 86.82 91.47

57.06 65.24 69.52

83.84 83.41 90.17

77.39 78.49 83.72

Ours

LIME

65.50

36.62

72.71

57.41

27

Figure 13 shows qualitative examples from the three AVS-ProcTHOR question types. Each example is shown on a separate row, with two rows each for Existence, Counting, and State. LIME’s predicted view is qualitatively meaningful and exposes the relevant object or region. Within each question type, the two examples include one case where Gemini-2.5-Flash answers correctly from the rendered view and one case where it does not. This illustrates that failures can arise not only from an unhelpful predicted pose, but also from the downstream answer verifier failing to extract the correct answer from an otherwise informative view.

28

Figure 13: Qualitative AVS-ProcTHOR examples showing the start view, LIME rendered view, and held-out reference view. Each example occupies one row; rows are ordered as two Existence examples, two Counting examples, and two State examples.

29

C.6

LIME for Multi-step Robot Tasks

We further test LIME in two preliminary robot settings that reuse the same sequential camera-motion interface. First, LIME can be used for language-instructed object scanning by repeatedly prompting the robot with a fixed intent such as “look at the object from a different angle.” At each step, the robot observes the current wrist-camera image, samples multiple candidate camera motions from the flow-matching head, and executes the sample with the largest deviation from previously visited views to encourage novel coverage. This produces a multi-view scanning trajectory around the target object without explicitly training a separate scanning policy. Figure 14 shows an example scanning evaluation trajectory. Second, we evaluate mid-distance navigation by repeatedly prompting LIME with the same targetdirected command over multiple sequential camera-motion steps. Although LIME predicts only local relative camera motions, the repeated execution shows that the robot can continue making progress toward a target across several viewpoints. Figure 15 shows an example mid-distance navigation evaluation trajectory.

Figure 14: Language-instructed object scanning example. Repeated prompts and novelty-biased sampling produce a multi-view trajectory around the target. The reconstruction result on the right is generated with VGGT-Omega [70].

Figure 15: Mid-distance navigation example. Repeated target-directed prompting drives progress toward the goal over three local camera-motion iterations. The full processes for these two tasks can be found in the supplementary videos, together with additional real-world results.

References [1] R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza. Introduction to autonomous mobile robots. MIT press, 2011. [2] R. Bajcsy, Y. Aloimonos, and J. K. Tsotsos. Revisiting active perception. Autonomous Robots, 42(2):177–196, 2018. 30

[3] M. F. Ahmed, K. Masood, V. Fremont, and I. Fantoni. Active slam: A review on last decade. Sensors, 23(19):8097, 2023. [4] J. A. Placed, J. Strader, H. Carrillo, N. Atanasov, V. Indelman, L. Carlone, and J. A. Castellanos. A survey on active simultaneous localization and mapping: State of the art and new frontiers. IEEE Transactions on Robotics, 39(3):1686–1705, 2023. [5] I. Lluvia, E. Lazkano, and A. Ansuategi. Active mapping and robot exploration: A survey. Sensors, 21(7):2445, 2021. [6] K. Li, M. Mantovani, R. J. Wood, L. Sabattini, and S. Gil. Motion-uncertainty-aware nextbest-view planning for moving object reconstruction. arXiv preprint arXiv:2605.17593, 2026. [7] D. Batra, A. Gokaslan, A. Kembhavi, O. Maksymets, R. Mottaghi, M. Savva, A. Toshev, and E. Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171, 2020. [8] J. Zhang, K. Wang, S. Wang, M. Li, H. Liu, S. Wei, Z. Wang, Z. Zhang, and H. Wang. Uninavid: A video-based vision-language-action model for unifying embodied navigation tasks, 2024. [9] Y. Hong, Q. Wu, Y. Qi, C. Rodriguez-Opazo, and S. Gould. A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1643–1653, June 2021. [10] M. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. [11] B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. [12] B. Yamauchi. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’, pages 146– 151. IEEE, 1997. [13] D. S. Chaplot, M. Dalal, S. Gupta, J. Malik, and R. R. Salakhutdinov. Seal: Self-supervised embodied active learning using exploration and 3d consistency. Advances in neural information processing systems, 34:13086–13098, 2021. [14] B. Yu, H. Kasaei, and M. Cao. Frontier semantic exploration for visual target navigation. arXiv preprint arXiv:2304.05506, 2023. [15] L. Schmid, M. Pantic, R. Khanna, L. Ott, R. Siegwart, and J. Nieto. An efficient samplingbased method for online informative path planning in unknown environments. IEEE Robotics and Automation Letters, 5(2):1500–1507, 2020. [16] B. Sun, H. Chen, S. Leutenegger, C. Cadena, M. Pollefeys, and H. Blum. Frontiernet: Learning visual cues to explore. IEEE Robotics and Automation Letters, 10(7):6576–6583, 2025. doi: 10.1109/LRA.2025.3569122. [17] J. Yan, X. Lin, Z. Ren, S. Zhao, J. Yu, C. Cao, P. Yin, J. Zhang, and S. Scherer. Mui-tare: Multiagent cooperative exploration with unknown initial position. arXiv preprint arXiv:2209.10775, 2022. 31

[18] Y. Kompis, L. Bartolomei, R. Mascaro, L. Teixeira, and M. Chli. Informed Sampling Exploration Path Planner for 3D Reconstruction of Large Scenes. IEEE Robotics and Automation Letters, 6(4):7894–7901, 10 2021. ISSN 23773766. doi:10.1109/LRA.2021.3101856. [19] J. Li, B. Sun, L. D. Giammarino, H. Blum, and M. Pollefeys. Actloc: Learning to localize on the move via active viewpoint selection. In J. Lim, S. Song, and H.-W. Park, editors, Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pages 1225–1245. PMLR, 27–30 Sep 2025. URL https://proceedings.mlr.press/v305/li25b.html. [20] Z. Zhang and D. Scaramuzza. Beyond point clouds: Fisher information field for active visual localization. pages 5986–5992. IEEE, 2019. [21] M. Chang, T. Gervet, M. Khanna, S. Yenamandra, D. Shah, S. Y. Min, K. Shah, C. Paxton, S. Gupta, D. Batra, et al. Goat: Go to any thing. 2024. [22] J. Zhang, L. Dai, F. Meng, Q. Fan, X. Chen, K. Xu, and H. Wang. 3d-aware object goal navigation via simultaneous exploration and identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6672–6682, 2023. [23] Z. Zhou, Y. Hu, L. Zhang, Z. Li, and S. Chen. Beliefmapnav: 3d voxel-based belief map for zero-shot object navigation, 2025. [24] W. Xie, H. Jiang, Y. Zhu, J. Qian, and J. Xie. Naviformer: A spatio-temporal context-aware transformer for object navigation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14708–14716, 2025. [25] J. Gu, E. Stefani, Q. Wu, J. Thomason, and X. Wang. Vision-and-language navigation: A survey of tasks, methods, and future directions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7606–7623, 2022. [26] Y. Zhang, Z. Ma, J. Li, Y. Qiao, Z. Wang, J. Chai, Q. Wu, M. Bansal, and P. Kordjamshidi. Vision-and-language navigation today and tomorrow: A survey in the era of foundation models. arXiv preprint arXiv:2407.07035, 2024. [27] K. Kawaharazuka, J. Oh, J. Yamada, I. Posner, and Y. Zhu. Vision-language-action models for robotics: A review towards real-world applications. IEEE Access, 2025. [28] M. Wei, C. Wan, X. Yu, T. Wang, Y. Yang, X. Mao, C. Zhu, W. Cai, H. Wang, Y. Chen, et al. Streamvln: Streaming vision-and-language navigation via slowfast context modeling. arXiv preprint arXiv:2507.05240, 2025. [29] S. Zeng, D. Qi, X. Chang, F. Xiong, S. Xie, X. Wu, S. Liang, M. Xu, X. Wei, and N. Guo. Janusvln: Decoupling semantics and spatiality with dual implicit memory for vision-language navigation. arXiv preprint arXiv:2509.22548, 2025. [30] H. Yin, X. Xu, L. Zhao, Z. Wang, J. Zhou, and J. Lu. Unigoal: Towards universal zero-shot goal-oriented navigation. arXiv preprint arXiv:2503.10630, 2025. [31] A.-C. Cheng, Y. Ji, Z. Yang, X. Zou, J. Kautz, E. Biyik, H. Yin, S. Liu, and X. Wang. Navila: Legged robot vision-language-action model for navigation. In RSS, 2025. [32] Z. Chu, S. Xie, X. Wu, Y. Shen, M. Luo, Z. Wang, F. Liu, X. Leng, J. Hu, M. Yin, et al. Abotn0: Technical report on the vla foundation model for versatile embodied navigation. arXiv preprint arXiv:2602.11598, 2026. [33] M. Wei, C. Wan, J. Peng, X. Yu, Y. Yang, D. Feng, W. Cai, C. Zhu, T. Wang, J. Pang, et al. Ground slow, move fast: A dual-system foundation model for generalizable visionand-language navigation. arXiv preprint arXiv:2512.08186, 2025. 32

[34] E. Padilla, B. Sun, M. Pollefeys, and H. Blum. Openfrontier: General navigation with visuallanguage grounded frontiers. arXiv preprint arXiv:2603.05377, 2026. [35] Y. Long, W. Cai, H. Wang, G. Zhan, and H. Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv preprint arXiv:2406.04882, 2024. [36] D. Goetting, H. G. Singh, and A. Loquercio. End-to-end navigation with vision language models: Transforming spatial reasoning into question-answering. arXiv preprint arXiv:2411.05755, 2024. [37] M. Habibpour and F. Afghah. History-augmented vision-language models for frontier-based zero-shot object navigation. arXiv preprint arXiv:2506.16623, 2025. [38] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. π0 : A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. [39] H. Xiong, X. Xu, J. Wu, Y. Hou, J. Bohg, and S. Song. Vision in action: Learning active perception from human demonstrations. arXiv preprint arXiv:2506.15666, 2025. [40] J. Kerr, K. Hari, E. Weber, C. M. Kim, B. Yi, T. Bonnen, K. Goldberg, and A. Kanazawa. Eye, robot: Learning to look to act with a bc-rl perception-action loop. arXiv preprint arXiv:2506.10968, 2025. [41] Y. Zou, C. Shi, W. Yu, H. Xue, J. Lv, Y. Pan, C. Wen, and C. Lu. Activeglasses: Learning manipulation with active vision from ego-centric human demonstration. arXiv preprint arXiv:2604.08534, 2026. [42] Y. Wang, C. Qian, R. Fan, and E. Johns. Observer actor: Active vision imitation learning with sparse view gaussian splatting. arXiv preprint arXiv:2511.18140, 2025. [43] Z. Liu, Y. Gu, Y. Wang, X. Xue, and Y. Fu. Activevla: Injecting active perception into visionlanguage-action models for precise 3d robotic manipulation. arXiv preprint arXiv:2601.08325, 2026. [44] Y. Huang, Z. Wang, W. Tang, C. Lu, and P. Cai. I-perceive: A foundation model for active perception with language instructions, 2026. URL https://arxiv.org/abs/2603.00600. [45] J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee. Beyond the nav-graph: Visionand-language navigation in continuous environments. arXiv preprint arXiv:2004.02857, 2020. [46] M. Khanna, R. Ramrakhya, G. Chhablani, S. Yenamandra, T. Gervet, M. Chang, Z. Kira, D. S. Chaplot, D. Batra, and R. Mottaghi. Goat-bench: A benchmark for multi-modal lifelong navigation. arXiv preprint arXiv:2404.06609, 2024. [47] N. Yokoyama, R. Ramrakhya, A. Das, D. Batra, and S. Ha. HM3D-OVON: A dataset and benchmark for open-vocabulary object goal navigation. arXiv preprint arXiv:2409.14296, 2024. [48] A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra. Embodied question answering, 2017. URL https://arxiv.org/abs/1711.11543. [49] A. Majumdar, A. Ajay, X. Zhang, P. Putta, S. Yenamandra, M. Henaff, S. Silwal, P. Mcvay, O. Maksymets, S. Arnaud, K. Yadav, Q. Li, B. Newman, M. Sharma, V. Berges, S. Zhang, P. Agrawal, Y. Bisk, D. Batra, M. Kalakrishnan, F. Meier, C. Paxton, S. Sax, and A. Rajeswaran. Openeqa: Embodied question answering in the era of foundation models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 33

[50] A. Z. Ren, J. Clark, A. Dixit, M. Itkina, A. Majumdar, and D. Sadigh. Explore until confident: Efficient exploration for embodied question answering. In Robotics: Science and Systems, 2024. [51] K. Jiang, Y. Liu, W. Chen, J. Luo, Z. Chen, L. Pan, G. Li, and L. Lin. Beyond the destination: A novel benchmark for exploration-aware embodied question answering. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025. [52] J. Koo, D. Choi, S. Youn, P. Y. Lee, and M. Sung. Toward ambulatory vision: Learning visually-grounded active view selection, 2025. URL https://arxiv.org/abs/2512. 13250. [53] K. Sakamoto, T. Miyanishi, D. Azuma, S. Kurita, S. Morikuni, N. Chiba, M. Kawanabe, Y. Iwasawa, and Y. Matsuo. E3vs-bench: A benchmark for viewpoint-dependent active perception in 3d gaussian splatting scenes, 2026. URL https://arxiv.org/abs/2604.17969. [54] S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, W. Ge, Z. Guo, Q. Huang, J. Huang, F. Huang, B. Hui, S. Jiang, Z. Li, M. Li, M. Li, K. Li, Z. Lin, J. Lin, X. Liu, J. Liu, C. Liu, Y. Liu, D. Liu, S. Liu, D. Lu, R. Luo, C. Lv, R. Men, L. Meng, X. Ren, X. Ren, S. Song, Y. Sun, J. Tang, J. Tu, J. Wan, P. Wang, P. Wang, Q. Wang, Y. Wang, T. Xie, Y. Xu, H. Xu, J. Xu, Z. Yang, M. Yang, J. Yang, A. Yang, B. Yu, F. Zhang, H. Zhang, X. Zhang, B. Zheng, H. Zhong, J. Zhou, F. Zhou, J. Zhou, Y. Zhu, and K. Zhu. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. [55] Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li. On the continuity of rotation representations in neural networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5738–5746, 2019. doi:10.1109/CVPR.2019.00589. [56] T. Li and K. He. Back to basics: Let denoising generative models denoise. arXiv preprint arXiv:2511.13720, 2025. [57] K. Grauman, A. Westbury, L. Torresani, K. Kitani, J. Malik, T. Afouras, K. Ashutosh, V. Baiyya, S. Bansal, B. Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19383–19400, 2024. [58] L. Ma, Y. Ye, F. Hong, V. Guzov, Y. Jiang, R. Postyeni, L. Pesqueira, A. Gamino, V. Baiyya, H. J. Kim, et al. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In European Conference on Computer Vision, pages 445–465. Springer, 2024. [59] D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, et al. The epic-kitchens dataset: Collection, challenges and baselines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):4125– 4141, 2020. [60] R. Zheng, D. Niu, Y. Xie, J. Wang, M. Xu, Y. Jiang, F. Castañeda, F. Hu, Y. L. Tan, L. Fu, et al. Egoscale: Scaling dexterous manipulation with diverse egocentric human data. arXiv preprint arXiv:2602.16710, 2026. [61] S. Kareer, D. Patel, R. Punamiya, P. Mathur, S. Cheng, C. Wang, J. Hoffman, and D. Xu. Egomimic: Scaling imitation learning via egocentric video. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13226–13233. IEEE, 2025. [62] H. Lin, S. Chen, J. H. Liew, D. Y. Chen, Z. Li, G. Shi, J. Feng, and B. Kang. Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647, 2025. [63] M. Han, L. Ma, K. Zhumakhanova, E. Radionova, J. Zhang, X. Chang, X. Liang, and I. Laptev. Roomtour3d: Geometry-aware video-instruction tuning for embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27586– 27596, 2025. 34

[64] M. T. I. SpatialVerse Research Team. Interiorgs: A 3d gaussian splatting dataset of semantically labeled indoor scenes. https://huggingface.co/datasets/spatialverse/ InteriorGS, 2025. [65] C. Yeshwanth, Y.-C. Liu, M. Nießner, and A. Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023. [66] H. Chen, B. Sun, A. Zhang, M. Pollefeys, and S. Leutenegger. VidBot: Learning generalizable 3d actions from in-the-wild 2d human videos for zero-shot robotic manipulation. 2025. [67] R. Wang, S. Xu, Y. Dong, Y. Deng, J. Xiang, Z. Lv, G. Sun, X. Tong, and J. Yang. Moge2: Accurate monocular geometry with metric scale and sharp details. Advances in Neural Information Processing Systems, 38:35928–35959, 2026. [68] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URL https://arxiv.org/abs/2306. 03310. [69] Physical Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y. Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. Vuong, H. Walke, A. Walling, H. Wang, L. Yu, and U. Zhilinsky. π0.5 : a vision-languageaction model with open-world generalization, 2025. URL https://arxiv.org/abs/2504. 16054. [70] J. Wang, M. Chen, S. Zhang, N. Karaev, J. Schönberger, P. Labatut, P. Bojanowski, D. Novotny, A. Vedaldi, and C. Rupprecht. VGGT-Ω. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

35