arXiv:2606.18235v1 [cs.AI] 16 Jun 2026
EvolveNav: Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation Qi Chai∗
Wenhao Shen∗
Nanjie Yao
Yue Xia
HKUST(GZ) Guangzhou, China [email protected]
Nanyang Technological University Singapore, Singapore [email protected]
HKUST(GZ) Guangzhou, China [email protected]
Xi’an Jiaotong University Xi’an, China [email protected]
Kaiyong Zhao
Jie Ma
Guosheng Lin
Hao Wang†
XGRIDS Shenzhen, China [email protected]
Xi’an Jiaotong University Xi’an, China [email protected]
Nanyang Technological University Singapore, Singapore [email protected]
HKUST(GZ) Guangzhou, China [email protected]
Figure 1: Method Comparison. Left: Our agent learns from (1) past episode history to dynamically update (2) evolved rules. During navigation, it adopts (3) preflection to proactively avoid rule-violating or unpromising rooms, while continuously (4) evolving its rule set from trajectories. Right: Baseline method relies on passive reaction, leading to inefficient exploration.
Abstract Zero-Shot Object-Goal Navigation (ZS-OGN) requires embodied agents to explore and locate target objects without any prior training. To this end, recent methods leverage foundation models. But they typically rely on static priors and lack adaptation, which leads to repeated errors and costly trial and error. In this paper, we propose a self-evolving ZS-OGN framework that enables continuous test-time improvement. Specifically, we build an agentic rule memory by extracting actionable knowledge from past trajectories. Then, we propose a retrieval strategy based on upper confidence bound,
∗ Equal Contribution. † Corresponding author.
selecting effective rules by balancing semantic relevance and historical success. In addition, we introduce a memory-guided preflection module that forecasts potential outcomes before action, reducing inefficient exploration. Extensive experiments show that our method outperforms existing zero-shot baselines, achieving a 10.1% improvement in success rate with fewer unnecessary steps.
1
Introduction
Object-Goal Navigation (OGN) is a fundamental task in embodied artificial intelligence [32, 35]. It requires an agent to perceive, reason, and act within an unseen space and locate a specific target. Existing OGN methodologies fall into training-based and trainingfree paradigms. Training-based methods [10, 12, 31, 36, 40, 41], relying on Reinforcement Learning (RL) or Imitation Learning (IL),
Qi Chai, Wenhao Shen, Nanjie Yao, Yue Xia, Kaiyong Zhao, Jie Ma, Guosheng Lin, and Hao Wang
demand massive datasets or extensive simulation. This leads to sample inefficiency and poor generalization in unseen environments. To address this, recent training-free methods [4, 15, 50, 58] leverage foundation models like Large Language Models and VisionLanguage Models (LLMs/VLMs) as semantic planners. However, these foundation models only provide general and fixed semantic priors. As a result, Zero-Shot OGN (ZS-OGN) agents cannot adapt to real-time feedback and often repeat identical mistakes, e.g., being consistently misled by visually similar objects. Another key limitation is the high cost of trial and error. Under limited step budgets, each action is expensive. Existing methods often rely on post-hoc correction [20, 37], where errors are detected only after execution, leading to wasted steps. To improve efficiency, agents must anticipate outcomes before acting and avoid low-value decisions in advance. Therefore, this paper aims to resolve a critical challenge in OGN: how to jointly achieve online adaptation and action anticipation, under limited test-time step budgets. Specifically, we propose a self-evolving and training-free OGN framework EvolveNav. The key idea is to transform navigation into a continuous self-improvement process, where the agent learns actionable navigation rules from past successes and failures. To this end, we introduce a self-evolving agentic rule memory. After each episode, the agent analyzes its trajectory and visual observations. It then extracts concrete rules, such as recognizing misleading layouts or identifying effective search patterns. This memory is updated across test-time episodes in a dynamic manner, which ensures the accumulated knowledge remains relevant and reliable. To utilize this knowledge during exploration, we design a rule retrieval strategy based on Upper Confidence Bound (UCB). Each rule is scored by balancing two factors: its semantic relevance to the current scene and its historical success rate. This allows the agent to select rules that are both contextually appropriate and empirically reliable. As a result, the agent continuously improves its behavior through experience, by updating the UCB scores only. In addition, to address the high cost of ZS-OGN agent trial and error, we propose a memory guided preflection module. Instead of correcting mistakes after actions are executed, the agent evaluates candidate waypoints before acting. It retrieves relevant rules and past failure cases to anticipate future observations and potential risks. This process enables the agent to filter out low-value or highrisk actions in advance, leading to more efficient and goal-directed exploration in unseen environments. Our main contributions are summarized as follows: • We introduce a self-evolving agentic rule memory to support online experience accumulation. With a UCB-based strategy that balances semantic relevance and historical success, the agent continuously refines its behavior. • We propose a memory-guided preflection module that shifts the ZS-OGN task from passive correction to proactive risk avoidance. By anticipating potential failures before execution, it effectively mitigates physical trial-and-error costs. • Extensive experiments show our method significantly outperforms zero-shot baselines, notably achieving a 10.1% success rate improvement on MP3D while minimizing redundant exploration.
2 Related Work 2.1 Object Goal Navigation Object goal navigation requires an agent to navigate unknown environments to find specified target objects. Early approaches primarily relied on end-to-end RL [52] or modular learning [7]. While RL-based methods directly map observations to actions, they often suffer from poor generalization and sim-to-real gaps. Modular methods mitigate training costs by building semantic maps but still struggle to generalize to unseen environments without task-specific training. Recently, LLMs [2, 34] and VLMs [22] have been introduced to enable zero-shot OGN. Methods like ZSON [28], CoW [14], and VLFM [54] utilize VLMs to evaluate semantic similarity between frontiers and the target to guide exploration. Meanwhile, approaches like L3MVN [56] and SG-Nav [53] leverage LLMs and 3D scene graphs for high-level spatial reasoning and long-horizon planning. To further improve efficiency and robustness in complex real-world scenarios, recent works have made significant advancements. ApexNav [58] introduces an adaptive exploration strategy balancing semantic and geometric exploration, along with a targetcentric semantic fusion to reduce visual misdetections of similar objects. MSGNav [18] constructs a multi-modal 3D scene graph that retains images as edge features, preventing visual information loss and enhancing robust open-vocabulary reasoning. To address the single-floor limitation, ASCENT [15] proposes an online crossfloor navigation framework featuring a hierarchical stair-aware topology and an LLM-driven coarse-to-fine reasoning module for efficient multi-floor exploration. However, these methods rely on fixed priors during deployment and struggle to adapt to real-time feedback, often repeating mistakes.
2.2
Self-evolution in Agents
LLM agents have demonstrated strong capabilities across a wide range of domains [5, 13, 27, 60]. In recent years, investigating the capacity for self-evolution within autonomous agents has garnered significant attention in recent research. Existing self-evolving techniques can be broadly divided into two main areas: prompt optimization and memory optimization. For prompt optimization, methods [38, 42, 43, 51, 61, 62] aim to automatically search for highquality instructions to guide language models toward producing more accurate outputs. For instance, PromptAgent [38] formulates prompt optimization as a strategic planning problem and leverages Monte Carlo tree search to efficiently navigate the expert-level prompt space. Similarly, OPRO [49] generates new instructions by prompting the model with previously generated candidates to iteratively refine prompt quality. On the other hand, memory optimization methods [8, 11, 39, 64] focus on managing contextual information and past experiences to support long-term decision-making and consistency. Reflexion [33] enables agents to reflect on task feedback and store episodic insights, which promotes continuous self-improvement over time without weight updates. Concurrently, MemoryBank [64] hierarchically summarizes events and dynamically updates memory based on recency and relevance to mitigate forgetting and enhance longterm retention. Despite these advancements, self-evolution remains under-explored in OGN. Current frameworks lack the capacity
EvolveNav : Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation
Intra-Episode Preflective Exploration
Inter-Episode Self-evolving Historical Analysis
Observation RGB
Depth Segmentation Semantic Map
Candidate Frontiers (f1, f2, f3) Semantic (e.g., Doorway)
Steps
Episode History
LLM
Key Steps Extraction
Credit Initiation Preflection
Δi = simi – simi-1 Credit =∑iΔi (i ∈ key step)
Thought f1: Wrong room, dead-end, no target
What’s the risk of frontiers…
Action
Thought f2: Wrong room, view blocked, no target
Key Steps Normal Steps
New Rule Generation Prioritize living room while searching for couch.
Semantic Similarity with target
Thought f3: Wrong room, no target
UCB Weight Update
Execution
𝑤𝑖 = 𝜇𝑖 +
Move to f3
Decision Making
Rule Bank (Rules & Weights)
Retrieve Top-K Rules
Credit
ln 𝑡𝑖 𝑁𝑖
Success rate
Figure 2: Method Overview of EvolveNav. The architecture consists of two core components for continuous optimization. Left (Intra-Episode Preflective Exploration): During test-time execution, the agent evaluates frontiers (𝑓1, 𝑓2, 𝑓3, . . . , 𝑓𝑛 ) by retrieving top-𝐾 rules from the rule bank for LLM-driven preflection. This assesses risks to determine the optimal direction, minimizing trial-and-error. Right (Inter-Episode Rule Self-Evolution): After each navigation episode, the framework distills new rules from the trajectory history. A UCB-based algorithm updates rule weights by balancing credit assignment and exploration potential, continuously refining the rule bank. to internalize exploration experiences, causing agents to repeat navigational errors with limited generalization.
3 Methodology 3.1 Problem Formulation In this paper, we focus on the zero-shot object-goal navigation (ZSOGN) task. Unlike traditional learning-based paradigms that rely on massive environment interactions to learn navigation policies, the training-free paradigm requires the agent to navigate in unstructured environments and locate an instance of a specific target object category 𝑐 (e.g., “couch”), without any fine-tuning. Formally, we follow the common practice [15, 18, 58] for the ZS-OGN task evaluation, an episode begins with the agent placed at an initial pose 𝑝 0 within an unexplored scene. At any given time step 𝑡, the agent captures an egocentric RGB-D image 𝐼𝑡 = 𝑟𝑔𝑏 𝑑𝑒𝑝𝑡ℎ ⟨𝐼𝑡 , 𝐼𝑡 ⟩ and obtains its spatial tracking pose 𝑝𝑡 = (𝑥𝑡 , 𝑦𝑡 , 𝜃 𝑡 ). Relying on these sensory inputs alongside its maintained memory state, the agent determines and performs a discrete action 𝑎𝑡 ∈ A. Specifically, the permissible action space A is defined as a set of six distinct commands: MOVE_FORWARD, TURN_LEFT, TURN_RIGHT, LOOK_UP, LOOK_DOWN, and STOP. A navigation episode ends when the agent executes the STOP action or exceeds the maximum step
limit 𝑇 . The task succeeds if the agent stops within 1 meter of the visible target object 𝑐. Otherwise, the episode is considered a failure.
3.2
Overview
In time-constrained training-free navigation, an agent must minimize inefficient physical trial and error and continuously selfimprove by leveraging historical successes and failures. To achieve this without any environment-specific parameter fine-tuning, we propose a novel navigation framework that tightly couples realtime spatial exploration with a self-evolving cognitive memory. As illustrated in Figure 2, our system is composed of two primary components: Intra-Episode Preflective Exploration and Inter-Episode Rule Self-Evolution: • Intra-Episode Preflective Exploration. Within a navigation episode, the agent actively constructs a 2D environment map and extracts candidate frontiers. Before moving, an LLM retrieves past empirical rules from our constructed memory. It then performs “preflection” to evaluate the potential risks of each frontier and select the optimal exploration direction. (Section 3.5) • Inter-Episode Rule Self-Evolution. After each episode, the agent performs a post-hoc analysis to enable continuous learning. It uses the LLM to review the trajectory, identify key decisions, and distill them into generalizable navigation rules. To manage
Qi Chai, Wenhao Shen, Nanjie Yao, Yue Xia, Kaiyong Zhao, Jie Ma, Guosheng Lin, and Hao Wang
this growing rule bank, we apply a UCB algorithm to dynamically balance the exploitation of proven rules and the exploration of new ones (Section 3.4).
3.3
Spatial Mapping and Low-Level Control
To effectively navigate in unknown and unstructured environments, the agent must first establish a geometric understanding of the surrounding physical space and possess reliable low-level locomotion capabilities. At each time step 𝑡, the agent leverages egocentric RGB-D observations and odometry to incrementally project 3D depth data into a global coordinate frame, dynamically maintaining a 2D Occupancy Grid Map. By mapping 3D data onto a 2D plane, this projection reduces computational complexity while retaining critical information regarding traversable areas and obstacle positioning. Based on this occupancy map, the system continuously extracts a set of candidate frontiers. A frontier is defined as the geometric boundary separating known free space from unexplored regions. By explicitly identifying these frontiers, the agent can systematically target the most informative edges of its current knowledge, ensuring that exploration is driven by spatial uncertainty rather than random wandering. Once a target coordinate is determined by the LLM planner (detailed in Section 3.5) or identified as the target object 𝑐 by a pretrained vision model, it becomes a 2D local waypoint. A low-level navigation controller [63] then navigates the agent to this waypoint. This hierarchical design enhances system efficiency by selectively invoking the LLM only when necessary, thereby minimizing redundant API calls. To prevent the agent from getting stuck, the controller executes movement for a maximum of 𝐻 steps. Execution stops early if the frontier is fully explored, the waypoint becomes unreachable, or a visual trigger starts a new decision cycle. This step bound ensures the agent stays reactive to new observations and recovers quickly from failures.
3.4
Self-evolving Agentic Rule Memory
In zero-shot navigation, agents lack the ability to update their model parameters, making them highly susceptible to repeating the same mistakes across different environments. To achieve continuous self-improvement without fine-tuning, our framework introduces a post-episode reflection mechanism that distills low-level trajectory data into highly generalizable cognitive rules. Regardless of whether an episode ends in success or failure, the system compiles the agent’s complete history of frontier decisions and executes a two-stage rule generation pipeline. First, the LLM reviews the global trajectory to isolate the pivotal decision steps that determined the episode’s outcome. Subsequently, a secondary LLM query analyzes the local observations and decision contexts of these specific steps to synthesize reusable navigation rules. Finally, these newly formulated rules are deposited into the rule memory bank M for future invocation and dynamic weight evaluation. 3.4.1 Semantic-driven Credit Assignment. Distilling high-quality rules from a lengthy exploration trajectory requires identifying exactly which actions led to the final outcome. We must therefore assign accurate “credit” to each step, mathematically quantifying its specific contribution and importance. Because a single episode can
encompass dozens of steps, simply treating all actions equally is ineffective. Inspired by information potential theory [44], we propose a credit assignment mechanism based on sequential improvements in semantic similarity. For the 𝑖-th step in the trajectory, we calculate a composite semantic similarity score S𝑖 between the currently observed scene and the target object, reflecting the potential value of that step: (𝑖 ) (𝑖 ) S𝑖 = 𝛼 · Sroom + (1 − 𝛼) · Sobj ,
(1)
(𝑖 ) (𝑖 ) where Sroom and Sobj represent the cosine similarities of the cur-
rently observed room type and the visible object set against the target category, respectively, and 𝛼 serves as a balancing coefficient. The initial credit 𝐶𝑖 assigned to the 𝑖-th step is then defined as the marginal gain in semantic similarity relative to the preceding step: 𝐶𝑖 = S𝑖 − S𝑖 −1 .
(2)
To differentiate the reward distribution between successful and failed episodes, we apply a clipping operation to 𝐶𝑖 to prevent extreme values and compute a normalized weight 𝑤𝑖 using a softmax function. Ultimately, the initial support score 𝑆𝑟 assigned to the newly generated rule 𝑟 is defined as the weighted sum of the contributions from its associated key steps: ∑︁ 𝑆𝑟 = 𝑤 𝑖 · 𝐶𝑖 . (3) 𝑖
3.4.2 Dynamic Memory Management via UCB. As the agent continuously explores multiple environments, the rule memory bank naturally expands. Given the prohibitive physical trial-and-error costs associated with training-free navigation, the system must rigorously balance the exploitation of historically proven rules with the exploration of newly formulated, unverified ones. To mathematically formalize this trade-off, we treat rule retrieval as a multi-armed bandit problem and employ a UCB management strategy. For any rule 𝑟 ∈ M, its expected utility 𝜇𝑟 combines historical performance with the latest episodic feedback. This utility is iteratively smoothed using a momentum-based update to incorporate the newly computed support score 𝑆𝑟 : 𝜇𝑟 ← 𝜂 · 𝜇𝑟 + (1 − 𝜂) · 𝑆𝑟 ,
(4)
where 𝜂 ∈ [0, 1] serves as a momentum decay coefficient. Prior to initiating a new navigation episode, the framework computes the UCB score for all rules residing in the memory bank: √︄ ln𝑇 UCB(𝑟 ) = 𝜇𝑟 + 𝛽 , (5) 𝑁𝑟 where 𝑇 represents the total number of exploration episodes the agent has completed thus far, 𝑁𝑟 is the historical frequency with which rule 𝑟 has been retrieved and applied, and 𝛽 is a hyperparameter modulating the exploration degree. Crucially, to ensure that newly generated rules receive a guaranteed initial validation phase, their UCB scores are initialized to infinity (∞) when 𝑁𝑟 = 0. Finally, the memory bank is sorted in descending order based on these computed UCB(𝑟 ) scores. The top-𝐾 rules are extracted and injected as experiential priors into the LLM’s context during the preflection phase (as detailed in Section 3.5), thereby successfully completing the self-improving cognitive loop of the framework.
EvolveNav : Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation
3.5
Memory-Guided Preflection
As established in Section 3.3, the mapping module identifies a set of candidate frontiers at each decision step. In environments with constrained step budgets, relying on greedy distance-based strategies or static commonsense priors often traps the agent in local optima or leads to repetitive exploration of irrelevant regions. To circumvent these prohibitively expensive physical trial-and-error costs, we propose an LLM-based memory-guided preflection mechanism. This module enables the agent to proactively anticipate and avoid potential failures before executing irreversible physical actions. To implement this, we construct a structured context prompt that integrates geometric distances with local semantic observations, such as visible objects and spatial layouts for each candidate. Unlike traditional training-free methods that rely exclusively on fixed commonsense priors, which often struggle with atypical environments, our approach dynamically injects the agent’s self-summarized decision memory into this reasoning context Specifically, at each decision stage, the system queries a selfevolving rule bank M (detailed in Section 3.4) to retrieve the top-𝐾 most relevant experiential rules. To balance the exploitation of reliable past rules with the exploration of newly formed insights, the retrieval is governed by an Upper Confidence Bound (UCB) selection criterion: R top𝐾 = TopK𝑟 ∈ M UCB(𝑟 ) , (6) where UCB(𝑟 ) denotes the confidence score of rule 𝑟 . By grounding the prompt in these dynamically retrieved rules, the agent leverages its own historical lessons to avoid repeating systemic mistakes. To mitigate hallucinations and impulsive decisions, we replace direct target selection with an explicit reasoning process. The LLM must first perform failure prediction for each candidate frontier by synthesizing injected rules with current observations. For example, it might predict a high risk of failure if a frontier’s visual cues conflict with established spatial rules. This mandatory risk assessment filters out deceptive paths, allowing the LLM to balance failure probabilities against exploration rewards to select the optimal, risk-averse frontier.
4 Experiment 4.1 Experimental Setup 4.1.1 Datasets. We evaluate our proposed method on two standard ObjectNav benchmarks derived from the Habitat Challenge. Specifically, we utilize the HM3D-Semantics-v0.1 dataset from the 2022 Habitat challenge [45], which comprises 2,000 evaluation episodes across 20 distinct scenes and 6 goal categories. Additionally, we evaluate on the updated MP3D dataset from the 2021 Habitat challenge [6], consisting of 2195 episodes, 11 scenes, 21 goal categories episodes distributed over the 11 scenes. 4.1.2 Evaluation Metrics. Following the standard evaluation protocol established in prior ObjectNav methods [58], we employ Success Rate (SR) and Success weighted by Path Length (SPL) to comprehensively evaluate the performance of our proposed method. SR measures the fundamental navigation efficacy, defined as the fraction of episodes where the agent successfully navigates to the target object and executes the stop action. SPL further assesses navigation
efficiency by weighting the binary success indicator of each episode against the ratio of the shortest path distance to the actual agent trajectory length, thereby penalizing circuitous routes. 4.1.3 Baselines. To evaluate the effectiveness of our proposed approach, we compare it against a comprehensive set of state-of-theart baselines, which are broadly categorized into learning-based and zero-shot methods. The learning-based category includes RIM [9], OVG-Nav [55], PIRLNav [31], and XGX [40], which heavily rely on task-specific training architectures to achieve navigation. In the zero-shot category, we evaluate foundational methods such as ZSON [28] and ActPept [16], which utilize contrastive models like CLIP or graph-based structures for semantic guidance, alongside progressive frameworks that incorporate LLMs and VLMs for explicit instruction interpolation. Specifically, we compare against L3MVN [56], PixNav [4], SG-Nav [53], OpenFMNav [21], and InstructNav [26], which integrate various iterations of GPT architectures to interpret language and guide exploration. Furthermore, we benchmark against recent advanced zero-shot frameworks including VLFM [54] and ApexNav [58], which employ BLIP-2 [22] paired with robust models like DeepSeek-V3 [23], as well as the latest state-of-the-art methods such as MFNP [57], ASCENT [15], and MSGNav [18], which leverage the Qwen model family [2, 46– 48] to construct reliable visual-linguistic mappings for complex navigation tasks. 4.1.4 Implementation Details. For the multimodal reasoning backbone, we employ the pre-trained BLIP-2 [22] as our vision foundation model and Qwen3-8B [46] as the language model instruction interpolator. In strict adherence to the zero-shot object-goal navigation paradigm, all network parameters remain entirely frozen during deployment, with no task-specific or environment-specific fine-tuning applied. The core functional components, namely the memory-guided preflection module and the self-evolving agentic rule memory, operate purely at inference time through dynamic prompt injection and UCB-based rule updating. Following established evaluation protocols from prior works [3, 15, 18], we set the success distance thresholds to 1.0 meters for both benchmarks. Regarding the computational infrastructure, all simulated experiments, environment rendering, and large model inference processes are executed on a single high-performance workstation equipped with a AMD EPYC 7542 32-Core Processor and 2 NVIDIA RTX A6000 GPUs. Additional hyperparameter configurations and further implementation details are provided in the Supplementary Material.
4.2
Quantitative Evaluation
We compare our proposed approach against state-of-the-art learningbased and zero-shot object goal navigation methods on the widely adopted HM3D and MP3D benchmarks. As demonstrated in Table 1, our method achieves state-of-the-art performance across both datasets, consistently outperforming existing training-free approaches. On the HM3D dataset, our method sets a new benchmark with an SR of 67.3% and a Success weighted by SPL of 33.9%, surpassing recently proposed frameworks such as MSGNav and ASCENT by 1.9% in SR and 0.4% in SPL. The performance gains are even more significant on the highly challenging MP3D dataset, which is
Qi Chai, Wenhao Shen, Nanjie Yao, Yue Xia, Kaiyong Zhao, Jie Ma, Guosheng Lin, and Hao Wang
Table 1: Comparisons with SOTA methods. The table compares learning-based and zero-shot methods on HM3D and MP3D datasets across the “Success Rate” (SR) and “Success Weighted by Path Length” (SPL) metrics. We introduce columns for Vision and Language models to specify the Instruction Interpolator components used by each approach. All baseline results from their original publications. Arrow “↑” denotes higher is better. Our method achieves state-of-the-art performance among zero-shot approaches, yielding the highest SR and SPL on both benchmarks.
Method
Venue
Zero-shot
RIM [9] OVG-Nav [55] VLFM [54] PIRLNav [31] XGX [40]
IROS’23 RAL’24 ICRA’24 CVPR’23 ICRA’24
ZSON [28] L3MVN [56] PixNav [4] VLFM [54] SG-Nav [53] OpenFMNav [21] ActPept [16] InstructNav [26] ApexNav [58] MFNP [57] MSGNav [18] ASCENT [15] EvolveNav (Ours)
Instruction Interpolator
HM3D [30]
MP3D [6]
Vision
Language
SR ↑
SPL ↑
SR ↑
SPL ↑
✗ ✗ ✗ ✗ ✗
BLIP-2 [22] -
-
57.8 52.5 64.1 72.9
27.2 30.4 27.1 35.7
50.3 35.8 36.4 -
17.0 12.3 17.5 -
NeurIPS’22 IROS’23 ICRA’24 ICRA’24 NeurIPS’24 NAACL-F’24 RAL’24 CoRL’24 RAL’25 ICRA’25 CVPR’26 RAL’26
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
CLIP [29] LLaMA-Adapter [59] BLIP-2 [22] LLaVA-1.6-7B [24] GPT-4V [1] GraphSAGE [17] GPT-4V [1] BLIP-2 [22] Qwen-VLChat-Int4 [2] GPT-4o [19] BLIP-2 [22]
RoBERTa-large [25] GPT-4 [1] GPT-4 [1] GPT-4 [1] GPT-4 [1] DeepSeek-V3 [23] Qwen2-7B [47] GPT-4o [19] Qwen2.5-7B [48]
25.5 50.4 37.9 50.9 54.0 54.9 58.0 59.6 58.3 63.0 65.4
12.6 23.1 20.5 23.6 24.9 24.4 20.9 33.0 26.7 31.4 33.5
15.3 34.9 32.5 40.2 37.2 39.8 39.2 41.1 44.5
4.8 14.5 15.9 16.0 15.7 17.4 17.8 15.4 15.5
-
✓
BLIP-2 [22]
Qwen3-8B [46]
67.3
33.9
49.0
19.1
characterized by larger spatial scales and complex multi-floor layouts. In this environment, our method achieves an impressive 49.0% SR and 19.1% SPL, yielding a substantial absolute improvement of 4.5% in SR and 3.8% in SPL over the strongest zero-shot baselines. This significant performance improvement on MP3D highlights a critical advantage of our framework. While prior zero-shot methods often experience severe performance degradation when transitioning from HM3D to more visually and geometrically complex scenes due to their reliance on fixed semantic priors, our self-evolving agentic rule memory effectively mitigates this generalization gap. By continuously accumulating and dynamically retrieving experiential rules via a UCB-based mechanism, the agent adapts to unseen environments and corrects its own behaviors online without requiring any parameter updates. Furthermore, the consistently superior SPL across both benchmarks indicates that our method not only locates targets more reliably but also generates highly efficient, purposeful navigation trajectories. Rather than relying on post-hoc corrections after irreversible actions, our memory-guided preflection module empowers the foundation model to forecast failure risks and filter out low-value candidate frontiers prior to physical execution. This risk avoidance successfully prevents the exhaustive and redundant exploration loops that frequently constrain conventional zero-shot implementations, significantly reducing wasteful physical trial-and-error steps.
Table 2: Ablation studies on core modules. We ablate the Preflection, Memory-Evolving modules to validate their effectiveness on the HM3D and MP3D datasets. Setting
4.3
Module Preflection
1 2 3
✓
4
✓
HM3D
MP3D
Memory-Evolve
SR ↑
SPL ↑
SR ↑
SPL ↑
✓
64.7 66.5 66.7
32.5 33.5 33.6
43.9 47.4 48.3
15.8 18.4 18.7
✓
67.3
33.9
49.0
19.1
Ablation Studies
4.3.1 Effectiveness of Preflection. To verify the effectiveness of the proposed preflection, we disable it during inference. As shown in Table 2, integrating the memory-guided preflection module (Setting 2) yields a noticeable performance boost over the baseline (Setting 1), improving the SR by 0.8% on HM3D and 3.5% on the more challenging MP3D dataset. Notably, we also observe a steady increase in Success weighted by SPL. This validates our core hypothesis that shifting from reactive, post-hoc correction to prior-to-action anticipation significantly mitigates the prohibitive costs of physical trial-and-error. By forcing the LLM to explicitly predict frontier failure risks before committing to irreversible physical actions, the agent effectively filters out deceptive local optima and avoids dead ends. This proactive risk avoidance allows the agent to conserve its limited step budget, resulting in the generation of more purposeful and highly efficient navigation trajectories.
EvolveNav : Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation
Rules for this episode：
1. Don't avoid the most obvious places like living room. 2. Avoid relying on indirect associations with spa, jacuzzi as these do not logically indicate the presence of a potted plant. 3. Avoid areas with no clear indicators as you may stuck.
Task: Find a [Potted Plant]
Rule 3 ↑ Episode Summary
Frontier 2
Target
Frontier 1
Rule 2 ↑
Frontier 2
Key Choice: Key Rules:
Step 106 & 121 Rule 2 & Rule 3
Final Results:
Success
New Rule (For future episodes) Frontier 3 Frontier 2
Frontier 1
Frontier 3
When looking for a potted plant, prioritize areas containing furniture such as armchairs.
Frontier 1
Step 22 Preflection
For Frontier 1： May incorrectly assume a potted plant could be near spa items. For Frontier 2： Likely to miss the potted plant due to focus on spa items. For Frontier 3： Unlikely to find a potted plant in a hall setting.
Decision
Frontier 3. Based on learned heuristics, avoiding areas with spa, jacuzzi, and bathroom accessories, and since Frontier 3 has no prior information, it is the least likely to contain a potted plant but is the safest choice given the heuristics.
Step 121
Step 106 Preflection
For Frontier 1： May incorrectly assume potted plants are common in bathrooms. For Frontier 2： Could lead to another Area. For Frontier 3： No clear indicators may lead to stuck
Decision Frontier 2. Frontier 2 contains elements like armchairs and chairs which are more likely to co-occur with a potted plant compared to spa and bathroom accessories in Frontier 1 or no clear indicators in Frontier 3.
Step 149
Preflection
For Frontier 1： The potted plant may be a visual misperception. For Frontier 2： Typical office furniture are not related to potted plant.
Final Observation
Decision Frontier 1. Frontier 1 already contains the potted plant. It is better than another frontier which seems not related to the potted plant.
Figure 3: Navigation process visualization of EvolveNav. We provide the agent’s trajectory alongside its reasoning process. During the episode, the agent utilizes Preflection to evaluate frontiers and proactively avoid unpromising rooms. After the episode, EvolveNav generates a new rule from the history and updates its Rule Bank. It illustrates how EvolveNav autonomously distills long-term navigation knowledge from short-term trial and error. 4.3.2 Effectiveness of Memory and Evolving. To evaluate the impact of continuous online experience accumulation, we isolate the self-evolving agentic rule memory alongside its UCB-based weight updating mechanism (Setting 3 in Table 2). Compared to the static baseline, empowering the agent to formulate and retrieve rules without explicit preflective reasoning still achieves an absolute SR improvement of 2.0% on HM3D and a substantial 4.5% on MP3D. The significant performance gain on the MP3D dataset highlights a critical bottleneck in existing zero-shot paradigms: the widely used static priors heavily relied upon by foundation models frequently fail in geometrically complex or unusual real-world environments. By dynamically extracting actionable rules from historical successes and failures via semantic-driven credit assignment, the agent successfully bridges this generalization gap, continuously adapting its behavior without requiring any parameter updates. 4.3.3 Effectiveness of LLM Backbone. To investigate the influence of the language foundation model on the overall navigation performance, we conduct an ablation study by changing the LLM backbone within the Instruction Interpolator, while keeping the vision foundation model fixed to BLIP-2. As presented in Table 3, we evaluate the framework using Qwen2.5-7B, Qwen3-8B, and Qwen3.5-9B architectures. Notably, the empirical results indicate that EvolveNav exhibits limited sensitivity to the scale and specific
Table 3: Ablation studies on LLM backbone. We investigate the impact of different LLM backbones within the Instruction Interpolator. The vision model is fixed to BLIP-2, and performance is evaluated on the HM3D and MP3D datasets.
Instruction Interpolator
HM3D
MP3D
Vision
Language
SR ↑
SPL ↑
SR ↑
SPL ↑
BLIP-2
Qwen2.5-7B Qwen3-8B Qwen3.5-9B
67.0 67.3 67.2
33.9 33.9 34.1
48.8 49.0 48.9
19.0 19.1 19.2
iteration of the underlying LLM backbone. Across both datasets, the performance variance is practically negligible. For instance, transitioning from the 7B to the 9B parameter model yields only marginal fluctuations, maintaining a robust SR of approximately 67.0%-67.3% on HM3D and 48.8%-49.0% on the MP3D benchmark. We attribute this stability to the core architectural design of our framework. This suggests that in OGN tasks, stronger language foundation models may not lead to better results. Meanwhile, EvolveNav explicitly grounds the agent’s decision-making in a dynamically maintained, self-evolving rule memory. During the preflection phase, the LLM
Qi Chai, Wenhao Shen, Nanjie Yao, Yue Xia, Kaiyong Zhao, Jie Ma, Guosheng Lin, and Hao Wang
a
b
c Task: Find a [Table]
Task: Find a [Nightstand]
Task: Find a [Potted Plant]
Start Point
Ascent’s Path
Ascent’s Destination
Our Path
Our Destination
Figure 4: Qualitative Comparison. We visualize the start points, paths and destinations of EvolveNav and ASCENT [15] on several MP3D scenes. Across the three illustrated scenarios, the Ascent’s trajectory is notably more convoluted. It frequently gets distracted by adjacent small rooms and narrow corridors, expending a significant number of steps to escape from these localized traps. In contrast, the paths of our proposed EvolveNav demonstrate better performance. This indicates EvolveNav achieves more efficient and robust results. is injected with structured local observations alongside high-value historical rules retrieved via the UCB selection criterion.
4.4
Qualitative Evaluation
To intuitively demonstrate the decision-making ability of EvolveNav, we provide a qualitative case study from the MP3D dataset, as shown in Figure 3. The agent is tasked with finding a <Potted Plant> in a multi-room layout. Instead of greedily exploring the nearest frontiers, EvolveNav demonstrates strong proactive risk avoidance. For instance, at Step 22, the agent triggers the preflection module before physical movement. By retrieving historical rules from the Rule Bank (e.g., Rule 2: "Avoid relying on indirect associations with spa, jacuzzi"), the LLM anticipates the failure risk of Frontier 1 and successfully bypasses it. Later at Step 106, preflection correctly guides the agent to prioritize an area with armchairs, anticipating a higher co-occurrence probability with the target. After the episode ends, EvolveNav continuously learns from its physical interactions. Following the trajectory, EvolveNav dynamically increases the UCB weights for the successful priors Rules 2 and 3. Furthermore, the LLM distills a new heuristic from this experience: "When looking for a potted plant, prioritize areas containing furniture such as armchairs." enriching the Rule Bank to guide future cross-scene explorations. To further demonstrate the superiority of our proposed EvolveNav, we visualize and compare the navigation trajectories across several indoor scenarios from the MP3D dataset against Ascent, the best baseline method. As illustrated in Figure 4, we present three distinct Object Goal Navigation tasks: finding a Potted Plant (scene a), a Nightstand (scene b) and a Table (scene c). Across the three illustrated scenarios, Ascent’s trajectory is notably more complex than that of our method. The red path frequently gets distracted by adjacent small rooms and narrow corridors, expending a significant
number of steps to escape from these localized traps. In contrast, our method actively observes and reorients at the entrances of narrow passages, proactively preventing the agent from getting stuck. By leveraging the evolved rules and the preflection module, our agent is capable of planning a more direct, globally optimal route. For instance, in the layout of scene (c), EvolveNav bypasses irrelevant areas entirely and heads straight toward the dining area to locate the table. In conclusion, these qualitative visualizations corroborate our quantitative findings, demonstrating that EvolveNav effectively minimizes redundant exploration and exhaustive trial-and-error, thereby achieving significantly more efficient and goal-directed navigation in unseen environments.
5 Discussion 5.1 Future Work We plan to extend the proposed EvolveNav from a single-agent paradigm to a collaborative multi-agent navigation system. While our current self-evolving memory and preflection mechanisms significantly mitigate the physical trial-and-error costs for an individual agent, navigating large-scale and complex environments often necessitates distributed coordination.
5.2
Limitations
Despite its robust performance, our framework presents two primary limitations. First, it relies on 2D visual foundation models. Consequently, persistent 2D detection errors can directly lead to navigation failures. In future work, this could be mitigated by multiview verification or by integrating active perception policies to enhance 3D spatial awareness. Second, the framework is potentially sensitive to unconventional layouts (e.g., a bed in a kitchen). Such environments may trap the agent in inefficient exploration loops.
EvolveNav : Proactive Preflection and Self-Evolving Memory for Zero-Shot Object Goal Navigation
Addressing this issue by validating the rules or refining the update strategy requires further research.
6
Conclusion
We propose EvolveNav, a self-evolving, training-free framework that addresses the limitations of static priors in ZS-OGN. Our approach introduces a self-evolving rule memory that extracts actionable knowledge from past trajectories, utilizing a UCB-based retrieval strategy to balance semantic relevance with historical success. To minimize physical trial-and-error, we develop a memoryguided preflection module that shifts the exploration paradigm from passive correction to proactive risk avoidance. Extensive experiments on HM3D and MP3D demonstrate that EvolveNav significantly outperforms state-of-the-art zero-shot baselines.
References [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). [2] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023). [3] Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. 2020. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171 (2020). [4] Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, and Hao Dong. 2024. Bridging zero-shot object navigation and foundation models through pixel-guided navigation skill. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5228–5234. [5] Qi Chai, Zhang Zheng, Junlong Ren, Deheng Ye, Zichuan Lin, and Hao Wang. 2025. Causalmace: Causality empowered multi-agents in minecraft cooperative tasks. arXiv preprint arXiv:2508.18797 (2025). [6] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158 (2017). [7] Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. 2020. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems 33 (2020), 4247– 4258. [8] Nuo Chen, Hongguang Li, Jianhui Chang, Juhua Huang, Baoyuan Wang, and Jia Li. 2025. Compress to impress: Unleashing the potential of compressive memory in real-world long-term conversations. In Proceedings of the 31st International Conference on Computational Linguistics. 755–773. [9] Shizhe Chen, Thomas Chabal, Ivan Laptev, and Cordelia Schmid. 2023. Object goal navigation with recursive implicit maps. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 7089–7096. [10] Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16537–16547. [11] Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413 (2025). [12] Ronghao Dang, Liuyi Wang, Zongtao He, Shuai Su, Jiagui Tang, Chengju Liu, and Qijun Chen. 2023. Search for or navigate to? dual adaptive thinking for object navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8250–8259. [13] Honghao Fu, Junlong Ren, Qi Chai, Deheng Ye, Yujun Cai, and Hao Wang. 2025. Vistawise: Building cost-effective agent with cross-modal knowledge graph for minecraft. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 21895–21909. [14] Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, and Shuran Song. 2023. Cows on pasture: Baselines and benchmarks for languagedriven zero-shot object navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23171–23181. [15] Zeying Gong, Rong Li, Tianshuai Hu, Ronghe Qiu, Lingdong Kong, Lingfeng Zhang, Guoyang Zhao, Yiyi Ding, and Junwei Liang. 2026. Stairway to Success:
An Online Floor-Aware Zero-Shot Object-Goal Navigation Framework via LLMDriven Coarse-to-Fine Exploration. IEEE Robotics and Automation Letters (2026). [16] Yu Guo, Jinsheng Sun, Ruiheng Zhang, Zhiqi Jiang, Zhenqiang Mi, Chao Yao, Xiaojuan Ban, and Mohammad S Obaidat. 2024. An object-driven navigation strategy based on active perception and semantic association. IEEE Robotics and Automation Letters 9, 8 (2024), 7110–7117. [17] Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017). [18] Xun Huang, Shijia Zhao, Yunxiang Wang, Xin Lu, Wanfa Zhang, Rongsheng Qu, Weixin Li, Yunhong Wang, and Chenglu Wen. 2025. MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation. arXiv preprint arXiv:2511.10376 (2025). [19] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024). [20] Jinyeon Kim, Cheolhong Min, Byeonghwi Kim, and Jonghyun Choi. 2024. Preemptive Action Revision by Environmental Feedback for Embodied Instruction Following Agents. In CoRL. [21] Yuxuan Kuang, Hai Lin, and Meng Jiang. 2024. Openfmnav: Towards openset zero-shot object navigation via vision-language foundation models. arXiv preprint arXiv:2402.10670 (2024). [22] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. PMLR, 19730–19742. [23] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024). [24] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems 36 (2023), 34892–34916. [25] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). [26] Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. 2024. InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment. In Conference on Robot Learning. [27] Jie Ma, Zhitao Gao, Qi Chai, Wangchun Sun, Pinghui Wang, Hongbin Pei, Jing Tao, Lingyun Song, Jun Liu, Chen Zhang, et al. 2025. Debate on graph: a flexible and reliable reasoning framework for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 24768–24776. [28] Arjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman, and Dhruv Batra. 2022. Zson: Zero-shot object-goal navigation using multimodal goal embeddings. Advances in Neural Information Processing Systems 35 (2022), 32340– 32352. [29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PmLR, 8748–8763. [30] Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. 2021. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238 (2021). [31] Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. 2023. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17896–17906. [32] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. 2019. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision. 9339–9347. [33] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems 36 (2023), 8634–8652. [34] Aaditya Singh and Adam Fry et al. 2025. OpenAI GPT-5 System Card. arXiv:2601.03267 [cs.CL] https://arxiv.org/abs/2601.03267 [35] Fuchun Sun, Runfa Chen, Tianying Ji, Yu Luo, Huaidong Zhou, and Huaping Liu. 2024. A comprehensive survey on embodied intelligence: Advancements, challenges, and future perspectives. (2024). [36] Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, and Bo An. [n. d.]. True Knowledge Comes from Practice: Aligning Large Language Models with Embodied Environments via Reinforcement Learning. In The Twelfth International Conference on Learning Representations. [37] Hanlin Wang, Chak Tou Leong, Jian Wang, and Wenjie Li. 2024. E2 CL: Exploration-based Error Correction Learning for Embodied Agents. In Findings of
Qi Chai, Wenhao Shen, Nanjie Yao, Yue Xia, Kaiyong Zhao, Jie Ma, Guosheng Lin, and Hao Wang
the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 7626–7639. doi:10.18653/v1/2024.findings-emnlp.448 [38] Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. 2023. Promptagent: Strategic planning with language models enables expert-level prompt optimization. arXiv preprint arXiv:2310.16427 (2023). [39] Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2024. Agent workflow memory. arXiv preprint arXiv:2409.07429 (2024). [40] Justin Wasserman, Girish Chowdhary, Abhinav Gupta, and Unnat Jain. 2024. Exploitation-guided exploration for semantic embodied navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2901–2908. [41] Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. 2019. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. arXiv preprint arXiv:1911.00357 (2019). [42] Yurong Wu, Yan Gao, Bin Benjamin Zhu, Zineng Zhou, Xiaodi Sun, Sheng Yang, Jian-Guang Lou, Zhiming Ding, and Linjun Yang. 2024. Strago: Harnessing strategic guidance for prompt optimization. In Findings of the Association for Computational Linguistics: EMNLP 2024. 10043–10061. [43] Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Fengwei Teng, Jinhao Tu, Xinbing Liang, Sirui Hong, Chenglin Wu, and Yuyu Luo. 2025. Self-supervised prompt optimization. arXiv preprint arXiv:2502.06855 (2025). [44] Yutao Xie, Nathaniel Thomas, Nicklas Hansen, Yang Fu, Li Erran Li, and Xiaolong Wang. [n. d.]. TIPS: Turn-level Information-Potential Reward Shaping for Search-Augmented LLMs. In The Fourteenth International Conference on Learning Representations. [45] Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Theo Gervet, John Turner, Aaron Gokaslan, Noah Maestre, Angel Xuan Chang, Dhruv Batra, Manolis Savva, et al. 2023. Habitat-matterport 3d semantics dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4927–4936. [46] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025). [47] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 Technical Report. eprint arXiv: 2407.10671 (2024). [48] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 Technical Report. arXiv preprint arXiv:2412.15115 (2024). [49] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. In The Twelfth International Conference on Learning Representations. [50] Zebin Yang, Sunjian Zheng, Tong Xie, Tianshi Xu, Bo Yu, Fan Wang, Jie Tang, Shaoshan Liu, and Meng Li. [n. d.]. EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [51] Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, et al. 2023. Retroformer: Retrospective large language agents with policy gradient optimization. arXiv preprint arXiv:2308.02151 (2023). [52] Xin Ye and Yezhou Yang. 2021. Efficient robotic object search via hiem: Hierarchical policy learning with intrinsic-extrinsic modeling. IEEE robotics and automation letters 6, 3 (2021), 4425–4432. [53] Hang Yin, Xiuwei Xu, Zhenyu Wu, Jie Zhou, and Jiwen Lu. 2024. SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation. Advances in Neural Information Processing Systems 37 (2024), 5285–5307. [54] Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. 2024. Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 42–48. [55] Hwiyeon Yoo, Yunho Choi, Jeongho Park, and Songhwai Oh. 2024. Commonsense-aware object value graph for object goal navigation. IEEE Robotics and Automation Letters 9, 5 (2024), 4423–4430. [56] Bangguo Yu, Hamidreza Kasaei, and Ming Cao. 2023. L3mvn: Leveraging large language models for visual target navigation. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 3554–3560. [57] Lingfeng Zhang, Hao Wang, Erjia Xiao, Xinyao Zhang, Qiang Zhang, Zixuan Jiang, and Renjing Xu. 2024. Multi-Floor Zero-Shot Object Navigation Policy. arXiv preprint arXiv:2409.10906 (2024). [58] Mingjie Zhang, Yuheng Du, Chengkai Wu, Jinni Zhou, Zhenchao Qi, Jun Ma, and Boyu Zhou. 2025. Apexnav: An adaptive exploration strategy for zero-shot object navigation with target-centric semantic fusion. IEEE Robotics and Automation Letters (2025). [59] Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. 2023. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199 (2023).
[60] Zheng Zhang, Yihuai Lan, Yangsen Chen, Lei Wang, Xiang Wang, and Hao Wang. 2025. Dvm: Towards controllable llm agents in social deduction games. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5. [61] Zheng Zhang, Nuoqian Xiao, Qi Chai, Deheng Ye, and Hao Wang. 2025. Multimind: Enhancing werewolf agents with multimodal reasoning and theory of mind. In Proceedings of the 33rd ACM International Conference on Multimedia. 5824–5833. [62] Zheng Zhang, Peilin Zhao, Deheng Ye, and Hao Wang. 2025. Enhancing jailbreak attacks on llms via persona prompts. arXiv preprint arXiv:2507.22171 (2025). [63] Xiaoming Zhao, Harsh Agrawal, Dhruv Batra, and Alexander Schwing. 2021. The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation. In Proc. ICCV. [64] Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38. 19724–19731.