Conceptio › Archive › arXiv CS

arXiv CSopen access

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

arXiv:2604.09508v1 [cs.CV] 10 Apr 2026

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning Yucheng Shen∗

Jiulong Wu∗

Jizhou Huang

Soochow University Baidu Inc. Beijing, China

Baidu Inc. Beijing, China

Dawei Yin

Lingyong Yan†

Min Cao†

Baidu Inc. Beijing, China

Soochow University Suzhou, China

Abstract

remarkable capabilities across a wide range of tasks [1–3, 16, 17], they remain constrained by fixed training data and are therefore prone to hallucination and knowledge gaps. Retrieval-Augmented Generation (RAG) [15, 19] mitigates these issues by grounding model outputs in externally retrieved evidence. Notably, real-world documents often contain rich visual content, such as charts, tables, and figures, which cannot be faithfully preserved through conventional text extraction. To bridge this gap, Visual RAG (VRAG) extends RAG to the visual domain [6, 12, 31], enabling models to retrieve and reason directly over images rendered from document pages. However, these VRAG works typically follow a fixed retrievethen-read pipeline that performs only a single round of retrieval, leaving them unable to gather the additional evidence needed when a question requires reasoning across multiple steps [26]. For this, agentic VRAG is proposed [13, 26, 27, 30], and interleaves reasoning with iterative retrieval actions, so as to dynamically collect evidence across multiple turns. Notably, recent work [26] enriches the agent’s action space with visual actions, such as crop-and-zoom, enabling it to selectively magnify specific regions of a retrieved image (e.g., a chart segment or table cell) for fine-grained analysis.

Visual Retrieval-Augmented Generation (VRAG) empowers VisionLanguage Models to retrieve and reason over visually rich documents. To tackle complex queries requiring multi-step reasoning, agentic VRAG systems interleave reasoning with iterative retrieval.. However, existing agentic VRAG faces two critical bottlenecks. (1) Visual Evidence Sparsity: key evidence is scattered across pages yet processed in isolation, hindering cross-page reasoning; moreover, fine-grained intra-image evidence often requires precise visual actions, whose misuse degrades retrieval quality; (2) Search Drift in Long Horizons: the accumulation of visual tokens across retrieved pages dilutes context and causes cognitive overload, leading agents to deviate from their search objective. To address these challenges, we propose VISOR (Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning), a unified single-agent framework. VISOR features a structured Evidence Space for progressive cross-page reasoning, coupled with a Visual Action Evaluation and Correction mechanism to manage visual actions. Additionally, we introduce a Dynamic Trajectory with Sliding Window and Intent Injection to mitigate search drift. They anchor the evidence space while discarding earlier raw interactions, preventing context from being overwhelmed by visual tokens. We train VISOR using a Group Relative Policy Optimization-based Reinforcement Learning (GRPO-based RL) pipeline with state masking and credit assignment tailored for dynamic context reconstruction. Extensive experiments on ViDoSeek, SlideVQA, and MMLongBench demonstrate that VISOR achieves state-of-the-art performance with superior efficiency for long-horizon visual reasoning tasks.

Visual Evidence Sparsity I can't reason coherently!

across images

... VRAG agent

analysis page i

analysis page n

reasoning gap

Some is uninformative, and some is wrong!

lost in the details

crop and zoom

Search Drift in Long-Horizon Interaction

• Computing methodologies → Intelligent agents.

agent loop context user: What's the ... assistant:<think><retrival> user:<image> assistant:<think><crop> user:<cropped imgae> assistant:<think><retrival> ...

Keywords Visual Retrieval-Augmented Generation, Agentic Reasoning, Longhorizon Reasoning, Evidence Management, Reinforcement Learning

analysis page 1

KeyInfo occupies only a small region.

within a single image

CCS Concepts

evidence across multi-pages

& evidence buried

search drift

Figure 1: Two critical bottlenecks in agentic VRAG: (Top) Visual Evidence Sparsity, where relevant clues are scattered across pages or confined to small regions within a single image; (Bottom) Search Drift in Long-Horizon Interaction, where accumulated visual tokens overwhelm the context, burying earlier evidence and causing the agent to lose track of its original objective.

Introduction

Although Language Models (LMs) and Vision-Language Models (VLMs) have advanced rapidly in recent years and demonstrated ∗ These authors contributed equally to this work. † Corresponding authors.

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Our main contributions are summarized as follows: • We propose VISOR, an agentic framework for long-horizon Visual RAG that addresses visual evidence sparsity and search drift via structured evidence accumulation, visual action correction, dynamic trajectory, and intent injection. • We design a two-stage training pipeline tailored for VISOR’s dynamic context, with a carefully curated SFT dataset and a reward function that jointly optimizes retrieval precision and answer correctness. • Extensive experiments on ViDoSeek, SlideVQA, and MMLongBench demonstrate that VISOR consistently outperforms strong fine-tuned baselines on both Qwen2.5-VL-7B and 3B.

Despite this progress, they still face two critical bottlenecks when handling complex questions over visually rich content as illustrated in Figure 1. (1) Visual Evidence Sparsity. Answering complex queries often requires locating and integrating key evidence from multiple document images, which is hindered by sparsity across two granularities. Across images, relevant clues are typically scattered across multiple pages; however, existing systems process each retrieved page in isolation and lack mechanisms for cross-page evidence accumulation, leaving partial observations fragmented and insufficient to support coherent multi-step reasoning. Within a single image, query-relevant information often occupies only a small region, e.g., specific cells in a dense table or a particular data series in a chart. Extracting such fine-grained evidence necessitates precise visual actions like crop-and-zoom. Yet, without an effective evaluation mechanism to guide these actions, agents risk performing suboptimal operations: cropping irrelevant regions introduces distracting content, or redundant zooms consume precious interaction turns without yielding useful signals. (2) Search Drift in Long-Horizon Interaction. This challenge is significantly amplified in the visual domain. Each retrieved document image consumes thousands of visual tokens, far exceeding the cost of textual content, causing the context window to saturate rapidly during multi-turn interactions. This leads to two compounding issues: first, previously gathered evidence becomes buried under accumulating visual data, making it difficult for the model to locate, or reuse earlier findings, ultimately undermining the coherence and completeness of the final answer; second, as visual tokens increasingly dominate the context, the agent’s attention drifts away from the original query, resulting in retrievals and actions that progressively deviate from user intent. To address these bottlenecks, we propose VISOR, a Visual Retrieval Augmented Generation via Iterative Search and Over-horizon Reasoning, illustrated in Figure 2. First, to mitigate visual evidence sparsity, VISOR introduces a structured Evidence Space that explicitly accumulates query-relevant observations across retrieval iterations. This enables joint cross-page reasoning, moving beyond isolated per-page processing. Also, a Visual Action Evaluation and Correction mechanism is developed to assess the utility and outcome of each visual action, thereby pruning noisy or irrelevant crops and selectively incorporating validated evidence into the Evidence Space. Second, to overcome search drift in long-horizon interaction, VISOR incorporates two key mechanisms: Dynamic Trajectory, which maintains a real-time, up-to-date Evidence Space pinned at the top of the context window, while recent interactions are retained via a sliding window; and Intent Injection, which reanchors the agent to the original user query at every retrieval step to preserve goal fidelity. This design effectively bounds context growth without sacrificing critical information. Notably, VISOR adopts an agent-based loop architecture and is thus trained endto-end: first via Supervised Fine-Tuning (SFT), followed by GRPObased Reinforcement Learning (RL) [22]. The reward function is carefully crafted to jointly optimize retrieval precision and final answer correctness. Extensive experiments on established benchmarks, ViDoSeek [25], SlideVQA [24], and MMLongBench [20], show that VISOR consistently outperforms existing agentic baselines even without task-specific fine-tuning, and its performance further improves with SFT+RL training.

2 Related Work 2.1 Visual Retrieval-Augmented Generation RAG has demonstrated significant advantages in addressing knowledge intensive problems [7, 15, 29], with traditional text-based methods retrieving relevant passages for answer generation. However, with the widespread adoption of visually rich documents such as slides, reports, and scanned PDFs, knowledge is no longer confined to plain text. Early approaches rely on OCR or document parsing to extract textual content from images [10], but such pipelines are lossy and fail to preserve layout, chart, and figure information. Recently, OCR-free retrieval methods [14] have emerged that directly align textual queries with document page images: ColPali [6] introduced late-interaction visual retrieval via token-level similarity between query text and image patch embeddings; VisRAG [31] further explored generation pipelines operating directly on retrieved visual content. Recent work also explores multimodal RAG systems [12] combining visual and textual information for more accurate retrieval and reasoning. EVisRAG [23] introduced evidence-based reasoning over multiple retrieved images to support multi-image understanding. Our work builds upon these visual retrieval foundations and further strengthens multi-image understanding.

2.2

Agentic RAG with Reinforcement Learning

The agentic paradigm for RAG was pioneered by ReAct [30], which interleaves reasoning with actions so that models can dynamically decide when and what to retrieve. Reinforcement learning has emerged as an effective approach for improving reasoning in language models [9, 11, 22], and recent work has extended it to VLMs for visual reasoning tasks [19, 28]. In the agentic RAG setting, Search-R1 [13] first applied RL to train language models for text-based agentic retrieval, and its visual extension Search-R1-VL adapted the framework to multimodal scenarios; R1-Router [21] trained a routing policy to dynamically decide retrieval actions; MMSearch-R1 [27] and WebWatcher [8] extended RL-based agents to multi-modal search and richer tool sets; VRAG-RL [26] defined a visual perception action space with crop-and-zoom operations and introduced multi-turn RL training for VRAG. However, these works do not address the challenges of multi-image evidence management and semantic drift in long-horizon visual reasoning. One natural direction to tackle these issues decomposes the task across specialized agents: ViDoRAG [25] proposed actor-critic iterative 2

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

Algorithm 1 VISOR Agent Loop

reasoning with separate agents for planning, retrieval, and answering; SLEUTH [18] and M3RAG [5] similarly decomposed the task into specialized modules. However, this decomposition can not support end-to-end optimization, as each component is trained or prompted independently with misaligned objectives, and sacrifices efficiency by routing all queries through the full pipeline regardless of complexity. Our work instead addresses these challenges within a single unified agent loop, achieving the benefits of both RL-based end-to-end training and robust long-horizon multi-image reasoning.

Require: Query 𝑞, corpus C Require: Max turns 𝑇 , sliding window size 𝑊 1: Initialize evidence space E ← ∅, history H ← [ ] 2: // 𝑜 𝑡 : environment observation returned after each action (retrieved image 𝐼𝑘 or cropped region 𝐼 ′ , bundled with the intentinjected prompt) 3: Construct initial prompt P from 𝑞 4: for 𝑡 = 1 to 𝑇 do 5: // Dynamic Trajectory Sliding Window (§3.2) 6: Construct context C𝑡 ← [ P; E; last 𝑊 turns of H ] Generate response r𝑡 : reasoning 𝜃 𝑡 (in ⟨think⟩ tags) followed 7: by action 𝑎𝑡 (𝑐𝑡 ), 𝑎𝑡 ∈ {search, crop, answer} (𝑐𝑡 is the action content: a search query, crop coordinates, or final answer string, respectively) 8: // Evidence Space update 9: Extract evidence from 𝜃 𝑡 ; update E [𝐼𝑘 ] for current page 𝐼𝑘 10: if 𝑎𝑡 = answer then 11: return 𝑐𝑡 as the final answer 12: else if 𝑎𝑡 = search then 13: Retrieve images from C using query 𝑐𝑡 14: // Intent Injection: re-state 𝑞 and E in observation 15: 𝑜𝑡 ← retrieved image 𝐼𝑘 with intent-injected prompt 16: // Visual Action Evaluation 17: Evaluation: guide model to evaluate whether to crop 𝐼𝑘 18: else if 𝑎𝑡 = crop then 19: Crop region 𝑐𝑡 from the current image → 𝐼 ′ 20: // Visual Action Correction 21: Correct: if crop is uninformative, redirect to pre-crop reasoning 22: 𝑜𝑡 ← 𝐼 ′ with intent-injected prompt 23: end if 24: Append (r𝑡 , 𝑜𝑡 ) to H 25: end for 26: Force answer: generate final answer 𝑎 based on the latest [ P; E; last 𝑊 turns of H ]

3 Method 3.1 Task Formulation Given a natural language query 𝑞 and a large corpus C = {𝐼 1, 𝐼 2, . . . , 𝐼 𝑁 } comprising 𝑁 page-level images sourced from visually rich documents (e.g., slides, reports), the goal is to generate a final answer 𝑎 by iteratively retrieving and reasoning over relevant images. In practice, C is heterogeneous: pages from documents of varying topics and formats are pooled into a single flat index devoid of document-level boundaries. Consequently, answering a query often necessitates synthesizing evidence scattered across multiple pages within this mixed corpus.

3.2

VISOR Agent Framework

As illustrated in Figure 2 and Algorithm 1, VISOR is an iterative agentic framework wherein a single agent interleaves retrieval, visual reasoning, and evidence accumulation to perform visual RAG. At each turn, the agent first reflects on its accumulated observations within a ⟨think⟩ block. It then issues one of three actions—search, crop, or answer—and subsequently receives a new observation from the environment. The loop continues until the agent either generates a final answer or reaches the maximum allowable number of turns. The framework relies on three core design components that respectively govern how the agent reasons, acts, and manages its context across turns. Evidence Collection. Existing agentic RAG systems [26, 27, 30] typically process retrieved pages in isolation, making it challenging to jointly reason over evidence scattered across multiple pages. VISOR addresses this limitation by introducing a persistent and structured evidence space E: for each visited page 𝐼𝑘 , the agent generates up to two text summaries directly into its ⟨think⟩ block: pre post 𝑒𝑘 upon initially inspecting the full page, and 𝑒𝑘 after examining a localized region via a crop action (which is omitted if no crop occurs). These summaries are extracted and stored in E: pre post E = 𝐼𝑘 ↦→ (𝑒𝑘 , 𝑒𝑘 ) | 𝑘 ∈ retrieved pages . (1) Based on the above design, the agent is explicitly prompted to record any potentially useful information from a page, even if that page alone is insufficient to answer the question. This strategy prevents VLMs from prematurely discarding partially relevant content and ensures that fine-grained clues from disparate pages can be effectively synthesized when formulating subsequent search queries or deriving the final answer.

<search> tags; the retrieval engine returns the most relevant page (top-1) from C as the subsequent observation. (2) Crop: the agent sends a <bbox> action specifying pixel coordinates to zoom into a specific region of the current page for fine-grained reading. (3) Answer: the agent wraps its final response in <answer> tags, terminating the loop. To minimize unnecessary crop actions, we introduce a Visual Action Evaluation and Correction mechanism. Upon receiving a retrieved page, the agent first evaluates whether the globally visible content is already sufficient; if so, no crop is performed. The agent issues a crop action only when a specific region appears to contain relevant content but lacks sufficient clarity or resolution for reading. Furthermore, if a crop yields uninformative results, a Correction step is applied: via prompting, the agent’s attention is redirected back to its pre-crop reasoning, thereby preventing irrelevant visual content from injecting noise into E.

Action Space. At each turn, the agent selects one of three actions: (1) Retrieval: The agent submits a text query wrapped in

Agent Loop Context. In the visual setting, naively appending every response and observation to the context is prohibitively costly: 3

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Action Space

Collected Evidence retrieved image_1 { pre chart shows Microsoft Edge... ; post: the cropped image shows... }

retrieved image_2: { pre: the image shows nothing related to... }

1.Retrival:<search> Query </search>

...

top 1 retrieval model

Evidence is extracted from <think>...</think>

Step i

User Query

VISOR

multimodal document

2.Crop:<bbox> [x1,y1,x2,y2] </bbox>

Visual Action Evaluation

what is recommended as a web browser according to...

retrieved image

cropped image

3.Answer: <answer> xxx</answer>

Agent Loop

e.g. The final answer is Microsoft Edge ...

Agent Loop Context

Turn i context

User Query

...

Collected Evidence

T i-2

imgae of relevant page

T i-1

assistant:<think>...</think> <action>...</action> user: <imgae> <intent injection>

T i

refer to [user query] and Collected Evidence to [analyse the image/...]

Sliding Window

Figure 2: Overview of VISOR. At each step 𝑖, the agent produces a ⟨think⟩. . . ⟨action⟩ response. Evidence extracted from the reasoning trace is accumulated in a structured Evidence Collection space E. The action space comprises three operations: search, crop, and answer. The agent loop context is reconstructed at each turn: the user query and E are always pinned at the top, while only the last 𝑊 turns of raw interactions are retained via a Sliding Window; each turn’s observation also includes an Intent Injection prompt to re-anchor the agent’s focus. each retrieved page-level image introduces thousands of tokens, rapidly diluting earlier evidence. To mitigate this, we replace the standard append-only trajectory with a Dynamic Trajectory mechanism that dynamically reconstructs the input context at each turn as: C𝑡 = P ; E𝑡 ; (r𝑡 −𝑊 +1, 𝑜𝑡 −𝑊 +1 ), . . . , (r𝑡 , 𝑜𝑡 ) , (2) | {z }

Intent Injection prompt. This prompt explicitly restates the original query 𝑞 and redirects the model’s attention to the evidence space E, reminding the agent to consult its accumulated findings rather than reasoning from scratch. This mechanism ensures the agent remains anchored to its primary objective throughout the trajectory, effectively countering the common tendency of long-horizon reasoning to drift into irrelevant tangents.

last 𝑊 turns

where P is the initial prompt containing 𝑞, E𝑡 is the current evidence space, r𝑖 and 𝑜𝑖 denote the agent’s response and the environment observation at turn 𝑖, and 𝑊 is the sliding window size that controls how many recent turns are retained. The query P and E𝑡 are persistently pinned at the top, ensuring that all cross-page findings accumulated thus far remain visible at every turn. We empirically set 𝑊 = 2 ; thus, when the agent issues a crop action, the context retains the preceding turn (containing the original full-page image and the initial reasoning). This allows the agent to observe the complete retrieve-then-crop reasoning chain. While earlier turns are evicted from the raw context window, their distilled evidence remains safely preserved in E𝑡 . This strategy strictly bounds the number of in-context image tokens throughout long-horizon reasoning tasks. Furthermore, every system-returned observation—whether triggered by a search, crop, or forced-answer event—incorporates an

3.3

Training Pipeline

Supervised Warm-up via Trajectory Distillation. Inspired by trajectory distillation [26], we distill agentic trajectories from Qwen3-VL-235B-A22B [3] to construct SFT data. We collect training queries from the training split of SlideVQA [24] and apply strict quality filtering——particularly on retrieval steps—discarding trajectories that exhibit redundant or uninformative searches. This yields about 2.5K curated trajectories (see Appendix B for details). This SFT stage endows VLM with two complementary capabilities. First, it learns the foundational agentic skills required by our framework: strict adherence to structured output formats (<think>, <search>, <bbox>, <answer> tags) and precise formulation of retrieval intents. Second, it acquires a verification search strategy: after gathering sufficient evidence, the agent executes one final search query to 4

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

confirm that no critical visual or textual information has been overlooked before generating its answer. This deliberate verification step contrasts sharply with prior agentic VRAG systems [26, 27], which typically suppress additional searches to mitigate context bloat. In contrast, our VISOR framework dynamically reconstructs and compresses trajectories during inference, enabling it to reward—rather than penalize—such verification behavior. We emphasize that the SFT data consists of conventional longhorizon trajectories, without evidence-space abstraction or slidingwindow context restructuring. During training, observation tokens are masked, ensuring that gradients propagate exclusively through the model’s own responses. Introducing the dynamic context format at the SFT stage would unnecessarily complicate data curation and is orthogonal to its objective: SFT is designed solely to learn basic agentic behaviors and format compliance, while leaving contextefficient reasoning to the subsequent RL stage.

on retrieval completeness to prevent reward hacking—without this gate, an agent that fails to retrieve the necessary pages could still earn high answer credit by producing fluent-sounding but fabricated responses that happen to fool the judge. Retrieval Reward (𝑟 ret ). Let 𝑡 ∗ be the turn at which all reference pages are first retrieved and 𝑇 the total number of search turns. We define Δ = 𝑇 − 𝑡 ∗ as the number of extra search turns after all necessary evidence has been gathered:  −1.0      −0.5  𝑟 ret =  0.0     −0.1×Δ 

incomplete retrieval (missing evidence pages) Δ = 0 (no verification search) Δ = 1 (exactly one verification, optimal) Δ ≥ 2 (over-searching) (5) Our reward shaping explicitly encourages exactly one verification search: stopping immediately after full retrieval (Δ = 0) is unrewarded, risking missed evidence, while excessive searching (Δ ≥ 2) is penalized. Failure to retrieve all necessary pages incurs the harshest penalty, as missing evidence is unrecoverable by reasoning alone. Format Penalty (Iformat ). Besides the above two reward scores, we also introduce an output penalty score Iformat for the whole trajectory format inspired by [9]. Iformat is set to 1 if the trajectory format is invalid (invokes no search action before the answer action), and 0 otherwise.

Reinforcement Learning. While SFT enables the agent to imitate expert trajectories, imitation alone provides only a weak and indirect signal for determining when to terminate retrieval: the teacher’s stopping behavior lacks explicit optimality guarantees and is not reinforced by reward feedback. The RL stage addresses this limitation. Through self-generated rollouts in the retrieval environment, the agent receives dense, task-aligned rewards, allowing it to discover and refine retrieval strategies that surpass what pure imitation can convey, including precise stopping criteria and adaptive search depth. We optimize the policy using GRPO [22] under the dynamic trajectory described above. Two designs address the unique challenges of multi-turn agentic training. First, observation masking: environment-provided observations, such as retrieved images or cropped regions, are external to the agent’s policy. To ensure proper credit assignment, we exclude these tokens from the policy gradient. And only the agent’s own generated tokens <think>/<action> contribute to parameter updates. Second, credit assignment under dynamic context: the sliding window is applied at generation time only. Specifically, each forward pass uses the dynamically reconstructed context for token prediction, whereas the loss is computed over the entire uncompressed response sequence. This ensures that both reward calculation and gradient backpropagation operate on the complete, untruncated trajectory, enabling accurate long-horizon credit assignment despite context compression during generation. The total reward 𝑟 first checks format validity, then combines retrieval and answer quality: 𝑟 = (𝑟 ans + 𝑟 ret ) 1 − Iformat − 1 · Iformat . (3)

4 Experiments 4.1 Experimental Settings Datasets and Metric. Following VRAG-RL [26], we evaluate our method on three challenging and visually rich benchmarks: ViDoSeek [25], SlideVQA [24], and MMLongBench [20]. For each benchmark, all images of document pages are pooled into a unified retrieval corpus, and the model retrieves relevant images from this corpus to answer each question. For fine-tuning, we use training data derived from the SlideVQA training split, comprising 2,500 samples for supervised fine-tuning (SFT) and 800 samples for reinforcement learning (RL). To evaluate the performance, we employ Qwen-max-latest as the judge model to assign a binary score (0 or 1) to each response by comparing it against the reference answer, and we report the mean score as the final accuracy. Further details are provided in Appendix D.

Answer Reward (𝑟 ans ). 𝑟 ans is gated on whether retrieval succeeded: ( ˆ 𝑎 ∗ ) ∈ {0, 1} LLM-judge(𝑎, retrieval complete 𝑟 ans = (4) ˆ ∈ {0, 0.2} retrieval incomplete honesty-judge(𝑎)

Baselines. For vanilla (non-fine-tuned) baselines, we compare against Vanilla RAG [6] and ReAct [30], as well as two multiagent architectures: ViDoRAG [25] and M3RAG [5]. For fine-tuned baselines, we compare with Search-R1-VL [13], VRAG-RL [26], MMSearch-R1 [27], R1-Router [21], and EVisRAG [23]. All methods use ColQwen2.5-v0.1 [6] as the shared retrieval backbone, ensuring a fair comparison. Implementation details of all baselines are provided in Appendix F.

When retrieval is complete, an LLM judge (Qwen-Max-Latest [3]) compares the predicted answer 𝑎ˆ against the reference 𝑎 ∗ and returns a binary correctness score. When retrieval is incomplete, we instead apply an epistemic honesty check: the agent receives partial reward (0.2) for explicitly acknowledging insufficient information rather than fabricating a response. We gate these two evaluations

As shown in Table 1, VISOR achieves state-of-the-art performance across all three benchmarks and all backbone sizes. Specifically, on SlideVQA, the improvement is most pronounced on multi-hop questions, where our VISOR scores 53.62%, substantially outperforming

4.2

Main Results

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Table 1: Main results on SlideVQA, ViDoSeek, and MMLongBench. We report accuracy (%). † denotes multi-agent architectures. ‡ denotes fine-tuned models. ★ denotes results taken from published papers under the same or comparable experimental settings. The best result in each column is bolded and the second-best is underlined. SlideVQA

Method

ViDoSeek

Single-hop

Multi-hop

Overall

Extraction

Vanilla RAG★ [6] ReAct★ [30] ViDoRAG† [25] M3RAG†★ [5] Search-R1-VL‡★ [13] VRAG-RL‡★ [26] MMSearch-R1‡ [27] EVisRAG‡ [23] R1-Router‡ [21] VISOR‡

29.10 34.80 72.15 – 48.30 69.30 52.06 78.21 69.66 78.82

17.40 20.40 39.86 – 42.30 43.10 40.21 42.32 45.33 53.62

26.10 31.11 63.88 65.82 46.76 62.59 49.03 69.09 63.43 72.37

26.40 27.50 66.05 – 40.50 60.60 55.97 67.75 64.19 73.49

Vanilla RAG★ [6] ReAct★ [30] ViDoRAG† [25] Search-R1-VL‡★ [13] VRAG-RL‡★ [26] EVisRAG‡ [23] R1-Router‡ [21] VISOR‡

19.40 15.70 41.44 26.30 65.30 75.42 64.93 74.58

12.20 10.90 19.93 20.10 38.60 47.70 42.15 50.79

17.56 14.47 35.94 24.71 58.45 68.35 59.10 68.49

10.10 6.70 31.32 20.10 63.10 66.05 62.64 67.75

Logic

MMLongBench Overall

Text

Table

Chart

Figure

Layout

Overall

32.88 33.85 69.00 69.36 44.77 66.78 57.53 69.79 66.11 74.87

13.10 10.10 24.40 – 19.90 26.10 16.84 26.80 26.46 27.49

14.70 12.40 23.96 – 13.40 26.30 17.97 27.65 23.50 23.96

15.90 10.20 21.91 – 12.90 24.80 19.10 24.72 22.47 27.53

4.30 6.20 24.14 – 11.40 25.90 18.28 22.41 28.28 23.79

7.60 7.10 20.34 – 10.20 21.20 11.86 19.49 16.95 22.88

– – 25.50 – – – 18.42 27.98 26.92 28.45

13.23 9.96 34.33 24.32 67.76 68.82 65.59 69.00

2.20 2.70 7.90 8.50 22.70 27.14 26.80 26.80

4.10 3.60 6.91 7.80 16.10 28.64 23.50 23.96

5.20 3.40 5.06 7.90 21.90 25.13 21.91 26.97

4.70 3.10 8.97 9.30 21.40 24.83 25.17 23.45

4.30 5.10 7.63 7.60 19.50 17.80 21.19 22.03

– – 8.50 – – 28.34 25.74 27.86

Qwen2.5-VL-7B 41.30 42.10 72.83 – 50.30 74.80 59.56 72.43 68.61 76.66

Qwen2.5-VL-3B 17.30 14.20 38.23 29.80 73.80 72.43 69.42 70.62

Table 2: Ablation study of VISOR on Qwen2.5-VL-7B on SlideVQA and ViDoSeek. We report overall accuracy (%). II = Intent Injection, ES = Evidence Space with Visual Action Evaluation and Correction, SW = Sliding Window.

VRAG-RL (43.10%) and EVisRAG (42.32%). This demonstrates that our Evidence Space effectively supports cross-page reasoning. In contrast, performance on single-hop questions remains comparable to the best baseline (78.82%), indicating that VISOR introduces no overhead on simpler queries. On ViDoSeek, gains are markedly larger for Extraction task (+12.89% over VRAG-RL) than for Logic task (+1.86%). This suggests that our Evidence Space is particularly beneficial for retrieving and aggregating factual content across pages, whereas logical inference—once relevant pages are identified—is already well addressed by existing RL-trained reasoning modules. On MMLongBench, VISOR leads on Chart and Layout subtasks (+2.81% and +3.39% over EVisRAG, respectively), highlighting the importance of our selective crop-and-zoom mechanism for fine-grained visual understanding. With the smaller Qwen2.5-VL-3B backbone, VISOR achieves 68.49% on SlideVQA, 69.00% on ViDoSeek, and 27.86% on MMLongBench, outperforming EVisRAG on the first two benchmarks. The slight performance gap on MMLongBench (vs. 28.34% from EVisRAG) stems primarily from differing training objectives: EVisRAG is fine-tuned on large-scale multi-image understanding data, granting it stronger multi-image comprehension capability, while VISOR focuses on agentic retrieval and long-horizon reasoning. On visually richer content, the reduced capacity of the 3B backbone amplifies this architectural divergence. Notably, multi-agent methods suffer severe degradation at this scale, e.g., ViDoRAG performance drops sharply from 69.00% to 34.33% on ViDoSeek. In contrast, fine-tuned single-agent methods exhibit only marginal decline (e.g., EVisRAG: 69.79%→68.82%), underscoring that end-to-end optimization becomes increasingly critical as model capacity decreases.

Method

4.3

ViDoSeek

SlideVQA

Vanilla

Fine-tuned

Vanilla

Fine-tuned

VISOR (Full)

57.88

74.87

54.00

72.37

w/o II w/o ES w/o SW w/o ES & SW w/o II & ES & SW

49.04 39.58 53.94 42.11 37.57

69.00 47.81 70.93 68.48 66.78

43.52 39.10 47.47 40.81 36.98

69.07 51.33 67.54 64.61 62.59

Ablation Study

As shown in Table 2, we ablate VISOR by decomposing three key components: Intent Injection (II), Evidence Space with Visual Action Evaluation and Correction (ES), and Sliding Window (SW). All ablation studies are conducted using Qwen2.5-VL-7B backbone on both ViDoSeek and SlideVQA. The results demonstrate that the removal of any single component leads to consistent performance degradation across both datasets and evaluation settings, underscoring the necessity and complementary nature of each module in our framework. Detailed Analysis of Evidence Space (ES) and Sliding Window (SW). (1) The removal of ES incurs the largest accuracy drop across both benchmarks and settings. This indicates that parameter updates alone cannot compensate for the loss of evidence distillation capability. (2) Eliminating SW also consistently degrades 6

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

performance, as the model is forced to process an unbounded sequence of visual tokens without any context budgeting strategy, quickly exceeding its effective receptive capacity. (3) Most revealing, however, is their interaction: when ES is absent, further removing SW partially recovers performance. This counterintuitive result demonstrates that SW is only beneficial when paired with ES. Without ES to extract and preserve critical evidence from discarded turns, SW truncation becomes purely destructive, discarding raw context with no compensatory signal. Conversely, without SW, the model retains all historical turns but rapidly exhausts its context window, leading to inefficiency and degradation. (4) Together, these findings reveal a tight coupling between the two components: SW enables efficient, bounded-context processing by compressing the interaction trajectory, while ES ensures that semantically valuable information lost during compression is retained through structured evidence distillation. Only when jointly deployed do ES and SW effectively tackle the central challenge of long-horizon visual retrieval; in isolation, each component proves either inadequate or actively detrimental. Detailed Analysis of Intent Injection (II). In the absence of II, the model lacks explicit query re-anchoring at each interaction turn. This leads to a noticeable performance drop on both benchmarks. However, the gap narrows substantially after fine-tuning, suggesting that the model can partially remedy query-aware behavior through training. Thus, II primarily serves as an architectural inductive bias that guides untrained models toward query-aligned reasoning, while still offering a consistent (though reduced) benefit in fine-tuned settings. In summary, removing all three components (II, ES, and SW) reduces VISOR to a vanilla agentic baseline. The persistent performance gap across both benchmarks underscores that these modules jointly mitigate two core challenges in long-horizon visual retrieval: sparse evidence signals and search drift. Their synergy—not just their individual contributions—is key to VISOR’s effectiveness.

4.4

67.0

65 58.6

60 Acc

56.4

50.2

49.8

41.2

40 35

Inference w/ Noise Upper Bound

33.4

30 1

3 4 Num of Reference Images

Figure 3: Accuracy of Qwen2.5-VL-7B on SlideVQA (500 samples) as a function of the number of retrieved reference images. The upper bound (67.0%) is the accuracy when correct reference images are directly provided to the model. EVisRAG) are structurally constrained: a fixed top-𝑘 retrieval cannot recover missing evidence if relevant pages lie outside the initial set, especially for multi-hop questions. ViDoRAG mitigates this by retrieving up to 10 pages by default, achieving the highest completeness, and yet this brute-force expansion introduces substantial noise into the context. On the drift axis, methods like VRAG-RL iteratively gather evidence over multiple interaction steps, improving coverage beyond a single retrieval pass; yet without explicit evidence management, each new image appends raw visual tokens to an ever-growing context, shifting the failure mode from missing evidence to accumulated irrelevance. Our VISOR addresses this dilemma at the architectural level. By distilling verified evidence into a compact, structured representation and enforcing a bounded context window, VISOR enables verification-driven search without uncontrolled noise accumulation, simultaneously achieving high retrieval completeness and sustained contextual coherence. Table 3: Retrieval completeness (%) and average number of retrieved images per trajectory on SlideVQA (2,215 samples). Completeness = fraction of questions where all ground-truth evidence pages are retrieved.

Analysis

Retrieval Behavior Analysis. Understanding the limitations of existing retrieval logic is key to motivating our design. In particular, methods like VRAG-RL illustrate a fundamental tension between retrieval completeness and context length, a tradeoff that our VISOR framework explicitly seeks to resolve. Figure 3 analyzes Qwen2.5-VL-7B on 500 SlideVQA samples, plotting accuracy as a function of the number of retrieved reference images. At low retrieval counts, the model often retrieves only visually similar but non-informative pages or misses critical evidence required for multi-hop reasoning, forcing it to answer with incomplete information. As more images are retrieved, however, irrelevant content accumulates, inducing semantic drift that obscures earlier findings and ultimately degrades performance. The upper-bound curve, representing accuracy when ground-truth reference images are directly provided, confirms that completeness of retrieved content, and semantic drift caused by long context, remains a primary bottleneck. As shown in Table 3, this tradeoff between completeness and drift manifests differently across methods. On the completeness axis, methods that perform retrieval only once at the beginning (e.g.,

Method EVisRAG [23] VRAG-RL [26] ViDoRAG [25] VISOR

Completeness (%)

Avg. Retrieved

80.2 76.8 91.3 84.2

3 (fixed) 1.78 10 (fixed) 2.34

Time Efficiency. As shown in Figure 4, VISOR is naturally slower than EVisRAG, which performs a single fixed top-𝑘 retrieval and one-shot answer, due to its iterative multi-turn interaction. Compared to VRAG-RL, another iterative method, VISOR introduces an additional verification turn yet remains competitive in overall latency. This is because the Sliding Window prevents context length from growing unboundedly across turns, significantly accelerating per-turn inference, while Visual Action Evaluation reduces unnecessary crop-and-zoom operations by issuing boundingbox actions only when fine-grained inspection is genuinely needed. 7

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Compared to ViDoRAG, VISOR is notably faster despite achieving higher accuracy. ViDoRAG retrieves up to 10 pages per query and subsequently runs repeated actor-critic cycles over the retrieved content through multiple specialized agents, incurring substantial VLM inference overhead. VISOR avoids this by adaptively retrieving only the pages needed per query, keeping the average retrieved count low while maintaining high completeness. Answer Search

EVisRAG

compatible with existing training paradigms and does not require specialized trajectory formats to benefit from. Visual Action Usage Analysis. Crop-and-zoom enables finegrained perception of information-dense regions, but indiscriminate cropping introduces visual noise and wastes interaction turns. Without an explicit gating mechanism, models tend to over-apply crop actions—Appendix H shows a representative case where a crop is applied to an already-legible region, gaining nothing while consuming an extra turn. VISOR’s Visual Action Evaluation mechanism explicitly prompts the model to assess whether cropping is necessary before acting, directly suppressing such redundant operations. As a result, VISOR’s bbox usage rate on ViDoSeek is only 6.65%, compared to 82.92% in VRAG-RL where no such gate exists. The crops that are issued under VISOR are genuinely informative, as reflected by the gains on Chart and Layout in Table 1.

Perception Verification

ViDoRAG VRAG-RL VISOR

Avg. Latency

Reliability of Model-as-Judge. We use an LLM judge (QwenMax-Latest) both as the answer reward signal during RL training and as the evaluation metric at test time. Compared to rule-based alternatives such as exact match, a model-based judge handles the linguistic variability inherent in open-ended visual question answering—where correct answers may be expressed in multiple valid forms—without over-penalizing semantically equivalent responses. Compared to recall-based soft matching, it is less susceptible to reward hacking through repetitive or loosely overlapping outputs. We verify the reliability of this judge by measuring its agreement with human annotations on a held-out subset; results (Appendix D) confirm high consistency, supporting its use as both a training signal and an evaluation criterion.

Figure 4: Breakdown of average per-sample inference latency on ViDoSeek. VISOR introduces a verification step yet remains faster than the multi-agent baseline ViDoRAG and competitive with VRAG-RL. Effect of Training. Table 4 examines how each training stage shapes the model’s behavior on ViDoSeek. The finish rate measures the proportion of episodes in which the model produces a non-empty final answer, reflecting whether the agent successfully completes the task within the allocated steps. The invalid action rate measures the proportion of episodes in which at least one action violates the required format constraints (e.g., missing mandatory <think> reasoning or issuing malformed tool calls); upon detecting such an action, the framework immediately returns a system-level error message prompting the model to correct its behavior before proceeding.

Case Study. We present representative success and failure trajectories of VISOR in Appendix H. In the success case, VISOR handles a multi-hop question spanning two separate pages: it retrieves each relevant page in turn, distills findings into the Evidence Space, and synthesizes a correct answer from the accumulated evidence—a capability that single-pass retrieval and page-isolated reasoning cannot provide. In the failure case, VISOR successfully retrieves the correct page yet produces an incorrect answer due to misinterpretation of a complex structural diagram. This suggests that while VISOR’s retrieval and evidence management are effective, visual understanding of intricate visual structures remains a bottleneck, pointing to a clear direction for future improvement.

Table 4: Effect of training stage on the model. Method

Invalid Action Rate ↓

Finish Rate ↑

Acc ↑

Vanilla SFT SFT+RL

0.61 0.53 0.09

74.08 79.42 96.23

57.88 63.57 74.87

The vanilla model already benefits from VISOR’s framework structure, but frequently produces invalid actions—failing to follow the required output format or issuing hallucinated crop coordinates—and often terminates prematurely without retrieving sufficient evidence. SFT on long-horizon trajectories substantially reduces the invalid action rate and improves finish rate, demonstrating that models trained in the standard trajectory format can be directly deployed within VISOR’s framework without architectural modification. RL further refines decision-making at each step: the model learns when to search, when to crop, and when to answer, leading to the lowest invalid rate, highest finish rate, and best accuracy. These results suggest that VISOR’s framework is broadly

Conclusion

We presented VISOR, a framework for agentic VRAG that tackles visual evidence sparsity and search drift in long-horizon document understanding. Through deliberate architectural design, VISOR achieves structured evidence management within an end-to-end trainable loop, without sacrificing retrieval flexibility or reasoning coherence. Experiments confirm state-of-the-art results on SlideVQA and ViDoSeek, and ablations show that the Evidence Space addresses a structural bottleneck that RL training alone cannot resolve. We hope VISOR offers a practical blueprint for building capable, efficient agentic VLMs over visually rich corpora. 8

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

References

arXiv:2402.03300 (2024). [23] Yubo Sun, Chunyi Peng, Yukun Yan, Shi Yu, Zhenghao Liu, Chi Chen, Zhiyuan Liu, and Maosong Sun. 2025. VisRAG 2.0: Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation. arXiv preprint arXiv:2510.09733 (2025). [24] Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. 2023. Slidevqa: A dataset for document visual question answering on multiple images. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13636–13645. [25] Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, and Feng Zhao. 2025. Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 9124–9145. [26] Qiuchen Wang, Ruixue Ding, Yu Zeng, Zehui Chen, Lin Chen, Shihang Wang, Pengjun Xie, Fei Huang, and Feng Zhao. 2025. Vrag-rl: Empower visionperception-based rag for visually rich information understanding via iterative reasoning with reinforcement learning. arXiv preprint arXiv:2505.22019 (2025). [27] Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. 2025. Mmsearch-r1: Incentivizing lmms to search. arXiv preprint arXiv:2506.20670 (2025). [28] Jiulong Wu, Yucheng Shen, Lingyong Yan, Haixin Sun, Deguo Xia, Jizhou Huang, and Min Cao. 2025. Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis. arXiv preprint arXiv:2511.10254 (2025). [29] Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze D Gui, Ziran W Jiang, Ziyu Jiang, et al. 2024. Cragcomprehensive rag benchmark. Advances in Neural Information Processing Systems 37 (2024), 10470–10490. [30] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations. [31] Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, et al. 2024. Visrag: Vision-based retrieval-augmented generation on multi-modality documents. arXiv preprint arXiv:2410.10594 (2024).

[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). [2] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023). [3] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. 2025. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025). [4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-VL Technical Report. arXiv preprint arXiv:2502.13923 (2025). [5] Haizhou Du and Wenhao Li. 2026. M3RAG: Orchestrating Multi-agent Reasoning for Multi-hop, Multi-modal Understanding. In International Conference on Multimedia Modeling. Springer, 364–378. [6] Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024. Colpali: Efficient document retrieval with vision language models. arXiv preprint arXiv:2407.01449 (2024). [7] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, Haofen Wang, et al. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 2, 1 (2023), 32. [8] Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, et al. 2025. Webwatcher: Breaking new frontier of vision-language deep research agent. arXiv preprint arXiv:2508.05748 (2025). [9] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025). [10] Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM international conference on multimedia. 4083–4091. [11] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720 (2024). [12] Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, et al. 2024. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959 (2024). [13] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516 (2025). [14] Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. Ocr-free document understanding transformer. In European Conference on Computer Vision. Springer, 498–517. [15] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33 (2020), 9459–9474. [16] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024). [17] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems 36 (2023), 34892–34916. [18] Keliang Liu, Zizhi Chen, Mingcheng Li, Jingqun Tang, Dingkang Yang, and Lihua Zhang. 2025. Resolving evidence sparsity: Agentic context engineering for long-document understanding. arXiv preprint arXiv:2511.22850 (2025). [19] Ziyu Liu, Yuhang Zang, Yushan Zou, Zijian Liang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Visual agentic reinforcement fine-tuning. arXiv preprint arXiv:2505.14246 (2025). [20] Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, et al. 2024. Mmlongbench-doc: Benchmarking long-context document understanding with visualizations. Advances in Neural Information Processing Systems 37 (2024), 95963–96010. [21] Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Yishan Li, Yukun Yan, Shuo Wang, Zhiyuan Liu, Yu Gu, Minghe Yu, Ge Yu, et al. 2025. Learning to route queries across knowledge bases for step-wise retrieval-augmented reasoning. arXiv preprint arXiv:2505.22095 (2025). [22] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint

Search Engine and Crop-and-Zoom Tool

Search Engine. We use ColQwen2.5-v0.1 [6] as our retrieval backbone. Each document page is pre-encoded into patch-level multivector embeddings offline and stored for fast lookup. At inference time, the agent’s text query is encoded by the same model and ranked against all page embeddings via the MaxSim operator, returning the top-𝑘 candidate pages. To avoid redundancy, the environment maintains a per-trajectory retrieval history: from the ranked list, the first page not previously shown to the agent is selected as the observation. If all top-𝑘 candidates have already been retrieved in earlier turns, the agent receives a prompt indicating that no new pages are available and is directed to produce a final answer. Crop-and-Zoom Tool. When the agent emits a <bbox>[x1, y1, x2, y2]</bbox> action, the pixel coordinates—expressed in the VLM’s displayed image space—are linearly mapped back to the original high-resolution page. Before cropping, the bounding box is expanded by a fixed margin of 28 pixels on each side to preserve contextual content near the boundary, and then clamped to the image extent to prevent out-of-bound access. The resulting region is cropped and resized to a standard resolution, enabling the agent to read fine-grained content (e.g., dense tables or small charts) that is difficult to perceive at full-page scale. Invalid coordinates result in an error message prompting the model to retry.

SFT Data Construction

Raw Data Collection. Following VRAG-RL [26], we construct SFT data by collecting expert trajectories via prompt-based distillation. Training queries are drawn from the training split of SlideVQA [24]. For each query, we prompt the teacher model to generate a complete 9

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Table 5: Key hyperparameters for SFT.

agentic trajectory—comprising interleaved search queries, cropand-zoom actions, and reasoning steps—using the same ReActstyle format defined in our system prompt. In contrast to VRAGRL, which employs separate models for trajectory generation and grounding, we use Qwen3-VL-235B-A22B [3] uniformly for all data collection steps.

Name Finetuning type Freeze vision tower Freeze multi-modal projector Freeze language model Cutoff length Epochs Batch size Gradient accumulation steps Learning rate LR scheduler type Warmup ratio

Data Filtering. We apply a series of quality filters to retain only high-quality trajectories. First, we enforce structural validity of crop actions: any trajectory where a <bbox> action is not immediately preceded by a retrieval step is discarded, as such actions lack grounding in the retrieved content. We additionally remove trajectories containing trivial full-image crops (coordinates [0, 0, 1000, 1000]), which provide no additional information over the original page view. Second, we apply a retrieval completeness criterion: a trajectory is retained only if all ground-truth reference pages appear among its retrieved images, ensuring that the SFT signal is grounded in sufficient evidence. Finally, trajectories with more than 10 retrieval steps are discarded to avoid excessively long context sequences. These filters together yield a candidate pool of trajectories, which are further filtered by answer correctness: only trajectories where the final answer matches the reference answer are retained. This yields approximately 2.5K high-quality training trajectories.

Table 6: Key hyperparameters for RL. Name Number of agent groups Warmup steps ratio Train batch size Mini batch size per GPU Micro batch size per GPU Learning rate (Actor) KL loss coefficient Tensor model parallel size Max prompt length Max response length Max turns Total steps GPU memory utilization

Data Augmentation. After filtering, we apply two targeted modifications to each trajectory. First, all search queries within a trajectory are replaced with the original question text. The teacher model often generates paraphrased or reformulated queries that may omit key details from the original question due to intent drift; the original question, by contrast, contains all critical information needed for retrieval, ensuring every search step stays anchored to the user’s intent. Second, we insert a mandatory verification round at the end of each trajectory. Specifically, the final think in the last assistant turn is appended with a verification intent statement, and the original <answer> action is replaced by a new <search> action. A previously unseen page from the same document is then randomly sampled as the verification image. We use Qwen3-VL-235B-A22B [3] to analyze the new image in the context of the already-found answer, producing a verification reasoning text. A final assistant turn is then appended, containing the verification reasoning followed by the original answer. This augmentation teaches the model to perform one last confirmatory retrieval before committing to an answer, directly corresponding to the verification behavior elicited at inference time by our Verification Hint prompt.

Value Full True True False 16384 3 16 2 1.0e-5 cosine 0.1

Value 5 0.285 8 1 1 1.0e-6 0.01 2 8192 2048 10 100 0.4

Evaluation Details

Judge Prompt. Following VRAG-RL [26], we use Qwen-max-latest as the LLM judge to evaluate model responses. Given the question, reference answer, and model prediction, the judge assigns a binary score (0 or 1) indicating whether the prediction is semantically correct. The exact prompt used is shown in Figure 5. Reliability of Model-as-Judge. The reliability of Qwen-max-latest as an LLM judge has been validated in VRAG-RL [26]. To further corroborate this, we additionally evaluate our results on ViDoSeek using DeepSeek-V3.2 [16] as an alternative judge. As shown in Table 7, the two judges yield highly consistent scores overall, with minor differences at the subtask level: the DeepSeek judge assigns slightly lower scores on Extraction and slightly higher scores on Logic. The small gap confirms that our reported results are not sensitive to the choice of judge model.

Training Hyperparameters

The detailed hyperparameters used during SFT and RL training are listed in Table 5 and Table 6, respectively. All experiments reported in the main paper are conducted with Qwen2.5-VL-7B [4] as the backbone on a single node of 8 NVIDIA A800 GPUs. While VISOR is model-agnostic and can be applied to VLMs of different scales, practitioners may find it beneficial to adjust hyperparameters such as learning rate and context length proportionally when scaling to larger or smaller models.

Table 7: Comparison of evaluation scores on ViDoSeek under two judge models. Judge Qwen-max-latest DeepSeek-V3.2 10

Extraction 73.49 71.47

Logic 76.66 78.67

Overall 74.87 74.61

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

Judge-model Prompt System Prompt:

Character Introduction You are an expert evaluation system for a question answering chatbot. You are given the following information: - the query - a generated answer - a reference answer Your task is to evaluate the correctness of the generated answer. Response Format Your response should be formatted as following: <judge>True or False</judge> If the generated answer is correct, please set "judge" to True. Otherwise, please set "judge" to False. Please note that the generated answer may contain additional information beyond the reference answer.

User Prompt:

Query: {Query Description} Reference Answer: {Reference Answer} Generated Answer: {Generated Answer}

Figure 5: The prompt template used for LLM-as-Judge evaluation.

Dataset Information

Implementation Details of the Baseline

Among all baselines, results for Vanilla RAG, ReAct, Search-R1-VL, and VRAG-RL are taken directly from the VRAG-RL paper [26], as our evaluation strictly follows the same experimental protocol. Results for M3RAG are taken from its original paper [5], where the evaluation setup differs from ours (simpler retrieval configuration). The remaining four baselines—ViDoRAG, EVisRAG, MMSearch-R1, and R1-Router—are reproduced by us under our unified evaluation framework.

We evaluate VISOR on three visually rich document benchmarks. SlideVQA.. SlideVQA [24] is a VQA benchmark built on presentation slides, covering a wide range of real-world slide decks across diverse topics. Questions are split into Single-hop (1,648) and Multi-hop (567) subsets, where multi-hop questions require aggregating evidence from multiple slides within the same deck. Our evaluation uses the full test split of 2,215 questions.

Vanilla RAG. Vanilla RAG is a straightforward retrieval augmented baseline that uses the original question to retrieve relevant pages from the corpus and directly provides the retrieved content to the model for answer generation, without any iterative reasoning or multi-turn interaction. We follow the visual-based variant, where page images are retrieved via ColQwen2.5-v0.1 [6] and fed directly to the VLM.

ViDoSeek. ViDoSeek [25] is a benchmark targeting retrieval augmented question answering over large-scale visually rich corpora, with approximately 6,000 document page images spanning text, tables, charts, and figures. The test set of 1,142 questions is divided into two non-overlapping subsets by question type: Extraction (645 questions) asks models to locate and directly read out specific information from retrieved pages, while Logic (497 questions) requires further inference or computation over the retrieved content to derive the answer.

ReAct. ReAct [30] structures the agent’s behavior as an interleaved Thought–Action–Observation loop, enabling multi-turn retrieval-augmented reasoning. At each turn, the model issues a search query based on its current reasoning state and receives a retrieved page image as the observation, continuing until it produces a final answer.

MMLongBench. MMLongBench [20] is a multi-modal document understanding benchmark emphasizing long-context perception across heterogeneous content types. We retain only questions with verifiable reference answers, yielding 847 questions for evaluation. Questions are tagged by content type—Text (291), Table (217), Chart (178), Figure (290), and Layout (118)—and a single question may carry multiple tags, so the subsets overlap and their counts sum to more than 847.

Search-R1-VL. Search-R1-VL is a visual extension of Search-R1 [13], which introduces multi-turn RL-based reasoning into the RAG loop. The visual variant adapts this framework to image-based retrieval, training on the same dataset and with the same reward and postprocessing methods as VRAG-RL, initialized from a cold-start checkpoint. 11

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

VRAG-RL. VRAG-RL [26] trains an agentic VLM via GRPO-based RL to iteratively retrieve and reason over document page images. It introduces a crop-and-zoom tool for fine-grained perception and uses trajectory-level rewards that jointly optimize retrieval and answer quality.

the model recognizes the crop as unhelpful and falls back to issuing a new search query—restarting the retrieval process unnecessarily. This case illustrates two compounding failure modes: (1) imprecise coordinate prediction that targets an already-legible and non-critical region, and (2) the absence of a correction mechanism, which allows the wasted turn to propagate without recovery guidance. VISOR addresses both issues via Visual Action Evaluation, which gates the crop action before execution, and Visual Action Correction, which redirects the model back to its pre-crop reasoning context when a crop proves uninformative.

M3RAG. M3RAG [5] is a multi-agent framework that decomposes the retrieval-reasoning pipeline into specialized agents for multi-modal document understanding. As its evaluation settings differ from ours in retrieval configuration, we report results directly from the original paper. ViDoRAG. ViDoRAG [25] adopts an actor-critic multi-agent architecture with separate agents for planning, retrieval, and answer generation, enabling iterative reasoning over visually rich documents. We use ColQwen2.5-v0.1 [6] as the single-modal search engine, retrieving the top-10 pages per query. The backbone VLM is Qwen2.5-VL-7B [4]. All other components and procedures follow the original ViDoRAG pipeline without modification.

Figure 7 shows a representative success case of VISOR on a multihop question from SlideVQA: “In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?” Answering this question requires a two-step reasoning chain: first locating the year when Amazon reached 50 fulfillment centers, then retrieving sales breakdown data for that specific year. The trajectory unfolds over five retrieval steps. At Step 1, the model issues a search and receives an irrelevant page about Amazon India—a different document with no bearing on the question. The model correctly identifies this as uninformative and continues searching. At Step 2, VISOR retrieves a timeline slide from the Amazon whitepaper showing that 50 fulfillment centers were reached in 2010. This intermediate finding is distilled into the Evidence Space, bridging the two sub-questions. At Step 3, VISOR retrieves a category evolution slide covering Amazon’s product mix from 1995 to 2011, providing useful context but no sales figures for 2010. At Step 4, VISOR retrieves a bar chart showing the Media vs. Others sales split from 1997 to 2010. Grounded by the year stored in the Evidence Space, the model directly reads off the 2010 value: Others (∼60%) > Media (∼40%), and arrives at the answer. At Step 5, VISOR performs a verification search and retrieves a page from a different analytics document containing a bar chart—but the chart’s content is unrelated to the question. The model correctly recognizes this page as irrelevant and commits to the answer already derived from accumulated evidence. Notably, the retrieved pages span three distinct source documents: Steps 2–4 all come from the Amazon whitepaper, while Step 1 and Step 5 each originate from entirely unrelated documents. This cross-document retrieval is handled transparently by VISOR’s per-trajectory deduplication and Evidence Space, which keeps the agent anchored to the original question across all turns regardless of what document each page originates from. A method without persistent evidence management would risk losing the intermediate finding (2010) as irrelevant pages accumulate, leading to search drift or an incorrect answer. Figure 8 illustrates a representative failure case of VISOR on a multi-hop question from MMLongBench: “As of Q3 2015, is Vietnam’s adoption rate of iOS 9 higher or lower than the global average rate? What is the difference in percentage?” This question requires retrieving two separate slides from the same report and combining a country-level bar chart with a global pie chart. VISOR correctly retrieves both reference pages. At Step 1, the model fetches the Vietnam iOS breakdown slide, which contains a

EVisRAG. EVisRAG [23] introduces evidence-based reasoning over multiple retrieved images, explicitly extracting and structuring evidence from each retrieved page to support multi-image understanding. We use ColQwen2.5-v0.1 [6] as the retriever in place of the original VisRAG-Ret, retrieving the top-3 pages per query. The model weights are the officially released EVisRAG-7B. All other settings follow the original EVisRAG configuration. MMSearch-R1. MMSearch-R1 [27] integrates multimodal search into the reasoning loop, employing cross-modal retrieval mechanisms to fetch contextually aligned information in both visual and textual forms. The original MMSearch-R1 provides two tools: text search and image search (image-to-image retrieval). In our setting, we adapt text search to retrieve images of document pages via ColQwen2.5-v0.1 [6]. Since image search is not applicable in the document retrieval scenario, invoking it returns a prompt instructing the model to use text search instead. The model weights are the officially released MMSearch-R1-7B. All other settings follow the original configuration. R1-Router. R1-Router [21] employs a dynamic routing mechanism trained via Step-GRPO, generating intermediate sub-queries during the reasoning process and directing them selectively to the most appropriate retrieval tool, thereby minimizing unnecessary retrievals while adaptively integrating external evidence. In our setting, all retrieval tools are adapted to retrieve images of document pages via ColQwen2.5-v0.1 [6], returning the top-5 pages per query. The maximum number of interaction turns is set to 3. All other settings follow the official R1-Router configuration.

Case Study

Bbox Case

Figure 6 shows a representative failure case of indiscriminate cropping from VRAG-RL. The question asks about the DBO contract award schedule. At Step 𝑥, the model correctly identifies a relevant section in the retrieved page and issues a <bbox> action targeting the top-left corner of the image. However, the specified coordinates capture only the heading text “so far” (part of “Progress so far”), missing the actual content entirely. The resulting cropped patch is uninformative, yet consumes a full interaction turn. At Step 𝑥+1, 12

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

Step x: <think>The user is asking about the DBO contract award schedule. The image contains a section that mentions 'DBO contract award by Dec 2018'. To provide a clearer view of this specific information, I will crop the relevant part of the image.</think> <bbox>[92, 10, 155, 32]</bbox> Step x+1: <think>The cropped image shows the text 'so far', which is part of the heading 'Progress so far'. This indicates that the relevant information about the DBO contract award schedule is likely located above this section. I will now search for the full image to find the complete 'Progress so far' section.</think> <search>full image of 'Progress so far' section</search>

Figure 6: A representative case illustrating redundant crop usage in VRAG-RL. grouped bar chart clearly annotated with per-version adoption rates. The model accurately reads Vietnam’s iOS 9 adoption for Q3/2015 as 13%. At Step 2, the model retrieves the global iOS breakdown slide, which shows a three-sector pie chart with iOS 9 (blue, 51%), iOS 8 (gray, 41%), and Earlier (light gray, 8%). However, the model misidentifies the sectors: it reads the gray iOS 8 sector (41%) as the iOS 9 share, overlooking the legend. As a result, the model computes 41% − 13% = 28% instead of the correct 51% − 13% = 38%. The directional answer (“lower”) is correct, but the numerical answer is wrong due to this legend misinterpretation. This failure mode cannot be remedied by better retrieval or longer reasoning: the bottleneck is purely visual—the model fails to correctly associate legend colors with the corresponding pie sectors, a task that requires precise fine-grained perception of smallfont legend entries alongside visually similar chart elements. It highlights an inherent limitation of the current 7B backbone and suggests that improved visual grounding capabilities in future VLMs would directly translate to higher benchmark performance.

<think> content is extracted and appended to the Evidence Space under the corresponding image source filename. The accumulated Evidence Space is then injected into subsequent prompts (e.g., the Image Observation Prompt and Forced-answer Prompt) so that the agent can reason over all prior findings without relying on raw conversation history. Image Observation Prompt. Appended after each retrieved page image, this prompt embeds the Intent Injection—repeating the original question—to keep the agent anchored to the user’s query regardless of the retrieved content. It instructs the agent to analyze the image and record findings in <think> before issuing the next action. Crop Observation Evaluation Prompt. This prompt is appended alongside the Image Observation Prompt and serves as the Visual Action Evaluation gate: it reminds the agent to propose <bbox> only if critical details are visually unclear, discouraging unnecessary crop actions on already legible regions. Crop Observation Correction Prompt. Returned after a <bbox> action along with the cropped image, this prompt implements Visual Action Correction: it constrains the agent’s next action to either <search> or <answer>, preventing chained crops, and instructs the agent to refer back to its previous <think> if the cropped region turns out to be uninformative. Verification Hint. Injected when the system detects that the current retrieval step may be a verification search (triggered by the agent’s prior <think> signaling readiness to confirm), this prompt instructs the agent to issue a final <answer> immediately if the new image confirms the accumulated evidence, and to continue searching only if a direct contradiction is found. No-image Prompt. Triggered when all top-𝑘 retrieved pages have already been shown to the agent in prior turns, this prompt notifies the agent that no new pages are available and directs it to produce a final answer solely from the accumulated Evidence Space. Forced-answer Prompt. Injected at the final turn when the maximum number of turns is reached, this prompt requires the agent to produce an answer unconditionally, providing the full Evidence Space as context to ground the response. Invalid Action Prompt. Returned whenever the agent produces a malformed output (e.g., missing <think> tags, an unrecognized

Prompt Templates

All prompt templates used in VISOR are illustrated in Figure 9. The prompts described below are designed for the 7B backbone (Qwen2.5-VL-7B); for smaller models such as the 3B variant, we lightly simplify the wording to reduce comprehension burden and improve format adherence. System Prompt. Injected once at the start of each trajectory, the System Prompt defines the agent’s action space and behavioral rules. It specifies three valid actions: <search> to retrieve a new page, <bbox> to zoom into an unclear region, and <answer> to produce a final response. Crucially, it mandates that every response begin with a <think> block, and requires the agent to perform one final verification search before committing to an answer. It also instructs the agent to extract all potentially useful information from each retrieved image into its <think>, as these thoughts are recorded into the persistent Evidence Space for later reference. User Prompt. Appended immediately after the System Prompt, the User Prompt provides the original question for the current trajectory. Evidence Space. The Evidence Space is a structured ledger maintained across all turns. After each retrieval step, the agent’s 13

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Good Case User Query: In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category? Reference Answer: Others

Step 1: <think>To answer this question, I need to determine the year in which Amazon had 50 fulfillment centers and then compare the number of products sold in the 'media' category versus the 'others' category for that year. Since the provided image does not contain this information, I will perform a search to find relevant data about Amazon's fulfillment centers and product sales categories.</think> <search>In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?</search> Step 2: <think>The image provides information about Amazon India's fulfillment centers, stating it has two centers in Mumbai and Bangalore, but it does not mention having 50 fulfillment centers or any data on product sales by category. Therefore, I need to search for additional information to determine the year Amazon had 50 fulfillment centers and compare media versus others category sales.</think> <search>In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?</search> Step 3: <think>The image clearly shows the growth of Amazon's fulfillment centers over the years, with a milestone of '50 fulfillment centers' in 2010. However, it does not provide any information about product sales by category (media vs. others). To answer the question, I need to find additional data on Amazon's product sales distribution for the year 2010. I will perform a search to gather this specific information.</think> <search>In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?</search> Step 4: <think>The image shows the evolution of Amazon's category offerings from 1995 (books) to 2011 (16 main categories). However, it does not provide specific data on sales figures for the 'media' category (e.g., books, music, movies) compared to other categories in 2010, the year Amazon had 50 fulfillment centers. To answer the question, I need to find sales data or market reports that compare these categories for that specific year. I will perform a search to gather this information.</think> <search>In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?</search> Step 5: <think>The bar chart shows the percentage of sales for the 'Media' category (Books, Music, and DVD/Video) compared to the 'Others' category in the U.S. e-commerce market from 1997 to 2010. In 2010, the year Amazon had 50 fulfillment centers, the 'Media' category accounted for approximately 40% of sales, while the 'Others' category accounted for the remaining 60%. This indicates that more products were sold in the 'Others' category compared to the 'Media' category in 2010. I can now answer the question. I want to do a verification round, so I will search again.</think> <search>In the year in which Amazon had 50 fulfillment centers, were more products sold in the media category or others category?</search>

Figure 7: A success case of VISOR on a multi-hop SlideVQA question. Retrieved page images are shown at the top for layout reasons; the corresponding reasoning steps are presented below. Step 1 retrieves an irrelevant page from a different document. Step 2 identifies the year 2010 from an Amazon fulfillment center growth chart. Step 3 retrieves a category evolution slide providing context. Step 4 locates the Media vs. Others sales breakdown chart, from which the answer is read. The final verification search (Step 5) returns a page with an unrelated chart from a different document, yet VISOR correctly produces the answer from accumulated evidence. 14

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

Bad Case User Query: As of Q3 2015, is vietnam's adoption rate of iOS 9 higher or lower than a global average rate? What is the difference in percentage? Reference Answer: ['lower', '38'] Step 1: <think>To determine whether Vietnam's adoption rate of iOS 9 is higher or lower than the global average and calculate the difference, I need to find specific data on Vietnam's iOS 9 adoption rate and the global average. I will start by searching for this information.</think> <search>As of Q3 2015, is vietnam's adoption rate of iOS 9 higher or lower than a global average rate? What is the difference in percentage?</search>

Step 2: <think>The image provides Vietnam's adoption rate for iOS 9 as 13% specifically for Q3/2015. However, the global average adoption rate for iOS 9 during the same period is not directly available in the image. I will need to search for the global average adoption rate to calculate the difference.</think> <search>As of Q3 2015, is vietnam's adoption rate of iOS 9 higher or lower than a global average rate? What is the difference in percentage?</search>

Step 3: <think>The image provides the global adoption rate for iOS 9 as 41% (as of September 19, 2015). Vietnam's adoption rate for iOS 9 is 13% as of Q3/2015. Comparing these values, Vietnam's adoption rate is lower than the global average. The difference in percentage is 41% - 13% = 28%. I can now answer the question. I want to do a verification round, so I will search again.</think> <search>As of Q3 2015, is vietnam's adoption rate of iOS 9 higher or lower than a global average rate? What is the difference in percentage?</search>

Step 4: <think>The new image focuses on Android OS adoption in Vietnam, specifically showing Lollipop at 35%, but does not provide data on iOS 9 adoption or global iOS 9 averages for comparison. This does not support or contradict the answer about Vietnam's iOS 9 adoption being 13% with a global average of 41%, as the image lacks the necessary iOS 9 figures for verification. The answer is confirmed.</think> <answer>As of Q3 2015, Vietnam's adoption rate of iOS 9 is lower than the global average rate. The difference in percentage is 28% (41% - 13%).</answer>

Figure 8: A failure case of VISOR on a multi-hop MMLongBench question. Both reference pages are retrieved correctly, but the model misreads the legend of a pie chart, confusing the iOS 8 slice (gray, 41%) for iOS 9 (blue, 51%), leading to an incorrect final answer despite successful retrieval.

Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao

Prompt Templete for VISOR System Prompt: You are a visual reasoning agent. You will search for images to answer the user's question. Rules: 1. Every response must start with <think> </think> where you reason about what you see and what to do next. 2. After thinking, output exactly one action: - <search>query</search> to retrieve images. Each search returns one new image; if you repeat a query, you will get a different image from the same document. Use the original question as your query unless you have a specific reason to change it. - <bbox>[x1,y1,x2,y2]</bbox> to zoom into an unclear region (normalized to 0-1000, only on full images). - <answer>your answer</answer> once you are ready to give your final answer. 3. Before answering, you must do one final search using the original question to verify your answer. After receiving the new image, give your <answer> immediately — unless the new image provides a directly conflicting answer to the question, in which case search once more and then give your <answer> immediately regardless. 4. When given an image, analyze it fully in <think> </think> and extract every potentially useful piece of information — your thoughts will be recorded into a COLLECTED EVIDENCE table for later reference, so be as thorough as possible. If the image contains no relevant information, explicitly state that (e.g., "This image does not contain information related to the question.") to avoid polluting the evidence. Only propose <bbox> if critical details are visually unclear. A "COLLECTED EVIDENCE" section may appear summarizing findings from previous steps — use it alongside new images to guide your reasoning. Good example: <think>This image shows the Nordic Ecolabelling Portal login page. The page clearly states that Microsoft Edge or Google Chrome is recommended as the web browser. I have enough information to answer. I will do one final verification search.</think><search>Apply for Nordic Swan Ecolabel license, what is recommended as a web browser?</search> <think>This image shows a different section of the portal with no information about browser recommendations. It does not contradict my answer. My answer stands.</think><answer>Microsoft Edge or Google Chrome</answer>

User Prompt:

Query: {Query Description}

Evidence Space:

### COLLECTED EVIDENCE Image Source: {filename} - {thought_1} - {thought_2} ...

Image Observation Prompt with Intent Injection:

Image loaded, analyze any possible useful information for the question: [{Query Description}] in your think, then continue your action after <think> </think>.

Crop Observation Correction Prompt:

This is the cropped image, analyse it based on the question: [{Query Description}], and after <think> </think>, you can just use <search> or <answer> this time. If the cropped image is incorrect, please pay attention to the previous <think>.

Crop Observation Evaluation Prompt: Only propose <bbox> if critical details are visually unclear. Verification Hint:

This may be a verification search. Based on all the evidence collected so far, if the new image confirms the answer, provide it directly inside <answer> and </answer>. Only continue searching if you find a contradiction or missing information.

Invalid Action Prompt:

Your previous action is invalid. You must conduct reasoning inside <think> and </think> first every time you get new information. After reasoning, if you find you lack some knowledge, you can call a search engine by <search> query </search> and user will return the searched results. Every time you retrieve an image, you have the option to crop it to obtain a clearer view, the format for coordinates is <bbox>[x1, y1, x2, y2]</bbox>. You can search as many times as your want. If you find no further external knowledge needed, you can directly provide the answer inside <answer> and </answer>, without detailed illustrations. For example, <answer> Beijing </answer>. Please try again.

No-image Prompt:

No more new images can be retrieved. Based on all the evidence you have collected so far, please provide your final answer inside <answer> and </answer> after <think> and </think>. For example, <answer> Beijing </answer>.

Forced-answer Prompt:

This is your last response. Based on the COLLECTED EVIDENCE, provide your final answer for the question: [{Query Description}] inside <answer> </answer> tags after <think> </think>. ### COLLECTED EVIDENCE {Evidence Space}

Figure 9: All prompt templates used in VISOR. Each template corresponds to a distinct interaction event in the agent loop.

VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning

action token, or out-of-range <bbox> coordinates), this prompt describes the correct format and instructs the agent to retry.

Analysis. VISOR is largely insensitive to 𝑊 for small values (𝑊 ≤ 3), with a performance gap of less than 2 percentage points relative to the optimal setting. This robustness stems from the complementary role of the Evidence Space (ES): even when earlier turns are evicted from the raw context window, their distilled evidence is preserved in E𝑡 and re-injected at every subsequent turn, so the agent does not suffer a hard information loss. 𝑊 = 1 causes a mild accuracy drop (70.43% vs. 72.37%). When the agent issues a <bbox> crop action, a window of size 1 retains only the crop observation, discarding the original full-page image that motivated the crop. The agent can no longer directly compare the cropped detail with its broader page context, slightly impairing finegrained reasoning. 𝑊 = 2 recovers this capability: the preceding retrieve turn (full-page image plus initial reasoning) is kept alongside the crop result, giving the agent a complete retrieve-then-crop reasoning chain. Increasing 𝑊 to 4 leads to a more substantial drop (68.17%), approaching the ablation result of removing SW entirely (67.54%). With a larger window, older page images that are no longer actionable for the current turn are kept in the live context, introducing additional visual noise and increasing token consumption without proportional benefit. Taken together, 𝑊 = 2 strikes the best balance between context efficiency and information completeness, and we adopt this setting throughout the paper.

Sensitivity Analysis of Sliding Window Size

In the Dynamic Trajectory mechanism, the sliding window size 𝑊 controls how many recent interaction turns are retained in the live context at each step. A smaller 𝑊 bounds the number of incontext image tokens but risks discarding recent raw observations, while a larger 𝑊 preserves more interaction history at the cost of increased token consumption. We empirically set 𝑊 = 2 in all main experiments; here we investigate how sensitive VISOR’s performance is to this choice. Experimental Setup. We vary 𝑊 ∈ {1, 2, 3, 4} and evaluate the fine-tuned VISOR (Qwen2.5-VL-7B, SFT+RL) on SlideVQA (2,215 test questions). All other settings are identical to the main experiments. Results are reported in Table 8. Table 8: Sensitivity of VISOR to sliding window size 𝑊 on the fine-tuned model (SlideVQA). We report overall accuracy (%). Window Size 𝑊 1 2 3 4

SlideVQA 70.43 72.37 71.56 68.17

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

Record · ID 5995 · SHA-256 eacf46ff6e6a6ff8
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.