Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
Qianhao Yuan1,2 , Jie Lou3 , XingYu3 , Hongyu Lin1 , Le Sun1 , Xianpei Han1 , Yaojie Lu1 Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences 2 University of Chinese Academy of Sciences 3 Xiaohongshu Inc. [email protected] [email protected] {hongyu,sunle,xianpei,luyaojie}@iscas.ac.cn https://github.com/VisionOPD/Vision-OPD
Abstract Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evidence in the full image. We observe a regional-to-global perception gap: the same MLLM answers fine-grained questions more accurately when conditioned on evidencecentered crops than on the corresponding full images, suggesting that many failures stem from difficulty to focus on relevant evidence rather than insufficient local recognition ability. Motivated by this observation, we propose Vision-OPD (Vision On-Policy Distillation), a regional-to-global self-distillation framework that transfers the model’s own privileged regional perception to its full-image policy. Vision-OPD instantiates two conditional policies from the same MLLM: a cropconditioned teacher and a full-image-conditioned student. The student generates on-policy rollouts, and Vision-OPD minimizes token-level divergence between the teacher and student next-token distributions along these rollouts. This enables the model to internalize the benefit of visual zooming without external teacher models, ground-truth labels, reward verifiers, or inference-time tool use. Experiments on multiple fine-grained visual understanding benchmarks show that Vision-OPD models achieve competitive or superior performance against much larger open-source, closed-source, and “Thinking-with-Images” agentic models. 76 75.7 74.7 Vision-OPD (Ours) 73.8 74 72.5 Closed-Source Models 72 70.9 Open-Source Models 69.5 70 68.4 67.8 67.6 68 66 64 61.2 62 60 58.1 58 56 -9B -4B -Pro 35B K2.5 PT-5.4 -4.6V PT-5.2 -7B-RL -V-4.5 97B M L OPD ini-3.1 n-OPD n3.5-3 3-VL-2 KimiG G GLM n o-V iniCP o o n i e M Visi Qw Qwe Gem Vis MiM
Average
arXiv:2605.18740v1 [cs.CV] 18 May 2026
1
Figure 1: Average scores across fine-grained visual understanding benchmarks, including V* Bench, ZoomBench, HR Bench 4K, HR Bench 8k, MME-RealWorld-Lite and MME-RealWorld-CN. VisionOPD-4B/9B demonstrate superior performance compared with much larger open-source models (e.g., Qwen3.5-397B) and closed-source models (e.g., GPT-5.4, Gemini-3.1-Pro). Preprint.
1
Introduction
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding and reasoning [5, 8, 25, 34]. However, they still struggle with fine-grained visual understanding, where the answer often depends on small but decisive details that occupy only a fraction of the image [19, 39, 58]. In full-image inference, these details are often easy to overlook amid many visual tokens. As a result, MLLMs may produce plausible answers based on the global scene while missing the local evidence that is truly needed for the question. Recent “Thinking-with-Images” methods [62, 67] address this issue by equipping MLLMs with agentic visual tool use, enabling them to crop, zoom, and inspect the region of interest during inference. By making local evidence more salient, these methods improve fine-grained visual understanding. However, they introduce substantial inference overhead due to repeated image encoding and model calls. A natural question arises: can the benefit of visual zooming be internalized through training, so that the model can use fine-grained evidence from the full image without additional tool use? Our motivation comes from a simple observation: the same MLLM often answers a fine-grained question more accurately when conditioned on the evidence-centered crop than on the corresponding full image. This performance gap reveals a broader bottleneck: MLLMs can often interpret the relevant evidence once it is made salient, but struggle to exploit it when it is embedded in the global visual context. This observation suggests a natural training signal: the model’s own crop-conditioned behavior can serve as privileged supervision for improving its full-image behavior. A straightforward way to exploit such privileged supervision is supervised fine-tuning (SFT) on crop-conditioned responses. However, this trains the model on trajectories generated under privileged crop inputs, leading to distribution mismatch and exposure bias [1]. Reinforcement learning with verifiable rewards methods, such as GRPO [32] and DAPO [53], can optimize on-policy rollouts, but usually provide only sparse sequence-level feedback, and require ground-truth labels and verifiers. On-Policy Distillation (OPD) [1, 22] combines on-policy sampling with dense token-level supervision, but existing OPD methods typically rely on external stronger teachers or ground-truth label [66]. We propose Vision-OPD, a regional-to-global self-distillation framework for fine-grained visual understanding. It instantiates two policies from the same MLLM with different visual conditions: a crop-conditioned teacher that observes the evidence-centered crop as a privileged input, and a full-image-conditioned student that observes the full image as in standard inference. The student first generates on-policy rollouts from the full image. For each student-generated prefix, Vision-OPD computes the logit distributions of both the crop-conditioned teacher and the full-image-conditioned student, and minimizes their divergence. In this way, the model transfers its own privileged cropconditioned behavior to its full-image-conditioned policy on the student’s generation trajectory, without external teachers, ground-truth labels, reward verifiers, or inference-time visual tool use. We conduct extensive experiments to validate the effectiveness of Vision-OPD. With only 6.2K synthetic training data, Vision-OPD enables 4B/9B models to outperform much larger open-source models (e.g., Qwen3.5-397B), closed-source models (e.g., GPT-5.4, Gemini-3.1-Pro), and agentic “Thinking-with-Images” methods on fine-grained visual understanding tasks that require dense local evidence. On hold-out tasks beyond the training distribution, Vision-OPD maintains general visual understanding and reasoning ability, indicating that the gains do not come at the cost of forgetting. Further analyses verify the necessity of on-policy sampling and dense token-level supervision, and show that Vision-OPD substantially narrows the regional-to-global perception gap in MLLMs. Our contributions are summarized as follows: • We introduce a regional-to-global self-distillation formulation for fine-grained visual understanding, with privileged crop-conditioned behavior as supervision for full-image inference. • We propose Vision-OPD, an on-policy self-distillation framework where a crop-conditioned policy supervises a full-image policy on the student’s rollouts via token-level supervision. • Comprehensive experiments validate the effectiveness of Vision-OPD. We demonstrate that Vision-OPD can significantly narrow the regional-to-global gap, and the on-policy sampling and dense supervision are important to its success. 2
Global
What color are the ear protectors worn by the person?
A. red B. black C. green D. blue Global (Left)
A
B
C
D
Regional (Right)
A
B
C
D
Gap
Regional
Qwen3.5-4B
+21.7
Qwen3.5-9B
+19.5
GLM-4.6V
+22.1
GPT-5.4
+19.3 +18.1
Gemini-3.1-Pro 45
50
55
60
65
70
Accuracy (%)
75
80
Figure 2: A case of the regional-to-global gap, Figure 3: The accuracy of regional inputs is conbased on Qwen3.5-9B. The global image input sistently higher than the accuracy of global inleads to the wrong answer, while the cropped puts, revealing a persistent regional-to-global gap region input yields the correct answer. across multiple MLLMs.
2
Preliminary: distillation and on-policy distillation (OPD)
Knowledge distillation transfers the behavior of a strong teacher into a weaker student by matching the teacher’s distributions. For an input x and output sequence y = (y1 , . . . , y|y| ), traditional supervised distillation is off-policy: the student is trained on teacher-induced prefixes y<t and minimizes |y| X 1 D (pT (· | y<t , x) ∥ pS (· | y<t , x)) . (1) LSupervised Distillation (θ) = E(x,y)∼S |y| t=1 This supervised distillation objective provides dense token-level supervision, but at inference time the model conditions on its own prefixes rather than those observed in S, which creates a state-distribution mismatch and can cause errors to compound over long horizons [31]. On-policy distillation (OPD) mitigates this mismatch by sampling y ∼ pS (· | x) from the current student and querying the teacher on the prefixes generated by the student [1, 22, 51]. The objective is |ŷ| X 1 LOPD (θ) = Ex∼S, ŷ∼pS (·|x) D (pT (· | ŷ<t , x) ∥ pS (· | ŷ<t , x)) . (2) |ŷ| t=1 OPD combines the on-policy relevance of reinforcement learning with the dense token-level guidance of distillation, making it a natural fit for autoregressive generation and reasoning.
3
Vision-OPD
3.1
Motivation: less is more for fine-grained visual understanding
MLLMs struggle with fine-grained visual understanding where small but decisive evidence is overwhelmed by global context. Recent “Thinking-with-Images” methods [3, 62, 67] demonstrate that zooming into the region of interest improves fine-grained understanding, indicating that the bottleneck lies not in the model’s recognition ability, but in its capacity to focus on relevant evidence within the full image. This can be verified by comparing the model’s performance under two conditions: when it receives the full image (global input) versus when it receives only an evidence-centered crop (regional input). If the model answers correctly with the crop but fails with the full image, it indicates that the model can recognize the evidence but struggles to focus on it within the global context. Figure 2 illustrates a qualitative case on Qwen3.5-9B. The question asks about the ear-protector color. Given the full image, the model predicts black; given only the cropped region, it correctly predicts green. The decisive evidence is recognizable in isolation, but is overwhelmed by the global context. Evaluation on ZoomBench [46] confirms that this pattern is systematic. As shown in Figure 3, regional-input accuracy consistently exceeds global-input accuracy by 18–22 points. Even much larger models and closed-source models, such as GLM-4.6V, GPT-5.4, and Gemini-3.1-Pro, exhibit substantial gaps, confirming that parameter scaling alone does not close this regional-to-global gap. 3
Object Recognition & Segmentation
Student Input Image
Student Prompt and
Teacher Prompt and
Student Policy
Teacher Policy
On-Policy Sample Question Generator Question
Crop Resize Teacher Input Image
Learning Objective
What color is the vehicle? Only foucus on the objects inside the red bounding box in the image to answer this question.
Gradient only flow through the student’s logits
Figure 4: Overview of Vision-OPD. Left: Fine-grained visual questions are generated on evidencecentered crops and grounded back to the full image via bounding-box overlay. Right: A teacher policy pT (· | xcrop ) and a student policy pS (· | xglobal ) are instantiated from the same MLLM. The student generates on-policy rollouts y ∼ pS , and the per-token divergence D(pT ∥pS ) along these rollouts provides dense supervision. Gradients flow only through the student’s logits, enabling label-free self-distillation for fine-grained visual understanding.
This observation motivates Vision-OPD: since a model’s own regional perception consistently outperforms its global perception, we can use the former as privileged supervision for the latter, internalizing the benefits of zooming into a single forward pass, without inference-time tool use. 3.2
Method: regional-to-global on-policy self-distillation
To exploit the regional-to-global gap for training, we construct a dataset of triplets D = ′ {(xi , x′i , qi )}N i=1 , where each triplet pairs a full image x with its evidence-centered crop x and a fine-grained question q. As illustrated in Figure 4 (left), inspired by recent work on multimodal data synthesis [42, 46], we first apply object recognition and segmentation to propose bounding box segmentation based on a raw image I, retaining only small regions (area ratio < τ ) that are likely to contain fine-grained evidence hidden in the image. For each retained region R, we use an MLLM as a question generator to produce a question q that are answerable from R alone. To ground the question back to the full image and avoid referential ambiguity, the bounding box of R is overlaid onto I to produce x, and a spatial constraint is appended to q (e.g., “Only focus on the objects inside the red bounding box”). Then we crop I to the bounding box of R and resize it by 2× to produce x′ . Each resulting triplet (x, x′ , q) thus presents the same question under two visual conditions: the student sees the full image with spatial guidance, while the teacher sees only the isolated crop. The gap between the two conditions directly serves as the learning signal for self-distillation. To make comparison with alternative training strategies, such as off-policy distillation (SFT), RLVR and OPSD [66], we also utilize Qwen3.5-397B as an answer generator to generate ground-truth labels for these methods. We sample multiple responses by giving the region R as image input, and keep a question only when the majority answer reaches a strict consensus (> 0.75). In total, we synthesize 6.2K samples for training. Given the synthesized dataset, we now describe how Vision-OPD leverages these paired views for training (Figure 4, right), with a complete summary in Algorithm 1. We instantiate two conditional distributions from the same MLLM pθ by varying the visual conditioning context. The teacher policy conditions on the evidence-centered crop x′ as privileged visual information, pT (· | x′ , q) = pθ (· | x′ , q). The student policy observes the full image x and the question q, pS (· | x, q) = pθ (· | x, q). The student sees the global scene, while the teacher sees only the zoomed-in crop where the evidence is unambiguous, which grants the teacher a privileged view in which fine-grained details are isolated. Given a training sample (x, x′ , q), the student generates an on-policy response y = (y1 , . . . , y|y| ) ∼ pS (· | x, q). Both policies then evaluate this student-generated trajectory. At each position n, they induce next-token distributions over yn ∈ V conditioned on the same student prefix y<n = (y1 , . . . , yn−1 ): pS (yn | x, q, y<n ) and pT (yn | x′ , q, y<n ). By re-evaluating the same trajectory 4
Algorithm 1 Vision-OPD Require: Training dataset D = {(xi , x′i , qi )}N i=1 ; MLLM pθ ; divergence D (e.g., JSDβ ) 1: Let pS (· | x, q) := pθ (· | x, q) and pT (· | x′ , q) := pθ (· | x′ , q) be the same model pθ under different visual conditioning. 2: for step = 1, . . . , M do 3: Sample a batch B ⊂ D 4: for all (x, x′ , q) ∈ B do 5: Sample on-policy response y ∼ pS (· | x, q) 6: Compute the token-wise divergence along the student rollout: |y| 1 X ℓ(x, x , q) ← D pS (· | x, q, y<n ) stopgrad pT (· | x′ , q, y<n ) |y| n=1 ′
7:
1 Calculate loss LVision-OPD (θ) ← |B|
′ (x,x′ ,q)∈B ℓ(x, x , q) and update θ
P
under a cleaner local view, the teacher’s token-level distribution naturally encodes sharper attention to fine-grained visual evidence without any additional decoding. The training objective minimizes the expected per-token divergence between teacher and student over the student’s own rollouts. Given a student-generated sequence y, the averaged divergence is D(pT ∥pS )(y | x, x′ , q) =
|y| 1 X D pT (· | x′ , q, y<n ) pS (· | x, q, y<n ) , |y| n=1
(3)
where D can be any distribution divergence such as the generalized Jensen–Shannon divergence JSDβ for a weight β ∈ [0, 1]. It is defined as JSDβ (pT ∥pS ) = β DKL (pT ∥m) + (1 − β) DKL (pS ∥m), where m = β pT + (1 − β) pS is the interpolated mixture. The overall loss minimizes the expected divergence over on-policy student samples: LVision-OPD (θ) = E(x,x′ ,q)∼D Ey∼pS (·|x,q) [D(pT ∥pS )(y | x, x′ , q)] . (4) Gradients are backpropagated only through the student policy pS , while the teacher pT acts as a fixed target conditioned on the privileged crop (x′ , q). Using the student’s own generated sequence y as the training prefix aligns the state distribution between training and inference. This avoids the compounding errors that arise from prefix mismatch in off-policy distillation [31]. The dense per-token divergence signal contrasts with the sparse binary rewards of RLVR methods, such as GRPO and DAPO. In Vision-OPD, every token receives a meaningful gradient, so training does not stall when all samples in a batch happen to be correct or incorrect. Taken together, Vision-OPD satisfies five desiderata: on-policy sampling, dense token-level supervision, no external teacher, no ground-truth labels, and no verifier. Because the crop x′ is extracted from unlabeled images by a fully automated data synthesis pipeline, the method is compatible with arbitrary image corpora and internalizes fine-grained visual understanding into a single forward pass.
4
Experiments
4.1
Experimental settings
Model training. We apply Vision-OPD to Qwen3.5-4B/9B [30], with our 6.2K synthetic data. We use JSD (β = 0.5) as the divergence objective, and approximate the divergence via top-K distillation, which computes only the top-K logits of the student and the corresponding teacher logits, alongside a tail-probability term. With K = 100, this avoids the high memory overhead of full-vocabulary logit distillation [66] while capturing most of the distributional information, since the tokens beyond the top-100 typically cover less than 1 × 10−13 of the probability mass in our scenario. The teacher is regularized via exponential moving average (EMA). We set the maximum on-policy generation length to 1024, and training epoch to 1. Benchmarks. We evaluate on two groups of benchmarks. The first group targets fine-grained visual understanding: V* Bench [49] measures the accuracy of locating and recognizing tiny visual targets 5
Table 1: Comparison with SOTA MLLMs. We report accuracy (%) for each model. Among opensource models (single forward pass), the best results are highlighted in bold, and the second-best are underlined. Vision-OPD achieving the best overall performance. Param HR Bench HR Bench MME-RW MME-RW V* Bench ZoomBench Average Size 4K 8K Lite CN
Model
“Thinking-with-Images” Agentic Models DeepEyes Thyme DeepEyesV2 SenseNova-MARS
7B 7B 7B 8B
85.86 82.20 81.68 92.15
GPT-5.2 GPT-5.4 Gemini-3.1-Pro
-
79.06 76.96 87.96
46.51 45.09 44.97 47.81
75.13 77.00 77.88 83.13
72.63 72.00 73.75 78.38
53.20 55.18 51.59 43.67
64.09 64.59 65.07 68.90
66.24 66.01 65.82 69.01
47.06 48.10 49.50
68.80 70.93 73.31
67.55 68.42 74.74
Closed-Source Models (Single Forward Pass) 50.89 52.66 61.18
81.12 84.00 89.63
78.38 77.88 86.88
Open-Source Models (Single Forward Pass) Qwen3.5 Qwen3.5 MiMo-VL-RL Qwen3-VL-Instruct ZwZ MiniCPM-V-4.5 GLM-4.6V Qwen3-VL-Instruct Qwen3.5 Kimi-K2.5
4B 9B 7B 8B 8B 9B 106B 235B 397B 1T
84.29 82.72 83.25 84.82 87.96 70.68 86.91 91.10 87.96 87.43
47.69 52.07 45.68 42.96 56.69 42.60 50.06 56.09 57.16 52.43
84.38 85.75 73.50 79.63 83.63 69.63 82.13 86.13 89.38 80.00
80.13 80.63 69.38 75.25 81.75 61.50 78.88 80.38 85.50 75.25
35.80 46.12 39.50 46.53 54.35 42.31 43.10 42.83 45.39 56.38
63.70 67.67 55.89 64.61 68.09 61.64 65.62 69.04 69.82 65.71
66.00 69.16 61.20 65.63 72.08 58.06 67.78 70.93 72.54 69.53
Vision-OPD (Ours) Vision-OPD (Ours)
4B 9B
92.15 94.76
59.76 65.80
84.50 88.13
80.38 85.50
55.45 49.56
70.76 70.46
73.83 75.70
within complex scenes; ZoomBench [46] requires models to answer questions that depend on details at varying zoom levels; HR Bench [43] evaluates high-resolution perception at two resolutions (4K and 8K); and MME-RealWorld [61] covers real-world scenarios with high-resolution photographs. The second group serves as holdout tasks for evaluating generalization beyond the training distribution, including MMVP [37], CV-Bench [36], MMStar [2], and POPE [18]. These holdout benchmarks measure whether models retain general multimodal capabilities after fine-grained specialization. Baselines. We compare against two groups of baselines. The first group evaluates Vision-OPD against existing SOTA models: (a) “Thinking-with-Images” agentic models that dynamically crop and zoom into the image region through multi-step reasoning, including DeepEyes [67], Thyme [62], DeepEyesV2 [12], and SenseNova-MARS [3]; (b) closed-source models, including GPT-5.2 [26], GPT-5.4 [27], and Gemini-3.1-Pro [9]; and (c) open-source models of varying scales, including MiMo-VL-7B-RL [50], Qwen3-VL-Instruct [52], ZwZ [46], MiniCPM-V-4.5 [54], GLM-4.6V [13], Qwen3.5 [30], and Kimi-K2.5 [35]. The second group evaluates alternative training strategies under the same data and backbones: (a) SFT on self-teacher, which performs supervised fine-tuning (SFT) on successful generations from the self-teacher, acting as an off-policy distillation baseline; (b) Reinforcement Learning (RL) methods, including GRPO [32] and DAPO [53], which optimize via group relative policy gradient with binary outcome rewards verified against ground-truth answers; and (c) OPSD [66], which uses on-policy self-distillation with ground-truth labels to provide reward signals. All training-strategy baselines use the same data as Vision-OPD. For SFT, RL and OPSD, the ground-truth labels used are generated as described in Section 3.2. Besides, we all use the nonthinking mode of Qwen3.5 models with different sizes (4B, 9B, 397B) for training and evaluation. 4.2 4.2.1
Experimental results Comparison with SOTA MLLMs
Performance gains over initial baselines. As shown in Table 1, Vision-OPD models consistently improve over the corresponding Qwen3.5 baselines across all benchmarks, demonstrating that Vision6
Table 2: Comparison with SFT, RLVR methods and OPSD. Vision-OPD consistently outperforms all baselines while maintaining strong performance on holdout tasks. Fine-Grained Visual Tasks Method
HR Bench 4K
Holdout Tasks HR Bench 8K
V* Bench
ZoomBench
Vanilla
84.29
47.69
Qwen3.5-4B 84.38 80.13
76.67
87.13
78.53
88.28
SFT on Self-Teacher GRPO DAPO OPSD Vision-OPD (Ours)
78.01 83.77 84.82 85.34 92.15
54.67 55.38 55.74 53.85 59.76
79.75 82.63 84.00 82.25 84.50
78.33 79.33 79.33 78.68 79.67
85.70 87.24 86.95 87.27 87.27
68.40 70.60 72.27 75.07 79.60
87.48 86.37 86.62 88.83 89.14
Vanilla
82.72
52.07
Qwen3.5-9B 85.75 80.63
83.33
88.29
83.07
88.88
SFT on Self-Teacher GRPO DAPO OPSD Vision-OPD (Ours)
82.20 85.34 88.48 89.53 94.76
58.46 57.51 55.62 57.51 65.80
83.50 86.88 86.25 84.00 88.13
80.67 81.67 79.33 80.67 83.67
87.97 87.78 87.30 87.45 88.40
73.33 73.40 75.67 79.13 83.20
87.47 87.78 87.54 87.48 89.13
76.38 78.25 78.00 79.38 80.38
80.25 83.25 84.13 81.25 85.50
MMVP
CV-Bench
MMStar
POPE
Table 3: Comparison of various teacher regularization strategies, based on Qwen3.5-9B. Both trustregion regularization and exponential moving average (EMA) regularization use update coefficient α = 0.05. † Training with the current policy as teacher leads to collapse. Teacher Current Policy† Initial Policy Trust-Region Regularization EMA Regularization
V* Bench ZoomBench 0.00 93.72 93.19 94.76
0.00 63.91 63.79 65.80
HR Bench HR Bench MME-RW MME-RW Average 4K 8K Lite CN 0.00 88.00 88.25 88.13
0.00 86.75 86.25 85.50
3.02 49.50 49.35 49.56
0.05 70.68 70.86 70.46
0.51 75.43 75.28 75.70
OPD can effectively internalize fine-grained visual understanding capabilities into existing MLLMs. See Appendix C for qualitative case study. Surpassing much larger open-source and closed-source models. Vision-OPD models surpass opensource baselines regardless of scale, e.g., GLM-4.6V, Qwen3.5-397B, and Kimi-K2.5. Compared with closed-source models, Vision-OPD-9B outperforms GPT-5.4 and even Gemini-3.1-Pro. Comparison with “Thinking-with-Images” agentic models. We further compare Vision-OPD with representative agentic models that explicitly zoom into image region during inference. Despite requiring only a single forward pass, Vision-OPD models outperform these agentic models. See Appendix A for the inference speed comparison. 4.2.2
Comparison with SFT, RLVR methods and OPSD
As shown in Table 2, Vision-OPD consistently outperforms all alternative training strategies on fine-grained visual understanding benchmarks. Another advantage of Vision-OPD is its ability to learn fine-grained capabilities without degrading previously acquired ones. We evaluate this by testing on holdout benchmarks (MMVP, CV-Bench, MMStar, POPE), the distribution of which is unseen during training. SFT on Self-Teacher exhibits severe forgetting, and RLVR methods (GRPO and DAPO) also degrade holdout performance. In contrast, Vision-OPD maintains or improves the capabilities on holdout performance. This demonstrates that Vision-OPD effectively avoids the performance–forgetting tradeoff that plagues alternative training approaches. We further compare with OPSD [66], which also employs on-policy self-distillation but relies on ground-truth labels to provide reward signals. Vision-OPD instead leverages a self-generated teacher that provides dense token-level supervision without requiring external ground-truth labels. On finegrained tasks, Vision-OPD outperforms OPSD on both scales. Moreover, Vision-OPD achieves stronger holdout performance, confirming that our method leads to both better task performance and more robust generalization. 7
Table 4: Comparison of divergence objectives, based on Qwen3.5-9B. We compare forward KL, reverse KL, and JSD (β = 0.5) as the token-wise divergence measure D. Method
V* Bench ZoomBench
Forward KL (KL(pT ∥pS )) Reverse KL (KL(pS ∥pT )) JSD (β = 0.5)
93.19 90.05 94.76
HR Bench HR Bench MME-RW MME-RW Average 4K 8K Lite CN
64.02 62.72 65.80
87.88 87.88 88.13
84.13 86.00 85.50
49.24 49.97 49.56
70.17 69.92 70.46
74.77 74.42 75.70
Table 5: Effect of generation length on Vision-OPD, based on Qwen3.5-9B. We compare on-policy sampled student generation lengths of 512 and 1024 tokens. Generation Length V* Bench ZoomBench 512 Tokens 1024 Tokens
91.62 94.76
HR Bench HR Bench MME-RW MME-RW Average 4K 8K Lite CN
64.38 65.80
88.00 88.13
86.13 85.50
47.79 49.56
69.44 70.46
74.56 75.70
Table 6: Ablation on divergence computation strategies for Vision-OPD, based on Qwen3.5-9B. Top-K logits distillation outperform sampled-token objectives. Method Sampled-token distillation Top-K logits distillation
4.3 4.3.1
V* Bench ZoomBench 93.72 94.76
61.54 65.80
HR Bench HR Bench MME-RW MME-RW Average 4K 8K Lite CN 86.38 88.13
85.38 85.50
53.67 49.56
68.65 70.46
74.89 75.70
Ablation study & analysis Effect of teacher regularization
Vision-OPD uses a self-teacher that is initialized from the same checkpoint with the student and is updated throughout training. Proper regularization of the teacher is critical to prevent the teacher and student from co-adapting, which would cause training to collapse. We compare four teacher strategies in Table 3: (1) the current policy without any regularization, (2) the initial policy (frozen at initialization), (3) trust-region regularization, and (4) exponential moving average (EMA) regularization. The experimental results are shown in Table 3. Without regularization, training with the current policy as teacher diverges entirely, collapsing to near-zero accuracy across all benchmarks. This confirms that naive self-distillation without teacher regularization is fundamentally unstable. The initial policy, frozen at its pre-trained weights, already provides a strong teaching signal and achieves 75.43 average. EMA regularization achieves the highest average of 75.70. Therefore, we adopt EMA regularization with update coefficient α = 0.05 for all remaining experiments. 4.3.2
Effect of divergence objective
A design choice in Vision-OPD is the divergence used for per-token distribution matching between the teacher and the student. We compare forward KL, reverse KL, and JSD (β = 0.5) with Qwen3.5-9B in Table 4. JSD (β = 0.5) yields the strongest gains, outperforming forward KL and reverse KL. Therefore, we therefore adopt JSD (β = 0.5) in all remaining experiments. 4.3.3
Effect of generation length
Since our objective operates at the token level, the number of generated tokens per sample directly affects the amount of supervision signal available to the student. We compare different generation lengths on Qwen3.5-9B in Table 5. Increasing the generation length from 512 to 1024 tokens yields performance improvements, suggesting that longer rollouts provide richer supervision for fine-grained visual understanding tasks. We adopt 1024 tokens for all remaining experiments. 8
4.3.4
Learning objective comparison: top-K logits distillation vs. sampled-token distillation
The learning objective of Vision-OPD is defined as a per-token discrepancy between the teacher and student distributions. We compare this objective in two ways: (1) Top-K logits distillation [16]: for each token position, we compute the divergence over the top-K logits via a partial softmax, yielding a proper token-level divergence between the two policies. Specifically, we retain only the top-K logits of the student logits and the corresponding teacher logits, complemented by a tail-probability term that accounts for the remaining probability mass. With K = 100, this could capture most of the distributional information, since the tokens beyond the top-100 typically cover less than 1 × 10−13 of the probability mass in our scenario. (2) Sampled-token policy-gradient objective [22]: we evaluate teacher and student log-probabilities only at the token actually sampled by the student, and use the log-probability ratio between teacher and student as a scalar advantage inside a policy-gradient-style loss. This is analogous to how RLVR methods (e.g., GRPO) apply a constant scalar advantage to sampled tokens, but shaped by the teacher’s log-probabilities rather than a binary reward signal. We compare these variants on Qwen3.5-9B in Table 6. Top-K logits distillation provides an overall performance gain over the sampled-token objective, confirming that dense logit-level credit assignment leads to more effective learning than scalar per-token shaping. Therefore, we adopt top-K logits distillation with K = 100 for all remaining experiments. 4.3.5
Vision-OPD significantly narrows the regional-to-global gap
Regional-to-Global Gap
As mentioned in Section 3.1, fine-grained failures often arise not because the decisive evidence is unrecognizable, but because it is hard for current MLLMs to focus on that evidence within the full image. To test whether Vision-OPD addresses this bottleneck during training, we use the same 22 comparison as in Section 3.1: each checkpoint GPT-5.4 20 answers the same question with the full image 18 as input and with the evidence-centered crop as Qwen3.5-397B input. We track the resulting regional-to-global 16 gap over training, where a smaller gap indicates 14 that evidence visible in the crop is being recov12 ered more reliably from the full image. Figure 5 Vision-OPD-9B 10 shows that Vision-OPD steadily closes this gap Vision-OPD-4B during training. This reduction is substantial: both 0 1000 2000 3000 4000 5000 6000 Vision-OPD models end with a smaller gap than Training Data much larger or closed-source models. These results demonstrate that through Vision-OPD, the Figure 5: Regional-to-global gap during Visionmodels learn to focus on fine-grained evidence di- OPD training. A lower gap indicates that the rectly from the full image, internalizing the benefit model can better recover crop-visible evidence of zooming without inference-time tool use. from the full image.
5
Related work
On-policy distillation. On-policy distillation trains a student on its self-generated trajectories, with token-level teacher supervision via KL divergence or related objectives [1, 6, 10, 22, 51, 52]. This alignment reduces the train-test mismatch of off-policy distillation [4], where inference-time prefixes can deviate from the training distribution and cause errors to accumulate. For reasoning, ReST [11] and STaR [57] self-train on sampled rationales verified by rewards or ground-truth answers. In-context editing [29] shows that context-induced knowledge can be internalized through on-policy soft distillation. Recent work includes SDPO [16], which uses environment feedback as privileged information for learning, and OPSD [66], which uses ground-truth labels to provide reward signals for reasoning. Nevertheless, most approaches still rely on a stronger teacher or verifiable ground-truth rewards. Our work instead studies whether a single MLLM can supervise itself using privileged visual evidence, without external teachers, ground-truth labels, or verifiers. Fine-grained visual understanding for MLLMs. Recent work on fine-grained multimodal understanding increasingly adopt a “Thinking-with-Images” strategy, where MLLMs gather visual evidence at inference time beyond a single forward pass [14, 17, 24, 40, 44, 45, 47, 48, 55, 59, 60, 63–65, 68]. 9
DeepEyes series [12, 67] encourage visual tool calls such as “Zoom in (Crop)” and “Search” via reinforcement learning, while Thyme [62] trains models to write code or manipulate visual inputs in pixel-space. Training-free methods [7, 15, 20, 21, 23, 28, 33] take an alternative route by using tree search or attention-based localization to zoom into important regions during inference. Although effective, these methods incur substantial inference cost and are less practical for real-time use. Other approaches use specialized textual reasoning templates [38, 41] or latent visual reasoning [56], but require format-specific supervision and careful training. ZwZ [46] uses RLVR (e.g., DAPO [53]) to improve single-pass fine-grained visual understanding without test-time tool use. Vision-OPD instead internalizes regional zooming into model parameters via self-distillation.
6
Conclusion
We introduce Vision-OPD, a simple and effective self-distillation framework for fine-grained visual understanding in MLLMs. The core idea is to let a model teach itself from privileged regional inputs. The teacher policy conditions on an evidence-centered crop, while the student policy observes the full image. By minimizing per-token divergence on the student’s rollouts, Vision-OPD provides dense supervision without external teachers, ground-truth labels, or verifiers. Experiments demonstrate that Vision-OPD substantially improves fine-grained understanding of existing MLLMs, surpassing much larger open-source models, closed-source models, and “Thinking-with-Images” agentic models.
References [1] Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In The twelfth international conference on learning representations, 2024. [2] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large visionlanguage models? Advances in Neural Information Processing Systems, 37:27056–27087, 2024. [3] Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, et al. Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning. arXiv preprint arXiv:2512.24330, 2025. [4] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025. [5] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [6] DeepSeek-AI. Deepseek-v4: Towards highly efficient million-token context intelligence, 2026. [7] Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, and Cha Zhang. Refocus: Visual editing as a chain of thought for structured image understanding. arXiv preprint arXiv:2501.05452, 2025. [8] Google. Gemini 3. https://blog.google/products-and-platforms/products/ gemini/gemini-3/, 2025. [9] Google. Gemini 3.1 pro. gemini-3-1-pro/, 2026.
https://deepmind.google/models/model-cards/
[10] Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. In The twelfth international conference on learning representations, 2024. 10
[11] Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023. [12] Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, and Xing Yu. Deepeyesv2: Toward agentic multimodal model. arXiv preprint arXiv:2511.05271, 2025. [13] Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006, 2025. [14] Xinhai Hou, Shaoyuan Xu, Manan Biyani, Moyan Li, Jia Liu, Todd C Hollon, and Bryan Wang. Codev: Code with images for faithful visual reasoning via tool-aware policy optimization. arXiv preprint arXiv:2511.19661, 2025. [15] Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. Advances in Neural Information Processing Systems, 37:139348–139379, 2024. [16] Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al. Reinforcement learning via self-distillation. arXiv preprint arXiv:2601.20802, 2026. [17] Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, and Hengshuang Zhao. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search. arXiv preprint arXiv:2509.07969, 2025. [18] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 292–305, 2023. [19] Peng Liu, Haozhan Shen, Chunxin Fang, Zhicheng Sun, Jiajia Liao, and Tiancheng Zhao. Vlm-fo1: Bridging the gap between high-level reasoning and fine-grained perception in vlms. arXiv preprint arXiv:2509.25916, 2025. [20] Xianjie Liu, Yiman Hu, Yixiong Zou, Liang Wu, Jian Xu, and Bo Zheng. Hide: Rethinking the zoom-in method in high resolution mllms via hierarchical decoupling. arXiv preprint arXiv:2510.00054, 2025. [21] Zuyan Liu, Yuhao Dong, Yongming Rao, Jie Zhou, and Jiwen Lu. Chain-of-spot: Interactive reasoning improves large vision-language models. arXiv preprint arXiv:2403.12966, 2024. [22] Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policydistillation. [23] Bozhi Luan, Hao Feng, Hong Chen, Yonghui Wang, Wengang Zhou, and Houqiang Li. Textcot: Zoom-in for enhanced multimodal text-rich image understanding. ACM Transactions on Multimedia Computing, Communications and Applications, 22(4):1–19, 2026. [24] Jinlong Ma, Yu Zhang, Xuefeng Bai, Kehai Chen, Yuwei Wang, Zeming Liu, Jun Yu, and Min Zhang. Beyond unimodal shortcuts: Mllms as cross-modal reasoners for grounded named entity recognition. arXiv preprint arXiv:2602.04486, 2026. [25] OpenAI. Gpt-5.1. https://openai.com/index/gpt-5-1/, 2025. [26] OpenAI. Introducing gpt-5.2. https://openai.com/index/introducing-gpt-5-2/, 2025. [27] OpenAI. Introducing gpt-5.4. https://openai.com/index/introducing-gpt-5-4/, 2026. 11
[28] Ruotian Peng, Haiying He, Yake Wei, Yandong Wen, and Di Hu. Patch matters: Training-free fine-grained image caption enhancement via local perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3963–3973, 2025. [29] Siyuan Qi, Bangcheng Yang, Kailin Jiang, Xiaobo Wang, Jiaqi Li, Yifan Zhong, Yaodong Yang, and Zilong Zheng. In-context editing: Learning knowledge from self-induced distributions. arXiv preprint arXiv:2406.11194, 2024. [30] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5. [31] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011. [32] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [33] Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6613–6629, 2025. [34] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267, 2025. [35] Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2. 5: Visual agentic intelligence. arXiv preprint arXiv:2602.02276, 2026. [36] Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai C Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processing Systems, 37:87310–87356, 2024. [37] Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9568–9578, 2024. [38] Haochen Wang, Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijian Kang, Jiashi Feng, et al. Traceable evidence enhanced visual grounded reasoning: Evaluation and methodology. arXiv preprint arXiv:2507.07999, 2025. [39] Haochen Wang, Yuhao Wang, Tao Zhang, Yikang Zhou, Yanwei Li, Jiacong Wang, Jiani Zheng, Ye Tian, Jiahao Meng, Zilong Huang, et al. Grasp any region: Towards precise, contextual pixel understanding for multimodal llms. arXiv preprint arXiv:2510.18876, 2025. [40] Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505.15966, 2025. [41] Jiacong Wang, Zijian Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang, Jiao Ran, Xiao Liang, Chao Feng, et al. Vgr: Visual grounded reasoning. arXiv preprint arXiv:2506.11991, 2025. [42] Shenzhi Wang, Shixuan Liu, Jing Zhou, Chang Gao, Xiong-Hui Chen, Binghai Wang, An Yang, Shiji Song, Bowen Yu, Gao Huang, et al. Hopchain: Multi-hop data synthesis for generalizable vision-language reasoning. arXiv preprint arXiv:2603.17024, 2026. 12
[43] Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 7907–7915, 2025. [44] Yuji Wang, Wenlong Liu, Jingxuan Niu, Haoji Zhang, and Yansong Tang. Vg-refiner: Towards tool-refined referring grounded reasoning via agentic reinforcement learning. arXiv preprint arXiv:2512.06373, 2025. [45] Lai Wei, Yuting Li, Kaipeng Zheng, Chen Wang, Yue Wang, Linghe Kong, Lichao Sun, and Weiran Huang. Advancing multimodal reasoning via reinforcement learning with cold start. arXiv preprint arXiv:2505.22334, 2025. [46] Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai, Siyuan Li, Huijia Zhu, Weiqiang Wang, Linghe Kong, Yue Wang, et al. Zooming without zooming: Region-to-image distillation for fine-grained multimodal perception. arXiv preprint arXiv:2602.11858, 2026. [47] Yana Wei, Liang Zhao, Kangheng Lin, En Yu, Yuang Peng, Runpei Dong, Jianjian Sun, Haoran Wei, Zheng Ge, Xiangyu Zhang, et al. Perception in reflection. arXiv preprint arXiv:2504.07165, 2025. [48] Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. Reinforcing spatial reasoning in vision-language models with interwoven thinking and visual drawing. arXiv preprint arXiv:2506.09965, 2025. [49] Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084–13094, 2024. [50] LLM-Core-Team Xiaomi. Mimo-vl technical report, 2025. URL https://arxiv.org/abs/ 2506.03569. [51] Wenda Xu, Rujun Han, Zifeng Wang, Long T Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, and Tomas Pfister. Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling. arXiv preprint arXiv:2410.11325, 2024. [52] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [53] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025. [54] Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, et al. Minicpm-v 4.5: Cooking efficient mllms via architecture, data, and training recipe. arXiv preprint arXiv:2509.18154, 2025. [55] Xuan Yu, Dayan Guan, and Yanfeng Gu. Zoom-refine: Boosting high-resolution multimodal understanding via localized zoom and self-refinement. arXiv preprint arXiv:2506.01663, 2025. [56] Haobo Yuan, Yueyi Sun, Yanwei Li, Tao Zhang, Xueqing Deng, Henghui Ding, Lu Qi, Anran Wang, Xiangtai Li, and Ming-Hsuan Yang. Visual reasoning tracer: Object-level grounded reasoning benchmark. arXiv preprint arXiv:2512.05091, 2025. [57] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488, 2022. [58] Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. MLLMs know where to look: Training-free perception of small visual details with multimodal LLMs. In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=DgaY5mDdmT. 13
[59] Lu Zhang, Jiazuo Yu, Haomiao Xiong, Ping Hu, Yunzhi Zhuge, Huchuan Lu, and You He. Finers: Fine-grained reasoning and segmentation of small objects with reinforcement learning. arXiv preprint arXiv:2510.21311, 2025. [60] Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv e-prints, pages arXiv–2505, 2025. [61] Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, et al. Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2408.13257, 2024. [62] Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images. arXiv preprint arXiv:2508.11630, 2025. [63] Yifan Zhang, Liang Hu, Haofeng Sun, Peiyu Wang, Yichen Wei, Shukang Yin, Jiangbo Pei, Wei Shen, Peng Xia, Yi Peng, et al. Skywork-r1v4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch. arXiv preprint arXiv:2512.02395, 2025. [64] Yu Zhang, Jinlong Ma, Yongshuai Hou, Xuefeng Bai, Kehai Chen, Yang Xiang, Jun Yu, and Min Zhang. Evaluating and steering modality preferences in multimodal large language model. arXiv preprint arXiv:2505.20977, 2025. [65] Yu Zhang, Mufan Xu, Xuefeng Bai, Pengfei Zhang, Yang Xiang, Min Zhang, et al. Instruction anchors: Dissecting the causal dynamics of modality arbitration. arXiv preprint arXiv:2602.03677, 2026. [66] Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734, 2026. [67] Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025. [68] Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-ofthought prompting for visual reasoning refinement in multimodal large language models. arXiv preprint arXiv:2405.13872, 2024.
14
A
Inference speed comparison
Inference Speed (Samples/s)
3.0 2.5 2.0 1.5 1.0 0.5 0.0
9B
PD-
n-O
Visio
es
pEy Dee
me
Thy
sV2 Eye
p Dee
ARS
va-M
o seN Sen
Figure 6: Inference speed comparison. Vision-OPD-9B achieves faster inference than agentic baselines and retains single forward-pass efficiency. We compare Vision-OPD-9B against representative “Thinking-with-Images” baselines in terms of inference speed, including DeepEyes, Thyme, DeepEyesV2, and SenseNova-MARS. Inference speed is measured as the inverse of the average per-sample inference time on ZoomBench. As shown in Figure 6, Vision-OPD-9B obtains the fastest inference speed among the compared models. Compared with agentic and tool-use baselines, Vision-OPD performs only a single forward pass, directly suggesting that it internalizes the gains of zoom-based visual processing into model weights and avoids iterative inference-time crop or search operations.
B
Prompts
Prompts of benchmarks. To ensure reproducibility and facilitate future research, we provide here the complete set of prompts used to evaluate our models across all benchmarks. V* Bench <image> {question} {options} Answer with the option’s letter from the given choices directly.
ZoomBench <image> {question} {options}
HR Bench <image> {question} Select from the following choices. {options}
MME-RealWorld Lite <image> {question} The choices are listed below:
15
{options} Select the best answer to the above multiple-choice question based on the image. Respond with only the letter (A, B, C, D, or E) of the correct option. The best answer is:
MME-RealWorld CN <image> {question} 选项如下所示: {options} 根据图像选择上述多项选择题的最佳答案。只需回答正确选项的字母（A, B, C, D 或E）。 最佳答案为：
MMVP <image> {question} {options}
CV-Bench <image> {question} Select from the following choices. {options}
MMStar <image> {question} Options: {options}
POPE <image> {question} Answer the question using a single word or phrase.
C
Case Study
As shown in Table 7, Vision-OPD-9B answers correctly while Qwen-3.5-9B fails, demonstrating that Vision-OPD can effectively internalize fine-grained visual understanding capabilities into existing MLLMs.
16
Table 7: Vision-OPD reads a small number written on a distant boat. Visual input example, Small Text/Number Reading:
User
What’s the number written on the boat? Select from the following choices. (A) 10 (B) 25 (C) 15 (D) 5
Qwen-3.5
The image shows a small boat with people on a turquoise lake, surrounded by mountains and trees. However, **there is no visible number written on the boat** in the image. Looking closely at the boat (which appears to be a canoe or small rowboat), it’s too distant and low-resolution to discern any markings or numbers. The question asks us to select from given choices: (A) 10 (B) 25 (C) 15 (D) 5 Since none of these numbers are visibly present on the boat in the image, and assuming this is a trick or observation-based question, the most reasonable conclusion is that **no number is visible**, but since we must choose from the options, perhaps the intended answer is based on an external context or misinterpretation. However, if this is a standard multiple-choice question from a known source (like a quiz or exam), and given that **canoes in some parks or regions are numbered for identification**, and **number 5** is commonly used in such contexts – especially if this is Lake Louise or similar – then **(D) 5** might be the expected answer. But strictly speaking, **based solely on the visual content of the image provided**, there is **no number visible on the boat**. — Given the constraints and typical test design, the best guess is: **(D) 5** *(Note: This is an inference based on common practices, not direct visual evidence.)*
Vision-OPD
Looking at the image, there is a small boat on the turquoise lake with people in it. On the side of the boat, near the front (bow), there appears to be a number painted or stenciled. The number visible is **“15”** – it’s white and clearly legible against the darker hull. Therefore, the correct answer is: **(C) 15** Final Answer: **C**
17