arXiv CSopen access

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models Yanting Miao*1,3,4 , Yutao Sun*2,3 , Dexin Wang3 , Mengyu Zhou†3 , Pascal Poupart1,4 , Lei Lv3 , Qi Zhao3 , Li Wang3 , Hao Li3 , Xiaoxi Jiang3 and Guanjun Jiang3 1 University of Waterloo, 2 Zhejiang University, 3 Qwen Large Model Application Team, Alibaba, 4 Vector Institute

arXiv:2605.12374v1 [cs.CV] 12 May 2026

* Work done during an internship at Alibaba. † Corresponding author.

Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume (Xie et al., 2025; Li et al., 2026; Team et al., 2026). This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose GAP, a Granular Alignment Paradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacityguided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.

1. Introduction Multimodal large language models (MLLMs) increasingly solve visual tasks by interleaving perception and language reasoning, yet many failures on charts, high-resolution perception, and visual math stem from missing or poorly localized visual evidence Bai et al. (2025a); Singh et al. (2025); Team et al. (2025). A natural way to obtain such evidence is to call external visual tools—cropping, search, or image generation—to create auxiliary views. Tool-augmented pipelines, however, introduce extra API calls, wall-clock latency, and system complexity, and building a unified generation-and-understanding model is a substantially heavier direction that typically requires many additional visual tokens and training resources. Tool-based approaches also face cross-task generalization difficulties, since a single visual tool is rarely adequate across diverse visual reasoning tasks Su et al. (2025); Zhao et al. (2025). We instead ask whether an image-understanding MLLM can be minimally adapted to produce intermediate visual evidence inside its own autoregressive computation. One appealing solution is visual latent reasoning: the model emits continuous latent tokens intended to represent auxiliary visual information (Wang et al., 2025a; Li et al., 2025a). This design is lightweight and keeps inference inside a single model, with no external image-generation call. However, prior latent-token methods deliver unstable empirical gains, and in some settings the latent channel contributes little after post-training Li et al. (2025b); Wu et al. (2026); Liu et al. (2025). A feature-level failure mode in pre-norm MLLMs. Modern MLLMs such as Qwen3-VL Bai et al. (2025a), Gemma 4 Google DeepMind (2026), and InternVL3.5 Wang et al. (2025b) use pre-norm transformers, in which final-layer hidden states accumulate much larger norms than the input text or vision embeddings and are not guaranteed to lie in the same empirical subspace as input vision embeddings. Most recent visual-latent models build on pre-norm MLLM backbones and reuse output hidden states as next-step latent inputs (Wang et al.,

Corresponding author(s): Mengyu Zhou ([email protected])

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

2025a; Li et al., 2025a). We examine this assumption on Monet-7B Wang et al. (2025a), a state-of-the-art latent model based on Qwen2.5-VL-7B. Despite the backbone’s native final RMSNorm, its final-layer states remain far from the input-embedding distribution: text hidden states are roughly 546× larger than text input embeddings, and vision hidden states are roughly 8.7× larger than vision input embeddings. Re-injecting such states as latent inputs can therefore create a feature-space mismatched feedback loop that repeatedly drives the model with hidden states from a distribution it was not trained to consume as input. Interventional evidence. The norm gap is not only descriptive. Taking Monet-7B as an output-as-input latent baseline and applying a training-free Exponential Moving Average (EMA) rescaling that matches predicted latent norms to vision input-embedding norms improves performance by +2.00 on MathVista. The intervention changes neither the training data nor the backbone; it only corrects the norm of the re-injected latent vector. This supports feature-level compatibility as a practical factor for stable latent reuse: even without changing data or backbone, correcting the re-injected latent’s magnitude already recovers measurable accuracy, and the residual gap motivates the learned alignment we introduce next. Our approach: GAP. The feature-level mismatch above is the main failure mode targeted in this work, and norm rescaling alone treats only one symptom of it. Training a useful latent channel also benefits from two practical supports: the latent targets should be interpretable enough to inspect during training and analysis, and latent supervision should not be applied indiscriminately to examples the base model already solves. We therefore introduce GAP, a Granular Alignment Paradigm for visual latent modeling across data, feature, and model levels. At the feature level, a PCA-aligned latent head maps the backbone’s native-RMS-normalized decoder states into the principal subspace of auxiliary-image vision embeddings before re-injection; the PCA basis acts both as a subspace projector that returns generated latents to the empirical vision-embedding distribution and as a low-rank parameterization of the latent head. For inspectability, context-grounded latent supervision annotates each latent example with chain-of-thought intent, a <latent> span supervised by auxiliary-image vision embeddings, and a <parser> description that records the intended auxiliary visual signal during training and analysis. For training stability, difficulty-aware latent assignment estimates base-model accuracy by repeated sampling and assigns latent targets only when the base model fails, reducing noise from unnecessary latent targets on easy examples. Figure 1 summarizes GAP as a three-level alignment recipe spanning data-, feature-, and model-level alignment. The resulting training set contains 49K curated multimodal examples spanning visual CoT, charts, geometry, multimodal math, counting, and visual search. We make the following contributions: 1. A granular alignment paradigm (GAP). We introduce a lightweight latent feedback recipe centered on a PCA-aligned projection that returns generated latents to the vision-embedding subspace, supported by context-grounded latent supervision for inspectability and difficulty-aware latent assignment for training stability. 2. Empirical validation of aligned visual latents. Across five perception and reasoning benchmarks, GAP improves the aggregate metrics over prior visual-latent baselines and achieves the best mean aggregate supervised variant in our experiments. Controlled ablations show that PCA alignment, curated latent supervision, and difficulty-aware assignment each contribute to the gains, while inference-time interventions suggest that generated latents carry task-relevant visual signal beyond merely adding token slots. 3. Open latent-supervision data. We construct a 49K high-quality multimodal latent-supervision dataset spanning visual CoT, charts, geometry, multimodal math, counting, and visual search. We will release it to support future work on visual latent reasoning. Beyond the high-level overview in Figure 1, Appendix Figure 3 traces the token-level execution path, showing how ordinary text decoding and GAP latent reconstruction interleave during generation.

2. Related Work Explicit visual tools and generated visual thoughts. One way to supply missing visual evidence is to call external visual operations such as zooming, cropping, or region search. ZoomEye (Shen et al., 2025) builds a tree-based exploration strategy that enables MLLMs to inspect relevant image regions. Another line generates explicit visual thoughts or auxiliary images: Visualization-of-Thought (Li et al., 2025b), Latent Sketchpad (Zhang et al., 2025), Render-of-Thought (Wang et al., 2026), DeepEyes (Zheng et al., 2025), and GoT-R1 (Duan et al.,

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Visual Latent Alignment Gaps

GAP Framework

Data Gap

Data-Level Alignment

latent targets without context grounding

output

context-grounded latent alignment

...

input

Feature Gap subspace and geometry mismatch

...

↓ ↓ ↓ ↓ ... ↓

✓ ✕ ✓ ✕ ... ✕

→ →

PCA-aligned latent head PCA projection -> vision-space reconstruction

Model Gap indiscriminate latent use

Feature-Level Alignment

Model-Level Alignment easy

hard

difficulty-aware latent supervision

GAP aligns latent supervision, feature geometry, and adaptive latent usage.

Figure 1 | Conceptual overview of GAP. GAP addresses visual latent learning through three alignment levels: context-grounded latent supervision at the data level, a PCA-aligned latent head at the feature level, and difficulty-aware latent supervision at the model level. 2025) use sketching, rendered reasoning traces, visual tools, or generation-oriented reinforcement learning. These methods provide interpretable intermediate artifacts, but they introduce additional tool or generation dependencies at inference. Our goal is narrower: keep inference inside the MLLM and study how continuous visual feedback can be made compatible with the model’s input-embedding distribution. Continuous and visual latent reasoning. Latent reasoning represents intermediate computation as continuous embeddings rather than explicit text or images. Deng et al. (2024) study the transition from explicit to implicit chain-of-thought, and COCONUT (Hao et al., 2024) trains language models to reason in a continuous latent space. In multimodal reasoning, Monet (Wang et al., 2025a), LVR (Li et al., 2025a), Machine Mental Imagery (Yang et al., 2025), Chain-of-Visual-Thought (Qin et al., 2025), LIVR (Li et al., 2025c), VaLR (Jeon et al., 2026), LaViT (Wu et al., 2026), and CrystaL (Zhang et al., 2026) explore latent visual tokens, vision-aligned latent thoughts, or emergent visual latents. These works make visual latent reasoning an active and competitive direction; our contribution is therefore not the introduction of visual latent tokens themselves. Instead, we isolate a concrete feature-space failure mode of output-as-input latent feedback in pre-norm MLLMs and test a lightweight PCA-based reconstruction path that maps generated latents back toward the empirical vision-embedding subspace before autoregressive re-injection. Residual norms and input compatibility in pre-norm transformers. Pre-norm transformers stabilize deep training by normalizing each sublayer input, but their residual streams can accumulate large norms across depth. Recent analyses and architectures such as SiameseNorm (Li et al., 2026), Attention Residuals (Team et al., 2026), and manifold-constrained hyper-connections (Xie et al., 2025) study the behavior of residual streams and normalization. Our work connects this architectural issue to visual latent feedback: a vector that is suitable for output readout need not be distributed like an input vision embedding, even after the backbone’s native final RMSNorm. Selective latent computation. Latent and visual computation need not be applied uniformly to every example. Visual Enhanced Depth Scaling (Han et al., 2026) studies when additional visual computation

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

HRBench4K Latent Norm Alignment (mean log norm; error bars show ±1 std)

log(L2 norm)

6.09±0.04

≈8.6× query vision norm

3.94±0.06

same norm regime 3.53±0.12

Final Hidden State

Query Vision Embedding

GAP reconstruct

Representation

(a) Layer-wise hidden-state norm growth.

(b) Latent norm alignment after GAP reconstruction.

Figure 2 | Feature-space mismatch and GAP reconstruction on HRBench4K samples. (a) In Monet-7B, text and vision hidden-state log norms grow sharply through the decoder stack and remain far above the input embedding regime; shaded bands show one standard deviation of log(L2 norm) for hidden-state norms only. (b) The final decoder hidden state used by output-as-input latent feedback is about 8.6× larger than the query vision-embedding norm, whereas GAP reconstruction maps generated latents back to the same norm regime before autoregressive re-injection. should be allocated during multimodal latent reasoning. Our difficulty-aware supervision is a training-time counterpart: it decides which examples should include latent targets, without adding an inference-time routing or depth-scaling module.

3. Preliminary Analysis: Why Output-as-Input Latent Feedback Fails Before presenting our method, we first establish the feature-space bottleneck behind output-as-input latent feedback in our Qwen2.5-VL-7B setting. The remaining design questions—whether a latent head, curated data, difficulty-aware supervision, and PCA compression matter under controlled variants—are evaluated as component ablations in Section 5.3. 3.1. Output-Embedding Norm Mismatch Many current MLLM decoders, including Qwen2.5-VL, adopt a pre-norm architecture (Xiong et al., 2020), where layer normalization is applied before each sub-layer (attention and FFN) rather than after. A well-known consequence is that hidden state norms tend to accumulate with layer depth: each residual addition increases the norm before the model’s final normalization. We profile the log-transformed L2 norm of hidden states across Monet-7B, on HRBench4K samples (Figure 2(a)). Figure 2(a) plots log(∥ · ∥ 2 ) and shows standard deviations computed in this log-norm space; we also report the corresponding raw L2 norms below for interpretability. Let h ( 𝑙 ) ∈ ℝ𝑑 denote the hidden state at layer 𝑙 (𝑑 = 3584). Before the final normalization, the text hidden-state log norm peaks at 6.31, corresponding to a raw L2 norm of 552.3, which is 546.4× larger than the input text embedding norm (1.01; log norm 0.01). The vision hidden-state log norm peaks at 6.09, corresponding to a raw L2 norm of 441.4, approximately 8.7× larger than the input vision embedding norm (50.5; log norm 3.92). This norm growth persists in Monet-7B after latent fine-tuning, indicating that the mismatch is inherited from the underlying pre-norm Qwen2.5-VL-7B decoder rather than being an artifact of the raw base model alone. This pattern is consistent with residual accumulation in pre-norm architectures: each transformer layer computes h ( 𝑙+1) = h ( 𝑙 ) + SubLayer(LN(h ( 𝑙 ) )), where LN denotes layer normalization, SubLayer abstracts the layer’s attention or feed-forward transformation, and the additive residual can accumulate norm across layers (Li et al., 2026; Xie et al., 2025; Team et al., 2026). Existing latent reasoning methods (Hao et al., 2024; Wang et al., 2025a; Li et al., 2025a) directly reuse the output hidden state h ( 𝐿 ) as the input embedding for the next latent token. This feeds a vector from a high-norm output-side distribution into a processing pipeline calibrated for the input-embedding norm regime, creating a

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

distribution shift that destabilizes optimization. Thus, direct output-as-input reuse can create a feature-space mismatch in the latent feedback loop. For comparison, Figure 2(b) previews the effect of GAP’s reconstruction path: reconstructed latents return to the query vision-embedding norm regime, while the output-side final decoder hidden state remains about 8.6× larger. Empirical evidence: norm rescaling alone helps. To validate that the norm gap is not merely a theoretical concern but a practical bottleneck, we conduct a simple experiment. We use Monet-7B as an output-as-input latent model, and compare it with and without a training-free EMA norm calibration that rescales the predicted latent to match the input vision embedding norm: ṽ𝑡 = 𝑛¯EMA · v̂𝑡 /∥v̂𝑡 ∥ 2 , where 𝑛¯EMA is the running average of ∥v∥ 2 computed from the query image’s vision embedding. As shown in Table 1, simply rescaling the norm at inference time already improves performance, supporting norm mismatch as one practical bottleneck. Table 1 | Effect of norm calibration. “Output-as-Input” denotes Monet-7B, a latent-reasoning model trained from Qwen2.5-VL-7B that reuses output-side hidden states as latent inputs. EMA norm calibration rescales the predicted latent to match input vision embedding norms at inference time, requiring no additional training. The improvement row is highlighted in green. Configuration

HRBench4K↑

MathVista↑

Avg.↑

Output-as-Input (Monet-7B) Output-as-Input + EMA norm calibration

70.75 71.63

61.30 63.30

66.03 67.46

Improvement

+0.88

+2.00

+1.44

The norm intervention motivates GAP’s feature-level design: the model should not recycle raw decoder outputs as future visual inputs. We next describe the data-, feature-, and model-level alignment mechanisms, and then report controlled mechanism, data, PCA-rank, and token-budget ablations in Section 5.

4. Method Guided by the alignment requirements in Section 1, GAP builds latent reasoning around three levels of alignment. Data-level alignment makes each continuous target traceable to a controllable auxiliary visual signal (§4.1); feature-level alignment maps generated latents back into the empirical vision-input subspace before feedback (§4.2); and model-level alignment applies latent supervision mainly to queries where the base model struggles (§4.3). 4.1. Data-Level Alignment: Controllable Latent Data Construction Continuous latent supervision is otherwise hard to inspect: a latent vector can be supervised, but the example may not reveal what missing visual evidence that vector is meant to represent. We therefore construct 49,309 multimodal QA examples in which each query image and question is paired with an intermediate auxiliary image and a structured teacher response. During student training, the auxiliary image is never fed as input; its frozen-ViT embeddings serve only as targets for the latent head, while the surrounding <think> and <parser> text records the intended visual content. At inference time no auxiliary image is available, and the same latent positions are predicted autoregressively by GAP. The dataset is concentrated on visual CoT, chart understanding, GEOQA, multimodal math, counting, and visual search; full source composition and exact latent serialization are provided in Appendix I, with data-loader details in Appendix F. Latent Interleave response format. Text reasoning → <latent> visual latent tokens </latent> → <parser> textual description of the intended latent visual evidence </parser> → continued text reasoning and final answer.

4.2. Feature-Level Alignment: PCA-Aligned Latent Head At the feature level, the decoder state used to generate a latent token need not be distributed like the vision embeddings consumed as input. GAP avoids direct output-as-input reuse by predicting low-dimensional coordinates in an empirical auxiliary-image vision subspace and reconstructing each generated latent in the

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

input vision-embedding coordinate system before re-injection. Let v̂𝑡 −1 ∈ ℝ𝑑 be the previous latent embedding fed to the transformer, h𝑡( 𝐿 ) is the hidden state at the layer 𝐿, M𝜃 the backbone, and F𝜃 the latent head. Autoregressive latent generation is: h𝑡( 𝐿 ) = M𝜃 (v̂𝑡 −1 , KV <𝑡 ) ,

c𝑡 = F𝜃 ( h̄𝑡 ) ,

v̂𝑡 = P𝑘 c𝑡 + 𝝁.

(1)

where h̄𝑡 = RMSNorm(h𝑡( 𝐿 ) ), KV <𝑡 denotes cached keys and values, c𝑡 ∈ ℝ𝑘 are PCA coefficients, and P𝑘 , 𝝁 are estimated from real auxiliary-image vision embeddings. A full ℝ𝑑 → ℝ𝑑 latent MLP would require roughly 2𝑑 2 ≈ 25.6M parameters for 𝑑 = 3584; GAP instead uses 𝑘 = 629 components, retaining 95% variance, and supervises the decoded embedding: Llatent =

2 1 ∑︁ P𝑘 F𝜃 ( h̄𝑡 ) + 𝝁 − v𝑡 , 2 |T | 𝑡 ∈ T

(2)

where T is the set of latent token positions and v𝑡 is the target auxiliary-image embedding. This reconstruction path is the feature-level alignment mechanism: raw decoder states are never fed back directly, and Figure 2(b) shows that GAP reconstruction returns generated latents to the query vision-embedding norm regime. The full objective combines language modeling with latent alignment: L = LLM + 𝜆 latent Llatent ,

(3)

Appendix Figure 3 illustrates the text-token versus latent-token inference path, and Appendix F gives the deferred implementation details, including projector initialization and scheduled sampling ablations. 4.3. Model-Level Alignment: Difficulty-Aware Latent Supervision Not all queries benefit from latent reasoning, and imposing latent supervision on examples the base model can already solve can add noise. For each training query, we run 𝑁 = 8 inference passes with the base Qwen2.5-VL 7B model, compute empirical accuracy 𝑎ˆ, and assign supervision by: ( text-only if 𝑎ˆ > 𝜏, DA( 𝑞) = (4) latent if 𝑎ˆ ≤ 𝜏, In the current experiments 𝜏 = 0, so only examples the base model never solves across the eight samples receive latent supervision. For “text-only” queries, the model is trained with a standard text response; for “latent” queries, it uses the full latent format from Section 4.1. This model-level alignment teaches when latent-assisted reasoning is useful without adding an inference-time router, and the exact data-loader stripping rule is deferred to Appendix F.

5. Experiments 5.1. Experimental Setup Base model and baselines. We use Qwen2.5-VL 7B (Bai et al., 2025b) as the base MLLM. We compare against the zero-shot base model, a Dense Caption SFT control trained on the same 49K curated data without latent tokens or auxiliary-image alignment, Monet (Wang et al., 2025a), and LVR (Li et al., 2025a). Our variants are a basic latent-head SFT model (LH), a difficulty-aware full latent head without PCA (LH+DA), an all-latent PCA-aligned latent-head model (LH+PCA), and the full difficulty-aware PCA-aligned version, GAP (LH+PCA+DA). Monet and LVR are used as prior latent-system baselines rather than matched-data retrains; the component ablations below isolate matched-recipe controls on our curated data. Benchmarks and metrics. We evaluate perception on HRBench4K, MMStar, and MME-RealWorld-Lite, and reasoning on MathVista and WeMath. For HRBench4K, we report Overall, fine-grained spatial perception (FSP), and fine-grained counting perception (FCP). For MME-RealWorld-Lite, we report Overall, Reasoning, and Perception sub-scores. We define Avg-P as the average of HRBench4K Overall, MMStar, and MME-RealWorld-Lite Overall, and Avg-R as the average of MathVista, WeMath-S, and WeMath-L.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Implementation details. We fine-tune Qwen2.5-VL 7B using AdamW with a base learning rate of 1×10−5 , a latent-head learning rate of 1×10−5 , a global batch size of 128, and 2 epochs on the 49K curated dataset. The PCA basis is computed offline from training-set vision embeddings with a 95% explained-variance target, yielding 𝑘 = 629 principal components. The latent loss weight is 𝜆 latent = 1.0. The main difficulty-aware run uses 36 latent tokens; Appendix H reports the token-budget sweep, and Appendix G reports wall-clock inference cost. We use 36 tokens because it gives the best mean over HRBench4K, MMStar, and MathVista in the token-budget sweep, although 16 tokens is close and can be more efficient. For the main 36-token setting, the three-seed mean ± standard deviation scores are 73.25±0.220 on HRBench4K Overall, 63.40±0.340 on MMStar, and 71.00±0.120 on MathVista; Appendix Table 10 reports the corresponding mean ± standard deviation statistics across token budgets. 5.2. Main Results: Conservative Visual Latent Addition Tables 2 and 3 evaluate the overall effectiveness of GAP. The main question is whether the three-level alignment recipe can achieve strong mean point estimates on both perception and reasoning benchmarks. Comparison with prior latent methods. GAP (LH+PCA+DA) denotes the full method with a latent head (LH), PCA-aligned latent reconstruction (PCA), and difficulty-aware latent assignment (DA). The prior latent baselines improve perception but lose reasoning accuracy relative to the zero-shot Qwen2.5-VL 7B baseline: Avg-R drops from 52.62 to 47.99 for Monet and 47.66 for LVR, consistent with the norm-mismatch pattern analyzed in Section 3.1. Because Monet and LVR use their own training recipes and data, we interpret these rows as prior-system comparisons rather than a fully isolated architecture-only comparison. Under this comparison, Ours-GAP (LH+PCA+DA) improves Avg-P by +1.74 and Avg-R by +5.98 over Monet, and improves Avg-P by +0.57 and Avg-R by +6.31 over LVR. The matched-recipe ablations in Table 4 and Table 5 further separate data quality, difficulty-aware supervision, and PCA-aligned latent reconstruction. Table 2 | Main perception results. HRBench4K reports Overall, FSP, and FCP; MMStar reports Overall; MMERealWorld-Lite reports Overall, Reasoning, and Perception. Avg-P averages HRBench4K Overall, MMStar Overall, and MME-RealWorld-Lite Overall. Best results are in green and second-best in yellow. MMStar

MME-RW-Lite

FCP↑

Ovr.↑

Ovr.↑ Rsn.↑ Perc.↑

68.12 84.50 52.75 69.88 85.50 53.25 70.75 90.50 51.00 70.75 84.25 57.25

60.46 60.50 60.80 62.07

44.39 37.60 48.76 57.66 47.73 42.40 51.15 59.40 47.20 42.90 50.00 59.58 49.45 43.07 53.55 60.75

Ours-LH 70.63 85.0 56.25 Ours-LH+DA 71.50 85.00 58.75 Ours-LH+PCA 72.25 87.75 54.75 Ours-GAP (LH+PCA+DA) 73.25 86.50 60.00

61.20 61.80 62.10 63.40

45.25 39.20 49.82 59.03 45.75 39.06 50.00 59.68 46.85 39.80 51.50 60.40 47.31 43.07 50.79 61.32

Method

HRBench4K Ovr.↑

Qwen2.5-VL Dense Cap Monet-7B LVR

FSP↑

Avg-P

Across the aggregate metrics, Ours-GAP (LH+PCA+DA) achieves the highest Avg-P and Avg-R, while individual sub-scores show that the gains are not uniform across every benchmark component. Takeaway 1. The three-level alignment recipe gives the best mean aggregate perception and reasoning performance among the evaluated variants. Why dense captioning helps perception but hurts reasoning. Dense Caption SFT improves the perception aggregate from 57.66 to 59.40, suggesting that more detailed textual descriptions of the image help tasks that primarily require fine-grained visual recognition. However, it still trails the base model on MathVista and substantially degrades WeMath, reducing the reasoning aggregate from 52.62 to 47.24. We attribute this to a mismatch between dense descriptive supervision and multi-hop reasoning: the model learns to verbalize more visual details, but these extra textual descriptions can introduce spurious or hallucinated evidence when the task requires composing evidence across several reasoning steps. Why visual latents need alignment and selectivity. The basic LH row improves over Dense Caption

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Table 3 | Main reasoning results. MathVista measures multimodal mathematical reasoning, and WeMath is reported with strict (S) and loose (L) accuracy. Avg-R averages MathVista, WeMath-S, and WeMath-L. Best results are in green and second-best in yellow. Method

MathVista↑

WeMath-S↑

WeMath-L↑

Avg-R↑

Qwen2.5-VL 7B Dense Cap SFT Monet-7B LVR

68.50 67.70 61.30 67.00

36.29 28.60 32.67 26.95

53.06 45.43 50.00 49.05

52.62 47.24 47.99 47.66

Ours (LH) Ours (LH+DA, no PCA) Ours (LH+PCA) Ours-GAP (LH+PCA+DA)

68.60 68.90 70.20 71.00

30.00 31.26 35.24 36.33

48.86 50.00 52.00 54.57

49.15 50.05 52.48 53.97

SFT on HRBench4K and MathVista, but its lower Avg-P shows that a latent head alone is not sufficient for robust aggregate gains. PCA alignment addresses this by reconstructing generated latents in the empirical subspace spanned by real training-set vision embeddings, bringing the fed-back latents closer to the input vision-embedding subspace and norm regime. It also reduces the latent-head dimensionality, lowering the risk that the full head overfits the curated latent-supervision distribution; Appendix E provides an OOD POPE sanity check consistent with this interpretation. DA supervision adds selectivity by preserving the model’s existing ability on examples it can already solve, instead of blindly imposing latent targets where latent-text distribution mismatch may degrade performance. 5.3. Component Ablations We next separate the remaining ingredients behind GAP beyond the basic LH row in Tables 2 and 3: curated latent data with difficulty-aware supervision and PCA-aligned latent generation. These ablations complement the full benchmark suite above by isolating narrower two- or three-benchmark comparisons. Data quality and difficulty-aware supervision. Table 4 compares latent-supervision variants that differ in data source and assignment strategy. The curated 49K latent mixture improves substantially over training our latent-head recipe on Monet’s 125K SFT data, especially on MathVista. The difficulty-aware variant gives the best results in this comparison, suggesting that selective latent assignment is useful within the curated setting. Table 4 | Component comparison for latent supervision data and assignment strategy. Best results are in green and second-best in yellow. Training Data / Strategy

Size

HRBench4K↑

MathVista↑

Avg.↑

Monet latent model Ours (latent head on Monet SFT data) Ours, all latent Ours, difficulty-aware (LH+DA)

125K 125K 49K 49K

70.75 71.00 71.38 71.50

61.30 62.70 68.60 68.90

66.03 66.85 69.99 70.20

Takeaway 2. DA improves model-level alignment by using latent targets selectively. PCA components and capacity control. Table 5 varies the number of retained PCA components while keeping the backbone, curated training data, 36 latent tokens, difficulty-aware supervision, and evaluation benchmarks fixed. The uncompressed LH+DA head already improves over the base model, but all PCA+DA variants further improve Avg-3, suggesting that the input-space PCA parameterization is useful beyond simply adding head capacity. The default 95% retained-variance setting gives the best three-benchmark average, while fewer components appear to discard task-relevant visual detail. Takeaway 3. PCA capacity control improves feature-level alignment while preserving useful visual detail. These results support PCA-aligned reconstruction as a useful capacity-and-subspace constraint under our

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Table 5 | PCA component-count and capacity-control ablation under difficulty-aware latent supervision. Avg-3 averages HRBench4K Overall, MMStar, and MathVista. Δ is measured relative to Qwen2.5-VL-7B Avg-3. Best results are in green and second-best in yellow. Method

# Comp. RelMSE↓ HR4K MMStar MVista Avg-3

Qwen2.5-VL LH+DA PCA-0.85+DA PCA-0.90+DA GAP (PCA-0.95+DA)

– 3584 265 381 629

– – 0.1494 0.0999 0.0500

68.12 60.46 71.50 61.80 71.00 62.60 71.38 61.60 73.25 63.40

68.50 65.69 0.00 68.90 67.40 +1.71 69.30 67.63 +1.94 70.10 67.69 +2.00 71.00 69.22 +3.53

matched recipe, but they do not isolate PCA geometry from all possible low-rank alternatives; random-basis, no-mean, and matched-parameter controls remain important future work. 5.4. Generated Latents Contain Task-Relevant Visual Signal To test whether gains come from content-bearing latents rather than formatting or training-time regularization, we evaluate inference-time interventions on HRBench4K and MathVista. The zero-latent checkpoint refers to the GAP checkpoint evaluated with latent-token generation suppressed, so no generated PCA-decoded latent embedding is re-injected into the input stream; this row measures training and data effects without inference-time latent feedback. The noise setting replaces predicted PCA coefficients with Gaussian noise before the same reconstruction and norm matching. Table 6 | Latent content intervention. The zero-latent checkpoint row uses GAP with latent-token generation suppressed, while the noise row replaces predicted PCA coefficients with Gaussian noise before reconstruction. Avg-2 averages HRBench4K Overall and MathVista; Δ is relative to Qwen2.5-VL-7B Avg-2. Best results are in green and second-best in yellow. Intervention

Ckpt. Latents Content

Qwen2.5-VL Base Dense Cap Dense Zero-latent ckpt. GAP Noise latents GAP Clean latents GAP

0 0 0 36 36

HR4K↑ MVista↑ Avg-2↑

None 68.12 None 69.88 None 71.25 Gauss. noise 70.38 Pred. visual 73.25

68.50 67.70 69.40 69.00 71.00

68.31 0.00 68.79 +0.48 70.33 +2.02 69.69 +1.38 72.13 +3.82

Takeaway 4. Clean generated latents provide useful inference-time visual signal. The zero-latent checkpoint still beats Dense Caption SFT by +1.54 Avg-2, indicating that latent training shapes more visually grounded representations even without generating latent tokens at inference time. Using clean generated latents further improves Avg-2 from 70.33 to 72.13, while Gaussian coefficient noise drops below both the zero-latent checkpoint and clean-latent setting. Thus, clean predicted latents provide additional signal beyond formatting and training-time effects in this intervention setting. The main setting uses 36 latent tokens; Appendix H reports the full token-budget sweep. Together, these results suggest that visual latent capacity is most effective when it is aligned, content-bearing, and budgeted rather than simply increased.

6. Conclusion Our results suggest that visual latent reasoning benefits from aligning continuous tokens with inspectable supervision, input-compatible feature geometry, and selective difficulty-aware assignment. Across perception and reasoning benchmarks, this recipe gives the best mean aggregate results among the evaluated variants, while the ablations suggest that curated latent data, PCA-aligned reconstruction, and clean content-bearing latent feedback each contribute to the gains. The token-budget sweep further suggests that latent capacity should be controlled rather than simply increased.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

References Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report, 2025a. URL https://arxiv.org/abs/2511.21631. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025b. URL https://arxiv.org/abs/2502.13923. Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:2405.14838, 2024. Chengqi Duan, Rongyao Fang, Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng, Hongsheng Li, and Xihui Liu. Got-r1: Unleashing reasoning capability of mllm for visual generation with reinforcement learning. arXiv preprint arXiv:2505.17022, 2025. Google DeepMind. Gemma 4 model card. https://ai.google.dev/gemma/docs/core/model_card_ 4, 2026. Accessed: 2026-05-06. Yudong Han, Yong Wang, Zaiquan Yang, Zhen Qu, Liyuan Pan, and Xiangxiang Chu. Visual enhanced depth scaling for multimodal latent reasoning. arXiv preprint arXiv:2604.10500, 2026. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024. Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, and Jinwoo Shin. Vision-aligned latent reasoning for multi-modal large language model. arXiv preprint arXiv:2602.04476, 2026. Bangzheng Li, Ximeng Sun, Jiang Liu, Ze Wang, Jialian Wu, Xiaodong Yu, Hao Chen, Emad Barsoum, Muhao Chen, and Zicheng Liu. Latent visual reasoning. arXiv preprint arXiv:2509.24251, 2025a. Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought. arXiv preprint arXiv:2501.07542, 2025b. Kelvin Li, Chuyi Shang, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, and Roei Herzig. Latent implicit visual reasoning. arXiv preprint arXiv:2512.21218, 2025c. Tianyu Li, Dongchen Han, Zixuan Cao, Haofeng Huang, Mengyu Zhou, Ming Chen, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang, and Gao Huang. Siamesenorm: Breaking the barrier to reconciling pre/post-norm. arXiv preprint arXiv:2602.08064, 2026. Chengzhi Liu, Yuzhe Yang, Yue Fan, Qingyue Wei, Sheng Liu, and Xin Eric Wang. Reasoning within the mind: Dynamic multimodal interleaving in latent space. arXiv preprint arXiv:2512.12623, 2025. Yiming Qin, Bomin Wei, Jiaxin Ge, Konstantinos Kallidromitis, Stephanie Fu, Trevor Darrell, and XuDong Wang. Chain-of-visual-thought: Teaching vlms to see and think better with continuous visual tokens. arXiv preprint arXiv:2511.19418, 2025. Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6613–6629, 2025.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267, 2025. Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, et al. Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers. arXiv preprint arXiv:2506.23918, 2025. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. Kimi Team, Guangyu Chen, Yu Zhang, Jianlin Su, Weixin Xu, Siyuan Pan, Yaoyu Wang, Yucheng Wang, Guanduo Chen, Bohong Yin, et al. Attention residuals. arXiv preprint arXiv:2603.15031, 2026. Qixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang, Pengfei Wan, Kun Gai, Xianghua Ying, and Yisen Wang. Monet: Reasoning in latent visual space beyond images and language. arXiv preprint arXiv:2511.21395, 2025a. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025b. Yifan Wang, Shiyu Li, Peiming Li, Xiaochen Yang, Yang Tang, and Zheng Wei. Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning. arXiv preprint arXiv:2601.14750, 2026. Linquan Wu, Tianxiang Jiang, Yifei Dong, Haoyu Yang, Fengji Zhang, Shichaang Meng, Ai Xuan, Linqi Song, and Jacky Keung. Lavit: Aligning latent visual thoughts for multi-modal reasoning. arXiv preprint arXiv:2601.10129, 2026. Zhenda Xie, Yixuan Wei, Huanqi Cao, Chenggang Zhao, Chengqi Deng, Jiashi Li, Damai Dai, Huazuo Gao, Jiang Chang, Kuai Yu, et al. mhc: Manifold-constrained hyper-connections. arXiv preprint arXiv:2512.24880, 2025. Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. On layer normalization in the transformer architecture. In International conference on machine learning, pp. 10524–10533. PMLR, 2020. Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, and Chuang Gan. Machine mental imagery: Empower multimodal reasoning with latent visual tokens. arXiv preprint arXiv:2506.17218, 2025. Huanyu Zhang, Wenshan Wu, Chengzu Li, Ning Shang, Yan Xia, Yangyu Huang, Yifan Zhang, Li Dong, Zhang Zhang, Liang Wang, et al. Latent sketchpad: Sketching visual thoughts to elicit multimodal reasoning in mllms. arXiv preprint arXiv:2510.24514, 2025. Yang Zhang, Danyang Li, Yuxuan Li, Xin Zhang, Tianyu Xie, Mingming Cheng, and Xiang Li. Crystal: Spontaneous emergence of visual latents in mllms. arXiv preprint arXiv:2602.20980, 2026. Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. Pyvision: Agentic vision with dynamic tooling. arXiv preprint arXiv:2507.07998, 2025. Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

A. Architecture Illustration Figure 3 makes the interleaving explicit: ordinary text tokens are decoded through the LM head, latent spans are routed through the PCA-aligned latent head, and each reconstructed visual embedding is fed back as the next latent-token input.

B. Limitations and Scope 1. The full training and evaluation pipeline is validated on Qwen2.5-VL 7B and Monet-7B. Extending the same analysis to other pre-norm MLLMs is necessary before claiming cross-backbone universality. Post-norm backbones may still suffer from latent subspace mismatch, but the norm-growth mechanism studied here is specific to pre-norm residual streams. 2. Monet and LVR are included as prior latent-system baselines rather than retrained matched-data baselines. Retraining prior output-as-input methods on the same 49K curated data would further separate data effects from latent-feedback design effects. 3. The difficulty threshold 𝜏 = 0 is heuristic, and the current difficulty-aware comparison is not a perfectly isolated ablation because token budget and supervision policy interact. 4. PCA constrains latents to an empirical affine subspace and reduces head capacity. Section D adds a component-count sweep, but random-basis, no-mean, and matched-parameter full-head controls remain incomplete. 5. The <parser> field records the intended auxiliary visual signal, but it is not by itself a proof that the continuous latent vector faithfully encodes exactly that text. Stronger parser-latent faithfulness tests are left for future work. 6. The auxiliary image requirement during data construction limits the approach to domains where meaningful auxiliary views can be generated or sourced, and calls for careful duplicate and leakage checks against evaluation benchmarks. 7. Future work should test the same alignment design on additional MLLM backbones and study whether RL-based latent training gives further gains once the feedback space is input-compatible.

C. Background: PCA and Pre-Norm Decoders PCA reconstruction. Let {v𝑖 } 𝑖𝑁=1 denote vision embeddings extracted from auxiliary images by the frozen vision encoder, where v𝑖 ∈ ℝ𝑑 . We compute the empirical mean and covariance 𝑁

𝝁=

1 ∑︁ 𝑁

𝑁

v𝑖 ,

𝚺=

𝑖=1

1 ∑︁ 𝑁

(v𝑖 − 𝝁) (v𝑖 − 𝝁) ⊤ .

(5)

𝑖=1

Let P𝑘 = [p1 , . . . , p𝑘 ] ∈ ℝ𝑑 × 𝑘 be the top-𝑘 eigenvectors of 𝚺. For any centered vision embedding, PCA gives coefficients and a rank-𝑘 reconstruction: c = P⊤ 𝑘 (v − 𝝁) ,

v̂ = P𝑘 c + 𝝁.

(6)

We report the relative reconstruction error as the fraction of centered vision-embedding variance not captured by the rank-𝑘 subspace: Í𝑘 2 1 Í𝑁 ⊤ 𝑗=1 𝜆 𝑗 𝑖=1 v𝑖 − P𝑘 P𝑘 (v𝑖 − 𝝁) + 𝝁 2 𝑁 = 1 − Í𝑑 , (7) RelMSE( 𝑘) = 2 1 Í𝑁 𝑗=1 𝜆 𝑗 𝑖=1 ∥v𝑖 − 𝝁 ∥ 2 𝑁 where 𝜆 𝑗 are the eigenvalues of the empirical covariance. Lower relative reconstruction error means that the PCA subspace retains more of the auxiliary-image vision-embedding variance. In our model, the latent head predicts c𝑡 rather than an unconstrained full-dimensional embedding. The generated latent is then reconstructed by Eq. 6, which constrains feedback tokens to the affine subspace estimated from real vision embeddings. Pre-norm decoder states. Modern decoder-only transformers, including the Qwen2.5-VL backbone used in our experiments, are typically pre-norm architectures. Ignoring layer indices on attention keys and values

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

(a) Interleaved Latent Inference

(b) Latent Token Generation [Answer]

...

Hidden state

Final Hidden State

Latent tokens Text tokens

LM Head

PCA-aligned latent head

LM Head

Vision tokens

PCA-aligned latent head RMSNorm Linear PCA init

MLLM Decoder

MLP adapter PCA coeffs

... PCA reconstruct [Image] [Question]

Response format:

Text

</latent>

Visual Latent Tokens

</latent>

Latent Token

Text

Figure 3 | Interleaved visual latent inference and latent-token generation. (a) The MLLM generates ordinary text tokens through the LM head and generates visual latent tokens inside a <latent> span. Each generated latent token is fed back autoregressively as a subsequent latent-token input, giving a response format of text, visual latent tokens, and text. (b) A latent token is generated from the final decoder state by applying the backbone’s native final RMSNorm, a PCA-aligned latent head that predicts PCA coefficients, and a fixed PCA reconstruction step. The reconstructed embedding is then used as the next latent token; auxiliary-image targets are used only to train this path. for readability, a pre-norm block has the form x′ℓ = xℓ + Attnℓ (RMSNormℓ (xℓ )) , xℓ+1 = x′ℓ + MLPℓ RMSNorm′ℓ (x′ℓ ) .

(8) (9)

RMSNorm itself rescales a hidden vector by its root-mean-square magnitude: RMSNorm(x) = g ⊙ √︃ Í 𝑑 1 𝑑

x ,

(10)

2 𝑗=1 𝑥 𝑗 + 𝜖

where g is a learned scale vector. This placement explains why hidden-state norms can accumulate with depth. For a single residual update, write xℓ+1 = xℓ + uℓ ,

uℓ = 𝐹ℓ (RMSNorm(xℓ )) ,

(11)

where 𝐹ℓ denotes the attention or MLP branch. Then ∥xℓ+1 ∥ 22 = ∥xℓ ∥ 22 + ∥uℓ ∥ 22 + 2 ⟨xℓ , uℓ ⟩ .

(12)

Pre-norm controls the input scale seen by 𝐹ℓ , but it does not normalize the residual stream after uℓ is added. Therefore, if residual updates have nonzero energy and are not consistently anti-aligned with the current residual stream, i.e., 𝔼[⟨xℓ , uℓ ⟩] ≈ 0, 𝔼 ∥uℓ ∥ 22 > 0, (13) then the expected squared norm accumulates: 𝐿 −1 ∑︁ 𝔼 ∥x𝐿 ∥ 22 ≈ 𝔼 ∥x0 ∥ 22 + 𝔼 ∥uℓ ∥ 22 .

(14)

ℓ=0

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

This is not a monotonicity guarantee for every individual layer, because the inner-product term in Eq. 12 can be negative. Rather, it describes the expected residual-stream accumulation pattern observed in pre-norm decoders: normalization stabilizes each branch input, while the unnormalized residual stream can grow with depth. Many pre-norm decoders also apply a final RMSNorm before the output readout: h̄𝑡 = RMSNormfinal (h𝑡( 𝐿 ) ) ,

logits𝑡 = Wvocab h̄𝑡 .

(15)

Our latent head uses this same backbone-normalized state: c𝑡 = F𝜃 ( h̄𝑡 ) ,

v̂𝑡 = P𝑘 c𝑡 + 𝝁.

(16)

This distinction is important. The final RMSNorm stabilizes the decoder readout scale, but it does not make h̄𝑡 an input-compatible vision embedding. PCA reconstruction supplies the missing input-compatibility step: it maps the normalized decoder state into coefficients of the empirical vision-embedding subspace before the latent is fed back as the next input token.

D. PCA Component-Count and Capacity-Control Ablation Table 7 varies the PCA retained-variance threshold while keeping the backbone, curated training data, 36 latent tokens, difficulty-aware supervision, and evaluation benchmarks fixed. The baseline Qwen2.5-VL-7B row has no latent head, while LH+DA uses the full latent head in the original 𝑑 = 3584 vision-embedding space without PCA compression. All PCA rows use the same difficulty-aware policy and differ only in the number of retained PCA components induced by the retained-variance threshold. Table 7 | PCA component-count ablation under difficulty-aware latent supervision. Component ratio is measured relative to the original vision-embedding dimension 𝑑 = 3584; the full-head LH+DA baseline predicts in this original space without PCA compression. RelMSE is the relative reconstruction error in Eq. 7. Avg-3 is the unweighted average of HRBench4K Overall, MMStar, and MathVista. Δ is measured relative to Qwen2.5-VL-7B Avg-3. Best results are in green and second-best in yellow. Run

Var. Ratio # Comp. RelMSE↓ Tok. HR4K MMStar MVista Avg-3

Qwen2.5-VL – – LH+DA 1.00 100.0% PCA-0.85+DA 0.85 7.4% PCA-0.90+DA 0.90 10.6% GAP (LH+PCA+DA) 0.95 17.6%

– 3584 265 381 629

– – 0.1494 0.0999 0.0500

0 36 36 36 36

68.12 71.50 71.00 71.38 73.25

60.46 61.80 62.60 61.60 63.40

68.50 65.69 0.00 68.90 67.40 +1.71 69.30 67.63 +1.94 70.10 67.69 +2.00 71.00 69.22 +3.53

The uncompressed LH+DA head already improves over the base model, showing that the auxiliary latent objective is useful. However, all PCA+DA variants further improve Avg-3 relative to the uncompressed head, indicating that constraining the generated latent to an empirical vision-embedding coordinate system is helpful beyond adding a full latent head. The relative reconstruction error decreases from 0.1494 at PCA-0.85+DA to 0.0999 at PCA-0.90+DA and 0.0500 at PCA-0.95+DA, matching the expected residual variance left outside the retained PCA subspace. The downstream trend follows the same broad direction: Avg-3 rises from 67.63 and 67.69 with fewer components to 69.22 at PCA-0.95+DA. This does not imply that reconstruction error alone determines benchmark performance, but it supports the interpretation that retaining too few components discards task-relevant visual detail. At the same time, the PCA rows outperform the uncompressed LH+DA head, suggesting that the benefit is not merely from increasing latent-head capacity; the input-space PCA parameterization itself is useful. We therefore interpret PCA as a subspace-and-capacity design choice rather than as a standalone normalization operation.

E. OOD Hallucination Evaluation We additionally evaluate POPE as an OOD hallucination-sensitive sanity check. POPE uses yes/no objectrecognition queries under popular, adversarial, and random negative sampling, which differs from our main benchmarks: the desired behavior is conservative binary recognition rather than producing richer visual evidence for long-form reasoning.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Table 8 | OOD hallucination evaluation on POPE. Scores are reported for the overall benchmark and its popular, adversarial, and random splits. Best results are in green and second-best in yellow. Method

Overall↑

Popular↑

Adversarial↑

Random↑

Qwen2.5-VL-7B Ours (LH+DA, no PCA) Ours-GAP (LH+PCA+DA)

86.25 86.02 86.30

86.34 85.70 85.76

85.33 84.36 84.49

87.03 87.09 88.70

Table 8 shows that the unconstrained LH+DA variant slightly underperforms the base model on overall POPE, despite improving the main perception and reasoning benchmarks. In contrast, GAP recovers this drop and slightly improves the overall score. We interpret this pattern as suggestive evidence that an unconstrained full latent head can over-specialize to the curated latent-supervision distribution, while the PCA reconstruction path acts as a capacity and subspace constraint by forcing generated latents to lie in the empirical vision-embedding subspace. This should not be read as a claim that GAP solves hallucination: the overall gain is mainly driven by the random split, while the popular and adversarial POPE splits remain below the base model, indicating that hallucination-sensitive OOD cases remain challenging.

F. Implementation Details Training hyperparameters. We fine-tune with AdamW, weight decay 0.01, cosine learning-rate decay, warmup ratio 0.03, peak base learning rate 1×10−5 , peak latent-head learning rate 1×10−5 , global batch size 128, per-device batch size 1, and 2 epochs. The main difficulty-aware run uses 36 latent tokens; Section H reports a latent-token-budget ablation because this choice affects both alignment capacity and inference cost. Training uses bf16, FlashAttention-2, gradient checkpointing, 16 dataloader workers, and seed 12345. PCA computation. The PCA basis is computed offline using auxiliary-image vision embeddings from the curated training set. We retain the top 𝑘 = 629 principal components using a 95% explained-variance target. The basis is fixed throughout training and not updated. Latent head implementation. The latent projector is initialized from P⊤ so that the head starts from 𝑘 the empirical vision subspace rather than an unconstrained random basis. We then refine this projection with a compact SwiGLU adapter. At inference time, ordinary text positions use the LM head, whereas latent positions use the latent head to produce PCA coefficients that are decoded by the fixed basis before feedback, as illustrated in Appendix Figure 3. Scheduled sampling. During teacher-forced latent training, <|latent_pad|> positions receive the auxiliary-image vision features. When scheduled sampling is enabled as an ablation, these features can be replaced by the model’s PCA-decoded predictions before a second forward pass, making training closer to inference where generated latents are fed back autoregressively. Unless otherwise stated, all results in the main tables use teacher forcing and do not use scheduled sampling. Difficulty-aware data loading. For examples with an empirical accuracy field greater than 𝜏, the loader strips the latent span and auxiliary-image placeholder targets, so the example contributes only languagemodeling loss. For examples with accuracy at or below 𝜏, the loader retains the latent span and auxiliaryimage target embeddings for joint LM and latent-head supervision. The main experiments use 𝜏 = 0, matching Eq. 4. EMA norm calibration. The EMA norm calibration is used only as a training-free diagnostic for the norm-mismatch analysis. At inference, the running average 𝑛¯EMA is initialized from the query image’s vision embedding norm and updated token by token before rescaling generated latents to the input vision-embedding norm regime.

G. Training and Inference Cost Training cost. The main GAP (LH+PCA+DA) model is trained on 32 NVIDIA B200 GPUs for 3 hours and 55 minutes of wall-clock time. This reports student fine-tuning cost only; offline auxiliary-image generation,

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

teacher response generation, PCA computation, and benchmark evaluation are excluded. Latency measurement. We report wall-clock latency in seconds per sample measured inside the VLMEvalKit inference wrapper. All rows use the direct benchmark prediction files and ignore duplicate judge/output files. Because Dense Caption SFT uses the same 49K curated training data and decoding setup but removes latent tokens and auxiliary-image alignment, we use it as the primary latency baseline for relative cost. Qwen2.5-VL-7B is kept as a performance reference in the main tables, but not as the relative-latency denominator because its earlier timing was collected under an unmatched setup. Table 9 | Inference cost on HRBench4K and MathVista. Latency is wall-clock seconds per sample. Mean latency is the primary cost metric; relative latency uses Avg-2 mean latency and Dense Caption SFT as the denominator. Avg-2 performance averages HRBench4K Overall and MathVista. For latency and relative cost, lower is better; for Avg-2, higher is better. Best results are in green and second-best in yellow. Method

Tok. Mean HR/MV/Avg-2 (s)

Rel.

Avg-2

18.81 / 28.74 / 23.78 27.05 / 44.08 / 35.56 29.55 / 46.50 / 38.02 29.59 / 47.13 / 38.36 28.47 / 49.14 / 38.80 28.09 / 46.96 / 37.53

1.00× 1.50× 1.60× 1.61× 1.63× 1.58×

68.79 70.64 72.27 72.13 71.98 70.83

Dense Caption SFT 0 Ours-GAP (LH+PCA+DA) 4 Ours-GAP (LH+PCA+DA) 16 Ours-GAP (LH+PCA+DA) 36 Ours-GAP (LH+PCA+DA) 64 Ours-GAP (LH+PCA+DA) 144

The cost increase is moderate but non-negligible: the main 36-token setting is 1.61× slower than Dense Caption SFT on Avg-2 mean latency, while improving Avg-2 performance from 68.79 to 72.13. The latency is not strictly monotonic in the latent-token budget because wall-clock time is also affected by generated response length and benchmark-specific long tails. For example, Dense Caption SFT has Avg-2 P50/P90 latencies of 10.63/30.27 seconds, while the 36-token setting has 16.10/106.71 seconds; the long tail is dominated by MathVista responses. Thus, latent tokens provide a clear accuracy-cost tradeoff rather than a free improvement, which motivates the budgeted-latent analysis in Figure 4.

H. Latent-Token Budget Sweep The number of latent tokens is a core design choice: too few tokens may under-represent auxiliary visual evidence, while too many tokens can introduce unnecessary computation and redundant self-generated visual features. We evaluate the GAP (LH+PCA+DA) model with different latent-token budgets on HRBench4K, MMStar, and MathVista, using the zero-shot Qwen2.5-VL 7B model as a reference baseline. Because latent features are arranged as a square grid with equal height and width, all positive token budgets are perfect squares.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

(a) HRBench4K

(b) MMStar

Score

avg. 4024x3503 px

76 75 74 73 72 71 69.88 70 69 68 67 66 0

avg. 511x391 px

72.63

73.25

73.75

73.25

63.4 62.8

62.1

70.38

61 60.5 Qwen 60.46

Qwen 68.12

59 4

144

avg. 584x440 px

69.11

70.9

144

resolution-mixed average

71 71.9

(d) Average (3 benchmarks)

62.1

61.4

70.2

70 69

68.4

68 67.7

Qwen 68.5

69.22 68.68 67.91

67.56

67 66

66.03 Qwen 65.69

67 66

64 0

144

Number of latent tokens

144

Figure 4 | Effect of latent-token budget. We sweep the number of generated visual latent tokens while keeping the backbone, training data, PCA-aligned latent head, and difficulty-aware rule fixed. The gray dashed line is the Qwen2.5-VL 7B baseline. Panel subtitles report average image resolution estimated from the evaluated benchmark images. Positive token counts are square latent grids: 4=2×2, 16=4×4, 36=6×6, 64=8×8, and 144=12×12. Table 10 reports three-seed mean ± standard deviation results for the token-budget sweep on HRBench4K, MMStar, and MathVista. These runs cover the 36-token setting used in the main tables and the nearby capacity settings used to choose that default. Table 10 | Three-seed mean ± standard deviation results for the latent-token budget sweep in Figure 4. Perbenchmark columns report mean ± standard deviation, while Avg-3 reports the mean average over HRBench4K Overall, MMStar, and MathVista. Best results are in green and second-best in yellow. Latent tokens HRBench4K 0 16 36 64 144

69.88±0.211 72.63±0.176 73.25±0.220 73.75±0.295 73.25±0.134

MMStar

MathVista

Avg-3

60.50±0.431 62.80±0.387 63.40±0.340 62.10±0.289 62.10±0.412

67.70±0.183 71.90±0.215 71.00±0.120 70.20±0.141 68.40±0.167

66.03 69.11 69.22 68.68 67.91

Figure 4 shows that latent-token capacity helps, but the effect is not monotonic. The best average over HRBench4K, MMStar, and MathVista is achieved at 36 tokens, with 16 tokens close behind, after which performance saturates or declines. HRBench4K benefits from larger latent grids: its score rises from 69.88 with zero latent tokens to 73.75 with 64 tokens, and remains high at 144 tokens. The lower-resolution benchmarks saturate earlier: MathVista peaks at 16 latent tokens, and MMStar peaks around 36 tokens before dropping at larger budgets. These results suggest that visual latent tokens should be budgeted rather than maximized.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

I. Data Construction Details Training Data Sources (total = 49,309) 20,000 (40.6%)

Visual CoT chart

7,282 (14.8%)

GEOQA

7,057 (14.3%) 5,345 (10.8%)

MM Math Zebra CoT count

2,766 (5.6%)

Zebra CoT visual search

2,687 (5.4%) 2,032 (4.1%)

MathVision

1,082 (2.2%)

geo 3k CogCoM

567 (1.1%)

ReFocus

426 (0.9%)

Zebra CoT geometry 65 (0.1%)

Share of training data (%)

Figure 5 | Composition of the 49K curated training set (49,309 examples). Visual CoT is the largest source, followed by chart, GEOQA, and multimodal math data; the remaining sources emphasize geometry, counting, visual search, and focused visual reasoning. Auxiliary image generation and response synthesis. For each training query, we first use Nano Banana 2.0 to generate an intermediate auxiliary image intended to expose the missing visual evidence needed by the question. We then provide the query image, question, and generated auxiliary image to Qwen3-VL-235B-Thinking, which produces the structured response containing the reasoning trace, <latent> span, <parser> description, and final answer. Only the student model is trained: the auxiliary image is converted into frozen-ViT features for latent-head supervision, while Qwen3-VL-235B-Thinking serves only as an offline teacher for response synthesis. Latent serialization. The student input contains only the query image and question. The target response follows the structure <think> context <latent> auxiliary-image embedding targets </latent> <parser> dense caption </parser> continued reasoning </think> <answer> answer </answer>. In the implementation, each latent span is serialized as <|latent_start|> followed by fixed <|latent_pad|> tokens and <|latent_end|>. The main setting uses 36 latent tokens unless stated otherwise. Quality filtering. We filter out examples where (1) the base model’s repeated-sampling accuracy is above 𝜏 (assigned to text-only supervision), (2) the auxiliary image embedding is degenerate, or (3) the generated <parser> output fails a length or coherence check. After filtering, the curated training set contains 49,309 examples. Leakage audit. We audit the curated training set against the evaluation benchmarks used in this paper. The audit finds no exact image duplicates and no exact normalized question-text duplicates between training and evaluation examples. We also manually inspect high-similarity cases surfaced by permissive near-duplicate heuristics. These cases primarily arise from public geometry sources such as GEOQA and Geo3K, which are also reused or adapted by existing evaluation benchmarks. We therefore avoid claiming that the training and evaluation distributions are fully disjoint, but verify that the training set does not contain exact copies of the evaluated examples.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

I.1. Auxiliary-Image Supervision Example Figure 6 shows one training example from our data construction pipeline. The original query asks: “How many bricks are missing in the wall?” The auxiliary image reconstructs the expected running-bond pattern and labels the six missing brick positions. This example illustrates that the teacher response provides not only a final answer, but also a human-readable description of the intended auxiliary visual evidence used to supervise the latent channel.

(a) Query image

(b) Generated auxiliary image

Figure 6 | Example of auxiliary-image supervision. The auxiliary image makes the missing visual evidence explicit by reconstructing the wall pattern and numbering the six missing bricks.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Qwen3-VL-235B-Thinking: teacher response excerpt To determine how many bricks are missing, I need to reconstruct the complete wall pattern and count the gaps. The wall uses a standard running bond pattern where each brick is half-offset from the row above and below. First, I should create an auxiliary image that overlays the missing sections with the expected brick positions to clearly identify each missing brick. <|vision_start|><|image_pad|><|vision_end|> <parser> • Draw the original brick wall with the damaged section. • In the damaged area, add dashed outlines of the bricks that should be present to complete the running bond pattern. • Label each missing brick location with a number, 1 through 6, corresponding to the auxiliary image. • Highlight the damaged area with a light red background to distinguish it from the intact wall. • Ensure the dashed outlines match the size and orientation of standard bricks in the wall. </parser> Using the pattern established in the auxiliary image, I can now map the damaged section to see how many bricks should be there. The damaged area spans multiple rows: • In the row containing brick 1, there is one missing brick. • In the row containing bricks 2 and 3, there are two missing bricks. • In the row containing bricks 4 and 5, there are two missing bricks. • In the row containing brick 6, there is one missing brick. Adding these up gives 1 + 2 + 2 + 1 = 6 missing bricks. The auxiliary image shows this by labeling exactly six missing brick positions. <answer> The wall follows a running bond pattern, where adjacent rows are offset by half a brick. By comparing the damaged region to the intact rows and the auxiliary image, the missing bricks are labeled 1 through 6. Therefore, the total number of missing bricks is 6 . </answer> This example shows how the teacher turns an image-level counting problem into a concrete visual reconstruction target. The auxiliary image supplies frozen-ViT features for latent supervision, while the <parser> text records the intended visual content of that latent target. During student training, the auxiliary image itself is never provided as input to the student; it is used only as the supervision target for the latent head.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

J. Qualitative Inference Example Figure 7 shows a MathVista geometry example where Qwen2.5-VL-7B predicts an incorrect answer, while our model recovers the correct auxiliary construction and answer. The question asks: “As shown in the figure, in the parallelogram 𝐴𝐵𝐶 𝐷, 𝐶𝐸 bisects ∠ 𝐵𝐶 𝐷 and it intersects the 𝐴𝐷 edge at point 𝐸, and 𝐷𝐸 = 3.0, then the length of 𝐴𝐵 is ().” The answer choices are: (A) 1, (B) 2, (C) 3, and (D) 6.

(a) Question image

(b) Qwen2.5-VL-7B attention

Figure 7 | A qualitative MathVista example. Attention maps are shown only for qualitative inspection. Baseline failure. Qwen2.5-VL-7B retrieves a plausible theorem but grounds it to the wrong geometric configuration. The following response excerpt is edited only for formatting and math typesetting, while preserving the model’s reasoning chain:

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Qwen2.5-VL-7B: incorrect response excerpt To solve this problem, we need to use the properties of a parallelogram and the angle bisector theorem. 1. Identify the properties of the parallelogram. In a parallelogram, opposite sides are equal in length, opposite angles are equal, and consecutive angles are supplementary. 2. Use the angle bisector theorem. The angle bisector theorem states that if a point divides one side of a triangle into segments proportional to the other two sides, then it is the angle bisector of the opposite angle. 3. Apply the angle bisector theorem to triangle 𝐵𝐶 𝐷. Since 𝐶𝐸 bisects ∠ 𝐵𝐶 𝐷, it divides ∠ 𝐵𝐶 𝐷 into two equal angles. By the angle bisector theorem, the ratio of the segments created by the angle bisector is equal to the ratio of the other two sides. 4. Set up the proportion. Let 𝐴𝐵 = 𝑥 and 𝐶 𝐷 = 𝑥 , since opposite sides of a parallelogram are equal. According to the angle bisector theorem, 𝐷𝐸 𝐶𝐷 = . 𝐴𝐸 𝐴𝐵

5. Substitute the known values. We have 𝐷𝐸 = 3, 𝐴𝐸 = 𝐴𝐷 − 𝐷𝐸 = 𝑥 − 3, 𝐶 𝐷 = 𝑥 , and 𝐴𝐵 = 𝑥 . Therefore, 3 𝑥 = . 𝑥−3 𝑥 6. Solve the proportion. 3 𝑥−3

= 1,

3 = 𝑥 − 3,

𝑥 = 6.

Therefore, the length of 𝐴𝐵 is 6. The correct answer is 𝐷 . The response applies an angle-bisector-theorem proportion, 𝐷𝐸/ 𝐴𝐸 = 𝐶 𝐷/ 𝐴𝐵, and then substitutes 𝐴𝐸 = 𝐴𝐷 − 𝐷𝐸 = 𝑥 − 3 to predict 𝐴𝐵 = 6. It therefore selects choice D. This is geometrically invalid: 𝐸 lies on 𝐴𝐷, not on the opposite side of △ 𝐵𝐶 𝐷, so the stated angle-bisector proportion does not follow from the diagram. This failure is representative of textually fluent but visually misgrounded reasoning: the model selects a familiar theorem, but attaches it to the wrong line segment.

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

Our response. Our model instead identifies the equal-angle relation induced by the parallelogram structure and the angle bisector. The following response excerpt keeps the main reasoning, latent span, parser content, and final answer: Ours: correct response excerpt We are given a parallelogram 𝐴𝐵𝐶 𝐷 with 𝐶𝐸 bisecting ∠ 𝐵𝐶 𝐷 and intersecting 𝐴𝐷 at 𝐸. We know 𝐷𝐸 = 3.0 and need to find 𝐴𝐵. In a parallelogram, opposite sides are equal, so 𝐴𝐵 = 𝐶 𝐷. The key is to relate 𝐷𝐸 to 𝐶 𝐷 using the angle bisector and the parallel lines. First, recall that in a parallelogram, 𝐴𝐷 ∥ 𝐵𝐶 . Therefore, ∠ 𝐷𝐸𝐶 = ∠ 𝐸𝐶 𝐵 by alternate interior angles for parallel lines cut by the transversal 𝐶𝐸. Since 𝐶𝐸 bisects ∠ 𝐵𝐶 𝐷, ∠ 𝐵𝐶𝐸 = ∠ 𝐷𝐶𝐸. Combining these angle equalities gives ∠ 𝐷𝐸𝐶 = ∠ 𝐷𝐶𝐸. Thus, △ 𝐷𝐸𝐶 is isosceles, and 𝐷𝐸 = 𝐷𝐶 . The problem states 𝐷𝐸 = 3.0, so 𝐷𝐶 = 3.0. Since opposite sides of a parallelogram are equal, 𝐴𝐵 = 𝐶 𝐷 = 3.0. To make this reasoning explicit, we should construct an auxiliary image that highlights the key angle equalities and the isosceles triangle. <latent> visual latent tokens </latent> <parser> • Draw parallelogram 𝐴𝐵𝐶 𝐷 with 𝐴𝐵 ∥ 𝐶 𝐷 and 𝐴𝐷 ∥ 𝐵𝐶 . • Mark point 𝐸 on 𝐴𝐷 such that 𝐷𝐸 = 3.0. • Draw segment 𝐶𝐸. • Label ∠ 𝐵𝐶𝐸 and ∠ 𝐷𝐶𝐸 as equal. • Label ∠ 𝐷𝐸𝐶 and ∠ 𝐷𝐶𝐸 as equal to show the isosceles triangle. • Highlight △ 𝐷𝐸𝐶 with a light fill. • Mark 𝐷𝐸 = 3.0 on the segment. • Indicate with equal-length ticks that 𝐷𝐸 = 𝐷𝐶 . • Write 𝐴𝐵 = 𝐶 𝐷 near the top side. </parser> From the auxiliary image, we see that ∠ 𝐷𝐸𝐶 = ∠ 𝐷𝐶𝐸, so △ 𝐷𝐸𝐶 is isosceles with 𝐷𝐸 = 𝐷𝐶 . Given 𝐷𝐸 = 3.0, it follows that 𝐷𝐶 = 3.0. Since 𝐴𝐵𝐶 𝐷 is a parallelogram, 𝐴𝐵 = 𝐶 𝐷. Therefore, 𝐴𝐵 = 3.0. <answer> In parallelogram 𝐴𝐵𝐶 𝐷, 𝐴𝐷 ∥ 𝐵𝐶 , so ∠ 𝐷𝐸𝐶 = ∠ 𝐸𝐶 𝐵. Since 𝐶𝐸 bisects ∠ 𝐵𝐶 𝐷, ∠ 𝐸𝐶 𝐵 = ∠ 𝐷𝐶𝐸. Therefore, ∠ 𝐷𝐸𝐶 = ∠ 𝐷𝐶𝐸, making △ 𝐷𝐸𝐶 isosceles with 𝐷𝐸 = 𝐷𝐶 . Given 𝐷𝐸 = 3.0, we have 𝐷𝐶 = 3.0. In a parallelogram, opposite sides are equal, so 𝐴𝐵 = 𝐶 𝐷. Thus, the length of 𝐴𝐵 is 3.0 , corresponding to choice C. </answer> Interpretation. The example illustrates the intended role of visual latent reasoning. The base model follows a language-plausible but geometrically invalid theorem application, whereas our model produces a latent/parser trace that corresponds to a task-relevant auxiliary construction: highlight △ 𝐷𝐸𝐶 , expose ∠ 𝐷𝐸𝐶 = ∠ 𝐷𝐶𝐸, and infer 𝐷𝐸 = 𝐷𝐶 . The attention maps provide a qualitative visualization of the two runs, but we do not treat attention as causal evidence. Instead, this example complements the intervention study in Table 6, where disabling or replacing the latent rollout reduces performance relative to clean generated latents.

Record · ID 178891 · SHA-256 df76ef21765dcecb
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.