ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both Ziyu Guo1,2 , Rain Liu1 , Xinyan Chen2 , Pheng-Ann Heng2
arXiv:2605.15198v1 [cs.CV] 14 May 2026
1
Meta AI, 2 The Chinese University of Hong Kong
Visual reasoning, often interleaved with intermediate visual states, has emerged as a promising direction in the field. A straightforward approach is to directly generate images via unified models during reasoning, but this is computationally expensive and architecturally non-trivial. Recent alternatives include agentic reasoning through code or tool calls, and latent reasoning with learnable hidden embeddings. However, agentic methods incur context-switching latency from external execution, while latent methods lack task generalization and are difficult to train with autoregressive parallelization. To combine their strengths while mitigating their limitations, we propose ATLAS, a framework in which a single discrete “word”, termed as a functional token, serves both as an agentic operation and a latent visual reasoning unit. Each functional token is associated with an internalized visual operation, yet requires no visual supervision and remains a standard token in the tokenizer vocabulary, which can be generated via next-token prediction. This design avoids verbose intermediate visual content generation, while preserving compatibility with the vanilla scalable SFT and RL training, without architectural or methodological modifications. To further address the sparsity of functional tokens during RL, we introduce Latent-Anchored GRPO (LA-GRPO), which stabilizes the training by anchoring functional tokens with a statically weighted auxiliary objective, providing stronger gradient updates. Extensive experiments and analyses demonstrate that ATLAS achieves superior performance on challenging benchmarks while maintaining clear interpretability. We hope ATLAS offers a new paradigm inspiring future visual reasoning research. Project Page: https://atlas-oneword.github.io Correspondence: [email protected]
1
Introduction
The rapid evolution of Vision-Language Models (VLMs) Bai et al. (2025a); An et al. (2025); Bai et al. (2025b); Seed (2026); Team et al. (2024); Li et al. (2024) has advanced multimodal intelligence from perception toward reasoning Jiang et al. (2025). In these tasks, purely textual reasoning is often insufficient, as problem solving frequently requires intermediate visual analysis Shao et al. (2024a); Zhao et al. (2025a); Chern et al. (2024). This capability, commonly studied as interleaved visual reasoning, involves generating, perceiving, and using intermediate visual states to guide subsequent inference Chen et al. (2025); Qiao et al. (2025b); Su et al. (2025). For instance, game solving may require updating the board state after each operation, while geometry solving may require constructing auxiliary lines to reveal hidden relations Hu et al. (2024); Zhang et al. (2024). Despite strong progress in direct visual understanding, current VLMs still remain limited in this dynamic visual reasoning process. Unified models Deng et al. (2025); Li et al. (2025b); Zhao et al. (2025a); Liu et al. (2025b); Wu et al. (2024); Xie et al. (2024) provide a straightforward solution by explicitly generating pixel-level images, as illustrated in Fig. 1I. This paradigm is intuitive: the model externalizes intermediate visual representations in the same modality as the input. However, generating new images introduces substantial inference cost and training difficulty. The model must allocate significant capacity to image decoding and re-encoding, and requires non-trivial framework-level architectural designs, which often necessitates pre-training from scratch.
1
Text Token
Image Token
Functional Token
High Training Overhead !
I. Unified Model’s Visual Reasoning Question: How much will it cost to fence the perimeter at £37/m?
Generated Image(s) Answer: £768.50.
Gen Head
I will generate a geometrical diagram with auxiliary lines…
From the diagram, I can solve for x,y… I will compute the cost…
II. Agentic Visual Reasoning
High Tool-execution Latency !
Question: How much will it cost to fence the perimeter at £37/m?
Code Executor
…
Tool Executor
…
First, I will write code to add an auxiliary line…
…
G
Answer: £768.50.
…
Next, I will call functions to label …
I can solve for x,y, find the perimeter … I will compute the cost…
Breaks Training Parallelization !
III. Latent Visual Reasoning Question: How much will it cost to fence the perimeter at £37/m?
Visual Loss
G
Visual Loss
Answer: £768.50. I will draw auxiliary lines…
IV.
Continuous Latent Embed.
✓ Efficient
ATLAS’s Visual Reasoning
Question: How much will it cost to fence the perimeter at £37/m?
Then, I will label FG…
Continuous Latent Embed.
Upon that, I can compute the cost…
✓ Native AR Compatibility
CE Loss
CE Loss
✓ Better Performance
CE Loss
Answer: £768.50. <|Line|>
“I will first add an auxiliary line…”
<|Text|>
“Next, I will calculate and label FG, BG, and EG…”
<|Text|>
“Upon that, I will ...”
“Finally, I …”
Figure 1 Comparison of Visual Reasoning Paradigms. I: Unified models generate intermediate pixel-level images. II: Agentic methods rely on external code or tool execution. III: Latent methods conduct intermediate reasoning through latent embeddings. IV: ATLAS briges agentic and latent visual reasoning through discrete vocabulary functional tokens
within the standard autoregressive generation loop, more efficient and effective.
To better preserve the standard VLM architecture, existing methods explore two alternative routes. First, agentic visual reasoning Gupta and Kembhavi (2023); Hu et al. (2024); Surís et al. (2023) in Fig. 1.II, treats the VLM as a high-level controller that generates code or tool calls to manipulate the visual input through external modules. Although its computational overhead is lower than that of generating full intermediate images, it still often requires verbose code or tool-call formulations even for simple visual operations, increasing output length and inference latency. Second, latent reasoning Wang et al. (2025b); Li et al. (2025a); Qin et al. (2025) in Fig. 1.III, performs intermediate reasoning in hidden representations rather than generating images or long textual operations. However, the supervision signals for latent embeddings are derived from a specific range of tasks, limiting their generalization to broader domains. More critically, they introduce recurrent latent dependencies Hao et al. (2024), which break the compatibility with standard parallel training and substantially increase training cost. 2
In this paper, we propose ATLAS, a framework in which only a single functional “word” serves as both an agentic operation and a latent reasoning unit, as illustrated in Fig. 1.IV. The key idea of ATLAS is to represent each visual operation as a standard discrete token in the tokenizer vocabulary, such as zooming into a region, constructing auxiliary lines, drawing shapes, adding arrows, or inserting textual labels. These tokens are generated through ordinary next-token prediction within the same sequence as natural language tokens, rather than being modeled as continuous latent states outside the autoregressive sequence. Compared with agentic methods, ATLAS provides a compact and efficient interface that internalizes complex code generation, tool calling, and external execution into a single token. Compared with latent methods, ATLAS maintains a standard autoregressive generation loop without any visual supervision, preserving compatibility with existing supervised fine-tuning (SFT) and reinforcement learning (RL) frameworks, enabling efficient parallel training with scalability to larger-size models and data. It is also worth noting that these functional tokens do not require image-level supervision. Instead, they are optimized with the standard cross-entropy (CE) objective over token sequences, allowing the model to learn from the reasoning context by iteself when and how to invoke them as effective visual operations. We adopt a two-stage training recipe for ATLAS. First, to provide a reliable cold start for using functional tokens, we curate a new dataset, ATLAS-178K, covering over 40 visual reasoning tasks collected and reformulated from existing efforts Qiao et al. (2025b). Each example is annotated with functional-token trajectories that specify the desired visual operations, enabling the model to learn when and how to invoke functional tokens within standard autoregressive generation. On top of this, we apply RL to enhance visual reasoning through outcome-driven optimization. Thanks to our designs that functional tokens are represented as ordinary vocabulary tokens, ATLAS can be optimized directly with standard GRPO Shao et al. (2024b), without introducing customized training modifications Liu et al. (2025a); Xue et al. (2025). We leverage a diverse reward ensemble that jointly encourages answer correctness, valid functional-token usage, and coherent reasoning behavior, which already yields improvements over the SFT model. However, during RL training, we observe a critical “gradient dilution” issue: the sparse functional tokens responsible for visual reasoning are overwhelmed by the much larger number of ordinary text tokens, leading to insufficient optimization. To mitigate this, we introduce Latent-Anchored GRPO (LA-GRPO), which augments the standard GRPO objective with a statically weighted token-level auxiliary loss anchored on the functional-token vocabulary. This auxiliary objective provides a persistent learning signal for functional tokens, yielding consistent performance gains across reasoning tasks. Our contributions are summarized as follows: • We propose ATLAS, a visual reasoning framework that represents visual operations as discrete functional tokens in the standard vocabulary, avoiding verbose intermediate visual states, while preserving compatibility with scalable autoregressive training. • We identify gradient dilution for sparse functional tokens during training and propose LA-GRPO, a token-anchored objective that strengthens functional-token optimization. • We show that ATLAS enables compact single-token visual reasoning, achieving strong performance on challenging benchmarks with substantially reduced overhead.
2
ATLAS
In this section, we present ATLAS, a framework that bridges agentic and latent visual reasoning through discrete functional tokens. We first introduce the overall model architecture in Sec. 2.1, including the design of functional tokens within the autoregressive sequence. We then describe the training paradigm in Sec. 2.2, which consists of an SFT on the curated ATLAS-178K dataset followed by a standard RL with GRPO Shao et al. (2024b). Finally, in Sec. 2.3, we present the proposed LA-GRPO objective for enhanced functional-token optimization.
3
Text Token
Functional Token
Question: How much will it cost to fence the perimeter at £37/m?
Special Token
Imagined Reasoning Image
ATLAS:
Tokenizer Vocabulary I image the
<|im_start|> assistant
will
𝑉𝑡𝑒𝑥𝑡
line …
Input Image:
point
I
will
first
add
auxiliary
lines
G
<|Line|>
<|im_start|>
Sample ,
I
will
label
point
G <|Text|> …
<|image_pad|>
<|Manip|> 2m
I
will
render length
of
𝑉𝑠𝑝𝑒𝑐
…
Then
<|im_end|>
<|vision_start|>
G
BC <|Text|> …
<|Line|> <|Arrow|>
𝑉𝑓𝑢𝑛𝑐
<|Shape|> <|Text|>
Figure 2 Overall Pipeline of ATLAS. ATLAS represents visual operations as functional tokens within the standard
autoregressive sequence, enabling the model to perform visual reasoning without generating intermediate images or invoking external tools.
2.1
Model Architecture
Building upon standard autoregressive architectures Bai et al. (2025b); Liu et al. (2024); Bai et al. (2025a), ATLAS formulates visual reasoning as next-token prediction by representing visual operations as discrete learnable functional tokens in the tokenizer vocabulary. We instantiate ATLAS with Qwen2.5-VL Bai et al. (2025b) and add five functional tokens, each corresponding to an internalized operation. Generated like ordinary words within the same autoregressive sequence, these tokens provide a compact and interpretable interface for active perception and visual construction, while avoiding external tool execution, pixel-level intermediate supervision, and recurrent latent dependencies. This preserves compatibility with existing VLM pipelines and supports efficient parallel training. Taxonomy of Functional Tokens. To internalize visual operations into the reasoning process, we expand the standard vocabulary V with a compact set of functional tokens. Formally, the full vocabulary is defined as V = Vtext ∪ Vspec ∪ Vf unc , where Vtext denotes natural language tokens, Vspec denotes the original special tokens of the VLM (e.g., <im_start>, <image_pad>), and Vf unc = {<|Manip|>, <|Shape|>, <|Line|>, <|Arrow|>, <|Text|>} denotes the five proposed functional tokens. We intentionally keep Vf unc compact to avoid excessive perturbation to the original token distribution of the base model. Instead of introducing many task-specific tokens, we abstract common visual operations into a small set of general categories. For instance, bounding boxes, masks, cropping, and zooming can all be represented by the generalized region-based token <|Shape|>. As summarized in Tab. 1, each functional token corresponds to a high-level visual operation that can support multi-step reasoning. This taxonomy is not intended to be exhaustive. Rather, it provides a simple and effective template for internalizing visual operations as discrete tokens. Future work can naturally extend the functional-token vocabulary to cover more diverse operations and scenarios. Unified Sequence Modeling. Unlike agentic approaches that pause generation to call external modules, or latent methods that produce continuous hidden embeddings, ATLAS keeps the entire reasoning process within a single discrete autoregressive sequence. Given a multimodal input context x, the model predicts an output sequence: y = {y1 , . . . , yT }, where yt ∈ V = Vtext ∪ Vspec ∪ Vf unc When a functional token yt ∈ Vf unc is predicted, it is treated as an ordinary sequence token while serving as an internal reasoning unit that specifies the type of visual operation needed at the current step. For 4
Table 1 Taxonomy of Functional Tokens. The imagined reasoning images illustrate the internal visual reasoning states, which will not be generated or supervised in the sequence. Functional Token
Description
Imagined Reasoning Image
<|Manip|>
Enhances image through denoising, sharpening, filtering, cropping, or zoom-in based inspection.
<|Shape|>
Marks regions for visual grounding, highlighting, or area reasoning.
<|Line|>
Adds lines for geometric construction, visual separation, structural cues, or emphasis.
<|Arrow|>
Indicates direction, motion, force, flow, causal relation, or highlights key visual elements.
<|Text|>
Adds symbolic labels, numerical values, or textual notes to support multi-step reasoning.
example, <|Line|> indicates that the model should reason with an auxiliary line, while <|Text|> indicates that symbolic labels or numerical annotations may be useful for the subsequent derivation. This formulation preserves the explicitness and interpretability of agentic reasoning, while avoiding the latency of tool execution and the cost of pixel-level image generation. Importantly, functional tokens do not require any image-level supervision. Instead, they are optimized with the same cross-entropy (CE) objective as ordinary text tokens: X Lf unc = − log pθ (yt | x, y<t ). yt ∈Vf unc
Through token-level supervision, the model learns from the surrounding reasoning context when and how to invoke functional tokens as effective visual operations. For example, when the reasoning context states, “Now I will add an auxiliary height to ...”, the next functional token can be <|Line|>, encouraging the model to associate such geometric construction intent with the corresponding functional token. Since all reasoning units remain within the autoregressive sequence, ATLAS is fully compatible with scalable next-token training and inference pipelines.
2.2
Two-stage Training Recipe
We train ATLAS in two stages. First, we curate ATLAS-178K, an SFT dataset tailored to our visual reasoning paradigm with functional tokens. This provide a cold start for functional-token invocation and improved interleaved visual reasoning. Second, we apply standard GRPO for RL, further enhancing reasoning performance through reward-guided optimization. Stage 1: SFT with ATLAS-178K. We construct ATLAS-178K to provide supervised reasoning trajectories for the SFT stage. Specifically, it is constructed through the following three steps: 1. Source Data and Token Extraction: We start from the publicly released preview subset of V-Interaction400K Qiao et al. (2025b), which provides image-construction code paired with visual reasoning problems, making it suitable for deriving functional-token supervision. We parse the original code and extract visual operations that can be naturally mapped to our functional-token space, including line drawing, text annotation, shape drawing, visual refinement, cropping, and other visually grounded transformations. We then filter the extracted samples and retain 138K high-quality examples covering over 40 tasks for functional-token trajectory construction.
5
2. Trajectory Construction and Polishing: After extracting the mapped operations, we convert them into reasoning trajectories with functional tokens. For each functional step, we insert a predefined transition template so that the functional token appears as an explicit part of the reasoning process. Since directly templated trajectories can be overly rigid, we further use Gemini-2.5-Pro Team et al. (2024) to polish them into more natural reasoning text while preserving the original semantics and functional-token order. 3. Perception Preservation: To preserve the model’s low-level perceptual ability, we also include VPerception-40K Qiao et al. (2025b) during SFT. This part of data does not contain functional tokens, but provides complementary supervision for fine-grained visual understanding and helps reduce catastrophic forgetting during fine-tuning. With this dataset, we train the model using the vanilla CE loss Mao et al. (2023), updating all tokens in the sequence and enabling the model to learn valid functional-token invocation from context. Stage 2: Standard RL with GRPO. While SFT provides a cold start for functional-token usage, complex multi-step reasoning further requires the model to decide when such operations are useful for reaching the correct answer. Thanks to the compatibility of ATLAS with standard autoregressive generation, we can directly adopt GRPO without introducing customized training procedures. Given a query q, the policy πθ samples a group of G outputs {o1 , . . . , oG }. We define a composite reward r(o) that encourages answer correctness, effective functional-token usage, and valid formatting, while penalizing overly long responses and excessive token invocation: r(o) = λacc racc + λfunc rfunc + λfmt rfmt − λlen plen − λspam pspam , where each λ controls the corresponding reward or penalty term defined as follows: • Answer Accuracy (racc ): Evaluates whether the final answer is correct. We use exact string matching and mathematical equivalence checking when applicable. The reward is 1 for a correct answer and 0 otherwise. • Functional Token Usage (rfunc ): To encourage the meaningful usage of functional tokens and prevent reward hacking, we implement a strict conditional reward mechanism. The functional-token reward is granted only if the model invokes at least one functional token and successfully get the correct final answer. • Format Adherence (rfmt ): Ensures that the final answer follows the required format for reliable parsing. It yields 1 if the required format is satisfied and 0 otherwise. • Length Penalty (plen ): Discourages overly verbose responses. If the output length L(o) exceeds a predefined threshold Lmax , we apply a linear penalty within a fixed buffer range, capped by a maximum penalty value. • Token Overuse Penalty (pspam ): Prevents the model from repeatedly generating functional tokens only to exploit the usage reward. Let Nf unc denote the number of functional tokens in the output. If Nf unc exceeds a threshold τspam , we apply a bounded linear penalty for excessive usage. This reward design reflects a simple principle: functional tokens should be encouraged only when they support effective reasoning. The standard GRPO objective then optimizes the policy according to the relative advantage within the sampled group: β G 1 X πθ (oi | q) Âi − βDKL (πθ ||πref ), LGRP O (θ) = − G i=1 πref (oi | q) where Âi is the advantage computed from the group rewards r(oi ), and β controls the KL penalty.
2.3
LA-GRPO
Directly applying standard GRPO to ATLAS suffers from a “gradient dilution” issue. As illustrated in Fig. 3, standard GRPO assigns a sequence-level advantage to each rollout response and propagates this signal 6
Standard GRPO for ATLAS 𝐺
ℒ𝐺𝑅𝑃𝑂 𝜃 = −
LA-GRPO for ATLAS ℒ𝐿𝐴−𝐺𝑅𝑃𝑂 = ℒ𝐺𝑅𝑃𝑂 + 𝛼
𝑖 − 𝛽𝔻𝕂𝕃 (𝜋𝜃 | 𝜋𝑟𝑒𝑓 𝐴
𝑖=1
Rollout Group
Advantage
…
…
…
…
<|Line|>
<|Shape|>
+ 0 -
…
…
𝐴2 < 0
…
…
Low Advantage
Vspec
𝐴1 > 0
<|Line|>
𝐴2 < 0
<|Shape|>
…
Advantages on Vocabulary
High Advantage
Vtext
Vfunc
<|Manip|> <|Line|> <|Arrow|> <|Shape|> <|Text|>
+ 0 -
…
Vtext
𝐴i
𝑡
ℒ𝑡𝑜𝑘𝑒𝑛
Advantage
𝐴1 > 0
𝐴𝐺 > 0
<|Text|>
𝑡∈𝑀𝑓𝑢𝑛𝑐
…
Advantages on Vocabulary
…
𝑀𝑓𝑢𝑛𝑐
Rollout Group
…
…
…
1
𝛽
1 𝜋𝜃 𝑜𝑖 𝑞 𝐺 𝜋𝑟𝑒𝑓 𝑜𝑖 𝑞
…
𝐴i
<|Text|>
𝐴𝐺 > 0
Low Advantage
High Advantage
Vspec
Vfunc
<|Manip|> <|Line|> <|Arrow|> <|Shape|> <|Text|>
Weak Gradient Updates on Vfunc !
✓ Stronger Gradient Update on Vfunc
Figure 3 Latent-Anchored GRPO. Standard GRPO provides sequence-level advantages to all generated tokens, which can
dilute the learning signal for sparse functional tokens. LA-GRPO adds a token-level auxiliary objective on Vf unc to stabilize functional-token optimization.
to all generated tokens. However, functional tokens occupy only a very small portion of the sequence. In ATLAS trajectories, an average response contains 203.7 generated tokens, but only 4.8 of them are functional tokens, corresponding to a ratio of 2.3%. As a result, the learning signal for these sparse but important visual-operation tokens is easily diluted by the much larger number of ordinary text tokens. This weakens updates on Vf unc and may cause the model to underuse functional tokens or learn unstable behaviors such as token spamming. To address this issue, we propose Latent-Anchored GRPO (LA-GRPO). The key idea is to keep the original sequence-level GRPO objective unchanged, while adding a functional-token anchor that explicitly strengthens optimization on Vf unc . Concretely, for each sampled rollout, we identify the positions where functional tokens appear and apply an additional token-level auxiliary objective only to these positions. This auxiliary term reuses the rollout advantage from GRPO, but concentrates the update on functional tokens such as <|Line|>, <|Shape|>, and <|Text|>. In this way, LA-GRPO preserves the global reward-driven optimization of standard GRPO while providing a stronger and more persistent learning signal for the tokens responsible for internalized visual operations. Specifically, for each rollout oi = {y1 , . . . , yT }, we collect the positions of functional tokens: Mfunc (oi ) = {t | yt ∈ Vf unc }. For each t ∈ Mfunc (oi ), we define a token-level clipped surrogate loss: (t) Ltoken = − min ρi,t Âi , clip(ρi,t , 1 − ϵ, 1 + ϵ)Âi ,
ρi,t =
πθ (yt | q, y<t ) . πθold (yt | q, y<t )
This objective anchors the group-level advantage directly to functional-token positions, producing stronger
7
Table 2 Mapping from Code Operations to Functional Tokens. We parse visual operations from image-construction code
and map them into a compact functional-token space. Functional Token
Representative Code Patterns
<|Manip|>
np.pad(...) cv2.blur(...) cv2.GaussianBlur(...) scipy.signal.convolve(...) cv2.filter2D(...)
<|Line|>
plt.plot([x1, x2], [y1, y2], ...) ax.plot([x1, x2], [y1, y2], ...) cv2.line(img, pt1, pt2, ...)
<|Arrow|>
plt.arrow(x, y, dx, dy, ...) ax.arrow(x, y, dx, dy, ...) cv2.arrowedLine(img, pt1, pt2, ...)
<|Shape|>
plt.fill(x, y, ...) ax.add_patch(Circle(...)) ax.add_patch(Rectangle(...)) cv2.rectangle(img, pt1, pt2, ...) cv2.polylines(img, pts, ...) img[y1:y2, x1:x2] PIL.Image.crop((x1, y1, x2, y2)) cv2.resize(crop, ...) torchvision.transforms.Resize(...)
<|Text|>
plt.text(x, y, s, ...) ax.text(x, y, s, ...) cv2.putText(img, text, org, ...)
updates on sparse visual-operation tokens. Then, the final objective is: X 1 (t) LLA-GRPO = LGRPO + α Ltoken , |Mfunc | t∈Mfunc
where α controls the anchor strength and the normalization stabilizes the auxiliary loss scale. Unlike standard GRPO, which spreads the advantage signal across the whole vocabulary, LA-GRPO explicitly reinforces Vf unc while retaining sequence-level reward optimization. This yields stronger gradients for functional tokens, stabilizes their invocation, and improves visual reasoning without modifying the autoregressive training pipeline.
3
Experiments
3.1
Implementation Details
Training Details. We adopt Qwen2.5-VL-7B Bai et al. (2025b) as our base model. During the SFT stage, we freeze the vision encoder and solely update the visual projector and the language model. For the RL stage, we continue to freeze the vision encoder while optimizing the aligner and language model for 1 epoch. Training Data Details. We parse the original code and extract visual actions that can be naturally mapped to our functional-token space. The detailed correspondences between code actions and functional tokens are summarized in Tab. 2. Based on that, for each functional step, we insert a predefined transition template so that the functional token appears as an explicit part of the reasoning process. After enhancing by Gemini2.5-Pro Comanici et al. (2025), the resulting SFT data contains functional-token reasoning trajectories with 8
Table 3 Performance Comparison on Visual Reasoning. We compare five groups of VLMs with our ATLAS models on
three challenging benchmarks. Method
V*
BLINK
WeMath Avg.
Art.
Count.
Forensic.
IQ
Jigsaw
M-view.
Spatial
49.2 59.2 55.0 78.3
79.5 35.6 30.3 89.4
31.3 30.0 25.3 43.3
55.3 53.3 48.7 85.3
59.4 47.4 43.6 50.4
69.2 62.2 58.0 90.2
58.3 43.3 13.3 37.5
0.8 25.0 24.2 21.2
18.7 20.7 20.3 16.0
31.3 38.7 34.7 33.3
0.0 33.8 48.9 31.6
20.3 47.6 44.8 42.7
25.0 60.8
11.7 37.1
14.3 30.0
2.0 57.3
3.0 39.8
27.3 69.2
25.0 19.7 32.9
20.7 18.7 28.7
52.7 42.0 56.1
44.4 51.1 49.4
63.6 43.4 67.2
Closed-source Models GPT-4o OpenAI (2024) Claude-4-Sonnet Anthropic (2025) Gemini-2.0-Flash Pichai et al. (2024) Gemini-2.5-Pro Comanici et al. (2025)
62.8 15.2 73.3 79.1
50.6 63.0 47.4 71.3
61.0 49.9 45.3 74.6
82.9 61.5 56.4 85.5
Standard VLMs Qwen2.5-VL Bai et al. (2025b) LLaVA-OneVision-7B An et al. (2025) MiniGPT-v2 Lin et al. (2023) Gemma-3-27B Team et al. (2025)
70.2 75.4 35.6 62.3
36.2 23.1 11.0 31.7
22.8 36.6 32.8 32.1
29.9 47.0 43.6 42.7
Unified Models Anole Chern et al. (2024) Bagel Deng et al. (2025)
25.4 55.5
24.7 39.4
Visual CoT Shao et al. (2024a) V-Thinker Qiao et al. (2025b) VTS-V Bai et al. (2025c)
44.5 41.4 74.9
28.6 32.5 42.8
16.4 51.1
31.6 63.2
Agentic Visual Models 44.4 35.0 51.2
47.0 26.9 62.4
57.5 43.3 61.7
Latent Visual Models LVR Li et al. (2025a) MCOT Zhang et al. (2023) CoVT Qin et al. (2025) Monet Wang et al. (2025b)
77.5 76.4 72.8 77.8
41.2 39.6 38.1 36.9
49.4 47.4 47.9 41.8
59.0 55.6 59.0 41.0
60.0 57.5 60.0 56.7
35.6 33.3 36.4 22.7
25.3 26.7 24.0 28.0
52.7 50.7 41.3 45.3
48.1 45.9 49.6 40.5
65.0 62.2 65.0 58.7
26.0 26.0 26.3
54.7 57.7
48.1 43.6
70.6
51.3
53.4
62.9
Ours ATLASSFT ATLASGRPO
77.9
28.9 40.3
46.0 50.5
50.4 57.3
59.2 61.7
26.5 34.1
ATLASLA-GRPO
75.4
45.0
51.3
65.0
62.5
37.9
77.5
57.3
logically consistent steps, natural transitions, and correct final answers. For GRPO and LA-GRPO, we use We-Math 2.0 (Qiao et al., 2025a), MMK12 (Meng et al., 2025), and ThinkLite (Wang et al., 2025c). These datasets cover different visual reasoning scenarios and provide diverse supervision for reinforcement learning. Evaluation Datasets and Metrics. We evaluate ATLAS across a suite of benchmarks, including V* Wu and Xie (2024), BLINK Fu et al. (2024), and WeMath Qiao et al. (2025a). We adopt a rigorous automated judging pipeline, where we first employ rule-based scripts to parse the answer from the model outputs and then utilize Qwen3-VL-235B-A22B-Instruct Bai et al. (2025a) as a deterministic LLM-as-a-judge to verify the correctness of the answer and the format.
3.2
Quantitative Analysis
The performance comparison across visual reasoning benchmarks is summarized in Tab. 3. Compared with the base model Qwen2.5-VL Bai et al. (2025b), ATLAS brings clear improvements across all benchmarks. The improvement is notable on BLINK, where Qwen2.5-VL achieves an average accuracy of 22.8%, while ATLASLA-GRPO reaches 51.3%. This shows that discrete vocabulary functional tokens can effectively enhance structured visual reasoning. Among our variants, ATLASSFT already improves the BLINK average from 22.8% to 46.0%, demonstrating that supervised fine-tuning provides a strong reasoning capability. ATLASGRPO further improves the overall 9
Question: How many of the letters printed on the climber's
Response from ATLAS: Imagined Reasoning Image
The climber is carrying a snowboard with some text on the base, but the distance and the
snowboard are consonants?
snowy background make it a bit hard to read. First, I’ll mark the area containing the logo to
A. 2 B. 5 C. 4 D. 3
focus the model's attention on the text.
Input Image:
<|Shape|> The text is still somewhat faint due to the lighting. To improve the legibility of the Imagined Reasoning Image
characters before I count them, I’ll apply a filter to enhance the local contrast of the letters. <|Manip|> Now, the text is clearly visible as "JONES". Let's break down the word: J: Consonant; O: Vowel; N: Consonant; E: Vowel; S: Consonant There are 3 consonants (J, N, S) in total. Therefore, the correct answer is D.
Question: How many people
are reflected in the mirror?
Response from ATLAS: Imagined Reasoning Image
To determine how many people are reflected in the mirror, I must first distinguish between the multiple reflections to avoid double-counting individuals seen from different angles. I
A. 2 B. 1 C. 3 D. 4
will outline the mirrored doorway area with a supporting shape to localize the relevant
Input Image:
visual information. <|Shape|> Within this focused view, I can identify the figures standing in the hallway. I will add textual Imagined Reasoning Image
1
2
annotations to the reflected individuals to track the variables and ensure an accurate count. <|Text|> By labeling the distinct figures, I find there are two people visible in the mirror's reflection. Therefore, the answer is A.
Imagined Reasoning Image
Question: How many plates on
Response from ATLAS:
the counter?
A: To determine the number of plates on the counter, I will first focus on the main
A. 4 B. 5 C. 3 D. 6
preparation area where several dishes are arranged for breading.
Input Image:
<|Shape|> Imagined Reasoning Image
In this region, I can identify three white plates. I also need to check the remaining counter space to see if any other plates are in use. <|Arrow|> I notice one additional plate next to the stove holding the finished food. To ensure an
Imagined Reasoning Image
accurate count, I will label these four distinct plates. <|Text|>
4
3 2
The total count of plates on the counter is four. Therefore, the answer is (A).
1
Figure 4 Qualitative Examples of ATLAS. The cloud-shaped regions in the middle illustrate the model’s imagined reasoning
states for better understanding.
performance, reaching 50.5% on BLINK and 40.3% on WeMath. It brings large gains on several subsets, such as Jigsaw and Spatial Relation, but the improvements are not uniform. For example, it drops on IQ and multi-view reasoning compared with ATLASSFT . This suggests that sequence-level preference optimization can improve final-answer accuracy, but may also introduce unstable functional-token behavior on structured reasoning tasks. With LA-GRPO, ATLAS achieves the best performance among our variants on WeMath and BLINK average, reaching 45.0% and 51.3%, respectively. It also improves over standard GRPO on several subsets, including Art Style, Counting, Forensic Detection, and Multi-view Reasoning, with multi-view reasoning increasing from 43.6% to 53.4%. Compared with existing visual reasoning methods, ATLASLA-GRPO obtains consistently competitive results. Although standard GRPO performs better on a few metrics such as V*, Jigsaw, and Spatial Relation, LA-GRPO gives a more balanced result across the benchmark. These results indicate that anchoring the optimization on functional tokens helps stabilize training and improves the overall effectiveness of ATLAS on complex visual reasoning tasks.
10
Question: The images are frames from a video. The first image is from the beginning of the video and the second image is from the end. Is the camera moving left or right when shooting the video?
(A) Left
(B) Right
Input Image:
Response from ATLAS: To determine the camera movement, I will analyze the perspective shift and the visibility of the bottle label. I will first mark the bottle's front label and the background price tag as reference points. <|Shape|> As the video progresses, the view of the bottle's label rotates to reveal more of its right side, while the bottle itself shifts leftward relative to the background price tag. I will indicate this change in perspective with a directional marker. <|Arrow|> This parallax effect, where the foreground subject moves left against the distant background, combined
1st Image
2nd Image
with the changing angle of the text, confirms the camera is translating to the right while rotating to keep the object in view. Therefore, the answer is B.
Question: Which image is most similar to the reference image? (A) The second image (B) The third image Input Image:
Response from ATLAS: The crabs in all images appear identical, so the primary distinguishing factor is the background
environment. The reference image shows a crab on a dark, textured surface. I will highlight this background to serve as the baseline for comparison.
<|Shape|> The second image features a crab on a reflective water surface, which is a clear mismatch. I will point to
the water to indicate this environmental difference. Reference Image
2nd Image
<|Arrow|>
The third image has a dark background that appears to be wood. To confirm the material, I will improve the perceptual clarity to see the grain more clearly.
<|Filter|> The enhanced view shows a textured wooden grain that matches the reference.
Therefore, the third image is the most similar. The answer is B. 3rd Image
Question: Which image is most similar to the reference image? (A) The second image (B) The third image Input Image
Response from ATLAS: All three images feature green lizards with similar skin textures. The reference image focuses specifically on a high-resolution view of the head and eye. I will draw a shape to highlight the head region in the reference to establish the focus and scale. <|Shape|> The second image provides a full-body perspective on a dirt surface, which deviates from the close-up composition of the reference. I will point to the ground in the second image to indicate this environmental
Reference Image
2nd Image
difference.
<|Arrow|> The third image is also a close-up of a lizard's head, showing similar scale patterns. To better compare the fine details, I will improve the perceptual clarity of the eye and surrounding scales.
<|Filter|> The enhanced view confirms that the third image shares the same close-up framing and detailed scale structure as the reference.
Therefore, the third image is the most similar. The answer is B. 3rd Image
Figure 5 Qualitative Examples of ATLAS. We show more visual reasoning examples where functional tokens help localize
relevant regions, indicate directions, refine visual evidence, and support multi-step reasoning.
3.3
Qualitative Analysis
Qualitative Examples. Fig. 4 and Fig. 5 show representative visual reasoning trajectories from ATLAS. As shown, the model invokes functional tokens at meaningful steps: <|Shape|> is used to localize relevant 11
Question: As shown in the figure, quadrilateral ABCD is a trapezoid with an area of 48
Input Image
cm². Its lower base is three times the length of its upper base, and the length of AD is
① Image Attention
② Image Attention
Around <|Line|>
Around <|Text|>
Highlight Height AF
Annotate Base Lengths
as shown in the figure. What is the height AF of the trapezoid? (A) 6 (B) 7 (C) 8 (D) 9 (E) No correct answer Response from ATLAS: To find the height AF, I first need to make the perpendicular height explicit in the diagram. The relevant vertical segment is the dashed line from A down to F, so I highlight this height line as the quantity to solve for. ① <|Line|> Now I use the given base information. The upper base AD is marked as 6cm, and the lower base BC is three times the upper base. Therefore, AD = 3cm, BC = 3 * 3 = 9cm. I annotate these known lengths to keep the trapezoid area calculation clear. ② <|Text|> The area of a trapezoid is S = 1/2 (AD + BC) * AF. Substituting the known values, 48 = 1/2 (3 + 9) * AF. So, 48 = 6 * AF, and therefore AF = 8cm. Thus, the height of the trapezoid is 8cm. The answer is C.
Question: Is the car beneath the cat? (A) Yes (B) No
① Image Attention
Input Image
Around <|Shape|>
Response from ATLAS: To answer whether the car is beneath the cat, I should first localize the cat clearly, since it is the reference object in the question. The attention is centered on the cat, especially around its body and lower part, so I first mark the cat region. ① <|Shape|> With the cat region identified, I can compare the car’s position relative to it. The car appears in the upper-left background of the image, while the cat is sitting lower and more toward the center-bottom. Since the car is positioned above the cat rather than below it, the car is not beneath the cat. Therefore, the answer is B.
Mark Cat Region
Figure 6 Attention Analysis of Functional Tokens. The highlighted regions show that functional tokens tend to attend to
task-relevant visual evidence like geometric lines and target objects. Table 4 Efficiency Comparison. Efficiency metrics are reported as per-query averages. ATLAS substantially reduces
generation overhead and latency while improving accuracy. Method
All Tokens ↓
Code Tokens ↓
Func. Tokens ↓
Latency ↓
Peak Mem. ↓
Acc. ↑
V-Thinker (Qiao et al., 2025b) ATLAS
489.57 99.85
350.35 -
0.81
18.83s 3.80s
2.55GB 1.43GB
42.0 57.7
Relative Gain
4.90×
4.96×
1.78×
+15.7
434.3×
regions, <|Arrow|> guides attention to additional evidence, and <|Text|> supports counting and labeling. These examples show that functional tokens are not used as isolated markers, but are naturally integrated into the reasoning process to support visual reasoning. Attention Scores. We further visualize the image attention around functional tokens in Fig. 6 and Fig. 7. For each functional token, we average the attention scores between image tokens and the nearby 10 tokens around it. The resulting maps show that different functional tokens tend to focus on relevant visual regions, e.g., in Fig. 6, <|Line|> attends to the height segment in the geometry problem, and <|Shape|> highlights the cat region for spatial comparison. These patterns suggest that functional tokens are associated with meaningful visual evidence rather than being used only as textual markers.
3.4
Efficiency Analysis
We further evaluate the inference efficiency of ATLAS on BLINK-Jigsaw Fu et al. (2024) and compare it with V-Thinker Qiao et al. (2025b), which relies on explicit agentic reasoning. As shown in Tab. 4, ATLAS greatly reduces both output length and operation-formulation overhead. This is because each visual operation is represented by a single functional token, rather than a long textual description or tool-call formulation. This compact formulation directly improves inference efficiency. ATLAS reduces the average latency from 18.83s to 3.80s and lowers peak memory usage from 2.55GB to 1.43GB, while maintaining competitive accuracy. These results show that representing visual operations as compact functional tokens can reduce inference cost while preserving effective visual reasoning. 12
Question: As shown in the figure, a circle is drawn with center C, and points A and B are
Input Image
Image Attention Around <|Line|>
exactly on the circle. The lengths of BC, AC, and CD are as shown in the figure, and AB = CD. What is the measure of ∠ABC? (A) 45 (B) 60 (C) 72 (D) 90 (E) No correct answer Response from ATLAS:
To determine \angle ABC , I first focus on the key radius AC , since C is the center and both A and B lie on the circle. Drawing segment AC makes the radius structure explicit and helps compare the given equal
lengths.
Trace Radius AC
<|Line|>
Now we use the given information. Because C is the center, AC and BC are both radii, so AC = BC . The problem also states that AB = CD , and from the given lengths we have CD equal to the radius as well. Therefore, AB = AC = BC. So triangle ABC is equilateral, and each angle in the triangle is 60^\circ . Hence, \angle ABC = 60^\circ. The answer is B.
Question: How many "one way" signs are pointing to the left in the image? (A) 3 (B) 5 (C) 2 (D) 4
Image Attention
Input Image
Around <|Arrow|>
Response from ATLAS: To count the "ONE WAY" signs that point to the left, I first focus on the cluster of signs along the central pole, since that is where most of the relevant signs appear. To make the left-pointing signs more salient, I highlight the signs whose arrows point left with red arrows. <|Arrow|> With the left-pointing signs highlighted, I can count them clearly. There are two "ONE WAY" signs pointing to the left, both stacked along the central pole. The other visible "ONE WAY" signs point to the right and are not included. Therefore, the number of "ONE WAY" signs pointing to the left is 3. The answer is A.
Highlight Left Signs
Question: Is the car beneath the cat? (A) yes (B) no
Input Image
Image Attention Around <|Shape|>
Response from ATLAS: The question asks whether the cat is touching the wine glass, so I need to inspect the possible contact region rather than the whole image. I focus on the cat’s front paw and the lower part of the wine glass, especially the stem and base where contact would be most likely. <|Shape|> After marking this region, the cat’s paw is close to the glass, but it does not visibly touch the glass stem or base. The wine glass stands on the table separately, and there is a small gap between the paw and the glass. Therefore, the cat is not touching the wine glass.
Inspect Contact Region
Question: How many people are drinking water? (A) 1 (B) 3 (C) 0 (D) 2 Response from ATLAS:
Input Image
Since the question asks how many people are drinking water, I should not count people
① Image Attention
② Image Attention
Around <|Arrow|>
Around <|Filter|>
Focus on Hand Object
Check Background People
who are merely holding food or sitting near drinks. I first direct attention to the object in the woman’s hand, because this is the most likely candidate for a drinking action. ① <|Arrow|> After focusing on her hand, the object she is holding appears to be food rather than a water cup. She is not drinking water. Next, I need to check whether any background person is drinking, since there are distant figures near the counter that are less clear. ② <|Filter|> With the background area inspected more carefully, the distant people do not appear to be drinking water either. One person may be near the counter, but there is no clear action of drinking and no clear water cup being raised to the mouth. Therefore, there are zero people drinking water. The answer is C.
Figure 7 Attention Visualizations around Functional Tokens. The attention maps show that functional tokens tend to focus
on task-relevant visual evidence across different examples.
3.5
Ablation Study of Negative Reward Penalties
We ablate the format reward (rfmt ), length penalty (plen ), and token spam penalty (pspam ) to verify the necessity of each reward constraint. As shown in Table 5, removing any of these components leads to a performance drop on BLINK, indicating that the negative reward design is important for stable functionaltoken alignment. The full LA-GRPO objective achieves the best BLINK average accuracy of 51.3. When the format reward is removed, the model becomes less reliable in producing parseable final answers, leading to a drop of 1.3. Removing the length penalty causes responses to become unnecessarily verbose, which increases 13
Table 5 Ablation Study of Negative Reward Penalties in LA-GRPO on BLINK. Removing any component leads to lower
BLINK average accuracy.
rfmt
plen
pspam
Acc.
✓ ✓ ✓
✓ ✓ ✓
✓ ✓ ✓ -
51.3 50.0 49.2 47.0
generation cost and hurts reasoning accuracy. The largest degradation occurs when removing the token spam penalty, where the BLINK average drops from 51.3 to 47.0. In this setting, we observe severe reward hacking: the model generates up to 18.7 functional tokens per sequence merely to accumulate rfunc , rather than using them to support effective reasoning. Similarly, removing plen increases the average sequence length by 43.8%, resulting in higher compute cost without accuracy gains. These results show that the format, length, and token-spam constraints are complementary: they encourage ATLAS to invoke functional tokens only when useful, while keeping the reasoning trajectory concise, well formatted, and robust.
4
Related Work
Agentic Visual Reasoning. Another line of research equips language or multi-modal models with the ability to act through external tools. Representative examples include program-based and code-based systems such as VISPROG Gupta and Kembhavi (2023) and ViperGPT Surís et al. (2023), where the model generates executable programs to invoke specialized vision modules. More recent agentic frameworks broaden this idea by integrating richer tool ecosystems, hierarchical planning, and mixed-modality execution Zheng et al. (2025); Qiao et al. (2025b); Shao et al. (2024a); Wang et al. (2025a). Related work also explores visual workspaces such as sketchpads, where the model produces intermediate drawings, marks, or auxiliary constructions to support subsequent reasoning Hu et al. (2024). These methods are effective because they allow the model to actively manipulate visual inputs rather than passively observe them once. However, the underlying execution typically happens through external programs, APIs, or auxiliary environments. This creates a disjoint reasoning loop in which visual action is performed outside the standard autoregressive computation graph. Consequently, such approaches are usually non-differentiable end-to-end, incur latency from context switching and tool execution, and often require verbose code or program generation even for relatively simple visual operations. Latent Visual Reasoning. Latent reasoning offers a promising alternative by moving intermediate computation from explicit text into compact hidden representations Zhang et al. (2023); Hao et al. (2024). In language modeling, recent work explores this direction from several perspectives, including self-generated latent rationales Zelikman et al. (2024), continuous thought representations Hao et al. (2024), and recurrentdepth architectures that scale test-time computation without emitting long reasoning traces Zhao et al. (2025b). In multi-modal settings, methods such as Heima Shen et al. (2025) compress explicit reasoning into hidden thinking tokens, while more recent approaches introduce latent visual tokens or latent visual trajectories to support multi-modal reasoning without full image generation Li et al. (2025a); Qin et al. (2025); Wang et al. (2025b). These studies show that latent reasoning can improve efficiency and, in some cases, performance. Nevertheless, existing methods still face important limitations. Many rely on auxiliary supervision, reconstruction, or distillation targets for latent states Qin et al. (2025); Wang et al. (2025b), which restricts flexibility and may limit generalization beyond the training setup. More importantly, several approaches introduce recurrent or non-standard computation patterns, which deviate from the standard next-token prediction pipeline and reduce compatibility with highly optimized autoregressive parallel training systems. In contrast, our method formulates agentic visual actions as discrete functional tokens within the normal vocabulary space, keeping the entire reasoning process strictly inside the standard autoregressive loop. This preserves end-to-end differentiability while avoiding the recurrent dependencies and external execution overhead that limit prior approaches.
14
5
Conclusion
In this paper, we introduced ATLAS, a visual reasoning framework that represents visual operations as discrete functional tokens within the standard autoregressive vocabulary. By internalizing visual reasoning into compact tokens, ATLAS avoids intermediate image generation, external tool execution, and verbose operation formulations, while preserving interpretability and compatibility with parallel autoregressive training. We further identified gradient dilution for sparse functional tokens during GRPO training and proposed Latent-Anchored GRPO to stabilize their optimization. Extensive experiments show that ATLAS establishes a distinct visual reasoning paradigm, achieving strong performance on complex benchmarks with reduced inference latency and memory usage.
15
References Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, et al. Llava-onevision-1.5: Fully open framework for democratized multimodal training. arXiv preprint arXiv:2509.23661, 2025. Anthropic. System card: Claude opus 4 & claude sonnet 4. Technical report, Anthropic, May 2025. https: //www.anthropic.com/claude-4-system-card. Available at https://www.anthropic.com/claude-4-system-card. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025a. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025b. Tianyi Bai, Zengjie Hu, Fupeng Sun, Jiantao Qiu, Yizhen Jiang, Guangxin He, Bohan Zeng, Conghui He, Binhang Yuan, and Wentao Zhang. Multi-step visual reasoning with visual tokens scaling and verification. arXiv preprint arXiv:2506.07235, 2025c. Xinyan Chen, Renrui Zhang, Dongzhi Jiang, Aojun Zhou, Shilin Yan, Weifeng Lin, and Hongsheng Li. Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning. arXiv preprint arXiv:2506.05331, 2025. Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06135, 2024. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025. Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pages 148–166. Springer, 2024. Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14953–14962, 2023. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024. Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. Advances in Neural Information Processing Systems, 37:139348–139379, 2024. Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency. arXiv preprint arXiv:2502.09621, 2025. Bangzheng Li, Ximeng Sun, Jiang Liu, Ze Wang, Jialian Wu, Xiaodong Yu, Hao Chen, Emad Barsoum, Muhao Chen, and Zicheng Liu. Latent visual reasoning. arXiv preprint arXiv:2509.24251, 2025a. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought, 2025b. https://arxiv.org/abs/2501.07542. Baijiong Lin, Weisen Jiang, Feiyang Ye, Yu Zhang, Pengguang Chen, Ying-Cong Chen, Shu Liu, and James T Kwok. Dual-balancing for multi-task learning. arXiv preprint arXiv:2308.12029, 2023. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next. https: //llava-vl.github.io/blog/2024-01-30-llava-next/, 2024. Accessed: 2025-02-14. Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.05470, 2025a.
16
Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhaochong An, Fanny Yang, Aditya Patel, et al. Tuna: Taming unified visual representations for native unified multimodal models. arXiv preprint arXiv:2512.02014, 2025b. Anqi Mao, Mehryar Mohri, and Yutao Zhong. Cross-entropy loss functions: Theoretical analysis and applications. In International conference on Machine learning, pages 23803–23828. pmlr, 2023. Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365, 2025. OpenAI. Gpt-4o. https://openai.com/index/hello-gpt-4o/, 2024. Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Introducing Gemini 2.0: our new AI model for the agentic era. Google Blog, 2024. https://blog.google/innovation-and-ai/models-and-research/google-deepmind/ google-gemini-ai-update-december-2024/. Accessed: 2024-12-11. Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma Gongque, Shanglin Lei, Yifan Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning? In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20023–20070, 2025a. Runqi Qiao, Qiuna Tan, Minghan Yang, Guanting Dong, Peiqing Yang, Shiqiang Lang, Enhui Wan, Xiaowan Wang, Yida Xu, Lan Yang, et al. V-thinker: Interactive thinking with images. arXiv preprint arXiv:2511.04460, 2025b. Yiming Qin, Bomin Wei, Jiaxin Ge, Konstantinos Kallidromitis, Stephanie Fu, Trevor Darrell, and XuDong Wang. Chain-of-visual-thought: Teaching vlms to see and think better with continuous visual tokens. arXiv preprint arXiv:2511.19418, 2025. Bytedance Seed. Seed1. 8 model card: Towards generalized real-world agency. arXiv preprint arXiv:2603.20633, 2026. Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, 37:8612–8642, 2024a. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024b. Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking. arXiv preprint arXiv:2501.19201, 2025. Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505.15966, 2025. Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11888–11898, 2023. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimczak-Plucińska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri,
17
Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Põder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report, 2025. https://arxiv.org/abs/2503.19786. Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505.15966, 2025a. Qixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang, Pengfei Wan, Kun Gai, Xianghua Ying, and Yisen Wang. Monet: Reasoning in latent visual space beyond images and language. arXiv preprint arXiv:2511.21395, 2025b. Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934, 2025c. Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024. Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084–13094, 2024. Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07818, 2025. Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629, 2024. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? ECCV 2024, 2024. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023. Shanshan Zhao, Xinjie Zhang, Jintao Guo, Jiakui Hu, Lunhao Duan, Minghao Fu, Yong Xien Chng, Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, et al. Unified multimodal understanding and generation models: Advances, challenges, and opportunities. arXiv preprint arXiv:2505.02567, 2025a. Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint arXiv:2505.19590, 2025b. Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362, 2025.
18