arXiv CSopen access

Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs Sai Srinivas Kancheti* † IIT, Hyderabad

Aditya Sanjiv Kanade* Microsoft Research India

[email protected]

Vineeth N. Balasubramanian Microsoft Research India

Tanuja Ganu Microsoft Research India

[email protected]

arXiv:2604.16060v1 [cs.CV] 17 Apr 2026

Abstract Multimodal Reasoning Models (MRMs) leveraging Chain-of-Thought (CoT) based thinking have revolutionized mathematical and logical problem-solving. However, we show that this paradigm struggles with generalized spatial intelligence. We perform a comprehensive evaluation of seventeen models across thirteen spatial benchmarks and identify a critical gap: CoT prompting consistently degrades performance in visual spatial reasoning. Furthermore, through a novel No-Image++ ablation, we demonstrate that MRMs and CoT prompted MLMs suffer from severe shortcut learning, and hallucinate visual details from textual priors even when the image is absent. These findings challenge the efficacy of text-only CoT for spatial tasks and underscore the need for vision-centric reasoning paradigms.

Introduction

The emergence of "System 2" Multimodal Reasoning Models (MRMs) — models post-trained via SFT and RL to generate step-by-step reasoning — has driven remarkable progress in mathematical and logical domains. By leveraging Reinforcement Learning (RL) (Lambert et al., 2024; Guo et al., 2025) and long Chain-of-Thought (CoT) (Wei et al., 2022; Wang et al., 2022) inference, MRMs demonstrate the ability to self-correct and reason through complex problems. Separately, CoT prompting is a general technique that instructs any Multimodal Language Model (MLM) to think step-by-step before answering. However, a fundamental question remains: does this text-centric reasoning paradigm translate to spatial intelligence? Spatial reasoning requires grounding, geometric intuition, and precise localization, which are skills that may not easily arise from verbose, text-based reasoning (Tong et al., 2024a,b). * Equal contribution. †

Work done while at Microsoft Research India.

In this work, we conduct a comprehensive evaluation of seventeen models, including nine state-ofthe-art open-source MRMs (e.g., GThinker, VisionR1, ViGoRL, Qwen3-VL) and eight diverse backbone MLMs. We benchmark these models across thirteen datasets covering static 2D relations, 3D geometry, and dynamic/temporal understanding. To isolate the impact of CoT reasoning, we standardize our evaluation using a uniform evaluation and scoring policy. Our findings reveal that contrary to trends in other domains, CoT prompting degrades performance in visual spatial tasks. Our contributions are as follows: (i) We show that MRMs consistently underperform their own backbone on generalized spatial benchmarks. In our experiments, 7 out of 8 reasoning models failed to surpass the backbone they were distilled from. (ii) We demonstrate in Figure 1 that CoT prompting lowers accuracy by an average of 3% across a diverse range of MLMs. (iii) Through a novel No-Image++ ablation, we show that MRMs suffer from severe shortcut learning. When presented with a blank image and a “Cannot determine” option, reasoning models continue to hallucinate visual details and confidently select incorrect answers based solely on textual priors. These results suggest that simply scaling textbased reasoning is insufficient for robust spatial intelligence, highlighting the need for vision-centric training paradigms.

Methodology

We begin by describing the baseline models we consider, datasets and our evaluation scheme. Baselines: We rigorously benchmark the performance of seventeen models: i) Qwen2.5VL backbone: We evaluate three scales of Qwen2.5-VL-Instruct series (3B, 7B, 72B) (Bai et al., 2025b), eight SOTA General purpose visual reasoning Multimodal Reason-

CoT 62.52 46.88 60.68 60.99 63.26 58.86 61.11 62.61

Non-CoT 39.38 (-23.14%) 47.84 (+0.96%) 62.52 (+1.84%) 62.18 (+1.19%) 62.85 (-0.41%) 59.6 (+0.74%) 62.6 (+1.49%) 62.74 (+0.13%)

Accuracy (%)

Model GThinker R1-Ov ViGoRL VL-Re. Vision-G1 Vision-R1 TreeVGR ThinkLite

Prompting CoT Non-CoT

+2.61%

+3.00%

+2.57%

+4.48%

+3.09%

+0.58%

40 30

-3B

n2.5 Qwe

n2.5

Qwe

-7B

n2.5

Qwe

B -72

+1.15%

3-8

L rnV

Inte

+0.64%

.5-3

VL3

rn Inte

6-7

. VA1

LLa

72 OVVALLa

L-8Bg n3-Vinkin Qwe -Th

Figure 1: (Left) CoT vs Non-CoT performance of open-source MRMs. (Right) Bar chart showing the average accuracy of various families of MLMs over 13 benchmark datasets. For each model, the left bar shows the accuracy achieved by CoT prompting, and the right bar shows for base prompt (non-CoT). We observe CoT prompting drops performance over a wide range of backbones and model scales, including Qwen3-VL-8B-Thinking (Bai et al., 2025a), a model with explicitly enhanced spatial perception.

ing Models (MRMs) trained using RLVR (Shao et al., 2024) GThinker-7B (Zhan et al., 2025) (Jun’25), ViGoRL-7B-Spatial (Sarch et al., 2025) (May’25), Vision-G1-7B (Zha et al., 2025) (Aug’25), R1-Onevision-7B (Yang et al., 2025b) (Mar’25), VL-Rethinker-7B (Wang et al., 2025b) (May’25), Vision-R1 (Huang et al., 2025) (Mar’25), TreeVGR (Wang et al., 2025a) (Jul’25), and ThinkLite-7B (Wang et al., 2025d) (Apr’25); Qwen3-VL-8B-Thinking (Bai et al., 2025a), a model with explicitly enhanced spatial perception; ii) InternVL backbone: InternVL3-8B, InternVL3.5-38B (Wang et al., 2025c); iii) LLaVA backbone: LLaVA-v1.6-Mistral-7B, LLaVAOneVision-Qwen2-72B (Li et al., 2024); and finally a proprietary model GPT-4o (Hurst et al., 2024) for a total of seventeen diverse baselines covering both reasoning and non-reasoning MLMs at various scales. Open-source MRMs. We choose eight diverse top performing open-source MRMs (listed above) that are trained for general visual reasoning including spatial tasks. ViGoRL-spatial and TreeVGR are explicitly trained to perform spatial reasoning, while the remaining MRMs (with the exception of Vision-R1) contain spatial domains in their training data and are trained as general purpose visual reasoners. All MRMs considered are built atop the Qwen2.5-VL-7B-Instruct backbone, which combines strong visual capabilities with emergent reasoning abilities, and has been widely used as a capable backbone. A key motivation for our study, highlighted in Tab. 2, is that the original papers for these models evaluate on math heavy datasets which are not vision-centric. We hence comprehensively evaluate the reasoning capabilities of these models on spatial tasks.

Datasets: We evaluate on thirteen datasets covering various aspects of spatial reasoning, which can be broadly categorized into static 2D datasets, and 3D/dynamic datasets. The former datasets are usually confined to single images and focus on planar spatial relationships, usually from the camera’s perspective. We place BLINK (Fu et al., 2025), CV-Bench2D (Tong et al., 2024a), MMVP (Tong et al., 2024b), RealWorldQA (xAI, 2025), SpatialBench (Cai et al., 2025), VSR (Liu et al., 2023), and V*Bench (Wu and Xie, 2024) in this category. We also consider benchmarks that require reasoning involving 3D geometry, depth, multiimage consistency, and temporal reasoning. These datasets often involve understanding the 3D position and relative orientation from the object’s perspective inside the image. 3DSRBench (Ma et al., 2024), CV-Bench3D (Tong et al., 2024a), MindCube (Yin et al., 2025), MMSIBench (Yang et al., 2025a), OmniSpatial (Jia et al., 2025), and SAT-Real (Ray et al., 2025) belong to this category. We choose these datasets as they have real-world objects set in natural scenes, are difficult to answer on account of being vision-centric (Tong et al., 2024a), and cover a wide rage of spatial capabilities. A summary of the datasets along with the various spatial facets they test is provided in Appx. Tab. 5. The left half of the table describe the 2D datasets, while the right half describes the rest. Evaluation: To ensure uniform evaluation, we follow VLMEvalKit (Duan et al., 2024) to provide a uniform system prompt as well as a uniform question format. All benchmarks, are multiple-choice questions (MCQs) with options provided in the question prompt. The question format for all datasets is: Question:<question>\nOptions:\nA.

Models

3DSRBench

BLINK

CV-Bench 2D

MindCube

MMSIBench

MMVP

Qwen2.5-VL-7Bcot Qwen2.5-VL-7B

57.110.39 55.380.06

53.440.30 56.040.03

75.920.03 77.170.03

76.090.31 83.780.04

30.830.25 35.110.18

27.470.21 26.870.09

72.440.68 75.780.32

GThinker-7B R1-Onevision-7B ViGoRL-7B-Spatial VL-Rethinker-7B Vision-G1 Vision-R1-7B TreeVGR-7B ThinkLite-7B

56.580.20 48.520.20 55.840.20 56.990.11 55.910.01 55.010.20 51.530.03 57.260.13

54.760.17 43.270.50 52.510.26 54.600.23 54.600.08 46.470.66 53.160.25 57.130.08

77.400.06 53.310.04 76.590.27 76.060.12 76.700.15 71.580.12 76.240.09 76.890.18

82.950.04 58.000.42 86.140.10 80.750.14 83.750.14 75.830.31 75.170.13 80.440.16

40.160.32 27.090.42 39.360.16 37.810.27 38.100.31 36.950.59 44.250.66 30.130.23

27.330.31 13.300.30 25.870.17 26.900.08 26.070.12 22.900.36 27.170.33 27.970.56

73.780.42 56.160.16 73.220.42 75.890.16 76.560.16 72.220.32 71.330.27 73.560.16

Models

OmniSpatial

RealWorldQA

SAT

SpatialBench

VSR

V*Bench

Avg.

Qwen2.5-VL-7Bcot Qwen2.5-VL-7B

40.400.80 45.230.11

63.050.27 69.020.00

59.220.57 63.110.16

61.750.70 62.870.00

81.830.35 85.380.04

76.270.25 79.060.00

59.68 62.68

GThinker-7B R1-Onevision-7B ViGoRL-7B-Spatial VL-Rethinker-7B Vision-G1 Vision-R1 TreeVGR-7B ThinkLite-7B

47.680.14 31.540.10 36.970.21 39.840.26 46.880.21 39.750.16 47.290.38 45.360.43

68.670.06 49.870.46 65.670.55 68.500.39 69.760.16 67.410.06 67.580.11 69.370.06

58.440.16 51.500.83 58.440.68 65.000.98 62.670.00 58.450.32 62.110.79 66.440.42

60.070.15 50.190.18 58.650.49 61.570.15 64.930.00 60.510.08 60.260.16 62.190.32

83.770.04 72.500.32 82.080.07 84.400.08 86.550.08 79.950.18 74.800.07 86.850.04

81.150.00 54.190.79 77.490.74 64.571.08 79.930.25 78.180.25 83.600.49 80.280.25

62.52 46.88 60.68 60.99 63.26 58.86 61.11 62.61

Table 1: Accuracy of SOTA MRMs on 13 spatial benchmarks. The top two rows shows performance of the base model Qwen2.5-VL-7B, which is competitive with MRMs trained to perform multimodal reasoning. We identify that open-source MRMs do not exhibit generalized spatial intelligence beyond their base model.

Baseline GThinker-7B R1-Onevision-7B ViGoRL-7B-Spatial VL-Rethinker Vision-G1 Vision-R1

Paper-Reported Datasets MMStar, RWQA, MMMU-Pro MathVision, Mathvista, Mathverse SAT-Val, BLINK MathVision, MMMU-Pro, MEGA MathVista, MMMU-Pro, MMStar, ChartQA MathVista, MMStar, ChartQA, MMEsum

Table 2: A summary of the evaluation datasets used in each MRM’s paper. Most eval datasets are Math heavy, are not vision-centric, and do not cover many aspects of spatial reasoning.

<optA>\nB.<optionB> ...\nPlease select the correct answer (letter and option text) from the options above. We append dataset-specific prompts for OmniSpatial & MindCube to ensure good performance. The prompts are detailed in Appx. § A. Metric. We use vLLM (Kwon et al., 2023) version 0.10.0 for performant, batched inference of MLMs on 4 NVIDIA A100 GPUs. We use a batch size of 16, set max new tokens generated to be 32768, set model context length as 32768, and perform inference on bf loat16 precision. We use pass@1 accuracy under greedy decoding (with temperature set to 0) as our metric. All results are over 3 seeds. System Prompts. We evaluate models in two

settings, using a i) base prompt/non-CoT prompt such as “You are a spatial-reasoning assistant. The user asks a question, and the Assistant solves it.” or a ii) CoT prompt where we append “First output the thinking process in <think> </think> tags and then output the final answer in <answer> </answer> tags.” to the base prompt. For CoT evaluation of MRMs, we use the custom CoT prompt they train on (instead of default shown above) for best performance as shown in the table below. The custom Model Custom CoT Simple CoT GThinker-7B 62.52 59.57 CoT prompt Vision-G1 63.26 62.06 used for each MRM is shown in Appx § A. Scoring the generations. We use an LLM-asa-judge along with a carefully designed prompt, shown in Appx. § A.3, to score all generations. We pick Qwen3-30B-A3B-Instruct-2507, a small non-reasoning text model as our judge for scoring since our evaluation is on MCQs with short answers i.e the final answer of the model is not free-form but restricted to the options provided. To validate our choice, we re-score the generations of

−−−−→ Models

Random Qwen2.5cot Qwen2.5 GThinker R1-Ov ViGoRL VL-Re. Vision-G1 Vision-R1 TreeVGR ThinkLite

No-Image 38.83 No-Image++

37.45 43.4

38.59 76.41

44.17 5.55

28.1 11.22

43.18 30.95

41.26 47.73

44.46 25.28

41.15 7.29

41.91 11.35

42.48 36.00

Table 3: Results of two variant of the No-Image ablation, where the images are replaced with an uninformative full gray image. For No-Image, MRMs show much higher average performance, indicating their ability to shortcut an answer just from the question (random is better). For No-Image++ (higher better), where a “cannot determine” option is added, we find Qwen_cot as well as MRMs still choose other options as they are biased by the text trace.

Vision-G1 using GPT-4o as the judge, and compare it with our chosen judge. We observe a Cohen’s kappa score (Cohen, 1960) of > 0.99 indicating near-perfect agreement of judges.

Results and Analysis

(i) CoT Prompting Hurts Visual Spatial Reasoning. Contrary to trends in math and logic domains, we observe that CoT prompting frequently hurts performance in visual spatial tasks. Figure 1 (left) shows the performance of open-source MRMs under both CoT and non-CoT prompting. Surprisingly, these models, which have been explicitly trained via RL to reason, often perform better when this reasoning capability is suppressed. Six of the eight MRMs achieve higher accuracy with the nonCoT prompt than with their native CoT prompts. We observe that GThinker is not robust to changes in its prompt and struggles to adhere to the directanswer format of the base prompt. We qualitatively observe that it generates ill-formed CoT traces even when instructed otherwise, causing a significant drop (−23.14%) in performance (see Figure 2 for an example). Figure 1 (right) shows that this trend holds across three families of models (Qwen, InternVL & LLaVA), and across a range of model strengths (params ranging from 3B to 72B). We additionally evaluate Qwen3-VL-8B-Thinking (Bai et al., 2025a), a model with explicitly enhanced spatial perception, and observe that Non-CoT still outperforms CoT by +0.64% at a competitive baseline of ∼65% (dataset-wise results in Appx. Table 9). (ii) RL-trained Multimodal Reasoning Models Underperform their Backbone. We present the accuracies of eight open-source MRMs on 13 datasets in Tab. 1. The first two rows indicate backbone results, where cot indicates evaluation under CoT prompt. We observe that the non-CoT Qwen2.5-VL backbone shows strong average performance of 62.68%. Surprisingly, despite extensive SFT and RL training designed to enhance visual reasoning, seven out of the eight MRMs fail to surpass this baseline. Even models explicitly fine-

Model GPT-4o (Hurst et al., 2024) GPT-4.1-mini (OpenAI et al., 2024) GPT-5 (Singh et al., 2025) GPT-5-mini (Singh et al., 2025) GPT-5-nano (Singh et al., 2025)

Avg. CoT

Avg. Non-CoT

65.55 (+0.50) 67.79 (+0.39) 69.00 69.86 (+0.08) 60.63

65.05 67.40 69.65 (+0.65) 69.78 61.86 (+1.23)

Table 4: CoT vs Non-CoT performance of proprietary models. Non-CoT outperforms CoT for GPT-5 and GPT-5-nano.

tuned for spatial tasks ViGoRL (−2%) & TreeVGR (−1.57%) underperform the backbone. Vision-G1 is the only exception, and outperforms the backbone by +0.6%. However, we show in the next paragraph that Vision-G1 exhibits the strongest reliance on textual priors, suggesting its performance may stem partly from dataset shortcuts rather than grounded visual reasoning. (iii) Reasoning Models Show Over-reliance on Text Rationale. We highlight a crucial shortcoming of text-only CoT reasoning, where we observe an over-reliance on the text modality which leads to hallucination of visual content. We perform a NoImage ablation, where we pass an uninformative fully gray image (of the same size and aspect ratio as the original image) as input along with the question. The first row of Table 3 reports the average accuracy across all 13 datasets. We observe that MRMs perform significantly better than random guessing (e.g., GThinker achieves 44.17%), indicating they can "shortcut" the answer by ignoring visual content and relying solely on question text, options, and world-knowledge priors. To confirm this behavior stems from hallucination, we introduce the No-Image++ setting. Here, we maintain the gray image input but append a "Cannot determine from the image" option to the answer choices, which serves as the ground truth. As shown in the second row of Table 3, most MRMs perform poorly on this metric. Rather than acknowledging the missing visual information, they continue to hallucinate reasoning traces for the original options. We present qualitative examples of this failure mode in Figure 2.

Example 1: GThinker degenerate output under Non-CoT prompt Question (RWQA): “Which item is larger than the other?” Output: <tool_call>\n\n\n\n\n\n\n\n\n\n\n\n\n... (repeats until termination) Example 2: ViGoRL hallucination under No-Image++ (blank image) Question (CVBench2D): “Where is the cave located with respect to the trees?” Output: <think> The cave entrance appears to be at the bottom of the image, while the trees are higher up, suggesting that the cave entrance is below the trees. (300, 450) ...The cave entrance is clearly situated at the bottom of the image, which is below the trees. (250, 550) </think> <answer> below </answer> Note: The input image is a blank gray image. The model hallucinates coordinates and spatial relationships using world knowledge (caves are usually below trees).

Figure 2: Qualitative examples of failure modes. Top: GThinker produces degenerate output when prompted without CoT. Bottom: ViGoRL hallucinates detailed spatial reasoning for a blank image in the No-Image++ setting. Additional examples in Appx. C.

(iv) Analysis of Proprietary Models. To evaluate the generalizability of our findings to proprietary models, we benchmark five models from the GPT family. Table 4 shows that non-CoT performance remains competitive with or exceeds CoT performance across the board. Notably, GPT-5 and GPT-5-nano show CoT degradation (+0.65% and +1.23% for Non-CoT, respectively), mirroring the open-source trend. While GPT-4o and GPT-4.1mini show marginal CoT gains, these are small (< 0.5%) relative to the additional inference compute required for reasoning. We analyze the CoT traces of proprietary models and find two notable differences from opensource models: (i) Proprietary models produce significantly shorter traces (∼350 characters for GPT-5-mini vs. ∼3600 characters for Qwen3-VL8B-Thinking), and (ii) proprietary traces lack the reflective phrases (e.g., “wait”, “let me reconsider”) and repetitive looping commonly observed in open-source MRMs. We hypothesize that this conciseness helps proprietary models avoid the hallucination-inducing verbosity that harms opensource models, though their training details remain opaque. These observations suggest that the quality and conciseness of reasoning traces, rather than their mere presence, may be key to preserving spatial reasoning performance under CoT prompting.

Conclusion

In this work, we show that the success of reasoning models in logic and mathematics does not yet extend to the spatial domain. Our benchmarking of seventeen models across thirteen datasets reveals that Chain-of-Thought prompting consistently degrades spatial reasoning performance, with specialized MRMs frequently underperforming their own base models. Crucially, our No-Image++ analysis identifies the mechanism behind this failure: current reasoning chains tend to hallucinate visual information based on textual priors rather than engaging in grounded perception. Our analysis of proprietary models further suggests that concise, non-repetitive reasoning traces may mitigate this degradation. Our work highlights the need for vision-centric training paradigms for MRMs. Promising future directions include (i) test-time visual verifiers that evaluate each reasoning step against image evidence and trigger backtracking on incorrect visual claims, and (ii) visual process reward models that incentivize grounded, perceptionfirst reasoning during training.

Limitations In this work, we have sought to cover a broad range of visual spatial reasoning datasets and R1-style MRMs. However, we do not claim that the 13 datasets included represent the entirety of the visual spatial reasoning domain. Given the current landscape of MRMs, it is challenging to completely isolate all confounding factors that may lead to performance improvements or declines across these datasets. We note that proprietary model training details remain opaque, limiting deeper analysis of their behavior. We believe this study offers a solid foundation for future research to further explore vision-centric reasoning paradigms.

Acknowledgments References Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. Qwen3-vl technical report. Preprint, arXiv:2511.21631. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei

Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025b. Qwen2.5-vl technical report. ArXiv, abs/2502.13923. Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. 2025. Spatialbot: Precise spatial understanding with vision language models. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 9490–9498. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37 – 46. Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, and Jiaqi et al. Wang. 2024. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 11198–11201.

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. Llava-onevision: Easy visual task transfer. ArXiv, abs/2408.03326. Fangyu Liu, Guy Emerson, and Nigel Collier. 2023. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11:635–651. Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Alan Yuille, and Jieneng Chen. 2024. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.

Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, WeiChiu Ma, and Ranjay Krishna. 2025. Blink: Multimodal large language models can see but not perceive. In Computer Vision – ECCV 2024, pages 148–166, Cham. Springer Nature Switzerland.

Arijit Ray, Jiafei Duan, Ellis L Brown II, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A. Plummer, Ranjay Krishna, Kuo-Hao Zeng, and Kate Saenko. 2025. SAT: Dynamic spatial aptitude training for multimodal language models. In Second Conference on Language Modeling.

Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, and Ruoyu Zhang et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. ArXiv, abs/2501.12948.

Gabriel Sarch, Snigdha Saha, Naitik Khandelwal, Ayush Jain, Michael J. Tarr, Aviral Kumar, and Katerina Fragkiadaki. 2025. Grounded reinforcement learning for visual reasoning. ArXiv, abs/2505.23678.

Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaoshen Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. ArXiv, abs/2503.06749.

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, JunMei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv, abs/2402.03300.

OpenAI Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and Aleksander Mkadry et al. 2024. Gpt-4o system card. ArXiv, abs/2410.21276.

Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex BakerWhitcomb, Alex Beutel, Alex Karpenko, and 465 others. 2025. Openai gpt-5 system card. Preprint, arXiv:2601.03267.

Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. 2025. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXiv:2506.03135. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles. Nathan Lambert, Jacob Daniel Morrison, Valentina Pyatkin, Shengyi Huang, and Hamish Ivison et al. 2024. Tülu 3: Pushing frontiers in open language model post-training. ArXiv, abs/2411.15124.

Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. 2024a. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. In Advances in Neural Information Processing Systems, volume 37, pages 87310–87356. Curran Associates, Inc. Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024b. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9568–9578.

Haochen Wang, Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijiang Kang, Jiashi Feng, Zhuochen Wang, and Zhaoxiang Zhang. 2025a. Traceable evidence enhanced visual grounded reasoning: Evaluation and methodology. ArXiv, abs/2507.07999. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025b. Vlrethinker: Incentivizing self-reflection of visionlanguage models with reinforcement learning. ArXiv, abs/2504.08837. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, and Zhe Chen et al. 2025c. Internvl3.5: Advancing opensource multimodal models in versatility, reasoning, and efficiency. ArXiv, abs/2508.18265. Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Qinghong Lin, Furong Huang, and Lijuan Wang. 2025d. Sota with less: Mcts-guided sample selection for dataefficient visual reasoning self-improvement. ArXiv, abs/2504.07934. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. ArXiv, abs/2203.11171. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903. Penghao Wu and Saining Xie. 2024. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13084–13094. xAI. 2025. Grok-1.5 vision. https://huggingface. co/datasets/xai-org/RealworldQA. License = CC BY-ND 4.0. Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. 2025a. Mmsi-bench: A benchmark for multi-image spatial intelligence. ArXiv, abs/2505.23764. Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. 2025b. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. ArXiv, abs/2503.10615. Baiqiao Yin, Qineng Wang, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Manling Li, Jiajun Wu, and Li Fei-Fei. 2025. Spatial mental modeling from limited views. Preprint, arXiv:2506.21458.

Yuheng Zha, Kun Zhou, Yujia Wu, Yushu Wang, Jie Feng, Zhi Xu, Shibo Hao, Zhengzhong Liu, Eric P. Xing, and Zhiting Hu. 2025. Vision-g1: Towards general vision language reasoning with multi-domain data curation. ArXiv, abs/2508.12680. Yufei Zhan, Ziheng Wu, Yousong Zhu, Rongkun Xue, Ruipu Luo, Zhenghao Chen, Can Zhang, Yifan Li, Zhentao He, Zheming Yang, Ming Tang, Minghui Qiu, and Jinqiao Wang. 2025. Gthinker: Towards general multimodal reasoning via cue-guided rethinking. Preprint, arXiv:2506.01078.

Prompts

In this section we present the dataset prompts as well as different system prompts used in the baselines. A.1

System Prompts

Base prompt. This is the simple no-thinking prompt used by Qwen2.5-VL-7B. You are a spatial-reasoning assistant. The user asks a question, and the Assistant solves it. CoT prompts. We give the list of CoT system prompts we use to evaluate the MRM baselines. Below we present prompts used by GThinker, R1Onevision, ViGoRL-Spatial, VL-Rethinker, VisionG1, and Vision-R1 respectively. GThinker: A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>. In the reasoning process enclosed within <think> </think>, each specific visual cue is enclosed within <vcues_*>...</vcues_*>, where * indicates the index of the specific cue. Before concluding the final answer, pause for a quick consistency check: verify whether the visual cues support the reasoning and whether each step logically follows from what is seen. If correct, conclude the answer; otherwise, revise the visual cues and reasoning, then conclude. R1-Onevision: You are a spatial-reasoning assistant. The user asks a question, and the Assistant solves it. First output the thinking process in <think> </think> tags and then output the final answer in <answer> </answer> tags. ViGoRl-Spatial: A conversation between User and Assistant. The User asks a question, and the Assistant solves it.

The Assistant systematically reasons through the problem step by step by checking and verifying possible solutions and image regions, while grounding reasoning steps to specific objects and their relationships in the image using (x,y) coordinates. There may be one image or two images concatenated together, in which case the Assistant must compare the spatial relationships between the two images.\n\n All reasoning processes must be enclosed within a single set of ’<think>’ tags, and reasoning steps must include specific reference coordinates:\n\n For example, <think> {Reasoning text}. {Further reasoning text} {more reasoning} </think> The final answer should be enclosed in ’<answer>’ tags in the format: <answer> {text of selected answer choice} </answer>\n\n The Assistant must help the user identify the correct answer choice from the options provided.\n - If the correct answer is unclear, select the most relevant option based on the spatial relationships and dynamics within the image.\n - The Assistant should verify each step and check multiple possible solutions before selecting the final answer. VL-Rethinker: Please think step by step, and **regularly perform self-questioning, self-verification, self-correction to check your ongoing reasoning**, using connectives such as "Wait a moment", "Wait, does it seem right?", etc. Remember to put your final answer within . Vision-G1: You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think></think> tags. The final answer MUST BE put in . Vision-R1: A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process

and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>. A.2

Dataset Prompts

We use prompts from the respective papers for OmniSpatial, MindCube and Spatial457 as recommended. Task \n —– \n You will OmniSpatial: receive 1. **Image** - a single RGB frame depicting a scene. \n 2. **Question** a natural-language query about spatial relationships between objects in the image. \n 3. **Options** - >=2 answer candidates, each tagged by a capital letter (A, B, C, D. . . ).\n Based on the image and question, provide your answer. Always ground your answer in the visual evidence; do not hallucinate unseen objects. If uncertain, pick the most plausible option—never refuse or reply “insufficient information.” MindCube: Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints. A.3

LLM Judge Scoring Prompts

We show the prompts we use for judging MLM generations.

allowed set, or "0".\n - Do not output any explanation, spaces, punctuation, or additional text.\n\n Rules:\n 1) If the response explicitly names exactly one letter (patterns like "A", "A)", "Option A", "Answer is C"), return that letter immediately.\n 2) Only evaluate the explicitly provided choice. If the response is long and complex without an explicit final choice, return "0".\n 3) If multiple choices appear in the response, the last unambiguous one is the final choice.\n 4) Never judge factual correctness—only map the response to the best matching option letter from the given options.\n 5) If no explicit letter can be extracted from the response, compare the response’s meaning to option texts. If exactly one option clearly restates or is a synonym/number/name/unit match for the response, return its letter. (Example: response “1956” matches option “B. 1956”)\n 6) If the response uses standard MCQ phrases such as "none of the above" or "all of the above" and a matching option exists, map them. If there is no matching option, output "0".\n 7) If the response contains both an explicit letter and a conflicting phrase, prefer the explicit letter. If conflicts remain or are unclear, output "0".\n 8) If the response says "I don’t know", "Cannot determine", or similar, output "0".\n \n\n - Example 1\n Response:\n Rome\n Options:\n A. Paris\n B. Berlin\n C. Rome\n D. Madrid\n\n Output -> C\n\n - Example 2\n Response:\n I don’t know\n\n Options:\n A. Glucose\n B. Fructose\n C. Sucrose\n D. Lactose\n\n Output -> 0\n\n - Example 3\n Response:\n A. B\n\n Options:\n A. B\n B. D\n C. A\n D. C\n\n Output -> A\n\n

MCQ Scoring: You are a helpful assistant. \n\n Your task: given (1) a free-form "Response" and (2) a list of "Options", decide which option the response most likely corresponds to and return the option letter. If no option clearly matches, output "0". \n\n Inputs: - VQA Scoring: You are a helpful Response: free-form text that may assistant.\n\n Task: Given a short include a letter, a phrase, or an free-form "Response" and a gold-standard explanation. - Options: A series of "Gold", decide if the Response expresses choices, each starting with a single the SAME answer as Gold. Output "1" uppercase letter followed by ".", one for match, "0" otherwise.\n\n Inputs:\n option in each line.\n\n Output format: - - Gold: the gold-standard answer which STRICTLY OUTPUT EXACTLY ONE CHARACTER: a is either (i) a short phrase, (ii) single uppercase option letter from the an integer, or (iii) "Yes"/"No".\n -

Response: a few words or a short phrase, possibly will include reasoning steps before the final answer.\n\n Output format:\n - STRICTLY OUTPUT EXACTLY ONE CHARACTER: "1" if matching, "0" if not.\n - Do not output any explanation, spaces, punctuation, or additional text.\n\n Rules:\n 1) Compare only the final answer in the Response to Gold. Ignore any reasoning steps or intermediate answers present in the Response.\n 2) If multiple conflicting answers or uncertainty like "I don’t know" appear in the Response, output "0".\n 3) Do not use external knowledge; judge only based on the text in Gold and Response.\n 4) Punctuation, grammar, and minor spelling errors should be ignored.\n uppercase/lowercase differences should be ignored.\n - hyphen and underscore are ignored. For ex, "double-bus" and "double bus" are considered the same.\n - synonyms of "Yes"/"No" like "Y"/"N", "True"/"False" must be considered the same.\n - word representations of numbers like "one"/"two"/"three" must be considered the same as "1"/"2"/"3".\n 5) Core concept and critical attributes must match. For example, "New York City" and "New York State" do not match. Other examples of non-matches are “bus” vs “double bus”; “red” vs “light red”; “dog” vs “golden retriever”; “apple” vs “green apple”.\n 6) If the response says "I don’t know", "Cannot determine", or similar, output "0".\n\n Examples:\n Gold: Double Bus | Response: This is a bus -> 0\n - Gold: Double Bus | Response: I can see a double-bus -> 1\n - Gold: Yes | Response: Y -> 1\n - Gold: 10 | Response: ten -> 1\n - Gold: red | Response: light red -> 0\n - Gold: stop sign | Response: a stop sign on a pole -> 1\n - Gold: person | Response: man -> 0\n\n Now read the following Gold and Response and output exactly one character: "1" or "0".\n

Expanded Tables

Below we present the full table for the No-Img Ablation containing results for all 14 datasets. In the expanded table below, we provide datasetwise number for CoT vs Non-CoT performance of MLMs of various backbones and sizes. Gthinker-7B: visual clues (description of salient part in text) within <vcues_*></vcues_*>tags, encourage rethinking (to enable reflection & relook). Build 7k sample CoT dataset using ‘pattern-guided cold start’ (from ScienceQA, M3CoT, Math, Sherlock etc). CoT data generated using a cascade of MLMs ensuring some samples have rethinking stages. RL data sources are very diverse and general (from llava-o1, r1onevision, mm-eureka). 4K samples are pick from RL sources post clustering to enforce diversity. RL & SFT data sources are different. ViGoRL-7B: base models do not perform visual verification & don’t perform backtracking/reflection. Vanilla GRPO also does not incentivize this behavior. Two step process, i) warm-start CoT SFT: MCTS to generate grounded reasoning steps, where each reasoning step anchors though to image coordinates < st , (xt , yt ) >. Pref MCTS over linear rollouts to enforce exploration and corrective reflection. Qwen2.5-VL-72B teacher is used to generate about 20k reasoning traces from 1400 images of SAT data (from a total of 32k images) ii) spatially grounded RL: Entire training set of 32k SAT questions. Vision-G1: multi-domain data curation. Training data spanning many domains (all domains of VisCoT) is collected. The training sources have cross image reasoning data IconQA, NLVR2, ImageCode and spatial reasoning datasets VQA-AS, Super-CLEVR. Multi-round RL with data curriculum is used (i.e after every round of RL data selection is performed to discard low quality data). Influence function based selection is done using LESS (LESS: Selecting Influential Data for Targeted Instruction Tuning). For difficulty based filtering, use prev round checkpoint to generate k rollouts for each sample, and retain those with avg. acc between 0.2&0.8 (dicard too easy and too hard samples). First IF selection gives 40K training samples, then prev round checkpt is used to perform difficulty-based filtering, on which current round RL training is done. Training done for 3 rounds. Unclear if selection & filtering is done fresh over entire data for every round, or selection done once

Benchmark BLINK (2025) CV-Bench2D (2024a) MMVP (2024b) RealWorldQA (2025) SpatialBench (2025) VSR (2023) V*Bench (2024)

#Questions