ConceptioArchivearXiv CS
arXiv CSopen access

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text

Unknown · 2026 · arxiv_cs
arXiv CS · Papers · License: Open Access · 2026
Open Source ↗Direct PDF ↓
artificialintelligenceknowledgerepresentationreasoning
artificial intelligence, reasoning, knowledge representation

2026-06-08

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text Yutong Bian1*

Dongjie Cheng1*

Heming Xia1

Yongqi Li1†

Wenjie Li1

1 The Hong Kong Polytechnic University

[email protected], [email protected], [email protected]

arXiv:2606.09585v1 [cs.AI] 8 Jun 2026

Abstract Chain-of-Thought (CoT) improves the performance of Large Language Models (LLMs) and has been extended to Multimodal Large Language Models (MLLMs). More recent work further moves from text-based multimodal reasoning toward interleaved-modal reasoning, where intermediate steps can incorporate both textual rationales and visual evidence. In this work, we propose a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? To explore this, we propose optical reasoning, which treats images as a standalone reasoning medium. We instantiate this concept with two variants: typographic-based optical reasoning, which optimizes visual layouts for compact rationale rendering, and graphical-based optical reasoning, which composes text and graphical elements into structured visual rationales. Across mathematical, scientific, and interleaved-modal reasoning benchmarks, optical reasoning can match or even exceed traditional text reasoning while reducing reasoning tokens by an average of 28.57% on language tasks and 16% on multimodal tasks, achieving 1.96× the token efficiency of text reasoning. These results show that images can effectively and efficiently encode rationales while providing a unified visual canvas for reasoning. The code is attached for reproducibility and subsequent open release.

§ Code: https://github.com/ModalityDance/Optical-Reasoning

reasoning [6, 7, 8], where reasoning steps may incorporate both textual rationales and visual evidence. As illustrated in the interleaved-modal reasoning part of Figure 1, this trend suggests that visual information could also contribute to the reasoning process.

1 Introduction Chain-of-Thought (CoT) [1] improves the performance of Large Language Models (LLMs) by eliciting intermediate reasoning steps before deriving final answers, as illustrated in Figure 1. Recently, Large Reasoning Models (LRMs) [2] further strengthen this paradigm by producing longer reasoning traces, leading to improved performance on complex reasoning tasks. Building on the success of reasoning in LLMs, recent studies [3, 4, 5] have extended CoT to multimodal reasoning, where Multimodal Large Language Models (MLLMs) answer questions by jointly interpreting textual prompts and visual inputs. In this setting, CoT improves performance by generating textual reasoning steps based on visual inputs and questions. More recent work further moves beyond purely textual rationales toward interleaved-modal

Based on the above process, we arrived at a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? We believe this transition yields two primary potentials. 1) First, inspired by optical context compression [9, 10, 11, 12], which transforms contexts into images to reduce input tokens, we believe that rendering rationales into images could maintain reasoning performance while substantially improving reasoning efficiency. 2) Second, given the critical role of visual information in multimodal tasks, images provide a unified visual canvas that can integrate text, graphical elements, and spatial layouts naturally.

∗ Equal contribution.

In this work, we propose optical reasoning, which

† Corresponding author.

1

Optical Reasoning Existing Methods Text Reasoning

We conducted experiments across mathematical reasoning, scientific reasoning, and interleavedmodal reasoning benchmarks using five frontier MLLMs: GPT-5.1 [13], Gemini 2.5 Flash [14], Claude Sonnet 4.51 , Kimi K2.5 [15], and Qwen3VL-235B [16]. For language tasks, T-OR matches or exceeds text reasoning in seven model-benchmark pairs while reducing reasoning tokens by an average of 28.57%. In the remaining cases where T-OR underperforms text reasoning, its best setting trails by an average accuracy gap of 0.027 while still reducing the number of reasoning tokens by 20%. For multimodal tasks, T-OR matches or exceeds text reasoning in five model-benchmark pairs while reducing reasoning tokens by an average of 16%; when T-OR falls behind, the average accuracy gap is only 0.014 with a 32% token reduction. More broadly, under the Marginal Accuracy Gain (MAG) metric in Eq. 14, each visual reasoning token achieves 1.96× the efficiency of a text reasoning token. These results validate that images enable effective and efficient compression of interleaved-modal rationales. The contributions are summarized as follows

Interleaved-Modal Reasoning

Step1: Let the height of the building be h. Initially, the angle of elevation is 45^\circ.... Step2: Let the new distance from the tower be d_2. \tan 60^\circ = \dfrac{h}{d_2} \Rightarrow d_2 = \dfrac{h}{\sqrt{3}}.... Step3: Let the constant speed of the car be v. Then v = .... Step4: Time to travel the remaining distance d_2 = \dfrac{h}{\sqrt{3}} is....

Step1: Let the height of the building be h....

Step2: Let the new distance from the tower be....

436 ×

524 ×

Optical Reasoning Typographic-Based

88 ×

Graphical-Based

436 ×

Figure 1: An example of different reasoning paradigms. Optical reasoning represents rationales as either dense typographic layouts or step-aligned graphical compositions. Dark blue boxes denote images, and light red boxes denote text.

• We introduce optical reasoning, positioning images as a promising reasoning medium that not only compactly encodes textual rationales but also offers a unified visual canvas to seamlessly integrate text and graphical elements. • We instantiate optical reasoning with typographicbased and graphical-based variants to explore the efficacy of images as a reasoning medium. Specifically, the former employs an optimized layout token strategy to maximize information density, while the latter leverages the complementarity between symbolic derivations and spatial relations to enable unified multimodal reasoning. • We evaluate optical reasoning across 5 benchmarks and 5 advanced MLLMs, demonstrating that images are an effective and efficient medium with unique capabilities for structuring rationales, achieving 1.96× the token efficiency of text reasoning.

treats images as the sole reasoning medium, and instantiate this concept through two variants. 1) Typographic-based optical reasoning (T-OR), illustrated in the typographic part of Figure 1, employs an optimal layout token strategy designed to maximize information density under a controllable set of reasoning tokens. Specifically, it searches over text width and font size to improve canvas utilization while preserving readability and complete reasoning content. 2) Graphical-based optical reasoning (G-OR), illustrated in the graphical part of Figure 1, employs a step-aligned composition strategy to unify textual rationales and graphical elements within an image. Specifically, G-OR decomposes each rationale into reasoning steps and assigns each step to a corresponding visual panel, where concepts and relations are represented as graphical structures while key formulas and text are retained as explicit reasoning anchors. Together, these two variants examine images as both compact carriers and unified visual canvases for reasoning.

1 https://www-cdn.anthropic.com/

963373e433e489a87a10c823c52a0a013e9172dd.pdf 2

Optical Reasoning

2 Related Work

3.1 Preliminary: Text Reasoning In standard text reasoning [1], language models generate intermediate rationales and final answers in text. We assume the rationale is available, either generated by a model or provided externally, and focus on how different media represent the same reasoning content. Let q = (q1 , . . . , q Nq ) denote its tokenized question. Let rtxt denote the textual rationale sequence:

Interleaved-modal reasoning. Multimodal reasoning is now shifting from text reasoning toward interleaved-modal reasoning. ICoT [6] introduced the concept of interleaved-modal reasoning, where the model identifies relevant visual regions via attention and integrates them into the reasoning process. MINT-CoT [17] further introduced interleaved visual tokens into mathematical CoT, enabling finegrained visual grounding during reasoning. DeepEyes [7] employed active perception with external tools. MVoT [18] enabled visual thinking by generating image visualizations of rationales. ZebraCoT [19] further provided a large-scale interleavedmodal dataset. Omni-R1 [8] proposed a unified generative paradigm for multimodal reasoning by generating intermediate images during reasoning. These methods enrich text-based reasoning with visual information, yet still rely primarily on textual rationales; in contrast, we explore images as standalone reasoning media.

rtxt = (r1txt , . . . , rtxt Ntxt ),

(1)

where each ritxt is a textual rationale unit, such as a text span or an equation, and Ntxt is the number of textual rationale units. For model inference, rtxt is serialized into text tokens. Given q and rtxt , the model πθ decodes the answer token sequence a: a ∼ πθ (· | q, rtxt ) .

(2)

Thus, rtxt serves as the reasoning medium.

3.2 Optical Reasoning

Optical compression. Images have recently shown strong potential for compressing textual content. DeepSeek-OCR [9] formalized this idea as context optical compression, where long textual contexts are encoded as compact images. Glyph [20] further explored optical compression for scaling context windows. Following this paradigm, CodeOCR [10], AgentOCR [12], and MemOCR [11] extend optical compression to more scenarios. Moreover, VTCR1 [21] incorporated optical compression into reasoning by rendering previous textual rationales into images and feeding them back to MLLMs. RoT [22] similarly rendered textual rationales into images, for training the latent reasoning. These studies mainly treat images as a compression of textual rationales, whereas we explore images as standalone reasoning media that can organize text, graphical elements, and spatial layouts.

Motivated by the potential of images to compactly replace rationales and to unify textual, graphical, and spatial information, optical reasoning represents intermediate rationales as images and uses them as the reasoning medium. To support both textual and visual rationales, we define a unified interleaved-modal rationale sequence as rmix = (r1mix , . . . , rmix Nmix ),

rimix ∈ {rtxt , rvis },

(3)

where rimix denotes the i-th rationale unit and Nmix represents the total number of interleaved rationale units. A rationale unit is textual when rimix = rtxt , encompassing text spans or equations, and visual when rimix = rvis , which includes images, diagrams, or visual evidence segments. When all rationale units equal rtxt , the sequence rmix corresponds to the textual rationale sequence rtext utilized in text reasoning. A renderer g maps the interleaved-modal rationale into an image: I = g(rmix ). (4)

3 Method In this section, we formalize text reasoning, introduce optical reasoning, and instantiate it using typographic-based and graphical-based variants.

Let Nvis be the number of visual reasoning tokens. The visual encoder ϕ(·) maps I to vis vis zvis = ϕ( I ) = (zvis 1 , z2 , . . . , z Nvis ),

3

(5)

Optical Reasoning

where zvis i is the i-th visual reasoning token. Under optical reasoning, the model derives the answer from question tokens and visual reasoning tokens: a ∼ πθ (· | q, zvis ) .

where ρ(ℓ) is the fill ratio, ϵ(ℓ) is the layout penalty, and λ is the penalty weight. Specifically, ρ(ℓ) measures the ratio between the occupied content and the canvas, while ϵ(ℓ) aggregates penalties for excessive margins and overly tight layouts that reduce readability. The optimal layout is selected as

(6)

Thus, optical reasoning represents rationales with image tokens rather than text tokens. Based on this formulation, we instantiate optical reasoning with two renderers: a typographic renderer gtypo and a graphical renderer ggraph .

ℓ⋆ = arg max S(ℓ). ℓ∈C B

In practice, the renderer first performs a coarse search over C with a font-size step δs , while retaining only candidates that satisfy the reasoning-token budget. If a feasible candidate layout reaches the threshold τmin , the renderer further examines neighboring font sizes skipped during the coarse search. If no feasible candidate reaches τmin , the renderer falls back to the feasible candidate with the highest score. This strategy preserves the full rationale content while reducing redundant visual space under the target reasoning-token budget.

Typographic-based optical reasoning. For T-OR, we render the interleaved-modal rationale sequence rmix into a compact typographic image under a controllable reasoning-token budget B. The renderer gtypo , implemented with XeLaTeX2 , preserves the original order of rationale units: textual units are typeset as text, equations, or tables, while visual units are inserted as image blocks. Formally, it maps the rationale sequence into a typographic rationale image: Itypo = gtypo (rmix ; ℓ⋆ ). (7)

Graphical-based optical reasoning. Graphicalbased optical reasoning transforms rmix into a unified image-based rationale that organizes reasoning with text, graphical elements, and spatial layouts. We define the graphical renderer ggraph as

Here, ℓ⋆ is the selected layout configuration. We define a layout configuration as

ℓ = (w, s, γ, p),

(8)

Igraph = ggraph (rmix ),

where w denotes the text width, s denotes the font size, γ denotes the line spacing, and p denotes the page padding. These variables control how rationale units are placed on the image canvas. In our implementation, gtypo searches over the candidate width set W and font-size set S , while using default values γ0 and p0 for line spacing and page padding. Therefore, the candidate layout set is

C = {(w, s, γ0 , p0 ) | w ∈ W , s ∈ S}.

(9)

(10)

Each feasible candidate image Itypo is evaluated by a layout score: S(ℓ) = ρ(ℓ) − λϵ(ℓ),

(13)

where rmix is the interleaved-modal rationale sequence. We instantiate ggraph with Nano Banana 23 via a structured prompt. The prompt provides the problem, the rationale, and optional visual images, and asks the renderer to produce a multi-panel graphical rationale following a step-aligned composition strategy. Specifically, the renderer decomposes the rationale into reasoning steps and assigns each step to a corresponding visual panel. Key reasoning text, equations are preserved as textual annotations, while graphical elements and spatial layouts are used to reorganize the rationale visually. The full prompt template is provided in Appendix B.2. The resulting graphical rationale image Igraph is encoded by ϕ(·) into visual reasoning tokens zvis for answer derivation. Compared with Itypo , which preserves the orig-

Given the budget B, the renderer searches for the most compact and readable layout ℓ⋆ . The feasible layout set under this budget is defined as

C B = {ℓ ∈ C | Nvis ≤ B} .

(12)

3 https://storage.googleapis.

(11)

com/deepmind-media/Model-Cards/ Gemini-3-1-Flash-Image-Model-Card.pdf

2 https://xetex.sourceforge.net/

4

Optical Reasoning

inal order of rationale units, Igraph reorganizes the rationale within a unified visual canvas.

to the patch-to-token counting rules described in Appendix C. We also prevented answer leakage by removing explicit answers from the rationales, thereby requiring models to infer the solutions independently. The evaluation prompts for all reasoning settings are provided in Appendix B.1. In addition, we disabled the reasoning mode so that models rely solely on given rationales. For inference, we set the temperature to 0 for all models except Kimi K2.5 [15], which only supports a fixed temperature of 0.6 in the nonthinking mode.

4 Experiments 4.1 Experimental Setup Datasets and evaluation. We evaluate optical reasoning on three reasoning categories: mathematical reasoning with AquaRat [23] and GSM8K [24], scientific reasoning with GPQA Diamond [25] and ScienceQA [26], and interleaved-modal reasoning with Zebra-CoT [19]. Detailed descriptions of these datasets are provided in the Appendix D. We used five frontier MLLMs for our evaluation. Specifically, we selected three closed-source models (GPT-5.1 [13], Gemini 2.5 Flash [14], and Claude Sonnet 4.5) and two open-source models (Kimi K2.5 [15], Qwen3-VL235B [16]). We use accuracy as the primary metric, first applying rule-based matching and then using a large language model judge for unmatched cases with the prompt in Appendix B.3.To better measure token efficiency, we utilized the Marginal Accuracy Gain (MAG) per reasoning token. Specifically, MAG is defined as the accuracy improvement over the no reasoning baseline, normalized by the number of reasoning tokens, capturing the accuracy gain per reasoning token. MAGm =

Accm − Accno , Nm

4.2 Results on Typographic-based Optical Reasoning Through the typographic-based variant, we examined whether images can serve as a more compact but effective medium for rationales. As shown in the Table 1, we observe two main trends across diverse benchmarks and models. First, under comparable reasoning tokens, typographic-based optical reasoning matches or outperforms text reasoning for multiple model-benchmark pairs, while achieving an average MAG 1.96× higher than that of text reasoning across all token-budget ratios. Second, on ZebraCoT [19], a benchmark requiring interleaved-modal reasoning, T-OR outperforms text reasoning on Gemini 2.5 Flash [14], Kimi K2.5 [15], and GPT-5.1 [13], and remains within 1% of performance drop relative to text reasoning on the remaining two models. By integrating rationales and multimodal evidence into images, T-OR demonstrates that images can serve as an effective and efficient reasoning medium. Figure 2 further shows how models respond to reasoning token compression. The performance of Gemini 2.5 Flash [14] remains competitive even under aggressive compression. In contrast, Kimi K2.5 [15] and Claude Sonnet 4.5 improve more consistently as visual tokens increase. This phenomenon indicates that sensitivity to visual information density differs across model families. Moreover, the loss of visual details caused by optical 2D mapping does not necessarily imply that the cues required for reasoning are lost, at least for certain models. We further analyzed this view in Section 4.5.

m ∈ {text, visual}. (14)

Baselines. We compared two reasoning settings. No reasoning provides only the task input and measures the model’s direct-answer capability. Text reasoning provides the textual rationale and serves as the upper bound in our setup. Implementation. In this work, we primarily used rationales from open-source CoT datasets to assess the effectiveness of optical reasoning. Nevertheless, externally provided rationales may also affect model behavior, thereby confounding the assessment of endto-end reasoning performance. To examine whether optical reasoning remains effective when models generate their own rationales, we further evaluated optical reasoning in Section 4.5. To ensure a fair and controlled evaluation, we strictly controlled the reasoning image resolution across all models according 5

Optical Reasoning

Reasoning Tokens

Token Reduction

No reasoning

0.0

-100%

Text reasoning

257.8

-0%

T-OR

51.8 103.7 155.6 207.4 259.3

-80% -60% -40% -20% -0%

No reasoning

0.0

T-OR

Scientific

Multimodal MAG

ScienceQA

Zebra-Cot

0.4961

0.3222

0.4646

0.6737

0.2633

0.4440

0.6811

0.8908

0.7475

0.9646

0.5733

0.7715

1.27

0.5610 0.5630 0.6044 0.5866 0.5866

0.8446 0.9098 0.9334 0.9249 0.9500

0.6970 0.7020 0.7626 0.7273 0.7778

0.9695 0.9673 0.9690 0.9679 0.9739

0.6300 0.6267 0.6133 0.6267 0.6167

0.7404 0.7538 0.7765 0.7667 0.7810

5.72 2.99 2.14 1.56 1.30

-100%

0.6969

0.6179

0.4646

0.6411

0.3500

0.5541

Text reasoning

257.8

-0%

0.7677

0.9962

0.7525

0.9314

0.7000

0.8296

1.07

51.8 103.7 155.6 207.4 259.3

-80% -60% -40% -20% -0%

0.7441 0.7165 0.7520 0.7283 0.7598

0.8340 0.9757 0.9894 0.9924 0.9932

0.5657 0.6364 0.7071 0.7626 0.7424

0.8720 0.9194 0.9341 0.9455 0.9504

0.6733 0.7033 0.6933 0.6933 0.7133

0.7378 0.7903 0.8152 0.8245 0.8318

3.55 2.28 1.68 1.30 1.07

No reasoning

0.0

-100%

0.6890

0.5967

0.4293

0.6737

0.2600

0.5195

Text reasoning

257.8

-0%

0.7323

0.9947

0.7626

0.9782

0.6867

0.8309

1.21

T-OR

51.8 103.7 155.6 207.4 259.3

-80% -60% -40% -20% -0%

0.7677 0.7520 0.7835 0.7402 0.7362

0.9894 0.9909 0.9894 0.9310 0.8923

0.7828 0.7323 0.7626 0.7071 0.7828

0.9597 0.9592 0.9592 0.9613 0.9613

0.6400 0.6367 0.6600 0.6600 0.6467

0.8279 0.8142 0.8309 0.7999 0.8039

5.95 2.84 2.00 1.35 1.10

No reasoning

0.0

-100%

0.6024

0.4193

0.4141

0.7102

0.2800

0.4852

Text reasoning

257.8

-0%

0.7441

0.9970

0.7727

0.9668

0.6600

0.8281

1.33

T-OR

51.8 103.7 155.6 207.4 259.3

-80% -60% -40% -20% -0%

0.7362 0.7520 0.7638 0.7559 0.7441

0.9568 0.9553 0.9704 0.9803 0.9795

0.6212 0.6566 0.6616 0.7020 0.7323

0.9423 0.9526 0.9477 0.9477 0.9499

0.6267 0.6200 0.6300 0.6567 0.6567

0.7766 0.7873 0.7947 0.8085 0.8125

5.63 2.91 1.99 1.56 1.26

No reasoning

0.0

-100%

0.7756

0.7384

0.5303

0.5294

0.3300

0.5807

Text reasoning

257.8

-0%

0.8465

0.9947

0.8434

0.9178

0.6700

0.8545

1.06

T-OR

51.8 103.7 155.6 207.4 259.3

-80% -60% -40% -20% -0%

0.7913 0.8189 0.8386 0.8465 0.8386

0.7589 0.8999 0.9591 0.8597 0.9515

0.5909 0.6768 0.7222 0.7828 0.8131

0.8557 0.8922 0.9031 0.9161 0.9254

0.5200 0.6100 0.6600 0.6633 0.6600

0.7034 0.7795 0.8166 0.8137 0.8377

2.37 1.92 1.52 1.12 0.99

Claude 4.5

GPT-5.1

GPQA Diamond

Kimi K2.5

Avg. Acc.

Gsm8k

Gemini 2.5 Flash

Mathematical

Qwen3-VL-235B

Setting

AquaRat

Table 1: Typographic-based optical reasoning(T-OR) evaluation results across five benchmarks. The Reasoning Tokens reports the average usage per sample across all five benchmarks. MAG reports the marginal accuracy gain over the no-reasoning baseline per 1,000 reasoning tokens. Bold and underlined values indicate the best and second-best results within each comparison group, respectively.

based optical reasoning. By integrating text, graphical elements, and spatial layouts within a unified visual canvas, graphical-based optical reasoning can convey more complex multimodal evidence naturally. This suggests that images are not merely compact containers for text but an expressive reasoning medium. We further illustrate this advantage in Section A.

4.3 Results on Graphical-based Optical Reasoning Building on prior experiments demonstrating that images can serve as a better reasoning medium than text, we further explored the unique advantages of the image with our graphical-based optical reasoning on AquaRat [23]. It is shown in Table 2 that graphicalbased optical reasoning achieves the best accuracy, outperforming both text reasoning and typographic6

Optical Reasoning

GPT-5.1

Claude Sonnet 4.5

Kimi K2.5

Gemini 2.5 Flash

AquaRat 5% 0%

Acc (%)

0%

0%

-5%

-5%

-2.5% -5%

-10%

-7.5%

-15%

-10%

-20% -80%

-60%

-40%

Token Reduction

-20%

-0%

-10% -15% -20% -25% -80%

-60%

Acc (%)

ScienceQA

-40%

Token Reduction

-20%

-0%

-5% -10% -15%

Token Reduction

-20%

-0%

-40%

Token Reduction

-20%

-0%

-20%

-0%

0% -2% -4% -6% -8% -10% -12% -14%

0%

-40%

-60%

Avg.

5%

-60%

-80%

ZebraCot

2% 1% 0% -1% -2% -3% -4% -5% -6% -80%

Text Reasoning

GPQA Diamond

5%

2.5%

-12.5%

Qwen3-VL-235B

Gsm8k

-80%

-60%

-40%

Token Reduction

-20%

-0%

-80%

-60%

-40%

Token Reduction

Figure 2: Analysis on token compression ratios. We report the accuracy change of T-OR over text reasoning under different reasoning tokens. Positive values indicate improvements over the text reasoning baseline.

Setting

Token Reduction

No reasoning

-100%

0.6890

Text reasoning

-0%

0.7323

-80%

0.7677

-60%

0.7520

-40%

0.7835

-20%

0.7402

-0%

0.7362

-

0.8150

T-OR

G-OR

Effects of layout style. We ablated color and font family by varying each rendering factor independently while keeping the others fixed. As shown in Table 3, red achieves the highest accuracy, outperforming the black baseline, whereas green leads to the lowest accuracy. For font family, "Heros" yields the best performance. These results indicate that visual appearance affects how effectively the model decodes typographic rationales in optical reasoning. A possible reason is that high-contrast colors and clean fonts make key textual regions easier for the visual encoder to identify and parse.

Acc.

Effects of layout density. We further studied font size and text width to examine the effect of layout density on visual reasoning. As shown in Table 3, very small fonts substantially reduce accuracy, while moderate font sizes perform better; meanwhile, a narrower text width achieves higher accuracy than wider layouts. These observations suggest that optical reasoning benefits from compact but readable visual layouts. This may be because overly dense layouts impair legibility, while excessively wide layouts spread related reasoning steps too far apart, making it harder for the model to attend to them effectively.

Table 2: Accuracy on AquaRat under no reasoning, text reasoning, T-OR, and G-OR. Bold and underlined values indicate the best and second-best results.

4.4 Ablation Study Since typographic-based optical reasoning relies on a text-to-visual rendering engine, we investigated the impact of key factors during the rendering process by evaluating the performance of the GPT-5.1 [13] model on the GPQA Diamond [25] dataset. Table 3 presents the detailed experimental results. 7

Optical Reasoning

Factor

Color

Font Family

Font Size

Text Width

Value

Acc.

Reasoning

Token

Tokens

Reduction

No reasoning

0.0

-100%

0.6890

Text reasoning

95.3

-0%

0.7323

1.2

-98.75%

0.7008

2.4

-97.50%

0.7283

4.8

-95.00%

0.7638

7.2

-92.50%

0.7992

19.1

-80.00%

0.7677

Setting

Acc.

Black

0.7727

Blue

0.7778

Green

0.7525

Red

0.7929

Heros

0.7778

Latin Modern

0.7626

Pagella

0.7626

Termes

0.7576

8 pt

0.7222

38.2

-60.00%

0.7520

10 pt

0.7677

57.4

-40.00%

0.7835

12 pt

0.7626

76.4

-20.00%

0.7402

14 pt

0.7727

95.6

-0.00%

0.7362

16 pt

0.7727

2.0 in

0.7929

Table 4: Accuracy on AquaRat under extreme reasoningtoken compression using T-OR with Gemini 2.5 Flash.

4.0 in

0.7576

Model

Pillow

Matplotlib

XeLaTeX

6.0 in

0.7475

Qwen3-VL-235B

0.6970

0.6970

0.7323

Kimi K2.5

0.7576

0.7273

0.7626

Gemini 2.5 Flash

0.8131

0.8182

0.7828

Claude Sonnet 4.5

0.7929

0.7727

0.8131

T-OR

Table 3: GPQA Diamond accuracy of GPT-5.1 under different T-OR renderer ablations.

4.5 In-depth Analysis

Table 5: Accuracy on GPQA Diamond using T-OR with different renderers. Pillow, Matplotlib, and XeLaTeX denote the rendering backends used to convert the same textual rationale into typographical rationale.

Extreme compression. To assess the robustness of optical reasoning under extreme token compression, we evaluated T-OR on Gemini 2.5 Flash [14] with the AquaRat [23] dataset. We progressively reduced the number of visual tokens, pushing the average estimated token budget to as few as 1.2 tokens per example. Table 4 demonstrates that optical reasoning remains effective even under extreme compression conditions. At a token budget ratio of −98.75%, the model still achieves an accuracy that can outperform the baseline without reasoning. Optimal performance occurs at an average budget of only 7.2 reasoning tokens per example, which surpasses the performance of both text reasoning and the full-budget optical reasoning. These results indicate that optical reasoning does not rely exclusively on high-resolution visual legibility. Instead, compact visual layouts can preserve coarse yet informative reasoning cues within highly limited token budgets.

Impact of renderers. To assess how renderer choices affect optical reasoning, we conducted a controlled study on GPQA Diamond [25] with four MLLMs [15, 14, 16]. Specifically, we evaluated T-OR under three representative rendering engines: Pillow4 , Matplotlib [27], and XeLaTeX. We fixed the reasoning tokens, rendering strategy, and reasoning content, and varied only the backend used to render rationales into images, thereby isolating the renderer’s effect. As shown in Table 5, different models prefer different rendering engines. Qwen3-VL [16] and Claude achieve their best results with XeLaTeX, respectively, whereas Gemini [14] performs best with Matplotlib. This indicates that the compatibility between rendering engines and MLLMs also affects the performance of optical reasoning, which may stem 4 https://github.com/python-pillow/Pillow

8

Optical Reasoning

Setting

Reasoning Tokens

Acc.

Setting

No reasoning

0.0

0.6890

Text reasoning

95.0

0.7323

17.7 LLMLingua-2

T-OR

Token Reduction

Acc.

No reasoning

-100%

0.4646

Free reasoning

-0%

0.6869

0.6890

-80%

0.6162

37.7

0.6890

-60%

0.6616

57.6

0.6929

-40%

0.6768

76.8

0.6929

-20%

0.6869

19.1

0.7677

-0%

0.6919

38.2

0.7520

57.4

0.7835

76.4

0.7402

T-OR

Table 7: Accuracy on GPQA Diamond using T-OR with rationales generated by GPT-5.1. We compare no reasoning, free reasoning under different budgets.

5 Conclusion and Future Work

Table 6: AquaRat results with LLMLingua-2 and T-OR.

In this study, we propose optical reasoning, rethinking images as the sole reasoning medium, and identify two core potentials. First, rendering rationales into images can preserve reasoning performance while substantially improving efficiency. Second, images provide a unified visual canvas that naturally integrates text, graphical elements, and spatial layouts. The typographic-based variant validates the former potential by using an optimal layout token strategy to maximize information density. The graphical-based variant targets the latter potential by organizing rationales into step-aligned compositions. Empirical results across diverse benchmarks and MLLMs show that: images can serve as an effective and efficient reasoning medium and exhibit unique capabilities for structuring interleaved-modal rationales. A key future direction is to mitigate graphical hallucinations in expressive rationale generation, thereby further unlocking the potential of images as an independent reasoning medium.

from MLLMs’ varying abilities to interpret different visual styles. Comparison with efficient text reasoning. To examine whether optical reasoning offers advantages over existing efficient text reasoning methods, we compared it with LLMLingua-2 [28], a representative text truncation method for reasoning compression, under equivalent reasoning tokens and the same rationales. We conducted this study on AquaRat [23] using Gemini-2.5 Flash [14]. As shown in Table 6, optical reasoning consistently outperforms LLMLingua2 [28]. This suggests that, unlike text truncation, optical reasoning can effectively preserve rationale content, because detail loss from optical 2D mapping does not necessarily remove the cues certain models need for reasoning. Analysis on model-generated rationales To verify whether optical reasoning generalizes to modelgenerated rationales, we evaluated GPT-5.1 [13] on the GPQA Diamond dataset [25]. The free reasoning baseline refers to text reasoning in which rationales are generated by MLLMs themselves. As shown in the table 7, optical reasoning achieves comparable or superior performance relative to the free reasoning baseline. This confirms that optical reasoning holds substantial practical value in realistic reasoning scenarios rather than limited settings.

Limitations This work has two limitations. 1) Model-dependent perception. Optical reasoning may behave differently across MLLMs. Its effectiveness can be affected by model-specific sensitivity to resolution, layout density, rendering style, and visual-token budgets. Future work may develop model-adaptive rendering strategies to improve robustness across different models. 2) Reliability of graphical rationales. Generated schematics may contain graphical inac9

Optical Reasoning

curacies. While graphical-based optical reasoning offers stronger visual expressiveness, its reliability can be improved through end-to-end fine-tuning or reinforcement learning with feedback on visual correctness and answer accuracy.

Ethics Statement This work uses publicly available reasoning benchmarks and does not collect new human-subject data. The datasets are used only for research evaluation and do not contain private user information to the best of our knowledge. The models, tools, and scientific artifacts used in this work are publicly accessible or available through official APIs, and are used in accordance with their intended purposes. The generated rationale images are only intermediate representations for benchmark evaluation rather than humanfacing outputs or deployment decisions. Therefore, we believe this work poses minimal ethical risk.

References [1] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL https://arxiv.org/abs/2201.11903. [2] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, et al. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645 (8081):633–638, Sept 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09422-z. URL http://dx. doi.org/10.1038/s41586-025-09422-z. [3] Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, and Godawari Sudhakar Rao. Kamcot: Knowledge augmented multimodal chain-ofthoughts reasoning, 2024. URL https://arxiv. org/abs/2401.12863. [4] Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1zero’s "aha moment" in visual reasoning on a 2b non-sft model, 2025. URL https://arxiv.org/ abs/2503.05132. [5] Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforce10

ment learning, 2025. URL https://arxiv.org/ abs/2503.07365. [6] Jun Gao, Yongqi Li, Ziqiang Cao, and Wenjie Li. Interleaved-Modal Chain-of-Thought, March 2025. URL http://arxiv.org/abs/2411. 19488. arXiv:2411.19488 [cs]. [7] Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning, March 2026. URL http://arxiv.org/abs/ 2505.14362. arXiv:2505.14362 [cs]. [8] Dongjie Cheng, Yongqi Li, Zhixin Ma, Hongru Cai, Yupeng Hu, Wenjie Wang, Liqiang Nie, and Wenjie Li. Omni-r1: Towards the unified generative paradigm for multimodal reasoning, 2026. URL https:// arxiv.org/abs/2601.09536. [9] Haoran Wei, Yaofeng Sun, and Yukun Li. DeepSeekOCR: Contexts Optical Compression, October 2025. URL http://arxiv.org/abs/2510. 18234. arXiv:2510.18234 [cs]. [10] Yuling Shi, Chaoxiang Xie, Zhensu Sun, Yeheng Chen, Chenxu Zhang, Longfei Yun, Chengcheng Wan, Hongyu Zhang, David Lo, and Xiaodong Gu. CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding, April 2026. URL http://arxiv.org/abs/2602. 01785. arXiv:2602.01785 [cs]. [11] Yaorui Shi, Shugui Liu, Yu Yang, Wenyu Mao, Yuxin Chen, Qi GU, Hui Su, Xunliang Cai, Xiang Wang, and An Zhang. MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning, March 2026. URL http://arxiv.org/abs/ 2601.21468. arXiv:2601.21468 [cs]. [12] Lang Feng, Fuchao Yang, Feng Chen, Xin Cheng, Haiyang Xu, Zhenglin Wan, Ming Yan, and Bo An. AgentOCR: Reimagining Agent History via Optical Self-Compression, February 2026. URL http://arxiv.org/abs/2601. 04786. arXiv:2601.04786 [cs]. [13] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, et al. Openai gpt-5 system card, 2026. URL https://arxiv.org/abs/ 2601.03267. [14] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URL

Optical Reasoning

https://arxiv.org/abs/2507.06261. [15] Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, S. H. Cai, Yuan Cao, Y. Charles, H. S. Che, Cheng Chen, Guanduo Chen, Huarong Chen, Jia Chen, Jiahao Chen, et al. Kimi k2.5: Visual agentic intelligence, 2026. URL https://arxiv.org/abs/ 2602.02276. [16] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report, 2025. URL https://arxiv.org/abs/ 2511.21631. [17] Xinyan Chen, Renrui Zhang, Dongzhi Jiang, Aojun Zhou, Shilin Yan, Weifeng Lin, and Hongsheng Li. Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning, 2025. URL https://arxiv.org/abs/2506.05331. [18] Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought, 2025. URL https: //arxiv.org/abs/2501.07542. [19] Ang Li, Charles Wang, Deqing Fu, Kaiyu Yue, Zikui Cai, Wang Bill Zhu, Ollie Liu, Peng Guo, Willie Neiswanger, Furong Huang, Tom Goldstein, and Micah Goldblum. Zebra-cot: A dataset for interleaved vision language reasoning, 2025. URL https://arxiv.org/abs/2507.16746. [20] Jiale Cheng, Yusen Liu, Xinyu Zhang, Yulin Fei, Wenyi Hong, Ruiliang Lyu, Weihan Wang, Zhe Su, Xiaotao Gu, Xiao Liu, Yushi Bai, Jie Tang, Hongning Wang, and Minlie Huang. Glyph: Scaling context windows via visual-text compression, 2025. URL https://arxiv.org/abs/2510.17800. [21] Yibo Wang, Yongcheng Jing, Shunyu Liu, Hao Guan, Rong cheng Tu, Chengyu Wang, Jun Huang, and Dacheng Tao. Vtc-r1: Vision-text compression for efficient long-context reasoning, 2026. URL https: //arxiv.org/abs/2601.22069. 11

[22] Yifan Wang, Shiyu Li, Peiming Li, Xiaochen Yang, Yang Tang, and Zheng Wei. Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning, 2026. URL https://arxiv. org/abs/2601.14750. [23] Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. CoRR, abs/1705.04146, 2017. URL http://arxiv.org/abs/1705.04146. [24] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [25] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof qa benchmark, 2023. URL https://arxiv.org/abs/2311.12022. [26] Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, KaiWei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022. URL https://arxiv. org/abs/2209.09513. [27] Paul Barrett, John Hunter, J Todd Miller, J-C Hsu, and Perry Greenfield. matplotlib–a portable python plotting package. In Astronomical data analysis software and systems XIV, volume 347, page 91, 2005. [28] Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression, 2024. URL https://arxiv. org/abs/2403.12968.

A Case Study To qualitatively illustrate why optical reasoning works, we analyzed the rationales from different modes in Figure 3. First, both T-OR and G-OR effectively integrate text, graphical elements, and spatial layouts within a unified visual canvas. T-OR faithfully preserves the original rationale. For example, the physics case retains key equations and derivations, while the robot case aligns action descriptions with corresponding visual thoughts. G-OR goes one step further by adopting a more flexible spatial organization, using panels and diagrammatic decomposition

Optical Reasoning

to separate different reasoning stages. As shown in the tower and geometry cases, G-OR explicitly visualizes intermediate variables and spatial relations, enabling the rationale to be conveyed through multimodal illustrations. These observations demonstrate that images, as an expressive reasoning medium, can naturally support interleaved-modal CoT and further strengthen reasoning by exploiting spatial relations within a unified visual canvas. However, we also revealed the limitations of G-OR. Although graphical elements can vividly express complex concepts, the generated schematics are not always accurate. For example, in the geometric case, the red segment is intended to indicate the key diagonal relation, but its placement deviates from the exact geometric constraint. This suggests that G-OR provides stronger visual expressiveness, but may also introduce graphical hallucination as a new failure mode. More case studies are provided in Figures 4, 5 and 6.

spatial layouts jointly carry the reasoning process.

B.3 LLM Judge Prompt We first apply rule-based answer extraction and matching. For predictions that cannot be resolved reliably by the rule-based matcher, we use an LLM judge as a fallback. The judge prompt in Table 10 is constrained to compare only the final prediction against the gold answer and to return a structured correctness verdict.

C Patch Token Mapping Following the setting of CodeOCR [10], we uniformly apply a Qwen3-VL-style patch mapping [16] across all models to estimate visual reasoning tokens. Concretely, for a rendered reasoning image with height H and width W, we estimate its visual token count as ⌈︂ ⌉︂ ⌈︂ ⌉︂ H W N= . (15) 32 32 Closed-source models [13, 14] do not publicly disclose their exact internal visual tokenization rules. We therefore use the same patch-based estimator for all models to keep token accounting comparable. When generating T-OR variants, we resize the rendered rationale image to match target token budgets corresponding to reduction ratios from −80% to 0% relative to the text-reasoning token count. This procedure enables controlled comparisons between text reasoning and visual reasoning under matched or compressed reasoning budgets.

B Prompt Templates This section provides the prompt templates used in our evaluation.

B.1 Evaluation Prompts For answer prediction, all evaluated models are instructed to output only the final answer in a boxed format. Table 8 first lists the baseline prompt settings: no reasoning receives only the problem text, text reasoning receives the problem followed by the rationale, and free reasoning asks the model to solve the problem step by step. For optical reasoning, TOR takes the problem text together with the rendered typographic rationale image, while G-OR takes the problem text together with the generated graphical rationale image. For datasets with fixed answer choices, the system prompt is specialized to require exactly one boxed choice label.

D Benchmark Details Table 11 summarizes benchmark categories, task formats, sizes, and evaluation metrics used in our experiments. We report accuracy for task performance and MAG for token efficiency, following Eq. 14. For Zebra-CoT [19], we construct the evaluation split reproducibly by applying reservoir sampling to each source subset with a fixed random seed of 42. The final evaluation set contains 300 examples from 15 sampled subsets: 2D Visual Reasoning (Visual Jigsaw, Visual Search), 3D Visual Reasoning (Embodied CoT, Robot Planning), Scientific Reasoning (Chemistry, Competitive Programming, Geometry, Physics), and Visual Logic & Strategic Games (Checkers, Chess, Ciphers, Connect Four, Maze, RPM, Tetris).

B.2 Graphical Rationale Generation Prompt For G-OR, we use Nano Banana 2 to transform the problem and answer-masked rationale into a compact rationale image. The prompt in Table 9 operationalizes G-OR by converting the answer-masked rationale into a step-aligned multi-panel visual canvas, where textual reasoning anchors, graphical elements, and 12

Optical Reasoning

Setting

Prompt template

No reasoning

System: You are an expert problem solver. Your only task is to provide the final answer in \boxed{ANSWER} format. Do not show your work, intermediate steps, or reasoning. User: {Problem}.

Text reasoning

System: You are an expert problem solver. Your only task is to provide the final answer in \boxed{ANSWER} format. Do not show your work, intermediate steps, or reasoning. User: {Problem} {rationale}.

Free reasoning

System: You are an expert problem solver. Solve the problem step by step. At the end, provide your final answer in the format \boxed{ANSWER}. User: {Problem}.

T-OR

System: You are an expert problem solver. Your only task is to provide the final answer in \boxed{ANSWER} format. Do not show your work, intermediate steps, or reasoning. User: {Problem}. The rendered typographic rationale image is provided as a separate visual input.

G-OR

System: You are an expert problem solver. Your only task is to provide the final answer in \boxed{ANSWER} format. Do not show your work, intermediate steps, or reasoning. User: {Problem}. The generated graphical rationale image is provided as a separate visual input. Table 8: Evaluation prompt templates for the main task settings.

Prompt template You are an expert educational illustrator. Task: Create a compact, step-by-step comic-style illustration that explains how to solve a problem. Input: Question: {Problem}. Solution: {Answer-masked rationale}. Strict requirements: The comic must follow the solution steps, but must not reveal the final answer explicitly. Preserve the reasoning text, transformations, and intermediate expressions from the provided solution visibly in the image. Include only visuals that directly help explain the solving process. Do not include irrelevant people, characters, decorations, or background scenery. Break the solution into 2–4 clear logical steps, with one panel per step. Layout and compactness: Use a tight multi-panel layout, either 2–4 panels in a single horizontal row or a compact 2-by-2 grid if needed. Keep panel spacing narrow, outer margins small, and content dense enough to fill most of the canvas. Keep each panel simple, information-rich, consistently sized, and easy to scan. Visual style: Clean educational comic style, simple shapes, clear labels, high contrast, plain white background, minimal clutter, crisp linework, and readable mathematical notation. Output: A single compact multi-panel comic illustration. Each panel should correspond to one logical step in the solution. Preserve the reasoning text visibly in the image, but do not include or complete the final answer. Table 9: Prompt template for generating graphical rationales in G-OR.

13

Optical Reasoning

LLM judge template System: You are an exact grader for final answers. Decide whether the student’s final answer is equivalent to the gold answer. Focus only on final-answer equivalence. Treat semantically equivalent, numerically equivalent, or format-equivalent answers as correct when they express the same answer. If the student’s answer is ambiguous, incomplete, or includes multiple conflicting final answers, mark it incorrect. Do not reward correct reasoning if the final answer is wrong. Return JSON only: {"verdict":"CORRECT|INCORRECT"}. User: Problem: {Problem}. Gold answer: {Gold answer}. Student answer: {Model answer}. Table 10: LLM judge prompt template for unresolved answer matching cases.

Benchmark

Task

Size

AquaRat [23]

Multiple-choice algebra and quantitative reasoning prob- 254 lems with five answer options.

Accuracy / MAG

Gsm8k [24]

Open-ended grade-school math word problems requiring multi-step arithmetic reasoning.

Accuracy / MAG

GPQA Diamond [25]

Graduate-level multiple-choice science questions span- 198 ning physics, chemistry, and biology.

Accuracy / MAG

ScienceQA [26]

Multimodal science question answering with image inputs for visually grounded examples.

Accuracy / MAG

Zebra-CoT [19]

Interleaved text-image visual reasoning problems contain- 300 ing question images and intermediate visual rationale images.

1,319

1,836

Metric

Accuracy / MAG

Table 11: Benchmark details for the main evaluation. Dataset sizes correspond to the processed JSONL files used in our experiments.

14

Optical Reasoning

Typographic-based Optical Reasoning For a magnetic field \\vec{B}, parallel to Oz, with small values of m (orbital magnetic quantum number) and B=1T, compare the order of magnitude of the paramagnetic coupling term with transition energy \\Delta E of Hydrogen atom with wavelength \\lambda=0.4861\\mu m.

Graphical-based Optical Reasoning

Please describe the sequence of robot actions from the initial state (the first image) to the goal state (the second image).

A car is being driven, in a straight line and at a uniform speed, towards the base of a vertical tower. The top of the tower is observed from the car and, in the process, it takes 10 minutes for the angle of elevation to change from 45° to 60°. After how much more time will this car reach the base of the tower?

A rectangular solid, 3 x 4 x 15, is inscribed in a sphere, so that all eight of its vertices are on the sphere. What is the diameter of the sphere?

'

Figure 3: Case study comparing T-OR and G-OR across text-only and multimodal reasoning tasks. T-OR preserves the original rationale in a dense typographic layout, while G-OR reorganizes the rationale into step-aligned graphical panels. Typographic-based Optical Reasoning A father wants to divide Rs. 5100 between his two sons, Mohan and Sohan who are 23 and 24 at present. He divides the amount in such a way that if their shares are invested at compound interest at 4% p.a. they will receive equal amount on attaining the age of 26 years. Find Mohan's share.

Graphical-based Optical Reasoning

The image shows a jigsaw puzzle with several missing pieces represented by gray areas. Identify which set of pieces from the options provided correctly fills the gaps in the puzzle.

What is the capital of Vermont? A. Montpelier B. Wilmington C. Burlington D. Hartford

Astronomers are studying a system of five exoplanets with circular orbits, discovered through the TTV method. The planets (designated as Planet_1 through Planet_5), which are in orbital resonance, have orbital periods in a ratio of 1:2:2.5:3.5:5. Assuming these planets possess the same albedo, what is the ratio of the equilibrium temperatures between Planet_4 and Planet_2?

Figure 4: Additional illustrations of T-OR and G-OR across text-only and multimodal reasoning tasks. 15

Optical Reasoning

Typographic-based Optical Reasoning Points A, B, C, D lie in this order on the circumference of a circle. Minor arc AC is 160°, and minor arc BD is 150°. If B bisects minor arc AC, then what is the measure of minor arc AD?

Graphical-based Optical Reasoning

Does the giraffe in the image make physical contact with the person?

You tag the inner capsid proteins of rotavirus with a GFP and deliver them to mice via oral gavage. By in vivo imaging, you observe that some of the fluorescence was destroyed by the stomach acid, and some were entangled in the intestinal mucus before they reached the secondary lymphoid organs. You identify the fluorescent Peyer patches and isolate the proliferating cell population. You sequence their variable heavy chain gene and observe high variability. Which process is taking place?

What is the name of the colony shown? A. New York B. Pennsylvania C. Delaware D. New Jersey

Figure 5: Additional illustrations of T-OR and G-OR across text-only and multimodal reasoning tasks.

Typographic-based Optical Reasoning X is a meson resonance. What is the mean decay distance? Knowing that the production energy is $E_{X}=8GeV$, the mass $m_{X}=1.2GeV$, and the width $\Gamma_{X}=320MeV$.

Graphical-based Optical Reasoning

Identify the person wearing glasses in the image provided.

Calculate the enthalpy of formation of (CH3)2C=CH-CH2-CH(CH3)-CH2CH=C(CH3)2. Given, Enthalpy of atomization of carbon = 1000 kJ/mol. Bond energy of H-H = 100 kJ/mol Bond energy of C-C = 200 kJ/mol Bond energy of C=C = 300 kJ/mol Bond energy of C-H = 400 kJ/mol

You come across an algorithm that gives the following output (written as input -> output): AGG -> 115 TGCTGA -> 176 What value does ACAGTGACC give?

Figure 6: Additional illustrations of T-OR and G-OR across text-only and multimodal reasoning tasks.

16

Record · ID 267703 · SHA-256 50ea848e65473359
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.