Official Review · ICLR 2026

This paper introduces a new comprehensive benchmark called MME-Reasoning, designed to evaluate the logical reasoning capabilities of Multimodal Large Language Models (MLLMs). The authors argue that existing multimodal reasoning benchmarks are insufficient for assessing logical reasoning, mainly because they lack a clear taxonomy and coverage of different reasoning types, inductive, deductive, and abductive, and often conflate perceptual ability or knowledge breadth with reasoning ability. 1. The benchmark follows clear design principles (comprehensiveness, going beyond perception, minimizing knowledge dependence, and diverse evaluation formats), and the construction pipeline is well described. 2. Data are sourced from textbooks, logic workbooks, online resources, exams, existing benchmarks, and author-designed/synthetic problems, with manual filtering to remove items that primarily rely on perception or complex domain knowledge. 3. A broad set of recent, representative MLLMs (closed/open source, chat/thinking models, and rule-based RL models) are evaluated. 4. The paper is well structured and clearly written. 1. The core goal of MME-Reasoning is to systematically assess MLLMs’ logical reasoning—explicitly covering inductive, deductive, and abductive types—while striving to decouple perception and domain knowledge from reasoning. However, many similar benchmarks already exist, e.g.: - Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal Reasoning Benchmark - VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models - MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models Their data sources likely also contain abductive items, so this benchmark may not clearly reveal deficiencies of current MLLMs relative to what existing benchmarks already show. 2. Much of the data comes from academic exams (e.g., K-12 math/physics), standardized logic questions (civil service exams), and hand-crafted puzzles, with limited coverage of real-world reasoning scenarios. This weakens the benchmark’s ability to assess “real-world reasoning.” 3. Some data (e.g., Chinese civil-service logic questions) may introduce evaluation bias for MLLMs developed by non-Chinese institutions. Such institutions may not train on similar data, whereas Chinese institutions might, making the benchmark less reliable for unbiased capability assessment. 4. The difficulty levels (easy/medium/hard) are defined mainly by the time human experts need to solve each problem, which introduces subjectivity. It would be better to incorporate more objective metrics or report inter-annotator agreement. The paper defines five core abilities, but how this taxonomy was derived—and whether it is standard or newly proposed—should be further justified for soundness and completeness. 5. Typographical error: “casual chain analysis” → “causal chain analysis.” Same as the Weaknesses.