Reasoning Correctness: The Missing Dimension in Evaluating Large Language Models

Recent advances in large language models (LLMs) have led to widespread claims that these systems exhibit increasingly sophisticated “reasoning” abilities. Contemporary LLM research focuses primarily on the behavioral success of reasoning-related tasks while paying comparatively little attention to the logical correctness/validity of the reasoning processes involved. Evaluations of mathematical problem solving, planning, scientific question answering, and chain-of-thought prompting have reinforced the perception that reasoning emerges from scale. However, from the viewpoint of Philosophy of Logic, two important conceptual issues remain insufficiently examined: First, the nature of “reasoning”, what exactly is “reasoning”? Second, the criteria for “reasoning correctness”, what exactly is “correct reasoning” and how evaluate “reasoning correctness”? Drawing on insights from Logic and Philosophy of Logic, this paper argues the above two important conceptual issues. We provide a definition of “reasoning” and, based on this definition, clearly distinguish between “reasoning correctness” and “reasoning appearance.” We point out that generating seemingly reasonable reasoning trajectories and intermediate steps, and even correct results do not prove the logical correctness/validity of the underlying reasoning process. We further argue that correctness should be treated as an independent dimension in the evaluation of any machine reasoning. Future progress may require explicit inferential evaluation criteria in addition to benchmark accuracy. The paper proposes a new correctness-oriented evaluation methodology for machine reasoning.