When Does LLM Self-Correction Help? A Control-Theoretic Markov Diagnostic and Verify-First Intervention
arXiv:2604.22273v1 [cs.AI] 24 Apr 2026
Aofan Liu1
Jingxiang Meng2,∗
Abstract— Iterative self-correction is widely used in agentic LLM systems, but when repeated refinement helps versus hurts remains unclear. We frame self-correction as a cybernetic feedback loop in which the same language model serves as both controller and plant, and use a two-state Markov model over {Correct, Incorrect} to operationalize a simple deployment diagnostic: iterate only when ECR/EIR > Acc/(1 − Acc). In this view, EIR functions as a stability margin and prompting functions as lightweight controller design. Across 7 models and 3 datasets (GSM8K, MATH, StrategyQA), we find a sharp near-zero EIR threshold (≲ 0.5%) separating beneficial from harmful self-correction. Only o3-mini (+3.4 pp, EIR = 0%), Claude Opus 4.6 (+0.6 pp, EIR ≈ 0.2%), and o4-mini (±0 pp) remain non-degrading; GPT-5 degrades by −1.8 pp. A verifyfirst prompt ablation provides causal evidence that this threshold is actionable through prompting alone: on GPT-4o-mini it reduces EIR from 2% to 0% and turns −6.2 pp degradation into +0.2 pp (paired McNemar p < 10−4 ), while producing little change on already-sub-threshold models. ASC further illustrates the stopping trade-off: it halts harmful refinement but incurs a 3.8 pp confidence-elicitation cost. Overall, the paper argues that self-correction should be treated not as a default behavior, but as a control decision governed by measurable error dynamics.
I. INTRODUCTION Iterative self-correction—where a language model reviews and revises its own outputs—has emerged as a central mechanism in agentic AI systems. The promise is intuitive: by reflecting on potential errors, models can catch and fix mistakes that arise in initial reasoning. This capability underpins self-refinement loops in autonomous agents [1], [2], multi-agent debate systems [3], and reasoning chains with verification steps [4]. However, a growing body of evidence questions the effectiveness of unbounded self-correction. Huang et al. [5] demonstrate that LLMs cannot reliably self-correct reasoning without external feedback, often degrading performance through overcorrection. A complementary pattern—which we term the accuracy–correction paradox—is that models with higher initial accuracy tend to benefit less from, or even be harmed by, self-correction; related critical surveys [14], [15] document similar trends. Chen et al. [3] find that excessive refinement paradoxically degrades performance in multi-agent systems. These findings converge on a critical question: What governs the transition from beneficial to harmful self-correction, and can we predict and exploit this transition? 1 Peking University. 2 University of Chicago. ∗ Corresponding author.
Prior work has approached this question empirically, characterizing saturation points for specific models and tasks. However, a theoretical understanding of why self-correction converges—or diverges—and when it should stop is lacking. This work. We treat iterative self-correction as a closedloop control problem: each revision applies a feedback action to the previous answer, and the central question is whether the loop is stabilizing or destabilizing. Our contribution is less a new Markov theorem than an operationalization of this feedback view into a measurable diagnostic and an actionable intervention. Specifically, we make three contributions: (1) A Markov-chain diagnostic for self-correction. We model iterative self-correction as a two-state Markov chain on {Correct, Incorrect}, parameterized by the Error Introduction Rate (EIR) and Error Correction Rate (ECR). The equilibrium, steady-state, and convergence expressions are standard consequences of this model; our contribution is to turn them into a directly measurable stop-or-iterate diagnostic for deployment. Relative to Yang et al. [6], who characterize convergence curves across rounds, we emphasize an engineering question: when should a controller continue applying self-correction, and when should it stop? (2) A near-zero EIR threshold, causally validated. Across 7 models and 3 datasets we identify a sharp threshold: only models with EIR ≲ 0.5% (o3-mini, Claude Opus 4.6, o4-mini) benefit from self-correction; five others degrade, including GPT-5. A verify-first prompt ablation drives GPT4o-mini’s EIR from 2% to 0% and turns −6.2 pp degradation into +0.2 pp (paired McNemar p < 10−4 ). The same prompt produces little change on already-sub-threshold models, as predicted by the diagnostic, suggesting that EIR is the causal control variable rather than a spurious correlate. (3) Analysis of stopping and capability trade-offs. We analyze ASC as an adaptive stop-rule combining instanced monitoring. d ECR level self-confidence with batch-level EIR/ ASC halts harmful refinement for GPT-4o-mini, but the confidence-elicitation prompt itself costs 3.8 pp of accuracy. We therefore treat ASC not as a headline gain, but as evidence for a two-tier capability view: EIR suppression (prompt-level, already achievable) prevents degradation, whereas ECR enhancement (likely training-level) is required for actual improvement. Our framework shifts the study of self-correction from observation to principled control: measure EIR on a calibration set, check whether the equilibrium condition is satisfied, and iterate only if it is. For high-accuracy models, π ∗ can be far below baseline, making unconditional self-correction a systematic form of compute waste.
II. RELATED WORK A. Self-Correction in Large Language Models Huang et al. [5] demonstrate that LLMs cannot selfcorrect reasoning without external feedback, finding that selfcorrection often degrades performance when models lack ground-truth signals. Kamoi et al. [14] and Stechly et al. [15] provide critical surveys showing that strong self-verification is rare in current LLMs, and that higher baseline accuracy can correlate with worse net gains from self-correction—a pattern we refer to as the accuracy–correction paradox. Yang et al. [6] recently develop a probabilistic theory for multi-round self-correction, modeling accuracy evolution as a Markov process and deriving closed-form convergence curves that align well with empirical trajectories across diverse models. Building on this line of analytical work, we focus on the practical question of when self-correction helps versus hurts: we identify a near-zero EIR threshold as a diagnostic for beneficial self-correction and validate it causally through a verify-first prompt ablation. Our work extends these analyses by examining error propagation across multiple iterations rather than single-step correction. B. Iterative Refinement Strategies Madaan et al. [1] introduce Self-Refine, demonstrating improvements across diverse tasks but without systematically quantifying diminishing returns. Shinn et al. [2] propose Reflexion with verbal reinforcement learning through selfreflection, but rely on external feedback. Chen et al. [3] introduce MAgICoRe, identifying “excessive refinement” as a key challenge but without measuring iteration-by-iteration trajectories. Our work complements these approaches by providing fine-grained empirical measurements of what happens at each refinement step. C. Agentic Reasoning Systems Agentic reasoning systems combine planning, tool use, and self-reflection to tackle open-ended problems through continual interaction with an environment. Du et al. [4] explore iterative reasoning through energy minimization, showing that adaptive computational budgets improve performance, while Schick et al. [12] show that language models can learn to invoke external tools within their reasoning trajectories. Gou et al. [16] further demonstrate that toolinteractive critiquing is substantially more reliable than intrinsic self-critique. Our focus on refinement depth complements this body of work by quantifying when iteration benefits diminish even in the purely intrinsic setting. D. Error Analysis in Mathematical Reasoning Prior work on mathematical reasoning has focused on improving baseline accuracy through better prompting [11], fine-tuning [8], or tool augmentation [12]. However, systematic analysis of error types and their evolution through refinement cycles remains limited. Existing error taxonomies [13] classify errors at a single point rather than tracking how errors transform through iterative refinement.
Our work addresses this gap by introducing a dynamic error classification framework that tracks error type transitions across iterations. By analyzing EIR and ECR separately, we provide fine-grained insights into which types of problems benefit from refinement and which are resistant to correction. Despite extensive work on self-correction [1], [2], [5], no prior study systematically measures iteration-by-iteration accuracy trajectories, error introduction versus correction rates, and error propagation patterns. Our work fills this gap. III. METHOD A. Problem Formulation Consider N reasoning problems {q1 , . . . , qN } with ground-truth answers {a∗1 , . . . , a∗N }. Iterative self-correction proceeds as: (0)
= M(qi ) (k) (k−1) ri = M(qi , ri , prefine ) ri
(1) for k ≥ 1
(2)
(k)
where ri is the response at iteration k and prefine is the (k) (k) refinement prompt. Define ci = ⊮[extract(ri ) = a∗i ] and PN (k) 1 Acc(k) = N i=1 ci . Definition 1 (Error Transition Rates): At iteration k, the Error Introduction Rate (EIR) and Error Correction Rate (ECR) are: (k+1)
EIR(k) = P (ci
(k)
= 0 | ci
= 1)
(3)
(k+1) (k) ECR(k) = P (ci = 1 | ci = 0)
(4)
The Net Benefit is NB(k) = Acc(k) − Acc(k − 1). B. Markov Chain Model We model each problem’s correctness as a two-state Markov chain on S = {C ORRECT, I NCORRECT}, illustrated in Figure 1, with transition matrix: 1 − EIR(k) EIR(k) T (k) = (5) ECR(k) 1 − ECR(k) C. Theoretical Results Theorem 1 (Equilibrium Condition): The net benefit is zero (NB(k + 1) = 0) if and only if: Acc(k) ECR(k) = (6) EIR(k) 1 − Acc(k) This provides an interpretable condition: refinement stops improving when the correction rate (weighted by error proportion) exactly balances the introduction rate (weighted by correct proportion). Theorem 2 (Steady-State Accuracy): If EIR(k) → EIR∗ and ECR(k) → ECR∗ , the steady-state accuracy is: ECR∗ (7) π∗ = EIR∗ + ECR∗ Theorem 3 (Convergence Rate): Under stationary rates, convergence is geometric: |Acc(k) − π ∗ | = |1 − EIR∗ − ECR∗ |k · |Acc(0) − π ∗ | (8) Proof sketch for Theorem 1. From Markov dynamics, Acc(k+1) = Acc(k)(1 − EIR(k)) + (1 − Acc(k))ECR(k).
THEORETICAL LAYER – Markov Chain
ALGORITHMIC LAYER – ASC CONTROLLER Batch-level monitor
Instance-Level Confidence
Refine Another Iteration ？ Or early-stop ？
EMPIRICAL-LAYER - 7 MODELS x 3 DATASETS GPT4o-mini
GPT-4.1
Claude Sonnet 4
GPT-5
Claude Opus 4.6
GPT-o3-mini
GPT-o4-mini
Five Convergence Modes 1. Monotonic degradation (EIR>ECR) 2. Absorbing lock (EIR ≈0, ECR ≈ 0) 3. Stepwise decline (alternating slip vs. stable)
5. Oscillating near-lock (frontier cap. Still - δ) 6. Beneficial convergence (EIR <= 0.5% -> +0.6.. +3.4 pp)
Fig. 1. Three-layer view of iterative self-correction as a Markov feedback loop. The theoretical layer formalises correctness evolution on {C, I} with EIR/ECR transitions, yielding equilibrium, steady-state, and convergence expressions. The control layer interprets EIR as a stability margin and verify-first d monitoring for early stopping. The empirical d ECR prompting as controller design; ASC adds instance-level confidence γ(k) ≥ τ with batch-level EIR/ layer evaluates 7 models × 3 datasets, confirming near-zero EIR (≲ 0.5%) as the threshold separating beneficial from harmful self-correction.
Setting NB = Acc(k+1) − Acc(k) = 0 and rearranging yields Eq. (6). Theorem 2 follows from the stationary distribution of the limiting transition matrix, and Theorem 3 from the spectral decomposition with subdominant eigenvalue λ2 = 1 − EIR∗ − ECR∗ . Implications. Theorem 3 shows that when both rates are low (typical for strong models on moderate tasks), convergence is slow (ρ ≈ 1). When rates are high, convergence is fast but the steady-state may be poor. Most critically, these results reveal a fundamental asymmetry: for high-accuracy models, π ∗ can be far below baseline accuracy, making all self-correction harmful. D. Adaptive Self-Correction (ASC) Motivated by the theoretical analysis, we propose ASC (Algorithm 1), which uses two complementary stopping criteria: ASC operates at two levels: (1) instance-level confidence: if self-assessed confidence γ (k) ≥ τ (default τ = 8 on 1–10 scale), refinement stops; (2) batch-level equilibrium: d further iterations would decrease d ≥ ECR, if running EIR accuracy by Theorem 1. E. Experimental Design Models. Seven models spanning four capability tiers: GPT-4o-mini (fast), GPT-4.1 and Claude Sonnet 4 (mid),
Algorithm 1 Adaptive Self-Correction (ASC) Require: Problem q, model M, max iterations K, threshold τ Ensure: Final response r, iterations used k 1: r (0) ← M(q); γ (0) ← ConfScore(M, q, r (0) ) 2: if γ (0) ≥ τ then 3: return r(0) , 0 4: end if 5: for k = 1 to K do 6: r(k) ← M(q, r(k−1) , prefine ) 7: γ (k) ← ConfScore(M, q, r(k) ) 8: if γ (k) ≥ τ then 9: return r(k) , k // Confidence criterion 10: end if d then d ≥ ECR 11: if EIR 12: return r(k−1) , k − 1 // Equilibrium criterion 13: end if 14: end for 15: return r (K) , K GPT-5 and Claude Opus 4.6 (frontier), and o3-mini plus o4-mini (reasoning/RLVR). Detailed per-iteration trajectories are reported for six core models on GSM8K (Table II); o4mini is evaluated as an additional RLVR validation run in Section IV-F.
TABLE I BASELINE ACCURACY (%) ACROSS MODELS AND DATASETS
TABLE III EIR AND ECR (%) ON GSM8K (500 PROBLEMS ).
(I TERATION 0). T HE SEVENTH MODEL ( O 4- MINI ) IS REPORTED 0→1
SEPARATELY AS AN RLVR VALIDATION RUN IN S ECTION IV-F.
1→2
2→3
3→4
EIR ECR EIR ECR EIR ECR EIR ECR
Tier
Model
GSM8K MATH StrQA
Fast Mid
GPT-4o-mini GPT-4.1 Cl. Sonnet 4 Frontier GPT-5 Cl. Opus 4.6 Reasoning o3-mini
91.2 94.6 96.8 96.2 97.6 93.2
72.8 79.8 78.2 85.8 86.0 75.8
75.5 82.5 87.5 84.5 89.5 47.0
TABLE II ACCURACY (%) ACROSS REFINEMENT ITERATIONS ON GSM8K.
Model
It. 0 It. 1 It. 2 It. 3 It. 4
GPT-4o-mini GPT-4.1 Cl. Sonnet 4 GPT-5 Cl. Opus 4.6 o3-mini
91.2 94.6 96.8 96.2 97.6 93.2
90.0 94.4 96.2 94.8 98.0 96.2
89.6 94.4 96.2 94.4 98.2 96.6
86.6 94.2 95.6 94.6 98.0 96.6
85.0 94.4 95.6 94.4 98.2 96.6
∆ −6.2 −0.2 −1.2 −1.8 +0.6 +3.4
Datasets. GSM8K [8] (500 problems), MATH [9] (400 problems), StrategyQA [10] (200 problems). Baselines. Generic refinement (4 iterations), SelfRefine [1], Reflexion [2], Self-Consistency [7]. IV. EXPERIMENTS A. Baseline Accuracy Table I shows baseline accuracies across all model–dataset pairs. Claude Opus 4.6 achieves the highest accuracy on all three benchmarks (97.6%, 86.0%, 89.5%), followed by GPT5. MATH accuracy is consistently 10–14 points lower than GSM8K, confirming the expected difficulty gradient. The one outlier is o3-mini on StrategyQA (47.0%): the model frequently refuses open-ended commonsense items by returning meta-commentary rather than yes/no, which our exact-match extractor scores as incorrect. Since our refinement analysis is conducted on GSM8K where extraction is unambiguous, this extraction artefact does not affect the EIR/ECR results reported below; we revisit it in the limitations. B. Accuracy Trajectories Table II reveals five distinct convergence patterns on GSM8K (500 problems each). Five modes emerge: (1) Monotonic degradation (GPT-4o-mini: 91.2%→85.0%, −6.2 pp). EIR escalates from 1.3% to 3.8% across iterations while ECR remains low, leading to compounding errors. (2) Absorbing lock (GPT-4.1: ±0.2 pp). After one iteration, both EIR and ECR drop to zero—no answers change. The model reaches a near-fixed point at 94.4%. (3) Stepwise decline (Claude Sonnet 4: −1.2 pp). Alternating “slip” iterations (−0.6 pp) and stable iterations produce a gradual decline with dynamic equilibrium. (4) Oscillating near-lock (GPT-5: −1.8 pp). Despite frontier capability (96.2% baseline), GPT-5 declines −1.4 pp
GPT-4o-mini GPT-4.1 Cl. Son. 4 GPT-5 Cl. Op. 4.6 o3-mini
1.3 0.4 0.8 1.9 0.2 0.0
0.0 3.7 6.2 10.5 25.0 44.1
0.9 0.0 0.2 0.8 0.2 0.0
4.0 0.0 5.3 7.7 20.0 10.5
3.8 0.2 0.8 0.0 0.4 0.0
3.8 0.0 5.3 3.6 11.1 0.0
2.1 0.0 0.4 0.4 0.2 0.0
1.5 3.4 9.1 3.7 20.0 0.0
at Iteration 1, then oscillates around 94.4%. Its initial EIR (1.9%) is the highest among mid/frontier models. (5) Beneficial convergence (o3-mini: +3.4 pp; Opus 4.6: +0.6 pp). Both improve and stabilize above baseline. o3mini maintains EIR = 0% across all 4 iterations; Opus 4.6 achieves EIR ≈ 0.2% with ECR/EIR ≈ 125, far exceeding the equilibrium requirement of 40.7. The GPT-5 vs. Opus 4.6 contrast is instructive: despite similar baseline accuracy (96.2% vs. 97.6%), a 10× difference in EIR (1.9% vs. 0.2%) flips the outcome from degradation to improvement. This demonstrates that the EIR threshold for beneficial self-correction is sharp. C. EIR/ECR Dynamics Table III presents per-iteration error transition dynamics for all six models. Key findings: (1) o3-mini and Opus 4.6 achieve nearzero EIR, acting as a “correctness guard”; (2) beneficial models achieve high ECR (o3-mini: 44.1%, Opus 4.6: 25.0% at Iteration 1); (3) EIR increases for weaker models (GPT-4o-mini: 1.3%→3.8%), indicating compounding errors; (4) instruction-tuned models fail the equilibrium condition—for GPT-4o-mini, ECR/EIR ≈ 1.0 vs. the required 8.6; (5) for o3-mini with EIR = 0, ECR/EIR → ∞, trivially satisfying Theorem 1. Findings (4)–(5) directly support H2. D. Baseline Comparisons Self-Refine. On 750 GSM8K problems with GPT-4omini, Self-Refine’s Iteration 0 accuracy (85.7%) is substantially lower than generic refinement (91.2%), likely because Self-Refine’s prompt format interferes with initial problemsolving. Both methods degrade: −4.6 pp for Generic, −3.6 pp for Self-Refine. This supports H5: structured selffeedback does not overcome EIR/ECR imbalance. Compute-Equivalent Comparison. With 3 API calls per problem (Table IV), Self-Consistency [7] achieves 93.4%, surpassing baseline by 2.2 pp and outperforming 3-iteration refinement by 6.8 pp. Self-Consistency generates independent samples, breaking the sequential dependency chain. With k = 3 independent samples at p = 0.912, the theoretical majority-vote accuracy p3 + 3p2 (1 − p) ≈ 97.9%; the observed 93.4% is lower due to correlated errors. E. Verify-First Prompt Ablation Can we induce near-zero EIR via prompting? We test a verify-first prompt on GPT-4o-mini: “Before making any
TABLE IV C OMPUTE - EQUIVALENT COMPARISON (3 API CALLS ) ON GSM8K WITH GPT-4 O - MINI . ∆ IS COMPUTED AGAINST THE SINGLE - SHOT BASELINE (91.2%) TO ENABLE CROSS - METHOD COMPARISON ; S ELF -R EFINE ’ S OWN I TER -0 ACCURACY IS 85.7% ( PER - METHOD DEGRADATION −3.6 PP ).
Method
Acc. (%)
∆
91.2 86.6 82.1 93.4
— −4.6 −9.1 +2.2
Baseline (single shot) 3-iter Generic Refine Self-Refine (3 iters) Self-Consistency
Prompt
It. 0 It. 1 It. 2 It. 3 It. 4
∆
GPT-4o-mini Standard 91.2 90.0 89.6 86.6 85.0 −6.2 Verify-First 91.2 91.2 91.4 91.4 91.4 +0.2 GPT-4.1 Standard 94.6 94.4 94.4 94.2 94.4 −0.2 Verify-First 94.6 95.0 95.0 94.6 94.6 0.0 Sonnet 4.5 Verify-First 97.2 97.2 97.0 96.8 97.2 0.0 GPT-4o-mini Std. EIR VF EIR
— —
1.3 0.0
0.9 0.0
3.8 0.0
Model
Ver.% Conf.% Chg.% EIR
Mode
GPT-4o-mini 98.8 GPT-4.1 99.5 Cl. Son. 4 100.0 GPT-5 96.2
95.5 99.0 99.1 95.2
3.4 0.5 0.9 1.2
2.02 Degrade 0.16 Absorb 0.57 Stepwise 0.78 Oscillate
Cl. Op. 4.6 o3-mini
99.2 83.8
0.7 1.2
0.26 Beneficial 0.00 Beneficial
99.9 84.8
TABLE VII H YPOTHESIS TESTS FOR GPT-4 O - MINI ON GSM8K (α = 0.05).
TABLE V V ERIFY- FIRST PROMPT ABLATION (GSM8K, 500 PROBLEMS ). ACCURACY TRAJECTORIES (%) PER ITERATION ACROSS THREE MODELS ; EIR ROWS (%) SHOWN FOR THE DEGRADING CASE .
Model
TABLE VI R ESPONSE PATTERNS ON GSM8K (500 × 4 ITERATIONS ).
2.1 0.0
changes, first verify whether your previous answer is correct by re-solving independently. Only change your answer if you find a concrete, specific error.” To test whether the effect generalizes, we replicate on GPT-4.1 and Claude Sonnet 4.5 (a point refresh of the Sonnet 4 endpoint used in Table II, selected because it is the version available at the time of the ablation run)—two models whose baseline EIR is already ≲ 0.5%. On GPT-4o-mini, verify-first achieves EIR = 0% across all 4 iterations, converting monotonic degradation (−6.2 pp) to slight improvement (+0.2 pp); the paired McNemar test on iter-4 outcomes gives p < 10−4 vs. standard refinement, with ∆ Acc = +6.4 pp (paired-bootstrap 95% CI [+4.2, +8.8]). On GPT-4.1 and Sonnet 4.5, whose baseline EIR is already below the threshold, verify-first produces no detectable accuracy change (paired McNemar p = 1.0 for both)—consistent with the prediction that the intervention is targeted: active where baseline EIR is harmful, inert where the threshold is already satisfied. This confirms near-zero EIR is the sufficient condition for preventing degradation, achievable through prompt engineering alone. However, it does not achieve the beneficial improvements of o3-mini (+3.4 pp), suggesting ECR enhancement requires deeper training-level capabilities. F. RLVR Validation: o4-mini To test generalization, we evaluate o4-mini—a third RLVR model. It exhibits average EIR = 0.2% with accuracy oscillating within ±0.4 pp of 96.8%. With three models tested (o3-mini: EIR = 0%, Opus 4.6: EIR ≈ 0.2%, o4-mini: EIR ≈ 0.2%), the near-zero EIR property appears consistent across all non-degrading models.
Hyp.
Test
Stat.
p
Result
H1: It. 0→1 McNemar 6.00 0.014 Sig. ↓ H1: It. 1→2 McNemar 0.67 0.414 n.s. H1: It. 2→3 McNemar 11.84 <0.001 Sig. ↓ H1: It. 3→4 McNemar 6.40 0.011 Sig. ↓ H5: Gen. vs. SR McNemar 0.50 0.480 n.s.
G. Response Pattern Analysis Table VI reveals a surprising finding: all models exhibit high verification phrase ratios (84.8–100%), including GPT4o-mini (98.8%), which degrades the most. The distinguishing factor is not how models verify but how often they choose to change answers—and whether those changes target genuinely incorrect answers. When models change answers on previously correct problems, the change is harmful in nearly 100% of cases across all models. The key mechanism behind near-zero EIR is an internal ability to accurately assess correctness, not surface-level verification rhetoric. H. Statistical Analysis Table VII presents hypothesis test results. Three of four transitions show significant decreases, providing strong evidence that iterative self-correction is actively harmful for this model. Self-Refine does not significantly outperform generic refinement (p = 0.48), confirming H5. I. Analysis: Why Self-Correction Fails Pool-size asymmetry. For GPT-4o-mini on GSM8K, the correctable error pool is small (44/500) while the correct pool susceptible to EIR is large (456/500). Even a small EIR (1.3%) on the large correct pool produces more errors than a moderate ECR on the small incorrect pool. This instantiates the accuracy–correction paradox introduced above: for any model with Acc > π ∗ , self-correction degrades performance. Why o3-mini and Opus 4.6 succeed. Both break the paradox by achieving near-zero EIR, dramatically reducing poolsize asymmetry. With EIR ≈ 0, any nonzero ECR guarantees improvement. We hypothesize both models implement an implicit “verify-then-edit” strategy. For o3-mini, this likely emerges from RL training on chain-of-thought reasoning; for Opus 4.6, it may reflect large-scale instruction tuning that develops strong internal consistency checking. Cross-dataset generalization (H3). GPT-4o-mini’s degradation generalizes: on MATH, accuracy declines from 72.8%
to 68.5% (−4.3 pp), confirming the pattern is not datasetspecific. ASC behavior and confidence–accuracy trade-off (H4). ASC halts at Iteration 0 for all 500 GSM8K problems with GPT-4o-mini, correctly identifying that further refinement would be harmful. However, adding the confidence elicitation prompt itself reduces accuracy: ASC achieves 87.4% vs. the baseline’s 91.2%, a loss of 3.8 pp. This suggests that requiring explicit self-assessment diverts reasoning capacity. A two-stage approach—first generating without confidence overhead, then selectively refining only uncertain cases— may mitigate this trade-off. Two-tier capability model. Our findings support a twotier model: (1) EIR suppression (achievable via prompt engineering or RL) prevents degradation by stopping the model from changing correct answers; (2) ECR enhancement (likely requiring RL training with verifiable rewards) enables actual improvement by developing the capacity to reliably identify and correct genuine errors. The verify-first prompt achieves tier 1 (+0.2 pp) but not tier 2 (+3.4 pp for o3-mini), confirming this distinction. V. CONCLUSIONS This work makes three contributions. First, we cast selfcorrection as a two-state Markov feedback system and use its standard equilibrium (Theorem 1), steady-state (Theorem 2), and convergence (Theorem 3) properties as an operational diagnostic. For high-accuracy models, π ∗ can be far below baseline, making repeated self-correction harmful. Second, we reveal a hierarchy of convergence behaviors across seven models. The key predictor is near-zero EIR (≲ 0.5%), not model architecture: o3-mini (+3.4 pp, EIR = 0%), Claude Opus 4.6 (+0.6 pp, EIR ≈ 0.2%), and o4-mini (±0 pp, EIR ≈ 0.2%) all achieve non-degrading self-correction. The GPT-5 vs. Opus 4.6 contrast (similar capability, opposite outcomes) shows that a 10× difference in EIR can flip the closed-loop outcome. Third, we validate a verify-first prompt that reduces EIR to 0% through prompt engineering alone, and we use ASC mainly to analyze stopping trade-offs rather than claim a universal performance gain. Practical implications: (1) for models with Acc > π ∗ , selfcorrection reduces accuracy, so practitioners should estimate EIR on calibration sets; (2) verify-first prompting behaves like lightweight controller design, converting harmful loops into stable ones; (3) Self-Consistency (93.4%) outperforms iterative refinement (86.6%) with the same compute; (4) beneficial self-correction requires a two-tier capability: EIR suppression (achievable via prompting) prevents degradation, while ECR enhancement (likely requiring RL training) enables actual improvement. A. Limitations and Future Directions Scope. Our detailed refinement analysis covers seven models on GSM8K with 4 iterations each. Extending to MATH and StrategyQA refinement would further test crossdomain generalization of the convergence modes. It would
also clarify whether the near-zero EIR threshold remains stable across tasks with different answer formats, verifier difficulty, and intrinsic ambiguity. Non-stationary rates. The Markov chain model assumes rates approach stationarity. We observe non-stationarity (EIR increases from 1.3% to 3.8%), suggesting time-varying models may be more appropriate. A natural next step is a controltheoretic formulation with iteration-dependent transition dynamics and adaptive stopping thresholds. External feedback. We evaluate intrinsic self-correction (no external oracle). Huang et al. [5] show that external feedback can make self-correction effective. Incorporating tool outputs, retrieval results, or verifier models could shift the EIR/ECR balance favorably. In control terms, such signals may provide the exogenous correction needed to move the loop from self-referential drift toward error attenuation. Domain generality. Our evaluation focuses on mathematical and factual reasoning with clear correctness criteria. Extending to open-ended generation requires different evaluation frameworks. It would also require redefining EIR/ECR with graded or preference-based notions of correctness rather than exact-match labels. R EPRODUCIBILITY We provide anonymous supplementary material with per-iteration trajectories, change logs, EIR/ECR estimates, verify-first outputs, and the ASC reference implementation. All table values and hypothesis tests can be reproduced from the supplied CSVs with a single driver script. R EFERENCES [1] A. Madaan et al., “Self-Refine: Iterative Refinement with SelfFeedback,” in NeurIPS, vol. 36, 2023. [2] N. Shinn et al., “Reflexion: Language Agents with Verbal Reinforcement Learning,” in NeurIPS, vol. 36, 2023. [3] J. C.-Y. Chen et al., “MAgICoRe: Multi-Agent, Iterative, Coarse-toFine Refinement for Reasoning,” in Proc. EMNLP, pp. 32663–32686, 2025. [4] Y. Du, S. Li, J. B. Tenenbaum, and I. Mordatch, “Learning Iterative Reasoning through Energy Minimization,” in ICML, pp. 5570–5582, 2022. [5] J. Huang et al., “Large Language Models Cannot Self-Correct Reasoning Yet,” in ICLR, 2024. [6] Z. Yang, Y. Zhang, Y. Wang, Z. Xu, J. Lin, and Z. Sui, “A Probabilistic Inference Scaling Theory for LLM Self-Correction,” in Proc. EMNLP, 2025. [7] X. Wang et al., “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” in ICLR, 2023. [8] K. Cobbe et al., “Training Verifiers to Solve Math Word Problems,” arXiv preprint arXiv:2110.14168, 2021. [9] D. Hendrycks et al., “Measuring Mathematical Problem Solving With the MATH Dataset,” in NeurIPS Datasets and Benchmarks, 2021. [10] M. Geva et al., “Did Aristotle Use a Laptop?,” TACL, vol. 9, pp. 346– 361, 2021. [11] J. Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” in NeurIPS, vol. 35, pp. 24824–24837, 2022. [12] T. Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools,” in NeurIPS, vol. 36, 2023. [13] D. Hendrycks et al., “Measuring Massive Multitask Language Understanding,” in ICLR, 2021. [14] R. Kamoi, Y. Zhang, N. Zhang, J. Han, and R. Zhang, “When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs,” TACL, vol. 12, pp. 1417–1440, 2024. [15] K. Stechly, K. Valmeekam, and S. Kambhampati, “On the SelfVerification Limitations of Large Language Models on Reasoning and Planning Tasks,” in ICLR, 2025.
[16] Z. Gou, Z. Shao, Y. Gong, Y. Shen, Y. Yang, N. Duan, and W. Chen, “CRITIC: Large Language Models Can Self-Correct with Tool-
Interactive Critiquing,” in ICLR, 2024.