Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents Yitao Wu1 , Si Shen2 , Rui Yang3∗ , Hong Peng3 , Bin Hu3 1
saofund.ai, Shenzhen, China Shenzhen Xingqing Zhiti Technology Co., Ltd., Shenzhen, China 3 School of Information Technology and Engineering, Lanzhou University, Lanzhou, China [email protected], [email protected], [email protected], [email protected], [email protected] 2
arXiv:2607.17641v1 [cs.AI] 20 Jul 2026
Abstract Verify-repair loops are a standard means for large language model (LLM) agents to correct faulty plans in code generation, mathematical reasoning, and tool use. When both the verifier and the repairer are noisy, repair can damage already-correct plans, and reported acceptance keeps rising while true validity falls, so existing methods lack a principled basis for deciding when repair should stop. We propose VRR-Stop, a robust stopping framework for noisy verify-repair-repeat (VRR) loops. A four-parameter noise model separates verifier false acceptance and false rejection from the repair and damage behavior of the repairer. Belief filtering turns repeated verification votes into an estimate of committed validity, and the loop commits or repairs according to the sign of the true marginal gain, which requires only sign identifiability rather than accurate recovery of all parameters. When verifier discrimination approaches zero, calibration itself fails and estimation error can flip the stopping sign, so we pair VRR-Stop with VRR-Guard, an estimation-free fallback that replaces the incumbent candidate only under a sufficient verification margin. On a GSM8K stress setting, VRR-Stop improves final true validity by 60.6 percentage points over fixed five-round repair at an average cost of 0.72 repair rounds. Across settings, stopping reliability is governed jointly by verifier discrimination and the decision margin rather than by the absolute size of estimation error.
Code — anonymous.4open.science/r/vrr-artifact-2583
1
Introduction
On complex multi-step tasks such as code generation, tool invocation, and web automation, a single plan generated by an LLM agent often contains logical gaps, constraint violations, or failing tool calls, and once such errors enter execution they can derail the entire task. Verify-repair loops are therefore widely adopted, in which a verifier checks the candidate plan, a repairer produces a revision from the feedback, and the loop iterates toward a valid solution. A large body of work shows that this feedback mechanism corrects explicit errors and significantly improves plan executability and reasoning quality (Madaan et al. 2023; Shinn et al. 2023; Lightman et al. 2024). In practice, however, the paradigm implicitly relies on two premises that do not always hold, namely that multi-round repair tends to improve true quality and that verifier outputs ∗
Corresponding author.
(a) Clean feedback Figure legend
User
Initial plan p0 (valid)
Accept (correct)
Commit p0 (valid)
User
Agent (Planner)
Agent (Verifier)
Plan
Agent (Repairer)
(b) Noisy feedback
User
Initial plan p0 (valid)
Reject (false reject)
Repair damages (valid → invalid) (probability β)
Accept (false accept)
Final output p1 (invalid)
Figure 1: Clean feedback accepts and commits the valid plan p0 , whereas noisy feedback can damage it into an invalid p1 . Verifier and repair errors can therefore compound, making continued repair harmful.
suffice to reflect true validity. When both the verifier and the repairer are noisy, the two premises fail simultaneously, turning the gains of the loop into an unpredictable systemic risk. Figure 1 illustrates a typical instance of this mismatch. A truly valid initial plan p0 can be falsely rejected by the verifier under noisy feedback, turned into an invalid plan p1 by a harmful repair, and finally falsely accepted by the verifier. Verifier errors and repairer errors thus compound inside the loop, so true quality can decline over rounds instead of improving. Prior studies likewise observe that proxy signals in self-repair, LLM judges, and inference-time search can decouple from true quality (Huang et al. 2024; Kamoi et al. 2024a; Pan et al. 2024a,b; Khalaf et al. 2025). Verification pass rates therefore cannot be equated with the true validity of plans, and multi-round repair does not guarantee monotone improvement. The central question of this paper is therefore how, in agent systems that rely on verify-repair loops, one can decide when repair has degenerated from improving true quality into fitting verifier noise. Existing feedback-driven research can be examined from three directions with respect to this stopping problem. Selfcorrection and reflective feedback let the model revise its output from its own feedback, verbal reflection, or the environment (Madaan et al. 2023; Shinn et al. 2023; Kamoi et al. 2024b), yet verifier feedback and repairer behavior remain entangled in the loop, and pass-rate fluctuations cannot be attributed to either source. How to separate verifier noise
from repairer noise, and to characterize how true validity evolves with repair rounds, is a key loop-noise modeling problem. Process supervision, external verifiers, and generative reward models constrain intermediate reasoning with explicit checking signals (Cobbe et al. 2021; Lightman et al. 2024; Zhang et al. 2025a; Khalifa et al. 2026), but their effectiveness presupposes reliable feedback. When the feedback itself is noisy, how to estimate parameters from weak supervision while guaranteeing that estimation error does not turn “should stop” into “should continue” is a difficult stopping identifiability problem. Studies of test-time search and proxy-signal optimization show that imperfect verifiers, false acceptance, self-bias, and reward over-optimization can limit or even reverse inference-time gains (Gao, Schulman, and Hilton 2023; Stroebl, Kapoor, and Narayanan 2026; Yu, Li, and Wang 2025; Dorner et al. 2026; Zhou et al. 2025; Lu et al. 2025). How to detect such failure in time, and to design a conservative stopping mechanism that does not rely on accurate parameter estimation so that continued repair no longer causes harm, is an urgent safety fallback problem. Targeting the basic dilemma that continued repair does not always improve true quality, we start from the three problems of loop-noise modeling, stopping identifiability, and safety fallback, and propose VRR-Stop, a robust stopping framework for noisy verify-repair-repeat (VRR) loops. The contributions of this paper are as follows. (i) We propose a four-parameter loop-dynamics model that explicitly separates verifier false acceptance, verifier false rejection, and the repair and damage behavior of the repairer, thereby characterizing how the true validity of plans evolves over multiple repair rounds. (ii) We derive a stopping criterion driven by the sign of the true marginal gain, and give a weakly supervised estimation method based on multi-round verification records and a small number of labeled transition samples. The stopping decision requires only that the gain sign be identifiable rather than accurate recovery of all parameters. (iii) We give an explicit criterion for stopping-sign identifiability, show that decision reliability is jointly determined by the verifier discrimination J and the decision margin ∆, and empirically expose this boundary in low-discrimination verifier scenarios. (iv) We design VRR-Guard, a conservative fallback that does not rely on accurate parameter estimation and avoids fixed-round-repair-style degradation under sign unidentifiability or distribution shift.
2
Related Work
Iterative self-correction. Self-Refine revises outputs with self-generated feedback, Reflexion stores verbal feedback as experience, CRITIC obtains verifiable critiques from external tools, and SCoRe trains self-correction with reinforcement learning (Madaan et al. 2023; Shinn et al. 2023; Gou et al. 2024; Kumar et al. 2025). Follow-up studies find that, without reliable external signals, models may fail to recognize their own errors and can corrupt previously correct answers (Huang et al. 2024; Kamoi et al. 2024a,b). This line focuses on producing better feedback and revisions, but does not formalize termination as a decision problem in which the verifier and the repairer are simultaneously noisy. VRR-Stop separates verifier false acceptance and false rejection from
repair and damage behavior, and stops according to the true marginal gain of continued repair. Verifier-guided inference and proxy over-optimization. Verifier-guided methods score candidates with outcome verifiers, process reward models, or generative verifiers for candidate selection, search, or further reasoning (Cobbe et al. 2021; Lightman et al. 2024; Zhang et al. 2025a; Khalifa et al. 2026), with recent work scaling sampling and verification budgets (Zhao, Awasthi, and Gollapudi 2025; Zhong et al. 2025). The gains are bounded by verifier coverage, ranking error, and ROC characteristics (Stroebl, Kapoor, and Narayanan 2026; Yu, Li, and Wang 2025; Dorner et al. 2026; Lu et al. 2025), and proxy scores in feedback loops decouple from true quality through training- and inference-time reward hacking (Pan et al. 2024a,b; Gao, Schulman, and Hilton 2023; Khalaf et al. 2025). HedgeTune (Khalaf et al. 2025) tunes the operating point of selection-style mechanisms, and ROC-n-reroll (Dorner et al. 2026) analyzes how imperfect verifiers limit resampling. Unlike these selection-style mechanisms over independent candidates, a verify-repair loop keeps rewriting the same candidate, so the repairer can both fix invalid plans and damage valid ones. VRR-Stop targets the state transitions and stopping decision of this path-dependent rewriting rather than re-establishing proxy over-optimization itself. Self-correction dynamics and reliable stopping. A probabilistic theory of self-correction describes accuracy converging to a fixed ceiling at a single rate (Yang et al. 2025), which cannot express the interior peaks and late-stage decline caused by joint verifier and repairer noise. Unlabeled accuracy estimation infers judge reliability from multiple classifiers (Platanios, Blum, and Mitchell 2016), Youden’s J measures how well a judge preserves class differences (Collot et al. 2026), and uncertainty-aware process verification flags unreliable reward-model steps (Ye et al. 2025). More recent work treats stopping itself as a test-time decision dimension, via adaptive early stopping and confidence-based trace filtering for chain-of-thought reasoning (Sun et al. 2026; Fu et al. 2026) or statistically valid evaluation under imperfect judges (Feng et al. 2026), but these methods address singlepass generation or independent sampling and do not model the true-state transitions induced by repair. VRR-Stop uses Youden’s J for stopping identifiability rather than static judge ranking, and switches to the guarded keep-best fallback when the sign is unidentifiable.
3 3.1
Preliminaries and Motivation
Verify-Repair Loops
We consider a canonical verify-repair loop. Given a task instance, the agent first generates an initial plan p0 . In round k, the system issues M independent verification queries on (m) the current plan pk ; each query returns a binary signal ak ∈ {0, 1} (1 for accept), yielding the acceptance count Sk = PM (m) ∈ {0, . . . , M }. Based on Sk and the history, m=1 ak the system updates the committed validity bk = Pr(yk = 1 | Hk ) and decides whether to commit the current plan or to repair it. The observables and the decision variable of a
single round are given in Eq. (1): XM (m) Sk = a , π(Hk ) ∈ {Commit, Repair}, m=1 k (1) where π(Hk ) is the stopping policy over the observable history Hk . If π(Hk ) = Commit, the loop terminates and commits pk ; otherwise the repairer produces the next plan pk+1 from the current plan and the verification feedback. To separate observable verifier outputs from unobservable true quality, let yk ∈ {0, 1} denote the true validity of pk , i.e., whether the plan actually satisfies the task constraints, is executable, and completes the goal. The observable history up to round k is defined in Eq. (2): Hk = (p0 , S0 , . . . , pk , Sk ),
yk ∈ {0, 1}.
(2)
Since yk is generally unavailable at deployment, the stopping policy must act on Hk alone. An accepted plan is therefore not necessarily valid, and a rejected plan is not necessarily invalid.
3.2
validity, and stop when it no longer exceeds a minimum-gain threshold, as specified in Eq. (4): Gk = E[yk+1 − yk | Hk , continue] , Repair, Gk > τ, π(Hk ) = Commit, Gk ≤ τ,
Noise Model
Observed signals do not equal true validity, and repair is not a monotone improvement process. To capture both kinds of uncertainty, we adopt a compact noise model, given in Eq. (3): (m)
ρ0 = Pr(ak
= 1 | yk = 0),
(m) ρ1 = Pr(ak = 0 | yk = 1),
α = Pr(yk+1 = 1 | yk = 0, repair), β = Pr(yk+1 = 0 | yk = 1, repair), J = 1 − ρ0 − ρ 1 ,
(3)
where ρ0 is the probability that an invalid plan is falsely accepted, ρ1 the probability that a valid plan is falsely rejected, α the probability that repair turns an invalid plan valid, and β the probability that repair damages a valid plan. J is the discrimination ability of the verifier. As J approaches 0, acceptance signals can no longer support fine-grained stopping decisions. The model treats repeated queries on the same plan as conditionally independent observations given the true state, and treats ρ0 , ρ1 , α, β as stable within a local decision window. The former supports Bayesian updates from acceptance counts; the latter lets the one-step marginal gain be characterized by compact parameters. The conditional-noise assumption also ignores instance-level difficulty heterogeneity. The same verifier may err at different rates on different plans, and repeated queries may be correlated; a beta-binomial estimator diagnoses such within-class heterogeneity (Appendix E). Both assumptions are local approximations rather than global stationarity claims; their applicable range under non-stationarity is analyzed in the experiments.
3.3
Figure 2: The population verification rate Āt can keep increasing while the true-validity rate Qt peaks and then declines. The shaded interval marks ∆Qt < 0, where continued repair becomes harmful.
Stopping Objective
The question is when, given Hk , the system should repair once more rather than commit. We define the true marginal gain of one more repair round as the expected change in true
(4)
where τ ≥ 0 converts extra computation into a minimum acceptable validity gain or acts as a conservative margin; τ = 0 when cost is ignored. Eq. (4) adopts the one-step gain as the stopping target, comparing only a commit now against exactly one more repair round, without look-ahead over multi-round trajectories. Since yk is unobservable at deployment, Eq. (4) only defines the target; estimating Gk is deferred to the method section.
3.4
Motivation
More repair is not always beneficial. When the damage probability is non-negligible, simply adding repair rounds can push true quality past a peak into sustained decline. Iterative refinement methods such as Self-Refine, Reflexion, and CRITIC assume that outputs improve over rounds of feedback (Madaan et al. 2023; Shinn et al. 2023; Gou et al. 2024). Let Qt = Pr(yt = 1) be the population true-validity rate (subscript t indexes population-level round dynamics, distinct from the per-instance index k). If every round triggers repair, then Qt+1 = Qt (1 − β) + (1 − Qt )α, so whenever Qt is already high or β is non-negligible, Qt+1 − Qt ≤ 0 and repair noise accumulates over rounds. As Fig. 2 shows, Qt enters a declining regime after t⋆ , the round at which the marginal gain turns negative. This risk is not a corner case under extreme parameters. Self-correction is known to corrupt correct answers (Huang et al. 2024; Kamoi et al. 2024a), and our measured repair trajectories reach β close to 0.94. Multi-round repair therefore cannot be treated as a safe default; an explicit stopping mechanism is required. Verifier signals can improve without true validity improving. Treating verifier pass rates as a quality metric makes the system overestimate the benefit of repair (Zhao, Awasthi, and Gollapudi 2025; Zhong et al. 2025; Khalifa et al. 2026). With Āt = E[St /M ], the population verification rate satisfies Āt = ρ0 + (1 − ρ0 − ρ1 )Qt , where 1 − ρ1 and ρ0 are the verifier true- and false-positive rates. When J = 1 − ρ0 − ρ1
is low or the noise parameters drift, the system can exhibit Āt+1 − Āt > 0 while Qt+1 − Qt ≤ 0: the dashed curve in Fig. 2 keeps rising while true validity already falls. Verifier calibration can degrade quickly after switching models, verifiers, or task distributions, so optimizing pass rates alone does not guarantee quality gains; verifier noise and true validity must be modeled separately. Stopping requires identifiable evidence. Heuristic stopping rules such as fixed round budgets, consecutive-pass counts, or score thresholds cannot reliably judge whether repair is still worthwhile. Existing test-time scaling and verifier-guided methods focus on exploiting more sampling, verification, or repair, and rarely model whether the parameters required for stopping are identifiable under weak supervision. What the decision actually needs is the sign of ∆Qt = Qt+1 − Qt , while the available signals are only the acceptance counts {S0 , . . . , St } and a few labeled repair dt , the systransitions. If estimation error flips the sign of ∆Q tem mistakes “should stop” for “should continue” and enters a regime of sustained damage. Class balance, verifier calibration, and Youden’s J all affect judge reliability, and accuracy estimation without labels itself requires assumptions (Collot et al. 2026; Platanios, Blum, and Mitchell 2016). A stopping policy should therefore be near-optimal when evidence suffices and fall back to a conservative decision when it does not.
4
Method
Figure 3 gives an overview. Given the verify-repair history Hk and one-off weakly supervised calibration data, the framework estimates the committed validity of the current plan round by round and outputs commit or repair. It consists of a belief estimator that turns the M verification votes of each round into the posterior bk (stage one), a stopping criterion that forms the marginal gain Gk and acts on its sign (stages two and three), and the conservative fallback VRR-Guard that takes over commitment when calibration is unreliable (stage four). The three components address, in order, the loop-noise modeling, stopping identifiability, and safety fallback problems raised in the introduction.
4.1
Belief Filtering and the Stopping Boundary
Given Hk , the committed validity bk = Pr(yk = 1 | Hk ) is the posterior probability that the current plan is truly worth committing; it is not the verification pass rate, which can rise merely through false acceptance. The belief evolves by an update–predict recursion. The initial belief is the generator prior b− 0 = π̂, estimated on the calibration folds as the true-validity rate of initial plans. In round k, the M queries are conditionally independent given yk , so Sk is binomial and the observation update is given in Eq. (5): bk =
Sk M −Sk b− k (1 − ρ1 ) ρ1
, Sk M −Sk + (1 − b− )ρSk (1 − ρ )M −Sk b− 0 0 k (1 − ρ1 ) ρ1 k (5) where b− k is the predictive belief before observing this round’s votes; the binomial coefficients cancel. If repair is
executed, the true state transitions by α, β and the predict step is b− k+1 = (1 − β)bk + α(1 − bk ). The recursion relies on one explicit assumption, namely that the repair transition is conditionally independent of Hk given yk . Although plans entering repair are filtered by verification signals, their transitions are still governed by α, β; the labeled transition samples are collected from frozen trajectories in which every round triggers repair, consistent with this assumption. Subtracting the current belief from the predictive belief gives the true marginal gain of one more repair round, and setting it to zero yields the critical posterior belief, as in Eqs. (6)–(7): Gk = b− k+1 − bk = (1 − bk )α − bk β,
(6)
b⋆ = α/(α + β),
(7)
α + β > 0,
where the first term of Eq. (6) is the expected gain from fixing an invalid plan and the second the expected loss from damaging a valid one. b⋆ is exactly the fixed point of the population dynamics Qt+1 = Qt (1 − β) + (1 − Qt )α of the motivation subsection. Repair can only push validity toward b⋆ , and once the belief crosses it, the expected direction of further repair is downward. The belief space thus splits into a repairbeneficial regime (bk < b⋆ ) and a repair-harmful regime (bk ≥ b⋆ ). The boundary depends only on the repairer’s α and β; verifier noise ρ0 , ρ1 does not move the boundary but determines, through bk , whether the comparison is reliable (next subsection). Stopping follows the one-step objective of Eq. (4) with Gk from Eq. (6). All experiments use τ = 0; a nonzero threshold shifts the boundary to b⋆τ = (α − τ )/(α + β), and no repair is issued if α ≤ τ . The rule ties the stopping time to a direct comparison between repair benefit and damage risk rather than to a preset budget or pass-rate threshold. When the round budget Kmax is exhausted, the current plan is committed.
4.2
Calibration and Sign Identifiability
At deployment, none of bk , ρ0 , ρ1 , α, β is directly observable. Calibration uses two kinds of weak supervision. For repeated verification records, per-plan acceptance counts are modeled as a two-component binomial mixture whose components correspond to truly valid and invalid plans; a binomial-mixture expectation-maximization (EM) estimator recovers the mixture weight and ρ̂0 , ρ̂1 without labels. For repair transitions, α̂, β̂ are frequency estimates from at most 300 labeled before-after plan pairs, with labels produced by each task’s validity-judging rule. Substituting the estimates into Eq. (6) gives Ĝk = (1 − b̂k )α̂ − b̂k β̂, with b̂k computed from Eq. (5) under ρ̂0 , ρ̂1 , so calibration error propagates into Ĝk . Let the error radius Bk satisfy Pr(|Ĝk − Gk | ≤ Bk ) ≥ 1 − η, where η is a userchosen tolerance; Bk shrinks as the calibration sample size, the verification budget M , and the discrimination J grow. The sign-identifiability condition is given in Eq. (8): Ĝk − Bk > τ
or Ĝk + Bk ≤ τ.
(8)
1 Committed validity
Task & loop context
estimation
2
Marginal gain characterization
3
Stopping criterion
Visual boundary (optimal point)
"Create a tool-use plan to schedule a meeting."
History
Current plan
Repair success (gain)
Planner
(
Two outcomes of another repair Repair success (gain)
not identifiable STOP
GO
4
Planner
LLM planner generates
Stop Point (Optimal Boundary)
Guarded fallback
(if calibration unreliable)
VS Marginal gain (one more repair)
Decision rule
Current Candidate (current plan)
Verify
LLM verifier history
Current Plan Validity
[1] Check availability [2] Reserve room
above the stop point
[3] Create calendar event Repairer
0%
100%
Likelihood that the plan will succeed if committed.
Initial plan
Best Champion So Far (champion)
If the current plan's quality is: Verifier votes (repeat
[4] Send confirmation
commit (identifiable stop)
Is the new plan better?
writes an initial tool-use plan Verifier
Final commit
identifiable stop , identifiable)
(true valid)
Repair gain (benefit)
Damage risk (cost)
Marginal Gain = Repair Gain - Damage Risk
Verifier
Verifier
Verifier
commit (fallback)
times) Verifier
STOP below the stop point
CONTINUE
calibrated from weak supervision
LLM repairer
Replace champion only when current plan receives significantly more votes (margin ).
continue & reliable calibration (next repair round)
Figure 3: Overview of VRR-Stop, which estimates committed validity bk and stops according to the marginal gain Gk . VRRGuard is evaluated separately as a conservative fallback when calibration is unreliable. Proposition 1 (Stopping-sign consistency) If Eq. (8) holds, the action taken under Ĝk agrees with the action under the true Gk with probability at least 1 − η. The proof and a sample-complexity bound are in Appendix A; the guarantee presupposes that Bk attains its nominal coverage. Writing the decision margin as ∆k = |Gk −τ |, whether the premise of Proposition 1 can be met is governed jointly by J, the calibration sample size, and ∆k . As J → 0 or ∆k → 0, the required evidence grows rapidly, and nearzero J can defeat even a large margin because the posterior itself cannot be located reliably. In the experiments, VRRStop acts directly on the sign of Ĝk ; Eq. (8) and Proposition 1 serve as reliability-analysis tools for the flip-risk evaluation in the experiments.
4.3
Guarded Keep-Best Fallback
When a separation test on held-out labeled samples reports Jˆ near zero, or Eq. (8) fails throughout the evaluation window, calibrated stopping is no longer trustworthy and VRR-Guard should be used instead. The test compares acceptance rates of valid and invalid plans directly on labeled samples and does not depend on the binomial-mixture EM, avoiding diagnosing a possibly broken estimator with its own output. We partition evaluation scenarios offline by this criterion rather than deploying an online threshold switch. VRR-Guard maintains the incumbent best candidate ck , initialized as c0 = p0 . For any plan p, let S(p) = PM (m) (p) be its acceptance count, so Sk = S(pk ). m=1 a Given a retention margin δ, the retention rule is given in Eq. (9): pk , S(pk ) ≥ S(ck−1 ) + δ, ck = (9) ck−1 , otherwise, and the incumbent ck , not the last plan, is committed at termination. Lemma 1 (Bound on erroneous replacement) Suppose the incumbent c is truly valid and the new plan p
truly invalid, queries are conditionally independent given true states, and J ≥ 0. Then the perround erroneous-replacement probability satisfies Pr[S(p) ≥ S(c) + δ] ≤ exp − (M J + δ)2 /(2M ) , and the probability of at least one such replacement within Kmax rounds is at most Kmax times this bound. The lemma follows from a Hoeffding bound on the difference of two independent binomials (proof and exact perverifier tail probabilities in Appendix A), showing that erroneous replacement decays exponentially in δ and M J and giving the margin an analytic basis. The bound is conservative; with M = 8, the replay ablation of Appendix I selects δ = 5 as the most robust setting. Because Eq. (9) compares vote counts across rounds, it implicitly assumes within-trajectory stationarity of verifier noise; degradation Clean feedback Noisy feedback Correct accept False experiments. reject under drift is tested in the shift VRR-Guard does not strictly dominate no-repair. It forfeits part of the achievable gain in repair-beneficial regimes in exchange for near-no-repair downside protection when calibration is unLLM repairer rewrites p0 reliable. Harmful repair Pseudocode integrating both stopping modes appears in Accepted p0 Final output Correctcoraccept Appendix A, where the mode-selection threshold Jmin responds to the held-out separation test above; scenarios are Correct final output if rejected, partitioned offline without tuning J online. Each round min repair again costs M verifier calls plus a constant-time belief update, and repair calls occur only on Repair. Calibration is a one-off cost Damaged plan p1 using repeated verification records Verify againand at most 300 labeled transitions. Hyperparameters are the verification budget M , False repair gain threshold τ , retention margin δ, and budget Wrong final outputKmax . accept Repairer
[1] Check availability [2] Reserve room
[3] Create calendar event [4] Send confirmation
[1] Check availability [2] Reserve room
[3] Create calendar event
- verifier accepts - true validity
[4] Send confirmation
Verifier
- verifier accepts
5
LLM verifier checks plan p1
- true validity
Experiments
We organize the evaluation around four research questions. RQ1: Does multi-round verify-repair monotonically improve true validity, or does reported acceptance rise while true validity falls? RQ2: When repair can be harmful, does VRR-Stop avoid the degradation of heuristic stopping and approach the true-parameter myopic reference? RQ3: With limited calibration samples and varying verifier discrimina-
tion, when is the stopping sign identifiable, and how does estimation error flip the decision? RQ4: When calibration fails or the distribution shifts, does the conservative fallback retreat to near no-repair instead of collapsing?
Setup
Tasks, models, and protocol. The evaluation covers mathematical reasoning (GSM8K (Cobbe et al. 2021), MATH-500 (Hendrycks et al. 2021; Lightman et al. 2024)), code generation (MBPP (Austin et al. 2021)), and tool use (BFCL (Patil et al. 2025)); true validity is judged by answer matching, symbolic verification, unit tests, and an executor, respectively. Generators and verifiers span the Qwen2.5 (Yang et al. 2024), Mistral (Jiang et al. 2023), and Llama (Grattafiori et al. 2024) families. End-to-end evaluation uses Kmax = 5 repair rounds and M = 8 verification queries per round; the non-stationary loop-dynamics diagnostic runs to round 6. All calibration parameters are estimated with five-fold cross-fitting, so no instance’s stopping decision touches its own label; the full protocol and model configurations are in Appendix B. Metrics and baselines. The primary metric is the true validity V of the final commit (95% bootstrap confidence intervals from B=10,000 resamples) and the mean repair rounds ⟨K⟩; verifier reliability is described by ρ0 , ρ1 and Youden’s J = 1 − ρ0 − ρ1 ; paired differences use exact McNemar tests and instance-level bootstrap. Baselines fall into three groups. Fixed-budget repair corresponds to the common configurations of Self-Refine, Reflexion, CRITIC, and SCoRe (Madaan et al. 2023; Shinn et al. 2023; Gou et al. 2024; Kumar et al. 2025). Heuristic stopping relies only on verification signals and includes majority stopping and confidence-threshold stopping (Wang et al. 2023). Reference policies include no-repair, the true-parameter myopic (TPM) reference, and hindsight best-round selection, the latter two serving only as diagnostics. Formal definitions are in Appendix A.
5.2
1.00
Probability
5.1
Repair loop: true validity Resample control: true validity Verifier acceptance
Loop Dynamics
Addressing RQ1, Fig. 4 shows that, under a 256-token initial budget, validity starts at 0.45, rises to 0.87 after two repair rounds, then collapses after prompt mismatch is injected at round 3, reaching only 0.12 at round 6, which forms an interior optimum at K ⋆ = 2, whereas an independentresampling control with the same budget stays between 0.47 and 0.85 throughout. The paired gain of the peak round over the final round is 74.7 percentage points (95% CI [+69.3, +79.7]), and the control exhibits no comparable path-dependent decline, so the degradation stems from repeated rewriting of the same candidate rather than sampling noise. Non-stationary loops thus contain an intrinsic boundary where the gain turns negative; reliable stopping depends on identifying the gain sign in time, not on preset budgets. The pattern is not specific to this diagnostic. Across eight settings, six exhibit monotone decline with damage probabilities between 0.615 and 0.938, BFCL multi-turn stays flat under a near-inert repair operator, and only the favorable setting improves, so harmful repair is the rule rather than the exception under stress. A stronger verifier does not remove
K ⋆ =2
0.75 0.50 0.25 0.00
0
1
2
3
4
5
6
Repair round k
Figure 4: Loop dynamics under the non-stationary stress setting (N = 300, prompt mismatch from round 3). True validity peaks at K ⋆ = 2 and then collapses, while independent resampling shows no comparable decline. the risk. With a process-reward-model verifier of J = 0.805 on GSM8K, fixed five-round repair still drives validity from 0.727 to 0.097, and on MATH-500 (J = 0.77) from 0.798 to 0.150 while the TPM reference holds 0.798, so verifier quality strengthens stopping reliability but cannot eliminate repair damage (Appendix H). On the N = 500 stress traces, 55% of instances see a correct plan repaired into an incorrect one, and 24% of these damaging repairs win majority acceptance, raw-label statistics that are independent of any stopping rule (Appendix C).
5.3
Stopping Performance
For RQ2, Table 1 evaluates end-to-end gains on the GSM8K / Qwen2.5-3B stress setting. Fixed five-round repair degrades the no-repair baseline from 0.700 to 0.116, whereas VRR-Stop reaches 0.722 with only 0.72 repair rounds on average. Against fixed five-round repair, the paired gain is 60.6 percentage points (95% CI [+56.0, +65.0], McNemar p < 2 × 10−85 ); VRR-Stop also beats majority stopping (+3.2 pp, p < 2 × 10−4 ) and the TPM reference (+2.8 pp, p < 6 × 10−4 ), while its difference from no-repair (+2.2 pp) has a CI crossing zero. Under the fixed-budget deployment mode of iterative-feedback baselines, Reflexion and Self-Refine end at 0.095 and 0.080 on this stress setting, while VRR-Stop on the same batches of trajectories reaches 0.740 and 0.710. The ordering replicates across sample sizes and domains, with three disjoint GSM8K windows giving 0.752±0.037 against the reference 0.757±0.058 and Qwen7B stress retaining 0.875 by learning to never repair. Calibrated stopping thus terminates the loop in time when repair is harmful, preserving validity at lower cost than heuristic voting; all remaining baselines and paired tests are in Appendices D and G.
5.4
Calibration and Identifiability
For RQ3, we sweep a controlled J–∆ grid (∆ is the decision margin of the calibration subsection) and evaluate four cross-family verifiers on GSM8K, with N = 300 calibration samples and M = 8 queries per plan. Figure 5 shows that
Table 1: Stopping performance on the GSM8K / Qwen2.5-3B prompt-mismatch stress setting (N =500, M =8, Kmax =5), abbreviated; the TPM reference applies the same stopping rule with ground-truth parameters as a diagnostic.
Table 2: Robustness under calibration failure and distribution shift, replayed on saved trajectories with M = 8 and δ = 5; shift codes M, V, T denote model, verifier, and task shift. J
Setting True validity V [95% CI]
⟨K⟩
No repair Majority stopping ConfStop-0.85 Fixed repair K=5 VRR-Stop
0.700 [.658,.740] 0.690 [.648,.730] 0.562 [.518,.604] 0.116 [.088,.144] 0.722 [.682,.760]
0.00 0.92 1.92 5.00 0.72
TPM reference
0.694 [.652,.734]
0.89
Method
0.6
0.5 synthetic grid limit
0.4
0.4 Qwen-7B: safe (beyond grid)
0.3
0.3
0.2
0.2 Mistral
0.1 0.0 0.0
Llama: collapsed (J-driven)
Pr[sign flip]
Youden index J
0.5
Qwen-3B
0.1
0.0
0.2
0.4
0.6
0.8
decision margin ¢ = jG(¼)j
Figure 5: Stopping-sign flip probability over J and ∆ under the deployed rule; markers denote real verifiers. Flip risk concentrates at low J and low ∆, and near-zero J fails even with a large margin (Llama). flip risk concentrates where low J meets low ∆. The flip probability reaches 0.183 for J ≤ 0.15 and ∆ ≤ 0.10, versus 0.014 for J ≥ 0.4 or ∆ ≥ 0.30, a roughly 13-fold gap. Among the real verifiers, calibrated stopping stays within 2.8 percentage points of the TPM reference for Qwen-3B, Qwen7B, and Mistral (all J ≥ 0.18), whereas Llama has J of only 0.03. Despite a decision margin of about 0.74, its validity collapses from the reference 0.803 to 0.223 (−58.0 pp, 95% CI [−63.7, −52.0], McNemar p < 2 × 10−48 ). The failure mechanism is that the likelihood surface of the binomialmixture EM flattens as J → 0. With N = 120 the estimate is ρ̂1 = 0.27, yet enlarging the sample to 300 worsens it to 0.077 against a true 0.609, and a parameter sweep shows calibrated stopping recovers only for ρ̂1 ≳ 0.30. Stopping reliability is thus determined by whether finite evidence can stabilize the gain sign, not by the absolute size of parameter error; near-zero J mandates abandoning fine-grained calibration for the conservative fallback (Appendix E).
5.5
Guarded Fallback under Shift
For RQ4, Table 2 examines calibration failure and distribution shift. Fixed five-round repair falls far below no-repair in most shifted settings (e.g., Qwen-7B drops from 0.875 to 0.075), whereas VRR-Guard stays near the no-repair level
None Fixed 5 Calib. Guard
Qwen-3B fav. (–) 0.39 0.740 Qwen-3B stress (–) 0.46 0.700 Qwen-7B stress (M) 0.23 0.875 Mistral-7B stress (V) 0.18 0.507 Llama-3-8B stress (M) 0.03 0.803 MATH-500 (T) 0.22 0.798 BFCL single (T) 0.07 0.812
0.875 0.116 0.075 0.047 0.057 0.150 0.372
0.845 0.722 0.875 0.463 0.223 0.798 0.782
0.810 0.742 0.875 0.487 0.793 0.796 0.810
in all seven settings and never collapses. In the Llama setting where calibrated stopping fails, VRR-Guard recovers to 0.793, exceeding the failed VRR-Stop by 57.0 percentage points (95% CI [+51.0, +62.7], McNemar p < 1 × 10−47 ) and fixed five-round repair by 73.7 points. Its conservatism costs about 2.0 and 0.3 pp against no-repair on Mistral and BFCL (the latter’s paired CI contains zero), and its 0.810 in the favorable setting trails fixed five-round repair’s 0.875. Against margin-free verifier-best selection, the retention margin contributes +45.0 pp in the Llama scenario and between −4.0 and +8.0 pp elsewhere, concentrating its value where verification signals are least trustworthy. The value of VRRGuard is to pull the system back to a near-no-repair safe state when the sign is unidentifiable, not to dominate everywhere; full results and paired tests are in Appendices F and G. Ablations of the calibration input, the retention margin δ, and the calibration sample size corroborate these design choices, with binomial EM the best input, δ = 5 most robust, and Ncalib = 300 sufficient (Appendix I).
6
Conclusion
This paper addresses the stopping decision in LLM verifyrepair loops, whose crux is that the paradigm implicitly assumes multi-round repair improves true quality and verification signals represent true validity, and both assumptions fail together when the verifier and the repairer are noisy. We propose VRR-Stop, which builds on a four-parameter noise model and couples belief filtering with a marginal-gain sign criterion, turning stopping into a calibratable, identifiable sign decision, with the fallback VRR-Guard covering the unidentifiable regime. On the GSM8K / Qwen2.5-3B stress setting, VRR-Stop improves final true validity by 60.6 percentage points over fixed five-round repair and slightly exceeds the true-parameter myopic reference, while VRRGuard restores validity from 0.223 to 0.793 when calibration fails at near-zero discrimination. The method is limited by its local-stationarity approximation and binary validity representation; future work will model round-varying repair dynamics and online switching between the two modes.
Acknowledgments This work was supported in part by the Brain Science and Brain-like Intelligence Technology-National Science and Technology Major Project (No. 2021ZD0200600, No.
Derivations, Algorithms, and Reference Policies
This appendix provides the complete derivations of the stopping quantities in the main text, proofs of the two formal results, the construction of the calibration estimators and the error radius, and formal definitions of all comparison policies. Pseudocode integrating the two stopping modes is given in Algorithm 1; the mode-selection threshold Jmin corresponds to the separability check on held-out labeled samples in the guarded-fallback subsection of the main text. Evaluation scenarios were partitioned offline according to this check, and the specific value of Jmin was not tuned online.
A.1
Belief Recursion and the Stopping Boundary
Conditioned on the true state yk , the M verification queries are conditionally independent, so the acceptance count is binomial: Sk | yk =1 ∼ Bin(M, 1 − ρ1 ) and Sk | yk =0 ∼ Bin(M, ρ0 ). Applying Bayes’ rule to the predictive belief b− k , the two binomial coefficients cancel between numerator and denominator, which yields Eq. (5). Given yk , the repair transition is conditionally independent of the history Hk , so Pr(yk+1 =1 | Hk , repair) = (1 − β)bk + α(1 − bk ); this is the prediction step. Subtracting the current belief from the predictive belief gives Eq. (6). Setting Gk = 0 yields the boundary b⋆ = α/(α + β) of Eq. (7), which is also the fixed point of the affine map b 7→ (1 − β)b + α(1 − b) and coincides with the fixed point of the population dynamics in the motivation subsection of the main text. Under a nonzero threshold, Gk > τ is equivalent to bk < b⋆τ = (α − τ )/(α + β). Fig. 6 illustrates the geometry of this boundary with three empirically measured repair operators.
A.2
Proof of Proposition 1
Let E = {|Ĝk − Gk | ≤ Bk }; by the definition of Bk , Pr(E) ≥ 1 − η. If Ĝk − Bk > τ , then on E we have Gk ≥ Ĝk − Bk > τ , so the true action and the estimated action are both Repair. If Ĝk + Bk ≤ τ , then on E we have Gk ≤ Ĝk + Bk ≤ τ , so both are Commit. In either case the two actions agree with probability at least Pr(E) ≥ 1 − η, which completes the proof.
A.3
Proof and Numerical Evaluation of Lemma 1
Suppose the current candidate c is truly valid and the new plan p is truly invalid; then S(p) ∼ Bin(M, ρ0 ) and S(c) ∼ Bin(M, 1 − ρ1 ), and the two are independent given the true states. Pairing the m-th queries, the differences Xm = a(m) (p) − a(m) (c) ∈ [−1, 1] are i.i.d. with E[Xm ] = ρ0 − (1 − ρ1 ) = −J. Applying Hoeffding’s inPM equality to D = m=1 Xm gives Pr(D − E[D] ≥ t) ≤ exp(−t2 /(2M )); taking t = M J + δ ≥ 0 yields the singleround bound, and the statement over Kmax rounds follows from a union bound, which completes the proof.
marginal gain G(b)
A
Favorable
0.4
2021ZD0200408), and in part by the National Natural Science Foundation of China (Grant No. U24B20186), and supported by the Supercomputing Center of Lanzhou University.
continue
b * = 0.564
BFCL multi-turn
b * = 0.954
0.0 b * = 0.289
GSM8K stress -0.4
stop
-0.8 0.0
0.2
0.4
0.6
0.8
1.0
belief b
Figure 6: Expected benefit G(b) = (1 − b)α − bβ of running one more repair round, where b is the probability that the current plan is already correct, α is the probability that repair fixes an incorrect plan, and β is the probability that repair breaks a correct one; the three lines use the (α, β) values measured in Table 5. Wherever a line lies above zero, one more repair is expected to help; below zero it is expected to hurt; the crossing b⋆ = α/(α+β) (vertical dashes) marks the confidence level at which repairing stops paying off. Because b⋆ ranges from 0.954 (favorable) down to 0.289 (stress), the same confidence can mean continue in one setting and stop in another, so no fixed round budget or universal confidence threshold is safe across settings.
This bound is conservative relative to the exact tail probability. PThe exact value is the finite sum Pr[S(p) − S(c) ≥ δ] = i−j≥δ Pr[S(p)=i] Pr[S(c)=j]. Substituting the calibrated values of each verifier in Table 5 (M = 8, δ = 5), the single-round erroneous-replacement probabilities are 5.6×10−6 for Qwen-3B, 1.0×10−6 for Qwen-7B, 5.3×10−5 for Mistral, and 5.9 × 10−3 for Llama; even in the worst case of near-zero J, the five-round union bound does not exceed 3%.
A.4
Calibration Estimators and Identifiability
Calibration from repeated verification records treats the round-1 acceptance counts as a two-component binomial mixture, S ∼ π Bin(M, 1 − ρ1 ) + (1 − π) Bin(M, ρ0 ). The E-step of expectation-maximization (EM) computes each plan’s responsibility of belonging to the valid component, and the M-step updates the mixture weight and the two acceptance rates from responsibility-weighted frequencies; after multiple random restarts the solution with the highest log-likelihood is retained, and the component labels are fixed by the convention that the component with the higher acceptance rate corresponds to the valid class. The mixture is identifiable if and only if the two component acceptance rates differ, that is, J ̸= 0. As J → 0 the likelihood surface flattens and the estimator no longer carries information about ρ0 , ρ1 ; this is exactly the mechanism behind the failure of calibrated stopping in the Llama scenario. The transition
parameters are estimated from labeled pre/post-repair plan pairs: α̂ is the frequency of invalid-to-valid transitions and β̂ the frequency of valid-to-invalid transitions. The prior π̂ is the validity rate of initial plans on the calibration folds, and all estimation is performed outside the held-out folds of the cross-fitting.
A.5
Error Radius and Sample Complexity
The error radius Bk is constructed by propagating confidence bounds on the calibrated estimates through Eq. (5) and the expression for Ĝk , taking the worst-case excursion of Ĝk over the endpoints of each parameter interval. In an idealized single-query loop, the calibration sample size required for the stopping sign to be identifiable satisfies N ≳ C1 log(1/δc )/(M J 2 ∆2 ), where δc is the allowed failure probability. This bound is derived under the idealized single-query loop; we do not claim a corresponding explicit bound for the deployed rule, and decision reliability under deployment conditions is characterized directly by the fliprisk simulation in the identifiability experiments of the main text.
A.6
Formal Definitions of Baselines and References
No-repair commits p0 directly. Fixed-K repairs unconditionally for K rounds and then commits pK . Majority stopping commits the current plan when Sk > M/2 and otherwise keeps repairing until the budget is exhausted. ConfStopτconf replaces the commit condition with Sk /M ≥ τconf . Accepted-first commits the first candidate on the trajectory that receives majority acceptance, and commits the final round at budget exhaustion when no round is majorityaccepted. Last-accepted commits the last majority-accepted candidate, and commits the initial plan when no round is accepted. Verifier-best-of-trajectory commits the candidate with the highest acceptance count, breaking ties toward the earlier round. The true-parameter myopic (TPM) reference executes the same myopic stopping rule as VRR-Stop but with the true loop parameters (ρ0 , ρ1 , α, β, π); it isolates the effect of calibration error on the decisions and is not a hindsight policy that selects trajectories after observing ground truth. Hindsight best-round selection picks the best round on each trajectory according to the true labels; it serves only as a trajectory-structure diagnostic and is not deployable. The stationarity and independence assumptions of the fourparameter model are local approximations; their scope of validity and failure modes are discussed in Appendix H.
B
Full Experimental Setup
This appendix provides the complete experimental setup required for auditing and reproduction; the runtime environment, random seeds, and prompt templates are given in Appendix J. Task windows and verifiers. Table 4 lists the verifier type, evaluation windows, and label source of each task; the verifier for MATH-500 is a process reward model (PRM). The calibration probe issues 16 verification queries per plan
Algorithm 1 VRR-Stop with the guarded fallback (intended deployment procedure; experiments evaluate the two modes separately on saved trajectories) Require: plan p0 ; calibrated π̂, ρ̂0 , ρ̂1 , α̂, β̂; thresholds τ, η, Jmin ; margin δ; budgets M, Kmax 1: mode ← Stop if labeled-sample diagnostic gives Jˆ ≥ Jmin else Guard 2: b− ← π̂; c ← p0 ; Sc ← −∞ 3: for k = 0, 1, . . . , Kmax do 4: query the verifier M times on pk and collect Sk 5: bk ← posterior update of b− with Sk {Eq. (5)} 6: if Sk ≥ Sc + δ then 7: c ← pk ; Sc ← Sk {keep-best tracking, Eq. (9)} 8: end if 9: if mode = Stop then 10: Ĝk ← (1 − bk )α̂ − bk β̂; construct Bk 11: if Ĝk + Bk ≤ τ then 12: return pk {commit, stop sign identifiable} 13: else if Ĝk − Bk ≤ τ then 14: mode ← Guard {sign not identifiable, Eq. (8)} 15: end if 16: end if 17: if k = Kmax then 18: return pk if mode = Stop else c 19: end if 20: pk+1 ← repair pk with verifier feedback 21: b− ← (1 − β̂)bk + α̂(1 − bk ) {predict step} 22: end for
to stabilize the acceptance-rate estimates, whereas the endto-end loop issues M = 8 queries per round; the two share the same verifier configuration. The calibration stage uses at most 300 labeled samples. Construction of evaluation settings. The favorable setting is the standard generate-verify-repair loop without perturbation. In the prompt-mismatch (PM) stress setting, the copy of the problem statement received by the repairer is injected with numeric- or condition-level perturbations, while the verifier and the ground-truth judgment always use the original statement; the repairer can therefore produce plausible-looking but incorrect revisions from a corrupted premise. The repair temperature is 1.0 in the stress setting and 0.7 elsewhere. The main text’s non-stationarity diagnostic adds two further constructions: initial drafts are generated under a 256-token budget, and perturbations are injected from round 3 onward and run through round 6. Calibration protocol and cross-fitting. To prevent the same instance from being used both for parameter estimation and for stopping evaluation, the VRR-Stop results and the true-parameter myopic (TPM) reference reported in the main text both use five-fold cross-fitting: the stopping decision for each instance uses only the ρ0 , ρ1 , α, β and prior parameters estimated from the other four folds. Decisions from the five held-out folds are pooled at the instance level before computing the final validity and the mean number of repair rounds; the random seed of the fold split is fixed and
Table 3: Generator and verifier models of each evaluation setting. In every shift scenario the whole loop (generation, repair, and judging) runs on the target models; the shift codes of Table 2 mark the dominant change dimension relative to the Qwen-3B baseline.
Table 4: Datasets, verifiers, and evaluation windows. Probe windows feed the calibration probe (16 votes per plan); e2e and stress windows feed end-to-end evaluation (M =8). Task
Verifier type
GSM8K
LLM judge
Windows (N )
probe 300 / e2e 200 stress 200, 500 MATH-500 PRM 500 MBPP LLM judge 150 BFCL LLM judge / executor 400 / 199 1.0
Qwen-3B stress
Qwen-7B stress
Mistral stress
MATH-500
BFCL single
BFCL multi
Favorable
0.5
true validity
released with the replay artifacts. In-sample calibration results serve only as a sensitivity audit, not as the deployment protocol. The prior π̂ is the frequency estimate of the true validity rate of initial plans on the calibration folds. The transition parameters are estimated from the pre/post plan pairs of the first repair round on the frozen trajectories; during data generation, repair was triggered in every round without verifier gating, so the transition samples are free of selection bias induced by the acceptance signal. Definition of shift scenarios. Each shift scenario reruns the entire loop (generation, repair, and judging) on the target model or task. The shift codes in Table 2 mark the dominant change dimension of each scenario relative to the Qwen-3B baseline: M denotes a change of generator family or scale, V a change of verifier family, and T a change of task together with its accompanying verifier; the evaluation protocol and the calibration pipeline remain unchanged. The generator and verifier models of each setting are listed in Table 3.
0.0 1.0
Llama stress
0.5
0.0
Setting
Generator
Qwen-3B fav./stress (–) Qwen2.5-3B-Instruct same model (LLM judge) Qwen-7B stress (M) Qwen2.5-7B-Instruct same model (LLM judge) Mistral-7B stress (V) Mistral-7B-Instruct-v0.3 same model (LLM judge) Llama-3-8B stress (M) Llama-3-8B-Instruct same model (LLM judge) MATH-500 (T) Qwen2.5-Math-7B-Instruct Qwen2.5-Math-PRM-7B BFCL single (T) Qwen2.5-7B-Instruct same model (LLM judge) BFCL multi (T) Qwen2.5-7B-Instruct official executor
Hyperparameters. End-to-end evaluation uniformly uses M = 8, Kmax = 5 (the non-stationarity diagnostic runs through round 6), τ = 0, δ = 5, and B = 10,000 bootstrap resamples. The coverage level η is a confidence parameter of the reliability-analysis layer and does not enter end-toend deployment. Only two hyperparameters were tuned: the retention margin δ was swept over {0, . . . , 8} by pure replay on all seven settings, selecting the value most robust across settings, and the calibration sample size was examined over Ncalib ∈ {50, 100, 200, 300}. The remaining settings were fixed a priori and not tuned: M = 8 and Kmax = 5 follow common verify-repair budgets, τ = 0 by definition of the cost-free objective, and systematic sweeps of M and τ are left to future work (Appendix I). A manifest mapping each figure and table of the main text to its raw data artifacts is released with the anonymous code repository.
C
Full Loop Dynamics and Failure Cases
This appendix examines whether the phenomenon of RQ1 persists across settings and unfolds the complete chain of a harmful repair on a single instance.
C.1
0
Verifier
Cross-Setting Loop Dynamics
Table 5 summarizes the round-1 parameter estimates and trajectory shapes of the eight settings on their respective evaluation windows. Apart from the favorable setting, which
2
5 0
2
5 0
2
5 0
2
5
repair round
Figure 7: Mean true validity per repair round when every plan is repaired unconditionally, averaged over each trace’s evaluation window; the eight panels are the settings of Table 5. Round 0 equals the no-repair baseline and round k equals the fixed-budget value Fixed K=k, so each curve traces the full path between the two extremes reported in the tables. Six settings collapse within the first rounds and never recover, BFCL multi-turn stays flat, and only the favorable setting improves. rises monotonically, and BFCL multi-turn, which is approximately flat, true validity declines monotonically with the round index in the remaining six settings; the damage probability β ranges from 0.615 to 0.938 and is typically several times the repair probability α, so the stopping boundary b⋆ is pushed below 0.29. The MATH-500 row shows that a strong verifier (J = 0.77) does not prevent a high-β repairer from damaging correct plans, whereas the near-inert operator of BFCL multi-turn (α = 0.02, β = 0.04) provides a counterexample of multi-round repair without collapse. The finite-trace diagnostic in the last column shows that, outside the flat setting, the gap between myopic stopping and the hindsight best round is at most 4 percentage points. Fig. 7 plots the per-round true-validity trajectories of all settings.
C.2
A Representative Failure Trace
Table 5 and Fig. 4 give the aggregate view; this section unfolds the same mechanism on a single instance. The pattern is not a hand-picked pathology: on the N = 500 stress traces, the round-1 validity of 0.70 drops to 0.25 after a single re-
Table 5: Full loop dynamics across the eight settings, where PM-stress denotes the prompt-mismatch stress setting. All parameters are per-call round-1 estimates on each trace’s evaluation window (the Qwen2.5-3B PM-stress row uses the N =500 window), b⋆ = α/(α + β) is the stopping boundary of Eq. (7), and the last column is a finite-trace diagnostic rather than a formal dynamic-programming regret. Read each row by weighing repair α against damage β: wherever β dominates, b⋆ collapses toward zero and the trajectory shape turns into a monotone decline, so harmful repair is the rule across the stress rows rather than an exception. Setting
Model
Favorable PM-stress PM-stress PM-stress PM-stress MATH-500 BFCL single BFCL multi
Qwen2.5-3B Qwen2.5-3B Qwen2.5-7B Mistral-7B Llama-3-8B Qwen2.5-Math-7B Qwen2.5-7B Qwen2.5-7B
Verifier
ρ0
ρ1
J
α
β
b⋆ (Eq. 7)
Shape
Best-K − myopic
LLM judge LLM judge LLM judge LLM judge LLM judge PRM LLM judge Executor
0.438 0.364 0.725 0.707 0.358 0.109 0.875 0.659
0.169 0.177 0.048 0.111 0.609 0.118 0.057 0.223
0.39 0.46 0.23 0.18 0.03 0.77 0.07 0.12
0.423 0.320 0.000 0.014 0.051 0.020 0.147 0.048
0.020 0.786 0.909 0.862 0.938 0.617 0.615 0.037
0.954 0.289 0.000 0.015 0.051 0.031 0.192 0.564
monotone ↑ monotone ↓ monotone ↓ monotone ↓ monotone ↓ monotone ↓ monotone ↓ flat
0.025 0.016 0.000 0.040 0.000 0.000 0.027 −0.015
pair round, 55% of instances experience a correct plan being repaired into an incorrect one, and 24% of these damaging repairs receive majority acceptance among the eight judgments. All of these are raw-label statistics, independent of the stopping rule and its calibration. The trace comes from GSM8K test instance #292 (Qwen2.5-3B, M = 8 judgments per round); the complete causal chain is shown in Fig. 8. As shown in Fig. 8, the initial plan correctly derives 75 dollars from 150/10 = 15 billing intervals, yet receives only four of the eight votes and suffers a false rejection. The upstream perturbation corrupts only the repairer’s copy of the problem statement, changing every 10 minutes to every 17 minutes; the repairer then competently rewrites the plan from the corrupted premise into an incorrect answer of 40 dollars, and the verifier, which judges against the uncorrupted original problem, instead gives it six votes and commits it. True validity flips from 1 to 0 while the reported acceptance rises. This is the trade-off between the repair benefit (1−bk )α and the damage loss bk β of the belief-filtering subsection of the main text unfolding on a single instance: a high-β rewrite compounded by the missed detection of a verifier false acceptance. The case text is taken from a replay capture of the stresssetting pipeline, and a second instance exhibits the same type of chain; both full transcripts (including the eight judgment responses per round, the two candidate solutions, and the exact repair prompts) are released with the anonymous code repository. Per-instance sampling is not bitwise reproducible across inference-engine versions, so the case cites the replay transcripts rather than the original frozen trajectories; the aggregate conclusions are unaffected.
D
Full Stopping and Cross-Benchmark Results
This appendix examines the stability of the RQ2 conclusions across additional baselines, larger samples, different models, and different tasks; all numbers are reported under the fivefold cross-fitting protocol. Full baselines. Table 6 completes the full set of comparison strategies on the stress setting of Table 1, where ∆V
denotes the difference relative to the true-parameter myopic (TPM) reference. The three families of baselines exhibit a clear stratification. Heuristic stopping rules that rely solely on the verifier signal approach but never exceed no-repair (Majority stopping and Accepted-first both at 0.690, Verifier-best at 0.696); confidence-threshold stopping and Last-accepted degrade moderately (0.562 and 0.504); fixed-budget repair collapses severely and worsens monotonically with the budget (0.246, 0.122, and 0.116 for K = 1, 3, 5). VRR-Stop at 0.722 is the only deployable strategy that exceeds both the no-repair baseline and the TPM reference. The reference row at the bottom of the table is diagnostic only and is not deployable. Sample size and multi-window replication. On the N = 200 window of the same setting, VRR-Stop attains 0.705 (95% CI [0.640, 0.765], ⟨K⟩ = 0.82) against a TPM reference of 0.695, directionally consistent with 0.722 versus 0.694 at N =√500; the confidence intervals shrink by roughly a factor of 2.5, and the strategy ranking is unchanged. Across three disjoint GSM8K windows, calibrated stopping attains 0.752 ± 0.037, the TPM reference 0.757 ± 0.058, and fixed 5-round repair 0.118 ± 0.007. Calibrated stopping and the reference strategy alternate in per-window ranking; the two are close but not identical, which is the expected behavior of a calibration that tracks the true parameters. Cross-model and cross-domain replication. The Qwen2.5-7B stress setting exhibits the largest collapse in the paper: fixed 5-round repair falls from the no-repair level of 0.875 to 0.075, whereas VRR-Stop preserves 0.875 with zero repair rounds on average, matching the TPM reference; in this setting, calibrated stopping learns not to repair at all. On MBPP, under both the favorable and the stress settings, calibrated stopping coincides with the TPM reference (0.533 and 0.493, respectively), remaining stable across domains and calibration protocols. Direct comparison with iterative-feedback baselines. Under the fixed-budget deployment mode of the stress setting, Reflexion and Self-Refine reach final validities of 0.095 and 0.080, whereas VRR-Stop on the same trajectories attains 0.740 and 0.710 (TPM reference: 0.710 and 0.705). This comparison audits the fixed-budget deployment mode
original problem — what the verifier always grades corrupted copy: 'every 10 minutes' → 'every 17 minutes'
REPAIR UNDER A CORRUPTED PREMISE
ROUND 0: GENERATE + VERIFY
USER TASK Susan earns $5 every 10 minutes for an online task. She works between 8 a.m. and 11 a.m. and pauses in between for half an hour. How much money does she earn?
Gold answer: $75
Planner
[1] 2.5 h = 150 min; Repairer
[2] 150 / 10 = 15;
LLM planner generates
[4] 15 × $5 = $75;
falsely REJECTED
rejected plan
LLM verifier
revised plan
LLM repairer
(true invalid)
Verifier votes(6/8 accept)
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
Verifier
[3] the leftover 14 minutes earn nothing. [4] ........
sent to repair
Verifier
[2] '8 full intervals at $5 each give $40;
Repairer Verifier
Verifier
[1] 150 / 17 = 8 remainder 14;
Verifier votes(4/8 accept) Verifier
still grades the ORIGINAL problem
Verifier
Premise corruption (repairer's copy only)
[3] billable intervals;
Initial plan
ROUND 1: VERIFY + COMMIT
LLM repairer
Revised plan
falsely ACCEPTED
(true valid)
deviates from the gold answer.
True validity: 1 → 0 (falls)
$40 (wrong)
Reported acceptance: 4/8 → 6/8 (rises)
Figure 8: End-to-end failure trace of GSM8K instance #292. The two top rails show the fork that drives it: the original problem (blue, solid) is what the verifier always grades, while the corrupted copy (amber, dashed) that turns every 10 minutes into every 17 minutes reaches only the repairer. A valid plan under-accepted at 4/8 is competently rewritten from the corrupted premise into an invalid one, which the verifier accepts at 6/8 and commits to the user, so true validity falls from 1 to 0 while reported acceptance rises. (a) Per-plan rates
of iterative feedback and does not claim to reproduce the full training details of the original methods.
Invalid (n=79) Valid (n=221)
8
0.75
6
density
Full Calibration and Identifiability Results
acceptance rate k/16
E
This appendix examines when the parameter estimates can be trusted, when they fail, and locates the mechanism of failure. Cross-family calibration stress. Table 7 shows that the dominant calibration bias points in a different direction for each of the four verifiers: the Qwen family underestimates the false-acceptance rate, Llama underestimates the false-rejection rate, and the false-acceptance bias of Mistral reaches 0.66. Decision consequences are decoupled from bias magnitude: the largest bias, on Mistral, costs only 0.3 percentage points, whereas the smaller bias on Llama produces a 58.0-point collapse; the dividing line is the discrimination ability J, not the bias itself. Fig. 9 supplies the upstream evidence for all of this: on the probe window, repeated verification yields acceptance-rate distributions for the two classes that overlap yet remain separable, which is why weakly supervised calibration is feasible—and why it is fragile. Identifiability collapse on Llama. Table 8 exposes the failure mechanism. Binomial expectation-maximization (EM) yields ρ̂1 = 0.27 at N = 120, yet increasing the sample to 300 worsens the estimate to 0.077: as J approaches zero the likelihood surface flattens, and additional samples only make EM converge more confidently to a degenerate solution. The capped beta-binomial estimator gives 0.620 at N = 120 but is unstable across random seeds. A parameter sweep under the deployed rule completes the picture: calibrated decisions recover the stop-early optimum only when ρ̂1 is no lower than 0.30 to 0.35, with the midpoint crossing at ρ̂1 ≈ 0.17, so the EM estimate of 0.077 sits deep in the collapse zone. Stopping-decision agreement numbers for the estimatorinput ablation are given in Appendix I; the flip-rate and
(b) Distributions
1.00
0.50
0.25
4
2 True EM
0.00
0 Invalid
Valid
0.0
0.5
1.0
acceptance rate k/16
Figure 9: Per-plan judge acceptance on the N =300 probe window (M =16), split by true validity, shown as per-plan rates with class means in panel (a) and as class distributions in panel (b). Solid lines mark the true mean acceptance of each class and dashed lines the label-free EM estimates; EM recovers the valid-class rate almost exactly (0.91 vs 0.88) but underestimates the invalid-class rate (0.23 vs 0.49), the dominant source of downstream calibration error. The two classes overlap yet remain separable, which is why weakly supervised calibration is feasible but fragile.
identifiable-fraction grids underlying Fig. 5 are released with the anonymous code repository.
F
Full Robustness Results
This appendix examines whether the gains of VRR-Guard are confined to a single scenario and delineates the boundaries of its costs. Full results. Table 9 extends Table 2 with the TPM reference column and replay bootstrap intervals for Guard. VRR-
Table 6: Full stopping results on the GSM8K / Qwen2.5-3B prompt-mismatch stress setting (N =500); ∆V is measured against the TPM reference (positive = better), verifier-only baselines are offline replays on the same saved trajectories, and the final row is a non-deployable diagnostic. VRR-Stop is the only deployable strategy above both the no-repair baseline and the TPM reference. V [95% CI]
⟨K⟩
∆V (pp)
No repair Majority stopping ConfStop-0.85 Fixed repair K=1 Fixed repair K=3 Fixed repair K=5 Accepted-first Last-accepted Verifier-best-of-trajectory VRR-Stop
0.700 [.658,.740] 0.690 [.648,.730] 0.562 [.518,.604] 0.246 [.208,.284] 0.122 [.094,.152] 0.116 [.088,.144] 0.690 [.648,.730] 0.504 [.460,.546] 0.696 [.654,.736] 0.722 [.682,.760]
0.00 0.92 1.92 1.00 3.00 5.00 0.92 2.28 0.73 0.72
+0.6 −0.4 −13.2 −44.8 −57.2 −57.8 −0.4 −19.0 +0.2 +2.8
TPM reference
0.694 [.652,.734]
0.89
0.0
Method
Table 7: Cross-family calibration stress (per-row N is each trace’s evaluation window, J uses the family-level convention, and Gap is Vstop − VTPM in percentage points; the TPM reference is not a hindsight upper bound). Dominant bias reports the largest EM estimation error as estimate minus truth, so negative values are underestimates. Decision damage tracks J rather than the size of the dominant calibration bias. Judge
Family
N
J
Dominant bias
Vstop
VTPM
Gap (pp)
Qwen2.5-7B Qwen2.5-3B Mistral-7B Llama-3-8B
Qwen Qwen Mistral Llama
200 500 300 300
0.23 0.46 0.18 0.03
∆ρ0 = −0.29 ∆ρ0 = −0.19 ∆ρ0 = −0.66 ∆ρ1 = −0.53
0.875 0.722 0.463 0.223
0.875 0.694 0.467 0.803
+0.0 +2.8 −0.3 −58.0
Table 8: Identifiability collapse on the Llama-3-8B judge (target ρ1 = 0.609). Under the deployed rule, a parameter sweep shows calibrated stopping recovers the stop-early optimum only for ρ̂1 ≳ 0.30–0.35 with the midpoint crossing at ρ̂1 ≈ 0.17, so the EM estimate of 0.077 sits deep in the collapse zone. Estimator Binomial EM Capped beta-binomial EM
ρ̂1 (N =120)
ρ̂1 (N =300)
0.27 0.620
0.077 unstable (seed)
Guard stays near the no-repair level in all seven settings, whereas fixed 5-round repair degrades sharply in every setting except favorable. The Mistral row also provides a direct counterexample showing that the TPM reference is not a hindsight upper bound: its 0.467 falls below the no-repair 0.507. Fig. 10 places the repair cost and true validity of two representative scenarios on the same plane. Honest boundaries. Guard does not dominate everywhere. On Mistral and BFCL it falls below no-repair by 2.0 and 0.3 percentage points, with paired intervals of [−3.7, −0.7] and [−1.3, +0.5] respectively, the latter containing zero (Appendix G). Under the favorable setting, Guard’s 0.81 is below the 0.875 of fixed 5-round repair— the opportunity cost of conservatism in a scenario where repair helps. Role of the retention margin. Compared with margin-
free Verifier-best on the same trajectories, Guard is 45.0 percentage points higher in the Llama scenario (95% CI [+39.0, +51.0]), while the differences in the remaining settings lie between −4.0 and +8.0 percentage points. The retention margin δ is nearly neutral in most scenarios; its value is realized precisely where the verifier signal is least trustworthy. A no-collapse counterexample, and cost. Under the near-inert repair operator of BFCL multi-turn (α = 0.02, β = 0.04), all strategies coincide within confidence intervals: multi-round repair is not inherently harmful, and severe degradation appears only together with a high break probability. Regarding cost, each round consumes a fixed M verification calls and at most one repair call, so the cost ordering of the strategies is exactly the ⟨K⟩ ordering reported in each table; VRR-Guard operates by replaying saved trajectories and incurs no additional verification calls.
G
Paired Statistical Tests
This appendix provides same-trajectory paired statistical support for the percentage-point claims in the main text. All comparisons are paired per instance on the same batch of task instances and saved repair trajectories; differences in true validity are assessed with 10,000 instance-level bootstrap resamples, directional binary comparisons use exact McNemar tests, and a positive ∆V indicates that the lefthand strategy is better on the same instances. The decision sequences of VRR-Stop and the TPM reference follow the
Table 9: Full guarded-stopping results with the TPM reference column and replay bootstrap CIs; shift codes M, V, and T mark a change of model family, verifier family, or task relative to the Qwen-3B baseline. The TPM reference is a diagnostic rather than a hindsight upper bound (note Mistral: TPM 0.467 < No repair 0.507). VRR-Guard stays within a few points of no-repair in every row, including the Llama row where calibrated stopping collapses.
0.9
Setting
J
No repair
Fixed K=5
VRR-Stop
TPM ref.
VRR-Guard
Guard 95% CI
Qwen-3B favorable (–) Qwen-3B stress (–) Qwen-7B stress (M) Mistral-7B stress (V) Llama-3-8B stress (M) MATH-500 (T) BFCL single (T)
0.39 0.46 0.23 0.18 0.03 0.22 0.07
0.740 0.700 0.875 0.507 0.803 0.798 0.812
0.875 0.116 0.075 0.047 0.057 0.150 0.372
0.845 0.722 0.875 0.463 0.223 0.798 0.782
0.850 0.694 0.875 0.467 0.803 0.798 0.780
0.810 0.742 0.875 0.487 0.793 0.796 0.810
[.755,.865] [.702,.780] [.825,.920] [.430,.543] [.747,.837] [.760,.832] [.770,.848]
(a) Qwen-3B stress
(b) Llama stress
VRR-Guard
estimates.
No repair
VRR-Stop
TPM
TPM
H.1
Majority No repair
true validity
0.6
Verifier-best ConfStop-0.85
Gk = (1 − bk )α − bk β.
Verifier-best
0.3
Fixed K=1
VRR-Stop Majority Fixed K=1
Fixed K=3
Fixed K=3
Fixed K=5
ConfStop-0.85 Fixed K=5
0.0 0
1
2
3
4
mean repair rounds
5
0
1
2
3
4
5
mean repair rounds
Figure 10: True validity versus mean repair rounds for each stopping strategy under a working calibration (a, Qwen-3B stress) and a failed one (b, Llama stress); up-left is better, and the hollow marker is the ground-truth-parameter diagnostic TPM rather than a deployable strategy. Fixed-budget repair forms a dominated slide that pays more rounds for less validity, and in panel (b) calibrated VRR-Stop itself falls into that region while VRR-Guard stays at the no-repair corner. The pair thus covers both regimes, with calibrated stopping earning the low-cost, high-validity corner when calibration holds and the guarded fallback keeping that corner when it fails.
same cross-fitting protocol as the main text.
H
Repair Benefit and Damage Risk
Whether to continue repairing depends on the relative magnitude of the repair benefit and the damage loss, not on the number of repair rounds already executed. By Eq. (6), the true marginal gain of a single repair round is
VRR-Guard
Extended Discussion
This appendix elaborates on applicability conditions and method boundaries that the main text does not cover in detail, including how harmful repair arises, how calibration error affects stopping decisions, the failure modes of different verifiers, and the benefits and costs of the conservative fallback strategy. A common thread across the subsections is that calibration quality should be measured by whether the decision sign flips, not by the absolute error of parameter
The first term captures the expected benefit of an invalid plan becoming valid through repair, and the second term captures the expected loss of a valid plan being damaged by repair. Even under the same maximum repair budget, different values of bk , α, β yield different optimal stopping times; verifier noise ρ0 , ρ1 shifts this timing indirectly through the posterior belief bk . The results in Appendix C exhibit pronounced differences across settings. In the GSM8K stress setting and on MATH500, the repairer rarely corrects invalid plans yet frequently damages valid ones: fixed five-round repair reduces true validity from 0.700 to 0.116 and from 0.798 to 0.150, respectively. BFCL multi-turn instead shows nearly inert repair dynamics, with α = 0.02 and β = 0.04, and no statistically distinguishable difference among stopping policies. This counterexample shows that multi-round repair is not inherently harmful; substantial degradation emerges only when the repairer has a non-negligible capacity to damage valid states. Verifier discrimination ability and repairer safety also require separate examination. On GSM8K, the process reward model verifier Qwen2.5-Math-PRM-7B (Zhang et al. 2025b) attains a Youden’s index of J = 0.805 (this setting uses Qwen2.5-3B as the generator and is independent of the MATH-500 row in Table 5), yet the stronger verifier does not prevent the repairer from damaging correct plans. In this setting, fixed five-round repair still reduces validity from 0.727 to 0.097, whereas the true-parameter myopic (TPM) reference remains at 0.663. Improving verifier quality therefore strengthens the reliability of stopping decisions but cannot eliminate the risk introduced by the repair operation itself.
H.2
Non-Stationarity and Interior Peaks
The repairer’s state-transition probabilities may change across rounds. In the favorable setting of Qwen2.5-3B, the re-
Table 10: Paired replay statistics for load-bearing comparisons. Positive ∆V means the left strategy has higher true validity on the same saved instances. Every headline percentage claim in the main text maps to one row of this table. Section / setting
Comparison
RQ1 interior peak RQ1 interior peak RQ2 Qwen-3B stress RQ2 Qwen-3B stress RQ2 Qwen-3B stress RQ2 Qwen-3B stress RQ2 Qwen-3B stress RQ3 Llama stress RQ4 Llama stress RQ4 Llama stress RQ4 Llama stress RQ4 Mistral stress RQ4 BFCL
round 2 vs final round round 0 vs final round VRR-Stop vs Fixed-K=5 VRR-Stop vs ConfStop-0.85 VRR-Stop vs Majority stopping VRR-Stop vs TPM reference VRR-Stop vs No repair VRR-Stop vs TPM reference VRR-Guard vs failed VRR-Stop VRR-Guard vs Fixed-K=5 VRR-Guard vs Verifier-best (unguarded) VRR-Guard vs No repair VRR-Guard vs No repair
pair success rate under the per-round convention drops from 0.415 in the first round to 0.032 in the fifth (Table 5 reports a first-round value of 0.423 under the per-call convention), while the damage probability stays low throughout. This pattern admits a natural interpretation: easily repairable errors are eliminated first, and the remaining errors are harder to fix. Describing the entire repair process with transition probabilities estimated from the first round should therefore be regarded only as a locally stationary approximation. The non-stationarity diagnostic trajectory in the main text’s loop-dynamics experiment further illustrates the interior peak that repair can produce. Early repair completes initial solutions truncated by the generation-length limit, raising true validity to 0.87 at round two. After context drift is introduced at round three, true validity gradually declines to 0.12. This trajectory demonstrates that the optimal submission time can occur at an intermediate round, but it does not imply that a stopping policy based on stationary parameters can necessarily locate that peak. In paired replay, VRR-Stop and the TPM reference attain 0.693 and 0.720 (cross-fitted convention), both far below the 0.867 obtained by post hoc selection of round two. This result delineates the capability boundary of the current method. VRR-Stop judges whether one more repair round still yields a positive gain under the current local model; it does not search the full trajectory for the post hoc optimal round. When the repair mechanism changes abruptly at some round, α and β estimated from the first round cannot represent this change in advance. The interior-peak experiment thus both confirms the necessity of the stopping problem and indicates room for further research on rounddependent dynamic models.
H.3
Decision Robustness under Calibration Error
Parameter estimation error affects the stopping action only when it changes the sign of Gk − τ . For Mistral-7B, the estimation bias of the false-acceptance rate reaches ∆ρ0 = −0.66, yet VRR-Stop attains a true validity of 0.463, within 0.3 percentage points of the TPM reference at 0.467. Despite the large parameter error, the corresponding marginal gain
∆V [95% paired CI]
Paired test
+0.747 [+0.693, +0.797] +0.327 [+0.257, +0.397] +0.606 [+0.560, +0.650] +0.160 [+0.126, +0.194] +0.032 [+0.016, +0.050] +0.028 [+0.014, +0.044] +0.022 [−0.004, +0.048] −0.580 [−0.637, −0.520] +0.570 [+0.510, +0.627] +0.737 [+0.683, +0.787] +0.450 [+0.390, +0.510] −0.020 [−0.037, −0.007] −0.003 [−0.013, +0.005]
paired bootstrap paired bootstrap McNemar p < 2 × 10−85 McNemar p < 1 × 10−18 McNemar p < 2 × 10−4 McNemar p < 6 × 10−4 paired bootstrap McNemar p < 2 × 10−48 McNemar p < 1 × 10−47 McNemar p < 2 × 10−62 paired bootstrap paired bootstrap paired bootstrap
stays in the same decision region, so the final action does not change. Llama-3-8B exhibits the opposite outcome. Its verifier discrimination ability is only J = 0.03, and the binomialmixture estimate underestimates the false-rejection rate by about 0.53, which systematically inflates the posterior belief bk and thus substantially underestimates the damage risk represented by bk β. The estimation error eventually crosses the stopping boundary, driving VRR-Stop down to 0.223 from the TPM reference of 0.803. The direct cause of the degradation is not the parameter bias itself but the fact that the bias flips the decision sign between continuing and stopping. BFCL single-call provides further corroboration. Its verifier has a J of only 0.07, but the states it visits carry large decision margins, and VRR-Stop differs from the TPM reference by only 0.3 percentage points. A uniform trust threshold therefore cannot be set from the J value alone. The controlled experiment shown in Fig. 5 likewise indicates that when low J coincides with a low decision margin ∆, the sign-flip rate reaches 0.183, whereas it is only 0.014 in regions with high J or high ∆. The reliability of stopping decisions is jointly determined by verifier discrimination ability, decision margin, and calibration sample size.
H.4
Family-Conditional Verifier Failures
The current experiments further show that different model families can exhibit different directions of verifier failure. Across the eight Qwen configurations, the binomial-mixture estimate underestimates the false-acceptance rate in every case, with ∆ρ0 ranging from −0.16 to −0.31 (the full list of biases is released with the anonymous code repository). For Llama-3-8B, the false-acceptance rate is estimated essentially accurately, but the false-rejection rate is markedly underestimated (by about 0.53 at N =300). Mistral-7B instead shows a pronounced tendency toward false acceptance, with a true false-acceptance rate of about 0.71, yet its large decision margin prevents the stopping sign from flipping. These results indicate that verifier noise should not be treated as a fixed bias transferable across models. The same calibration method can underestimate the false-acceptance
rate in one model family and the false-rejection rate in another, and the two biases propagate through the posterior belief into Eq. (6) in different directions. That said, the existing experiments cover only three model families, and some configurations share tasks and samples. These findings should therefore be read as family-conditional patterns observed within the current experimental scope, not as universal laws determined by model architecture. The process reward model and the executors used in our experiments also impose different calibration conditions. A deterministic verifier produces no new stochastic observations under repeated queries on the same plan, so the binomialmixture estimate based on repeated acceptance counts cannot be applied directly. The stopping criterion itself remains applicable in this case, but ρ0 and ρ1 must be estimated directly from a small number of labeled samples.
H.5
Safety and Upside of the Guarded Fallback
When Eq. (8) fails to hold, VRR-Guard no longer relies on precise noise parameters; instead, it raises the evidential strength required to replace the current candidate. The policy retains the best candidate so far and performs a replacement only when the verification acceptance count of a new candidate exceeds that of the current one by at least δ. It does not prevent the system from generating subsequent candidates; it avoids committing a possibly degraded repair result when the evidence is insufficient. In the Llama-3-8B scenario, VRR-Guard restores true validity to 0.793, an improvement of 57.0 percentage points over the failed VRR-Stop and 73.7 percentage points over fixed five-round repair. When unsupervised calibration approaches non-identifiability, retaining the existing candidate is thus more robust than continuing to rely on erroneous parameters. This robustness comes with an observable opportunity cost. In the favorable setting, VRR-Guard reaches a validity of 0.810, below the 0.875 of fixed five-round repair. In the Mistral-7B and BFCL settings, VRR-Guard falls short of norepair by 2.0 and 0.3 percentage points, respectively, and the paired interval for BFCL contains zero. VRR-Guard therefore carries no theoretical guarantee of strictly dominating norepair. It is better viewed as a conservative fallback close to the no-repair baseline: it bounds the downside risk when calibration is unreliable, at the cost of forgoing part of the potential benefit in scenarios where repair is beneficial.
H.6
Scope and Deployment Implications
This work represents whether a plan is truly valid with a binary state. This binary validity representation suits tasks with well-defined labels, such as answer correctness, tool executability, and constraint satisfaction, but it cannot fully describe partial correctness, multiple error types, or longhorizon plans with staged goals. For such tasks, the state space needs to be extended to multi-level or structured representations, and the corresponding stopping boundary may no longer be given by a single threshold. The strict condition α + β ≤ 1 holds on seven of the eight empirical trajectories, but the GSM8K stress trajectory of
Qwen2.5-3B clearly violates it. Conclusions on that trajectory therefore rest on single-round marginal gains and finitetrajectory diagnostics rather than on global monotonicity. The TPM reference likewise serves only to isolate decision differences caused by parameter calibration; it is not a globally optimal upper bound over the finite horizon. Post hoc selection of the best round can be used to analyze trajectory structure but cannot serve as a deployable policy. Deploying VRR-Stop should begin with estimating the noise parameters and their error ranges from repeated verification records and a small number of labeled transition samples. Only when Ĝk − Bk > τ or Ĝk + Bk ≤ τ does the available evidence suffice to support a definite continue or stop action. If the confidence interval straddles the stopping boundary, or if the model, verifier, or task distribution shifts, the system should switch to VRR-Guard or recollect labeled samples for recalibration. Existing parameters should not be transferred directly to a new model family, verifier, or repair prompt. Future work can model round-dependent αt and βt to detect late context drift and interior peaks. The binary true state can also be extended to multi-level plan quality, describing partial correctness and distinct error types. Another direction is to estimate instance-conditional noise that depends on plan features, so that plans of different difficulty receive different stopping boundaries. Finally, per-instance marginal gains can support cross-task compute allocation, selecting the instances most worth further repair when the total inference budget is limited.
I
Additional Ablations
This appendix reports sensitivity analyses that have already been run and can be replayed at zero cost. Calibration-input ablation. In the GSM8K stress setting, the final validity under three calibration inputs— binomial expectation-maximization (EM), method of moments (MoM), and labeled maximum-likelihood estimation (MLE)—is 0.722, 0.690, and 0.690, respectively, with the differences in stopping decisions concentrated on a few borderline instances. Beta-binomial EM (BB-EM) yields 0.666 (95% CI [0.624, 0.706], ⟨K⟩ = 1.26): its overly flexible fit pushes ρ̂0 up to 0.508, which systematically depresses the posterior belief and induces excessive repair. Table 8 provides a comparative diagnosis of the two EM variants under within-class heterogeneity. Retention-margin ablation. A pure replay sweep over δ ∈ {0, . . . , 8} (M = 8) across all seven settings shows that δ = 5 is the most robust choice, trailing the per-setting optimal δ by at most 4.0 percentage points in any setting. Calibration-sample-size sensitivity. Under subsampled re-estimation with Ncalib ∈ {50, 100, 200, 300}, the decision agreement rate rises from 0.973 to 0.997, the sign-flip rate falls from 0.016 to 0.002, and the absolute validity gap to the TPM reference shrinks from 0.012 to 0.001. Systematic ablations of the verification budget M and the gain threshold τ were not run within the scope of this work and are left for future work.
J
Reproducibility Details
All experiments were run on a single server equipped with eight NVIDIA A800 80GB PCIe GPUs, with each experiment completed on a single GPU. The software environment comprises Ubuntu 22.04, CUDA 12.8, PyTorch 2.10.0 (cuDNN 9.10.2), and the inference engine vLLM 0.19.1 (Kwon et al. 2023). The temperature for generation, judging, and repair is 0.7, with the repair temperature raised to 1.0 in the stress setting. Code versions, random seeds, the complete generation, judging, and repair prompts for each task, and an index of raw data files are released with the anonymous code repository. True validity is adjudicated by exact match on GSM8K, symbolic verification on MATH500, sandboxed unit tests on MBPP (5-second timeout), and abstract syntax tree (AST) matching plus the official executor on BFCL; Appendix B describes how the generator prior is obtained. Per-instance LLM trajectories are single stochastic realizations and are not bit-wise reproducible across inferenceengine versions. All numbers in this paper are computed by deterministic replay of frozen trajectories and do not depend on regeneration.
References Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.; Terry, M.; Le, Q.; and Sutton, C. 2021. Program Synthesis with Large Language Models. arXiv:2108.07732. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168. Collot, S.; Fraser, C.; Zhao, J.; Shen, W. F.; Willi, T.; and Leontiadis, I. 2026. Balanced Accuracy: The Right Metric for Evaluating LLM Judges - Explained through Youden’s J statistic. In Matusevych, Y.; Eryiğit, G.; and Aletras, N., eds., Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track), 927–936. Rabat, Morocco: Association for Computational Linguistics. ISBN 979-8-89176-384-5. Dorner, F. E.; Chen, Y.; Cruz, A. F.; and Yang, F. 2026. ROCn-reroll: How Verifier Imperfection Affects Test-Time Scaling. In The Fourteenth International Conference on Learning Representations (ICLR). Feng, C.; Shen, M.; Balashankar, A.; Gerner-Beuerle, C.; and Rodrigues, M. R. D. 2026. Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judges. In The Fourteenth International Conference on Learning Representations (ICLR). ArXiv:2601.20913. Fu, Y.; Wang, X.; Tian, Y.; and Zhao, J. 2026. Deep Think with Confidence. In The Fourteenth International Conference on Learning Representations (ICLR). Gao, L.; Schulman, J.; and Hilton, J. 2023. Scaling Laws for Reward Model Overoptimization. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research.
Gou, Z.; Shao, Z.; Gong, Y.; Shen, Y.; Yang, Y.; Duan, N.; and Chen, W. 2024. CRITIC: Large Language Models Can SelfCorrect with Tool-Interactive Critiquing. In The Twelfth International Conference on Learning Representations (ICLR). OpenReview.net. Grattafiori, A.; Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; et al. 2024. The Llama 3 Herd of Models. CoRR, abs/2407.21783. Hendrycks, D.; Burns, C.; Kadavath, S.; Arora, A.; Basart, S.; Tang, E.; Song, D.; and Steinhardt, J. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. In Vanschoren, J.; and Yeung, S., eds., Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (NeurIPS Datasets and Benchmarks 2021). Huang, J.; Chen, X.; Mishra, S.; Zheng, H. S.; Yu, A. W.; Song, X.; and Zhou, D. 2024. Large Language Models Cannot Self-Correct Reasoning Yet. In The Twelfth International Conference on Learning Representations (ICLR). OpenReview.net. Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; de las Casas, D.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; Lavaud, L. R.; Lachaux, M.-A.; Stock, P.; Le Scao, T.; Lavril, T.; Wang, T.; Lacroix, T.; and El Sayed, W. 2023. Mistral 7B. CoRR, abs/2310.06825. Kamoi, R.; Das, S. S. S.; Lou, R.; Ahn, J. J.; Zhao, Y.; Lu, X.; Zhang, N.; Zhang, Y.; Zhang, R. H.; Vummanthala, S. R.; Dave, S.; Qin, S.; Cohan, A.; Yin, W.; and Zhang, R. 2024a. Evaluating LLMs at Detecting Errors in LLM Responses. In First Conference on Language Modeling (COLM). Kamoi, R.; Zhang, Y.; Zhang, N.; Han, J.; and Zhang, R. 2024b. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs. Transactions of the Association for Computational Linguistics, 12: 1417–1440. Khalaf, H.; Verdun, C. M.; Oesterling, A.; Lakkaraju, H.; and Calmon, F. d. P. 2025. Inference-Time Reward Hacking in Large Language Models. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025). Spotlight. Khalifa, M.; Agarwal, R.; Logeswaran, L.; Kim, J.; Peng, H.; Lee, M.; Lee, H.; and Wang, L. 2026. Process Reward Models That Think. Transactions on Machine Learning Research. Kumar, A.; Zhuang, V.; Agarwal, R.; Su, Y.; Co-Reyes, J. D.; Singh, A.; Baumli, K.; Iqbal, S.; Bishop, C.; Roelofs, R.; Zhang, L. M.; McKinney, K.; Shrivastava, D.; Paduraru, C.; Tucker, G.; Precup, D.; Behbahani, F.; and Faust, A. 2025. Training Language Models to Self-Correct via Reinforcement Learning. In The Thirteenth International Conference on Learning Representations (ICLR). OpenReview.net. Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Flinn, J.; Seltzer, M. I.; Druschel, P.; Kaufmann, A.; and Mace, J., eds., Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23), 611–626. Koblenz, Germany: ACM. ISBN 9798400702297. Lightman, H.; Kosaraju, V.; Burda, Y.; Edwards, H.; Baker, B.; Lee, T.; Leike, J.; Schulman, J.; Sutskever, I.; and Cobbe,
K. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations (ICLR). OpenReview.net. Lu, J.; Teehan, R.; Jin, J.; and Ren, M. 2025. When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers. arXiv:2512.02304. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; Gupta, S.; Majumder, B. P.; Hermann, K.; Welleck, S.; Yazdanbakhsh, A.; and Clark, P. 2023. Self-Refine: Iterative Refinement with Self-Feedback. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 46534–46594. Curran Associates, Inc. Pan, A.; Jones, E.; Jagadeesan, M.; and Steinhardt, J. 2024a. Feedback Loops With Language Models Drive In-Context Reward Hacking. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, 39154–39200. PMLR. Pan, J.; He, H.; Bowman, S. R.; and Feng, S. 2024b. Spontaneous Reward Hacking in Iterative Self-Refinement. arXiv:2407.04549. Patil, S. G.; Mao, H.; Cheng-Jie Ji, C.; Yan, F.; Suresh, V.; Stoica, I.; and E. Gonzalez, J. 2025. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. In Forty-second International Conference on Machine Learning. Platanios, E. A.; Blum, A.; and Mitchell, T. M. 2016. Estimating Accuracy from Unlabeled Data: A Bayesian Approach. In Proceedings of the 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 8634–8652. Curran Associates, Inc. Stroebl, B.; Kapoor, S.; and Narayanan, A. 2026. The Limits of Inference Scaling Through Resampling. In The Fourteenth International Conference on Learning Representations (ICLR). Sun, R.; Cheng, W.; Li, D.; Chen, H.; and Wang, W. 2026. Stop When Enough: Adaptive Early-Stopping for Chain-ofThought Reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics. San Diego, California, USA: Association for Computational Linguistics. ArXiv:2510.10103. Wang, X.; Wei, J.; Schuurmans, D.; Le, Q.; Chi, E.; Narang, S.; Chowdhery, A.; and Zhou, D. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In The Eleventh International Conference on Learning Representations (ICLR). OpenReview.net. Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Wei, H.; et al. 2024. Qwen2.5 Technical Report. arXiv:2412.15115.
Yang, Z.; Zhang, Y.; Wang, Y.; Xu, Z.; Lin, J.; and Sui, Z. 2025. A Probabilistic Inference Scaling Theory for LLM Self-Correction. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 13573– 13587. Suzhou, China: Association for Computational Linguistics. Ye, Z.; Melo, L. C.; Kaddar, Y.; Blunsom, P.; Staton, S.; and Gal, Y. 2025. Uncertainty-Aware Step-wise Verification with Generative Reward Models. In ICLR 2025 Workshop on Quantify Uncertainty and Hallucination in Foundation Models. Yu, F.; Li, Y.; and Wang, B. 2025. Scaling Flaws of Verifier-Guided Search in Mathematical Reasoning. arXiv:2502.00271. Zhang, L.; Hosseini, A.; Bansal, H.; Kazemi, M.; Kumar, A.; and Agarwal, R. 2025a. Generative Verifiers: Reward Modeling as Next-Token Prediction. In The Thirteenth International Conference on Learning Representations (ICLR). OpenReview.net. Zhang, Z.; Zheng, C.; Wu, Y.; Zhang, B.; Lin, R.; Yu, B.; Liu, D.; Zhou, J.; and Lin, J. 2025b. The Lessons of Developing Process Reward Models in Mathematical Reasoning. In Che, W.; Nabende, J.; Shutova, E.; and Pilehvar, M. T., eds., Findings of the Association for Computational Linguistics: ACL 2025, 10495–10516. Vienna, Austria: Association for Computational Linguistics. ISBN 979-8-89176-256-5. Zhao, E.; Awasthi, P.; and Gollapudi, S. 2025. Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. Vancouver, BC, Canada: PMLR. Zhong, J.; Li, Z.; Xu, Z.; Wen, X.; Li, K.; and Xu, Q. 2025. Solve-Detect-Verify: Inference-Time Scaling with Flexible Generative Verifier. arXiv:2505.11966. Zhou, Y.; Xu, A.; Wang, P.; Xiong, C.; and Joty, S. 2025. Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. Vancouver, BC, Canada: PMLR.