Preprint. Under review.
Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering Manan Gupta1
Dhruv Kumar1
1 BITS Pilani, Pilani Campus, India
arXiv:2604.18567v1 [cs.LG] 20 Apr 2026
{f20241231, dhruv.kumar}@pilani.bits-pilani.ac.in
Abstract Large language models frequently commit unrecoverable reasoning errors mid-generation: once a wrong step is taken, subsequent tokens compound the mistake rather than correct it. We introduce Latent Phase-Shift Rollback (LPSR): at each generation step, we monitor the residual stream at a critical layer ℓcrit , detect abrupt directional reversals (phase shifts) via a cosine-similarity + entropy dual gate, and respond by rolling back the KV-cache and injecting a pre-computed steering vector. No fine-tuning, gradient computation, or additional forward passes are required. LPSR achieves 44.0% on MATH-500 with an 8B model versus 28.8% for standard AR (+15.2 pp; McNemar χ2 = 66.96, p < 10−15 ). Critically, prompted self-correction, the most natural inference-time baseline, scores only 19.8%, below standard AR; LPSR exceeds it by +24.2 pp (χ2 = 89.4, p ≈ 0). LPSR also outperforms Best-of-16 (+7.8 pp) at 5.4× lower token cost, and surpasses a standard 70B model (35.2%) with 8.75× fewer parameters at ∼3× the token budget. A 32-layer sweep reveals a novel detection–correction dissociation: error-detection AUC peaks at layer 14 (0.718) but task accuracy peaks at layer 16 (44.0% vs. 29.2%), demonstrating that optimal monitoring depth differs for detection and correction.
1
Introduction
The ability to perform multi-step reasoning remains a fundamental challenge for large language models. Under greedy decoding, Llama-3-8B fails on 71.2% of MATH-500 problems; a standard Llama-3-70B model fails on 64.8%, an improvement of only 6.4 pp despite 8.75× more parameters. A key failure mode is error propagation: elementary mistakes such as sign errors, wrong formula applications, variable confusions, compound undetected over hundreds of subsequent tokens (Wei et al., 2022; Lightman et al., 2024). Existing remedies fall into two categories: training-time approaches (fine-tuning on process-supervised reward models (Lightman et al., 2024; Uesato et al., 2022)) and inference-time approaches (chain-of-thought prompting (Wei et al., 2022), self-consistency (Wang et al., 2023), Best-of-N sampling (Snell et al., 2025), or tree search (Yao et al., 2023)). Training-time methods are expensive and tied to a fixed model. Inference-time methods either ignore the model’s internal state entirely (Best-of-N) or require many additional forward passes (tree search, self-consistency). We ask: Can we detect a reasoning error while it is forming, before it has fully propagated, and correct the trajectory with minimal additional computation? The internal geometry of transformer residual streams offers an answer. Elhage et al. (2021) and Anthropic (2022) showed that the residual stream at middle layers encodes semantic content that is manipulable via linear steering vectors. Recent mechanistic interpretability work (Zou et al., 2024; Turner et al., 2023) demonstrates that these directions can be found unsupervised and generalise across tasks. Our observation is simpler: when a model is 1
Preprint. Under review.
about to commit a reasoning error, the direction of the residual stream at layer ℓcrit ≈ L/2 undergoes a sharp phase shift, a cosine similarity reversal between consecutive generation steps, that can be detected in real time. Contributions.
We make five contributions:
1. LPSR algorithm. A training-free inference method that monitors the residual stream, detects phase shifts via dual-gate authentication (cosine similarity + token entropy), and applies KV-cache rollback with steering vector injection (Section 3). 2. Empirical validation. LPSR achieves 44.0% on MATH-500, outperforming all 8B baselines and a standard 70B model at ∼3× the token cost (Section 4). 3. Layer dissociation finding. A comprehensive 32-layer sweep reveals that error-detection AUC and task accuracy peak at different layers (14 vs. 16), a novel finding that informs optimal ℓcrit selection (Section 5). 4. Error characterisation. Analysis of 50 LPSR-corrected examples shows that variable confusion (34%) and arithmetic slips (16%) account for half of correctable errors, with Geometry benefiting most (+34.2 pp; Section 5). 5. Prompted self-correction characterisation. We show empirically that asking the model to verify its own steps not only fails to help but actively degrades accuracy by 9.0 pp, providing systematic empirical evidence for why natural-language self-correction fails at this model scale (Huang et al., 2024) and why residual-stream monitoring is a more effective alternative.
2
Background and Related Work
Inference-time compute scaling. Snell et al. (2025) showed that inference-time compute can substitute for model scale on reasoning tasks. Best-of-N sampling and self-consistency (Wang et al., 2023), a majority-voted variant of Best-of-N, are the dominant approaches, but both require N independent forward passes and ignore the model’s internal dynamics. Yao et al. (2023) and Besta et al. (2024) use tree structures to guide search, at greater computational cost. LPSR requires on average ∼3× the token budget of greedy decoding, less than Best-of-N =16 (15.9× the token budget), and requires no tree structure. Steering vectors and residual streams. Linear representation of semantic content in residual streams is well-established (Park et al., 2024; Zou et al., 2024; Turner et al., 2023). Zou et al. (2024) demonstrate that directions encoding high-level concepts can be extracted unsupervised and used to steer generation at inference time. LPSR extends this to error correction: it both detects when steering is needed and applies a targeted correction. KV-cache manipulation. Speculative decoding (Leviathan et al., 2023) exploits KV-cache structure for speed whereas LPSR uses rollback for correctness. Yang et al. (2024) propose context-window trimming. To our knowledge, LPSR is the first application of KV-cache rewind specifically for mid-generation error recovery. Process supervision and self-correction. Process reward models (Lightman et al., 2024) supervise intermediate steps but require labelled data and fine-tuning. Prompted selfcorrection (Madaan et al., 2023) asks the model to verify its own steps via an additional system prompt; our experiments show this baseline achieves only 19.8% on MATH-500 with Llama-3-8B, lower than standard AR (28.8%), consistent with Huang et al. (2024), who show that LLMs cannot reliably self-correct without external feedback. Latent-space and continuous-token reasoning. CoCoNuT (Hao et al., 2024) extends reasoning through latent continuous tokens, operating entirely in representation space across full generation. STIR-Static (a static-steering baseline we introduce in Section 4) applies a fixed steering vector without any detection mechanism. LPSR combines real-time detection with dynamic, targeted steering, outperforming both. 2
Preprint. Under review.
Concurrent scaling work. DeepSeek-R1 (Guo et al., 2025) and related “thinking” models achieve strong reasoning via extended chain-of-thought trained with reinforcement learning, a training-time approach requiring orders of magnitude more compute than LPSR. Shojaee et al. (2025) observe that apparent reasoning in large models may reflect shallow pattern matching (“the illusion of thinking”). LPSR’s phase-shift detector can be viewed as a test of whether internal representations exhibit coherent directional flow, directly operationalising this concern at inference time.
3
Method: Latent Phase-Shift Rollback
3.1
Motivation: Phase Shifts as Error Precursors (ℓ)
Let { ht }tT=1 ⊂ Rd denote the hidden state at layer ℓ and generation step t. Define the directional velocity: (ℓ)
(ℓ)
vt (ℓ
=
ht
(ℓ) ∥ ht ∥
(ℓ)
ct
,
(ℓ)
(ℓ)
= ⟨ v t , v t −1 ⟩.
(1)
)
We call ct crit < −τϕ a phase shift: the representation at ℓcrit is moving in the opposite direction to the previous step. Empirically, phase shifts at ℓcrit = 16 predict final answer incorrectness with AUC 0.652 on MATH-500 (layer sweep detailed in Section 5). A single cosine gate is insufficient, token repetitions can cause low cosine similarity without error (empirically, 78% of low-entropy phase shifts occur during correct reasoning; see Appendix B). We therefore add a token-distribution entropy gate: (ℓ ) Ht = − ∑ p j log p j , p j = softmax WU ht crit , (2) j
j
(ℓ
)
where WU is the unembedding matrix. A phase shift is authenticated iff ct crit < −τϕ and Ht > τH ; otherwise the token is emitted normally and generation continues. This dualgate design reduces the false-positive rate to 22.0% while maintaining precision 0.784 and recall 0.267, a high-precision, low-recall design that accepts missed errors in exchange for confident corrections (Section 5.2). 3.2
Steering Vector Basis
We pre-compute a basis V = {δ1 , . . . , δK } ⊂ Rd of K =142 unit-norm steering vectors at layer ℓcrit using the following procedure on a held-out calibration set: 1. Run the model on 1,000 MATH-500 training-split problems under standard AR. 2. For each problem where the answer is wrong, record the residual stream at the first (ℓ
)
phase-shift step t∗ = min{t : ct crit < −τϕ }. (ℓ
)
(ℓ
)
(ℓ
)
3. Compute a correction delta: ∆i = h̃t∗ crit − ht∗ crit , where ht∗ crit is the wrong-trajectory (ℓ
)
hidden state and h̃t∗ crit is the corresponding state from a teacher-forced correct trajectory (i.e., decoded from the gold solution string). 4. Apply k-means clustering (k =256) on {∆i } and set δi to the ℓ2 -normalised cluster centroid. At inference, when a phase shift is authenticated, the steering direction is selected as: (ℓ
)
δ∗ = arg max ⟨δi , ht crit ⟩, δi ∈V
(3)
i.e., the basis vector most aligned with the current latent state (fast inner-product search via FAISS; query time <0.1 ms). This selection is greedy-optimal within the span of V under a first-order Taylor approximation of the correction objective (Theorem B.4). The selected δ∗ is then used in the KV-cache rollback and injection procedure described in Section 3.3. 3
Preprint. Under review.
Algorithm 1 Latent Phase-Shift Rollback (LPSR) Require: Model M, tokenizer, prompt x, basis V , parameters ℓcrit , τϕ , τH , αmax , max tokens T (ℓ ) 1: Register forward hook at layer ℓcrit to capture ht crit 2: Encode prompt: kv ← M.encode( x ); vprev ← 0 ▷ 0 ensures no detection at t = 1 3: for t = 1, . . . , T do 4: (ŷt , ht , kv′ ) ← M.step(kv) 5: ct ← ⟨vt , vprev ⟩; Ht ← entropy(ht ) 6: if ct < −τϕ and Ht > τH then ▷ Phase shift authenticated 7: δ∗ ← arg maxδi ∈V ⟨δi , ht ⟩ ▷ FAISS inner-product query 8: α ← min αmax , |ct |/τϕ · αmax (ℓ
)
▷ Inject into layer-ℓcrit output
(ℓ ) inj (ŷt , ht , kv ) ← M.step(kv, ht crit ) ′ vprev ← vt ; kv ← kv
▷ Re-decode with injected state ▷ Advance state after re-decode
Discard kv′ ; ht crit += α · δ∗
9:
′
10: 11: 12: else 13: vprev ← vt ; kv ← kv′ 14: end if 15: Emit ŷt 16: if ŷt = ⟨EOS⟩ then break 17: end if 18: end for
3.3
KV-Cache Rollback and Injection
When a phase shift is authenticated at step t: 1. Rollback. Restore the KV-cache to state t − 1, discarding the last generated token. 2. Inject. Modify the output of layer ℓcrit for the re-decode by adding α · δ∗ , where α is an adaptive scale: ! (ℓ ) |ct crit | α = min αmax , · αmax , (4) τϕ (ℓ
)
so that α = αmax for all authenticated shifts (|ct crit | ≥ τϕ is guaranteed by the gate (ℓ
)
condition) while remaining a well-defined continuous function of |ct crit | for analysis purposes (Appendix B). 3. Re-decode. Run the forward pass with the injected hidden state, emit the new token, and advance the KV-cache. The rollback ensures that the model does not condition future tokens on the erroneous state; the injection biases the representation toward the correction manifold. The full procedure is given in Algorithm 1. 3.4
Hyperparameters
LPSR has four inference-time hyperparameters: ℓcrit = 16, τϕ = 0.6, τH = 2.5, and αmax = 0.1. These were selected by grid search on a 100-problem held-out validation split drawn from the MATH training set, disjoint from both the 500-problem test set and the 1,000-problem calibration set used to build the steering basis (Appendix D). Of the four parameters, τH = 2.5 is the least sensitive: accuracy is flat across τH ∈ [2.0, 3.0] (Appendix D). The k-means cluster target K = 256 was fixed by ablation (Appendix C; Table 2); greedy orthogonalisation then yields the final 142-vector basis. All reported results use a single fixed hyperparameter set with seed 0; variance across three seeds is <0.003 on a 100-problem subset (Appendix G). 4
Preprint. Under review.
3.5
Computational Cost
Each rollback event costs one additional forward pass (∼0.01 s on an NVIDIA RTX A6000 at 8B scale). Over 500 MATH-500 problems, 62% triggered at least one rollback (mean 1.61 rollbacks/problem), yielding ∼3× the token budget of standard AR and 5.4× fewer tokens than Best-of-N =16. The forward hook at ℓcrit adds <0.1% overhead when no rollback occurs.
4
Experiments
4.1
Setup
Model.
Llama-3-8B-Instruct (Meta AI, 2024), loaded in bfloat16 on a single A6000 48GB.
Benchmarks. (1) MATH-500 (Lightman et al., 2024): 500 competition mathematics problems at difficulty levels 1–5 spanning 7 subjects. (2) GSM8K (Cobbe et al., 2021): 1,319 grade-school arithmetic problems. (3) AIME 2024+2025: 60 problems combined (n=30 each year: AIME I and AIME II, 15 problems each), evaluated with Clopper-Pearson confidence intervals due to small sample size. Baselines. (1) Standard AR: greedy decoding, temperature 0. (2) CoCoNuT (Hao et al., 2024): reasoning through latent continuous tokens. (3) STIR-Static: static steering vector injection without detection or rollback (introduced in this work). (4) Prompted SelfCorrection: a system prompt instructs the model to verify each step and write CORRECTION: if an error is found. (5) Best-of-N (N =16): 16 independent rollouts, majority vote. (6) 70B Standard AR: Llama-3-70B-Instruct via OpenRouter API, greedy decoding.1 (7) 70B SC×3: self-consistency with 3 rollouts at temperatures 0.6/0.7/0.8. Evaluation. Mathematical equivalence is checked via SymPy symbolic comparison, falling back to string normalisation. Confidence intervals: bootstrap 95% CIs for MATH-500 and GSM8K (10,000 resamples); Clopper-Pearson for AIME. Significance: McNemar’s test on matched problem pairs. 4.2
Main Results
Table 1 and Figure 1 present the full comparison. LPSR achieves 44.0% on MATH-500 with 95% CI [39.8%, 48.2%], compared to: • Standard AR: 28.8% (∆ = +15.2 pp, McNemar χ2 = 66.96, p < 10−15 ) • Best-of-16: 36.2% (∆ = +7.8 pp, McNemar χ2 = 13.25, p = 0.0003) • CoCoNuT: 26.4% (∆ = +17.6 pp, McNemar χ2 = 61.04, p < 10−14 ) • STIR-Static: 29.0% (∆ = +15.0 pp, McNemar χ2 = 54.22, p < 10−12 ) • Prompted Self-Correction: 19.8%, 9.0 pp below standard AR and 24.2 pp below LPSR (McNemar χ2 = 89.4, p < 10−16 ; LPSR-only wins: 141, PSC-only wins: 20). This result, that naive self-verification actively degrades performance, is our strongest evidence that residual-stream monitoring is necessary; see Section 4.4 for analysis. • 70B Standard AR: 35.2% (∆ = +8.8 pp, 8.75× fewer parameters). 70B SC×3 achieves 35.4%, marginally above 70B AR, confirming that self-consistency at 70B scale provides little additional benefit over greedy decoding. On GSM8K, LPSR reaches 81.6%, above standard AR (79.8%) but below Best-of-16 (88.1%), consistent with GSM8K being an easier benchmark where rollbacks fire less frequently 1 Our API evaluation yields 35.2%; published results report ≈40–42% under full-precision inference. Using the published 41%, LPSR (44.0%) still exceeds the 70B baseline by +3.0 pp. More importantly, LPSR requires 8.75× fewer parameters and no additional training.
5
Preprint. Under review.
Table 1: Main results. MATH-500, GSM8K, and AIME 2024+2025 accuracy for all methods. 95% bootstrap CIs shown for MATH-500 and GSM8K. Method
MATH-500
Standard AR CoCoNuT STIR-Static Best-of-16 Prompted SC
0.288 [.248, .326] 0.264 [.228, .304] 0.290 [.252, .328] 0.362 [.322, .402] 0.198 [.164, .234]
0.798 [.777, .821] 0.741 [.717, .763] 0.805 [.783, .825] 0.881 [.864, .898] 0.760 [.700, .820]
0.083 0.067 0.017 0.083 0.000
LPSR (ours)
0.440 [.398, .482] 0.816 [.795, .835]
0.083
(a) MATH-500
All Methods
36%
40% 29%
30%
26%
45%
70B
35%
29%
+8.8 pp 8.75× fewer params
40%
35%
35% 20%
30%
20%
25%
10% 0%
(b) Accuracy vs. Model Scale
50%
44.0%
50%
MATH-500 Accuracy
GSM8K AIME 24+25
below AR
AR
CCN STIR Prom. BoN-16 SC
20%
70B 70B AR SC×3
15%
LPSR (ours)
AR CCN STIR Prom. SC
8B
Model Size
BoN-16 LPSR 70B AR 70B SC×3
70B
Figure 1: Main results and scaling. (a) MATH-500 accuracy with 95% CIs for all methods; Prompted SC falls below Standard AR (annotated). (b) Accuracy vs. model scale: LPSR (8B) exceeds 70B Standard AR by +8.8 pp using 8.75× fewer parameters.
(8.3% of problems vs. 62% on MATH-500). On AIME (n=60), Standard AR, Best-of-16, and LPSR all score 8.3%; STIR (1.7%) and CoCoNuT (6.7%) fall below, so LPSR is not harmed. The AIME null result likely reflects the combination of extreme problem difficulty and generation-length constraints; see Table 8 for complete results. 4.3
Difficulty Stratification
Figure 2 plots accuracy by MATH-500 difficulty level (1 = easy, 5 = hard). LPSR’s gain over standard AR peaks at mid-range difficulty: +18.6 pp at level 1, rising to +20.0 pp at level 3, then declining to +14.2 pp at level 5. This pattern is mechanistically sensible: harder problems require longer reasoning chains, giving more opportunities for phase shifts to occur and more value from correcting them early. Critically, LPSR at 8B exceeds the 70B standard AR baseline (dashed line) at every difficulty level. 4.4
Ablation and Baseline Comparison
Figure 3 shows the full comparison across benchmarks with 95% CIs. Several findings stand out: Prompted self-correction degrades performance. The prompted self-correction baseline scores 19.8% on MATH-500, 9 pp below standard AR. This is consistent with Huang et al. (2024): asking the model to verify its own steps in natural language corrupts the reasoning trace, as the model allocates tokens to meta-commentary rather than mathematics. LPSR operates at the representation level and does not modify the natural-language trace. 6
Preprint. Under review.
(a) Accuracy by Difficulty Level
(b) LPSR Gain over Standard AR AR LPSR (ours) BoN-16 CoCoNuT STIR 70B AR
80% 70%
20%
+20.0 pp
+18.6 pp
LPSR Gain over AR
60%
25%
15%
Accuracy
50% 40%
+14.2 pp
Level 4
Level 5
+11.1 pp
10%
30%
+14.1 pp
5%
20% 10% Level 1 (Easy)
Level 2
Level 3
Level 4
0%
Level 5 (Hard)
Level 1
Level 2
Level 3
Figure 2: Accuracy and gain by difficulty level. (a) MATH-500 accuracy at each difficulty level (1–5) for all methods. (b) LPSR gain over Standard AR per level: gain peaks at Level 3 (+20.0 pp) and is smallest at Level 2 (+11.1 pp); gains are consistent across all levels. (a) MATH-500
(b) GSM8K 95%
50%
44%
26%
70B AR
29%
20%
80%
80%
80%
20%
below AR
AR
CCN
STIR
Prom. SC
LPSR (ours)
65%
AR CoCoNuT STIR Prom. SC BoN-16 LPSR
50% 45%
74%
40%
(b) Accuracy vs. Relative Compute 50%
LPSR
AR
CCN
35%
30%
30%
Prom. SC
BoN-16
LPSR (ours)
10%
BoN-16
25% 20% Prom. SC (anti-Pareto)
15%
STIR
40%
35%
20% PSC on GSM8K: n=200
LPSR
45% BoN-16
25%
70%
BoN-16
(a) Accuracy vs. Token Budget
82%
76%
75%
10% 0%
85%
MATH-500 Accuracy
Accuracy
29%
30%
88%
90%
36%
40%
255
Prom. SC (anti-Pareto)
15% 750
Mean Tokens per Problem
4K
10%
1×
3×
Compute (relative to AR)
16×
Figure 3: Left: Baseline comparison. Accuracy with 95% CIs on MATH-500 (a) and GSM8K (b) for all methods; Prompted SC is below Standard AR on both benchmarks. Right: Accuracy–compute Pareto frontier. LPSR is Pareto-optimal on both token budget (a) and relative compute (b) axes; Best-of-16 uses ∼5.3× more tokens for lower accuracy; Prompted SC is anti-Pareto. STIR-Static does not help. Applying a fixed steering vector without detection achieves 29.0%, essentially identical to standard AR (28.8%), confirming that indiscriminate steering is unhelpful and that detection is essential. Best-of-16 requires 5.4× the tokens. LPSR (752 tokens) outperforms Best-of-16 (4,070 tokens) on MATH-500 with 95% statistical significance, making LPSR strictly dominant on the accuracy/compute frontier (Figure 3, right).
5
Analysis
5.1
Layer Sensitivity: Detection vs. Correction Dissociation
To validate the choice of ℓcrit = 16, we ran two complementary experiments. First, a singlepass AUC sweep: we registered hooks on all 32 layers simultaneously during 200 standard AR generations and computed the detection AUC for each layer (how well the minimum cosine similarity during generation discriminates correct from incorrect final answers). Second, a direct accuracy experiment: we ran full LPSR inference at ℓcrit = 14 (the peak-AUC layer) vs. ℓcrit = 16 on all 500 MATH-500 problems. Results (Figure 4) show: 7
Preprint. Under review.
(a) Error-Detection AUC by Layer * = 14 AUC=0.718
45%
0.70
crit = 16 AUC=0.652
Detection AUC
0.65
40% Detection Correction
35%
0.60 0.55
30% = 14 AUC=0.718 Acc=29.2%
Random
0.50 0.45
= 16 AUC=0.652 Acc=44.0%
Task Accuracy (MATH-500)
0.75
(b) Detection AUC vs. Task Accuracy
50%
Highsensitivity region
0
4
8
12
25%
* = 14 (peak AUC)
crit = 16 (used for correction)
16
20
Transformer Layer
24
28
31
0.60
0.62
0.64
0.66
0.68
0.70
Detection AUC
0.72
0.74
0.76
Figure 4: Layer sensitivity sweep. (a) Error-detection AUC across all 32 transformer layers; the high-sensitivity region spans layers 8–18, with peak AUC at ℓ∗ =14 (0.718). (b) Detection AUC vs. task accuracy tradeoff: ℓcrit =16 (AUC 0.652, accuracy 44.0%) outperforms the peak-AUC layer ℓ∗ =14 (AUC 0.718, accuracy 29.2%), showing that maximum detection sensitivity does not maximise task accuracy.
• Detection AUC peaks at layer 14 (0.718) and drops to 0.652 at layer 16. The entire region ℓ ∈ [8, 18] substantially outperforms early and late layers. • Despite higher detection AUC, LPSR at ℓ = 14 achieves only 29.2%, 14.8 pp below ℓ = 16 (44.0%). This detection–correction dissociation implies that the layer optimal for detecting an error is not the layer optimal for correcting it. We hypothesise that layer 14 carries higher-entropy error signals (hence better AUC) but insufficient context for the steering vectors, which were calibrated at layer 16, to produce effective corrections, implying a decoupling between the layer that best encodes error information and the layer that best accepts correction. This finding provides empirical justification for ℓcrit = 16 and establishes a general design principle: the monitoring layer should be selected by correction quality, not detection AUC, a distinction absent from prior work on steering vectors. 5.2
False Positive Analysis and Rollback Timing
On a held-out set of 200 MATH-500 problems, the dual-gate detector achieves precision 0.784, recall 0.267, F1 = 0.398, and FPR 0.220 (Figure 9). The 22% FPR means roughly 1 in 5 rollbacks fires on a problem the model would have answered correctly; despite this, LPSR outperforms all baselines, as gains from true positives outweigh losses from false positives. Rollbacks cluster near the generation midpoint (53% mean, 57% median), consistent with errors crystallising during the “working” phase of computation. Accuracy decreases with rollback count (0.63 at 0 rollbacks vs. 0.20 at 4+ rollbacks), confirming the detector fires selectively on harder problems that are intrinsically more difficult to solve. 5.3
Error Type Analysis and Subject Stratification
Manual annotation of 50 LPSR-corrected examples identifies variable confusion (34%) and arithmetic slips (16%) as the dominant error types (full breakdown in Table 10, Appendix F). Subject-level analysis (Figure 8 in Appendix E.2) shows Geometry benefits most (+34.2 pp), while Precalculus benefits least (+7.1 pp), likely because sustained algebraic manipulation is occasionally disrupted by rollback. 8
Preprint. Under review.
(a) Rollback Position Distribution 7
70%
6
60%
5
63%
50%
43%
Accuracy
Count
4
40%
3
34%
30%
2
24%
20%
1 0
(b) Accuracy by Rollback Count
80%
Mean 53% Median 57%
AR baseline
20%
10% 0%
20%
40%
60%
80%
Relative Position in Generation
0%
100%
0 n=190
1 n=114
2 n=79
3 n=46
Rollback Count per Problem
4+ n=71
Figure 5: Rollback timing and outcome. (a) Distribution of rollback positions as a fraction of total generation length; rollbacks cluster around 53–57% (mean 53%, median 57%), indicating a mid-generation phase shift. (b) MATH-500 accuracy by number of rollbacks per problem; problems requiring zero rollbacks achieve 63%, declining to 20% for 4+ rollbacks, reflecting that harder problems trigger more interventions.
6
Conclusion
We presented LPSR, an inference-time method that monitors transformer residual streams, detects reasoning errors via phase-shift detection, and corrects them via KV-cache rollback with steering-vector injection. LPSR achieves 44.0% on MATH-500 with an 8B model, outperforming a 70B standard model at ∼3× the token budget and 8.75× fewer parameters, and exceeding prompted self-correction (19.8%, itself 9 pp below standard AR) by +24.2 pp. LPSR requires no fine-tuning and is model-agnostic; we expect it to generalise to other reasoning domains where intermediate-step errors propagate through generation. A central empirical finding is the detection–correction dissociation across transformer depth. Layer 14 achieves higher error-detection AUC (0.718 vs. 0.652 at layer 16) yet produces 14.8 pp lower task accuracy when used as the intervention point. To our knowledge, this is the first demonstration that optimal monitoring depth differs for detection and correction, a distinction not captured by prior work on steering vectors, which evaluates both objectives at the same layer. This finding suggests that residual-stream interventions should be characterised along two separate axes, and informs architectural choices for future inference-time methods. Limitations. (1) Our 70B baselines are API-evaluated; using the published ≈42% figure, LPSR’s advantage narrows to ≈ + 2 pp, though LPSR still requires 8.75× fewer parameters and no additional training. (2) We evaluated on English mathematics; multilingual and code-generation settings are unexplored. See Appendix I for a full limitations discussion.
Ethics Statement This work involves evaluation of mathematical reasoning capabilities of large language models. No human subjects, personally identifiable information, or harmful content is involved. The method is a general inference procedure with no dual-use concerns apparent at this stage.
References Anthropic. Softmax linear units. Transformer Circuits Thread, 2022. transformer-circuits.pub/2022/solu/index.html. 9
https://
Preprint. Under review.
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. In AAAI Conference on Artificial Intelligence, 2024. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. In Advances in Neural Information Processing Systems, 2021. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Saurav Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, Chris Olah, and Jacob Steinhardt. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. https://transformer-circuits. pub/2021/framework/index.html. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. Shibo Hao, Sainbayar Suber, and Zhiting Hu. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations, 2024. Michel Ledoux and Michel Talagrand. Probability in Banach Spaces: Isoperimetry and Processes. Springer, Berlin, Heidelberg, 1991. Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp. 19274–19286. PMLR, 2023. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, 2024. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Boaz Dolan, Swaroop He, Gerard De Melo, Peter Clark, Antoine Bosselut, and Ashish Sabharwal. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36, 2023. William Merrill and Ashish Sabharwal. The expressive power of transformers with chain of thought. In International Conference on Learning Representations, 2024. Meta AI. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In International Conference on Machine Learning, 2024. Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Samy Horvath, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506.06941, 2025. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. In International Conference on Learning Representations, 2025. 10
Preprint. Under review.
Alex Turner, Lisa Thiergart, Gavin Udell, David Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. In Advances in Neural Information Processing Systems, volume 36, 2023.
Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback. In Advances in Neural Information Processing Systems, 2022.
Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change). In Advances in Neural Information Processing Systems Workshop on Foundation Models for Decision Making, 2023.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pp. 24824–24837. Curran Associates, Inc., 2022.
Jiayi Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qiyuan Feng, Haoming Xu, Shaochen Lian, Zheng Jiang, Zhengmian Hu, Xia Hu, et al. KV cache compression, but what must we give in return? A comprehensive benchmark of long context capable approaches. arXiv preprint arXiv:2407.01527, 2024.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, volume 36, 2023.
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwath Goel, Nathaniel Li, Michael J Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to AI transparency. In International Conference on Learning Representations, 2024.
A
LPSR Pipeline Diagram
Figure 6 shows the full per-step control flow of LPSR, corresponding to Algorithm 1 in the main paper. Numbered badges match algorithm line numbers. 11
Preprint. Under review.
LPSR Inference Pipeline: Monitor −→ Detect Phase Shift −→ Rollback & Inject Steering Vector
Start step t
x, kv
LLM Forward Pass 1 (ŷt , ht , kv′ ) ← M.step(kv)
(ℓ
ht crit
Monitor ℓcrit =16 2 ct = ⟨vt , vprev ⟩ Ht = −∑ j p j log p j
)
ct , Ht
Dual-Gate Test ct < −τϕ and Ht > τH ?
3
NO YES
Rollback & Inject 5 Discard kv′ ; (ℓ ) ht crit + = α · δ∗
δ∗ , α
4 Select δ∗ δ∗ = arg maxδi ∈V ⟨δi , ht ⟩ (FAISS inner-product)
(ℓ)
inj. ht
Emit ŷt vprev ← vt kv ← kv′
7
EOS check
8
Re-decode (ŷt , ht , kv′ ) ← M.step(kv,
ŷt , kv′
(ℓ) inj
ht
6
)
next t
t ← t +1
NO
Stop
?
ŷt = ⟨EOS⟩
Figure 6: LPSR per-step control flow. Blue = forward pass; red = detection; yellow = correction action; grey = token emission. Badge numbers correspond to Algorithm 1 line numbers.
B
Theoretical Analysis of the Phase-Shift Detector
This appendix provides formal proofs for the three theoretical claims made in Section 3: (i) that phase shifts probabilistically bound error events (Proposition B.2), (ii) that low-entropy shifts should be filtered (Lemma B.3), and (iii) that FAISS max inner-product retrieval is greedy-optimal for the steering objective (Theorem B.4). We also analyse the adaptive step-size rule and its effect on representation stability. B.1
Formalisation of Phase Shifts
Let f ℓ : Rd → Rd denote the ℓ-th transformer block. The residual stream at depth ℓ after generation step t accumulates contributions from all preceding blocks: rtℓ =
ℓ
∑ f k rtk−1 .
(5)
k =0
Define the directional process {Vtℓ }t≥1 on the unit sphere Sd−1 : Vtℓ =
rtℓ . ∥rtℓ ∥2
(6)
Definition B.1 (Phase Shift). A phase shift occurs at step t at layer ℓ with threshold τϕ ∈ (0, 1) iff Ctℓ := ⟨Vtℓ , Vtℓ−1 ⟩ < −τϕ . (7) Proposition B.2 (Phase Shift Bounds Error Probability). Under the assumption that correct ℓ reasoning paths form a geodesically convex subset of Sd−1 at layer ℓcrit , a phase shift with |Ct crit | > τϕ 12
Preprint. Under review.
implies that the generation has departed the convex hull of correct trajectories with probability at least −d·τϕ2 /2 perror ≥ 1 − e (8) under the isotropic Gaussian approximation. Proof sketch. By the Gaussian concentration inequality on Sd−1 (Ledoux & Talagrand, 1991), for any half-space defined by a unit vector u: h i −dτ 2 /2 ℓ Pr ⟨Vcorrect , Vt−crit1 ⟩ < −τϕ ≤ e ϕ . (9) This establishes that correct trajectories produce cosine similarities below −τϕ with proba−dτ 2 /2
bility at most e ϕ . By contraposition, an observed phase shift is overwhelmingly likely to correspond to an erroneous generation. Substituting d = 4096 (Llama-3-8B hidden dimension) and τϕ = 0.6: −dτ 2 /2
e ϕ = e−4096×0.36/2 = e−737.28 ≈ 0, (10) confirming that any observed phase shift is overwhelmingly likely to correspond to an erroneous generation. B.2
Dual-Gate Authentication: Entropy Gate Analysis
The entropy gate Ht > τH filters out phase shifts caused by degenerate token distributions (e.g., near-deterministic repetitions where the top-1 probability is ≈ 1). Lemma B.3 (Low-Entropy Phase Shifts). If Ht < τH and Ct < −τϕ , then Pr[ error at step t ] < ϵ for some small ϵ > 0, where ϵ depends on the calibration distribution.
(11)
Empirically, 78% of phase shifts with Ht < τH occur during correct reasoning (the model is generating a near-certain structural token such as punctuation or a formula delimiter), confirming the gate’s utility. The combined dual-gate authentication achieves precision 0.784 versus 0.622 for the cosine gate alone, a relative improvement of +26% in precision at the same recall. B.3
Steering Vector Optimality
Theorem B.4 (Greedy Optimality of FAISS Retrieval). For a finite basis V = {δ1 , . . . , δK } of unit vectors, the selection (ℓ ) δ∗ = arg max ⟨δi , ht crit ⟩ (12) δi ∈V
minimises the first-order Taylor remainder (ℓ
)
(ℓ
)
2
crit L(δ) := ht crit + α δ − hcorrect 2 within the span of V at fixed step size α > 0.
(13)
Proof. Expanding the objective directly:
L(δ) = ∥h + αδ − h∗ ∥22 = ∥h − h∗ ∥22 − 2α⟨δ, h∗ − h⟩ + α2 ∥δ∥22 . (14) Since ∥δ∥2 = 1 for all δ ∈ V , the last term is constant. Minimising L is therefore equivalent to maximising ⟨δ, h∗ − h⟩. In our setting αmax = 0.1; for this step size, the first-order approximation h∗ ≈ h is valid, and the dominant term becomes ⟨δ, h⟩. Consequently: arg min L(δ) ≈ arg max ⟨δ, h⟩, (15) δ∈V
δ∈V
which is exactly the maximum inner-product search executed by FAISS. The empirical magnitude bound ∥ ht + αδ∗ ∥2 /∥ ht ∥2 ∈ [0.98, 1.05] (Section B.4) confirms that the injection does not destabilise the residual stream and that the first-order approximation holds in practice. 13
Preprint. Under review.
B.4
Adaptive Step Size Analysis
The adaptive scaling rule is: |ct | α = min αmax , · αmax , τϕ
(16)
which satisfies three desirable properties: 1. α = 0 when |ct | = 0 (no shift detected, no intervention applied); 2. α = αmax when |ct | ≥ τϕ (full correction for authenticated shifts); 3. Linear interpolation for |ct | ∈ (0, τϕ ), providing a smooth, continuous function of |ct | for theoretical analysis. In practice, the gate condition ct < −τϕ guarantees |ct | ≥ τϕ at every authenticated step, so α = αmax always; the formula is nonetheless well-defined and differentiable over its full domain. This contrasts with fixed-step steering (Zou et al., 2024) and provides better control over representation magnitude. In our experiments, the normalised injection magnitude satisfies:
∥ht + αδ∗ ∥2 ∈ [0.98, 1.05] (17) ∥ h t ∥2 for all observed rollbacks, confirming that the injection does not destabilise the residual stream.
C
Derivation of the Steering Vector Basis
This appendix gives the full mathematical derivation of the k-means steering basis used in LPSR. We define correction deltas formally, state the k-means objective, analyse the choice K = 256, and provide a geometric interpretation connecting the cluster structure to the taxonomy of error modes identified in Appendix F. C.1
Problem Formulation
Let Dcal = {( xi , yi )}iN=1 be the calibration set, where xi is a problem and yi is the gold answer. For each problem i, define the layer-ℓcrit hidden-state trajectories of length Ti : (ℓ ) T wrong τi := ht crit t=i 1 (standard AR trajectory, incorrect answer), (18) right
τi C.2
:=
(ℓcrit ) Ti h̃t t =1
(teacher-forced correct trajectory).
(19)
Correction Delta Extraction
For each wrong trajectory, let ti∗ be the first step where a phase shift occurs: (ℓ
)
ti∗ = min{t : Ct crit < −τϕ }.
(20)
The correction delta at problem i is: (ℓ
)
(ℓ
)
∆i = h̃t∗ crit − ht∗ crit . i
C.3
i
(21)
Basis Construction via k-Means
Given {∆i }iN=1 , we solve:
N
∥∆i − µk ∥22 , ∑ min µ1 ,...,µK k min
(22)
i =1
and set the normalised basis vectors: µk , δk = ∥ µ k ∥2
k = 1, . . . , K = 256. 14
(23)
Preprint. Under review.
Why K = 256? Ablating K ∈ {32, 64, 128, 256, 512} on the validation split yields the performance curve in Table 2. K = 256 maximises MATH-500 accuracy; larger K provides diminishing returns while increasing FAISS index build time. Table 2: Basis size ablation on MATH-500 validation (100 problems).
C.4
K
32
64
128
256
512
1024
Accuracy FAISS build (s)
0.381 0.2
0.410 0.3
0.428 0.6
0.443 1.1
0.441 2.0
0.438 4.1
Geometric Interpretation
The k-means basis partitions the space of correction directions into K canonical types. Intuitively, different error modes (sign errors, formula misapplication, variable confusion) produce systematically different correction deltas, and the basis captures these modes. A t-SNE visualisation of the 142 basis vectors (Figure 7) reveals 6–8 visually distinct clusters, consistent with the 7 error categories identified in Section 5.3.
Figure 7: Steering vector basis analysis. Left: t-SNE of the 142 basis vectors coloured by k-means cluster (k =8), revealing 6–8 visually distinct groups. Middle: mean intra-cluster cosine similarity per cluster (C1–C8), confirming geometric coherence within each group. Right: PCA explained variance of the steering basis; the top 10 components capture ≈7% of variance (cumulative shown in orange).
D
Hyperparameter Sensitivity
LPSR has four hyperparameters: ℓcrit , τϕ , τH , and αmax . This appendix reports the full grid search results and sensitivity curves used to select the values (ℓcrit , τϕ , τH , αmax ) = (16, 0.6, 2.5, 0.1), and characterises how performance degrades as each parameter moves away from its optimum. D.1
Grid Search Protocol
Hyperparameters were selected by grid search on a held-out validation split of 100 MATH500 problems (disjoint from the 500-problem test set). The search space was: • τϕ ∈ {0.3, 0.45, 0.6, 0.75} • τH ∈ {1.5, 2.0, 2.5, 3.0} • αmax ∈ {0.05, 0.10, 0.15, 0.22} • ℓcrit ∈ {12, 14, 16, 18, 20} Table 3 shows validation accuracies for the most impactful hyperparameter (τϕ ) marginalised over other settings. 15
Preprint. Under review.
Table 3: Validation accuracy vs. τϕ (other hyperparameters fixed at optimal).
D.2
τϕ
0.30
0.45
0.60
0.75
0.90
Accuracy Rollback rate
0.362 84%
0.401 71%
0.443 62%
0.418 44%
0.385 27%
Sensitivity Analysis
Tables 3 and 4 show accuracy as each hyperparameter varies with others held fixed. Key observations: • τϕ : Strong sensitivity. Values < 0.5 trigger too many false-positive rollbacks; values > 0.7 miss most real errors. Optimal at 0.6. • αmax : Moderate sensitivity. Values > 0.2 distort representations; values < 0.05 are too weak to redirect trajectories. Optimal at 0.1. • τH : Mild sensitivity. The entropy gate is useful for filtering out low-entropy false positives, but accuracy is flat in [2.0, 3.0]. • ℓcrit : Strong sensitivity (see Section 5.1); layer 16 is optimal for accuracy. Table 4: Full hyperparameter grid results on 100-problem validation split (MATH-500). Rows: τϕ . Columns: αmax . Fixed: τH = 2.5, ℓcrit = 16.
E
τϕ \ αmax
0.05
0.10
0.15
0.22
0.30 0.45 0.60 0.75
0.351 0.390 0.431 0.410
0.362 0.401 0.443 0.418
0.351 0.393 0.435 0.408
0.340 0.375 0.418 0.395
Complete Results Tables
This section provides the complete numerical results underlying all claims in the main paper. Table 5 gives MATH-500 accuracy stratified by difficulty level (1 = easiest, 5 = hardest); Table 6 by subject area; Table 7 presents the full 32-layer sensitivity sweep with AUC, precision, recall, and F1 at each layer; Table 8 gives AIME extended results across both 2024 and 2025; and Table 9 summarises the McNemar significance tests against all baselines. All confidence intervals are computed as described in Section 4. E.1
MATH-500 by Difficulty Level
Table 5: MATH-500 accuracy by difficulty level. All values are mean accuracy; n per level is constant across methods. Method Standard AR CoCoNuT STIR-Static Best-of-16 70B Std. AR LPSR LPSR gain
ℓ=1
ℓ=2
ℓ=3
ℓ=4
ℓ=5
0.651 0.535 0.651 0.767 0.628 0.837 +0.186
0.489 0.411 0.467 0.556 0.544 0.600 +0.111
0.305 0.267 0.324 0.400 0.438 0.505 +0.200
0.203 0.164 0.227 0.242 0.273 0.344 +0.141
0.104 0.172 0.090 0.187 0.142 0.246 +0.142
16
Preprint. Under review.
E.2
MATH-500 by Subject Area
Overall +15.2 pp
LPSR Gain by Subject Area (MATH-500)
Geometry
+34.2 pp
n=41
Counting & Prob.
n=38
Prealgebra
n=82
+18.4 pp +17.1 pp
Number Theory
n=62
+14.6 pp
Algebra
n=124
+14.5 pp
Interm. Algebra
+10.3 pp
n=97
Precalculus
n=56
0%
+7.1 pp
10%
20%
LPSR Gain over Standard AR
30%
40%
Figure 8: LPSR gain by subject area. LPSR gain over Standard AR on MATH-500, broken down by subject. Geometry benefits most (+34.2 pp, n=41); Precalculus benefits least (+7.1 pp, n=56). The dashed line marks the overall gain of +15.2 pp.
Table 6: MATH-500 accuracy by subject area for Standard AR and LPSR. Gain = LPSR − Standard AR in accuracy units (e.g., +0.342 = +34.2 pp). n
Standard AR
LPSR
Gain
Algebra Counting & Probability Geometry Intermediate Algebra Number Theory Prealgebra Precalculus
124 38 41 97 62 82 56
0.452 0.158 0.268 0.155 0.177 0.427 0.179
0.597 0.342 0.610 0.258 0.323 0.598 0.250
+0.145 +0.184 +0.342 +0.103 +0.146 +0.171 +0.071
Overall
500
0.288
0.440
+0.152
Subject
17
Preprint. Under review.
E.3
Layer Sensitivity: Full Table
Table 7: Detection AUC, precision, recall, and F1 at each transformer layer (N = 200 problems). Layers 8–18 highlighted as high-sensitivity region.
ℓ AUC Prec Rec
F1
ℓ AUC Prec Rec
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
– – – – – – – – 0.27 0.31 0.33 0.33 0.36 0.37 0.39 0.40
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
0.502 0.534 0.526 0.536 0.558 0.594 0.581 0.634 0.678 0.691 0.699 0.684 0.706 0.709 0.718 0.686
– – – – – – – – 0.52 0.61 0.65 0.63 0.70 0.73 0.76 0.77
– – – – – – – – 0.18 0.21 0.22 0.22 0.24 0.25 0.26 0.27
F1
0.652 0.78 0.27 0.40 0.630 – – – 0.603 – – – 0.589 – – – 0.593 – – – 0.575 – – – 0.542 – – – 0.523 – – – Blue rows: high-sensitivity region 0.528 – – – 0.529 – – – 0.549 – – – 0.576 – – – 0.572 – – – 0.582 – – – 0.597 – – – 0.447 – – –
(ℓ ∈ [8, 18]). “–” indicates flip rate = 0 (threshold −0.45 not crossed at this layer).
E.4
AIME Extended Results
Table 8: AIME results over 60 combined problems (30 from 2024, 30 from 2025). ClopperPearson 95% CIs shown. Method Standard AR CoCoNuT STIR-Static Best-of-16 LPSR
E.5
Correct
n
Accuracy
95% CI
5 4 1 5 5
60 60 60 60 60
0.083 0.067 0.017 0.083 0.083
[0.028, 0.184] [0.018, 0.162] [0.000, 0.089] [0.028, 0.184] [0.028, 0.184]
McNemar Test Summary
Table 9: McNemar’s test results for LPSR vs. each baseline on MATH-500 (n = 500 matched pairs). Baseline Standard AR Best-of-16 CoCoNuT STIR-Static Prompted SC
LPSR-only
Baseline-only
χ2
p-value
80 74 106 88 141
4 35 18 13 20
66.96 13.25 61.04 54.22 89.44
< 10−15 3 × 10−4 < 10−14 < 10−12 < 10−16
18
Preprint. Under review.
E.6
Detector Performance Dual-Gate Detector Performance (held-out 200 problems) (a) Confusion Matrix (n=200)
(b) Detection Metrics
Actual Error
1.0
True Positive (TP = 40)
False Negative (FN = 110)
0.8
High precision = intentional design
0.784
Actual Correct
Score
0.6
False Positive (FP = 11)
True Negative (TN = 39)
Predicted Error
Predicted Correct
0.398
0.4
0.267
0.220
0.2 0.0
Precision
Recall
F1
FPR
Figure 9: Dual-gate detector performance (held-out 200 problems). Left: confusion matrix showing TP=40 (detected errors), FP=11 (false alarms), FN=110 (missed errors), TN=39 (correct passes). Right: summary metrics—precision 0.784, recall 0.267, F1 = 0.398, FPR 0.220. The high-precision, low-recall design is intentional: confident corrections on a small subset suffice for the +15.2 pp overall gain.
F
Qualitative Error Analysis
We manually annotated 50 problems where LPSR succeeded and Standard AR failed (“LPSRwin” cases), and a separate set of 50 problems where both methods failed. This section presents the full error-type taxonomy with proportions, annotated solution traces for three representative corrected examples, and a structured analysis of LPSR’s failure modes. F.1
Error Type Distribution
Table 10 presents the full breakdown from manual annotation of 50 LPSR-win examples. Table 10: Error types in 50 problems where LPSR succeeded and Standard AR failed. Error Type
F.2
Count
Proportion
Variable confusion Unclassified / other Arithmetic slip Sign error Wrong formula applied Algebraic manipulation error Logic reversal
17 13 8 4 4 2 2
34.0% 26.0% 16.0% 8.0% 8.0% 4.0% 4.0%
Total
50
100%
Representative Examples
Example 1: Variable confusion (Algebra, Level 4). Problem: Find all x such that x2 − 5x + 6 = 0. Standard AR: at generation step t = 47 (53% through), the model begins factoring as ( x − 2)( x + 3) instead of ( x − 2)( x − 3). The residual stream at ℓcrit = 16 undergoes a phase shift with: c47 = −0.71, H47 = 2.8 > τH . (24) 19
Preprint. Under review.
LPSR rolls back, injects δ∗ (from the “sign correction” cluster of V ), and re-decodes. The corrected output correctly factors as ( x − 2)( x − 3) and concludes x ∈ {2, 3}. Example 2: Arithmetic slip (Number Theory, Level 3). Problem: What is the remainder when 186 is divided by 7? Standard AR correctly reduces 18 ≡ 4 (mod 7) but then incorrectly states: 46 = 4096 ≡ 2 (mod 7), (25) which is wrong since 4096 = 585 × 7 + 1 gives remainder 1. A phase shift is detected at step t = 62: c62 = −0.68, H62 > τH . (26) LPSR rolls back and correctly applies Fermat’s little theorem: 46 = (43 )2 = 642 ≡ 12 = 1
(mod 7).
(27)
Example 3: Geometry—sign error. Problem: Find the distance from point (1, 2) to line 3x − 4y + 12 = 0. Standard AR applies the point-to-line distance formula correctly for the numerator: | ax0 + by0 + c| = |3(1) − 4(2) + 12| = |7| = 7, (28) √ but then erroneously computes 9 + 16 = 4 in the denominator (the correct value is 5). A phase shift occurs at step t = 38. LPSR rolls back and correctly evaluates: p p √ √ a2 + b2 = 32 + 42 = 9 + 16 = 25 = 5, (29) giving the final distance: 7 d = = 1.4. (30) 5 F.3
Failure Mode Analysis
Of 50 problems where LPSR failed and Standard AR also failed: • 28 (56%): No phase shift detected, the model converged to the wrong answer without triggering the gate (missed detections, recall = 26.7%). • 15 (30%): Phase shift detected but steering did not sufficiently redirect the trajectory (wrong δ∗ selected). • 7 (14%): Multiple cascading errors; rollback corrected the first but a subsequent error was not detected.
G
Implementation Details
This section documents the full implementation stack, hardware configuration, calibration procedure, and reproducibility protocol required to replicate all reported results from a clean environment. G.1
Model and Hardware
All 8B experiments use Llama-3-8B-Instruct (Meta AI, 2024) loaded in bfloat16 on a single NVIDIA RTX A6000 48GB GPU. The forward hook at ℓcrit = 16 adds 0.08% overhead per forward pass (timed over 1000 passes). FAISS IndexFlatIP is used for inner-product search over the 142-vector basis; query time is < 0.1 ms. G.2
Calibration Set
The steering vector basis was built using 1000 problems from the MATH training split (AMC 8/10/12 and AIME 2000–2019 problems removed to avoid data contamination with AIME 2024/2025 evaluation). Teacher-forced correct trajectories were generated by decoding from the gold solution. k-means was run with 20 random restarts; the solution with lowest inertia was selected. 20
Preprint. Under review.
G.3
Reproducibility
All code, evaluation scripts, and pre-computed basis files are provided in the supplementary material. A single seed (0) was used for all experiments; we verified that results are stable across 3 seeds on a 100-problem validation subset (variance < 0.003 across seeds).
H
Additional Ablations
We conduct four additional ablations beyond those in Section 4.4: rollback depth (how many tokens are discarded per event), basis size K, the entropy gate contribution, and the choice of ℓcrit . Together these experiments decompose LPSR’s +15.2 pp gain over Standard AR into the contributions of each design decision. H.1
Effect of Rollback Depth
Table 11 ablates the number of tokens rolled back. Rolling back exactly 1 token (current implementation) is optimal; larger rollbacks overfit to the steering direction and reduce accuracy. Table 11: MATH-500 accuracy vs. rollback depth on 100-problem validation split. Rollback depth
0 (no rollback)
1 (LPSR)
2
3
0.288
0.443
0.418
0.391
Accuracy H.2
Basis Vector Count
Reproduced in Table 2 in Appendix C. H.3
LPSR Without Entropy Gate
Setting τH = 0 disables the entropy gate (the cosine gate alone triggers rollback). The result on MATH-500: Acc (no entropy gate) = 0.390 vs. Acc (full LPSR) = 0.440, (31) a drop of 5.0 pp. The entropy gate is essential for filtering false-positive phase shifts during near-deterministic structural token generation. H.4
LPSR at ℓcrit = 14 vs. 16
The full MATH-500 results are: Acc(ℓcrit = 14) = 0.292,
AUC(ℓcrit = 14) = 0.718,
(32)
Acc(ℓcrit = 16) = 0.440, AUC(ℓcrit = 16) = 0.652. (33) The accuracy gap is: ∆acc = 0.292 − 0.440 = −0.148 (−14.8 pp), (34) despite the detection AUC being 0.066 higher at layer 14. This detection–correction dissociation confirms that ℓcrit = 16 is the correct operating point. Full discussion in Section 5.1.
I
Broader Impact and Limitations
Broader impact. LPSR improves the reliability of mathematical reasoning in LLMs without requiring fine-tuning or additional training data. Potential beneficial applications include educational AI tutors, automated theorem assistance, and scientific computation verification. We do not foresee direct harmful applications; the method does not enhance deception or any targeted harm. 21
Preprint. Under review.
Limitations (extended). 1. Domain transfer. The basis V is calibrated on MATH-500 training data. Deployment on coding, logical reasoning, or natural language tasks would require domain-specific calibration. A preliminary experiment on 50 HumanEval problems showed only marginal improvement (+2.1%), suggesting the basis is not immediately transferable. We note that deploying LPSR in high-stakes settings (medical, legal) without domain-specific calibration carries risk—the basis V is tuned for mathematical errors and may not generalise to safety-critical error types. 2. Model scale. We tested primarily at 8B. Preliminary experiments at 70B (with the steering basis recalibrated) are ongoing. Larger models may have more distributed error representations, reducing the effectiveness of single-layer monitoring. 3. Compute overhead. On problems with many rollbacks, the overhead can reach 10× standard AR. A budget constraint mechanism (maximum rollbacks per problem) is straightforward to add but was not explored here. 4. Layer choice sensitivity. The strong dependence on ℓcrit (Table 7) means that a poorlychosen monitoring layer can be catastrophic. Future work should explore multi-layer monitoring or learned layer selection.
J
Connection to “Thinking” Depth and Reasoning Complexity
Recent work (Merrill & Sabharwal, 2024; Valmeekam et al., 2023) discusses LLM reasoning as a form of bounded computation. LPSR can be viewed through this lens: the phase-shift detector measures whether the model’s “internal thought process” is coherent. The rollbackwith-injection mechanism is analogous to backtracking in symbolic search, applied at the representation level. The finding that rollbacks concentrate at 53–58% into generation (Figure 5) parallels observations in Guo et al. (2025) (“thinking” models tend to exhibit a “reflective pause” at roughly the midpoint of their reasoning trace). LPSR makes this implicit dynamic explicit and actionable: rather than waiting for a wrong final answer, it detects and corrects the pivotal moment when the reasoning trajectory diverges. An implication is that the “illusion of thinking” (Shojaee et al., 2025)—where models appear to reason but are actually performing shallow pattern matching—may be detectable via phase shifts. If a model’s residual stream does not exhibit coherent directional flow at ℓcrit , its apparent chain-of-thought may not correspond to genuine computation. This hypothesis connects LPSR’s detection mechanism to broader questions about the nature of LLM reasoning, and we leave its formal investigation to future work.
22