Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
Tyler Alvarez 1 Ali Baheri 1
1. Introduction
arXiv:2605.13772v1 [cs.CL] 13 May 2026
Abstract
Large language models (LLMs) now solve mathematical problems, multi-hop questions, and code generation tasks by producing long chains of reasoning steps [25]. The same models also hallucinate within these chains, generating fluent steps that are nonetheless incorrect [15, 14], and a single early error typically propagates to a confidently wrong final answer. Detecting where reasoning first goes wrong, in a single forward pass, is therefore a prerequisite for trustworthy deployment of reasoning models.
Large language models hallucinate during multistep reasoning, but most existing detectors operate at the trace level: they assign one confidence score to a full output, fail to localize the first error, and often require multiple sampled completions. We frame hallucination instead as a property of the hidden-state trajectory produced during a single forward pass. Correct reasoning moves through a stable manifold of locally coherent transitions; a first error appears as a localized excursion in transport cost away from this manifold. We operationalize this view with a label-conditioned teacher that builds a trace-specific contrastive PCA lens and scores each step with seven geometric transition features, and a deployable BiLSTM student distilled from the teacher that operates on raw hidden states without inference-time labels. We prove that contrastive PCA is the optimal projection for a transport-separation objective between first-error and correct states, and that single-pass first-error localization holds whenever the first error creates a positive transport margin over preceding correct transitions. On ProcessBench, PRM800K, HaluEval, and TruthfulQA, both models outperform entropy-based, probingbased, and attention-based baselines in-domain; the teacher transfers stably across language models and datasets, while the student collapses under shift, a gap our distillation theory predicts. These results recast step-level hallucination detection as a problem of trajectory dynamics and identify the central obstacle to deployment: preserving the contrastive transport margin under distribution shift.
Most existing hallucination detectors operate at the trace level. Token-level entropy and P(True) probing [16] and Bayesian uncertainty methods [12] are not calibrated for multi-step reasoning structure. Semantic entropy [17] and self-consistency [24] sample many completions per prompt and aggregate, requiring multiple forward passes and producing one score per output. SelfCheckGPT [19] and INSIDE [9] likewise reduce a full reasoning trace to a single confidence value. None of these methods localize the first error within a reasoning chain. Process reward models [18, 23, 10] do score individual steps, but require expensive human annotation at training time and a separately trained verifier at inference. Hidden-state probes [8, 3] classify the truthfulness of static factual statements, not the dynamics of an unfolding reasoning trace. We propose to treat the sequence of hidden representations produced during a single forward pass as a trajectory in representation space, and to detect a hallucination as a localized excursion in transport cost away from the manifold of locally coherent transitions. We instantiate this view with two components. A label-conditioned teacher uses step-level correctness labels to construct a trace-specific contrastive PCA lens and assigns each step a transport-instability score. The teacher is not deployable, since it requires labels at inference; rather, it is a diagnostic upper bound that quantifies the hidden-space signal. A student, distilled from the teacher, learns to reproduce this score directly from raw hidden states, with no sampling, labels, or external verifier required at inference time.
1
Rochester Institute of Technology, Rochester, NY, USA. Correspondence to: Tyler Alvarez <[email protected]>, Ali Baheri <[email protected]>.
Contributions.
Preprint. May 14, 2026.
1
Where Does Reasoning Break? Hidden-State Transport Geometry
1. A geometric framing of step-level hallucination detection. We formulate hallucination detection as a problem of trajectory dynamics in hidden-state space, characterizing a first reasoning error as a localized transport excursion away from the manifold of locally coherent transitions. We instantiate this view with a label-conditioned teacher that builds a trace-specific contrastive PCA lens and a deployable BiLSTM student distilled from it that requires no labels, sampling, or external verifier at inference.
verifier. Probing, Interpretability, and Representation Engineering. Truth probes [8, 3] train linear classifiers to predict whether a statement is true, activation patching [20] localizes circuits responsible for factual recall, and representation engineering [27] extracts conceptual directions for reading and steering high-level behaviors such as honesty. These methods analyze static representations of static claims; we instead model the trajectory of hidden states across reasoning steps.
2. Theoretical guarantees. We prove that contrastive PCA is the optimal projection under a transportseparation objective between first-error and correct states (Theorem 3.1), establish a single-pass first-error localization bound under a transport margin assumption (Theorem 3.2), and reduce teacher–student decision agreement to a margin-preservation condition (Proposition 3.3).
Methodological Foundations. Contrastive PCA [1] identifies low-dimensional directions enriched in a target distribution relative to a background; we use it in a tracespecific frame to expose first-error displacements. Optimal transport, in particular the squared 2-Wasserstein distance [22, 21], gives a clean transition score against the cloud of correct transitions, and the Davis-Kahan sin Θ theorem [11] supplies the perturbation bounds for finite-sample stability. Recent adjacent work also supports geometry- and structure-aware views of reliability and representation learning [2, 6, 7, 4, 5]. Our deployable detector follows the standard knowledge distillation framework [13], although our analysis identifies margin rather than mean error as the right distillation target for cross-domain transfer.
3. Empirical findings. On ProcessBench, PRM800K, HaluEval, and TruthfulQA, both models beat entropybased, probing-based, and attention-based baselines in-domain. The teacher transfers stably across models and datasets while the student does not, a gap predicted by our distillation theory and identifying margin preservation under shift as the central deployment obstacle.
Positioning. We depart from prior work along two axes. First, we score transitions between reasoning steps rather than static states, using velocity, acceleration, and directional persistence in a contrastive lens, so that the detection signal is geometric rather than semantic. Second, we separate the question of whether a hidden-space signal exists (the label-conditioned teacher) from whether it can be recovered without labels at inference (the distilled student). This separation lets us prove guarantees for the geometric signal itself (Theorems 3.1–3.2) and identify margin preservation under shift (Proposition 3.3) as the precise bottleneck for deployment.
2. Related Work Hallucination in LLMs. Hallucination, the generation of content that conflicts with source material or factual knowledge, is a well-surveyed phenomenon in language generation [15] and large language models [14]. For multistep reasoning models, factual and reasoning-driven failures compound across steps, so the practically relevant target is to identify the step at which the trace first deviates from a coherent reasoning path. Uncertainty and Hallucination Detection. Token-level entropy and P(True) probing [16] and Bayesian uncertainty methods [12] are not calibrated for multi-step reasoning. Semantic entropy [17] and self-consistency [24] sample many completions and aggregate, requiring multiple forward passes and producing only trace-level scores. SelfCheckGPT [19] and INSIDE [9] similarly reduce a full trace to one confidence value and do not localize where it first goes wrong.
3. Methodology GeoReason detects reasoning failures by treating a generated solution as a trajectory in the hidden space of a language model. The central premise is geometric: correct reasoning may move through many semantic regions, but its step-to-step motion remains close to a stable manifold of locally coherent transitions; a first hallucination or reasoning error appears as a localized transport excursion away from this manifold. Our method has two components (Figure 1). First, a label-conditioned teacher constructs a contrastive geometric lens and converts hidden-state trajectories into transition-instability scores. This teacher is not deployable because it uses step labels to estimate its reference geometry. Second, a student learns to reproduce the teacher’s
Process Supervision. Process reward models [18, 23, 10] train supervised classifiers on human step-level correctness labels, and ProcessBench [26] measures this ability directly. They rely on costly annotation, on a separately trained verifier at inference, and on dataset-specific labeling conventions, which limit transfer across models and tasks. Our deployable detector requires neither sampling nor a separate 2
Where Does Reasoning Break? Hidden-State Transport Geometry
instability signal from raw hidden states alone, yielding a single-pass post-hoc detector for generated traces.
distribution of first-error states, optionally including posterror states with smaller weight ρ ∈ [0, 1]. Denote their b0 ) and (b b1 ). empirical means and covariances by (b µ0 , C µ1 , C GeoReason forms the contrastive transport matrix
3.1. Problem setup and step representations For each prompt, let the model generate a trace T = (s1 , . . . , sm ), where each st is a reasoning step, sentence, or phrase-level unit. At a fixed transformer layer ℓ, we map each step to one vector ht ∈ Rd by mean pooling the token hidden states belonging to that step, ht =
1 X (ℓ) aj , |It |
cα = (b b1 − αC b0 , M µ1 − µ b0 )(b µ1 − µ b0 )⊤ + C
where α controls how aggressively high-variance correct directions are suppressed. The projection U ∈ Rd×k is the cα , and each step is matrix of the top k eigenvectors of M represented in the geometric lens as
(1)
zt = U ⊤ e ht ∈ Rk .
j∈It
τ = ∞ if the trace has no labeled error.
3.3. Transition features and the teacher detector
(2)
The cPCA projection exposes where the trajectory leaves the correct manifold; transition features describe how it leaves. Let
For first-error localization, all steps after τ are treated as incorrect, since they are conditioned on a corrupted reasoning state even if their surface form later becomes plausible. The objective is to learn scores pt ∈ [0, 1] from a single generated trace such that pt is high exactly at and after the first error; the predicted first error is τb = min{t : pt ≥ θ}, with no-error declared if no step crosses the threshold.
∆2 zt = zt − 2zt−1 + zt−2 , (6)
∆zt = zt − zt−1 ,
with zero padding for missing previous steps. GeoReason uses the following feature block: xt = zt , rt , r̄t , vt , at , et , dt , rt − medianu∈C ru rt = ∥zt ∥2 , r̄t = , MADu∈C (ru ) + ε
3.2. Label-conditioned contrastive geometry
at = ∥∆2 zt ∥2 ,
vt = ∥∆zt ∥2 ,
Raw hidden states contain many nuisance directions: prompt topic, syntax, answer length, and model-specific representation choices. The teacher therefore first converts each trace into a local coordinate system centered at its correct prefix. Let C = {t : yt = 0} be the correct steps in a labeled trace. We compute 1 X ht , |C| t∈C 1 X ∥ht − h̄0 ∥22 , σ02 = d|C|
(5)
When ρ = 0, the lens isolates the first-error displacement; when ρ = 1, it uses all incorrect steps and recovers the simpler implementation in which all post-error steps are assigned the incorrect class. In practice, intermediate ρ values prevent re-stabilized post-error states from diluting the first-error direction.
(ℓ)
where It is the token index set for step st and aj is the hidden state of token j at layer ℓ. Other deterministic pooling rules, such as last-token pooling, can be used without changing the method. During training and evaluation, a labeled trace has step labels yt ∈ {0, 1}, where yt = 0 denotes a correct step and yt = 1 denotes an incorrect step. We define the first-error index as τ = min{t : yt = 1},
α ≥ 0, (4)
t 1 X et = vj2 + a2j , w j=t−w+1
dt =
(7)
⟨∆zt , ∆zt−1 ⟩ . (∥∆zt ∥2 + ε)(∥∆zt−1 ∥2 + ε)
Here rt and r̄t measure position in the contrastive space, vt and at measure local motion, et smooths transient noise, and dt distinguishes coherent continuation from an abrupt change of direction. A lightweight MLP teacher fθ maps xt to a probability
h̄0 =
(3)
t∈C
ht − h̄0 e ht = . σ0 + ε
pTt = σ(fθ (xt )), X LT (θ) = − yit log pTit
This normalization makes the remaining signal relative to what the same model and prompt considered locally coherent before the error.
(8)
i,t
+ (1 − yit ) log(1 − pTit ) .
We then learn a contrastive PCA (cPCA) lens. Let P0 be the distribution of trace-normalized correct states and P1 the
The teacher is best understood as a diagnostic upper bound on the hidden-space signal: it uses labels to construct the 3
Where Does Reasoning Break? Hidden-State Transport Geometry
Figure 1. The GeoReason teacher–student architecture. The teacher (top) uses step-level labels and reasoning-trace hidden states to construct a contrastive PCA (cPCA) projection, extracts a geometric feature set in this lens, and maps the features through an MLP to step-level hallucination probabilities. The student (bottom) is a BiLSTM that contextualizes raw hidden states and feeds a step classifier head, trained from three signals: supervised step labels, probability distillation from the teacher, and feature distillation through a training-only auxiliary head. At inference, the student requires only hidden states.
correct-step reference frame and the contrastive geometry, and is therefore not a valid standalone hallucination detector at deployment time.
sequence model that takes only raw hidden states (Figure 1, bottom). The student is a BiLSTM followed by an MLP: pSt = σ(gψ (ct )).
c1:m = BiLSTMψ (h1:m ),
Algorithm 1 GeoReason teacher: label-conditioned geometric instability
(9)
The BiLSTM makes the detector post-hoc rather than online: it uses the whole generated trace, but it requires only one forward pass through the language model and no sampling, self-consistency, or external verifier. We train the student with a mixture of supervised step labels and soft teacher targets,
Require: Labeled traces {(hi1:mi , yi1:mi )}ni=1 , cPCA dimension k, background weight α, window w. for each trace i do Ci ← {t : yit = 0} and compute h̄i0 , σi0 from Eq. (3). Normalize each step: e hit ← (hit − h̄i0 )/(σi0 + ε). end for b0 ) from all normalized correct steps and Estimate (b µ0 , C b1 ) from first-error steps, optionally adding post(b µ1 , C error steps with weight ρ. cα using Eq. (4); set U ← TopEigk (M cα ). Form M for each trace i and step t do Project zit ← U ⊤ e hit and compute xit using Eq. (7). end for Train the MLP teacher pTit = σ(fθ (xit )) with Eq. (8).
LS (ψ) = λ
X
BCE(yit , pSit )
i,t
+ (1 − λ)τd2
X
T KL Bern(qit )∥
i,t
(10)
S Bern(qit ) , T T qit = σ(logit(pit )/τd ),
S qit = σ(logit(pSit )/τd ),
return teacher probabilities pTit and geometric features xit .
where τd is the distillation temperature and λ ∈ [0, 1]. At test time, the student returns the first threshold crossing τbS = min{t : pSt ≥ θ},
3.4. Deployable student by margin-preserving distillation
τbS = ∞ if no crossing occurs.
The deployable model cannot use step labels to build C, P0 , or P1 at inference. We therefore distill the teacher into a
(11)
Unless otherwise tuned on a validation split, we use θ = 0.5. 4
Where Does Reasoning Break? Hidden-State Transport Geometry
vector and X1 ∼ P1 a trace-normalized first-error hidden vector, with means µ0 , µ1 and covariances C0 , C1 . For U ∈ Rd×k with U ⊤ U = Ik , define
Algorithm 2 GeoReason student: deployable first-error detector Require: Training traces {hi1:mi }ni=1 , optional labels yit , teacher probabilities pTit , threshold θ. Train BiLSTM + MLP student with Eq. (10).
Γ(U ) = EX1 W22 (δU ⊤ X1 , U# P0 ) − EX0 W22 (δU ⊤ X0 , U# P0 ),
procedure Infer(h1:m ) c1:m ← BiLSTM(h1:m ). pSt ← σ(g(ct )) for t = 1, . . . , m. if maxt pSt < θ then
where U# P0 is the pushforward of P0 under U ⊤ . Then Γ(U ) = Tr U ⊤ M U , (16) M = (µ1 − µ0 )(µ1 − µ0 )⊤ + C1 − C0 .
return no hallucination. else
Consequently, the maximizer of Γ(U ) over all kdimensional orthonormal projections is the top-k eigenspace of M , and the optimal value is the sum of the top k eigenvalues of M .
return hallucination with first-error estimate min{t : pSt ≥ θ}. end if end procedure
Proof sketch. For any fixed x, W22 (δU ⊤ x , U# P0 ) = EY ∼P0 ∥U ⊤ (x − Y )∥22 . Taking expectation over X1 gives a projected second moment with covariance C1 + C0 and mean shift (µ1 − µ0 )(µ1 − µ0 )⊤ ; taking expectation over X0 gives 2 Tr(U ⊤ C0 U ). Subtracting yields Eq. (16). The eigenspace claim follows from the Ky Fan variational principle. Full proofs and finite-sample perturbation bounds are deferred to Appendix A.
3.5. Main theoretical results We now justify the geometry used above. The statements are intentionally assumption-explicit: GeoReason is guaranteed when the first semantic error induces a detectable transport-margin event in hidden-state trajectory space. If a model makes an error while remaining hidden-state indistinguishable from correct trajectories, no geometry-only detector can be guaranteed to localize it.
Theorem 3.1 explains why cPCA is the appropriate lens for GeoReason. It selects directions that make first-error states far from the correct-state cloud while penalizing directions in which correct reasoning already has high variance. If trace normalization removes the mean shift, the objective reduces to the usual contrastive covariance C1 − C0 ; adding the parameter α in Eq. (4) gives the generalized backgroundpenalized form C1 − αC0 .
Transport score for a reasoning transition. For an orthonormal projection U , define the augmented transition vector ϕt (U ) = zt , ∆zt , ∆2 zt ∈ R3k . (12)
Theorem 3.2 (First-error localization under a transport margin). Let τ be the first error and let S(t) be the ideal transport score in Eq. (13). Suppose that, for all t < τ ,
Let R0U be the distribution of ϕt (U ) over correct transitions. For any positive semidefinite ground-cost matrix A ⪰ 0, define the point-to-cloud transport instability score
P{S(t) − µc ≥ u} ≤ exp{−c min(u2 /ν 2 , u/b)},
2 SU (t) = W2,A (δϕt (U ) , R0U )
:=
inf
π∈Π(δϕt (U ) ,R0U )
(15)
∀u > 0,
⊤
E(x,y)∼π (x − y) A(x − y).
for constants c, ν, b > 0. Suppose also that the first error has margin P{S(τ ) ≥ µc + γ} ≥ 1 − β and that the b satisfies empirical score S(t) b P max |S(t) − S(t)| ≤ γ/4 ≥ 1 − α. (18)
(13) Because one marginal is a point mass, the coupling is unique and SU (t) = EY ∼R0U (ϕt (U ) − Y )⊤ A(ϕt (U ) − Y ).
(17)
(14)
t≤τ
Thus SU (t) is the cost of transporting the observed transition to the empirical cloud of correct transitions. The features in Eq. (7) are low-order summaries of this quadratic transport cost: position, velocity, acceleration, local energy, and directional persistence.
Then the threshold θ = µc + γ/2 and first crossing rule b ≥ θ} obey τb = min{t : S(t) P{b τ = τ} ≥ 1 − α − β 2 γ γ , . − (τ − 1) exp −c min 16ν 2 4b (19)
Theorem 3.1 (cPCA maximizes a transport-separation objective). Let X0 ∼ P0 be a trace-normalized correct hidden 5
Where Does Reasoning Break? Hidden-State Transport Geometry
Proof sketch. On the estimation event, the first-error score remains above µc + 3γ/4 whenever the margin event holds, and hence crosses θ. A false alarm before τ can occur only if some correct step has S(t) ≥ µc + γ/4. Applying the sub-exponential tail bound and a union bound over t < τ gives Eq. (19). See Appendix A for full details.
all later steps are evaluated as post-error states, matching the objective in Eq. (2). Splits are prompt-level and stratified by dataset and error presence, so no reasoning trace appears in more than one split. Hidden-state extraction and model settings. The crossmodel experiments use one representative instruction-tuned model from each family: Qwen, Llama, and Mistral. For every generated trace, we run a single forward pass, extract the residual-stream hidden states from the final transformer block before the language-model head, and mean-pool tokens within each step as in Eq. (1). Unless otherwise stated, the cPCA lens uses rank k = 16, contrastive penalty α = 1, post-error weight ρ = 0.25, and smoothing window w = 3. Thresholds for first-error localization are tuned only on the validation split and then frozen for test evaluation.
The result matches the empirical design goal: post-error states need not remain anomalous forever. Localization only requires the first wrong step to create a transport excursion before any preceding correct transition does. Proposition 3.3 (Distillation preserves first-error decisions when margins survive). Let sT (t) and sS (t) be teacher and student scores on the same trace with common threshold θ, and define the teacher decision margin mT = min |sT (t) − θ|. 1≤t≤m
(20)
If maxt |sS (t)−sT (t)| ≤ ε < mT , then teacher and student assign identical labels to every step and return the same first-error index. For random traces,
Training details and baselines. The teacher is a two-layer MLP over the feature block in Eq. (7). The student is a twolayer BiLSTM with a step-classification head and a trainingonly auxiliary head for feature distillation. Both models are trained with AdamW, early stopping on validation AUROC, and prompt-level mini-batches. Baselines are evaluated under the same splits and hidden-state extraction protocol: TL-Entropy and TL-Perplexity use token-level likelihood statistics, Linear Probe trains a linear classifier on pooled step representations, and LLM-Check uses attention-derived scores. The teacher should be read as an oracle diagnostic for the existence of a label-conditioned geometric signal; the student is the deployable model because it does not use inference-time labels, sampling, or an external verifier.
P{b τS ̸= τbT } ≤ P{mT ≤ ε}+P{max |sS (t)−sT (t)| > ε}. t (21) Proposition 3.3 identifies the main failure mode of the deployable student. Strong in-domain performance requires only small average distillation error, but robust first-error transfer requires preserving the teacher’s margin under shifts in model family, prompt distribution, and dataset style. This is why we report both the label-conditioned teacher and the deployable student: the teacher measures whether a contrastive transport signal exists in hidden space, while the student measures how much of that signal can be recovered without inference-time labels.
Figure 2 visualizes the central phenomenon behind the method: the first error can produce a localized geometric excursion even when later post-error states move back toward the region occupied by correct steps. This supports the use of transition scores and first-crossing rules rather than a single trace-level endpoint confidence.
4. Experiments We evaluate GeoReason on two tasks: step-level hallucination detection and first-error localization. Step-level detection is measured with AUROC, while first-error localization is measured by the accuracy of the first step whose score crosses a validation-selected threshold. The main in-domain results are reported in Table 1 and Table 2.
Across datasets, both the teacher and student models achieve strong performance relative to prior baselines. For step-level detection (Table 1), the teacher attains the highest AUROC on three of the four datasets, including ProcessBench (91.0), HaluEval (94.0), and TruthfulQA (96.0), while the student achieves the best performance on PRM800K (99.8). Baseline methods such as TL-Entropy, TL-Perplexity, Linear Probe, and LLM-Check generally perform worse across most datasets.
Benchmarks and preprocessing. We use ProcessBench, PRM800K, HaluEval, and TruthfulQA because they cover process-level mathematical reasoning, annotated solution steps, generated hallucinations, and factual truthfulness. Each example is converted to a single ordered trace. When a benchmark provides step boundaries, we preserve them; otherwise, we split generated text at newline and sentencelevel delimiters and discard empty fragments. Step labels are mapped to binary correctness. For localization, the first annotated incorrect step is treated as the first-error index and
For first-error detection (Table 2), the teacher achieves the highest accuracy on ProcessBench (68.7), PRM800K (88.4), and HaluEval (68.7), while the student achieves the best performance on TruthfulQA (96.8) and PRM800K (92.9). Baselines again trail behind both proposed models in most settings, although the linear probe performs competitively 6
Where Does Reasoning Break? Hidden-State Transport Geometry
Figure 2. Hidden states projected into cPCA space for (a) correct, (b) first-error, and (c) post-error steps. First-error steps lie largely outside the correct-step distribution (83.9% outside 1σ), while post-error steps partially overlap (41.1%). This supports our view of hallucinations as trajectory deviations from a stable reasoning manifold.
on certain datasets such as TruthfulQA. Because the tables report point estimates, we interpret small differences cautiously and focus on the repeated pattern across benchmarks and generalization settings.
across all train–test combinations (Table 3). Step-level AUROC remains stable across models, ranging from 90.1 to 91.7, regardless of the training model. First-error detection accuracy is also consistent, with values between 72.2 and 78.3 across all settings.
Overall, these results show that both the teacher and student models are effective for step-level hallucination detection and first-error localization across a range of benchmarks, consistently outperforming standard entropy-based, probingbased, and attention-based baselines.
The student model exhibits strong performance when evaluated on the same model it was trained on (Table 4). For example, training and testing on Qwen, Llama, and Mistral yields AUROC values of 93.6, 93.5, and 93.9, respectively, with corresponding first-error accuracies of 75.7, 74.4, and 75.3. However, performance varies substantially across different train–test combinations. In particular, when evaluated on models different from the training model, AUROC values range from 33.4 to 58.5, and first-error accuracy ranges from 27.0 to 35.4.
Table 1. Step-level hallucination detection performance (AUROC) across benchmarks. Best results per column are in bold. Methods Teacher (non-deployable) Student (deployable) TL-Entropy TL-Perplexity Linear Probe LLM-Check (attention)
ProcessBench
PRM800K
HaluEval
TruthfulQA
91.0 75.0 57.1 51.2 67.8 61.9
98.5 99.8 54.5 45.8 91.3 48.0
94.0 88.4 50.8 48.4 78.6 55.7
96.0 96.5 64.4 67.1 90.0 69.8
Overall, these results show that both models achieve strong in-domain performance, while cross-model evaluation reveals differences in performance consistency across training and testing configurations.
Table 2. First-error detection accuracy across benchmarks. Best results per column are in bold. Methods Teacher (non-deployable) Student (deployable) TL-Entropy TL-Perplexity Linear Probe LLM-Check (attention)
ProcessBench
PRM800K
HaluEval
TruthfulQA
68.7 34.4 46.3 43.2 24.4 43.8
88.4 92.9 43.8 39.0 68.8 49.2
68.7 78.6 42.3 46.4 68.7 49.8
93.2 96.8 68.5 78.2 89.3 50.0
Table 3. Cross-model generalization for the Teacher. Models are trained on a single LLM and evaluated on all LLMs. We report step-level AUROC (top) and first-error accuracy (bottom in parentheses).
We evaluate cross-model generalization, where models are trained on a single LLM and evaluated on all three LLMs (Qwen, Llama, and Mistral). Tables 3 and 4 report results for the teacher and student models, respectively. We report step-level AUROC and first-error detection accuracy (in parentheses).
Train ↓ / Test → Qwen Llama Mistral
The teacher demonstrates consistently strong performance 7
Qwen
Llama
Mistral
91.6 ( 77.5 ) 90.7 ( 72.2 ) 90.1 ( 73.4 )
91.7 ( 77.8 ) 91.7 ( 78.2 ) 90.6 ( 74.6 )
90.9 ( 75.1 ) 91.2 ( 75.9 ) 91.5 ( 78.3 )
Where Does Reasoning Break? Hidden-State Transport Geometry Table 4. Cross-model generalization for the Student. Models are trained on a single LLM and evaluated on all LLMs. We report step-level AUROC (top) and first-error accuracy (bottom in parentheses).
Train ↓ / Test → Qwen Llama Mistral
Qwen
Llama
Mistral
93.6 ( 75.7 ) 51.5 ( 31.9 ) 48.8 ( 32.1 )
39.1 ( 30.0 ) 93.5 ( 74.4 ) 58.5 ( 35.4 )
35.4 ( 27.0 ) 33.4 ( 28.4 ) 93.9 ( 75.3 )
6. Conclusion GeoReason frames step-level hallucination detection as hidden-state trajectory geometry: a label-conditioned teacher exposes the geometric signal of a first error via tracespecific cPCA, and a deployable student distills this signal for single-pass detection from raw hidden states. We prove that cPCA is the optimal lens under a transport-separation objective (Theorem 3.1), that localization holds whenever a transport margin exists (Theorem 3.2), and that teacherstudent agreement reduces to margin preservation (Proposition 3.3). The teacher transfers across models and datasets while the student does not, identifying margin preservation under shift, rather than detection of the geometric signal, as the central deployment obstacle.
We evaluate cross-dataset generalization under a leave-oneout setup, where each method is trained on all datasets except one and evaluated on the held-out dataset. Table 5 shows that the teacher maintains strong performance across all held-out datasets (AUROC 59.2 to 91.3, first-error 49.8 to 89.0), while the student is consistently weaker on PRM800K, HaluEval, and TruthfulQA, with the gap reaching ∆ = +39.9 AUROC and +40.1 first-error accuracy.
Table 5. Cross-dataset generalization under a leave-one-out setup. Each method is trained on all datasets except the held-out one. We report step-level AUROC and first-error detection accuracy. ∆ denotes the performance gap (Teacher − Student). Held-out Dataset ProcessBench PRM800K HaluEval TruthfulQA
Step-Level AUROC Teacher Student ∆ 59.2 69.0 86.7 91.3
62.5 51.3 58.3 51.4
-3.3 +17.7 +28.4 +39.9
First-Error Acc. Teacher Student ∆ 54.2 49.8 69.5 89.0
38.7 9.7 59.8 50.4
+15.5 +40.1 +9.7 +38.6
5. Analysis The teacher computes geometric features in a labelconditioned, trace-specific cPCA space and outperforms all baselines both in-domain and on out-of-domain models and datasets. Because it requires step labels at inference, we treat it not as a deployable detector but as a diagnostic upper bound that demonstrates the framing of hallucination as trajectory instability. The student, which removes this requirement, also outperforms the baselines in-domain but collapses to near-random AUROC under cross-model and cross-dataset shift. We attribute this gap to what each model learns. The teacher captures a mechanistic instability signal that is intrinsic to the geometry of correct vs. first-error transitions; the student, operating in the full latent space without label-conditioned normalization, learns a representational signal that absorbs model- and dataset-specific encoding quirks. Improving deployment therefore requires distillation that preserves the teacher’s transport margin rather than only its mean predictions. 8
Where Does Reasoning Break? Hidden-State Transport Geometry
References
[12] Gal, Y. and Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pp. 1050–1059, 2016.
[1] Abid, A., Zhang, M. J., Bagaria, V. K., and Zou, J. Exploring patterns enriched in a dataset with contrastive principal component analysis. Nature Communications, 9(1):2134, 2018. doi: 10.1038/ s41467-018-04608-8. Article number 2134.
[13] Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network, 2015. NIPS Deep Learning and Representation Learning Workshop, 2015.
[2] Amiri Shahbazi, M. and Baheri, A. Geometry-aware uncertainty quantification via conformal prediction on manifolds. arXiv preprint arXiv:2602.16015, 2026. doi: 10.48550/arXiv.2602.16015.
[14] Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., and Liu, T. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2024. Final version: ACM Trans. Inf. Syst. 43(2), Article 42, January 2025.
[3] Azaria, A. and Mitchell, T. The internal state of an LLM knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 967–976, Singapore, 2023.
[15] Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., and Fung, P. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023. doi: 10.1145/ 3571730.
[4] Baheri, A. Logic-guided vector fields for constrained generative modeling. arXiv preprint arXiv:2602.02009, 2026. doi: 10.48550/arXiv.2602. 02009. [5] Baheri, A. and Alm, C. O. LLMs-augmented contextual bandit. In NeurIPS 2023 Workshop on Foundation Models for Decision Making, 2023. FMDM@NeurIPS 2023.
[16] Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022.
[6] Baheri, A. and Amiri Shahbazi, M. Conformal prediction across scales: Finite-sample coverage with hierarchical efficiency. Results in Applied Mathematics, 26: 100589, 2025. doi: 10.1016/j.rinam.2025.100589.
[17] Kuhn, L., Gal, Y., and Farquhar, S. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In International Conference on Learning Representations, 2023.
[7] Baheri, A. and Wei, P. Multi-fidelity temporal reasoning: A stratified logic for cross-scale system specifications. Logics, 3(2):5, 2025. doi: 10.3390/ logics3020005.
[18] Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023.
[8] Burns, C., Ye, H., Klein, D., and Steinhardt, J. Discovering latent knowledge in language models without supervision. In International Conference on Learning Representations, 2023.
[19] Manakul, P., Liusie, A., and Gales, M. J. F. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9004–9017, 2023.
[9] Chen, C., Liu, K., Chen, Z., Gu, Y., Wu, Y., Tao, M., Fu, Z., and Ye, J. INSIDE: LLMs’ internal states retain the power of hallucination detection. In International Conference on Learning Representations, 2024.
[20] Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems, volume 35, pp. 17359–17372, 2022.
[10] Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
[21] Peyré, G. and Cuturi, M. Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning, 11(5–6):355–607, 2019.
[11] Davis, C. and Kahan, W. M. The rotation of eigenvectors by a perturbation. III. SIAM Journal on Numerical Analysis, 7(1):1–46, 1970.
[22] Villani, C. Optimal Transport: Old and New, volume 338 of Grundlehren der mathematischen Wissenschaften. Springer, 2009. 9
Where Does Reasoning Break? Hidden-State Transport Geometry
[23] Wang, P., Li, L., Shao, Z., Xu, R., Dai, D., Li, Y., Chen, D., Wu, Y., and Sui, Z. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9426–9439, Bangkok, Thailand, 2024. [24] Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Selfconsistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, 2023. [25] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022. [26] Zheng, C., Zhang, Z., Zhang, B., Lin, R., Lu, K., Yu, B., Liu, D., Zhou, J., and Lin, J. ProcessBench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559, 2024. [27] Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., Goel, S., Li, N., Byun, M. J., Wang, Z., Mallen, A., Basart, S., Koyejo, S., Song, D., Fredrikson, M., Kolter, J. Z., and Hendrycks, D. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023.
10
Where Does Reasoning Break? Hidden-State Transport Geometry
A. Proofs and Additional Theoretical Analysis This appendix gives the formal details for the theoretical section. The statements are intentionally assumption-explicit: GeoReason can be guaranteed only when the first reasoning error induces a measurable transport-margin event in hiddenstate trajectory space. If a model makes a semantic error while remaining indistinguishable from correct trajectories in the chosen hidden representation, no unsupervised geometry-only detector can be guaranteed to find it. A.1. Notation and transport identities For a distribution P on Rd and a matrix U ∈ Rd×k , U# P denotes the pushforward distribution of U ⊤ X for X ∼ P . For a positive semidefinite matrix A, define the squared optimal-transport cost with ground cost cA (x, y) = (x − y)⊤ A(x − y) by 2 W2,A (P, Q) =
inf π∈Π(P,Q)
E(X,Y )∼π (X − Y )⊤ A(X − Y ).
(22)
When A = I, we write W22 . Lemma A.1 (Point-to-cloud transport). Let Q be any distribution on Rm with finite second moment and mean µQ , and let A ⪰ 0. Then, for any x ∈ Rm , 2 W2,A (δx , Q) = EY ∼Q (x − Y )⊤ A(x − Y ) = (x − µQ )⊤ A(x − µQ ) + Tr(ACQ ),
(23)
where CQ is the covariance of Q. Proof. The only coupling between the point mass δx and Q is the law of (x, Y ) with Y ∼ Q. This gives the first equality. For the second equality, write Y = µQ + ξ with Eξ = 0 and Eξξ ⊤ = CQ : E(x − Y )⊤ A(x − Y ) = (x − µQ )⊤ A(x − µQ ) + Eξ ⊤ Aξ, where the cross term vanishes and Eξ ⊤ Aξ = Tr(ACQ ). The detector score in the main text is the special case in which x is the augmented transition vector ϕt (U ) = zt , ∆zt , ∆2 zt , zt = U ⊤ h̃t , with ∆zt = zt − zt−1 and ∆2 zt = zt − 2zt−1 + zt−2 . Lemma A.1 shows that this score is a quadratic deviation from the correct-transition cloud. If A is block diagonal, the score is a weighted sum of position, velocity, and acceleration deviations. If A has off-diagonal blocks, the score also includes directional-persistence terms such as ⟨∆zt , ∆zt−1 ⟩ after expanding the quadratic form. Thus the hand-designed features used by GeoReason are a low-order coordinate system for a learned transport cost. A.2. Proof of the contrastive transport theorem Theorem A.2 (Contrastive transport projection). Let X0 ∼ P0 and X1 ∼ P1 be hidden vectors in Rd with means µ0 , µ1 and covariances C0 , C1 . For U ∈ Rd×k with U ⊤ U = Ik , define Γ(U ) = EX1 W22 (δU ⊤ X1 , U# P0 ) − EX0 W22 (δU ⊤ X0 , U# P0 ).
(24)
Then Γ(U ) = Tr U ⊤ M U ,
M = (µ1 − µ0 )(µ1 − µ0 )⊤ + C1 − C0 . (25) Pk The maximizers of Γ(U ) are the top-k eigenspaces of M , and the maximum value is i=1 λi (M ), where λ1 (M ) ≥ · · · ≥ λd (M ). Proof. Let Y0 ∼ P0 be independent of X0 , X1 . By Lemma A.1, EX1 W22 (δU ⊤ X1 , U# P0 ) = E∥U ⊤ (X1 − Y0 )∥22 = E Tr U ⊤ (X1 − Y0 )(X1 − Y0 )⊤ U 11
Where Does Reasoning Break? Hidden-State Transport Geometry
= Tr U ⊤ {C1 + C0 + (µ1 − µ0 )(µ1 − µ0 )⊤ }U . Similarly, for an independent copy Y0′ of X0 , EX0 W22 (δU ⊤ X0 , U# P0 ) = E∥U ⊤ (X0 − Y0′ )∥22 = 2 Tr(U ⊤ C0 U ). Subtracting gives Γ(U ) = Tr(U ⊤ M U ). Maximization over orthonormal U is exactly the Ky Fan variational problem, whose solutions are the top-k eigenspaces of M . Connection to cPCA. If trace-normalization removes the first-order mean shift, then µ1 ≈ µ0 and M ≈ C1 − C0 , the standard contrastive covariance matrix. A background penalty αC0 corresponds to optimizing Γα (U ) = Tr{U ⊤ [(µ1 − µ0 )(µ1 − µ0 )⊤ + C1 − αC0 ]U }, which favors directions where first-error variance or displacement is large relative to correct-step variance. This is the theoretical objective behind the label-conditioned teacher projection. A.3. Finite-sample stability of the contrastive projection The teacher uses empirical means and covariances. The following proposition records the stability needed for the main theorem to remain meaningful with finite traces. c be the empirical version of M and suppose ∥M c − M ∥op ≤ εM . Proposition A.3 (Finite-sample gap preservation). Let M b c Let U⋆ be a top-k eigenspace of M and U a top-k eigenspace of M . Then b ) ≥ Γ(U⋆ ) − 2kεM . Γ(U
(26)
If additionally λk (M ) − λk+1 (M ) = ∆ > 2εM , then the subspace error obeys b , U⋆ )∥op ≤ 2εM . ∥ sin Θ(U ∆
(27)
cU ) over orthonormal U , b maximizes Tr(U ⊤ M Proof. Since U b ⊤M cU b ) ≥ Tr(U⋆⊤ M cU⋆ ). Tr(U c − M )U )| ≤ kεM for any orthonormal U gives Using | Tr(U ⊤ (M b ⊤M U b ) ≥ Tr(U b ⊤M cU b ) − kεM ≥ Tr(U⋆⊤ M cU⋆ ) − kεM ≥ Tr(U⋆⊤ M U⋆ ) − 2kεM . Tr(U c. This is Eq. (26). Eq. (27) follows from the Davis-Kahan sin-theta theorem applied to M and M A typical concentration rate. If the hidden vectors in both classes are sub-Gaussian with parameter κ and sample sizes n0 , n1 , then standard covariance concentration yields, with probability at least 1 − δ, s 2
εM ≲ κ
d + log(1/δ) + κ∥µ1 − µ0 ∥2 n0 ∧ n 1
s
d + log(1/δ) , n0 ∧ n 1
(28)
up to universal constants and lower-order terms. The first term is covariance estimation; the second arises from the rank-one mean-shift component. In practice the effective dimension is the layer-wise intrinsic rank after trace normalization, which is often much smaller than the raw hidden width. 12
Where Does Reasoning Break? Hidden-State Transport Geometry
A.4. Proof of first-error localization Theorem A.4 (Localization under a transport margin). Let τ be the first error. Suppose that, for t < τ , P{S(t) − µc ≥ u} ≤ exp{−c min(u2 /ν 2 , u/b)}
for all u > 0,
(29)
and that P{S(τ ) ≥ µc + γ} ≥ 1 − β.
(30)
b P max |S(t) − S(t)| ≤ γ/4 ≥ 1 − α.
(31)
Assume the empirical score satisfies
t≤τ
b ≥ θ}. Then Set θ = µc + γ/2 and τb = min{t : S(t) 2 γ γ , . P{b τ = τ } ≥ 1 − α − β − (τ − 1) exp −c min 16ν 2 4b
(32)
Proof. Let Eest be the event in Eq. (31), and let Eerr be the event in Eq. (30). On Eest ∩ Eerr , b ) ≥ S(τ ) − γ/4 ≥ µc + 3γ/4 > θ, S(τ so the first error is detected. A false alarm before τ can occur on Eest only if, for some t < τ , b − γ/4 ≥ θ − γ/4 = µc + γ/4. S(t) ≥ S(t) By the union bound and Eq. (29), 2 γ γ P{∃t < τ : S(t) ≥ µc + γ/4} ≤ (τ − 1) exp −c min , . 16ν 2 4b Combining with the failure probabilities of Eest and Eerr gives the result. Interpretation. The theorem does not require all post-error steps to remain anomalous. This matters for the empirical phenomenon in which the trajectory may jump at the first error and then return toward the correct region. The proof only needs the first error to cross the transport threshold before any earlier correct step does. A.5. Distillation transfer Proposition A.5 (Teacher-student decision preservation). Let ST (t) and SS (t) be teacher and student scores on a trace, with a common threshold θ. Let mT = min |ST (t) − θ|. 1≤t≤T
If maxt |SS (t) − ST (t)| ≤ ε < mT , then the teacher and student assign identical binary labels to all steps and therefore return the same first-error index. For random traces, P{b τS ̸= τbT } ≤ P{mT ≤ ε} + P{max |SS (t) − ST (t)| > ε}. t
(33)
Proof. If |SS (t) − ST (t)| < |ST (t) − θ| for every t, then SS (t) − θ and ST (t) − θ have the same sign for every t. Hence every step label is identical, and the first threshold crossing is identical. The probabilistic statement is the complement of this deterministic event. This proposition clarifies why a student can match the teacher in-domain yet fail under dataset or model shift. The teacher is built from the contrastive transport matrix, so it depends on a low-dimensional instability direction. The student observes full hidden states. If nuisance directions vary across datasets or LLM families, the approximation error maxt |SS (t) − ST (t)| can increase. Alternatively, if the teacher scores concentrate near threshold on the shifted domain, mT shrinks. Either mechanism breaks decision preservation even when average regression loss appears acceptable. 13
Where Does Reasoning Break? Hidden-State Transport Geometry
A.6. Relation to the seven GeoReason features The theoretical score in Eq. (23) is quadratic in the augmented transition variables. Expanding with a block matrix A00 A01 A02 A = A10 A11 A12 A20 A21 A22 gives terms of the form zt⊤ A00 zt , zt⊤ A01 ∆zt ,
∆zt⊤ A11 ∆zt ,
∆2 zt ⊤ A22 ∆2 zt ,
∆zt⊤ A12 ∆2 zt ,
zt⊤ A02 ∆2 zt ,
plus linear and constant terms from the correct-transition mean. The scalar features used by GeoReason can be interpreted as computationally cheap summaries of these quantities: • projected state zt and magnitude ∥zt ∥ estimate position in the contrastive transport lens; • normalized magnitude estimates point-to-cloud distance after trace-wise scale correction; • velocity ∥∆zt ∥ and acceleration ∥∆2 zt ∥ estimate local transition cost; • rolling energy estimates a local average of S(t), reducing false positives from isolated noise; • directional persistence estimates cross terms between successive increments, distinguishing coherent progress from erratic jumps. The MLP is therefore not required to invent a new geometric statistic from scratch; it learns a nonlinear calibration of a transport-motivated sufficient feature family. A.7. Scope of the guarantee The results above rely on three substantive assumptions. First, the hidden layer must encode reasoning correctness through a contrastive transport direction; if P0 and P1 are identical after projection, no geometry-based detector can separate them. Second, the first error must have a margin γ larger than normal correct-step fluctuations; very subtle errors may be detectable only with additional semantic supervision. Third, deployable performance requires the student to preserve the teacher margin under distribution shift. These assumptions match the empirical structure of GeoReason: the teacher is a diagnostic upper bound for the hidden-space signal, while the student measures how much of that signal can be recovered without inference-time labels. A.8. Computational cost and implementation details P For N = i mi total steps, hidden dimension d, and cPCA rank k, naive covariance construction costs O(N d2 ). Our implementation uses a matrix-free randomized eigensolver for Eq. (4), requiring O(N dk) time and O(dk) working memory after streaming the normalized hidden states. Feature extraction costs O(N k) and teacher training is negligible relative to hidden-state extraction. Student inference costs one LLM forward pass to obtain hidden states plus O(mH 2 ) for a BiLSTM with hidden width H on a trace of length m. No step requires sampling multiple completions or querying an external verifier, which distinguishes GeoReason from self-consistency and process-supervision pipelines.
14