arXiv:2606.23590v1 [cs.AI] 22 Jun 2026
The Topology of Ill-Posed Questions: Persistent Homology for Detection and Steering in LLMs
Guangyu Jiang The George Washington University [email protected]
Sizhe Tang The George Washington University [email protected]
Mahdi Imani Northeastern University [email protected]
Tian Lan The George Washington University [email protected]
Abstract Ill-posed questions, including ambiguous, underspecified, or contradictory queries, may admit no valid answer or multiple plausible answers, posing a challenge for large language models (LLMs). Existing approaches largely analyze ill-posedness through model outputs and often focus on specific subclasses. We investigate whether diverse sources of ill-posedness can be represented within a unified topology of LLM internal states and whether this structure can be used to steer response behavior. We model the contextual hidden states of prompt tokens at each transformer layer as a point cloud and characterize its geometry using finite zerodimensional persistent homology. Each layer is summarized by three compact descriptors: mean finite lifetime, normalized lifetime entropy, and largest-lifetime concentration. Concatenating these descriptors across layers yields a topology representation of the question. We further introduce topology-conditioned activation steering, which retrieves topologically similar examples and constructs query-specific activation interventions that encourage source-aware clarification or abstention. Across three open-weight LLMs, topology features consistently outperform prompt-based and pooled-hidden-state baselines for ill-posedness classification, improving average accuracy from 67.4% to 78.9% on AmbigQA, from 79.9% to 88.5% on SituatedQA, and from 57.6% to 69.6% on CLAMBER 9-way classification. Topology-conditioned steering increases the average total acceptable response rate from 61.4% to 70.6% and grounded acceptable responses from 11.9% to 16.4%. These results show that persistent homology provides both an interpretable representation of ill-posedness and an effective mechanism for targeted response steering.
1
Introduction
Large language models (LLMs) are increasingly deployed as general-purpose question answering (QA) systems. In many cases, however, a user question may be ill-posed, admitting no valid answer or multiple plausible answers. Following Hadamard, a problem is well-posed when a solution exists, is unique, and depends continuously on the data [1]. Many datasets and benchmarks show that ill-posed questions in QA can originate from several distinct sources: ambiguities (lexical or semantic) with multiple plausible interpretations [2–4], underspecified entity or event references [2], missing temporal or geographic context [5], unclear user intent [4], or over-specified problems with conflicting statements/conditions [6, 7]. Even on well-specified questions where LLMs perform reliably, the same models tend to falter once a question becomes ill-posed, where the difficulty lies as much in the question’s formulation as in the model’s world knowledge. Preprint.
Recent work has studied several subclasses of ill-posed problems. Notable advances include disambiguated input rewrites [2], synthesizing answers across alternative interpretations [3], selective abstention or rejection with respect to ambiguity [8], and active information acquisition and elicitation [4, 9–11]. These works establish that a strong response to ill-posed problems requires LLMs to adapt their behaviors to the underlying sources of ill-posedness. Crucially, such adaptation should not collapse to a fixed refusal template: a useful response should preserve the question-specific entities, constraints, and context while targeting the missing, ambiguous, or conflicting information. However, most prior approaches treat LLM reasoning as a black box and focus on particular subclasses of ill-posedness through input-output analysis. This raises a broader question: Can various sources of ill-posedness be represented in a unified layer-wise topological space via a topological representation of the model’s internal states, and can this structure be used to steer reasoning toward behaviors aligned with their underlying sources? This paper demonstrates that the model’s prefill stage provides a natural locus for such unified analysis and steering. Before generation begins, the model has already transformed the entire question into a sequence of layer-wise contextual token states. Recent work shows that intermediate activations can serve as both readout and intervention sites: truthfulness-related directions can be identified and used to shift behavior during inference [12]; high-level representations can be monitored and controlled through representation engineering and contrastive activation steering [13, 14]; and more specific activation features have been associated with refusal [15], unanswerability and abstention [16], and question ambiguity [17]. We study this internal state from a topological perspective. Instead of representing a question by a single pooled hidden vector with selected dimensions, we treat the contextual hidden states of its prompt tokens at each transformer layer as a point cloud. This token cloud captures the relational structure among the words or subwords in the question after they have been contextualized by the model. We analyze the shape of this cloud using persistent homology, a method from topological data analysis (TDA) that tracks topological features across a range of distance scales [18]. Persistence diagrams have been used as text representations [19] and can be vectorized into machine-learning features such as persistence images [20]. We use compact finite H0 descriptors rather than highdimensional vectorizations to obtain three compact, fixed-dimensional descriptors. Our analysis focuses on finite zero-dimensional persistent homology H0 , which records how initially disconnected token components merge as the distance threshold increases. These merge patterns provide a compact description of the internal connectivity structure of a question representation during the reasoning process. This view casts ill-posedness as a layer-wise transformation of the question’s internal token cloud, quantified by finite zero-dimensional persistent homology H0 . A well-posed question induces token states that organize around a coherent interpretation. An ill-posed question instead preserves separated or unevenly connected token groups, each corresponding to an unresolved entity, context, or constraint of the specific ill-posedness subtype. We summarize each layer’s finite H0 diagram with three compact statistics: the mean finite lifetime, the normalized lifetime entropy, and the concentration ratio of the five largest lifetimes. Concatenating these descriptors across all layers yields a low-dimensional topology representation for each question. The topological representation enables accurate detection of ill-posedness and supports a topologyconditioned steering mechanism. Rather than applying a single global abstention direction that may produce generic refusals, our method retrieves topologically similar positive and negative examples for each query and forms a local activation-space contrast. This query-specific direction edits the model’s internal states during prefill and decoding, moving the response toward source-aware abstention or clarification while preserving information from the original question. Persistent homology determines which samples enter the contrast for each case; the intervention itself remains in the model’s native activation space. Our contributions are threefold. 1) We introduce a layer-wise token-cloud representation for ill-posed question analysis, treating prompt-token hidden states at each transformer layer as a point cloud and quantifying its multiscale connectivity through finite H0 persistence. 2) We show that three compact, fixed-dimensional descriptors of finite H0 lifetimes (mean lifetime, normalized lifetime entropy, and largest-lifetime concentration) concatenated across layers form a question-level topology vector that supports both binary and 9-way fine-grained ill-posedness classification across three open-weight model families. 3) We develop topology-conditioned local activation steering, where topology-space
2
neighbors select query-specific contrast pairs whose activation difference shifts response behavior toward ill-posedness-aware outputs while preserving the semantic content of the original question. Evaluation on AmbigQA and SituatedQA for binary ill-posedness detection, and on CLAMBER for fine-grained 9-way ill-posedness classification demonstrates the effectiveness of the proposed topological representation. The CLAMBER setting includes one well-posed N ONE class and eight ill-posedness subtypes. These subtypes cover unfamiliar-entity or missing-knowledge queries; contradictory queries, including conflicts induced by in-context examples; lexical polysemy, where a word or phrase has multiple meanings; semantic or co-reference indeterminacy, where the intended referent or interpretation is unclear; and four missing-constraint categories in which the intended response depends on unspecified personal, temporal, spatial, or task-specific information. Across Gemma-7Bit, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.3, our H0 topology features improve average accuracy over the strongest baselines from 67.4% to 78.9% on AmbigQA, from 79.9% to 88.5% on SituatedQA, and from 57.6% to 69.6% on CLAMBER 9-way classification. Averaged across the nine dataset–model settings, topology-conditioned steering improves grounded acceptance from 11.9% to 16.4% and total acceptance from 61.4% to 70.6%. These results show that topology-conditioned steering does not merely induce a fixed refusal pattern, but can better preserve question-specific information while moving the model toward source-aware abstention or clarification.
2
Related Work
Ill-posedness, abstention, and clarification in question answering Prior work on ill-posed QA has studied several response behaviors. AmbigQA asks systems to recover multiple plausible questionanswer pairs and produce disambiguated rewrites [2], while ASQA requires long-form answers that synthesize information across different interpretations [3]. SituatedQA studies questions whose answers depend on temporal or geographic context [5]. Other work treats ill-posedness as a reason to abstain or clarify: Cole et al. [8] distinguishes uncertainty about the world from uncertainty about the question, while CLAMBER, Qulac, and CAmbigNQ evaluate clarification-question generation for underspecified information needs [4, 9, 10]. Alignment with Perceived Ambiguity further uses a model’s own perceived ill-posedness signal to improve handling of unclear queries [21]. These works primarily evaluate final response behavior, such as rewriting, answer synthesis, abstention, or clarification. Our work instead studies how ill-posedness is represented within the model before generation, using the layer-wise topology of question-token clouds for both detection- and abstentionoriented steering. Internal representations of answerability, ill-posedness, and abstention A growing body of work suggests that LLMs encode reliability-relevant properties in intermediate activations. InferenceTime Intervention identifies truthfulness-related directions in attention-head activations [12], while representation engineering frames LLM control as monitoring and manipulating high-level internal representations [13]. Activation steering methods operationalize this idea by adding contrastive directions computed from positive and negative examples, as in Contrastive Activation Addition [14]. Related work shows that refusal can be mediated by low-dimensional activation directions [15], and that unanswerability directions can be used for both detection and causal control of abstention [16]. Most relevant to our work, Zhang et al. [17] show that question ambiguity, a central form of ill-posedness, is detectable during prefill and that a sparse set of neurons can shift responses from direct answering to abstention. Our framework complements these direction- and neuron-based approaches by representing each question as a layer-wise token cloud, covering fine-grained illposedness subtypes, and conditioning steering on topological similarity rather than a single global direction. Topological data analysis for language representations Topological data analysis provides tools for summarizing the shape of high-dimensional point clouds. Persistent homology tracks connected components, cycles, and higher-dimensional holes across distance scales, producing diagrams or barcodes that can be vectorized for machine learning [20]. Early NLP work used persistent homology as a text representation over embedded document units [19]. More recent work applies topology to language representation spaces, including cross-lingual word-embedding geometry [22], local topology of contextual language-model states [23], semantic-search ill-posedness over sentence-embedding neighborhoods [24], and zigzag-persistence analysis of layer-wise LLM dynamics [25]. These works mainly use topology as a diagnostic representation tool. Beyond 3
Topology based classification and steering for ill-posed questions H0 topology features summarize token-cloud geometry, support subtype classification, and guide construction of activation-space steering directions.
1. Ill-posed question
2. Token states
3. Topology features
4. Classification
5. Topology steering
before
token cloud -> H0 lifetimes
What is the release date of T2?
after
Binary: none vs ill-posed
Layer 31 none
6. Steered response Before steering
topology classifier
Layer 32
POLYSEMY
The release date of the movie 'Terminator 2: Judgment Day' (T2) is July 3, 1991.
ill-posed
...
WHEN
mean persist.
Give me a bulleted list of the last seven World Series winners
Multi-class: none + subtypes
Layer 2
none
polysemy
After topology steering
co-reference
entropy
Layer 1 CO-REFERENCE
top-5 fraction
Matthew told Joshua that he offended the client. Who offended the client?
Representative fine-grained ill-posed question types.
is
the
release
of
T2
?
when
where
whom
in-context learning
not known
activation-space steering vector
Feature vector across layers
question tokens What
what
date
During prefilling, each token has a hidden state at every layer.
Topology features support binary and multi-class prediction.
Question-token clouds are summarized by H0 persistence statistics.
Topology features select similar contrast cases; their activation difference defines the steering direction.
I need a bit more information about which T2 you are referring to. There are several movies and products with the name T2. Could you please provide more context or specify which T2 you are looking for?
direct answer -> abstention
Figure 1: Overview of our topology-based framework. We collect prompt-token hidden states across layers, treat each layer as a token point cloud, compute finite H0 persistence features, and use the resulting topology vectors for ill-posedness classification and topology-conditioned activation steering. NLP, reinforcement learning work has used topology to decompose temporal-difference signals in non-Markovian environments [26] and geometric coherence to structure value functions in Markov decision processes [27]. In contrast, we compute finite H0 persistence over the token cloud of a single question at each prefill layer, relate the resulting descriptors to binary and fine-grained ill-posedness detection, and use topology to condition local activation steering from direct answering toward abstention.
3
Method
We propose a topology-based framework for analyzing and steering ill-posed questions in LLMs. As shown in Figure 1, we collect prompt-token hidden states during prefill, treat each layer as a token point cloud, summarize its finite H0 persistent homology, and use the resulting topology vectors for classification and topology-conditioned steering. Topology selects behaviorally matched contrast examples, while the intervention itself remains in the model’s native activation space. 3.1
Layer-wise question token clouds
Let q be a question prompt and let M be a decoder-only language model with L transformer layers. During the prefill stage, the model processes all prompt tokens before generating the first (l) output token. At layer l, let ht (q) ∈ Rd denote the hidden state of the t-th prompt token, and let (l) (l) H (l) (q) = [h1 , . . . , hT ] ∈ RT ×d be the matrix obtained by stacking these T token states as rows. We view the rows of H (l) (q) as a point cloud of contextual token representations. This representation preserves the token-level relational structure that mean pooling and last-token pooling discard. A pooled vector compresses the prompt into a single point, while the cloud retains the geometric arrangement of token states, which we measure with persistent homology. Computing persistent homology directly in the full hidden dimension is unnecessary and can amplify noise. We therefore fit a layer-specific PCA map on training-set token states and project each token cloud as (l) (l) X(l) (q) = [x1 , . . . , xT ]⊤ = H (l) (q) − 1a⊤ Pl ∈ RT ×r . l
(1)
where Pl ∈ Rd×r denotes the matrix of the top r principal directions for layer l, and al ∈ Rd is the (l) (l) corresponding training-set mean. For simplicity, we use X (l) (q) = {x1 , . . . , xT } to denote the resulting point cloud in the reduced representation space. All PCA maps are fit only on the training split and then fixed for validation and test examples. 3.2
Persistent homology of question representations
We compute persistent homology separately for each question q and transformer layer l. The input to (l) (l) the topological analysis at layer l is the reduced token point cloud X (l) (q) = {x1 (q), . . . , xT (q)} ⊂ 4
Rr , defined from the layer-l prompt hidden states in Eq. (1). Thus, the persistent-homology diagram is constructed from the geometry of the token states after the model has processed the question up to layer l, rather than from a pooled prompt representation or from the final generated response. Given the layer-wise token cloud X (l) (q), we construct a Vietoris–Rips filtration [28]. For each scale (l) (l) parameter ϵ ≥ 0, the Vietoris–Rips complex VRϵ (X (l) (q)) contains a simplex [xi0 (q), . . . , xim (q)] whenever all pairwise distances among its vertices are at most ϵ: (l)
(l)
[xi0 (q), . . . , xim (q)] ∈ VRϵ (X (l) (q))
⇐⇒
(l)
(l)
∥xia (q) − xib (q)∥2 ≤ ϵ
for all a, b.
(2)
As ϵ increases, these complexes form a nested sequence, VRϵ1 (X (l) (q)) ⊆ VRϵ2 (X (l) (q)) whenever ϵ1 ≤ ϵ2 . Persistent homology tracks when topological features appear and disappear along this filtration. A feature born at scale bi and dying at scale di is represented by a point (bi , di ) in a persistence diagram, and its lifetime is ℓi = di − bi . Longer lifetimes correspond to features that persist across a wider range of distance scales. In this work, we focus on zero-dimensional persistent homology, H0 . For the layer-wise token cloud X (l) (q), the H0 diagram describes how connected components of the token cloud merge as the filtration radius grows. At ϵ = 0, each token point initially forms its own connected component. As ϵ increases, nearby components merge until the entire token cloud becomes connected. After all components have merged, one connected component remains; its corresponding H0 bar has infinite lifetime, so we exclude it from our finite-lifetime statistics. The remaining finite H0 bars record the scales at which connected components of the token cloud merge. Thus, finite H0 lifetimes provide a compact description of the multiscale fragmentation structure of the question representation at layer l. This choice is motivated by both interpretability and computational efficiency. Question-level ill-posedness often arises when the prompt leaves multiple entities, events, contexts, or constraints unresolved. Such unresolved alternatives induce separated or unevenly connected regions in the contextual token cloud. Finite H0 lifetimes capture this structure directly: large lifetimes correspond to merge events between token groups that remain separated until a relatively large scale. Moreover, H0 features are interpretable since they are closely related to single-linkage clustering and the edge lengths of a minimum spanning tree over the token cloud. MST equivalence. For a finite metric point cloud under the Vietoris–Rips filtration with edge threshold ϵ, the multiset of finite H0 death times equals the multiset of edge weights selected by Kruskal’s algorithm [29] for a minimum spanning tree, up to ties and filtration-scale convention. Each finite H0 death occurs when an edge first connects two previously disconnected components; Kruskal’s algorithm selects exactly these component-merging edges in increasing distance order. This equivalence yields an efficient implementation through pairwise distances and an interpretable connectivity skeleton of the contextual token cloud. For each question q and layer l, we denote the finite H0 persistence diagram of X (l) (q), excluding the infinite bar, by (l)
D0 (q) = {(bi , di )}ni=1 ,
L(l) (q) = {ℓ1 , . . . , ℓn },
ℓi = di − bi .
(3)
Here, L(l) (q) is the layer-specific finite H0 lifetime multiset used to compute the topological descriptors in the next subsection. For T token points, n = T − 1. 3.3
Compact topological descriptors
For each question q and layer l, we compress the finite H0 lifetime multiset L(l) (q) = {ℓ1 , . . . , ℓn } into three scalar descriptors. These descriptors summarize the token cloud with a small number of interpretable features rather than a high-dimensional persistence vector. First, we compute the mean finite lifetime: n
µ(l) (q) =
1X ℓi . n i=1
(4)
This measures the average scale at which connected components merge. A larger value indicates that token groups remain separated until larger filtration radii, suggesting a more fragmented token cloud. 5
Second, we compute normalized lifetime entropy (Eq. (5)): E (l) (q) = −
n 1 X pi log pi , log n i=1
ℓi pi = Pn
j=1 ℓj
.
(5)
This feature measures whether lifetime mass is spread across many comparable merge events or concentrated in only a few. When the finite lifetimes are nearly uniform, E (l) (q) is high, while it is low when a small number of merge events dominate the topology. Third, we compute the largest-lifetime concentration ratio (Eq. (6)). Pmin(5,n) (l) ℓ[j] (q) j=1 (l) . R (q) = Pn (l) ℓ (q) j=1 [j] (l)
(l)
(6)
(l)
Here ℓ[1] (q) ≥ ℓ[2] (q) ≥ · · · ≥ ℓ[n] (q) denotes the finite H0 lifetimes sorted in decreasing order, while the numerator sums the five largest finite lifetimes, or all finite lifetimes if n < 5. This ratio measures the extent to which the token-cloud topology is dominated by the largest component-merge events. High R(l) (q) indicates that a small number of large separations explain most of the H0 structure, whereas low R(l) (q) indicates a more diffuse multiscale merge pattern. Together, these three features summarize complementary aspects of the question token cloud: µ(l) (q) captures the average fragmentation scale, E (l) (q) captures the spread of lifetime mass, and R(l) (q) captures the dominance of the largest merge events. They are fixed-dimensional regardless of prompt length, making them suitable for both binary ill-posedness detection and fine-grained ill-posedness classification. 3.4
Layer-wise topology vectors for ill-posedness classification
Ill-posedness is rarely encoded at a single transformer layer, so we compute topology descriptors at every layer. For each l ∈ {1, . . . , L}, define ϕ(l) (q) = [µ(l) (q), E (l) (q), R(l) (q)] ∈ R3 . We concatenate all layer descriptors into z(q) = [ϕ(1) (q), . . . , ϕ(L) (q)] ∈ R3L , and standardize z(q) using coordinate-wise training-set means and standard deviations to obtain ze(q). We train a supervised classifier gθ : R3L → ∆C−1 , where C = 2 for binary well-posed versus ill-posed detection and C equals the number of subtypes for fine-grained classification. 3.5
Topology-conditioned local steering
We use topology not only for diagnosis, but also to construct query-specific steering directions. Standard activation steering constructs a single global contrast vector between positive and negative examples [14]. This can be too coarse for ill-posed questions, whose missing information may concern different entities, times, locations, task constraints, or interpretations. We instead retrieve behaviorally matched examples in topology space and construct a local activation-space direction. Let D+ be training questions whose base responses are ill-posedness-aware, and let D− be questions whose base responses are direct answers. For a test question q, we compute its standardized topology vector ze(q) and retrieve k nearest neighbors D+ (q) ⊂ D+ and D− (q) ⊂ D− using Euclidean disPTi (l⋆ ) tance in standardized topology space. For each retrieved example qi , let h̄(l⋆ ) (qi ) = T1i t=1 ht (qi ) be its mean-pooled prompt representation at intervention layer l⋆ . We define the local steering vector as X X 1 1 (l⋆ ) vlocal (q) = h̄ (q ) − h̄(l⋆ ) (qj ). (7) i |D+ (q)| |D− (q)| + − qi ∈D (q)
qj ∈D (q)
We then add this vector to the hidden states at layer l⋆ . During prefill, (l )
(l )
ht ⋆ (q) ← ht ⋆ (q) + αvlocal (q),
t = 1, . . . , T,
(8)
and during decoding we apply the same update to the current generated-token hidden state at layer l⋆ . The scalar α controls steering strength. Topology is used to select local contrast examples, while the intervention itself remains a standard activation-space addition; this avoids inverting non-smooth persistent-homology features back into token activations. 6
4
Experiment
Our experiments evaluate three questions. First, do ill-posed questions induce distinctive topological trajectories across transformer layers during prefill? Second, can compact finite H0 descriptors detect binary ill-posedness and fine-grained ill-posedness subtypes? Third, can topology-conditioned neighborhoods improve activation steering toward ill-posedness-aware responses, including abstention and clarification? 4.1
Datasets and Models
We evaluate three ill-posedness-oriented QA benchmarks. For binary ill-posedness detection, we use AmbigQA and SituatedQA. AmbigQA studies open-domain questions that may admit multiple plausible interpretations and provides disambiguated rewrites for resolving them [2]. SituatedQA focuses on questions whose answers depend on extra-linguistic temporal or geographic context [5]. For fine-grained ill-posedness classification, we use CLAMBER as a 9-way task: one well-posed N ONE class plus eight ill-posedness subtypes. These subtypes cover epistemic misalignment, including unfamiliar or missing-knowledge queries and contradictions induced by in-context examples; linguistic indeterminacy, including lexical polysemy and semantic or co-reference uncertainty; and missing output constraints, including unspecified personal, temporal, spatial, or task-specific information [4]. We report the main results using three open-weight LLMs: Gemma-7B-it [30], Llama-3.1-8B-Instruct [31], and Mistral-7B-Instruct-v0.3 [32] to acquire the internal representation of the questions. 4.2
Evolution of ill-posedness topology across layers
We first analyze how the topology of question token clouds changes across transformer layers during prefill. For each model and each CLAMBER ill-posedness subtype, we compute the three finite H0 descriptors introduced in Section 3: mean persistence, persistence entropy, and largest-lifetime concentration. We then average each descriptor over examples of the same subtype at every layer. Figure 2 visualizes these layer-wise trajectories for Gemma, Llama, and Mistral. The results show that ill-posedness-related topology is not static across the network. Instead, different ill-posedness subtypes follow distinct trajectories as the model transforms the question representation. The mean finite lifetime generally increases toward later layers, with especially sharp increases near the final layers in several models, suggesting that later representations can amplify large-scale separations among token components. The entropy and concentration curves provide a more subtypespecific view: some ill-posedness types maintain high entropy or low concentration of the largest lifetime, indicating a more diffuse merge structure, while others are dominated by a small number of large-component merges. These trends support persistent homology as an interpretability lens for ill-posedness and motivate its later use as a retrieval space for steering. 4.3
Ill-posedness classification
We next evaluate whether compact finite H0 descriptors can predict ill-posedness labels. For binary ill-posedness detection, we evaluate on AmbigQA and SituatedQA; for fine-grained ill-posedness classification, we evaluate on the 9-way CLAMBER taxonomy [2, 5, 4]. For each question, we concatenate the three finite H0 descriptors across all transformer layers and train a logistic regression classifier on the standardized topology vector ze(q). We compare against AEN (ambiguity-encoding neurons) [17], a sparse-neuron probing baseline that selects the top-k neurons most predictive of ambiguity and trains a classifier on their activations. We also compare against prompt-based baselines, including CLAMBER zero-shot [4], CLAM few-shot [33], CLAMBER few-shot chain-of-thought [4, 34], and InfoGain prompting [21]. All results are reported as accuracy and macro-F1 percentages. Table 1 shows that finite H0 topology features consistently outperform prompt-based baselines and AEN across both binary and fine-grained ill-posedness classification. On the binary tasks, the topology classifier reaches 77.8–79.5% macro-F1 on AmbigQA and 86.6–90.4% macro-F1 on SituatedQA across the three models. Averaged over Gemma, Llama, and Mistral, this improves over AEN from 67.4% to 78.9% accuracy on AmbigQA and from 79.9% to 88.5% on SituatedQA. The gains over prompt-based baselines are larger: the best prompt baseline averages 55.1% accuracy on AmbigQA and 58.2% on SituatedQA. On CLAMBER 9-way classification, the H0 topology classifier reaches 69.1–70.6% accuracy, improving the average over AEN from 57.6% to 69.6% 7
none
polysemy
co-reference
H0 mean persistence
0
5
10
15
20
0.85
25
Gemma Llama
60 40 20 0
5
10
15
20
25
30
ICL
NK
H0 top-5 fraction 0.6 0.4
0.975 0.950 0.925 0.900 0.875
0.2 0
5
10
15
20
25
0
5
10
15
Layer
20
25
30
0.00
20
25
0.2 0
5
10
15
20
25
30
0
5
10
0
5
10
15
20
25
30
15
20
25
30
1.0 0.8 0.6 0.4 0.2
0.25 0
15
0.4
0.50
50
10
0.6
0.75
100
5
0.8
1.00
150
Mistral
whom
0.90
50
0
where
0.95
100
0
when
0.8
150
0
what
H0 persistence entropy
0
5
10
15
Layer
20
25
30
Layer
Figure 2: Layer-wise topology of question token clouds on CLAMBER. Rows correspond to models and columns correspond to the three finite H0 descriptors: mean persistence, persistence entropy, and largest-lifetime concentration. Each curve averages examples from one ill-posedness subtype. The trajectories show that ill-posedness subtypes induce different topological patterns across layers, supporting the view that ill-posedness is transformed through the prefill computation rather than appearing only as an output-level behavior. Table 1: Ill-posedness classification across AmbigQA, SituatedQA, and CLAMBER. Each entry reports Accuracy/F1 in percent. AmbigQA and SituatedQA are binary ill-posedness detection tasks, while CLAMBER is a fine-grained 9-way classification task. Prompt-baseline metrics are computed only on valid parsed outputs; valid counts are omitted for compactness. InfoGain only works for the binary tasks and is therefore skipped for CLAMBER. Boldface marks the best method in each dataset–model row. Dataset / Model
AEN
H0 topology Zero-shot
Few-shot Few-shot CoT InfoGain
AmbigQA Gemma-7B-it 61.6 / 61.6 Llama-3.1-8B-Instruct 63.4 / 63.2 Mistral-7B-Instruct-v0.3 77.3 / 77.3
79.5 / 79.5 79.3 / 79.3 77.9 / 77.8
50.4 / 41.8 50.4 / 36.3 50.0 / 45.9 52.5 / 35.5 49.4 / 34.0 58.4 / 58.4
42.7 / 42.7 49.1 / 45.8 49.2 / 43.1
52.5 / 50.7 52.9 / 40.9 59.8 / 59.8
SituatedQA Gemma-7B-it 73.9 / 73.9 Llama-3.1-8B-Instruct 80.2 / 80.2 Mistral-7B-Instruct-v0.3 85.5 / 85.5
86.6 / 86.6 90.4 / 90.4 88.4 / 88.4
51.5 / 42.5 47.4 / 32.6 57.1 / 54.6 54.9 / 36.3 54.3 / 46.1 54.4 / 43.1
48.6 / 47.9 52.9 / 52.7 56.4 / 56.3
57.7 / 57.6 55.2 / 46.2 61.6 / 59.5
CLAMBER 9-way Gemma-7B-it 54.7 / 46.6 Llama-3.1-8B-Instruct 57.8 / 50.3 Mistral-7B-Instruct-v0.3 60.2 / 51.3
69.1 / 62.2 69.1 / 62.1 70.6 / 63.6
26.3 / 6.2 12.2 / 8.6 16.4 / 15.4 19.1 / 20.8 27.3 / 14.6 57.7 / 44.9
27.6 / 26.2 34.0 / 31.0 37.4 / 32.4
– – –
and outperforming the best prompt baseline average of 33.0%. These results suggest that compact finite H0 descriptors capture ill-posedness-relevant structure in the internal question representation, including subtype-specific structure required for fine-grained classification beyond the level of binary separation. 4.4
Topology-conditioned steering
We next evaluate whether topology-conditioned local steering can shift direct-answer behavior toward ill-posedness-aware responses. We focus on held-out questions for which the unsteered model produces a direct-answer-like response. We compare our topology-local method against a global, full-vector activation-steering baseline, following the standard contrastive activation-steering setup [14]. Both methods use the same activation intervention mechanism and differ only in how the examples used to construct the steering direction are selected: the global baseline averages over all behavior-labeled training examples, whereas topology-local steering selects the nearest neighbors in the standardized topology space. We apply interventions at layer l⋆ = 14. For topology-local steering, 8
Table 2: Steering results across AmbigQA, SituatedQA, and CLAMBER. Entries are percentages judged as G ROUNDED ACCEPTABLE, G ENERIC ACCEPTABLE, or T OTAL ACCEPTABLE; T OTAL is grounded plus generic. Boldface marks the higher G ROUNDED or T OTAL rate between methods.
Dataset
Model
Global full-vector
H0 topology-local
Grounded Generic Total Grounded Generic Total Gemma-7B-it Llama-3.1-8B-Instruct Mistral-7B-Instruct-v0.3
0.0 5.0 11.1
64.6 81.5 80.8
64.6 86.4 91.9
1.3 11.1 11.8
64.5 72.6 68.1
65.8 83.7 79.9
Gemma-7B-it SituatedQA Llama-3.1-8B-Instruct Mistral-7B-Instruct-v0.3
2.6 45.1 18.4
13.2 41.3 78.0
15.8 86.5 96.4
1.6 39.8 42.1
87.3 47.7 51.1
88.9 87.5 93.2
Gemma-7B-it CLAMBER Llama-3.1-8B-Instruct Mistral-7B-Instruct-v0.3
11.0 10.3 3.2
19.9 38.0 28.3
31.0 48.4 31.5
3.2 18.8 17.7
42.0 32.9 22.2
45.2 51.6 39.9
AmbigQA
we sweep k ∈ {3, 5, 10, 20, 40, 50, 100}, and for both steering methods we sweep α ∈ {1, 2, . . . , 15}. Table 2 reports the best held-out results across hyperparameter sweeping. We evaluate generations with a four-way LLM-judge rubric. G ROUNDED ACCEPTABLE responses recognize the ill-posedness and address its specific source, such as a missing, ambiguous, or conflicting element; G ENERIC ACCEPTABLE responses avoid unsupported direct answers but give only generic refusal or clarification. U NACCEPTABLE responses directly answer without addressing ill-posedness, and N EITHER denotes malformed, irrelevant, or non-classifiable outputs. We report grounded, generic, and total acceptable rates, where total is the sum of grounded and generic acceptable responses. Table 2 shows that topology-conditioned steering can produce more meaningful ill-posedness-aware behavior rather than merely inducing a fixed refusal. The key metric is G ROUNDED ACCEPTABLE, which requires the response to preserve information from the original question and address the specific missing, ambiguous, or conflicting element. The H0 topology-local method improves grounded acceptance in six of the nine dataset/model settings, including all three AmbigQA models, Llama and Mistral on CLAMBER, and Mistral on SituatedQA. In several cases, topology-local steering increases grounded responses while reducing generic abstentions, such as AmbigQA with Llama (5.0% → 11.1% grounded; 81.5% → 72.6% generic) and SituatedQA with Mistral (18.4% → 42.1% grounded; 78.0% → 51.1% generic). This suggests that topology-local steering uses the question-specific internal topology to guide source-aware abstention or clarification, rather than simply maximizing broad refusal.
5
Conclusion
We introduced a topological framework for detecting and steering ill-posed questions in LLMs. Treating layer-wise prompt-token hidden states as point clouds, we summarized their finite H0 persistence with three compact statistics that capture how ill-posedness reshapes internal connectivity across layers. These descriptors deliver strong binary and fine-grained classification of ill-posedness across three open-weight models. The same topology vectors serve as a retrieval key for local activation steering, producing responses that engage with the specific source of ill-posedness while preserving question content. Our study has several limitations. Experiments are limited to three openweight instruction-tuned models and three QA benchmarks. Promising directions include extending to H1 and persistent path homology for richer topological descriptors, learning a layer-aggregation weighting in place of concatenation, and using topology to condition not only abstention but also which specific clarifying question to ask. 9
References [1] Jacques Hadamard. Sur les problèmes aux dérivées partielles et leur signification physique. In Princeton University Bulletin, page 49–52, 1902. [2] Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 5783–5797, 2020. [3] Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. Asqa: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273–8288, 2022. [4] Tong Zhang, Peixin Qin, Yang Deng, Chen Huang, Wenqiang Lei, Junhong Liu, Dingnan Jin, Hongru Liang, and Tat-Seng Chua. Clamber: A benchmark of identifying and clarifying ambiguous information needs in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10746–10766, 2024. [5] Michael Zhang and Eunsol Choi. Situatedqa: Incorporating extra-linguistic contexts into qa. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7371–7387, 2021. [6] Jingyuan Ma, Damai Dai, Zihang Yuan, Rui Li, Weilin Luo, Bin Wang, Qun Liu, Lei Sha, and Zhifang Sui. Large language models struggle with unreasonability in math problems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 32428–32436, 2026. [7] Boyang Xue, Qi Zhu, Rui Wang, Sheng Wang, Hongru Wang, Minda Hu, Fei Mi, Yasheng Wang, Lifeng Shang, Qun Liu, et al. Reliablemath: Benchmark of reliable mathematical reasoning on large language models. arXiv preprint arXiv:2507.03133, 2025. [8] Jeremy Cole, Michael Zhang, Dan Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 530–543, 2023. [9] Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W Bruce Croft. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of the 42nd international acm sigir conference on research and development in information retrieval, pages 475–484, 2019. [10] Dongryeol Lee, Segwang Kim, Minwoo Lee, Hwanhee Lee, Joonsuk Park, Sang-Woo Lee, and Kyomin Jung. Asking clarification questions to handle ambiguity in open-domain qa. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11526–11544, 2023. [11] Zeyu Fang, Tian Lan, and Mahdi Imani. Mint: Minimal information neuro-symbolic tree for objective-driven knowledge-gap reasoning and active elicitation. arXiv preprint arXiv:2602.05048, 2026. [12] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inferencetime intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36:41451–41530, 2023. [13] Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023. [14] Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, 2024. 10
[15] Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. Advances in Neural Information Processing Systems, 37:136037–136083, 2024. [16] Maor Juliet Lavi, Tova Milo, and Mor Geva. Detecting (un) answerability in large language models with linear directions. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 682–699, 2026. [17] Zhuoxuan Zhang, Jinhao Duan, Edward Kim, and Kaidi Xu. Sparse neurons carry strong signals of question ambiguity in llms. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16092–16110, 2025. [18] Robert Ghrist. Barcodes: the persistent topology of data. Bulletin of the American Mathematical Society, 45(1):61–75, 2008. [19] Xiaojin Zhu. Persistent homology: An introduction and a new text representation for natural language processing. In Ijcai, number 2013, pages 1953–1959, 2013. [20] Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. Persistence images: A stable vector representation of persistent homology. Journal of Machine Learning Research, 18(8):1–35, 2017. [21] Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang-goo Lee, and Taeuk Kim. Aligning language models to explicitly handle ambiguity. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1989–2007, 2024. [22] Ondřej Draganov and Steven Skiena. The shape of word embeddings: Quantifying non-isometry with topological data analysis. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12080–12099, 2024. [23] Benjamin Matthias Ruppik, Michael Heck, Carel van Niekerk, Renato Vukovic, Hsien-Chin Lin, Shutong Feng, Marcus Zibrowius, and Milica Gasic. Local topology measures of contextual language model latent spaces with applications to dialogue term extraction. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 344–356, 2024. [24] Thomas Roland Barillot and Alex De Castro. Blowfish: Topological and statistical signatures for quantifying ambiguity in semantic search. arXiv preprint arXiv:2406.07990, 2024. [25] Yuri Gardinazzi, Karthik Viswanathan, Giada Panerai, Alessio Ansuini, Alberto Cazzaniga, and Matteo Biagetti. Persistent topological features in large language models. In International Conference on Machine Learning, pages 18811–18830. PMLR, 2025. [26] Zuyuan Zhang, Sizhe Tang, and Tian Lan. Cochain perspectives on temporal-difference signals for learning beyond markov dynamics. arXiv preprint arXiv:2602.06939, 2026. [27] Zuyuan Zhang, Zeyu Fang, and Tian Lan. Structuring value representations via geometric coherence in markov decision processes. arXiv preprint arXiv:2602.02978, 2026. [28] Leopold Vietoris. Über den höheren zusammenhang kompakter räume und eine klasse von zusammenhangstreuen abbildungen. Mathematische Annalen, 97(1):454–472, 1927. [29] Jon Kleinberg and Eva Tardos. Algorithm design. Pearson Education India, 2006. [30] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024. [31] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 11
[32] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://arxiv.org/abs/2310.06825. [33] Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Clam: Selective clarification for ambiguous questions with generative language models. arXiv preprint arXiv:2212.07769, 2022. [34] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. [35] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267, 2025.
A
Dataset Details and Classification Baselines
This appendix provides additional details on the datasets, splits, classification baselines, and parsing protocol used in Section 4.3. All supervised classifiers are trained only on the training split and evaluated on held-out test examples. Feature-based classifiers standardize input features using training-set statistics and train a balanced logistic regression classifier unless otherwise specified. A.1
Datasets and train-test splits
We evaluate classification on three datasets covering both binary ill-posedness detection and finegrained subclass prediction. AmbigQA. AmbigQA is used as a binary clear-versus-ill-posed detection dataset. Each example is labeled as either clear/well-posed or ambiguous/ill-posed. The processed data are organized into paired examples, where a clear and an ambiguous version are associated with the same underlying question. We use an 80/20 pair-level split. The final split contains 2,240 training examples and 560 held-out test examples. The test set is balanced, with 280 clear and 280 ambiguous examples. SituatedQA. SituatedQA is also used as a binary clear-versus-ill-posed detection dataset. Here, ill-posedness is often caused by missing temporal, geographical, or other situational context. As with AmbigQA, we use an 80/20 pair-level split and keep paired clear/ambiguous examples in the same split. The final split contains 2,240 training examples and 560 held-out test examples, balanced with 280 clear and 280 ambiguous examples. CLAMBER. CLAMBER is used for fine-grained 9-way ill-posedness classification. The label set is {none, polysemy, co-reference, what, when, where, whom, ICL, NK}. Here, none denotes well-posed questions. The other labels denote ill-posedness subtypes: polysemy indicates lexical ambiguity, co-reference indicates referential or semantic indeterminacy, what/when/where/whom indicate missing task-specific, temporal, spatial, or personal constraints, ICL indicates conflict induced by in-context examples, and NK indicates unfamiliar or missing-knowledge queries. We use the predefined CLAMBER train/test split. The training set contains 2,562 examples and the test set contains 640 examples. The test set contains 160 none examples, 80 examples each for ICL, NK, co-reference, and polysemy, and 40 examples each for what, when, where, and whom. A.2
Classification baselines
AEN k = 5 sparse-neuron baseline. AEN follows the ambiguity-encoding-neuron setting of Zhang et al. [17]. For each LLM and dataset, the method selects five hidden-state dimensions and trains a classifier using only those five scalar activations. This is a strong interpretability-oriented baseline because prediction is restricted to a small number of individual neurons rather than a full hidden-state vector. In our tables, k = 5 means that exactly five selected neurons are used. The classifier is trained and evaluated on the same train/test split as the topology method. 12
Table 3: Dataset splits and test-label distributions for classification experiments. Dataset
Task
Train / Test Test label distribution
AmbigQA Binary clear vs. ill-posed 2,240 / 560 280 clear; 280 ambiguous SituatedQA Binary clear vs. ill-posed 2,240 / 560 280 clear; 280 ambiguous CLAMBER 9-way subtype classification 2,562 / 640 160 none; 80 each for ICL, NK, co-reference, and polysemy; 40 each for what, when, where, and whom
Zero-shot prompting. The zero-shot baseline directly asks the LLM to make the ill-posedness decision without labeled demonstrations. For AmbigQA and SituatedQA, the model is prompted to decide whether the question is clear or ill-posed, or equivalently whether it should answer directly or seek clarification. For CLAMBER, the model is prompted with the 9-way label definitions and asked to output one subclass label. This baseline measures whether the model can classify ill-posedness from its instruction-following behavior alone, without training a separate classifier. Few-shot prompting. The few-shot baseline adds a small number of labeled demonstrations to the prompt before the test question. For binary datasets, demonstrations show questions annotated as clear or ill-posed. For CLAMBER, demonstrations show representative questions paired with 9-way subclass labels. This baseline follows the general clarification-oriented prompting setup used in prior work such as CLAM [33], and tests whether in-context examples improve classification behavior relative to zero-shot prompting. Few-shot chain-of-thought prompting. The few-shot chain-of-thought baseline extends few-shot prompting by adding short rationales before the final label. The intended effect is to encourage the model to explicitly identify why a question is underspecified, ambiguous, contradictory, or clear before producing a prediction. For CLAMBER, demonstrations include brief rationales connecting the question to the target subclass. We score only the final parsed label, not the reasoning text itself. This baseline is based on the general chain-of-thought prompting paradigm [34] and on CLAMBER-style ambiguity classification prompts [4]. InfoGain prompting. The InfoGain baseline is used for binary ill-posedness detection. It first prompts the LLM to produce a disambiguated version of the input question. The model is then run on both the original and disambiguated forms, and the method compares generation entropy before and after disambiguation. If the entropy reduction exceeds a fixed threshold, the question is predicted as ill-posed. The intuition is that an ill-posed question should become easier for the model to answer after relevant missing context is supplied. This follows the information-gain signal used in Alignment with Perceived Ambiguity [21]. We do not use InfoGain for CLAMBER 9-way classification because it produces a binary ambiguity score rather than a fine-grained subclass label. A.3
Prompt-output parsing and valid predictions
For all prompt-based baselines, predictions are obtained by parsing the generated text into one of the allowed labels. If the generated text cannot be mapped to a valid label, the prediction is marked invalid. Accuracy and macro-F1 for prompt-based baselines are computed only over valid parsed outputs. This is why prompt baselines report valid-output counts in the full result tables, while trained classifiers such as AEN and 3-H0 always produce a prediction for every test example. The main paper omits valid-count columns for compactness.
B
Steering Evaluation Details
B.1
Four-way LLM judge prompt
To evaluate whether a steered response appropriately handles an ill-posed question, we use a four-way LLM-based judge. The judge distinguishes between question-specific abstention and generic abstention. It assigns one of four labels: GROUNDED_ACCEPTABLE, GENERIC_ACCEPTABLE, UNACCEPTABLE, or NEITHER. 13
The system prompt is: You are a precise evaluator of ill-posedness-aware LLM responses. The user prompt is: I would like your feedback on the response of a large language model to a given user query. You must evaluate both: 1. whether the response recognizes that the user query is ill-posed, and 2. whether the response is grounded in the specific content of the query. An ill-posed query may be ambiguous, underspecified, missing a condition, missing temporal/spatial/personal/task-specific information, internally contradictory, or dependent on information not provided in the query. There are four classes of responses: GROUNDED_ACCEPTABLE: The response correctly recognizes that the query is ill-posed AND addresses the specific source of ill-posedness in the given query. The response must preserve question-specific information such as entities, constraints, time, location, referents, requested object, or missing condition. A response can be GROUNDED_ACCEPTABLE if it does one or more of the following: - asks a targeted clarification question; - explicitly identifies what information is missing; - explains the ambiguity, contradiction, or underspecification; - gives conditional answers for different plausible interpretations; - abstains while giving a question-specific reason. Example: Question: "Who won the final?" Response: "Which final do you mean? The answer depends on the sport, league, and year." Label: GROUNDED_ACCEPTABLE Example: Question: "What is the capital?" Response: "I cannot determine the answer because you did not specify the country or region." Label: GROUNDED_ACCEPTABLE GENERIC_ACCEPTABLE: The response avoids giving a direct answer, but it is generic, boilerplate, or
14
weakly grounded. It says the question is ambiguous, unclear, or lacks information, but does not identify the specific missing information or preserve the specific content of the query. If the same response could be copied unchanged to many unrelated ill-posed questions, label it GENERIC_ACCEPTABLE rather than GROUNDED_ACCEPTABLE. Example: Question: "Who won the final?" Response: "The question is unclear. Please provide more details." Label: GENERIC_ACCEPTABLE Example: Question: "What is the capital?" Response: "I do not have enough information to answer. Please clarify." Label: GENERIC_ACCEPTABLE UNACCEPTABLE: The response gives a direct final answer without addressing the ill-posedness of the query. It does not ask for clarification, does not explain missing information, and does not acknowledge ambiguity or contradiction. Example: Question: "What is the capital?" Response: "The capital is Paris." Label: UNACCEPTABLE NEITHER: The response is malformed, irrelevant, nonsensical, empty, or impossible to classify under the above labels. Do not use NEITHER merely because the response is generic. Generic refusals should be GENERIC_ACCEPTABLE. Important decision rules: - Do NOT label a generic refusal as GROUNDED_ACCEPTABLE. - Do NOT reward a response only because it abstains. - Reward responses that retain information from the original query and identify the specific missing, ambiguous, or conflicting element. - If the response removes most of the query-specific content and gives only a broad disclaimer, label it GENERIC_ACCEPTABLE.
15
- If the response directly answers without challenging the ill-posedness, label it UNACCEPTABLE. - Prefer GROUNDED_ACCEPTABLE only when the response is both ill-posedness-aware and question-specific. Now evaluate the following question-response pair. QUESTION: "{QUESTION}" RESPONSE: "{RESPONSE}" Return your answer in the following exact format: <issue>Briefly identify the specific source of ill-posedness in the question.</issue> <information_retention_score>0, 1, 2, or 3</information_retention_score> <specificity_score>0, 1, 2, or 3</specificity_score> <label>GROUNDED_ACCEPTABLE | GENERIC_ACCEPTABLE | UNACCEPTABLE | NEITHER</label> <rationale>Briefly explain why this label was chosen.</rationale>
In our analysis, we report both GROUNDED_ACCEPTABLE and GENERIC_ACCEPTABLE. Their sum gives the overall acceptable rate, while GROUNDED_ACCEPTABLE isolates question-specific, contentpreserving abstention or clarification behavior. The judge LLM is set to be GPT-5 nano [35] during the experiment. B.2
Qualitative steering examples
We provide representative stored generations from the steering experiments. The examples are reproduced as saved, including cases that end mid-sentence. Labels in parentheses are assigned by the four-way judge described above. These cases are chosen to highlight the difference between generic abstention and grounded, question-specific abstention or clarification: both steering methods may avoid an unsupported direct answer, but topology-local steering more often preserves the ambiguous entity, title, sense, or missing context from the original query. Example 1: title ambiguity. Question. Who wrote the song be still and know i am? Unsteered. The song "Be Still and Know" is a popular Christian worship song, and it’s actually a cover of a song written by Reuben Morgan and Martin Smith, but the most well-known version is by Hillsong Worship, and also by the band Kari Jobe. Global full-vector (GENERIC_ACCEPTABLE). The song "Be Still" is a popular worship song, and there are multiple versions of it. H0 topology-local (GROUNDED_ACCEPTABLE). I don’t have any information on a song called "be still and know I am". Could you please provide more context or more information about the song, like the artist or any other details? The topology-steered response preserves the full queried title, “be still and know I am,” and asks for artist or contextual information. The global response gives a generic statement about multiple versions of a shortened title, while the topology-local response makes the missing disambiguating information explicit.
16
Example 2: missing media type. Question. What is the genre of Switched? Unsteered. The genre of the novel "Switched" by Amanda Hocking is Young Adult Fantasy, specifically a paranormal romance and a faerie romance. Global full-vector (GENERIC_ACCEPTABLE). I do not have information on the genre of Switched. H0 topology-local (GROUNDED_ACCEPTABLE). I need a bit more context about "Switched". Could you please provide more information about what you are referring to, such as a book, movie, or something else? The topology-steered response preserves the title Switched and identifies the missing media type as the source of ambiguity. The global response abstains but gives no useful clarification target, while topology-local steering asks whether the user means a book, movie, or another object.
Example 3: missing referent for a title. Question. What is the genre of Vettai? Unsteered. Vettai is a 2012 Indian Tamil-language action comedy film directed by N. Linguswamy. The genre of Vettai is Action Comedy. Global full-vector (GENERIC_ACCEPTABLE). I am unable to verify the genre of Vettai. H0 topology-local (GROUNDED_ACCEPTABLE). I don’t have enough information to determine the genre of "Vettai". Can you please provide more context or specify which "Vettai" you are referring to? The topology-steered response preserves the title Vettai and asks the user to specify which referent is intended. The global response is a generic unverifiability statement, whereas topology-local steering converts the missing referent into an actionable clarification.
C
Additional Classification Diagnostics
C.1
Binary confusion matrices
Figure 3 shows row-normalized confusion matrices for the 3-H0 topology classifier on the binary AmbigQA and SituatedQA tasks. Rows correspond to true labels and columns correspond to predicted labels. On AmbigQA, the topology classifier identifies ambiguous questions with recall above 80% for all three models, while the main residual errors come from clear questions predicted as ambiguous. On SituatedQA, both clear and ambiguous examples are classified more reliably, with diagonal entries around 86%–93%, consistent with the stronger binary results reported in Table 1. C.2
CLAMBER 9-way confusion matrices
Figure 4 shows row-normalized confusion matrices for the 3-H0 topology classifier on CLAMBER 9-way classification. Rows correspond to true ill-posedness subclasses and columns correspond to predicted subclasses. These matrices complement the aggregate accuracy and macro-F1 scores in Table 1 by showing which fine-grained subclasses are separated by topology and where residual confusions remain.
D
Ablation Studies
D.1
Ablation: individual H0 descriptors
We first ablate the three finite H0 descriptors used in the main topology vector. For each model and dataset, we train the same classifier using only one descriptor trajectory across all layers: mean 17
Row-normalized 3-H0 topology confusion matrices
Binary classification on the held-out 80/20 test split; rows are true labels and columns are predictions.
77.9%
clear
Llama-3.1-8B Acc 79.3, F1 79.3
22.1%
75.7%
clear
clear
75.7%
100
24.3% 80
ambiguous
18.9%
81.1%
clear
ambiguous
ambiguous
17.1%
82.9%
clear
ambiguous
Gemma-7B-it Acc 86.6, F1 86.6 SituatedQA True label
24.3%
Mistral-7B-v0.3 Acc 77.9, F1 77.8
87.1%
clear
ambiguous
20.0%
80.0%
clear
ambiguous
Llama-3.1-8B Acc 90.4, F1 90.4
12.9%
92.5%
clear
7.5%
60
Mistral-7B-v0.3 Acc 88.4, F1 88.4 clear
88.2%
40
% of true class
AmbigQA True label
Gemma-7B-it Acc 79.5, F1 79.5
11.8% 20
ambiguous
13.9%
86.1%
clear
ambiguous
ambiguous
11.8%
88.2%
clear
ambiguous
Predicted label
ambiguous
11.4%
88.6%
clear
ambiguous
Predicted label
0
Predicted label
Figure 3: Row-normalized confusion matrices for the 3-H0 topology classifier on binary ill-posedness classification. Rows correspond to gold labels and columns correspond to predictions. The figure reports results on the held-out 80/20 test splits for AmbigQA and SituatedQA across Gemma-7B-it, Llama-3.1-8B, and Mistral-7B-v0.3. CLAMBER 9-way compact 3-H0 confusion matrices, row-normalized
True
Gemma: acc=0.691, macro-F1=0.622
Llama: acc=0.691, macro-F1=0.621
Mistral: acc=0.706, macro-F1=0.636
1.0
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.95
0.00
0.00
0.04
0.01
0.00
0.00
0.00
0.00
0.96
0.00
0.00
0.03
0.00
0.01
0.00
0.00
0.00
0.97
0.00
0.00
0.03
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.03
0.00
0.49
0.02
0.15
0.08
0.06
0.17
0.00
0.02
0.00
0.53
0.03
0.11
0.04
0.07
0.21
0.00
0.03
0.00
0.52
0.02
0.08
0.06
0.07
0.22
0.00
0.01
0.00
0.05
0.85
0.03
0.03
0.01
0.03
0.00
0.05
0.00
0.04
0.80
0.03
0.01
0.05
0.03
0.00
0.04
0.00
0.03
0.78
0.04
0.04
0.04
0.05
0.00
0.05
0.00
0.15
0.07
0.23
0.15
0.10
0.25
0.00
0.00
0.00
0.10
0.07
0.10
0.28
0.17
0.28
0.00
0.05
0.00
0.12
0.07
0.15
0.28
0.10
0.23
0.00
0.03
0.00
0.12
0.03
0.17
0.28
0.20
0.17
0.00
0.03
0.00
0.12
0.03
0.23
0.45
0.10
0.05
0.00
0.03
0.00
0.07
0.10
0.12
0.30
0.20
0.17
0.00
0.03
0.00
0.15
0.12
0.10
0.17
0.33
0.10
0.00
0.03
0.00
0.10
0.07
0.20
0.07
0.38
0.15
0.00
0.03
0.00
0.10
0.07
0.07
0.12
0.45
0.15
0.00
0.00
0.00
0.10
0.05
0.07
0.07
0.05
0.65
0.00
0.00
0.00
0.20
0.03
0.07
0.12
0.07
0.50
0.00
0.00
0.00
0.07
0.00
0.05
0.03
0.03
0.82
0.8
0.6
0.4
0.2
Predicted
om wh
en
ere
wh
wh
y
at wh
lys em
po
ce
ne no
NK
fer en
-re
ICL
om wh
en
ere
wh
wh
y
at wh
lys em
ce
ne no
NK
fer en
po
co
Predicted
co
-re
ICL
om wh
en
ere
wh
wh
y
at wh
lys em
po
ce
ne no
NK
fer en
co
-re
ICL
0.0
Predicted
Figure 4: Row-normalized confusion matrices for the 3-H0 topology classifier on CLAMBER 9-way classification. The classifier uses the three finite H0 descriptors across all layers: mean persistence, persistence entropy, and largest-lifetime concentration. Rows correspond to true subclasses and columns correspond to predicted subclasses.
finite lifetime, normalized lifetime entropy, or largest-lifetime concentration. We compare these single-descriptor variants with the full 3-H0 representation, which concatenates all three descriptors across layers. Table 4 shows that the three finite H0 descriptors provide complementary information. Across all datasets and models, using all three descriptors improves over the best single-descriptor variant. The average macro-F1 gain over the best single descriptor is 2.1 points on AmbigQA, 5.6 points on SituatedQA, and 5.0 points on CLAMBER 9-way classification. This suggests that mean finite lifetime, normalized lifetime entropy, and largest-lifetime concentration capture distinct aspects of token-cloud topology: average component separation, spread of lifetime mass, and dominance by the largest merge events. 18
Table 4: Ablation of finite H0 descriptors for ill-posedness classification. Each entry reports Accuracy/F1 in percent. Single-descriptor variants use one descriptor across all transformer layers, while “All 3 H0 ” concatenates mean finite lifetime, normalized lifetime entropy, and largest-lifetime concentration across layers. H0 mean
H0 entropy H0 largest-5 frac.
All 3 H0
Dataset
Model
AmbigQA
Gemma-7B-it 76.5 / 76.5 68.5 / 68.5 Llama-3.1-8B-Instruct 76.8 / 76.7 64.5 / 64.5 Mistral-7B-Instruct-v0.3 78.5 / 78.4 68.3 / 68.3
77.9 / 77.9 78.8 / 78.7 78.5 / 78.4
79.7 / 79.7 81.1 / 81.1 80.7 / 80.6
SituatedQA
Gemma-7B-it 79.3 / 79.3 76.2 / 76.2 Llama-3.1-8B-Instruct 81.1 / 81.1 70.3 / 70.3 Mistral-7B-Instruct-v0.3 80.7 / 80.6 75.7 / 75.7
74.1 / 74.0 75.4 / 75.4 75.6 / 75.6
84.5 / 84.4 87.9 / 87.9 85.6 / 85.6
Gemma-7B-it 63.0 / 55.4 63.4 / 55.4 CLAMBER 9-way Llama-3.1-8B-Instruct 68.6 / 60.9 62.2 / 56.9 Mistral-7B-Instruct-v0.3 63.4 / 57.0 62.3 / 55.1
54.4 / 48.4 61.1 / 53.0 62.3 / 54.2
69.2 / 62.5 69.1 / 62.1 70.6 / 63.6
D.2
Ablation: single-layer topology versus all-layer topology
We next test whether ill-posedness can be detected from the topology of a single transformer layer, or whether the full layer-wise trajectory is needed. For each layer, we train a classifier using only that layer’s three H0 descriptors: mean finite lifetime, normalized lifetime entropy, and largest-lifetime concentration. We compare these single-layer classifiers with the all-layer 3-H0 classifier used in the main experiments. Single-layer 3-H0 topology classification
Gemma-7B-it
Llama-3.1-8B
Mistral-7B-v0.3
All-layer 3-H0
Solid curves use one layer's H0 mean persistence, H0 persistence entropy, and H0 top-5 persistence fraction; dashed lines use the same 3-H0 statistics across all layers.
AmbigQA
SituatedQA
90
82
Test accuracy (%)
CLAMBER 9-way
70
81
85
80
60
79
80
78
50 75
77 76
40
70
75 0
5
10
15
20
25
30
Layer
0
5
10
15
Layer
20
25
30
0
5
10
15
20
25
30
Layer
Figure 5: Single-layer versus all-layer topology classification. Solid curves show test accuracy when the classifier uses only one layer’s three finite H0 descriptors. Dashed horizontal lines show the corresponding all-layer 3-H0 classifier. Across AmbigQA, SituatedQA, and CLAMBER 9-way classification, single-layer topology features are generally weaker than the all-layer topology vector, indicating that ill-posedness is better captured as a layer-wise trajectory rather than a property of one isolated layer. Figure 5 shows that the all-layer 3-H0 representation consistently outperforms single-layer topology features. On AmbigQA, individual layers can achieve moderate accuracy, but they remain below the all-layer dashed baselines for all three models. On SituatedQA, the gap is larger: single-layer classifiers fluctuate across depth, while the all-layer representation remains substantially stronger. On CLAMBER 9-way classification, single-layer performance is especially unstable and far below the all-layer baseline, suggesting that fine-grained ill-posedness subtypes depend on how topology evolves across layers rather than on a single representational snapshot. Together, this ablation supports our use of all-layer topology vectors in the main classification experiments.
E
Compute Resources
All reported experiments were run on a single shared GPU server with four NVIDIA RTX A6000 GPUs, each with 49 GiB of GPU memory, two AMD EPYC 7513 CPUs with 64 physical cores and 128 hardware threads in total, 503 GiB of system RAM, and approximately 1.3 TB of local storage. 19
Across the experiments reported in the paper, the total local compute is on the order of hundreds of GPU-hours, with most cost coming from LLM forward passes for feature extraction, prompt baselines, and steering sweeps.
20