LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

L O D E S TA R: T RUSTWORTHY E NTROPY I S NAVIGATED , N OT M ERELY M EASURED R EINFORCED P OLARIZER K EEPS A F ROZEN LLM FROM B EING C ONFIDENTLY M ISLED BY THE W RONG E VIDENCE Po-Jen Ko1∗ Che-Cheng Wu1∗ Hung-Chun Hsu1∗† Li-Yang Chang1 Chuan-Ju Wang1‡ 1

Research Center for Information Technology Innovation, Academia Sinica, Taiwan

arXiv:2608.11922v1 [cs.CL] 12 Aug 2026

A BSTRACT Predictive-distribution entropy makes a strong selection rule in existing retrievalaugmented question answering work. We first verify it on our own benchmarks. Across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer F1 from 0.4769 to 0.5148 over the retriever’s top-ranked passage without any gold answer. Yet in this paper we show that this lowest-entropy selection rule, which prior entropy-based selectors adopt, fails in a specific and consequential way. A misleading passage makes the respondent confidently wrong, driving its entropy down precisely where the signal looks most trustworthy. Existing work either scopes its claims away from this failure mode or documents it without repairing it; we instead show that the failure comes from the passage that the respondent reads, and the context that passage is read in is an input we can intervene on. We therefore introduce L ODESTAR, to the best of our knowledge the first method to score a text intervention by the uncertainty it induces in a third-party frozen respondent, compared across one question’s candidates, which extends what the respondent reads instead of retraining what it is. L ODESTAR uses reinforcement learning to train, once and offline, a polarizer—a short fixed natural-language string inserted into the respondent’s prompt and never into its weights. Its training labels are built offline from gold answers and two LLM judges; inference reads neither. We then evaluate L ODESTAR comprehensively, running every competing selector under the same frozen respondent and the same retrieved candidate pool. On 5,008 questions from those five benchmarks, L ODESTAR attains the highest mean F1 of any inference-ready selector, the highest exact match (0.4136), and the highest GPT-4o judge score of the frozen-respondent configurations judged (0.6435); its three-seed mean wins all 70 method-by-dataset cells on F1 against fourteen published configurations, and it takes the highest macro exact match of any of them, 0.4136 against the best baseline’s 0.4039, while remaining pairedsignificant on F1 against every one. It raises the same five-benchmark F1 mean from 0.5148 to 0.5339, a further +3.71%. The gain holds on both sides of the domain split: 0.4643 to 0.4789 in-domain on NQ-Open, and 0.5274 to 0.5476 averaged over SQuAD, TriviaQA, EntityQuestions and WebQuestions. We further ablate the polarizer string and find that it is what makes the respondent read a misleading passage less often, 26.0% of the time against 30.3%.

∗

Equal contribution. Project lead; proposed the methodology, conducted the experiments, and wrote the manuscript. R10946017@citi.sinica.edu.tw ‡ Corresponding author: cjwang@citi.sinica.edu.tw †

1

Table 1: Judged-misleading rate of the retrieved candidate set. Percentage of a question’s top-10 passages the frozen judge (Section 3) labels MISLEADING; MACRO averages the five columns. Candidate pool

Reference

NQ SQuAD TriviaQA EntityQ WebQ MACRO n=1,000 n=1,000 n=1,000 n=1,008 n=1,000 n=5,008

Contriever (Izacard et al., 2022) + bge-reranker-v2-gemma (Chen et al., 2024b) + Qwen3-Reranker-8B (Zhang et al., 2025b)

24.2 32.5 38.6

21.8 31.5 33.8

11.8 17.5 18.7

22.3 34.6 39.3

21.4 30.7 32.6

20.3 29.4 32.6

bge-m3 (Chen et al., 2024b) + bge-reranker-v2-gemma (Chen et al., 2024b) + Qwen3-Reranker-8B (Zhang et al., 2025b)

33.4 33.6 39.5

27.2 32.8 34.8

16.0 18.6 19.8

36.3 41.2 45.6

31.4 34.1 35.5

28.9 32.1 35.0

Qwen3-Embedding-8B

35.1

30.9

17.7

34.5

30.2

29.7

1

(Zhang et al., 2025b)

I NTRODUCTION

Retrieval-augmented generation (RAG) answers a question from a retrieved corpus rather than from what the model memorized in training. In open-domain question answering the answer therefore turns on which passage the model was handed. A typical workflow has two steps: (1) a retriever returns the corpus passages most similar to the question, and (2) a respondent LLM, typically frozen, reads those candidates alongside the question and produces the answer. The trouble is that not all retrieved passages support the correct answer. Some are topically relevant but factually misleading, and nothing at inference time marks which is which. A misleading passage is worse than a useless one, because it supplies a false fact and the respondent’s answer then disagrees with the gold answer. In Table 1 we report how often that happens, on about a thousand questions from each of five common QA datasets, where, depending on the retrieval system, between 20.3% and 35.0% of the retrieved candidates mislead the frozen respondent. What is even worse is that using a stronger retriever or adding a reranker drives that misleading-passage rate up rather than down. As a result, deciding which candidate’s answer to trust without gold answers is an active research question in the RAG literature. Entropy-based selection is an effective way to choose among the candidates without gold answers, taking the entropy of the answer tokens produced under each candidate as that candidate’s score. The rule is then to keep the answer the respondent produces with the least uncertainty (i.e., the lowest entropy). Across the same five QA datasets, even the simplest entropy signal lifts mean answer F1 from 0.4769 to 0.5148 over simply taking the retriever’s top-ranked passage. Prior work computes this confidence in several forms, all from the respondent’s own distribution: entropy over sampled generations clustered by meaning (Farquhar et al., 2024), over the generation trajectory (Moslonka et al., 2026; Qiu et al., 2025), and over a short probe rollout (Song et al., 2026). Yet the confidently-wrong problem is where that effectiveness ends. The passage this rule selects still yields an answer with zero exact match 59.6% of the time. That low uncertainty does not imply correctness is documented (Taparia et al., 2026; Ma et al., 2025), and so is the finding that stronger retrieval systematically lowers predictive entropy (Soudani et al., 2025). Neither result says which of one question’s own candidates to trust; that is where the direction reverses, and within a single question a misleading passage lowers the respondent’s entropy (Figure 1a). The failure is more often sidestepped than met (Section 2), leaving the confidently-wrong problem unsolved. To proactively address the confidently-wrong problem we introduce L ODESTAR (Learned Orientation of Directed Entropy, Steering Trustworthy Answer Retrieval), which leaves the respondent frozen and learns instead one short natural-language polarizer ψ ⋆ , inserted after the passage p and before the question q. The prompt then reads [ p ; ψ ⋆ ; q ]. Across the same five datasets, L ODESTAR further lifts mean answer F1 from entropy selection’s 0.5148 to 0.5339 (Section 5). Trained once and offline by reinforcement learning, ψ ⋆ is optimized against a reward that is not answer correctness but the within-question separation of the frozen respondent’s entropy, raised on misleading passages and kept low on supporting ones (Figure 1b). We call the resulting signal directed entropy, because the polarizer steers the respondent’s entropy in a chosen direction rather than leaving it to be passively measured. L ODESTAR runs at inference with no extra model, no sampling and no supervision; the only thing it keeps from training is the polarizer ψ ⋆ , one fixed string, enough by itself to outperform every published method the paper tests (Section 5). Our contributions can be summarized as follows: 2

1. We show that ranking a question’s candidates by entropy alone is unreliable, and repair it with one learned string. Within a single question, the respondent’s entropy does not reliably separate misleading passages from supporting ones (Section 3); one learned string is enough to make it separate them. Ranking by entropy alone reads a misleading passage more often on average than drawing one of the ten at random, 30.3% against the pool’s own 28.9%. One learned string puts the selection below that floor on all five benchmarks, 26.0% macro-averaged (Tables 1 and 3). 2. We benchmark fourteen published methods re-purposed as selectors; L ODESTAR leads them on every metric it is measured on. Fourteen published configurations, each run on identical pools with an identical frozen respondent under the strongest setting its released code or paper supports (Section 4; per-row deviations in Appendix J), spanning prompt-search optimizers, uncertainty signals scored from the respondent’s own output, and trained rerankers that never see it. L ODESTAR attains the highest mean answer F1 (0.5339) and exact match (0.4136) of any of them, and the highest GPT-4o judge score of the frozen-respondent configurations judged (0.6435); its F1 lead is paired-significant against every configuration tested (Appendix D). 3. We map the polarizer over all nine train–inference pairs of three frozen respondents, Llama-3.1-8B, Qwen2.5-7B and Qwen3.5-9B; every diagonal helps, though the diagonal is not always where it helps most. The better entropy signal tracks the model family: first-token H1 on Llama, all-token on both Qwen models. What holds across all three is the polarizer’s effect. Section 6 reports the cross-respondent comparison.

(a) Naive entropy selection

(b)

question q : “who played the beast in the movie beauty and the beast” top-K retrieved passages pi ∈ PtopK

retrieval (e.g., bge-m3) 1

2

3

supporting (rank 1)

4

6

7

8

9

top-K retrieved passages pi ∈ PtopK

10

animated film . . . includes

“Beauty and the Beast” and

Emma Watson and Dan

Emma Watson and Dan

its numerous sequels”

Stevens”

passage p ∥ question q

Stevens”

polarizer ψ⋆

✗ mispick ↓

0.0021 0.0013

frozen respondent R

3

reimagining of Disney’s 1991

in the Disney animated film

animated film . . . includes

2

“a 2017 . . . live-action

“known as the voice of Beast

reimagining of Disney’s 1991

retrieval (e.g., bge-m3) 1

supporting (rank 1)

misleading (rank 10)

“a 2017 . . . live-action

arg mina(q,p) entropy(q, p)

5

L ODESTAR: directed entropy question q : “who played the beast in the movie beauty and the beast”

4

entropy

lowest entropy of all ten sits on the wrong passage

answer: “Robby Benson” ✗

7

8

9

10

“known as the voice of Beast in the Disney animated film “Beauty and the Beast” and its numerous sequels” 0.0121

passage p ∥ ψ ⋆ ∥ question q

arg mina(q,p;ψ⋆ ) entropy(q, p, ψ ⋆ )

6

misleading (rank 10)

✓ correct ↓ 0.0039

frozen respondent R

(lower = more confident)

5

×9.5

entropy (lower = more confident)

ψ ⋆ raises the misleading side; the arg min moves

answer: “Dan Stevens” ✓

Figure 1: Why minimum-entropy selection is misled, and what L ODESTAR changes. (a) The misleading passage makes the frozen respondent become confidently wrong: the respondent’s entropy is lowest on that passage (0.0013, the minimum over all ten, against the supporting passage’s 0.0021), and the lowest-entropy rule keeps the answer that passage induces, which is wrong. (b) L ODESTAR inserts one learned polarizer ψ ⋆ between each passage and the question, optimized to raise the respondent’s entropy on misleading passages while leaving supporting ones near the entropy floor. The misleading passage rises 9.5× on the unrounded probes, to 0.0121, while the supporting one rises only to 0.0039, so the same lowest-entropy rule now selects the supporting passage and the answer is correct. No gold answers are used at selection time.

3

2

R ELATED W ORK

The confidently-wrong problem. That low uncertainty does not imply correctness is documented in the literature (Taparia et al., 2026; Ma et al., 2025), and Soudani et al. (2025) establish that stronger retrieval systematically lowers predictive entropy. As we show in Table 1, retrieving with bge-m3 puts a higher average share of misleading passages in the top-10 than retrieving with Contriever, 28.9% against 20.3%. Moreover, on our five pools the respondent’s mean first-token entropy is 1.22 nats1 on judged-misleading candidates against 1.33 on the rest of the candidate set. A selector that keeps the lowest-entropy candidate is therefore drawn toward misleading passages and the incorrect answers they induce. Yet what a selector needs is directional. Reading a misleading passage should leave the respondent less certain, its answer-token entropy the higher one. Neither prior result meets this need. Both describe how entropy behaves across questions, while a selector only compares the candidates of one question, a gap L ODESTAR closes by training its polarizer to make misleading passages carry the higher entropy inside each question (Section 3). Uncertainty estimation for generation. Predictive entropy and its refinements are the standard estimators for measuring how uncertain a language model is about its answer. Semantic entropy (Farquhar et al., 2024) clusters sampled generations by entailment before computing entropy; EigenScore (Chen et al., 2024a) and SeaKR (Yao et al., 2025) read the dispersion (e.g., covariance eigenvalues, Gram determinant) of internal states across samples; EPR (Moslonka et al., 2026) and CL E H E (Qiu et al., 2025) score the generation trajectory, the step-by-step entropy of the answer as the respondent decodes it. Three further signals reach this task from outside hallucination detection. Min-K% Prob (Shi et al., 2024) and its vocabulary-normalized successor Min-K%++ (Zhang et al., 2025a) were introduced to test whether a text was part of a model’s training data. A text the model has seen keeps even its least-likely tokens probable. Re-purposed as selectors, they score the answer conditioned on each passage and keep the passage whose answer scores highest. SPS (Hu et al., 2026) scores a passage from its representation alone, without seeing the question. None of these signals changes the input that produced it. Yet with a frozen respondent the input is the only part of the system a selector is free to change; L ODESTAR uses exactly that freedom, learning one string that moves the respondent’s entropy instead of merely reading it. Entropy-based answer selection. Uncertainty also governs the upstream decision of whether to retrieve at all (Moskvoretskii et al., 2025); our question is which candidate’s answer to keep for the frozen respondent once retrieval has happened. The closest prior method that does select is IGP (Song et al., 2026). It ranks the retrieved candidates p ∈ P (q) by the information gain IG(p, q) = H̄L (q) − H̄L (q, p) each delivers to a frozen respondent, with H̄L (q, p) the mean normalized entropy of the L tokens that compose a short probe answer and H̄L (q) the same entropy with no passage in context. Because H̄L (q) is constant once the question is fixed, it cancels within a question, and the rule reduces to a minimum-entropy selector over the answers a(q, p) the candidates induce: IGP max IG(p, q) |â {z } = arg a(q,p), p∈P (q) | {z }

selected answer

information gain

= arg

min H̄L (q, p) . a(q,p), p∈P (q) | {z } measured entropy

(1)

A second strategy learns the decision rather than scoring it, and this does not by itself escape the confidently-wrong problem either. MBA-RAG (Tang et al., 2025) learns a bandit policy conditioned on the question alone; a policy that never reads the candidates can decide how to retrieve for a question, but not which of that question’s candidates to trust (Section 5). Learned guidance for retrieval-augmented answering. A line of work trains a model to emit text that guides how a retrieval-augmented system answers. Self-RAG (Asai et al., 2024) trains a model to generate reflection tokens; GainRAG (Jiang et al., 2025) aligns retriever and respondent through a perplexity-based gain signal whose teacher reads the gold answer; CTRL-RAG (Tan et al., 2026) uses reinforcement learning to make generation more context-faithful; and CRITIC-R1 (Xiao et al., 2026) leaves the generator frozen and trains a critic instead, rewarding it for aligning its verdicts with a teacher’s judgement of the answer. What they share is where the training target is read: from the text itself, whether that is an answer’s correctness, its groundedness in the passage, the passage’s relevance, or the quality of a critique of the trajectory. We stress that Self-RAG and 1

The unit of entropy taken with the natural logarithm.

4

CTRL-RAG train the answering model itself. Their respondent is not frozen, so they solve a different problem from ours, improving the model that answers rather than the evidence a fixed model answers from. L ODESTAR changes only the training signal. To the best of our knowledge, no prior work scores a text intervention by the uncertainty it induces in a third-party frozen respondent, compared across the candidates of one question.

3

M ETHODOLOGY

The problem: answer selection for a frozen respondent. A frozen respondent R is only as good as the single passage (possibly misleading, possibly supporting) it reads. That choice decides whether its answer is correct. Given a question q and a retrieved candidate set P (q) = {p1 , . . . , pK }, each candidate read alone leads R to one candidate answer, so the pool induces K answers to choose among. The signal: within-question entropy separation. Within a single question, a misleading passage lowers the respondent’s entropy; selection by answer-token entropy is therefore drawn to the wrong answers R produces confidently from misleading passages, over the less confident but correct ones. We denote H̄L (q, p) as the mean normalized entropy of the L tokens of the answer R generates from passage p alone, and a(q, p) as that answer. The answer selector keeps arg mina(q,p), p∈P (q) H̄L (q, p), which is the rule Equation 1 showed IGP reduces to. Let Mis(q) and Sup(q) be the passages of question q labelled misleading and supporting. The quantity L ODESTAR acts on is the within-question separation between the entropies R produces on Mis(q) and on Sup(q), which the polarizer is trained to drive positive inside each question: ∆Mis−Sup H̄L (q) =

mean H̄L (q, p) − mean H̄L (q, p).

p∈Mis(q)

p∈Sup(q)

(2)

The intervention: directed entropy. L ODESTAR’s methodological contribution is the RL-learned polarizer, one fixed natural-language string ψ ⋆ that turns the respondent’s entropy from a quantity we passively measure into one we intervene on. L ODESTAR leaves R frozen and inserts ψ ⋆ after the passage p and before the question q (the prompt is given in Appendix B). We call the result, H̄L (q, p, ψ ⋆ ), the directed entropy. The polarizer ψ ⋆ is optimized to move it with an intended sign, upward on misleading passages, unchanged on supporting ones. Under directed entropy the answer selector differs from Equation 1 by one symbol, the entropy it ranks now computed under ψ ⋆ : min H̄L (q, p, ψ ⋆ ) . â = arg |{z} {z } a(q,p;ψ ⋆ ), p∈P (q) |

selected answer

(3)

directed entropy

Training objective. The optimal polarizer ψ ⋆ is the string that maximizes the within-question separation its insertion induces. A generator policy proposes candidate polarizers from a single fixed prompt that contains neither labels nor gold answers,2 and is trained on their reward by GRPO (Shao et al., 2024). Each candidate polarizer ψ is credited with the separation of Equation 2 evaluated under ψ, baseline-corrected against the same question’s natural separation: i h b q∈B S(ψ) = E ∆Mis−Sup H̄L (q; ψ) − ∆Mis−Sup H̄L (q) , | {z } | {z } the natural separation, Equation 2 separation with polarizer ψ inserted (4) with ∆Mis−Sup H̄L (q; ψ) = mean H̄L (q, p, ψ) − mean H̄L (q, p, ψ), p∈Mis(q)

p∈Sup(q)

b q∈B averages over the question set B scored at that step, whose labelled passages are samwhere E pled from the top-K retrieved. S(ψ) is the polarizer’s reward, zero whenever ψ changes nothing and larger being better. The reward functional never reads an answer, gold or generated. The partition Mis(q)/Sup(q) that the reward contrasts is nonetheless built from gold- and judge-derived labels, so gold-freedom is a property of inference, not of training-data construction. Appendix B states the GRPO objective and its advantage in full, together with every setting. 2 The converged strings are reported in Appendix H, where independent seeds are shown to agree on what to say even though the reward never scores wording.

5

Passage labels. Two LLM judges from different model families (Zheng et al., 2023; Verga et al., 2024; Thomas et al., 2024), gpt-oss-120b (OpenAI, 2025) and Qwen2.5-72B-Instruct (Yang et al., 2024), read each passage independently. A passage counts as MISLEADING only when both agree that it is, as SUPPORTING when the respondent’s answer from it exactly matches a gold answer after normalization, and as NEUTRAL otherwise; the rare passage meeting both criteria is routed to NEUTRAL. On the 5,000 NQ-Open questions from which the training pool is drawn they agree with Cohen’s κ = 0.675, and on a held-out audit set an independent third-party judge returns their consensus label 96% of the time (Appendix G). Inference. Inference adds no model, no sampling, no supervision and no passage label. The respondent is run once per candidate to obtain H̄L (q, p, ψ ⋆ ), and Equation 3 returns the argmin. ψ ⋆ stays in the prompt while that answer is decoded, which is why Equation 3 selects among a(q, p; ψ ⋆ ) rather than a(q, p). The only artifact training carries forward is ψ ⋆ , one category-level string, and selection reads nothing but the respondent’s own entropy.

4

E XPERIMENTAL S ETTINGS

4.1

B ENCHMARK

Datasets and evaluation pools. We evaluate L ODESTAR and all baselines on five open-domain QA datasets: in-domain, Natural Questions (Kwiatkowski et al., 2019); out-of-domain, SQuAD, TriviaQA, EntityQuestions and WebQuestions (Rajpurkar et al., 2016; Joshi et al., 2017; Sciavolino et al., 2021; Berant et al., 2013), none of whose training data is ever seen. Evaluation pools are fixed in advance and shared by every method: n = 1,000 each except EntityQuestions at n = 1,008, each with the top ten bge-m3 (Chen et al., 2024b) candidates from one shared Wikipedia index. Frozen respondent and answer evaluation protocol. The frozen respondent is Llama-3.1-8BInstruct (Grattafiori et al., 2024), its parameters never updated.3 It answers greedily from the selected passage alone, under one shared answering template that carries ψ ⋆ for L ODESTAR and nothing extra for every other configuration. Every F1 in this paper scores that generated answer against the gold answer(s); exact match (EM) does the same after normalization. An LLM judge, GPT-4o (OpenAI, 2024), additionally rates whether the generated answer is equivalent to any gold answer, under a judging prompt adopted verbatim from Farquhar et al. (2024). Appendix F carries the exact-match table and full judge protocol, and Appendix G reports the judge’s numbers per dataset. 4.2

P OLARIZER TRAINING

Policy, data and budget. We adopt Qwen3-4B-Instruct (Yang et al., 2025) as the polarizer policy, trading generation quality against training cost at a scale we can afford to run three times, and train it with GRPO (Appendix B). The polarizer’s training questions are the full 1,939-question NQOpen training pool, disjoint by construction from every evaluation pool. The reported ψ ⋆ , and every polarizer reported in this paper, is the final checkpoint of a completed, fixed 300-step budget. Entropy estimator. Reproducing IGP, whose gain term averages entropy over all L answer tokens, we found that scoring the first answer token alone is no worse on the macro mean under a shared answering prompt (0.5148 against 0.5001 mean answer F1 ; Appendix C). L=1 is therefore the setting L ODESTAR uses, in its training reward and in its selector alike. The choice has precedent on both counts: reading a selection score from the first decoded position alone is established in neural reranking, where Gangi Reddy et al. (2024) take a full listwise ordering from the logits of the first generated identifier; and approximating a multi-token entropy by its first token is a standing convention, one Clark et al. (2025) test directly and caution against. The measurement above asks their question of this task, and answers it. Each baseline keeps the estimator its own paper specifies. H̄L becomes the respondent’s first-token entropy H1 , and the decoding a candidate needs to be scored collapses from L tokens to one, an L-fold saving on every candidate of every question. 3 We additionally run the cross-respondent comparison across two model generations, Qwen2.5-7B and Qwen3.5-9B, training a polarizer for each respondent and evaluating every pairing both in-domain and out-ofdomain; every respondent improves under its own polarizer on both sides of the split (Section 6).

6

Table 2: Passage-selection F1 with a frozen Llama-3.1-8B-Instruct respondent. All configurations select one of the same ten bge-m3 candidates and answer with the same frozen respondent, so the rows differ only in how that passage is chosen. Mean, EM and the LLM-judge column are five-pool macro means of F1 , of exact match (Appendix F) and of a GPT-4o answer-equivalence verdict (Appendix G). The L ODESTAR row is the mean of the three runs listed beneath it in all three. Answer F1 ↑ Method

In-domain

Reference

Out-of-domain

Answer EM ↑

LLM -Judge ↑

Mean

Mean full table in Appendix F

Mean full table in Appendix G

NQ

SQuAD

TriviaQA

EntityQ

WebQ

0.4789 0.4783 0.4707 0.4877

0.5968 0.5916 0.5954 0.6033

0.7181 0.7163 0.7215 0.7164

0.4839 0.4943 0.4661 0.4912

0.3917 0.3908 0.3877 0.3966

0.5339 0.5343 0.5283 0.5390

0.4136 0.4148 0.4107 0.4152

0.6435 0.6420 0.6365 0.6520

P UBLISHED SIGNALS RE - PURPOSED AS SELECTORS Semantic entropy (Farquhar et al., 2024) Self-RAG (reflection) (Asai et al., 2024) SeaKR (Yao et al., 2025) CLeHe (Qiu et al., 2025) EPR (Moslonka et al., 2026) Min-K% Prob (Shi et al., 2024) IGP (Song et al., 2026) EigenScore (Chen et al., 2024a) SPS (Hu et al., 2026) Min-K%++¶ (Zhang et al., 2025a)

0.4616 0.4474 0.4462 0.4346 0.4333 0.4345 0.4307 0.4163 0.3005 0.2186

0.5650 0.5585 0.5708 0.5749 0.5643 0.5466 0.5169 0.5437 0.2606 0.1947

0.6997 0.6856 0.6944 0.6817 0.6763 0.6909 0.6847 0.6711 0.5688 0.5030

0.4430 0.4640 0.4386 0.4377 0.4367 0.4210 0.4367 0.4160 0.2457 0.1872

0.3798 0.3805 0.3748 0.3719 0.3732 0.3693 0.3531 0.3624 0.3146 0.2586

0.5098 0.5072 0.5050 0.5002 0.4968 0.4924 0.4844 0.4819 0.3380 0.2724

0.4039 0.3833 0.3959 0.3783 0.3753 0.3797 0.3607 0.3711 0.2439 0.1994

0.6001 0.6212 0.6006 0.6063 0.6025 0.5886 0.5935 0.5754 0.4075 0.3250

T RAINED RERANKER ( RESPONDENT- FREE ) GainRAG (Jiang et al., 2025) MBA-RAG (Tang et al., 2025) CRITIC-R1 (Xiao et al., 2026)

0.4433 0.4191 0.4085

0.5057 0.5245 0.4987

0.6836 0.6513 0.6402

0.4675 0.4384 0.3980

0.3753 0.3491 0.3674

0.4951 0.4765 0.4626

0.3707 0.3591 0.3378

0.6054 0.5747 0.5712

S EARCH - BASED GUIDANCE GEPA (Agrawal et al., 2026) MIPROv2 (Opsahl-Ong et al., 2024)

0.4631 0.4589

0.5876 0.5814

0.7081 0.7018

0.4446 0.4510

0.3767 0.3725

0.5160 0.5131

0.3971 0.3987

0.6205 0.6141

R EFERENCE LINES random rank1 oracle

0.2798 0.4211 0.7080

0.2536 0.5245 0.8220

0.5549 0.6513 0.8438

0.2507 0.4384 0.6660

0.2966 0.3491 0.6039

0.3271 0.4769 0.7288

0.2351 0.3595 0.5880

0.3923 0.5749 0.8310

RL- LEARNED POLARIZER 5 L ODESTAR (ours) · – training-data seed 42 – training-data seed 43 ⋆ – training-data seed 44 ·

6

·

5

· 8

⋆ 5

· ·

·

· 6

· ⋆

·

5

mean over ten bge-m3 candidates bge-m3 top-1 candidate best of ten, in hindsight

¶

Min-K%++ (Zhang et al., 2025a) is the successor of Min-K% Prob (Shi et al., 2024), printed for completeness and not among the fourteen ranked configurations. Our reimplementation matches the official release term for term, and it still scores below random (Appendix J).

4.3

BASELINES

Every baseline is re-implemented as a passage selector under the strongest configuration its published settings or released code supports, and evaluated on the same pools with the same frozen respondent and the same answering template, with ψ ⋆ the only additional content L ODESTAR places in it. Fourteen ranked configurations fall into three families, Table 2’s three baseline group headers, differing in where the selection signal comes from: (i) a measurement from models as published, most often of uncertainty; (ii) a selector model we train; or (iii) a polarizer string we search for. We also report two system-level comparisons in Appendix I (Table 12): Self-RAG’s own trained-respondent configuration and our reproduction of the concurrent CTRL-RAG preprint (Tan et al., 2026). Both train the answering model itself, so they compare inference classes rather than selection quality and are not ranked. Table 2’s three reference lines (random, rank1, oracle) calibrate rather than compete. Per-row configurations, reference-line definitions, the load-bearing disclosures about how rows should be read, and every residual deviation from the official scripts are in Appendix J. Published signals re-purposed as selectors. Semantic entropy, Self-RAG (reflection), SeaKR, CLeHe, EPR, Min-K% Prob, IGP, EigenScore and SPS fill nine rows, each run exactly as published, with no training stage in our pipeline. Seven of them score each candidate from the frozen respondent’s own response, SPS scores the passage representation alone, and Self-RAG (reflection) selects with the method’s officially released critic, which is what keeps it under the frozen respondent. Search-based guidance. We also ask whether the polarizer can simply be searched for. GEPA and MIPROv2 answer that under L ODESTAR’s exact reward function, training draws and frozen respondent, so only the optimizer differs. Both configurations start from the same hand-written caution seed and select identically at inference (arg minp H1 (q, p, ψ)). Each row reports its strongest seed against L ODESTAR’s three-seed mean, a comparison deliberately set in the baselines’ favour. 7

in-domain (NQ, n = 1,000)

OOD mean (SQuAD / TriviaQA / EntityQ / WebQ)

evaluating respondent (selects and answers)

evaluating respondent (selects and answers)

polarizer ψ llama3.1-8b trained against Llama-3.1-8B polarizer ψ qwen2.5-7b trained against Qwen2.5-7B polarizer ψ qwen3.5-9b trained against Qwen3.5-9B

Llama-3.1-8B

Qwen2.5-7B

Qwen3.5-9B

0.4544

0.3903

0.3852

0.4363

0.3889

0.3629

+0.0210

+0.0028

0.4353 +0.0019

+0.0070

+0.0035

+0.0056

−0.0188

0.3868

0.4199

+0.0034

+0.0382

polarizer ψ llama3.1-8b trained against Llama-3.1-8B polarizer ψ qwen2.5-7b trained against Qwen2.5-7B polarizer ψ qwen3.5-9b trained against Qwen3.5-9B

Llama-3.1-8B

Qwen2.5-7B

Qwen3.5-9B

0.5375

0.4823

0.4848

0.5132

0.4730

0.4564

+0.0196

+0.0157

−0.0047

+0.0064

0.5078

0.4788

−0.0101

+0.0122

+0.0128

−0.0156

0.5221 +0.0500

Figure 2: Cross-respondent transfer of the polarizer under a mean-normalized-entropy (H̄L ) selector. Rows are the respondent the polarizer ψ was trained against, columns the frozen respondent that selects and answers; each of the eighteen facets prints answer F1 with ψ inserted (top) and ∆ against the same-pass configuration without it (bottom). Every diagonal is positive on both panels; all four negative facets are off-diagonal. Cell colour gives the sign of ∆, its depth the magnitude; largest gain +0.0500, largest harm −0.0188. Table 3: The polarizer ablation under two metrics. Both blocks compare L ODESTAR against L ODESTAR with ψ ⋆ removed, i.e., plain first-token entropy selection. Rates are P (MISLEADING | selected) at K=10 on the bge-m3 pool, whose own rate is 28.9; MACRO averages the five. Selection rule

NQ

EntityQ

WebQ

MACRO

J UDGED - MISLEADING RATE OF THE SELECTED PASSAGE (%) ↓ L ODESTAR w/o polarizer 36.7 24.7 15.9 39.1 L ODESTAR (ours) 32.7 21.4 13.9 32.7

35.0 29.5

30.3 26.0

A NSWER F1 ↑ L ODESTAR w/o polarizer L ODESTAR (ours)

0.3699 0.3917

0.5148 0.5339

0.4643 0.4789

SQuAD

0.5657 0.5968

TriviaQA

0.7129 0.7181

0.4610 0.4839

Trained rerankers. GainRAG, MBA-RAG and CRITIC-R1 replace the signal with a model trained on the same 1,939 NQ-Open questions as L ODESTAR, so NQ is in-domain and the other four out-of-domain. Each scores the same ten candidates: GainRAG a 278M cross-encoder distilled from a gold-conditioned gain teacher, MBA-RAG a DistilBERT bandit over candidate ranks, and CRITIC-R1 a GRPO-trained critic reading each cached draft answer. Re-run from stored artifacts on different hardware, all three reproduce their tabulated values (∆F1 =0.0000 on WebQuestions).

5

M AIN R ESULTS

Head-to-head against published signals. L ODESTAR attains the highest mean F1 of any frozenrespondent method, 0.5339 against 0.5148 for plain first-token entropy selection and 0.4769 for the retriever’s own top-1 passage. It takes the highest exact match (0.4136, Table 9) on the same pools, and the highest GPT-4o judge score of the frozen-respondent configurations judged (0.6435, Appendix G). Throughout this section mean F1 is the macro mean over the five pools, the Mean column of Table 2. The step from the entropy selector holds on both sides of the split that mean conceals: 0.4643 to 0.4789 in-domain on NQ, and 0.5274 to 0.5476 averaged over SQuAD, TriviaQA, EntityQuestions and WebQuestions. The L ODESTAR row of Table 2, the mean of its three training seeds, also wins all 70 method-by-dataset cells on F1 , and takes the highest macro exact match of any ranked row, 0.4136 against semantic entropy’s 0.4039 (Table 9). The sweep is a property of that row rather than of each run behind it: seed 43 loses the EntityQuestions F1 cell to GainRAG (0.4661 against 0.4675), while every run does win all five datasets against both entropy selection and rank1 (Appendix E). Those cells span the fourteen frozen-respondent-comparable baseline configurations we rank, meaning every configuration that picks one of those ten candidates and leaves the respon8

dent frozen. The three runs beneath the L ODESTAR row differ in the training-data seed alone. It reshuffles the grouping and ordering of the 1,939 training questions, while which questions exist, the train/held-out split and the trainer’s own seed are held fixed, so the three converged polarizers are scored on one held-out set (Appendix E). Every F1 advantage of L ODESTAR over a baseline configuration is paired-significant under both a t-test and Wilcoxon (Appendix D). Reporting conventions. Table 2 reports the mean of three training runs and lists them individually; the three differ only in their training-data seed, each a 300-step run over the same 1,939 questions in a different grouping and order, with their spread reported in Appendix E. A table that instead reports a single run says so in its caption. Every headline number uses the reward of Equation 4.

6

T HE P OLARIZER A BLATION

The ablation. We ablate L ODESTAR by removing the polarizer ψ ⋆ and changing nothing else. The frozen respondent, the pools and the arg min selection rule are identical, so the two configurations differ only in whether the string is inserted. That configuration is also the strongest untrained selector in our benchmark (Appendix J). Table 3 gives it per dataset under two metrics: answer F1 , the outcome, and how often the selected passage is judged MISLEADING, the quantity the reward acts on. On the latter the string helps on 5/5 datasets, from 2.0 points (TriviaQA) to 6.4 (EntityQuestions), and carries the rule below the pool’s own 28.9%, which entropy selection alone does not reach. Averaged over the three training seeds (Appendix E) the gain over the ablated configuration is +0.0191 ± 0.0054, with per-dataset gains from +0.005 (TriviaQA) to +0.031 (SQuAD) and 5/5 datasets positive under every seed. That lead is paired-significant under every seed individually, the three seeds giving pw = 2.4 × 10−8 , 1.8 × 10−4 and 2.9 × 10−11 , under the protocol of Appendix D, whose table carries the seed-42 run alone. Appendix H decomposes the entropy shift the string produces, by passage class. Cross-respondent transfer. Figure 2 repeats the comparison between inserting the polarizer ψ ⋆ and leaving it out, under a second selection signal (i.e., H̄L ) and across three frozen respondents from two model families, Llama-3.1-8B, Qwen2.5-7B and Qwen3.5-9B. All four negative facets are transfers to a respondent the string was never trained against. In the Qwen3.5-9B column the answer sequence ends at <|endoftext|> rather than <|im end|>, because that is where its generation actually stops. Restricting it to questions where both answers finished leaves the signs unchanged. Pooled across the figure’s two panels (5,008 questions) the diagonal stays at +0.0395 against +0.0477 unrestricted, and the Llama-3.1-8B transfer grows rather than shrinks.

7

C ONCLUSION

We first show that the frozen respondent’s first-token entropy chooses passages well enough to beat the retriever’s own ranking with no gold answer, 0.5148 mean answer F1 against its 0.4769, but that it fails in a way invisible from where prior work measures entropy. The standard entropy selector keeps the lowest-entropy candidate (Equation 1) and is drawn toward the passages it should avoid: pooled over all questions, judged-misleading candidates carry lower entropy than the rest (1.22 nats against 1.33; Section 2). Across the five pools it reads a judged-misleading passage more often on average than picking one of the ten at random (Table 3). We propose L ODESTAR, which repairs this without touching the respondent: a GRPO-trained policy writes the polarizer ψ ⋆ , one short natural-language string whose only job is to navigate the respondent’s entropy in the intended direction. Across 5,008 questions from five QA benchmarks that one string is enough to give L ODESTAR the highest mean F1 of any inference-ready selector, 0.5339, its three-seed mean winning every method-by-dataset F1 cell against fourteen published configurations (Table 2). Of that lead, the ablation removing ψ ⋆ with the respondent, pools and selection rule unchanged attributes +0.0191 ± 0.0054 F1 to the string itself, positive on 5/5 datasets under all three training seeds (Section 6). All of it costs one forward pass per candidate and no gold answers. Overall, entropy is the right signal here but only once directed rather than passively measured: one learned string turns within-question entropy from a signal that misleads into one that selects, with the respondent left frozen.

9

R EFERENCES Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations (ICLR), 2026. arXiv:2507.19457. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In International Conference on Learning Representations (ICLR), 2024. arXiv:2310.11511. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1533–1544, 2013. Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. INSIDE: LLMs’ internal states retain the power of hallucination detection. In International Conference on Learning Representations (ICLR), 2024a. arXiv:2402.03744. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL, pp. 2318–2335, 2024b. doi: 10.18653/v1/2024.findings-acl.137. arXiv:2402.03216. Christian Clark, Byung-Doh Oh, and William Schuler. How well does first-token entropy approximate word entropy as a psycholinguistic predictor? In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL), pp. 47–57, 2025. doi: 10.18653/v1/2025.ijcnlp-short.4. arXiv:2507.22209. Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630:625–630, 2024. doi: 10.1038/s41586-024 -07421-0. Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji. FIRST: Faster improved listwise reranking with single token decoding. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8642–8652, 2024. doi: 10.18653/v1/2024.emnlp-main.491. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022. arXiv:2106.09685. Zhanghao Hu, Qinglin Zhu, Siya Qi, Yulan He, Hanqi Yan, and Lin Gui. Beyond perplexity: Let the reader select retrieval summaries via spectrum projection score. In AAAI Conference on Artificial Intelligence, pp. 31104–31112, 2026. doi: 10.1609/aaai.v40i37.40371. arXiv:2508.05909. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. Transactions on Machine Learning Research, 2022. arXiv:2112.09118. Yi Jiang, Sendong Zhao, Jianbo Li, Haochun Wang, and Bing Qin. GainRAG: Preference alignment in retrieval-augmented generation through gain signal synthesis. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025. arXiv:2505.18710. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1601–1611, 2017. 10

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466, 2019. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1-zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025. arXiv:2503.20783. Huan Ma, Jiadong Pan, Jing Liu, Yan Chen, Joey Tianyi Zhou, Guangyu Wang, Qinghua Hu, Hua Wu, Changqing Zhang, and Haifeng Wang. Semantic energy: Detecting LLM hallucination beyond entropy. arXiv preprint arXiv:2508.14496, 2025. Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, and Alexander Panchenko. Adaptive retrieval without self-knowledge? bringing uncertainty back home. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025. ACL Anthology 2025.acl-long.319. Charles Moslonka, Hicham Randrianarivo, Arthur Garnier, and Emmanuel Malherbe. Learned hallucination detection in black-box LLMs using token-level entropy production rate. In European Conference on Information Retrieval (ECIR), volume 16483 of Lecture Notes in Computer Science, pp. 115–130. Springer, 2026. doi: 10.1007/978-3-032-21289-4 8. arXiv:2509.04492. OpenAI. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024. OpenAI. gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.10925, 2025. Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing instructions and demonstrations for multi-stage language model programs. In Proceedings of EMNLP, 2024. Zexuan Qiu, Zijing Ou, Bin Wu, Jingjing Li, Aiwei Liu, and Irwin King. Entropy-based decoding for retrieval-augmented large language models. In North American Chapter of the Association for Computational Linguistics (NAACL), 2025. ACL Anthology 2025.naacl-long.236. Qwen Team. Qwen3.5: Towards Native Multimodal Agents, 2026. https://qwen.ai/blog ?id=qwen3.5. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2383–2392, 2016. Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. Simple entity-centric questions challenge dense retrievers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021. arXiv:2109.08535. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In International Conference on Learning Representations (ICLR), 2024. arXiv:2310.16789. Zhipeng Song, Yizhi Zhou, Xiangyu Kong, Jiulong Jiao, Xinrui Bao, Xu You, Xueqing Shi, Yuhang Zhou, and Heng Qi. Less is more for RAG: Information gain pruning for generator-aligned reranking and evidence selection. arXiv preprint arXiv:2601.17532, 2026. Heydar Soudani, Evangelos Kanoulas, and Faegheh Hasibi. Why uncertainty estimation methods fall short in RAG: An axiomatic analysis. In Findings of the Association for Computational Linguistics: ACL, 2025. ACL Anthology 2025.findings-acl.852. 11

Zhehao Tan, Yihan Jiao, Dan Yang, Junjie Wang, Duolin Sun, Jie Feng, Xidong Wang, Lei Liu, Yue Shen, Jian Wang, and Jinjie Gu. CTRL-RAG: Contrastive likelihood reward based reinforcement learning for context-faithful RAG models. arXiv preprint arXiv:2603.04406, 2026. Xiaqiang Tang, Qiang Gao, Jian Li, Nan Du, Qi Li, and Sihong Xie. MBA-RAG: a bandit approach for adaptive retrieval-augmented generation through question complexity. In International Conference on Computational Linguistics (COLING), 2025. arXiv:2412.01572. Aditya Taparia, Ransalu Senanayake, Kowshik Thopalli, and Vivek Narayanaswamy. The anatomy of uncertainty in LLMs. arXiv preprint arXiv:2603.24967, 2026. Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. Large language models can accurately predict searcher preferences. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 1930–1940, 2024. doi: 10.1145/3626772.3657707. arXiv:2309.10621. Pat Verga, Sebastian Hofstätter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, and Patrick Lewis. Replacing judges with juries: Evaluating LLM generations with a panel of diverse models. arXiv preprint arXiv:2404.18796, 2024. Wenhan Xiao, Ziwei Zhang, Chuanyue Yu, Xingcheng Fu, Qingyun Sun, Runhua Xu, and Jianxin Li. CRITIC-R1: Learning structured critics for retrieval-augmented generation. arXiv preprint arXiv:2605.29886, 2026. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, and Juanzi Li. SeaKR: Self-aware knowledge retrieval for adaptive retrieval augmented generation. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 27022–27043, 2025. doi: 10.18653/v1/2025.acl-long.1312. arXiv:2406.19215. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Ru Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Yonghui Wu, and Mingxuan Wang. DAPO: An open-source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2503.14476. Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. Min-k%++: Improved baseline for detecting pre-training data from large language models. In International Conference on Learning Representations (ICLR), 2025a. arXiv:2404.02936. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025b. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023. arXiv:2306.05685.

12

A PPENDIX CONTENTS A

Notation. Every symbol used in the paper, collected.

B

GRPO objective and hyper-parameters. The objective of Equation 5 in full, and every hyper-parameter.

C

Aligned-probe ablation. What IGP’s gain term scores once its probe reads the answering prompt, against first-token entropy on the same five pools.

D

Paired significance tests. L ODESTAR against each method on F1 .

E

Run and seed stability. What separates the three training runs, and where the tested run sits among them.

F

Exact-match results and paired tests. The exact-match counterpart of Table 2, and the same paired tests on per-question EM differences.

G

LLM-judge scores per dataset. Table 2’s judge column opened up pool by pool, including the one pool where the judge and F1 disagree.

H

Learned polarizer text and its effect on the entropy. The converged strings themselves, the two framings independent seeds reach under a reward that never scores wording, and how the string shifts the entropy of supporting and non-supporting passages.

I

CTRL-RAG reproduction. Its strongest checkpoint, why no single checkpoint of that run can stand in for the method, what its reward was maximising while the format decayed, and the scope limits on what we do and do not claim at its published scale.

J

Baseline configurations and disclosures. Official settings, the disclosures about how individual rows should be read, and every residual deviation from the published scripts.

13

A

N OTATION

Table 4 collects the paper’s symbols for reference.

Table 4: Symbols used in the paper. Defined at first use in Sections 3 and 4; collected here for reference. Symbol

Meaning

q p P (q) = {p1 , . . . , pK } K R

a question a candidate passage the retrieved candidate set for q; we retrieve with bge-m3 (Chen et al., 2024b) the candidate-pool depth |P (q)|; K=10 throughout the main results the frozen respondent, marked where the tables record it: the LLM that answers from q and one p, parameters never updated (Llama-3.1-8B-Instruct) Shannon entropy of R’s first answer-token distribution, no polarizer the same entropy with polarizer ψ inserted mean normalized entropy of the L answer tokens R generates from passage p alone, no polarizer the same entropy with polarizer ψ inserted the misleading / supporting passages of q within-question separation, Equation 2 within-question separation with ψ inserted, Equation 4 a sampled polarizer; ψi the i-th of a group the inference-time polarizer: the single converged output. The star marks what survives to inference, so the policy πθ carries none the GRPO polarizer policy and its frozen initialization reward of polarizer ψ: the separation its insertion induces, baseline-corrected, Equation 4 group size (number of polarizers sampled per prompt) group-standardized advantage of ψi , Equation 6 token-level importance ratio the answer R generates from passage p alone the same answer with polarizer ψ inserted; Equation 3 keeps the entropyminimizing one the selected answer, Equation 3

H1 (q, p) H1 (q, p, ψ) H̄L (q, p) H̄L (q, p, ψ) Mis(q), Sup(q) ∆Mis−Sup H̄L (q) ∆Mis−Sup H̄L (q; ψ) ψ ψ⋆ πθ , πref S(ψ) G Âi ρi,t (θ) a(q, p) a(q, p; ψ) â

B

GRPO OBJECTIVE AND HYPER - PARAMETERS

This section states the GRPO objective in full: h
i   J (θ) = E min ρ Â, clip(ρ, 1 − εlow , 1 + εhigh ) Â − β DKL πθ ∥ πref ,

(5)

where ρ is the token-level importance ratio against πθold , ρi,t (θ) = πθ (ψi,t | x, ψi,<t ) / πθold (ψi,t | x, ψi,<t ), and πref is the frozen initial policy. At each training step πθ samples a group of G polarizers from the single fixed prompt x, and all G candidates are scored by Equation 4 on the same batch of labelled questions; a step draws 16 such groups, the 128 sampled strings of Table 5. GRPO (Shao et al., 2024) replaces a learned value baseline with the statistics of the sampled group: the advantage of polarizer ψi is its reward standardized over the G rewards of that group,  G S(ψi ) − mean S(ψj ) j=1 Âi = , (6)  G std S(ψj ) j=1 broadcast to every token of ψi . Following Liu et al. (2025) and Yu et al. (2025), we aggregate token-mean over the group, so the sequence-level reward is not silently re-weighted by polarizer length. The clip range is asymmetric, (εlow , εhigh ) = (0.2, 0.28), and the KL term uses β = 10−3 with the low-variance k3 estimator, kept in the loss rather than in the reward so that Equation 4 remains the only quantity the advantage ranks. Because all G polarizers of a group are scored on the 14

same questions under the same frozen respondent, this advantage ranks nothing but phrasings of the polarizer. Table 5 lists the configuration behind Equations 5 and 6. Every polarizer reported in this paper is read at the final step of the 300-step budget, with no per-run checkpoint selection of any kind. The polarizer policy emits its polarizer wrapped in <critique>...</critique> tags, a name inherited from the training harness whose contents are the polarizer ψ of Section 3 and nothing else, and the string is parsed out before scoring. The policy prompt contains only the fixed categorylevel instruction and unlabelled exemplar question–passage pairs, never a label, a gold answer, or respondent feedback. ψ ⋆ itself is inserted into the respondent’s prompt rather than the policy’s, after the passage p and before the question q: it reads Passages: p / An expert’s analysis of the passage above: ψ ⋆ / Question: q.

Table 5: GRPO configuration used for every training run reported in this paper.

C

Setting

Value

Polarizer policy πθ Reference policy πref Group size G Question groups per step Training budget Sampling temperature Maximum polarizer length Advantage Loss aggregation Clip range (εlow , εhigh ) KL regularization Reward Malformed samples

Qwen3-4B-Instruct frozen initial policy 8 16 (128 sampled strings per step) 300 steps (one epoch of the 4,800-row file from the 1,939-question pool) 1.1 96 tokens group-standardized (Equation 6) token-mean over the group (0.2, 0.28); dual-clip constant 3.0 in-loss, low-variance k3 estimator, β = 10−3 Equation 4; per-passage entropy shifts clipped to ±2 fixed penalty in place of a respondent evaluation

Trainer hardware Wall-clock

4× NVIDIA RTX PRO 6000 Blackwell (96 GB) ≈ 7 h for the 300-step run (median 85 s/step)

A LIGNED - PROBE ABLATION

Appendix J’s IGP entry attributes part of its position to a prompt mismatch rather than to its gain formulation: its published implementation probes with its own short prompt, whereas the polarizer ablation’s H1 (q, p) (Section 6) is read from the very prompt that produces the scored answer. We isolate that factor by re-scoring IGP’s per-passage all-token mean entropy H̄L (q, p) under the shared answering prompt, with the same frozen respondent on the same five pools (Table 6). The two macro means are close, 0.5148 for H1 against 0.5001 for the aligned all-token mean, but that closeness holds only on average: per pool the two disagree in both directions, H1 ahead by +0.0339 on TriviaQA and the aligned mean ahead by 0.0070 on SQuAD. The narrowing does not move IGP’s ranked row. Every baseline is ranked under the strongest configuration its published settings or released code supports (Section 4.3). Probing with the answering prompt is a change we make to IGP’s gain term, not an option those settings offer. Table 2 therefore keeps IGP at its published configuration’s macro mean of 0.4844, and 0.5001 stands as an ablation result. The aligned probe recovers about half the distance from that row to H1 ’s 0.5148, and leaves H1 ahead on the macro mean.

15

Table 6: First-token entropy against the same-prompt all-token mean that IGP’s gain term uses, on the five canonical pools, scored with the frozen respondent under the shared answering prompt (as in Table 2). The two macro means are close (0.5148 vs. 0.5001), but per pool the two variants disagree in both directions: H1 leads by its largest margin on TriviaQA (+0.0339), while the aligned all-token mean leads by its largest margin on SQuAD (−0.0070); the small mean gap is not per-pool equivalence. Answer F1 ↑ (generated vs. gold) Variant

In-domain

Out-of-domain

Mean

NQ

SQuAD

TriviaQA

EntityQ

WebQ

H1 (q, p), first-token entropy H̄L (q, p), aligned all-token mean

0.4643 0.4364

0.5657 0.5727

0.7129 0.6791

0.4610 0.4402

0.3699 0.3722

0.5148 0.5001

∆ (H1 − H̄L )

+0.0279

−0.0070

+0.0339

+0.0208

−0.0023

+0.0146

D

PAIRED SIGNIFICANCE TESTS

Because all methods select from the same ten candidates of the same question, we test per-question F1 differences with paired bootstrap (10,000 resamples), a paired t-test and Wilcoxon signed-rank (Table 7), every row of it scored on all five pools and all 5,008 questions. Every test in this appendix is paired on the seed-42 run’s per-question selections, that being the fixed-budget run whose polarizer is used at inference; Table 2’s L ODESTAR row is instead the mean of three training runs, and is characterized by its ±0.0054 spread rather than by a p-value. Three rows carry a disclosure with their marker. The §§ configuration’s original run predates the per-question logging these tests need and was re-run with it, greedy CLeHe reproducing its tabulated macro F1 to −0.0007, so the test is of the re-run. The ∗ configuration is reconstructed under first-wins tie-breaking, which favours official semantic entropy: its tie rate is 62–79% with ∼ 2 distinct scores per 10 candidates, so first-wins imports a rank-1 prior. Removing it (random tiebreak) puts that method at mean F1 0.4774, indistinguishable from rank1’s 0.4769. The ∥ row’s paired vector comes from EPR’s official-budget configuration (Appendix J), macro 0.4955, rather than from the ranked 24-token row, macro 0.4968, so its ∆F1 reads +0.0389 where the ranked row’s macro difference is +0.0375. Every ∆F1 , 95% confidence interval and p-value behind these comparisons is in Table 7, and the exact-match tests are in the last two columns of Table 9 (Appendix F). “Winning all 70 cells” is a statement about cell-wise values rather than about per-dataset statistical significance; the statistically grounded form of the claim is “highest mean F1 , paired-significant on both tests against all fourteen baseline configurations ranked”. Plain entropy against the retrieval prior, respondent by respondent. The polarizer’s value is easiest to read where the signal it steers has none. On NQ, and within a single pass that recomputes both configurations question by question (n = 1,000), selection by first-token H1 alone is statistically indistinguishable from taking the retriever’s top-ranked passage on the two Qwen respondents. Each figure comes from that respondent’s own pass, which recomputes both configurations together, so the contrast is exactly paired within it. On Qwen2.5-7B the difference is +0.0139 [−0.0113, +0.0391], pw = 0.42; on Qwen3.5-9B it is +0.003, pw = 0.64. Llama-3.1-8B is the one respondent of the three on which the untrained signal separates from the retrieval prior by itself—0.4643 against rank1’s 0.4211 on its own NQ pass, and 0.5148 against 0.4769 over the five pools (Table 2). Each respondent’s polarizer is trained on that same first-token statistic, so the comparison is matched, and inserting it carries those two past rank1, to 0.3701 over 0.3452 and 0.4086 over 0.3517. Two qualifications belong beside that. The all-token estimator, which we do not train on, does separate from rank1 on Qwen2.5-7B (+0.0395 [+0.0143, +0.0649], pw = 2.8 × 10−3 ), so the failure is first-token H1 ’s rather than every untrained signal’s. And a signal’s standalone value does not predict what the reward can extract from it, Qwen3.5-9B having the weakest standalone signal of the three and the largest gain over its own H1 configuration, +0.0544, pw = 8.3 × 10−10 .

16

Table 7: Paired significance of L ODESTAR against each method on per-question F1 differences. Every row is tested against the same L ODESTAR run on the same questions, so the rows differ only in the method being compared. ∆F1 is that difference in answer F1 as defined in Section 4, reported with its 95% bootstrap interval and two paired tests, and bold marks p < 0.05 on both. Rows are banded by method family, in Table 2’s order; §§ marks a configuration re-run for per-question logging and ∗ a configuration reconstructed under first-wins tie-breaking, and ∥ one whose paired vector is its official-budget run; all three are stated in full in the text. Method

Reference

∆F1 95% CI

pt

pWilcoxon

P UBLISHED SIGNALS RE - PURPOSED AS SELECTORS Semantic entropy∗ (Farquhar et al., 2024) Self-RAG (reflection) (Asai et al., 2024) SeaKR (Yao et al., 2025) CLeHe§§ (Qiu et al., 2025) EPR∥ (Moslonka et al., 2026) Min-K% Prob (Shi et al., 2024) IGP (Song et al., 2026) EigenScore (Chen et al., 2024a) SPS (Hu et al., 2026)

+0.0246 +0.0272 +0.0295 +0.0348 +0.0389 +0.0420 +0.0500 +0.0526 +0.1965

T RAINED RERANKER ( RESPONDENT- FREE ) GainRAG (Jiang et al., 2025) MBA-RAG (Tang et al., 2025) CRITIC-R1 (Xiao et al., 2026)

+0.0393 [+0.0289, +0.0496] +0.0579 [+0.0468, +0.0689] +0.0719 [+0.0613, +0.0826]

6.3×10−14 4.3×10−25 4.6×10−39

2.2×10−13 4.4×10−24 1.3×10−36

S EARCH - BASED GUIDANCE GEPA (Agrawal et al., 2026) +0.0183 [+0.0105, +0.0262] MIPROv2 (Opsahl-Ong et al., 2024) +0.0212 [+0.0134, +0.0290]

4.1×10−6 1.4×10−7

5.7×10−6 1.8×10−7

R EFERENCE LINES rank1 oracle

+0.0575 [+0.0464, +0.0685] −0.1943 [−0.2035, −0.1853]

9.1×10−25 9.0×10−24 ≈ 0 5.6×10−287

P OLARIZER ABLATION L ODESTAR w/o polarizer

+0.0197 [+0.0128, +0.0267]

2.1×10−8

E

[+0.0149, +0.0342] 4.0×10−7 4.4×10−7 [+0.0171, +0.0373] 1.8×10−7 4.1×10−7 [+0.0207, +0.0383] 7.3×10−11 7.9×10−11 [+0.0258, +0.0438] 2.3×10−14 5.1×10−14 [+0.0299, +0.0480] 3.5×10−17 1.1×10−16 [+0.0330, +0.0512] 1.3×10−19 4.1×10−19 [+0.0398, +0.0602] 2.6×10−21 1.2×10−20 [+0.0427, +0.0622] 5.1×10−26 4.4×10−25 [+0.1840, +0.2091] 8.4×10−206 3.8×10−169

2.4×10−8

RUN AND SEED STABILITY

Table 2 reports L ODESTAR as the mean of three training runs and lists those runs beneath its L ODESTAR row. This appendix says what separates them, which is the training-data seed and nothing else. Each of the three is its own 300-step run under the inference-time configuration, evaluated at its final checkpoint, and each consumes its own training file built by the same builder under seed 42, 43 or 44. That seed draws 4,800 rows of 16 question groups per step from the same 1,939-question training pool, so 300 steps is one epoch over the file and what the seed varies is the grouping and ordering of the training questions, not which questions exist. Nothing else differs: the train/held-out split seed is pinned at 42 in all three, so they share one held-out split byte for byte, and the trainer’s own seed is never overridden. They land at mean F1 0.5343 (seed 42), 0.5283 (seed 43) and 0.5390 (seed 44), that is 0.5339 ± 0.0054, with a gain over entropy of +0.0191 ± 0.0054 and every run winning all five datasets against both entropy selection and rank1. Two properties of that spread matter for how the rest of the paper should be read. No run involves a checkpoint choice of any kind, each being the final step of the fixed budget (Appendix B). And the seed-42 run, which carries every paired test in this paper, is the median of the three rather than the best of them, so the tested run is a representative draw and not a selected peak. An execution floor for that spread. A fourth run places the ±0.0054 against something. The seed-42 configuration was trained a second time on the identical training file, again for the full 300-step budget and read at its own final checkpoint, and it lands at 0.5333 against that seed’s 0.5343. What separates the two is execution alone, meaning floating-point accumulation order, rollout sampling, and where the reward readers were placed. The variability therefore has three levels of its own. A same-seed re-run moves the five-dataset mean by 0.0010, a change of trainingdata seed moves it by 0.0054 (sd), and the method effect across the three seeds runs from +0.0135 17

to +0.0243, so the quantity this paper reports is more than an order of magnitude above the noisiest level beneath it.

F

E XACT- MATCH RESULTS AND PAIRED TESTS

Table 9 repeats the comparison of Table 2 under exact match, with the same three training seeds averaged into the L ODESTAR row and listed beneath it. The ordering is largely preserved and every claim made on F1 holds under EM. Against the fourteen ranked configurations L ODESTAR takes the highest macro exact match, 0.4136 against official semantic entropy’s 0.4039, and it does so without winning every cell: on WebQuestions the three-seed mean trails that method, 0.2283 against 0.2300, a deficit belonging to the mean more than to every run, since seed 44 takes the cell on its own at 0.2310 while seeds 42 and 43 land at 0.2280 and 0.2260. The system-level Self-RAG row, marked § in the table, is excluded from that count, because it answers with its own trained generator in an extraction configuration and is not frozen-respondent-comparable; that disclosure is given under Trained generators in Appendix J, with every other per-row configuration. The POLARIZER ABLATION row carries no seed breakdown, that configuration having no polarizer and being identical under all three training seeds. Our CTRL-RAG reproduction is not a row here either, as in Table 2; it is reported in the case study of Appendix I, where its exact-match numbers are tabulated. The last two columns of Table 9 repeat the paired tests of Table 7 on the per-question EM differences of the same 5,008 aligned questions. Their values are aligned on the × so that the exponents can be compared down the column, and every one is printed rather than marked, so how close a comparison runs to 0.05 can be read off its own row. Every conclusion of the F1 tests survives under EM, the narrowest margin being the lead over official semantic entropy, which clears 0.05 on both tests but by less than any other row (pt = 2.3 × 10−2 , pWilcoxon = 4.8 × 10−2 ), which is unsurprising for a coarser integer-valued score on what was already the closest comparison under F1 . The polarizer ablation, the cleanest test of the contribution, remains significant under EM on both tests. LLM-as-judge protocol. Because both F1 and EM are lexical, we additionally audit answer correctness with an LLM judge, so that no conclusion rests on string overlap alone. To keep the audit independent of our own design choices, the judging prompt is adopted verbatim from the official code release of the semantic-entropy paper (Farquhar et al., 2024), in its multi-reference branch, since every dataset here ships a list of gold answers. The judge is gpt-4o-2024-08-06 (OpenAI, 2024), queried at temperature 0 with a ten-token decoding budget and constrained to a yes/no verdict, which we map to 1 and 0; a reply that is neither is re-queried once and otherwise counted as incorrect. With the per-example fields as placeholders, the prompt is reproduced in Table 8.

Table 8: The LLM-judge prompt, adopted verbatim from the official release of Farquhar et al. (2024) in its multi-reference branch. Braced tokens are the per-example fields substituted at query time. Prompt template Instruction

We are assessing the quality of answers to the following question: {question} The following are expected answers to this question: {correct answers}. The proposed answer is: {predicted answer} Within the context of the question, does the proposed answer mean the same as any of the expected answers? Respond only with yes or no.

Constrained answer cue Response:

18

Table 9: Exact match on the same pools and methods as Table 2. Its rows differ from that table’s in two places: Min-K%++, printed there for completeness, is not repeated, and the system-level Self-RAG row (§ ) is added. Every cell is 1 where the respondent’s generated answer matches a gold answer after SQuAD normalization and 0 otherwise, averaged over the pool, so the rows carry that table’s one-row-per-method rule and family grouping under a second metric. The last two columns are that row’s paired tests against L ODESTAR on per-question EM differences, under the protocol of Appendix D; a — there marks a row that admits no per-question vector, and § a system-level row, whose test therefore compares inference classes rather than selection quality. The L ODESTAR row is the mean of the three training seeds listed beneath it, as in Table 2; reference lines keep their band’s italic, and the oracle pair is the one comparison that runs against L ODESTAR rather than for it. Answer exact match (EM) ↑ Method

In-domain

Reference

Paired tests vs. L ODESTAR (ours)

Out-of-domain

Mean

pt

pWilcoxon

NQ

SQuAD

TriviaQA

EntityQ

WebQ

0.3663 0.3670 0.3620 0.3700

0.4817 0.4770 0.4800 0.4880

0.6423 0.6420 0.6470 0.6380

0.3492 0.3601 0.3383 0.3492

0.2283 0.2280 0.2260 0.2310

0.4136 0.4148 0.4107 0.4152

— — — —

— — — —

P UBLISHED SIGNALS RE - PURPOSED AS SELECTORS Semantic entropy (Farquhar et al., 2024) SeaKR (Yao et al., 2025) Self-RAG (reflection) (Asai et al., 2024) Min-K% Prob (Shi et al., 2024) CLeHe (Qiu et al., 2025) EPR (Moslonka et al., 2026) EigenScore (Chen et al., 2024a) IGP (Song et al., 2026) SPS (Hu et al., 2026)

0.3590 0.3450 0.3450 0.3260 0.3210 0.3210 0.3120 0.3150 0.2120

0.4710 0.4630 0.4450 0.4410 0.4530 0.4440 0.4430 0.4050 0.1800

0.6300 0.6210 0.6070 0.6150 0.5960 0.5890 0.5930 0.6030 0.4970

0.3294 0.3274 0.3135 0.3026 0.3095 0.3075 0.3026 0.2986 0.1657

0.2300 0.2230 0.2060 0.2140 0.2120 0.2150 0.2050 0.1820 0.1650

0.4039 0.3959 0.3833 0.3797 0.3783 0.3753 0.3711 0.3607 0.2439

2.3 × 10−2 4.4 × 10−5 4.3 × 10−9 2.1 × 10−13 2.3 × 10−15 6.1 × 10−18 1.6 × 10−17 3.6 × 10−23 7.1 × 10−147

4.8 × 10−2 4.1 × 10−4 4.0 × 10−7 2.5 × 10−10 9.0 × 10−12 1.1 × 10−13 2.4 × 10−13 1.8 × 10−17 6.0 × 10−98

T RAINED RERANKER ( RESPONDENT- FREE ) GainRAG (Jiang et al., 2025) MBA-RAG (Tang et al., 2025) CRITIC-R1 (Xiao et al., 2026)

0.3280 0.3140 0.3010

0.4010 0.4120 0.3800

0.6040 0.5750 0.5480

0.3194 0.3056 0.2669

0.2010 0.1890 0.1930

0.3707 0.3591 0.3378

9.0 × 10−16 7.4 × 10−22 2.6 × 10−42

4.5 × 10−12 1.6 × 10−16 4.3 × 10−31

S EARCH - BASED GUIDANCE MIPROv2 (Opsahl-Ong et al., 2024) GEPA (Agrawal et al., 2026)

0.3460 0.3430

0.4740 0.4680

0.6220 0.6290

0.3323 0.3274

0.2190 0.2180

0.3987 0.3971

1.2 × 10−4 1.7 × 10−5

8.8 × 10−4 2.0 × 10−4

T RAINED GENERATOR Self-RAG§

RL- LEARNED POLARIZER L ODESTAR (ours) – seed 42 – seed 43 – seed 44

5

6

· ·

·

5

· 8

⋆ 5

·

⋆

·

·

· 6

· ⋆

·

5

0.3990

0.0800

0.2990

0.0923

0.1120

0.1965

1.2 × 10−201

1.3 × 10−128

R EFERENCE LINES random rank1 oracle

0.1928 0.3160 0.5710

0.1731 0.4120 0.6930

0.4795 0.5750 0.7700

0.1752 0.3056 0.5099

0.1550 0.1890 0.3960

0.2351 0.3595 0.5880

— 1.5 × 10−21 2.4 × 10−229

— 2.6 × 10−16 3.5 × 10−143

P OLARIZER ABLATION L ODESTAR w/o polarizer ψ ⋆ removed: plain arg minp H1

0.3620

0.4600

0.6400

0.3433

0.2160

0.4043

2.1 × 10−3

7.6 × 10−3

G

(Asai et al., 2024)

LLM- JUDGE SCORES PER DATASET

Table 2 reports the LLM-judge column as a five-pool mean; Table 10 opens it up per dataset, in the same layout and row order. The F1 ordering is reproduced pool by pool with one exception worth stating: on WebQuestions the Self-RAG (reflection) configuration leads the seed-42 run that carries this table’s paired tests, 0.5510 against 0.5430 (0.5497 for the three-seed mean), where under token F1 L ODESTAR wins that cell. The disagreement is confined to one pool and does not move the mean, where L ODESTAR leads the best baseline by +0.0208 on that run and by +0.0223 on the three-seed mean, but it is the kind of per-pool reversal a five-pool average hides, which is why we print the breakdown. Every mean in Table 10 reproduces the corresponding cell of Table 2 exactly wherever both tables report one, which is the check that the two columns are the same measurement and not two independent judge runs. Semantic entropy’s 0.6001 is opened up per pool here as well, having been obtained the same way: the corrected run’s cached selections looked up in that one judging pass, with no query of its own. The two paired-test columns are the one place in this table where the three training seeds do not average. Their L ODESTAR configuration is the seed-42 run’s per-question verdict file, whose five means are the seed-42 row, so neither test speaks for the three-seed mean that the L ODESTAR row and Table 2 report. They add no query to the judging the rest of the table already rests on, that pass having scored all ten candidates of every one of the 5,008 questions, so each configuration’s per-question verdict is the verdict at the rank it selected. A — in those two columns is therefore a 19

Table 10: LLM-judge accuracy per dataset (GPT-4o answer-equivalence, five pools), opening up the five-pool mean of Table 2. Same rows and grouping as Table 9, with L ODESTAR in bold and reference lines in italic. The last two columns are that row’s paired tests against L ODESTAR on perquestion judge verdicts, under the protocol of Appendix D, and the oracle pair is the one comparison that runs against L ODESTAR rather than for it. The L ODESTAR row is the mean of the three training seeds listed beneath it, as in Table 2, while the two test columns read the seed-42 run alone. A — is never a zero and never a test that fell short: in the first six columns it is a measurement this table does not report, and in the last two a comparison we did not run. In-domain Method

Paired tests vs. L ODESTAR (ours)

Out-of-domain

NQ

SQuAD

TriviaQA

EntityQ

WebQ

Mean

pt

pWilcoxon

0.5803 0.5780 0.5720 0.5910

0.7237 0.7210 0.7220 0.7280

0.7570 0.7530 0.7590 0.7590

0.6068 0.6151 0.5833 0.6220

0.5497 0.5430 0.5460 0.5600

0.6435 0.6420 0.6365 0.6520

— — — —

— — — —

P UBLISHED SIGNALS RE - PURPOSED AS SELECTORS Self-RAG (reflection) (Asai et al., 2024) Semantic entropy (Farquhar et al., 2024) CLeHe (Qiu et al., 2025) EPR (Moslonka et al., 2026) SeaKR (Yao et al., 2025) IGP (Song et al., 2026) Min-K% Prob (Shi et al., 2024) EigenScore (Chen et al., 2024a) SPS (Hu et al., 2026)

0.5520 0.5520 0.5540 0.5530 0.5390 0.5450 0.5350 0.5050 0.3690

0.6870 0.6610 0.6840 0.6740 0.6860 0.6170 0.6470 0.6600 0.3260

0.7220 0.7390 0.7320 0.7270 0.7330 0.7290 0.7280 0.7090 0.5930

0.5942 0.5317 0.5377 0.5357 0.5218 0.5526 0.5179 0.5060 0.2986

0.5510 0.5170 0.5240 0.5230 0.5230 0.5240 0.5150 0.4970 0.4510

0.6212 0.6001 0.6063 0.6025 0.6006 0.5935 0.5886 0.5754 0.4075

7.6 × 10−4 3.5 × 10−13 5.1 × 10−11 8.0 × 10−13 1.9 × 10−14 1.9 × 10−15 1.3 × 10−21 8.6 × 10−30 5.0 × 10−212

3.6 × 10−3 3.6 × 10−10 1.5 × 10−8 6.7 × 10−10 4.2 × 10−11 7.9 × 10−12 2.4 × 10−16 3.1 × 10−22 3.9 × 10−134

T RAINED RERANKER ( RESPONDENT- FREE ) GainRAG (Jiang et al., 2025) MBA-RAG (Tang et al., 2025) CRITIC-R1 (Xiao et al., 2026)

0.5560 0.5120 0.5290

0.6120 0.6220 0.6130

0.7240 0.6840 0.6870

0.6002 0.5625 0.5040

0.5350 0.4930 0.5230

0.6054 0.5747 0.5712

3.9 × 10−9 3.3 × 10−24 1.6 × 10−27

3.7 × 10−7 3.2 × 10−18 1.3 × 10−20

S EARCH - BASED GUIDANCE GEPA (Agrawal et al., 2026) MIPROv2 (Opsahl-Ong et al., 2024)

0.5670 0.5530

0.7100 0.6900

0.7490 0.7490

0.5496 0.5635

0.5270 0.5150

0.6205 0.6141

7.0 × 10−6 9.3 × 10−9

1.0 × 10−4 7.0 × 10−7

T RAINED GENERATOR Self-RAG§

RL- LEARNED POLARIZER L ODESTAR (ours) – seed 42 – seed 43 – seed 44

Reference 5

6

· ·

·

5

· 8

⋆ 5

·

⋆

·

·

· 6

· ⋆

·

5

0.5190

0.6110

0.6850

0.6161

0.5570

0.5976

2.6 × 10−10

5.0 × 10−8

R EFERENCE LINES random rank1 oracle

0.3560 0.5130 0.8240

0.3195 0.6220 0.9220

0.5797 0.6840 0.8710

0.3008 0.5625 0.7778

0.4057 0.4930 0.7600

0.3923 0.5749 0.8310

— 4.4 × 10−24 1.4 × 10−255

— 3.9 × 10−18 2.1 × 10−156

P OLARIZER ABLATION L ODESTAR w/o polarizer ψ ⋆ removed: plain arg minp H1

0.5600

0.6830

0.7490

0.5585

0.5140

0.6129

3.4 × 10−12

1.9 × 10−9

(Asai et al., 2024)

comparison we did not run rather than a test that ran and fell short, and only two kinds of row still carry one. L ODESTAR and its seeds are the comparison’s own reference, and the random line is an expectation over the ten ranks rather than a per-question selection, so neither admits a paired vector at all. The system-level Self-RAG row is tested and reads § : that method trains its own answering model, so its test compares inference classes rather than selection quality and the row stays outside the fourteen frozen-respondent-comparable configurations of Table 2, which it is not a row of. The values themselves are aligned on the × so that the exponents can be compared down the column. Auditing the labelling judge. The judge above scores answers; the passage labels of Section 3 come from a separate pair of labelling judges, whose inter-judge agreement is Cohen’s κ = 0.675. A separate benchmark audits the same judge pair against an independent 70B bf16 third-party judge, Llama-3.1-70B-Instruct (Grattafiori et al., 2024), on 1,300 held-out rows: agreement with the dualjudge consensus label, measured on the 600 consensus rows, is 96%, with a 1.5% false-positive rate on 200 no-injection anchors. The two figures are not the same measurement and do not confirm each other: κ is chance-corrected and the 96% is not, and the two are computed on different sets of rows.

H

L EARNED POLARIZER TEXT AND ITS EFFECT ON THE ENTROPY

The converged strings are interpretable, and independent seeds agree on what to say even though the reward never scores wording. The inference-time ψ ⋆ opens by directing doubt: “The passage may address a similar entity or context but not the specific one asked for, and it might contain outdated, fictional, or misattributed information. . . ”. The second seed converges on a paraphrase of the same instruction: “a related or similarly named topic but does not directly or correctly answer the questioned topic; it could be referencing a different entity, event, or timeframe. . . ”. Since the 20

Table 11: The polarizer across the 300-step budget, greedy-decoded from the five retained checkpoints of the deployed run, one box per checkpoint. The decode is still moving through the first three; steps 240 and 300 emit byte-identical strings, so the four boxes cover all five checkpoints, and the last of them is the ψ ⋆ every number in this paper is computed from. Step 60 — still moving

The passage may directly address a surface-level fact but could mislead if it refers to a related entity, a different time period, a variant form of the subject, or a fictionalized version rather than the actual entity. It might also contain outdated statistics, approximate numbers, or misattributed details that appear plausible but are not accurate for the specific query.

Step 120 — still moving

The passage may address a similar entity or context but not the specific one asked for, and it might contain outdated, approximate, or misattributed figures that do not accurately reflect the correct quantity or detail. It could also misrepresent the scope or boundaries of the subject, leading to a misleading answer.

Step 180 — still moving

The passage may address a similar entity or context but not the specific one asked for, and it might contain outdated, approximate, or misattributed figures that misrepresent the correct answer.

Steps 240 and 300 — byte-identical, and the deployed ψ ⋆

The passage may address a similar entity or context but not the specific one asked for, and it might contain outdated, fictional, or misattributed information that misrepresents the true answer.

reward is defined purely on H1 separation, this convergence is evidence that the objective has a stable linguistic optimum rather than that any phrasing was supplied. The opening clause is not, however, a reliable predictor of quality. The third seed instead opens by affirming: “The passage may directly address the surface-level query but could mislead by providing outdated, incomplete, or contextually adjacent information. . . ”. It reaches mean F1 0.5283, below both direct-doubt runs (0.5343 and 0.5390) yet above plain entropy selection’s 0.5148 on the same pools. All three runs win all five datasets against both entropy selection and rank1 (Appendix E). Affirm-then-qualify and direct-doubt framings are therefore both reachable optima of the same objective, and we report the wording as interpretable rather than as diagnostic. How the string is reached, and in what sense it is one string. Table 11 shows ψ ⋆ at five checkpoints spanning the 300-step budget of the deployed run. The wording is not stable early: across the ten checkpoints dumped every 30 steps the greedy decode takes seven distinct values, and the early strings are longer and carry extra qualifying clauses (375 characters at step 60 against 194 at step 300). The deployed string first appears at step 240 and is byte-identical at 270 and 300. Within a step, ψ ⋆ is the survivor of a group rather than a single output: πθ samples G = 8 polarizers at temperature 1.1 and GRPO ranks them against one another (Table 5). Those training rollouts were not written to disk, so the within-step spread during training cannot be recovered from this run. What can be measured is the converged policy re-sampled afterwards, and it is close to a point mass: drawing 8 samples from the final checkpoint at the training temperature returns eight byte21

Table 12: Trained generators beside L ODESTAR, in the format of Table 2. Both comparators train the answering model itself, so these are comparisons of inference class rather than of selection quality, and the two rows sit outside Table 2’s frozen-respondent ranking. The RESPONDENT column marks that model as frozen ( ) or fine-tuned (\), and the judge column is the five-pool GPT-4o macro of Appendix G’s protocol. Method

Reference

R ESPONDENT

Answer F1 ↑ (generated answer vs. gold answers) In-domain NQ

L ODESTAR (ours) Self-RAG CTRL-RAG

(Asai et al., 2024) (Tan et al., 2026)

\ \

Out-of-domain SQuAD

TriviaQA

EntityQ

WebQ

Mean

LLM-Judge (GPT-4o)

0.4783

0.5916

0.7163

0.4943

0.3908

0.5343

0.6420

0.4796 0.5492

0.2587 0.6318

0.4684 0.7198

0.2977 0.5817

0.3147 0.4775

0.3638 0.5920

0.5976 0.6925

identical strings, and widening to 512 draws at temperature 1.0 returns the deployed ψ ⋆ 502 times (98.0%), the ten that differ occurring once each and moving only inside the clause that lists what may be wrong with the passage. A control at temperature 2.0 returns six distinct strings from eight draws, so the sampler is live rather than silently greedy. This measures the policy after training rather than the group the optimizer ranked at step 300, and it speaks to how the reported string was obtained, not to whether the polarizer helps. What the string does to the entropy. Decomposing the shift H1 (q, p, ψ ⋆ ) − H1 (q, p) by passage class over the five pools, a zero-inference calculation on the cached probes, shows ψ ⋆ acting almost entirely on one side. Passages the respondent answers correctly from move by +0.007 on average and passages it fails on by +0.258, a difference-in-differences of +0.251. The effect concentrates where selection fails. The per-question lure, the wrong passage with the lowest plain entropy and therefore the one a plain-entropy selector actually picks, is pushed up by +0.377, against +0.106 for that question’s best supporting passage. These figures are a descriptive decomposition of cached probe values, not a causal account of what the reward taught the polarizer policy.

I

CTRL-RAG REPRODUCTION

CTRL-RAG (Tan et al., 2026) is a concurrent preprint that trains the generator for context faithfulness rather than a selector, so it is a paradigm relative rather than a same-task competitor. We reproduced it end-to-end and report it as a case study rather than as a row of Tables 2, 7 and 9 for one measurable reason: the run has no stable protocol-compliant checkpoint, so every candidate number would misrepresent it, and each would do so differently. The clean short-answer format survives to roughly 90–106 steps of the 300-step protocol; after that the raw output collapses and stays collapsed—macro F1 0.1344 at step 120, 0.1276 at step 300 (Table 13), every exact match lost at both—and the extraction rule we pre-registered against the earlier failure mode fires on only 9%–10% of questions against the next variant. Table 12 still prints both trained generators beside L ODESTAR, and each of its two rows needs its provenance stated. The CTRL-RAG cells are the strongest checkpoint of the run, RL step 80, so Table 13’s trajectory rather than that row is the reproduction’s full record, and its judge column carries the same five-pool macro, so step 80 reads 0.6925 in both tables. The Self-RAG cells are that method’s system-level row, the configuration its § disclosure attaches to in Table 9, where its per-pool exact match is printed. Self-RAG still selects one passage, as every ranked row does, while CTRL-RAG reads all ten in retrieval order and exposes no selection step at all. The strongest checkpoint, stated plainly. The reproduction beats L ODESTAR on every pool, its best checkpoint reaching mean F1 0.5920 against L ODESTAR’s 0.5343 (Table 13). That reference row is the seed-42 training run in every column, where Table 2 reports the three-seed mean, 0.5339 macro F1 and 0.4136 EM. Every row of the table is read under the raw protocol, on which the pre-registered extraction step changes step 20 by less than 10−4 , and the mid RL row is the last checkpoint whose sampled responses never hit the length cap. Against step 20, on the same 5,008 aligned questions, the paired difference runs against us, ∆F1 = −0.0533 [−0.0646, −0.0420] and −0.0665 [−0.0792, −0.0538] under EM. Two qualifications belong next to that number, and neither subtracts from it. The comparison is one of inference class rather than selection quality, since 22

this checkpoint is a gold-supervised, fully fine-tuned copy of the very respondent every ranked row keeps frozen and it reads all ten passages per query. What that class costs is the concrete form of the distinction. It requires gold answer supervision, per-domain fine-tuning, serving of the respondent itself and a ten-passage prefill per query, none of which is available in the frozenrespondent settings this paper targets, where L ODESTAR selects at one forward pass per candidate, needs no gold answers, and serves no model beyond the respondent that answers. And the margin is bought by supervised fine-tuning on in-benchmark gold answers, not by the reinforcement objective the method is about: the SFT anchor scores 0.5860 before a single RL step, three hundred steps of RL add at most +0.0060 over it, and then remove everything. We report the anchor row precisely so that this decomposition is visible rather than implied.

What the judge says about the collapse. At step 300 every string metric is destroyed—F1 0.1276 and EM exactly 0.0000—yet the GPT-4o judge still credits 0.5845 of answers as equivalent, against 0.6727–0.6925 for the three intact checkpoints. Roughly 85% of the judge score survives an F1 collapse to a fifth of its value, which is direct evidence that the collapse is a change of surface form and not a loss of content. Every figure in that column is a five-pool macro, so it reads both down the rows of Table 13 and against the macros quoted elsewhere, where L ODESTAR stands at 0.6435 over three seeds (Table 10) and at 0.6420 on the seed-42 run this table’s reference row reports. Figure 3 plots what the run was maximising while that surface form decayed. The reward rises over exactly the steps in which the sampled responses pin to the length cap, and CTRL-RAG’s objective scores no property of the output format. The surface form that the string metrics of Table 13 measure is therefore not a quantity this run was optimising; what it was optimising rewards length outright, as the next paragraph derives.

Mean reward (higher is better ↑) mean 0.1577 at clip ratio = 0.00

0.8

mean 0.3466 at clip ratio = 1.00

0.6 0.4 0.2 0 response length/clip ratio (lower is better ↓) early

mid

late

1 format intact: no response reaches the cap

0.75 0.5

reaches 1.00 at step 117 every sampled response truncated at the 32-token cap

leaves zero at step 86

0.25 0 0

50

100

150

200

250

300

RL step (the 300-step budget; step 0 is the SFT anchor)

Figure 3: CTRL-RAG reproduction, the RL stage measured two ways. Both panels are the SFT-anchored seed-42 run whose checkpoints Table 13 reports, read from its training log. Top: critic/score/mean, the quantity the objective maximises. Bottom: the fraction of sampled responses truncated at the 32-token cap. That cap is not binding on a well-formed answer: under the policy’s own tokenizer the 1,848 gold answer strings of the NQ pool average 3.7 tokens and the longest runs to 13, so truncation records a change of output format rather than an answer that did not fit. The reward rises as the format degrades, from mean 0.1577 over the 84 steps at which no response reaches the cap to mean 0.3466 over the 165 steps at which every response does. 23

Table 13: CTRL-RAG reproduction, stage by stage. The four CTRL-RAG rows are its SFT anchor at zero RL steps and three points early, mid and late in the 300-step budget, all read under the raw protocol, so a column is comparable down the rows. Those three points are fixed by the run rather than selected from it: 20 is the first checkpoint it wrote under its 20-step save interval, 80 is the last whose sampled responses never reach the length cap (Figure 3; the next saved checkpoint, 100, already truncates 36% of them), and 300 is the final step of the budget, the checkpoint Appendix B’s no-per-run-selection rule reports for every trained configuration. Judge is the GPT-4o answer-equivalence protocol of Table 2, macro-averaged over the same five pools as every other judge figure in the paper. Bold marks the best of the four CTRL-RAG rows, and L ODESTAR is a reference row that is the seed-42 training run in every column, where Table 2 reports the three-seed mean. Answer F1 ↑

EM ↑

Judge ↑

Stage

NQ

SQuAD

TriviaQA

EntityQ

WebQ

Mean

Mean

Mean

SFT anchor, 0 RL steps + RL, step 20 (early) + RL, step 80 (mid) + RL, step 300 (late)

0.5412 0.5402 0.5492 0.1156

0.6063 0.6113 0.6318 0.1619

0.7176 0.7184 0.7198 0.1439

0.5870 0.5858 0.5817 0.1043

0.4779 0.4819 0.4775 0.1121

0.5860 0.5875 0.5920 0.1276

0.4810 0.4813 0.4713 0.0000

0.6727 0.6779 0.6925 0.5845

0.4783

0.5916

0.7163

0.4943

0.3908

0.5343

0.4148

0.6420

L ODESTAR (ours) (

respondent)

Why the optimum is format-degenerate. The collapse is not an instability the run fell into; it is the objective being maximised. Written out, the reward optimised here is
√ ′ T, Rhybrid (y) = RCLR (y) · 1(y ∗ ⊆ y), RCLR (y) = E(y) 1 E(y) > τ PT E(y) = S(y | D) − min+ S(y | D \ {d}), S(y | ·) = t=1 log P (yt | y<t , q, ·), d∈D

(7) with T the answer’s token count and τ = 1. Two properties of that form push the same way on T . First, E(y) is a difference of two sums over the same T tokens, so it is extensive in length: each grows only √ √ additional token contributes one more per-token contrast, while the normaliser as T . Any positive average per-token contrast c therefore leaves RCLR ≈ c T , rising without bound in T —a sub-linear normaliser does not cancel an extensive numerator, it only slows it, and the threshold pushes the same way because a longer y clears E(y) > τ more easily. Second, the accuracy factor is a containment test, and containment is monotone under appending: once y ∗ occurs in y, no continuation can remove it. Padding is thus free on the factor that checks correctness and strictly profitable on the factor that does not, and the in-group min-max rescale is order-preserving, so in every group the longer surviving sibling carries the larger advantage. The joint optimum is to emit the gold string and then continue to the cap, which is exactly the endpoint Table 13 records: exact match 0.0000 while the judge still credits 0.5845, i.e. y ∗ is still there and everything after it is reward being collected. The analysis fixes the mechanism; Figure 3 fixes its sign, the reward rising over precisely the steps in which the responses lengthen. Note that this argument is a property of the reward’s shape alone, so the frozen-scorer deviation of Appendix J does not bear on it. Scoped conclusion. Our claim is bounded by our adaptation, which Appendix J discloses in full: under that adaptation, the contrastive-likelihood objective’s optimum is format-degenerate, the SFT anchor delays that optimum in proportion to its strength without preventing it, and pre-declared cleaning rules are evaded by the next variant. We explicitly do not claim this of CTRL-RAG at its published scale, where the official 74k SFT corpus, a batch of 1024 and a reward that reads likelihoods from the policy itself rather than from a frozen copy are all untested here. The length incentive we audited exists under any conditional likelihood, but its interaction with a larger anchor and a smaller gradient noise scale is an open question.

J

BASELINE CONFIGURATIONS AND DISCLOSURES

This appendix gives the per-row configuration behind the bare method names of Table 2, together with the disclosures about how those rows should be read. Every configuration follows the fairness contract of the Baselines paragraph (Section 4), and every method-by-dataset cell of the entropy, rank1 and oracle columns is identical across all runs, confirming that every method ran on 24

exactly the same items. The three reference lines are random, the expected F1 of a uniform draw; rank1, the retriever’s own top-ranked passage; and oracle, the best of the ten in hindsight. GainRAG, CRITIC-R1 and MBA-RAG are the only baselines that need a training artifact of their own, so they are the rows for which the paper’s own numbers cannot answer whether we ran the other method correctly or ran it badly in our own favour. We therefore re-ran all three from their stored artifacts on the WebQuestions pool, on different hardware from the original runs, and all three return their tabulated value exactly (0.3753, 0.3674 and 0.3491; ∆F1 = 0.0000 in each case). MBA-RAG’s re-run coincides with the rank1 line because its bandit selects rank 1 on all 1,000 questions of that pool, which is that policy’s collapse onto the retrieval prior reproducing rather than a failed re-run. J.1

T RAINING - FREE SELECTION SIGNALS

These rows score each candidate from the frozen respondent’s own response and select by that score, except for SPS, which scores the passage representation alone, and Self-RAG (reflection), which scores with the method’s officially released critic. None of them is trained for this paper. Each row’s configuration and the disclosure attaching to it are given below, one method per entry, in Table 2’s order. Multi-sample methods use a uniform k = 5 budget unless an official row is given. • Semantic entropy (Farquhar et al., 2024) runs in its official strict-entailment form. Its row is inflated by the table’s first-wins tie-break and should not be read as second place: under that setting it ties on 62–79% of questions with only ∼ 2 distinct scores per 10 candidates, so first-wins silently imports a retrieval-rank-1 prior. Breaking ties at random gives mean F1 0.4774, statistically indistinguishable from rank1’s 0.4769. Official code at https://github.com/jlko/semantic_uncertainty. • Self-RAG (reflection) (Asai et al., 2024) selects with the method’s officially released critic. We score it over the full vocabulary instead of the official top-5000 log-prob truncation, a deviation strictly favourable to it. Official code at https://github.com/AkariAs ai/self-rag. • SeaKR (Yao et al., 2025) runs at its official n = 20. Our row omits the official newline stop-strings, fixed seed and empty-sample filtering; restoring all three moves mean F1 by +0.0006, mixed in sign per dataset, and leaves every per-dataset comparison against L ODESTAR unchanged. Official code at https://github.com/THU-KEG/SeaKR. • CLeHe (Qiu et al., 2025) is our adaptation of that method’s entropy signal into a selector. The official method is a token-level soft fusion that never selects a single passage, so this row carries no inferential value about the published method. Official code at https: //github.com/zexuanqiu/entropy-based-decoding. • EPR (Moslonka et al., 2026) runs its official sequence-mean rate. Our row generates 24 tokens where the official script uses 200; a rerun at the official budget moves mean F1 by −0.0012, so the row we rank against is the marginally stronger configuration. Official code at https://github.com/artefactory/artefactual, release tag ECIR2026. • Min-K% Prob (Shi et al., 2024) runs at its official k = 20, adapted to selection by scoring each passage’s own cached greedy answer under the shared answering prompt. Official code at https://github.com/swj0419/detect-pretrain-code. • IGP (Song et al., 2026) runs under its published configuration, which probes with its own short prompt rather than the answering prompt. That misalignment, not the gain formulation, is the likely main cause of its position (Section 2); the aligned-probe ablation of Appendix C recovers about half the gap. • EigenScore (Chen et al., 2024a) runs the released code at k = 5 (K = 10, T = 0.5, max new tokens = 256) without Feature Clipping, which the official repository also omits, so it corresponds to that paper’s “EigenScore (w/o)” variant. The released code also follows top-k = 10 where the paper states 5, and pools at the EOS position. Official code at https://github.com/alibaba/eigenscore. • SPS (Hu et al., 2026) runs its released scoring function, which reads the candidate text and never the question. In SPS’s own protocol the candidates are query-conditioned rewrites of one evidence set, so nothing is lost by this; ours are ten different passages, and a score 25

blind to the question cannot prefer the one that answers it. Its low row is that mismatch, which we state before reporting it, and we keep the row beside the random reference line so its position is checkable rather than asserted: its correlation with per-candidate answer F1 is −0.005 to −0.082 on our pools, its mean F1 0.3380 sits at that line’s 0.3271 (within −0.5 to +2.4 paired z per dataset), the implementation was verified against the released code element by element (subspace basis, PCA rule, max-pooled penultimate representation, bare-sentence input, residual score, argmin), and the argmax control that would have exposed a sign error is worse on all five datasets. Official code at https://github.c om/HU-xiaobai/Code-of-Beyond-Perplexity-Let-the-Reader-Sel ect-Retrieval-Summaries-via-Spectrum-Projection-Score. • Min-K%++ (Zhang et al., 2025a) runs the identical transplant, differing only in its vocabulary-normalized per-token statistic. Its paper fixes no default k, so the run also sweeps k=10..100, and the best point (k=10, macro 0.2912) still sits below every other ranked row. Its below-random row is a property of the statistic under this transplant, not an implementation defect: our reimplementation matches the official release term for term, and an argmin control does not rescue the score. Official code at https: //github.com/zjysteven/mink-plus-plus. Every ranked row with tie logging other than semantic entropy has a tie rate ≤ 1.2% and reproduces its first-wins numbers under random tie-breaking. Plain first-token entropy, the polarizer ablation of L ODESTAR, is not ranked as a baseline row of Tables 2 and 9; being our own method minus one component, it is reported with the ablations (Section 6) and paired-tested in Table 7. At mean F1 0.5148 it is nonetheless the strongest untrained selector in the benchmark, behind only the searched GEPA string among inference-ready configurations, so what L ODESTAR improves on is a demanding comparison rather than a favourable one. J.2

S EARCHED GUIDANCE

The prompt-optimization rows ask whether L ODESTAR’s GRPO stage can be replaced by searching the polarizer directly. Both configurations consume L ODESTAR’s exact reward function, training draws and frozen respondent, search under the same 96-token polarizer cap, and select identically at inference (arg minp H1 (q, p, ψ)); only the optimizer differs. Both also start from the same handwritten caution seed, a manually written polarizer directing doubt at passages that look relevant but do not answer the exact question, and on the never-searched pools the searched strings improve on it by at most +0.003 mean F1 . That seed is itself evaluated under the identical protocol and reaches mean F1 0.5129 and EM 0.4005, below the no-polarizer ablation’s 0.5148: writing the polarizer by hand adds nothing over inserting no polarizer at all, so the ordering across the three ways of obtaining the string is hand-written 0.5129, searched 0.5131–0.5160, GRPO-learned 0.5339. Each optimizer’s configuration is given below, one per entry, in Table 2’s order. • GEPA (Agrawal et al., 2026) runs its official implementation, reflective evolution with Pareto candidate selection and gpt-oss-120b as the reflection LM, at two settings. At its official sample-efficiency setting, all six runs we launched—three seeds at each of two reflection-minibatch sizes—returned the hand-written seed unchanged, and not for want of proposals: at GEPA’s default minibatch of three, 23–25 of the reflection LM’s rewrites per run cleared the minibatch screen and were fully validated without ever displacing the seed on the Pareto frontier, and at a minibatch of ten no rewrite cleared the screen at all. The second setting runs each seed to saturation under plateau early-stopping; the table reports the strongest saturated seed. • MIPROv2 (Opsahl-Ong et al., 2024), a non-reflective Bayesian instruction search, runs under the same harness at the same sample-efficiency setting. Unlike GEPA there, every seed improved on the hand-written seed internally (+0.005 to +0.107 separation)—the same harness accepting improvements whenever the optimizer finds one, which rules out a harness fault behind GEPA’s unchanged returns—and the table reports the strongest seed. Each row therefore prints its strongest seed against L ODESTAR’s three-seed mean, a comparison deliberately set in the baselines’ favour. On the never-searched pools, however, both searched strings land within +0.003 mean F1 of the untuned hand-written caution—and significantly below L ODESTAR (Table 7). Section 5 reports the full comparison. 26

J.3

T RAINED SELECTORS

These three rows replace the selection signal with a model trained on our data, while the same frozen respondent still answers.

• GainRAG (Jiang et al., 2025) is the only baseline that, like L ODESTAR, trains on our data, so we control its training distribution exactly. It enters only as its NQ-trained inferenceready distilled selector: its per-candidate gain signal reads the gold answer and is therefore not a selector one can run at inference, so we report that signal at the end of this entry rather than as a table row. Following its official recipe, per-candidate gains are synthesised as contrastive-decoding perplexities of the gold answer (α=0.5) under the same frozen respondent and the same answering prompt that produced the cached utilities; a respondentgenerated pseudo-passage joins the ten bge-m3 candidates (11 per question); questions whose highest-gain candidate fails GainRAG’s lenient containment-EM filter are dropped; and bge-reranker-base is distillation-finetuned on the soft gain distribution (KL loss on − log(PPL+1) targets, group size 11, lr 6 × 10−5 , 2 epochs, official defaults otherwise, final checkpoint, no model selection). Crucially, its training questions are exactly the 1,939 NQ-open train questions L ODESTAR trains on (the filter keeps 1,883), so both methods see the same supervision distribution and share an in-domain pool. Two benchmark adaptations from the paper’s setting: candidates come from the shared bge-m3 top-10 pools rather than Contriever top-20, and reranker input length is capped at the model’s true 512-token limit rather than the released script’s 512+512. At test time the selector scores the same ten candidates as every other row, one 278M cross-encoder forward per candidate with no respondent and no gold; its full workflow with the pseudo-passage action enabled is not cell-comparable and is excluded from the tables, where it would reach mean F1 0.4907. The teacher signal is likewise kept out of the tables, but we report it because it bounds what the distillation could have delivered—selecting by a gain that scores the gold answer is selecting with the answer in hand. Reproduced on our pools it reaches mean F1 0.6213 over the four datasets where it is available, against L ODESTAR’s 0.4888 on those same four, while the distilled selector retains 0.4480 on those four (0.4951 over all five pools, its Table 2 row). That gap is distillation loss rather than a reproduction failure, and it is why GainRAG’s inference-ready row sits mid-table. • MBA-RAG (Tang et al., 2025) trains a DistilBERT bandit policy with an ϵ-greedy reward ra = A(y, ŷa ) − λC(a). Its published arms are retrieval strategies; our benchmark fixes retrieval, so the arm set is adapted to the ten candidate ranks of the shared pools, and since every arm then costs exactly one respondent call the cost term is degenerate and λ is set to 0. Rewards are the same cached per-passage F1 utilities that define our labels. Result: the trained policy collapses onto the retrieval prior, reaching mean F1 0.4765; the paired comparison against L ODESTAR is in Table 7. • CRITIC-R1 (Xiao et al., 2026) trains a structured critic (verdict/location/reason/fix schema) with two-stage GRPO under consensus teacher supervision. We adapt it to selection: the trained critic scores each candidate’s cached draft answer by its verdict-token probabilities (P (CORRECT) − P (INCORRECT), one forward per candidate), the bestscored passage is selected, and the same frozen respondent answers. Training follows the official recipe at the official scale: Qwen2.5-3B-Instruct critic with LoRA (Hu et al., 2022) r=16/α=16, lr 10−6 , KL 0.003, 5,000 critique samples at the official five-epoch exposure, two stages (CJA then gated DQA) with the paper’s conservative verdict matrix, the training triples drawn only from L ODESTAR’s train questions and consensus supervision (K=3) from a gpt-oss-120b teacher over the same cached draft answers. Result: the critic trains healthily by its own reward yet its selection ends at mean F1 0.4626, below rank1’s 0.4769; the paired comparison against L ODESTAR is in Table 7. An implementation audit accompanies the number, and the residual gap is the method rather than the harness: teacher-verdict alignment and within-question ranking are decoupled skills on this benchmark, and the two training stages reward the former. 27

J.4

T RAINED GENERATORS ( SYSTEM - LEVEL )

In these two rows the answering model itself is trained, so they are system-level comparisons rather than cell-level ones and are grouped separately in the tables. • Self-RAG (Asai et al., 2024) is the released selfrag-llama2-7b generator running its official reflection-token inference over the same ten candidates. Unlike every other row it replaces the frozen respondent with a trained one, a Llama-2-7B fine-tuned on SelfRAG’s own 150k-instance corpus, so its row answers the question “does a purpose-trained retrieval-augmented generator beat a frozen respondent steered by one learned string?” Result and metric diagnostic. Its reflection-based passage selection is sound (frozenrespondent F1 at its chosen passages matches plain entropy within ±0.015 on every pool), but its answers are style-mismatched with token-level F1 : on well-formed out-of-domain questions it answers in full sentences (mean 11.9 words vs. the frozen respondent’s 4.2), which token F1 punishes even when the fact is present. Its table row therefore uses its stronger configuration under our one-row policy, official inference plus a gold-agnostic extraction step, which recovers mean F1 from 0.3346 to 0.3638 (Table 12); L ODESTAR loses only the NQ cell to it, by 0.001, consistent with NQ-style data in its training corpus. Under Self-RAG’s own official metric, containment match, the comparison is a statistical tie, because containment rewards its verbose style; we report both metrics rather than choosing either. • CTRL-RAG (Tan et al., 2026) also trains the generator itself rather than a selector: fullparameter GRPO on the very Llama-3.1-8B-Instruct checkpoint we otherwise keep frozen, ′ · 1(y ∗ ⊆ y), optimizing the paper’s contrastive-likelihood hybrid reward Rhybrid = RCLR whose leave-one-out likelihood gap is scored online by a frozen-likelihood service held at the initial policy. We keep its two-stage SFT-then-RL structure but rebuild the SFT stage inside the benchmark, on 1,839 of the 1,939 NQ training questions L ODESTAR trains on, with the first gold answer as the target, in place of the official 74k multi-hop SFT corpus, which would inject supervision from outside this comparison. It reads the shared top10 concatenation and answers directly, with no selection step, so its cells are not frozenrespondent-comparable and are not claimed in the per-cell win count. It is kept out of Tables 2, 7 and 9, and appears only beside Self-RAG in Table 12; its scores, formatting pathology and the reason no single checkpoint of the run can stand in for the method are reported together in the reproduction case study of Appendix I.

28

Related documents

• Learning Evidence Highlighting for Frozen LLMs#134582
• When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents#1155307
• Beyond Mechanisms, Models, and Robustness. When Explanatory Success Entitles Persistence Claims#110756
• Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade#346542
• SPACE: Sparse Predictive Attractor via Counterfactual Eviction for Streaming Video Memory#1116923