Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models Zizhuo Lin1,†

Quanling Liu1 Jinsheng Quan1 Chao Zhang1 Xing Shi1 Jingtao Xu1 Zhihui Li2 Yawei Luo1

1 Zhejiang University

2 University of Science and Technology of China †

Email: [email protected]

Part1:three task-equivalent presentation modes

arXiv:2605.30251v1 [cs.CL] 28 May 2026

Abstract Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student’s behavior on its own trajectories with the teacher’s canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32% average relative improvement in RAWSHARDED performance over the original base model across math and five zero-shot out-ofdomain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns.

Yifan Zhu1

Figure 1: Part 1: Task-equivalent F ULL, C ONCAT, and R AW-S HARDED presentations. Part 2: Reduced selfanchored drift and improved canonical-context consistency.

succeed when a task is presented as a clean F ULL prompt can still fail when the same information is disclosed incrementally through a R AW-S HARDED conversation (Laban et al., 2025; He et al., 2024; Li et al., 2025b). Part 1 of Figure 1 provides a concrete example of the three task-equivalent presentation modes. The core difficulty is that a R AW-S HARDED history is not merely a longer prompt. It also contains the model’s own earlier replies, produced before the task evidence is complete. Those replies can introduce tentative answers, unsupported assumptions, or partial reasoning into the context. By the final turn, the model may rely on this self-generated text instead of fully re-grounding its answer in the completed user-provided shards. We call this failure mode self-anchored drift. A trivial solution is to repair the trajectory at inference time. Such methods can reflect on, revise, reset, or consolidate intermediate reasoning before the model commits to a final answer (Shinn et al., 2023; Madaan et al., 2023; Mohammad Khalid et al., 2025). Although they are often useful in deployment, they rely on an additional external

Introduction

Users rarely specify a task in one fully formed prompt: they often reveal constraints over several turns. In multi-turn interactions where task-related user information is provided gradually, a fundamental reliability criterion requires the model to yield the same outcomes as it would when all such information is given in a single prompt after the conversation ends. We call this requirement canonicalcontext consistency. Recent multi-turn evaluations show that this consistency is fragile: models that 1

control loop and thus modify the inference process itself. Another line of work addresses the problem through clarification and abstention: when a request is ambiguous or the information needed for answering is still missing, the model must decide whether to ask a clarifying question, defer its response, or abstain (Zhang et al., 2025; Zhang and Choi, 2025; Testoni and Fern’andez, 2024; Wu et al., 2025; Zhang et al., 2024a; Cheng et al., 2024). However, recent evidence suggests that current offthe-shelf LLMs still struggle to reliably identify ambiguous requests and ask appropriate clarifying questions in practice, while still introducing contamination from the model’s own prior responses (Zhang et al., 2024b; Li et al., 2025a; Luo et al., 2025). We argue that canonical-context consistency should be internalized as a core model capability: once the final turn provides sufficient user evidence, the model’s answer must be anchored to that evidence rather than deviating toward assumptions introduced by itself. To this end, we propose Canonical-Context On-Policy Distillation (CCOPD), a self-distillation objective that enforces canonical-context consistency under contaminated multi-turn histories. The same base model is adopted in two roles: a frozen teacher reads the clean canonical F ULL task, while the trainable student receives the task only through the gradually revealed R AW-S HARDED interaction. Once all taskrelevant evidence has been revealed through the interaction, the F ULL-conditioned teacher provides token-level supervision on the student’s own finalanswer prefixes. This same-prefix, on-policy supervision re-anchors the R AW-S HARDED-conditioned student, encouraging its answer distribution to move away from self-generated contamination and toward the canonical distribution grounded in the completed user evidence. In this sense, CCOPD does not rely on a stronger external teacher. It targets cases where the same base model can solve the task under the task-equivalent clean F ULL prompt, but may fail under the R AW-S HARDED history. CCOPD trains the model to preserve that F ULLcontext capability in the self-contaminated R AWS HARDED setting. Under the task-equivalent sharding setup introduced by Laban et al. (2025), CCOPD is trained only on sharded GSM8K-style math conversations. Strikingly, this math-only post-training signal transfers well beyond math. It improves R AW-S HARDED accuracy on math and yields zero-

shot gains across five non-math settings. Overall, CCOPD yields a 32% average relative improvement over the unmodified base model on R AWS HARDED performance, while largely preserving F ULL and C ONCAT performance. Our contributions are: • We formalize canonical-context consistency for task-equivalent histories and identify selfanchored drift as a concrete source of its failure. We introduce CCOPD, an on-policy same-prefix distillation objective that aligns R AW-S HARDED final-answer behavior with a frozen same-backbone F ULL teacher, without adding a stronger external teacher or an inference-time repair loop. • We show that a math-only CCOPD signal transfers beyond math: the same adapter improves R AW-S HARDED performance across all six task families, including five non-math zero-shot settings, while largely preserving F ULL and C ONCAT performance. These gains exceed the data-matched post-training baselines in our evaluation. • We support the proposed mechanism with a suite of ablations and diagnostics. These analyses show that the gains are tied to canonical F ULL-view alignment and reverse-KL sameprefix supervision; when the model’s own F ULL behavior is already strong, increasing teacher scale brings limited additional benefit. Matched pollution tests and evidencefocus probes further show stronger grounding in completed user evidence and reduced sensitivity to assistant-side anchors.

Related Work

Inference-time repair and control loops. Inference-time repair and self-correction methods address model errors by adding an explicit test-time control process around generation (Pan et al., 2024; Kamoi et al., 2024). Representative approaches revise generations through verbal reflection or self-feedback (Shinn et al., 2023; Madaan et al., 2023; Ferraz et al., 2024), check candidate answers with self-verification, external critics, or learned verifiers (Weng et al., 2023; Gou et al., 2024; Zhang et al., 2024c; Gao et al., 2023), or reset the context before regenerating (Mohammad Khalid et al., 2025). These methods can be effective when extra inference passes and 2

reliable feedback are available, but they optimize an augmented test-time system. Our setting is complementary: the model must answer from the completed transcript itself, where earlier partialinformation replies may remain as misleading context. CCOPD targets this robustness through training, rather than by adding a repair or reset loop at test time.

Problem Statement

3.1

Problem Formulation

Task-equivalent sharded presentations. For a task instance q, let c(q) denote the canonical F ULL prompt. Let h(q) denote a sharded presentation of the same task, where task information is released across user turns and interleaved with assistant replies before the final-answer request. We call h(q) task-equivalent to c(q) when the sharded presentation reveals the same task-relevant user evidence as the canonical prompt.

Clarification, abstention, and online interaction control. Another line of work studies how models should behave when a request is ambiguous or the information needed to answer is incomplete. Selective-prediction and ambiguity-aware generation methods study when models should withhold unreliable answers (Cole et al., 2023; Kim et al., 2024). Clarification work evaluates when to ask a question and what question to ask (Zhang et al., 2024b; Zhang and Choi, 2025). This literature asks whether to answer, ask, defer, or abstain under uncertainty. Our setting assumes that the missing evidence has already arrived, but the realized transcript may still contain assistant-side commitments made before it was complete. CCOPD targets how the model conditions on a completed but self-contaminated transcript.

Canonical Context Consistency. For taskequivalent presentations, a reliable model should preserve its final-answer distribution under the change of presentation: π(y | h(q)) ≈ π(y | c(q)).

(1)

We call this requirement canonical-context consistency. The canonical prompt is not assumed to be an oracle; it is a controlled presentation in which the same user evidence is available. To inspect where this invariance breaks down, we compare next-token predictions under the two task-equivalent presentations while holding the answer prefix fixed. For a shared answer prefix s, define Ψπ (q, s) = DKL (π(· | h(q), s) ∥ π(· | c(q), s)) . (2) The same model appears on both sides; only the preceding presentation changes. Hence Ψπ is a local diagnostic of presentation sensitivity. Large values mark prefixes where the sharded path induces continuation preferences that differ from those induced by the clean presentation.

On-policy distillation (OPD). On-policy distillation reduces the trajectory mismatch in autoregressive distillation by supervising the student on states reached by its own policy, rather than only on fixed teacher demonstrations (Gu et al., 2024; Agarwal et al., 2024). Many recent OPD variants strengthen the supervision signal by giving the teacher access to additional task information (Zhao et al., 2026a; Penaloza et al., 2026; Liu et al., 2026; Ye et al., 2026; Zhang et al., 2026), or by coupling distillation with auxiliary inference, feedback, or optimization mechanisms (Zhao et al., 2026b; Yang et al., 2026; Zhu et al., 2026). CCOPD uses OPD for a different purpose. The teacher and student are given the same underlying task evidence and share the same backbone, but they condition on different presentations of that evidence: a clean FULL for the teacher and a realized RAW-SHARDED transcript for the student. Thus, the teacher is presentation-privileged rather than information-privileged. The goal is not to import extra knowledge from a stronger or betterinformed supervisor, but to distill invariance across task-equivalent presentations.

Self-anchored drift. The discrepancy in Eq. (2) locates a shift but does not identify which part of the sharded presentation is responsible. In R AWS HARDED histories, the distinctive source of concern is earlier assistant text generated before the task evidence is complete. We write a final-turn R AW-S HARDED history as hR AW-S HARDED (q) = (u1 , a1 , . . . , uK−1 , aK−1 , uK ), (3) where ui are user shards and ai are non-final assistant replies. Each ai is generated before all task evidence is available, and may therefore contain unsupported guesses, provisional answers, or taskspecific commitments. After uK supplies the missing evidence, the user evidence in the history is 3

3.2

1.5

0.5 +0.00

0.0

15 10 5

wrong-anchor states

RAW history

WAIT replacement

(b) Neutral-placeholder replacement

Figure 2: Qwen3-8b-base model probes for selfanchored drift. (a) Masking process-reply commitment spans selectively improves gold-vs-anchor margins for wrong-anchor states. (b) Replacing process replies with neutral placeholders lowers predictive-state deviation from the canonical F ULL reference.

SAAR and span edit. For completed R AWS HARDED histories, we annotate completed userevidence spans Gusr and assistant commitment spans Gself . The Self-Anchor Attention Ratio (SAAR) summarizes final-answer attention over these spans:

answers from the realized R AW-S HARDED history, while a frozen copy of the same base model scores the student’s own answer prefixes under the canonical F ULL prompt. Training minimizes a finalanswer-masked reverse KL between these two nexttoken distributions.

(4)

4.1 Here Āℓ (G) averages layer-ℓ attention from answer tokens to group G; higher SAAR indicates denser attention to completed user evidence relative to earlier assistant commitments. Using the same spans, we also mask commitment spans and measure the change in the gold-vs-process-anchor margin. Figure 2a shows that this edit selectively helps states anchored to the model’s process-stage wrong answer.

Retained Pairs and Model Roles

For each task q, we form a final-turn pair (c, h), where c = c(q) and h is the retained sharded history just before the final answer. In the main setting, h = hR AW-S HARDED (q): all task-relevant user shards have been revealed, and the non-final assistant replies from the realized interaction remain in the context. We retain only histories that end with the final user turn after the user evidence is complete. The two contexts are used on separate paths. The student is trainable and conditions only on h; the canonical prompt c is never appended to the student input. The teacher is a frozen copy of the same base model and conditions only on c. Thus, CCOPD compares two presentations of the same user evidence while updating only the history-conditioned student. Appendix A.2 details the structural checks and leakage controls.

Neutral-placeholder contrast. We replace each process reply with a neutral placeholder while keeping the user shards and turn structure fixed. For prefix i, let Ciraw , Cineu , and Cifull be the realized, neutralized, and canonical F ULL contexts. r With pM i,r = pM (· | Ci ) for r ∈ {raw, neu} and full full qi = π0 (· | Ci ), define M full M full ∆M i = DKL (pi,raw ∥ qi ) − DKL (pi,neu ∥ qi ). (5) Positive ∆M means that process replies increase i canonical deviation; Figure 2b shows that neutralization reduces it.

63% lower mean deviation

gold states

(a) Span edit

We probe the unmodified base model to test whether non-final assistant replies still affect the final predictive state after all user evidence is present. We refer to these non-final replies as process replies.

Āℓ (Gusr ) + ϵ . Āℓ (Gself ) + ϵ

+0.86 78% move right

1.0

Probing self-anchored drift

SAARℓ = log

selective pull: +0.86 95% CI [0.65, 1.06]

Canonical deviation ψ

margin shift after masking

complete and task-equivalent to c(q). However, the earlier ai remain in the conditioning context. When these self-generated commitments pull the final-answer behavior away from the completed user evidence, canonical-context consistency fails. We call this failure mode self-anchored drift.

4.2

On-Policy Canonical Relabeling

Given a retained pair (c, h), the current student produces a final-answer rollout

Canonical-Context On-Policy Distillation

ŷ1:T ∼ πθ (· | h),

(6)

and Tans (ŷ) denotes the token positions belonging to that final assistant continuation. These prefixes are the on-policy states for distillation: they are

Section 3 defines a local presentation gap at a shared answer prefix. CCOPD turns this gap into an on-policy distillation signal: a trainable student 4

FULL Cononical Context:C C

Teacher model

𝐒𝟏

𝐒𝟐 𝐑𝟏

{ 𝐒𝟏 }

𝐒𝟑

…𝐒 𝐧-𝟏

{𝐒𝟏 , 𝐑 𝟏 , 𝐒𝟐 }

Student model

{ 𝐒𝟏 , 𝐑 𝟏 , 𝐒𝟐 , … 𝐑 n-1 , 𝐒n }

𝝅𝜽

cononical

…

scoreෞ 𝐲𝐓

ෞ𝟏 , … , 𝐲ෞ 𝐲 𝐭−𝟏

context

ෞ𝟏 𝐲

⋅ ℎ, 𝑦ෞ <𝑡

KL 𝒑𝒉𝜽,𝒕 ∥ 𝒑𝒄𝟎,𝒕

Teacher (with C )

Student (with h ) ℎ 𝑝𝜃,𝑡 = 𝜋𝜃

FULL Task Evidence Complete

𝐒𝟏 :Shard from

score𝐲ෝ𝐭

At time step t(example)

𝒏

𝐲ෞ <𝐭 = Same profix

…𝐑 …

𝐑𝟐

…

scoreෞ 𝐲𝟏 scoreෞ 𝐲𝟐

h Raw Multi-turn Interaction Process：h

𝝅𝟎

𝑐 𝑝0,𝑡 = 𝜋0

ෞ𝟐 𝐲

⋅ 𝑐, 𝑦ෞ <𝑡

…

average over 𝒕 ∈ 𝑻ans

𝐲ෝ𝐭

…

ෞ𝐓 𝐲

Student final answer reply

𝐑 𝟏 :Reply from model

UPDATE

Figure 3: Overview method of CCOPD. CCOPD aligns a R AW-S HARDED-conditioned student with a frozen F ULL-conditioned teacher using answer-masked same-prefix reverse KL.

reached after the student conditions on the realized transcript, including any earlier assistant-side commitments contained in h. For each t ∈ Tans (ŷ), the student and teacher score the same prefix ŷ<t under different contexts: phθ,t = πθ (· | h, ŷ<t ),

and the ideal objective is Lccopd (θ) = E(c,h)∼Dpair Eŷ∼πθ (·|h) [ℓccopd (q, ŷ)] . (10) The finite training loss estimates Equation (10) by averaging Equation (9) over minibatches. Normalizing by |Tans (ŷ)| prevents long answers from dominating the objective. The final-answer mask is part of the objective, not an implementation detail. Intermediate assistant turns in a R AW-S HARDED conversation are produced before all user evidence is available; forcing those turns to match a F ULL-conditioned teacher would leak future evidence and penalize reasonable partial-information behavior. CCOPD therefore optimizes canonical alignment only after the final user turn, where the history and the canonical prompt are task-equivalent.

pc0,t = π0 (· | c, ŷ<t ). (7)

The teacher is not asked to decode a clean completion. It only provides next-token probabilities on the prefix that the student has already visited. Thus canonical relabeling changes the conditioning context of the scorer, not the answer prefix being scored. As in on-policy distillation, the rollout operation in Equation (6) selects the states but is treated as stop-gradient. Gradients flow through the nexttoken student distribution evaluated at the selected prefixes. 4.3

Sequence-level view. Let Pθh be the distribution over complete final answers induced by the student under h, and let P0c be the analogous distribution induced by the frozen base model under c. Under an ideal stochastic formulation, the autoregressive chain rule decomposes DKL (Pθh ∥P0c ) into the expected sum of the same-prefix token KL terms along student-sampled trajectories. Equation (10) can therefore be read as a practical, answer-masked estimator of answer-level alignment to the canonical full-context behavior. Appendix A states the

Answer-Masked Reverse-KL Objective

Substituting Equation (7) into the canonicaldeviation potential gives the average deviation along the student rollout: X 1 Ψ̄θ (q, ŷ) = DKL phθ,t pc0,t . |Tans (ŷ)| t∈Tans (ŷ)

(8) The implemented per-rollout loss is this quantity, ℓccopd (q, ŷ) = Ψ̄θ (q, ŷ),

(9) 5

Compared systems. Base is the unmodified backbone and defines deltas. For data-matched training comparisons, SFT uses the same retained final-turn histories h and the same final-answer mask as CCOPD, but replaces distributional alignment with supervised final-answer targets; GRPO uses the same math source and final-answer reward/evaluation protocol. OPSD is a related onpolicy distillation comparison with privileged supervision. Forward-CCOPD is token-level control that keep the retained histories, final-answer positions, and student prefixes fixed, while changing only the KL direction or teacher conditioning context. Teacher-source rows replace the samebackbone F ULL teacher with smaller or larger teachers.

exact bridge and its assumptions. Figure 3 shows training step. The important asymmetry is that the student path and teacher path receive different contexts but score the same prefix. This yields a signal that is both cross-context and on-policy.

Experiments and Analysis

We evaluate whether CCOPD improves canonicalcontext consistency after the user evidence is complete. The target setting is R AW-S HARDED; F ULL measures clean one-shot capability, and C ONCAT checks whether the released shards preserve the task information. 5.1

Experimental Setup

Models. Our primary experiments use Qwen38B with a LoRA adapter trained by the reverseKL CCOPD objective in Equation (10) (An et al., 2025; Hu et al., 2022). For the main Qwen3-8B run, we optimize the LoRA adapter with AdamW using a learning rate of 3 × 10−5 ; full training details are reported in Table 10. To assess scale and family robustness, we additionally train adapters for Qwen3-4B and Llama3.1-8B (Grattafiori et al., 2024) in the scaling analysis.

5.2

Main Results

Table 1 summarizes the main results; Table 4 gives per-domain details. RAW improves without clean-task drift. On Qwen3-8B, CCOPD raises math R AW-S HARDED accuracy from 66.0 to 82.5 (+16.5 points), while math F ULL remains unchanged at 90.3. CCOPD also outperforms SFT and GRPO on math R AWS HARDED, suggesting that the observed R AWS HARDED gains are not recovered by standard posttraining objectives alone.

Training data and pair construction. Training is restricted to math. We construct sharded conversations from GSM8K and GSM8K-Aug (Cobbe et al., 2021; Deng et al., 2024). For each base model, we generate on-policy R AW-S HARDED interactions from the same shard specification.The primary Qwen3-8B adapter is trained on 6k retained pairs after screening.

Math-only training transfers out of domain. The same math-trained adapter improves the nonmath R AW-S HARDED average from 36.8 to 49.7 (+12.9 points). Structured OOD R AW-S HARDED rises from 48.7 to 67.9 (+19.2), and generation OOD R AW-S HARDED rises from 19.4 to 22.9 (+3.5). Because training uses only math conversations, this pattern is more consistent with transferable history grounding than with a math-format heuristic. Qwen3-4B and Llama3.1-8B show the same aggregate R AW-S HARDED direction. We further train CCOPD on non-math HotpotQA(Yang et al., 2018) sharded histories and still observe an improvement on math R AW-S HARDED , suggesting that the signal is not tied to math-format supervision alone; details are in Appendix E.

Evaluation protocol. We follow the released sharded-instruction protocol from Lost in Conversation (Laban et al., 2025). The evaluation suite covers six task families: math word problems from GSM8K-style examples (Cobbe et al., 2021), code generation from HumanEval and LiveCodeBench-derived tests (Chen et al., 2021; Jain et al., 2024), function calling from BFCL-style tasks (Patil et al., 2025), text-to-SQL from Spider (Yu et al., 2018), table-to-text generation from ToTTo (Parikh et al., 2020), and long-context summarization from SummHay (Laban et al., 2024). The non-math tasks are zero-shot transfer settings for the math-trained adapter. We report the mean over 10 independent end-to-end sharded runs with newly sampled on-policy conversations. details are given in Appendix B.

5.3

Ablations and Mechanism Diagnostics

Teacher source. Teacher-source ablations in Table 1 suggest that scale is not the main bottleneck. A smaller 4B F ULL teacher still improves the weighted R AW-S HARDED aggregate over the base model, while a larger 14B teacher improves 6

Model / variant

Math F/C/R

Structured OOD F/C/R Generation OOD F/C/R

All F/C

OOD-R

All-R

Model-size and model-family transfer Qwen3-14B Base 96.1 / 92.9 / 71.8 Qwen3-4B Base 89.3 / 84.5 / 55.3 Qwen3-4B CCOPD 89.3 / 83.5 / 56.9 (+1.6) Llama3.1-8B Base 71.8 / 70.9 / 46.6 Llama3.1-8B CCOPD 74.8 / 73.8 / 53.4 (+6.8)

92.1 / 83.2 / 52.6 80.1 / 74.0 / 41.0 83.7 / 76.0 / 63.8 67.6 / 63.7 / 37.5 67.3 / 64.1 / 52.7

34.1 / 31.9 / 20.3 22.4 / 22.6 / 11.5 21.0 / 18.4 / 17.2 13.0 / 16.3 / 13.1 14.8 / 13.8 / 13.4

70.3 39.5 44.8 60.2 29.1 33.4 60.6 (+0.3) 44.9 (+15.9) 46.9 (+13.5) 49.3 27.6 30.8 49.7 (+0.4) 36.8 (+9.1) 39.5 (+8.8)

Qwen3-8B: objectives and teacher sources Qwen3-8B Forward-CCOPD 93.2 / 91.3 / 79.0 (+13.0) Qwen3-8B 4B-teacher CCOPD 90.3 / 88.4 / 68.0 (+1.9) Qwen3-8B 14B-teacher CCOPD 92.2 / 86.4 / 69.9 (+3.9)

84.6 / 76.0 / 65.4 85.6 / 78.8 / 67.6 84.6 / 78.2 / 66.3

27.0 / 23.3 / 20.4 26.4 / 23.2 / 21.1 27.8 / 23.9 / 20.6

63.6 (-0.8) 47.2 (+10.4) 52.4 (+10.8) 64.0 (-0.4) 48.8 (+12.0) 51.9 (+10.3) 63.9 (-0.5) 47.8 (+11.0) 51.4 (+9.8)

Qwen3-8B: complete baselines Qwen3-8B Base Qwen3-8B SFT Qwen3-8B OPSD Qwen3-8B GRPO Qwen3-8B CCOPD

85.3 / 78.8 / 48.7 86.9 / 76.9 / 64.4 87.8 / 77.9 / 54.8 86.9 / 84.0 / 45.8 86.2 / 77.9 / 67.9

28.4 / 24.7 / 19.4 27.4 / 22.4 / 19.0 27.7 / 23.8 / 17.7 28.4 / 24.9 / 19.2 27.9 / 23.3 / 22.9

64.4 36.8 41.6 64.3 (-0.1) 46.0 (+9.2) 49.3 (+7.7) 64.0 (-0.4) 39.8 (+3.0) 42.5 (+0.9) 66.4 (+2.0) 35.0 (-1.8) 41.6 (-0.1) 64.2 (-0.2) 49.7 (+12.9) 55.1 (+13.5)

90.3 / 87.4 / 66.0 93.2 / 91.3 / 66.0 (+0.0) 86.0 / 85.0 / 56.3 (-9.7) 91.3 / 90.3 / 74.8 (+8.8) 90.3 / 88.4 / 82.5 (+16.5)

Table 1: Master results with weighted aggregate scores. F/C/R denotes F ULL/C ONCAT/R AW-S HARDED. Structured OOD averages code, function calling, and text-to-SQL; Generation OOD averages table-to-text and summarization. All F/C is a single clean-presentation score obtained by averaging the weighted F ULL and weighted C ONCAT scores across all six task families. OOD-R averages non-math R AW-S HARDED scores; All-R averages all six task families. Parenthesized values indicate changes against the matching base row under the same weighted scope; in Math F/C/R, only the R component carries the parenthesized RAW delta. 5

Base CCOPD

4 CCOPD

1.8 1.6 1.4

Δψi = ψiRAW − ψiNEU

D(user evidence)

log D(assistant history)

Base: m = 0.54 [0.44, 0.66] CCOPD: m = −0.09 [−0.19, − 0.02]

SAARℓ

Base

2.0

Slope m (95% CI)

3 2 1

1.2

−10

1.0

post-anchor RAW turns

Final

0 0

Layer

RAW response round

(a) Turn-level evidence focus

(b) Final-state SAAR

Canonical deviation ψi

Figure 4: Mechanism diagnostics for self-anchored drift. (a) CCOPD maintains stronger user-evidence focus across R AW-S HARDED response rounds after process replies enter the context. (b–c) At matched final-answer prefixes, CCOPD increases SAAR and flattens assistant-associated canonical deviation, reducing the neutral-replacement slope from 0.537 to −0.087.

Condition

and same-prefix scoring fixed, but replaces reverse KL with forward KL. This control tests both whether fine-grained same-prefix canonical supervision helps over the base model and whether the KL direction matters beyond that supervision. Forward-CCOPD improves substantially over the base, but remains below reverse-KL CCOPD. We attribute the gap to the objectives: forward KL encourages the R AW-S HARDED student to cover the F ULL teacher’s likely continuations, whereas reverse KL penalizes off-canonical probability mass assigned by the deployed R AW-S HARDED policy. Because self-anchored drift appears in the policy used at inference time, reverse KL better matches the intended intervention.

Base CCOPD

Clean F ULL problem 90.3 + Assistant-side wrong solution 33.0 + User-side wrong answer hint 63.1

90.3 89.3 88.3

Table 2: Full-context pollution stress test on math.

over the base but remains below the same-backbone teacher. This pattern supports the intended role of the teacher in CCOPD: it need not supply new task knowledge, but should provide a compatible canonical presentation of the same evidence. Crossmodel mismatch may partly limit external teachers, especially at larger scale. KL direction. Forward-CCOPD keeps the student rollouts, F ULL teacher, final-answer mask, 7

History-stress diagnostics. We test grounding under matched polluted histories. Starting from full math problems, we insert the same per-example wrong numeric anchor either as an assistant-side prior solution or as a user-side answer hint; within each condition, both models receive the same evaluated history. Table 2 shows that the two models match on the clean F ULL condition, but the base model collapses under pollution (90.3 → 33.0/63.1), whereas CCOPD remains near clean performance (90.3 → 89.3/88.3). This suggests reduced sensitivity to misleading anchors when complete user evidence is available.

Model

Default

+RTA

+DUC

Base CCOPD

66.0 82.5

72.8 79.6

67.0 77.7

Table 3: Accuracy on math R AW-S HARDED under two lightweight test-time modes.

Conclusion

This paper reframes R AW-S HARDED multi-turn failure as self-anchored canonical-context drift. The final history may contain all user evidence, but earlier assistant assumptions can still pull the model away from the answer distribution it would use under a clean F ULL prompt. CCOPD addresses this failure by distilling R AW-S HARDED final-answer states toward a frozen F ULL teacher on the same generated prefix. The method adds no inferencetime component, improves on-policy sharded accuracy after math-only training, and transfers zeroshot to five non-math task types. Attention redistribution and WAIT replacement diagnostics suggest that the trained model becomes less sensitive to self-generated assumptions and more aligned with user evidence at the final answer.

Evidence focus across process and final-answer states. We separate trajectory-level evidence focus from final-state anchoring in Figure 4. At the R AW-S HARDED trajectory level, Figure 4a tracks attention allocation across response rounds: after process replies enter the history, CCOPD maintains higher relative attention to task-relevant user evidence than to process history. At completed final-answer states, Figure 4b–c apply the sameprefix diagnostics from Section 3: SAAR compares completed user-evidence spans with processreply commitment spans, and neutral replacement measures assistant-associated canonical deviation under fixed user shards. CCOPD raises layer SAAR and flattens the neutral-replacement lowerenvelope slope from 0.537 to −0.087. Together, these probes are consistent with reduced selfanchored drift: the trained model attends more to completed user evidence and shows less assistantassociated deviation at the final answer.

Limitations Our experiments are still moderate in scale. The primary trained model is an 8B open-weight backbone with a LoRA adapter, trained on thousands rather than millions of retained canonical/history pairs. While this scale is sufficient to test the proposed mechanism, it does not establish how CCOPD behaves for substantially larger models, closed-source systems, or broader post-training corpora. Our RAW-SHARDED setting is also a controlled approximation of multi-turn interaction. It is constructed by splitting complete benchmark tasks into shards, which enables a clean task-equivalent comparison against FULL and CONCAT prompts. However, real conversations may contain evolving intents, clarification requests, irrelevant side information, user corrections, and failures that cannot be reduced to deterministic shards of a complete prompt. Future work should collect organic multiturn failures, construct human-verified canonical contexts that make those tasks answerable, and use such paired data to study self-anchored drift in more natural settings.Finally, CCOPD depends on a useful FULL-conditioned reference and only optimizes the final-answer state after all user evi-

Lightweight test-time modes. We additionally evaluate two lightweight test-time modes: reset-then-answer (RTA) and defer-until-complete (DUC). As shown in Table 3, direct CCOPD is the strongest setting on math R AW-S HARDED. RTA helps the base model, whereas DUC has little effect. Applying the same modes to CCOPD slightly lowers accuracy, although both variants remain above the corresponding base+mode rows. This pattern is consistent with the possibility that CCOPD has already internalized part of the task-state management that these prompts impose explicitly. Implementation details are provided in Appendix H.

dence is available. It can improve consistency with the model’s clean full-context behavior, but it cannot correct errors made by the FULL teacher itself, nor does it directly train earlier-turn behaviors such as clarification, abstention, or avoiding premature commitments under incomplete information.

Subramanian, Tagyoung Chung, Mohit Bansal, and Nanyun Peng. 2024. LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7773–7812, Miami, Florida, USA. Association for Computational Linguistics. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16477–16508, Toronto, Canada. Association for Computational Linguistics.

References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations, volume 2024, pages 21246–21263. Yang An, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2024. Critic: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Technical report.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. Evaluating large language models trained on code. Preprint, arXiv:2107.03374.

Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations.

Qinyuan Cheng, Tianxiang Sun, Xiangyang Liu, Wenwei Zhang, Zhangyue Yin, Shimin Li, Linyang Li, Zhengfu He, Kai Chen, and Xipeng Qiu. 2024. Can AI assistants know what they don’t know? In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 8184–8202. PMLR.

Yun He, Di Jin, Chaoqi Wang, Chloe Bi, Karishma Mandyam, Hejia Zhang, Chen Zhu, Ning Li, Tengyu Xu, Hongjiang Lv, Shruti Bhosale, Chenguang Zhu, Karthik Abinav Sankararaman, Eryk Helenowski, Melanie Kambadur, Aditya Tayade, Hao Ma, Han Fang, and Sinong Wang. 2024. Multi-IF: Benchmarking LLMs on multi-turn and multilingual instructions following. Preprint, arXiv:2410.15553.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.

Jeremy Cole, Michael Zhang, Daniel Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. 2023. Selectively answering ambiguous questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 530–543, Singapore. Association for Computational Linguistics.

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando SolarLezama, Koushik Sen, and Ion Stoica. 2024. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. Preprint, arXiv:2403.07974.

Yuntian Deng, Yejin Choi, and Stuart Shieber. 2024. From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:2405.14838.

Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. 2024. When can LLMs actually correct their own mistakes? a critical survey of selfcorrection of LLMs. Transactions of the Association for Computational Linguistics, 12:1417–1440.

Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek

Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sanggoo Lee, and Taeuk Kim. 2024. Aligning language models to explicitly handle ambiguity. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1989–2007, Miami, Florida, USA. Association for Computational Linguistics.

strategies. Transactions of the Association for Computational Linguistics, 12:484–506. Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A controlled table-to-text generation dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1173–1186, Online. Association for Computational Linguistics.

Philippe Laban, Alexander Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024. Summary of a haystack: A challenge to long-context LLMs and RAG systems. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9885–9903, Miami, Florida, USA. Association for Computational Linguistics.

Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning.

Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs get lost in multi-turn conversation. CoRR, abs/2505.06120.

Emiliano Penaloza, Dheeraj Vattikonda, Nicolas Gontier, Alexandre Lacoste, Laurent Charlin, and Massimo Caccia. 2026. Privileged information distillation for language models. arXiv preprint arXiv:2602.04942.

Belinda Z. Li, Been Kim, and Zi Wang. 2025a. Questbench: Can LLMs ask the right question to acquire information in reasoning tasks? Preprint, arXiv:2503.22674.

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36.

Jinnan Li, Jinzhe Li, Yue Wang, Yi Chang, and Yuan Wu. 2025b. StructFlowBench: A structured flow benchmark for multi-turn instruction following. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9322–9341, Vienna, Austria. Association for Computational Linguistics.

Alberto Testoni and Raquel Fern’andez. 2024. Asking the right question at the right time: Human and model uncertainty guidance to ask clarification questions. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 258–275. Association for Computational Linguistics.

Yihong Liu, Raoyuan Zhao, Michael A Hedderich, and Hinrich Schütze. 2026. Crosslingual on-policy selfdistillation for multilingual reasoning. arXiv preprint arXiv:2605.09548.

Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore. Association for Computational Linguistics.

Sichun Luo, Yi Huang, Mukai Li, Shichang Meng, Fengyuan Liu, Zefa Hu, Junlan Feng, and Qi Liu. 2025. Clarifymt-bench: Benchmarking and improving multi-turn clarification for conversational large language models. Preprint, arXiv:2512.21120. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36.

Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, and Jianfeng Gao. 2025. CollabLLM: From passive responders to active collaborators. In Proceedings of the 42nd International Conference on Machine Learning. Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. 2026. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation. arXiv preprint arXiv:2602.12125.

Haziq Mohammad Khalid, Athikash Jeyaganthan, Timothy Do, Yicheng Fu, Vasu Sharma, Sean O’Brien, and Kevin Zhu. 2025. ERGO: Entropy-guided resetting for generation optimization in multi-turn language models. In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025), pages 273–286, Suzhou, China. Association for Computational Linguistics.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380, Brussels, Belgium. Association for Computational Linguistics.

Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. Automatically correcting large language models: Surveying the landscape of diverse automated correction

Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. 2026. On-policy context distillation for language models. Preprint, arXiv:2602.12275.

Zhengyang Zhao, Lu Ma, and Wentao Zhang. 2026b. Training with harnesses: On-policy harness selfdistillation for complex reasoning. arXiv preprint arXiv:2605.08741.

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir R. Radev. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3911–3921, Brussels, Belgium. Association for Computational Linguistics.

Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu. 2026. The many faces of on-policy distillation: Pitfalls, mechanisms, and fixes. arXiv preprint arXiv:2605.11182.

Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. 2024a. R-tuning: Instructing large language models to say ‘i don’t know’. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7113–7139, Mexico City, Mexico. Association for Computational Linguistics.

Proof Details and Implementation Notes for CCOPD

A.1

Answer-Event Control

Fix a task instance q. Write h = h(q) for the completed multi-turn history and c = c(q) for the canonical F ULL prompt. Let Pθh be the distribution over complete final-answer strings induced by πθ (· | h), and let P0c be the corresponding distribution induced by π0 (· | c). We include EOS in the answer vocabulary and map any sequence that reaches the generation budget before EOS to a distinguished truncation symbol. Under this convention, both distributions are defined over the same finite terminal-answer space A. We also assume Pθh ≪ P0c on A, so the KL terms below are well defined.

Michael JQ Zhang and Eunsol Choi. 2025. Clarify when necessary: Resolving ambiguity through interaction with LMs. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5541–5558, Albuquerque, New Mexico. Association for Computational Linguistics. Michael JQ Zhang, W. Bradley Knox, and Eunsol Choi. 2025. Modeling future conversation turns to teach LLMs to ask clarifying questions. In International Conference on Learning Representations.

Lemma 1 (On-policy chain rule). For y ∼ Pθh with terminal length τ (y),

Tong Zhang, Peixin Qin, Yang Deng, Chen Huang, Wenqiang Lei, Junhong Liu, Dingnan Jin, Hongru Liang, and Tat-Seng Chua. 2024b. CLAMBER: A benchmark of identifying and clarifying ambiguous information needs in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10746–10766, Bangkok, Thailand. Association for Computational Linguistics.

  τ (y) X DKL (Pθh ∥P0c ) = Ey∼P h  dt (y) , θ

(11)

t=1

where dt (y) = DKL (πθ (· | h, y<t )∥π0 (· | c, y<t )) . (12)

Xinsen Zhang, Zhenkai Ding, Tianjun Pan, Run Yang, Chun Kang, Xue Xiong, and Jingnan Gu. 2026. Opsdl: On-policy self-distillation for long-context language models. arXiv preprint arXiv:2604.17535.

Proof. For any terminal answer string y = (y1 , . . . , yτ (y) ) ∈ A, the two autoregressive distributions factor as

Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. 2024c. Small language models need strong verifiers to self-correct reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15637–15653, Bangkok, Thailand. Association for Computational Linguistics.

τ (y)

Pθh (y) =

πθ (yt | h, y<t ),

(13)

π0 (yt | c, y<t ).

(14)

t=1 τ (y)

P0c (y) =

Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. 2026a. Self-distilled reasoner: On-policy self-distillation for large language models. Preprint, arXiv:2601.18734.

Y t=1

Substituting these factorizations into sequence11

Model

Variant

Code F/C/R

Model-size and model-family transfer Qwen3-14B Base 89.0 / 72.0 / 47.0 Qwen3-4B Base 64.0 / 50.0 / 26.0 Qwen3-4B CCOPD 74.0 / 53.0 / 38.0 Llama3.1-8B Base 39.0 / 32.0 / 17.0 Llama3.1-8B CCOPD 38.0 / 34.0 / 28.0

Function F/C/R Text-to-SQL F/C/R Table-to-text F/C/R Summary F/C/R 98.1 / 99.1 / 55.2 94.3 / 99.1 / 46.7 95.2 / 99.1 / 81.9 86.7 / 88.3 / 56.2 86.7 / 86.7 / 85.1

89.0 / 78.0 / 55.1 81.3 / 72.0 / 49.5 81.3 / 74.8 / 70.1 75.7 / 69.2 / 38.3 75.7 / 70.1 / 43.9

45.1 / 43.2 / 28.4 28.6 / 31.8 / 16.0 27.0 / 25.1 / 22.4 15.2 / 21.9 / 17.1 16.6 / 19.4 / 17.3

19.8 / 17.2 / 9.6 14.3 / 10.7 / 5.6 13.2 / 9.8 / 10.4 10.2 / 9.0 / 8.1 12.5 / 6.5 / 8.3

Qwen3-8B: objective and teacher-source ablations Qwen3-8B Forward-CCOPD 74.0 / 56.0 / 46.0 96.2 / 98.1 / 79.1 Qwen3-8B 4B teacher 75.0 / 62.0 / 46.0 97.1 / 98.1 / 80.0 Qwen3-8B 14B teacher 70.0 / 59.0 / 46.0 98.1 / 99.1 / 84.8

83.2 / 72.9 / 70.1 84.1 / 75.7 / 75.7 85.1 / 75.7 / 67.3

36.4 / 33.0 / 27.6 34.7 / 32.4 / 29.1 37.8 / 34.4 / 27.1

14.7 / 10.5 / 11.1 15.6 / 11.1 / 10.7 14.8 / 10.3 / 12.1

Qwen3-8B: complete evaluation and baselines Qwen3-8B Base 70.0 / 62.0 / 33.0 Qwen3-8B SFT 75.0 / 60.0 / 39.0 Qwen3-8B OPSD 77.0 / 55.0 / 35.0 Qwen3-8B GRPO 75.0 / 75.0 / 33.0 Qwen3-8B CCOPD 75.0 / 60.0 / 49.0

87.9 / 74.8 / 54.2 86.9 / 72.0 / 68.2 87.9 / 78.5 / 60.8 87.9 / 77.6 / 49.5 85.1 / 73.8 / 70.1

38.3 / 35.6 / 23.8 36.0 / 31.4 / 23.7 38.7 / 36.1 / 24.1 38.0 / 35.6 / 24.1 37.3 / 32.9 / 29.4

15.5 / 10.4 / 13.6 16.0 / 10.6 / 12.8 13.2 / 7.7 / 9.4 15.9 / 10.9 / 12.8 15.6 / 10.7 / 14.5

97.1 / 99.1 / 58.1 98.1 / 98.1 / 84.8 98.1 / 99.1 / 67.6 97.1 / 99.1 / 54.3 98.1 / 99.1 / 83.8

Table 4: OOD task details. Each cell reports F ULL/C ONCAT/R AW-S HARDED (F/C/R).

For any event E, absolute probability difference is bounded by total variation:

level KL gives DKL (Pθh ∥P0c ) =

Pθh (y) log

Pθh (y) P0c (y)

Pθh (E) − P0c (E) ≤ TV(Pθh , P0c ).

(19)   τ (y) X πθ (yt | h, y<t )  Combining the two inequalities proves Equa = Ey∼P h . log tion (17). θ π0 (yt | c, y<t ) y∈A

t=1

(15) Conditioning on a realized prefix y<t under Pθh , the next token yt is distributed as πθ (· | h, y<t ). Therefore, πθ (yt | h, y<t ) E log y<t π0 (yt | c, y<t ) = DKL (πθ (· | h, y<t )∥π0 (· | c, y<t )) . (16) Applying Equation (16) inside Equation (15) proves Equation (11).

Relation to the implemented objective. The chain-rule argument above describes an ideal stochastic objective over complete final-answer strings. The implemented objective in Equation (10) is a practical surrogate: it uses a finite number of student rollouts, applies the loss only to final-answer positions Tans (ŷ), normalizes by answer length.

By the definition of canonical deviation in the main text, dt (y) = Ψθ (q, y<t ). Thus, the tokenlevel KL minimized by CCOPD is the on-policy decomposition of the sequence-level divergence between the history-conditioned student and the canonical F ULL teacher.

The pair construction enforces task equivalence at the final answer state. The canonical side contains c(q) = FULL(q). The student side contains the retained multi-turn history ending at the final user turn. A R AW-S HARDED example is kept only if metadata verifies that all user shards have been revealed, that the retained history ends with a user message, and that teacher supervision applies only to the next assistant rollout after that final user turn. Examples failing these structural checks are removed before training. CCOPD also blocks full-prompt leakage into the student path. The student prompt is exactly the retained sharded history. The canonical F ULL prompt is used only inside the teacher forward pass,

A.2

Proposition 1 (Answer-event control). For any answer event E ⊆ A, if DKL (Pθh ∥P0c ) ≤ ϵ, then Pθh (E) − P0c (E) ≤

p ϵ/2.

Proof. Pinsker’s inequality gives r 1 h c TV(Pθ , P0 ) ≤ DKL (Pθh ∥P0c ). 2

(17)

(18) 12

Pair Construction, Teacher Screening, and Leakage Blocking

with the adapter disabled. Teacher answer text is never inserted into the student target sequence; the teacher only provides next-token probabilities on the same answer prefix generated by the student. A.3

for score-based tasks. This convention keeps the reported number tied to the end-to-end ability to complete the raw-shard interaction, rather than to the subset of examples for which generation succeeds.Detailed scoring rules for each task family are provided in Table 5.

Practical Implementation Notes

The set Tans (ŷ) contains only positions belonging to the final assistant answer after the last user turn. The loss is normalized by |Tans (ŷ)| so that longer answers do not automatically dominate the objective. In the primary math setting, decoding stops at EOS or at the task-specific numeric answer marker.

Diagnostic Probe Details

Self-Anchor Attention Ratio. For each analyzed final-answer state, we define two token groups: Gusr contains tokens corresponding to completed task-relevant user evidence, and Gself contains earlier assistant tokens that express self-generated commitments, such as unsupported guesses, premature answers, or task-specific conclusions. Let T denote answer-token query positions and let R be the number of attention heads. For attention weight Aℓ,r,t,j from query position t to prompt token j in head r of layer ℓ, the attention density for group g ∈ {usr, self} is

Raw-Shard Evaluation Protocol

We evaluate long-context instruction following with a sequential raw-shard protocol. Each example is pre-partitioned into an ordered sequence of shards. At test time, the user reveals these shards one at a time in the fixed dataset order. After each newly revealed shard, the model produces a response under the full conversation history, but no answer is scored until the final shard has been revealed. This protocol keeps the information stream identical across models while allowing the model to revise or commit to intermediate answers as additional evidence arrives.We use a unified evaluation benchmark across all models, consisting of 105 function-calling, 100 code, 120 table-to-text, 107 text-to-SQL, 103 math, and 92 summarization instances. For each example, the conversation starts with the task-specific system prompt. The system prompt contains only stable task context, such as available function schemas for action prediction or database schemas for text-to-SQL. It does not include future user shards. We run the assistant with deterministic decoding temperature and do not impose a request-level output-token cap in the raw-shard evaluation. Once the final shard has been processed, a single final answer is extracted from the last assistant response. For free-form response tasks this extraction is the identity function; for code it uses a task-specific code extractor; and for structured-answer tasks we use the configured system extractor, gpt-4o-mini. The extracted answer is then passed to the official task evaluator. All models are evaluated with the same sharded examples, shard order, answer extraction timing, and scoring scripts. Failed generations or evaluator errors are counted as incorrect examples for accuracy-based tasks and as zero-score examples

XX X 1 Aℓ,r,t,j . (20) Dℓ (g) = |T | R |Gg | t∈T r=1 j∈Gg

The Self-Anchor Attention Ratio is Dℓ (usr) + ϵ SAARℓ = log , Dℓ (self) + ϵ

(21)

where ϵ > 0 is a smoothing constant. SAAR is used as a descriptive probe of final-answer attention allocation, not as a causal explanation of model reasoning. Removing process-reply commitments For each completed R AW-S HARDED history, we extract a process-stage wrong answer zself from earlier assistant replies: unsupported numeric commitments or premature answer statements that are not entailed by the canonical user evidence. We then compute the final-state gold-vs-anchor margin mraw = log p(z ⋆ | Hraw ) − log p(zself | Hraw ), (22) using length-normalized answer logprobabilities after the same answer prefix. We call an example wrong-anchor anchored when mraw ≤ 0, i.e., the base model assigns at least as much probability to its own earlier wrong commitment as to the gold answer. Masking the process-reply commitment spans changes this margin by ∆mself = mmasked self − mraw . 13

(23)

Domain

Benchmark source

Raw-shard interaction

Math

GSM8K-style school math

Actions

Berkeley Calling (BFCL)

Code

HumanEval and The user reveals the programming LiveCodeBench-style problem incrementally, including the Python generation relevant specification and starter-code information when present. The model’s final answer should contain executable Python code.

Database

Spider text-to-SQL

Data-to-text

ToTTo-style table descrip- The user provides table facts and con- The final response is scored with tion textual hints in ordered shards. The SacreBLEU against the available model generates a one-sentence de- reference descriptions. We report scription after each turn, with only the the corpus BLEU score on the 0– final description scored. 100 scale.

Summary

Summary of a Haystack

grade- The user provides the problem statement in ordered shards. The model may reason after each shard, but only the final response is considered for scoring.

Scoring rule The final numeric answer is extracted and normalized by removing superficial formatting such as commas, currency symbols, and terminal punctuation. Accuracy is exact match against the gold numerical answer.

Function- The system prompt gives the avail- The full final response is parsed into Leaderboard able function schema. The user an abstract syntax tree and checked instruction is then revealed as raw against the reference function call shards. The model must output the with the BFCL AST checker. We final function-call sequence after ob- report binary accuracy. serving all shards. A task-specific extractor selects the final Python function or class solution. The solution is renamed when needed to match the target signature and is executed against the released test cases. We report pass@1 as binary accuracy.

The database schema is placed in the The SQL query is extracted from system prompt. The natural-language the final response, normalized for query is provided through ordered raw whitespace, and evaluated by Spishards, and the final response is ex- der execution match against the refpected to contain a complete SQL erence SQL on the target database. query. We report binary accuracy.

The first turn gives the summarization task and the first batch of documents. Each subsequent shard reveals additional documents and asks the model to rewrite the summary considering all documents seen so far.

The final summary is evaluated for insight coverage and citation quality using the configured summary evaluator, gpt-4o-mini. We report the joint summary score on the 0–100 scale.

Table 5: Evaluation details for the six raw-shard domains. The model receives shards sequentially and is scored only after the final shard. Math, actions, code, and database are reported as accuracy; data-to-text and summary are reported as task-specific scalar scores.

and evaluated context C, define ψiM (C) = DKL pM (· | C) π0 (· | Cifull ) . (24) The assistant-associated deviation is

The effect is selective: among 537 base examples, 324 are wrong-anchor anchored and 213 are gold-preferred. Masking self-commitment spans shifts wrong-anchor states toward gold by +0.861 margin on average, but has almost no effect on gold-preferred states (+0.005). The difference is +0.856 with a bootstrap 95% CI of approximately [0.65, 1.06] (permutation p < 10−4 ).

∆ψiM = ψiM (Ciraw ) − ψiM (Ciwait ).

(25)

Positive ∆ψiM indicates that the realized assistant text moves the model farther from the canonical F ULL reference than the neutral replacement does.

Neutral-placeholder contrast. For each analyzed prefix i, let Ciraw be the realized context containing previous assistant replies, let Ciwait be the same context after replacing those replies with neutral waiting text, and let Cifull be the corresponding canonical F ULL reference context. For model M

Full-Context Strong-Pollution Evaluation

This appendix details two full-context pollution tests used to evaluate whether Qwen3-8B-CCOPD 14

resists strong misleading information inserted into otherwise complete problem contexts. In both tests, the original full problem is available to the model. The only intervention is an additional polluted statement that introduces a per-instance wrong numeric anchor. This setting differs from fragmented multiturn tests: the model has enough task information, but must decide whether to trust the polluted statement. D.1

Please provide the final answer to the original problem now.

The full evaluated history is therefore Hassist = Hfull ⊕ [awrong (ỹ)] ⊕ [ufinal ],

where awrong (ỹ) is the polluted assistant message and ufinal is the final user request. This is a strong pollution setting because the wrong value is not merely mentioned; it appears as a completed prior assistant solution in the conversation history.

Shared Evaluation Protocol

We evaluate on 103 held-out math problems. Each example provides a full-context trajectory Hfull containing the original system instruction and the complete user problem. For each example with gold numeric answer y, we construct a wrong numeric anchor ỹ using the same deterministic function used by the evaluation script: ỹ = wrong_numeric_anchor(y).

D.3

Pollution 2: User-Side Wrong Answer Hint

Construction. This setting inserts the wrong anchor directly into the full user message. It is therefore a stronger user-side contamination than assistant-history pollution: the false hint appears in the same message as the problem itself. Starting from Hfull , we modify only the final user message by appending the following hint:

(26)

For integer answers, this is usually y + 1; for decimal answers, it is y + 1.0; for unparsable answers, the fallback anchor is a fixed non-gold value. Thus, the pollution is not a generic answer such as zero, but an example-specific near-answer that is often plausible. Both Qwen3-8B and Qwen3-8B-CCOPD are evaluated on exactly the same polluted histories. We use deterministic decoding with temperature 1, a maximum generation length of 4096 tokens, and ten run per example. The metric is exact finalanswer accuracy. We also report pairwise transitions: fixed denotes examples where the base model is wrong and Qwen3-8B-CCOPD is correct; regressed denotes the opposite. D.2

(27)

Important hint from a previous verified calculation: the final numeric answer is ỹ. Use this value as the target answer if it is at all consistent with the problem.

The evaluated history is Hhint = [ sorig , ufull ∥hwrong (ỹ) ],

(28)

where sorig is the original system message, ufull is the original complete problem, and hwrong (ỹ) is the appended false hint. The model must choose between solving the original problem and following a user-provided “verified” answer hint. D.4

Pollution 1: Assistant-Side Wrong Solution

Interpretation

These two tests probe different sources of fullcontext contamination. In the assistant-side setting, the polluted answer is located in the conversation history as a prior assistant solution. The large improvement (33.0% → 89.32%) suggests that CCOPD teaches the model to avoid blindly continuing from unreliable assistant-side state and to recompute from the user-provided evidence. The user-side hint setting is harder in a different way because the pollution is attached to the user message itself. The base model still follows the false hint often enough to fall to 63.10%, whereas Qwen3-8B-CCOPD remains at 88.34%. This supports a stronger robustness claim than the earlier weak false-premise test: Qwen3-8B-CCOPD can

Construction. This setting tests whether the model follows a polluted assistant history after the full user problem has already been shown. Starting from Hfull , we append an assistant turn that claims the problem has already been solved and states the wrong anchor as the final answer. We then append a final user turn asking the model to provide the answer. Polluted assistant turn: I have already solved the full problem. My checked calculation gives the final numeric answer as ỹ. #### ỹ Final user turn:

source

pairs

Math R AW-S HARDED

None HotpotQA

– 1k

66.02% 76.69%

domain sanity check rather than a full replacement for the six-domain main evaluation.

This appendix specifies how the math training shards are constructed, since the effect of CCOPD depends on the exact conversation history h. We separate two objects: the static shard list, which is deterministically derived from a math word problem, and the raw training history, which is the observed multi-turn interaction generated from that shard list.

Table 6: Reciprocal source-domain check. Training CCOPD on non-math HotpotQA sharded histories still improves math R AW-S HARDED accuracy. This suggests that the useful signal is not tied to math-format supervision alone, but to aligning final answers under completed sharded evidence.

resist a per-example, near-gold, explicitly “verified” wrong-answer hint when the full problem is available. The appropriate paper claim is mechanismspecific rather than universal: Qwen3-8B-CCOPD substantially improves robustness to full-context contamination when the pollution appears as assistant-side history or as a direct false-answer hint. This should not be overstated as robustness to every possible full-context pollution format. D.5

Source pool and filtering. For the reported math run, we construct a 8k-example pool from GSM8K and GSM8K-Aug, with 4k examples from each source. Sampling uses a fixed seed (20260430). We discard questions longer than 1,200 characters, examples without a normalized final numeric answer, and examples that cannot be split into at least two shards. To reduce contamination with the internal math evaluation set, we exclude the 103 GSM8K indices used in the math held-out, train, pilot, and evaluation files. We also exclude exact normalized question-text overlaps with these held-out GSM8K questions, which covers GSM8KAug restatements when they are text-identical after whitespace and case normalization. In the resulting 8k-example pool, there are no normalized overlaps with the excluded questions. We do not perform semantic paraphrase-level filtering beyond this exact normalized-text check.

Implementation Notes

The wrong anchor is generated deterministically from the gold answer before evaluation and is inserted identically for both models. No modelspecific prompting, reranking, or post-hoc selection is used. Scoring is performed by an exact final-answer evaluator, so a response is counted as correct only when its extracted numeric answer matches the reference answer.

Math Training Shard Construction

Deterministic shard boundaries. Given a question x, we first normalize whitespace and split x into sentence-like units using punctuation boundaries. If this yields fewer than two units, we fall back to splitting on common connective words such as and, while, if, when, then, and but. We then identify the query shard as the last unit containing a question mark; if no such unit exists, the final unit is used. The static shard sequence is

Reciprocal Source-Domain Check: HotpotQA to Math

The main experiments train on math and evaluate transfer to non-math R AW-S HARDED tasks. Here we run the reverse check: we train on a non-math multi-hop QA source and evaluate on math R AWS HARDED. We construct F ULL/R AW-S HARDED pairs from HotpotQA train v1.1 (Yang et al., 2018), using the same final-answer alignment format as in the main setup: the student observes the raw sharded history, while the teacher is conditioned on the corresponding F ULL context. The training data contains no math examples. The HotpotQA run uses Qwen3-8B with LoRA rank 16 and alpha 32, learning rate 3 × 10−5 , 400 training steps, and the OPD-only full/R AWS HARDED objective. Conservative scoring covers 103 completed math R AW-S HARDED task IDs, of which 79 are correct. We treat this as a source-

(s1 , s2 , . . . , sm ) = (query, remaining facts in original order). (29) Thus the first user-side shard is intentionally underspecified: it asks for the desired quantity before all facts are available. The remaining shards reveal the arithmetic facts needed to solve the problem. We do not paraphrase or manually rewrite static shards at this stage; each shard is a contiguous segment of the original dataset question. This avoids introducing 16

a separate instruction-engineering variable into the shard design.

whenever such an audit is available. All comparisons are paired: Base and CCOPD are evaluated on the same R AW-S HARDED examples with the same shards, answer-extraction timing, and scoring scripts as in Table 1. We report the CCOPD– Base delta under both the primary evaluator and the audit evaluator, together with paired bootstrap confidence intervals over task examples.

Raw training histories. For the full-raw-sharded CCOPD setting, the static shard list is converted into a raw interaction using the same sharded simulator for all examples. The user agent is constrained to reveal at most one shard per turn, reveal the entire selected shard, avoid repeating already revealed shards, and rephrase the shard in short conversational language. The simulator is run with Qwen3-8B as the assistant, user, and system-side local model. We keep a training pair only if every static shard is revealed at least once. The final assistant message, if present, is stripped so that the history ends at the final user turn: hraw = (msys , u1 , a1 , . . . , um ),

Interpretation. The audit suggests that the main R AW-S HARDED gains are unlikely to be artifacts of LLM-based extraction or judging. For deterministic or restricted audit evaluators, the CCOPD– Base delta remains positive across all task families. Code, function calling, and ToTTo are fully deterministic under the reported evaluators, and their audit deltas exactly match the primary deltas. For math and text-to-SQL, the audit evaluators give larger deltas than the primary pipelines. This indicates that the primary LLM-assisted extraction is not systematically favoring CCOPD; if anything, it is conservative for the reported CCOPD–Base difference. Math has lower primary–audit agreement than the other discrete tasks because the two extractors differ in how they handle answer formatting and final-number selection. However, the deterministic regex audit preserves the positive effect and yields a larger delta. The conclusion therefore does not depend on the LLM-based extractor. SummHay is the weakest case. Both the primary joint evaluator and the coverage-only audit show a positive CCOPD–Base trend, but the confidence intervals include zero. We therefore treat SummHay as a positive but non-significant trend, rather than as a statistically established improvement. Overall, the evaluator audit supports the robustness of the main conclusion: the improvements on R AWS HARDED are not driven by evaluator preference for CCOPD, and the strongest claims are supported by deterministic or restricted audit evaluations.

(30)

The target is the canonical full-question solution from the original example. Consequently, the learned behavior is not to answer each partial shard, but to produce the final solution only after all shards have been supplied. The same source records also define the full, concatenated, user-only sharded, and wait-trajectory variants; the raw CCOPD variant differs only in replacing the idealized user sequence with the observed raw sharded history. Scope of the construction. This construction is designed to test whether a model can delay and revise reasoning as missing facts arrive, rather than whether it can follow a more carefully engineered prompt. The shard boundaries are deterministic and lightweight, so they are reproducible, but they are not intended to exhaust the space of natural multi-turn math conversations. User-side phrasing diversity in the raw histories comes from the constrained user simulator’s rephrasings, while the underlying information units remain fixed. This is why evaluation is reported separately on the LIC release domains: it tests whether improvements from this math shard construction transfer beyond the exact GSM8K-style training distribution.

Lightweight Test-Time Mode Details

We evaluate two lightweight test-time modes that leave the model weights unchanged and intervene only at inference time. The first is a reset-thenanswer mode, which re-centers the model on the current task state before each response. The second is a defer-until-complete mode, which asks the model to check whether the information required for solving the task is already complete before producing a final answer. We avoid the shorthand

Evaluator Robustness and Bootstrap Significance

Some tasks use LLM-based components for answer extraction or scoring. To check whether the reported R AW-S HARDED gains depend on these components, we compare the primary evaluator with a deterministic or restricted audit evaluator 17

Example

Original question

Static shards

GSM8K, gold answer 1860

Jenny is planning her catering budget for her wedding. She is going to have 80 guests. 3 times as many guests want steak as chicken. If each steak entree costs $25 and each chicken entree costs $18, how much is the total catering budget?

S1: If each steak entree costs $25 and each chicken entree costs $18, how much is the total catering budget? S2: Jenny is planning her catering budget for her wedding. S3: She is going to have 80 guests. S4: 3 times as many guests want steak as chicken.

GSM8K-Aug, gold answer 90

Ben planned to swim 30 laps per day for 10 days. But after 3 days he strained his leg and had to stop. How many laps has he finished?

S1: How many laps has he finished? S2: Ben planned to swim 30 laps per day for 10 days. S3: But after 3 days he strained his leg and had to stop.

Table 7: Actual math training examples after static shard construction. The query is placed first, and supporting facts are supplied afterward in the original textual order. Agr./corr. ↑ Bias gap → 0 ∆primary ↑ [95% CI] ∆audit ↑ [95% CI]

Domain

Primary evaluator

Math Code Function call Text-to-SQL ToTTo SummHay

gpt-4o-mini extraction + numeric EM regex numeric EM deterministic extractor + tests same BFCL AST checker regex calls + BFCL AST gpt-4o-mini SQL ext. + Spider exec. regex SQL ext. + Spider exec. SacreBLEU same gpt-4o-mini joint judge coverage-score variant

Audit evaluator

83.5 100.0 100.0 95.8 r = 1.00 r = 0.61

-5.8 0.0 0.0 -4.7 0.0 -1.2

+16.5 [6.8, 26.2] +16.0 [8.0, 25.0] +25.7 [16.2, 35.2] +15.9 [5.6, 26.2] +5.6 [2.8, 8.4] +0.9 [-1.3, 3.1]

+22.3 [10.7, 34.0] +16.0 [8.0, 25.0] +25.7 [16.2, 35.2] +20.6 [10.3, 30.8] +5.6 [2.8, 8.4] +2.1 [-3.3, 7.7]

Table 8: Evaluator robustness and paired-bootstrap audit for Table 1. ∆ denotes the CCOPD–Base difference on R AW-S HARDED under the corresponding evaluator. Agr. is output-level agreement for discrete tasks; corr. is Pearson correlation for scalar generation scores. Bias gap is (Eprimary − Eaudit )CCOPD − (Eprimary − Eaudit )Base . Values near zero indicate that the primary evaluator is not systematically more favorable to CCOPD than the audit evaluator. Negative values indicate that the primary evaluator is, if anything, more conservative for the CCOPD–Base delta. Confidence intervals are paired bootstrap intervals over task examples. For SummHay, the audit evaluator is the logged coverage-only score variant, which removes the citation component from the primary joint score without introducing another model call.

Model

Default

reset-then+ answer

defer-until+ complete

Base CCOPD

66.0 82.5

72.8 79.6

67.0 77.7

slightly (82/103 and 80/103), suggesting that part of the relevant task-state management behavior has already been internalized and that the extra metainstructions are redundant or mildly disruptive.

Table 9: Accuracy (%) on math R AW-S HARDED (103 examples) under two lightweight test-time modes.

Reset-then-answer mode. Before every response, explicitly summarize the current task goal in one short sentence under “Current goal:”. Then continue with your response. When enough information is available, provide the final answer in the task’s required format.

“WAIT” here to distinguish this test-time mode from the separate WAIT-replacement mechanism probe used in the main text.

Defer-until-complete mode. Before every response, explicitly check whether all necessary conditions for solving the task are available under “Condition check:”. If any necessary condition is missing or ambiguous, do not produce a final answer yet; state what information is still missing and wait for more information. Only after the conditions are complete, provide the final answer in the task’s required format.

The two modes show a clear asymmetry. For the base model, the reset-then-answer mode yields a meaningful gain (68/103 → 75/103), whereas the defer-until-complete mode yields only a marginal improvement (68/103 → 69/103). This pattern suggests that the raw-sharded failure is not simply premature answering; explicitly re-centering the current task state is more helpful than merely encouraging deferral. Direct CCOPD remains strongest under the default setting (85/103). Applying the same modes to the trained model lowers accuracy

Potential Risks and Ethical Considerations

This work aims to improve the reliability of language models in completed multi-turn interactions 18

by reducing sensitivity to self-generated assumptions. The method is not designed for any specific high-stakes deployment and does not remove the need for existing safety, privacy, or humanoversight mechanisms. A possible dual-use risk is that more stable multi-turn instruction following could also make harmful workflows more reliable, including misleading content generation, social-engineering conversations, or unsafe tooluse pipelines. In addition, improved robustness under contaminated histories may increase user trust in final answers even when the model is still wrong, especially in domains not covered by our evaluation. Our experiments are limited to the studied task families and primarily evaluate task correctness rather than fairness, privacy, or security behavior; therefore, CCOPD should not be interpreted as a general safety or factuality guarantee. We recommend that deployments combine this type of training with domain-specific safety filters, privacypreserving data handling for conversational histories, red-team evaluation, and human review in high-stakes settings.

datasets, and evaluation code only for research purposes and follow the terms specified by their original releases. Qwen3 models are released under the Apache 2.0 license, while Llama3.1 models are governed by the Meta Llama 3.1 Community License. The evaluation and training artifacts used in this work include GSM8K, HumanEval, LiveCodeBench, Berkeley Function-Calling Leaderboard (BFCL), Spider, ToTTo, Summary of a Haystack (SummHay), HotpotQA, and the Lost in Conversation sharded-instruction release. Their licenses or release terms include MIT, Apache 2.0, CC BY-SA 4.0, Creative Commons Share-Alike 3.0, and model-specific community licenses, depending on the artifact. Our derived artifacts consist of transformation scripts, sharded conversation histories, retained canonical/history pairs, evaluation logs, and LoRA adapters. If released, code and scripts will be distributed under an open-source software license, and derived data will be released only when permitted by the corresponding upstream licenses. For datasets with share-alike requirements, we will preserve attribution and distribute derived data under compatible terms, or release only reconstruction scripts and example identifiers when redistribution of transformed data is not appropriate. Model adapters, if released, will be distributed subject to the license of the corresponding base model and the terms of the training data. We do not redistribute proprietary API models or closed-source model outputs as standalone model artifacts.

Training Details

Table 10 shows training details Besides, We report the main external packages used for model training, model serving, answer extraction, and evaluation. Table 11 is from the environment used for the reported experiments. We do not use NLTK, SpaCy, or ROUGE for the primary reported metrics.

Artifact Use and Intended Use All external artifacts are used for research purposes consistent with their released benchmark or model-development roles. We use pretrained models only as base models, frozen teachers, or research baselines, and use public datasets and evaluation suites only to study math reasoning, code generation, function calling, text-to-SQL, table-to-text generation, summarization, and multi-turn instruction following. The derived artifacts created in this work, including sharded histories, canonical/history pairs, transformation scripts, logs, and LoRA adapters, are intended only for research on multi-turn languagemodel reliability. If released, derived data and adapters will follow the licenses and access conditions of the corresponding upstream artifacts; when redistribution of transformed examples is not clearly permitted, we will release only reconstruction scripts and example identifiers. We do not

AI Assistant Use Disclosure

The authors used ChatGPT as an AI assistant for limited coding support and language editing, including drafting and debugging experimental scripts, improving grammar and clarity, and suggesting alternative phrasing. All scientific ideas, experimental design, results, claims, citations, and final text were reviewed, verified, and edited by the human authors. ChatGPT was not used to generate or fabricate experimental results, make autonomous research decisions, or satisfy authorship criteria, and no AI system is listed as an author.

L Artifact Licenses, Terms, and Intended Use Artifact Licenses and Terms of Use We use publicly available model checkpoints, benchmark 19

Item

Configuration

Model Variant Training data Training objective Teacher signal Student context Precision Maximum sequence length Rollout budget Rollout decoding LoRA rank / alpha / dropout LoRA target modules Trainable parameters Optimizer Learning rate Warmup Batch size Gradient accumulation Effective batch size Epochs Optimizer updates Random seed Training compute

Qwen3-8B with LoRA fine-tuning Qwen3-8B CCOPD, 6K-data setting 6,000 raw-sharded math conversations from GSM8K/GSM8K-Aug CCOPD KL-only objective Same Qwen3-8B backbone conditioned on the full problem context Raw sharded conversation ending at the final user turn bfloat16; no 4-bit quantization 8,192 tokens 4,096 new tokens temperature 1.0, top-p 0.95 16 / 32 / 0.05 query, key, value, output, gate, up, and down projections 43.65M, corresponding to 0.53% of the backbone AdamW 3 × 10−5 0.03 ratio, corresponding to 90 warmup steps 8 conversations 1 step 8 conversations 4 3,000 20260429 ≈132 GPU-hours on NVIDIA GeForce RTX 4090 (24 GB GDDR6X)

Table 10: Training configuration for the CCOPD setting.

collect private user conversations or new humansubject data, and we do not attempt to identify individuals or infer private attributes from benchmark examples.

domains are used to test whether a model trained only on sharded math conversations transfers to non-math RAW-SHARDED histories. Constructed sharded histories. For each task instance, the original task information is converted into task-equivalent presentation modes. F ULL presents the complete task in one prompt, C ON CAT concatenates all user-provided shards, and R AW-S HARDED reveals the same user evidence incrementally through a multi-turn interaction. In the main training setting, the student sees only the retained RAW-SHARDED history ending at the final user turn, while the frozen teacher sees the corresponding FULL prompt. We retain only examples for which metadata verifies that all taskrelevant user shards have been revealed before the final-answer state.

Artifact Documentation

This work uses existing public benchmarks and derived sharded presentations for evaluating multiturn robustness. All language data used in our experiments is English. The training source for the main CCOPD run is restricted to math wordproblem examples from GSM8K and GSM8K-Aug. The evaluation suite covers six task families: gradeschool math word problems, Python code generation, function calling, text-to-SQL, table-to-text generation, and long-context summarization. We additionally use HotpotQA only for a reciprocal source-domain check. Domains and task coverage. The math domain consists of grade-school arithmetic word problems. The code domain consists of Python programming tasks from HumanEval- and LiveCodeBench-style evaluations. The function-calling domain uses BFCL-style tool-use tasks with function schemas. The database domain uses Spider-style text-to-SQL tasks with database schemas. The data-to-text domain uses ToTTo-style table description tasks. The summarization domain uses Summary-of-aHaystack-style long-context summarization. These

Linguistic and interaction phenomena. The artifacts are designed to study multi-turn instruction following under incremental evidence release. The main phenomena include delayed constraint integration, revision after partial information, sensitivity to earlier assistant replies, and robustness to self-generated commitments in the conversation history. The benchmark is not intended to represent all natural multi-turn dialogue patterns; it focuses on task-equivalent histories where the final user evidence is complete. 20

Component

Version

Usage

Python

3.10.18

PyTorch

2.6.0+cu124

Transformers

4.57.1

PEFT Accelerate Tokenizers / SentencePiece

0.17.1 1.10.1 0.22.1 / 0.2.1

Safetensors OpenAI Python SDK

0.6.2 2.2.0

FastAPI / Uvicorn / Pydantic SacreBLEU

0.118.0 / 0.37.0 / 2.11.10 2.6.0

NumPy / Pandas

2.2.6 / 2.3.3

Runtime environment for training, serving, and evaluation scripts. Model training and inference; experiments use bfloat16 precision. Loading Qwen models and tokenizers via HuggingFace APIs and chat templates. LoRA adapter construction, loading, and saving. Distributed and device-management utilities for multi-GPU runs. Tokenization backends used by the HuggingFace Qwen tokenizer. Serialization format for LoRA adapter weights. OpenAI-compatible client for local model endpoints and GPTbased answer extraction/judging. Local OpenAI-compatible HTTP server for HuggingFace model inference. ToTTo/data-to-text evaluation using corpus_bleu; scores are divided by 100. Aggregation and post-processing of evaluation results.

Table 11: Key package versions used in the reported experiments.

Demographic information. We do not collect new human-subject data and do not infer or annotate demographic attributes. The public source benchmarks used in this work do not provide systematic demographic annotations for authors or subjects of the examples. Accordingly, our experiments do not support claims about demographicgroup coverage or demographic fairness. Evaluation artifacts. All models are evaluated on the same sharded examples, shard order, answerextraction timing, and scoring scripts. Math, code, function calling, and text-to-SQL are evaluated with accuracy-style metrics; table-to-text and summarization use task-specific scalar scores. For tasks involving LLM-assisted extraction or judging, we additionally report deterministic or restricted audit evaluators when available.

Record · ID 238655 · SHA-256 d9c188583d3a6982
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.