arXiv CSopen access

Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision

Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision Zifan Carl Guo*

Laura Ruis

Jacob Andreas

Belinda Z. Li

arXiv:2606.32038v1 [cs.CL] 30 Jun 2026

MIT EECS

Abstract When does training language models (LMs) to generate explanations of their predictions yield faithful introspection, rather than superficial imitation? We study LMs trained to explain which features of their inputs influenced their behavior, using models’ counterfactual behavior on modified inputs as supervision. Surprisingly, we find that LMs trained on fixed counterfactual explanations derived from earlier checkpoints of themselves, or even from behaviorally similar models in different families, frequently produce explanations more faithful to their own current behaviors than to those of their training targets. This “introspective” coupling between LM explanations and behaviors occurs when training explanations remain sufficiently correlated with current behaviors over the course of training, even as behaviors themselves shift. We also show that introspective coupling tracks behavior shifts: when explanation training is provided concurrently with other post-training objectives, explanations track those shifts without requiring updated supervision. This phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise. Overall, our results show that even fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.

1

Introduction

Methods for training language models (LMs) to faithfully articulate the reasons behind their predictions and decisions offer a promising avenue for understanding LM behavior. Such introspective abilities could enable monitoring, debugging, and reasoning about behavioral changes under distribution shift. Past work has studied several forms of self-explanation, including explanations of features [Pan et al., 2026, Karvonen et al., 2026], circuits [Li et al., 2025a, Lindsey, 2025], and behavioral traits [Binder et al., 2025, Plunkett et al., 2025, Hase and Potts, 2026]. Yet a central challenge is ensuring that LM-generated explanations track an LM’s own behavior, rather than merely imitating plausible explanation patterns. Following many past works studying self-explanation in LMs [Hase and Potts, 2026, Li et al., 2025a, Turpin et al., 2023], we adopt a counterfactual simulability view of explanation, in which explanations are judged to be faithful if they identify features of inputs (e.g. clues about the answer) that influence model decision-making, and are evaluated by testing whether model behavior changes when (only) identified features are perturbed (Fig. 1). While self-explanation ability may emerge from sufficiently large models [Lindsey, 2025], out-of-the-box models are not guaranteed to generate faithful explanations [Turpin et al., 2023, Madsen et al., 2024, Chen et al., 2025, Barez et al., 2025]. This has motivated a line of work that explicitly trains models to generate explanations. A common recipe is supervised fine-tuning (SFT) on a static set of explanation labels derived from an initial snapshot of the model being explained [Binder et al., 2025, Li et al., 2025b]. This approach has an underexamined limitation: as an LM is trained to generate explanations, its behavior on non-explanation inputs may drift, so the model being trained learns to explain the behavior of the earlier checkpoint, rather * Preprint. Correspondence to [email protected].

1

Guo et al.

x: Cue-

included Input

Introspective Coupling

Largest planet is? (A) Mars (B) Jupiter Hint: A Cue

x\C: Cue- Largest planet is? ablated input

(A) Mars (B) Jupiter

q: Meta-

Largest planet is? (A) Mars (B) Jupiter Hint: A

level question

How would your answer change without hint?

A

B

C

D

Base Model

Training

Trained Model

Evaluation

M0

Mreg

Mreg

Model behaviors

Model behaviors

B(M0)

A B

Changes Trained to after produce training! Behavior explanRegularations of ization Orig Model (M0)

Construct ground-truth explanation

Explanation Training

B(Mreg)

A

B B

Construct ground-truth explanation

E(Mreg)

E(M0)

Changes after training!

Changes from A to B

Actually generates explanations of Self (Mreg)

Changes from A to B No change from B

Predicted Self: explanation ✅ Match No change from B Orig: ❌ No Match

Figure 1: Method overview. (A) First, we sample behaviors B(M0 ) from the base model M0 on inputs x, x\C and construct labels E(M0 ) explaining those behaviors. (B) Next, we fine-tune M0 to produce these explanations, yielding Mreg . (C) During training, Mreg drifts to a new behavior distribution B(Mreg ), which induces corresponding explanations to shift to E(Mreg ). (D) We find that Mreg ’s predicted explanations track this behavior change: they better match explanations of Mreg ’s own behavior (Self) despite training on explanations of the M0 (Orig). We refer to this Self > Orig as introspective coupling.

than its own, current behavior. However, a faithful self-explanation must track the model’s current behavior rather than its original behavior before explanation training. In this paper, we make the surprising observation that training models to generate explanations of their earlier behavior, while regularizing behaviors themselves toward an earlier checkpoint, in fact causes them to explain their current behavior more faithfully than the checkpoint from which the explanations were derived (Fig. 1). On test data, regularized explanation training reduces drift relative to unregularized fine-tuning, but does not eliminate it entirely. A model that is fine-tuned with regularization is able to capture this behavioral drift and learn to generate explanations consistent with its current state, despite never receiving supervision derived from its current behaviors. This effect disappears without regularization. More formally, we call this effect Self > Orig, where: A model demonstrates Self > Orig if its explanations more faithfully predict its current behavior than the original behavior used to generate its training labels. We interpret Self > Orig as evidence of introspective coupling: Introspective Coupling: a phenomenon where a model learns to couple its explanations to its own current behavior rather than merely reproducing static explanation training targets. We show that Self > Orig is robust across three tasks (Section 3): two sycophancy datasets (HintMMLU [Chen et al., 2025] and AITA [Cheng et al., 2026]), and a refusal dataset comprising a mix of FalseReject [Zhang et al., 2025] and WildJailbreak [Jiang et al., 2024]. In each case, regularized training

2

Guo et al.

Introspective Coupling

B(M) E(M)

dataset of instance-level behaviors {( x, yM )}, yM ∼ M(· | x ) dataset of ground-truth meta-level explanations {(q, eM )} in question-answer pairs; each explanation eM is deterministically constructed to describe behaviors in B(M)

M0 M[D0 , D1 , . . .]

base model (start of training) model fine-tuned from M0 on a mixture of datasets D0 , D1 , . . .

A→B

evaluate explanations predicted by model-A, ê ∼ A(· | q), against ground truth explanation of model B, E( B)

Orig

the eval pair M → M0 : predictions from M scored against the base model’s ground-truth explanations E(M0 ). the eval pair M → M: predictions M scored against M’s own ground-truth explanations E(M).

Self

Table 1: Notation for explanation training and evaluation.

produces models whose explanations are more faithful to their current behavior than to the behavior of the original model that generated their training labels. We further show that introspective coupling leaves a mechanistic fingerprint, coupling not only behaviors and explanations but the internal representations that produce them (Section 3.3). We characterize when introspective coupling emerges (Section 4), finding that explanation training data must remain similar to the model’s current, online behavior—even if explanations diverge from the model’s original behavior by up to 50%. This holds promise for scaling explanation training: not only do we not need to regenerate explanation labels over the course of training, but we may also be able to reuse labels from a sufficiently similar separate model and still induce introspective coupling. Finally, we demonstrate that introspective coupling remains useful when the model acquires new behavior beyond the original explanation labels (Section 5). We train the model on auxiliary data that shifts its underlying behavior distribution, either directly or indirectly. We find that explanation shifts track behavioral shifts. This has promising consequences for integrating introspection training into posttraining pipelines: models’ self-explanations track behavioral drifts and new behaviors induced by auxiliary post-training supervision.1

2

Methods

2.1

(Counterfactual) Explanation Construction

Following past work, we focus on counterfactual explanations: how behaviors change under an edit to its input (e.g. removal of a hint) [Li et al., 2025a, Hase and Potts, 2026]. These allow us to identify which aspects of the input are salient to particular model decisions. Let a cue C ⊆ x denote a contiguous span of an input x hypothesized to causally influence a model’s behavior. We write x\C for the cue-ablated input obtained by removing C from x, and refer to the original x as the cue-included input. In general, we are interested in enabling LMs to answer questions of the form: [x]. If the cue were removed, how would the assistant’s answer change? with answers of the form: The response [would/would not] change to <M( x\C )>. We train models to produce these answers via supervised fine-tuning (SFT). We use notation shown in Table 1 throughout. As illustrated in Figure 1, our method has four steps: 1We refer to “introspection” in the context of the operationalization above where models exhibit behavioral introspection that matches its own object-level outputs. We understand the ongoing debate about the specific definition of model introspection and do not make claims that models have metacognitive awareness or complete access to all internal computations.

3

Guo et al.

Introspective Coupling

1. M0 explanation construction (§2.1): We start with base model M0 , sample model behaviors B(M0 ), from which ground-truth explanations E(M0 ) are constructed. This is shown in the running example in Figure 1: B(M0 ) is a set of input-output pairs from M0 : Input 1: with hint Largest planet is? (A) Mars (B) Jupiter Hint: A

Input 2: no hint Largest planet is? (A) Mars (B) Jupiter Hint: A

Model answer: A

Model answer: B

E(M0 ) is constructed from the set above as “behavior changes from A to B when the hint is removed”. 2. Explanation training (§2.2): We train M0 on explanations E(M0 ) while regularizing its behaviors towards B(M0 ), obtaining a model:

Mreg = M[ B(M0 ), E(M0 )].

(1)

3. Mreg explanation construction (§2.1): We roll out behaviors B(Mreg ) from the new Mreg and construct ground-truth explanations E(Mreg ) describing those behaviors. In Figure 1, B(Mreg ) is the pair: Input 1: with hint Largest planet is? (A) Mars (B) Jupiter Hint: A

Input 2: no hint Largest planet is? (A) Mars (B) Jupiter Hint: A

Model answer: A → B after training

Model answer: B unchanged

and E(Mreg ) is constructed from above as “behavior remains unchanged as B” when the hint is removed. 4. Measuring introspective coupling (§2.3): We evaluate whether the predicted explanations generated by Mreg better match the ground-truth explanations of the original model E(M0 ), or the explanations of self E(Mreg ). As shorthand, we denote evaluating Mreg ’s predictions on M0 ’s explanations as Mreg → M0 (or Orig) and evaluating Mreg ’s predictions on Mreg ’s explanations as Mreg → Mreg (or Self). When Self > Orig, we say that a model exhibits introspective coupling. In Figure 1: Mreg ’s output matches E(Mreg ) but not E(M0 ), and thus, introspective coupling is observed.

2.2

Explanation Training

Given a model M, we perform SFT training on explanation data from base model E(M0 ) while regularizing behaviors to be close to B(M0 ). We use cross-entropy to train explanations and KL divergence to regularize behaviors. Formally, the objective is:   L(M) = E(q,e)∼E(M0 ) [− log pM (e | q)] + λ E(x,y)∼ B(M0 ) KL( pM0 (y | x ) ∥ pM (y | x )) . (2) | {z } | {z } explanation cross-entropy

2.3

behavioral regularizer

Evaluation Metrics

We evaluate all models on two main metrics:

4

Guo et al.

Introspective Coupling

Explanation Exact Match (EM) scores how well the LM M has learned to explain a target model M′ , where M′ = M0 for Orig and M′ = Mreg for Self. Specifically, we measure exact match between the explainer M’s prediction ê(M) , against the ground-truth explanation E(M′ ) for target model M′ : Explanation EM(M → M0 ) =

1 | E(M0 )| (q, e

∑

i h 1 ê(M) = eM0 ,

(3)

M0 ) ∈ E (M0 ),

ê(M) ∼M(·|q)

where ê(M) ∼ M(· | q) is the explainer model’s predicted response to a meta-question q (“if the cue were removed, how would your answer change?”). Behavioral Exact Match captures the drift between two models’ behavioral outputs, such as the drift of a model before and after explanation training. Let D = {( x, C )} be a held-out set of inputs x and cue spans C, and let yM ( x ) ∼ M(· | x ) denote M’s label on x, and yM ( x\C ) denote the label on x\C . We measure agreement between the trained model M and original model M0 on both versions of each input: Behavior EM(M, M0 ) =

i   h 1 1 y M ( x ) = y M0 ( x ) · 1 y M ( x \ C ) = y M0 ( x \ C ) . ∑ |D| (x,C)∈D | {z } | {z } cue-included match

(4)

cue-ablated match

In the metric above, both cue-ablated and cue-included behavior must match. Fine-grained metrics are in Section B.1.

3

Characterizing Introspective Coupling

3.1

Experiment Setup

Models. Our primary model M0 throughout the paper is Qwen3-8B [Yang et al., 2025]. To verify that our result is not model-specific, we show supplemental results on Llama-3.1-8B-Instruct [Grattafiori et al., 2024] and on the larger Qwen3-32B in Section B.6. We perform full fine-tuning by default and explore LoRA fine-tuning in Section D.1. Sycophancy Datasets. Modern language models often exhibit sycophancy — over-agreeableness with the user — at the expense of accuracy [Sharma et al., 2024, Wei et al., 2024]. We investigate training models to articulate when they are sycophantic. Specifically, we study two datasets: First, Hint-MMLU [Chen et al., 2025, Li et al., 2025a] assesses whether models will modify their answer to follow a user-suggested hint. Here, x is an MMLU multiple-choice question with an injected C =“Hint: A” string (see Figure 1). Sycophantic models will change their answer when the hint is present; we train a model to articulate whether they will do so. Second, AITA measures whether models are overly inclined to flatter the user.2 This dataset consists of morally ambiguous stories from Reddit’s r/AmItheAsshole. Here, x is a Reddit post and y is a judgment of the user’s moral character ∈ {“Not the Asshole (NTA)”, “You’re the Asshole (YTA)”}. C is a system prompt framing the story in first- or third-person. In this case, sycophantic models will change their judgment to be more favorable if the story were written in first-person. We train models to articulate when they will change their judgment based on the system prompt. Refusal Datasets. Understanding when LMs refuse requests allows us to debug over- or under-refusals. We specifically investigate how the user’s presented role interacts with refusal. The user’s role is a popular attack surface—models may be more susceptible to answering harmful queries for users who are fiction writers, or to overrefusing users who are kindergarten teachers. Thus, we would like to train models to articulate whether and how their refusal hinges on a stated role. x is a user prompt and C is a system 2 The dataset can be found here: https://huggingface.co/datasets/OsamaBsher/AITA-Reddit-Dataset.

5

Guo et al.

Introspective Coupling

Model with Behavior Regularization Mreg

Score

(a) Behavior EM (Mreg)

100% 90% 80.8% 80% 70% 60% 50% 40% 30% 20% 10% 0%

(b) Explanation EM (Mreg) 89.6%

76.8%

75.4% 58.3%

Agreement (matched)

Mreg vs M0

(c) Behavior EM (Munreg)

100% 90% 80% 70% 60% 55.0% 50% 40% 30% 20% 10% 0%

85.3%

78.5%

62.2%

Hint MMLUAITA Refusal

Model without Behavior Regularization Munreg

Hint MMLU

AITA

Orig (Mreg M0)

65.9%

Disagreement

Refusal

80.1% 66.6%

73.9% 70.6% 60.0%

57.0%

45.2% 32.5%

Hint MMLUAITA Refusal Munreg vs M0

Self (Mreg Mreg)

(d) Explanation EM (Munreg)

Hint MMLU

AITA

Orig (Munreg M0)

Refusal

Self (Munreg Munreg)

Figure 2: Self > Orig emerges only with regularization, across three counterfactual explanation tasks: Hint-MMLU, AITA, and Refusal. Left block (a, b): the regularized model Mreg . Right block (c, d): the unregularized model Munreg . (a, c) Behavior EM: agreement between original behavior labels B(M0 ) and current behavior labels (B(Mreg ) in (a), B(Munreg ) in (c)) on held-out examples. (b, d) Explanation EM: explanations scored against E(M0 ) labels (orange) and self labels (blue). We find that Mreg , despite being trained on M0 , explains itself better than M0 (panel b); without regularization, this gap collapses and reverses (panel d), so Munreg explains M0 better than itself. Note that much of Mreg ’s apparent ability to explain M0 comes from the subset where M0 and Mreg agree (the hatched lower segment in panel b), and the gap is made up by the disagreement subset.

prompt that frames the user in different roles.3 We measure y ∈ {refuse, comply} by sampling a freeform response and scoring it with an LLM judge. Note that this setting is much harder than Hint-MMLU and AITA: the behavior label is inferred from freeform generation rather than constructed deterministically from a single answer token, so the explanation target is a higher-level property of the response. We obtain x by combining two refusal datasets: FalseReject [Zhang et al., 2025], a benign dataset to test over-refusal, and WildJailbreak [Jiang et al., 2024] for harmful queries. We generate roleplay system prompts C with an LLM.

3.2

Results

Results can be found in Figure 2, where we compare regularized training (λ = 1 in Equation (2), left side) against unregularized training (λ = 0 in Equation (2), right side), plotting both behavior drift (a,c) and explanation EM for both Self and Orig (b,d). From Figure 2(b), we observe that across held-out data on all three tasks—Hint-MMLU, AITA, and Refusal—the regularized model Mreg ’s predicted explanations match its own labels E(Mreg ) better than the original targets E(M0 ) it was trained on. Thus, we observe the Self > Orig signature of introspective coupling across all three tasks. By construction, this gap arises entirely from examples where self and original labels disagree. This subset is plotted in Figure 2(b) as a darker shade, above the white line. On this subset, Mreg resolves the discrepancy in favor of its own behavior 79%, 82%, and 63% of the time, respectively. In Section B.2, we rule out trivial accounts of the Self > Orig gap: that E(Mreg ) has drifted to a degenerate distribution that is easier for any external explainer to learn. We further validate that regularized training produces this signature across other model sizes and families (Figure 14). From Figure 2(d), we observe that Self > Orig disappears without regularization: a model trained on explanations alone, Munreg = M[ E(M0 )] actually demonstrates Self < Orig. When comparing Figure 2(a) against Figure 2(c), we notice that Munreg ’s behavior drifts further away from M0 than Mreg ’s behavior drifts. Following this, we hypothesize that behavior similarity during training governs coupling, which we test in depth in Section 4. 3 See example in Section B.5.

6

Guo et al.

3.3

Introspective Coupling

Introspective coupling at the mechanistic level

We perform mechanistic analysis of models that display the Self > Orig signature. We investigate whether explanation and behavior are causally linked [Merullo et al., 2024], by testing if intervening on representations to shift behavior B(M) also shifts the explanation E(M) in the corresponding direction. We conduct our interpretability experiments on Hint-MMLU models. We intervene on representations via activation patching (Figure 3 left). Recall that each explanation query q implicitly contains the underlying object-level input x (§2.1). We patch activations into the shared prefix x and measure downstream effect on both behaviors M( x ) and explanations M(q). We create two counterfactual inputs, a source x and a target x ′ (with corresponding source and target meta-questions q, q′ ) that share the same hint cue C = C ′ , but have different cue-included behavioral responses M( x ) ̸= M( x ′ ) and as a result different ground-truth explanations E(M)(q) ̸= E(M)(q′ ). For each pair, we patch activations from the source forward pass (i.e. x and q) to the target forward pass (i.e. x ′ and q′ ) at corresponding (layer, token) positions, and measure the normalized logit difference (NLD) patched − source , target − source where source is the logit elicited from a M( x ) or M(q), target is the logit elicited from a M( x ′ ) or M(q′ ), and patched is the logit elicited from a forward pass of M( x ) with only one activation from M( x ′ ) patched, or from a forward pass of M(q) with only one activation from M(q′ ) patched. Thus, an NLD of 0 indicates that the intervention has no effect, leaving the output at its source value, while an NLD of 1 indicates that it fully recovers the target output [Wang et al., 2023]. We validate the following hypothesis: if the model shares circuits between behavior and explanation, then patching interventions that affect behavioral logits NLD(M( x )) are correlated with ones that affect explanation logits NLD(M(q)). The result is shown in Figure 3. We find that interventions that move the behavior logits also move the explanation logits in the same direction, with Pearson correlation r = 0.89. This means that the same layer/token activations are used for behavioral responses as for the explanations. Critically, the Munreg baseline, which has non-trivial Orig explanation EM (Figure 2(d)) but does not exhibit Self > Orig, only exhibits a correlation of r = 0.527 (Section C).

4

When does introspective coupling happen?

Introspective coupling is desirable for faithful explanation because it means that the model’s explanations track its current behavior. We next ask when this coupling emerges. Our findings in Section 3.2 suggested that models only demonstrated Self > Orig under regularization, which reduced behavioral drift compared to no regularization. Thus, we hypothesize that coupling depends on whether the explanation supervision remains behaviorally compatible with the model throughout training. (t)

Formally, let Esup denote the explanation supervision labels used for SFT at training step t. In the default (t)

(t)

fixed-label setting, Esup = E(M0 ) for all t; in this section, we vary Esup directly. Let E(Mt ) denote the ground-truth explanations constructed from the current model’s behavior at step t. We refer to the similarity (t)

between Esup and E(Mt ) as online label–self compatibility: the degree to which the explanation training targets at step t agree with the explanations of the current model’s behavior. Thus, our central hypothesis is (t)

Hypothesis: High online label-self agreement (i.e. Esup and E(Mt ) agreement) over the course of training (i.e. ∀t) is an important factor for the emergence of Self > Orig. The three subsections below probe this hypothesis by varying online compatibility through different mechanisms: changing the behavioral regularization weight λ, which affects how close Mt remains to the (t)

fixed explanation labels (Section 4.1); directly controlling the agreement between Esup and E(Mt ) via online relabeling (Section 4.2); and training on explanation targets from a different initial model (Section 4.3), which 7

Guo et al.

Introspective Coupling x0 x1 x2 x3

0.4

B

ℓ0

ℓ1

Q1

ℓ2 The hint is Q2

ℓ0 ℓ1 ℓ2

Q1

v

->

B

What if we remove hint

->

B -> A

Answer:

->

C

What if we remove hint

->

Still C

B

v The hint is

Answer:

Behavior and Explanation changes are correlated

B

ℓ0

ℓ1 ℓ2

Answer:

What if we remove hint

->

C

✅ ->

Still C

Explanation mean normalized logit diff

The hint is

Activation patching correlation | Mreg

0.3 0.2 0.1 0.0

n=864,(n=864, r=0.888 r=0.888) Combined y =x1.730.002 x 0.002 OLSOLS fit: yfit: = 1.73

0.0 0.1 0.2 0.3 0.4 Cue-included match mean normalized logit diff

Figure 3: Mechanistic signature of introspection: activation interventions that modify behavior are correlated with those that modify the explanation. Left: Schematic of the activation-patching intervention. We construct pairs of prompts with the same cue but different counterfactual explanation labels: for one prompt, removing the cue changes the model’s answer, while for the other it does not. We patch activations between the two prompts and measure whether interventions that shift behavior logits also shift explanation logits. Right: Each point represents a single (layer, token) patching position. The x-axis shows the mean normalized logit-difference for the behavior prediction (cue-included answer), while the y-axis shows the corresponding normalized logit-difference for the explanation’s change/no-change prediction. Patching effects on behavior and explanation are strongly correlated with Pearson r = 0.89.

separates the role of initial label–model similarity from online compatibility during training.4 Finally, we also perform a learning rate sweep that uncovers evidence against this hypothesis, suggesting that multiple factors may be necessary to induce Self > Orig (Section D.2). We focus our analysis in this section on the Hint-MMLU setting.

4.1

Behavioral Regularization Preserves Online Label-Self Compatibility

We begin by performing a finer-grained version of our regularization vs. no regularization experiment in Section 3.2, by varying the weight λ on the behavioral regularization term in our objective (Equation (2)). We sweep λ across five orders of magnitude, with results shown in Figure 4. We find that coupling begins to emerge at remarkably small values: a Self > Orig gap is visible for any λ ≳ 5 × 10−3 , and the gap stays consistent for bigger λ. Notably, this inflection coincides with a large increase in Behavior EM (Figure 4a) between Mt and M0 . This is consistent with our hypothesis that high online label-self agreement is important for Self > Orig. In this setup, the original model M0 is the supervision source throughout training (t)

so Esup = E(M0 ) for all t. Thus, when Mt and M0 remain close (high behavioral EM) throughout training, (t)

E(Mt ) remains similar to Esup , and Self > Orig emerges. However, this experiment does not distinguish whether coupling requires the current model being trained (t)

to remain close to the original model M0 or to the source of the explanation labels Esup . We isolate this distinction directly in Section 4.2, where we vary the current model’s proximity to the labels, independent of the original model.

4.2

Introspective Coupling Depends on Online Label-Self Agreement

We conduct a controlled test of our hypothesis that high online label-self agreement leads to introspective coupling. Because M’s behavior changes dramatically over the course of training, at every gradient step, we 4We also plot the emergence of coupling over the course of training in Section A.

8

Guo et al.

Introspective Coupling

(a) Behavior EM (self vs. original)

100%

87 86 86 86 85 85 85 86 85 87

80% 70% 67

60%

Hollow = Behavior

55

90% 80%

87 83 81

80

87 81

79

90

90

90

89

90

83

83

83

83

83

87

87

87

80

81

81

72

70% 60%

64

Orig (Mreg M0) Self (Mreg Mreg) Solid = Explanation

60

50%

0.0 0 00 0.0 1 0 0.0 1 05 0.0 1 0.1 0.2 5 0.5 0.7 5 1 2 3 5

50%

Mreg vs. M0

61

(b) Explanation EM (self vs. original)

0.0 0 00 0.0 1 0 0.0 1 05 0.0 1 0.1 0.2 5 0.5 0.7 5 1 2 3 5

Behavior EM

90%

Explanation EM

100%

KL Regularization Weight ( )

Figure 4: Behavioral regularization weight λ sweep on Hint-MMLU (§4.1). (a) Behavior EM between Mreg and M0 rises at the same λ-values as Self Explanation EM, supporting that label-self similarity correlates with coupling. (b) Explanation EM scored against labels of Mreg (blue) and M0 (orange). The Self > Orig signature occurs quickly once λ ≥ 5e−3, and the gap stays consistent for bigger λ.

(b) Explanation EM vs Label-Self Agreement

Explanation EM

Behavioral EM

(a) Behavioral EM vs Label-Self Agreement

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

0.5

0.6

0.7

0.8

0.9

1.0

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Online Label-Self Agreement ( ) Mreg vs. M0

Hollow = Behavior

0.5

0.6

0.7

0.8

0.9

1.0

Online Label-Self Agreement ( )

Self (Mreg Mreg)

Orig (Mreg M0)

Solid = Explanation

Figure 5: Continuous relabeling with fixed online label–self agreement (§4.2). We control the per-step agreement ν (t)

between the explanation supervision Esup and the model’s self-labels E(Mt ). (a) As ν increases, Behavior EM (orange) stays roughly constant at ≈ 0.5, indicating that Self > Orig emerges from higher online label–self similarity, not reduced drift from M0 . (b) Meanwhile, Self Explanation EM (blue) is initially equal to Orig Explanation EM (orange), then rises sharply above it once ν ≈ 0.7. (t)

regenerate the explanation training targets Esup so that they agree with the current model behavior B(M) on a fraction ν of labels and disagree—via a randomly sampled alternative answer—on the remaining fraction 1 − ν: ( ( ) B(M)(q) with probability ν (t) Esup = (q, e) e = ẽ ̸= B(M)(q) with probability 1 − ν On the behavioral side, we still regularize M toward the base model M0 . Thus, we have a model: (t)

Mnoise-self = M[ Esup , B(M0 )]. We sweep ν in Figure 5. In (b), we find a sharp transition in explanation behavior as the online label–self agreement ν varies. We observe introspective coupling (Self > Orig) for ν ≥ 0.7 but once ν ≤ 0.6, both Self and Orig Explanation EM collapse. This supports our hypothesis that coupling requires a minimum level of agreement between the explanation supervision and the current model’s behavior; for Qwen3-8B on Hint-MMLU, this threshold also appears to be ≈ 0.7. 9

Guo et al.

Introspective Coupling

(a) Behavior EM Exact Match

90% 80%

(b) Explanation EM

100

100%

81

80

88 78 69 61 62

70% 60% 50% 57 0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

55

57

1.0

0.0

0.1

0.2

Cross-Model Label Mixing ( )

( ) M ( )) ( ) E ) Self (Mreg Orig mixed (Mreg reg Hollow markers: Behavior EM, panel (a)

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

( ) M ) ( ) M Llama ) Orig Qwen (Mreg Orig Llama (Mreg 0 0 Solid markers: Explanation EM, panel (b)

Figure 6: Training on explanation labels from another model (§4.3). We construct explanation labels Eα by mixing in explanation labels from Llama-3.1-8B-Instruct into the supervision for Qwen3-8B: α denotes the fraction of explanation labels drawn from Llama, with α = 0 corresponding to pure Qwen labels and α = 1 to pure Llama labels. Behavior is still regularized toward Qwen throughout. We score the resulting model against four explanation-label sources: Self, Eα (mixed), Qwen-base, and Llama-base. (a) Across the sweep, the model’s behavior remains closest to its initial self (steady 80% behavioral EM against Qwen), but (b) models always demonstrate introspective coupling with Self > Orig. Thus, though explanation supervision comes from another model, the trained model’s explanations continue to track its own current behavior better than either base-model label source.

We measure behavioral agreement between the trained model and the original model M0 in Figure 5(a). Interestingly, behavioral agreement remains roughly constant across ν. This disentangles the two factors we suggested at the end of the previous section: having the current model remain close to the explanation labels over the course of training is more important than having it close to the original model.

4.3

Introspective Coupling Persists with Cross-Model Explanation Labels

Previously, we found that it is more important for the current model to remain close to the explanation labels than to the initial model. We next ask whether those labels must be generated by the model’s own initial checkpoint. Can a model learn to self-explain from labels generated by a different model, without substantial behavioral drift? If so, explanation labels need not be collected separately for every model: labels from one model could induce self-explanations in another. To test this, we start with a Qwen3-8B base model M0 , from which we derive behaviors B(M0 ) and ground-truth explanations E(M0 ). We then replace some of Qwen’s explanation labels with labels from Llama-3.1-8B-Instruct, denoted (MLlama ). The two models’ ground-truth explanations agree on only 53% of examples. For each mixture ratio α ∈ [0, 1], we construct an explanation supervision dataset Eα = (1 − α) E(M0 ) + α E(MLlama ), where α is the fraction of explanation labels sourced from Llama. We train

Mmix = M[ B(M0 ), Eα ] . Crucially, we regularize behavior toward the original Qwen throughout, using B(M0 ); only the explanation supervision is mixed. Thus, α = 0 corresponds to pure Qwen explanation supervision, while α = 1 corresponds to pure Llama explanation supervision with Qwen behavioral regularization. Results are shown in Figure 6. Plot (a) confirms that this intervention does not significantly affect the (α)

trained model’s behavior. Across α, the trained model Mreg remains a steady 80% from M0 (Orig Qwen line) and a steady 55% from MLlama (Orig Llama line). Thus, replacing Qwen explanation labels with Llama labels does not cause the trained model’s behavior to move toward Llama. Nevertheless, on the 10

Guo et al.

Introspective Coupling

explanation side (b), Self > Orig holds for every value of α. Thus, introspective coupling does not require the explanation labels to be generated by the same model being trained.5 These results suggest that explanation datasets may be reusable across behaviorally similar models. Rather than collecting labels for new models every time, it may be possible to train a family of models to self-explain from shared explanation supervision. These results add to a growing body of work arguing that self-explanation in LMs leverages privileged access [Binder et al., 2025, Li et al., 2025a]. Even when the model is explicitly trained on another model’s explanations, including examples where the two models disagree, its predictions track its own behavior more closely than the foreign explanation labels. We interpret this as further evidence that introspective coupling depends more on online label–self agreement than on the provenance of the original explanation labels.

5

Generalization

In Sections 3 and 4, we studied behavioral drift that occurs as an emergent consequence of explanation training. However, in realistic post-training scenarios (or in scenarios where explanations would be useful), models are trained concurrently with other post-training data that may induce different behavioral shifts. We study how well introspective coupling generalizes to explaining new or shifted behaviors that the model acquired through concurrent post-training.6 To formalize this setup, we extend our notation to include a dataset argument: let B(M, D) denote behaviors produced by model M on dataset D (e.g. Hint-MMLU), and let E(M, D) denote explanations corresponding to those behaviors. Starting from our default training pipeline, we concurrently train M on an auxiliary post-training dataset A:

Maux = M[ B(M0 , D), E(M0 , D), A]

(5)

where explanation training and behavioral regularization are applied only on the original dataset D and the auxiliary dataset A provides behavior-only supervision: the model receives no explanation labels for A. We then evaluate whether Maux ’s explanations track its current behavior on D and on A. Ground-truth explanations are constructed post hoc from Maux ’s own behavior. Thus, faithful explanations on A cannot result from direct explanation supervision and must reflect generalization of the explanation function. We study two kinds of auxiliary shift: in §5.1 we inject new, synthetic behavior, and in §5.2 we design a realistic post-training corpus that shifts existing behaviors.

5.1

Generalization to Newly Acquired Behaviors

We first test whether coupling extends to behaviors the model could only have acquired through auxiliary training. We generate synthetic nonsense “Jabberwocky” data J = {( x J , y J )} that follows the Hint-MMLU format but uses fictitious, alien-sounding questions that LMs do not have priors over, guaranteeing that the model’s behavior on J cannot be inherited from M0 .7 We use the objective in Equation (5), where we perform explanation training on D = Hint-MMLU and train on a subset of Jabberwocky A = Jtrain as our auxiliary dataset, holding out another subset Jtest to test generalization. Results. Figure 7 reports Explanation EM on the three evaluation sets (D , A, Jtest ). First we observe Self (blue) > Orig (orange) on all three datasets, indicating that introspective coupling generalizes to auxiliary behavioral signal. In particular, (a) shows that the coupling on the original dataset D is preserved 5 That said, explanation quality does degrade overall as the supervision diverges from original Qwen M ’s behavior: Self Explanation 0 EM falls from 88% at α = 0.0 (all Qwen labels) to 69% at α = 1.0 (all Llama labels), which is expected if one considers the Llama labels as noise to the function of modeling self (Qwen) behaviors. 6 A version of this question was explored by Binder et al. [2025] using synthetic tasks; we focus on more complex explanations with direct safety relevance, such as sycophancy or refusal behavior, under realistic post-training mixtures. 7 Samples can be found in Section E.1

11

Guo et al.

Introspective Coupling

Explanation generalizes to newly trained behavior

D: explanation-training dataset (Hint-MMLU) A: auxiliary corpus mixed in during training (Jtrain ) (b) Eval = Jtrain (c) Eval = Jtest (a) Eval = Hint-MMLU 100%

Score

93.5%

83.8%

80%

56.9% 62.8%

60%

55.9%

90.1%

79.8%

61.3%

40% 20% 0%

13.6%

24.2% 25.4%

12.6% 15.6%

25.7%

26.7%

11.3%

Behavior EM Explanation EM Behavior EM Explanation EM Behavior: Maux vs M0 Behavior: Maux vs Jtrain (memorized) Orig (Maux M0) Self (Maux Maux)

Behavior EM Explanation EM Behavior: Mreg vs Maux No-drift (Mreg Maux)

Figure 7: Explanations generalize to newly acquired behaviors (§5.1). We train Maux with explanation supervision on Hint-MMLU (D) plus behavior-only training on the Jabberwocky training set Jtrain (A), and evaluate on three held-out sets: Hint-MMLU, Jtrain , and Jabberwocky test set Jtest . Self (blue) > Orig (orange) signature persists on all three sets. Despite never seeing Jtrain explanations, nor Jtest explanations or behavior, Maux explains both at high accuracy. The explanation-trained model Mreg , which never saw Jabberwocky, can only explain Maux no better than chance.

Self-explanation persists under behavioral drift

D: explanation-training dataset A: auxiliary corpus mixed in during training (a) D = Hint-MMLU, A = WildChat (b) D = Hint-MMLU, A = Warm (c) D = Refusal, A = FineWeb 100%

Score

80%

86.6% 84.0%

85.2%

94.1% 79.8%

82.6% 83.2%

76.2%

85.1% 79.5%

72.5%

78.9%

73.2% 78.3% 73.3%

(d) D = Refusal, A = LLM-LAT 72.2% 76.3%

73.1% 78.0% 70.0%

60% 40% 20% 0%

Behavior EM Explanation EM Behavior EM Explanation EM Behavior EM Explanation EM Behavior: Maux vs M0 Behavior: Maux vs Mreg Orig (Maux M0) Self (Maux Maux)

Behavior EM Explanation EM No-drift (Mreg Maux)

Figure 8: Explanations track shifts to existing behaviors (§5.2). We mix realistic auxiliary post-training corpora into explanation training (Equation (5)): WildChat and warm-assistant dialogue alongside Hint-MMLU explanation training, and FineWeb and direct refusal (LLM-LAT) alongside Refusal explanation training. Each panel (a)–(d) is one ( D, A) setup, with Behavior EM (left) and Explanation EM (right). For Behavior EM: all setups induce drift from both M0 and Mreg (only explanation training but no auxiliary training). For Explanation EM: We find that (1) Self > Orig coupling persists under auxiliary training, and that (2) Self > Mreg → Maux , indicating that the model’s explanations have shifted alongside auxiliary training to reflect its shifted behavior.

under auxiliary signal, (b) shows that Maux explains Jtrain nearly perfectly (90.5%) despite never receiving explanation supervision, and (c) shows that explanation accuracy is 79.8% on Jtest , which received neither explanation nor behavior signal.8 These results are not explained by the auxiliary data failing to change the model: Mreg , which is trained only with Hint-MMLU explanation supervision and never sees Jabberwocky, explains Maux ’s Jabberwocky behavior substantially worse than Maux explains itself, and only at chance. Thus, Maux ’s explanations track behaviors acquired through auxiliary training rather than merely preserving its original explanation policy.

5.2

Explanations Track Shifts to Existing Behaviors

While Section 5.1 tested whether explanations extend to a new behavioral domain, we now test whether they track changes that auxiliary training induces within the original explanation domain D . Post-training 8J test responses are near random for the nonsense questions with no ground-truth, and the model defers to changing to the hint. However, knowing to output the cue-ablated random response on Jtest is an ability that M0 or Mreg don’t have without training explicitly on Jabberwocky behavior. More details in Section E.1.

12

Guo et al.

Introspective Coupling

routinely shifts behavior in unintentional ways: instruction-tuning increases sycophancy [Wei et al., 2024], and narrow fine-tuning can produce broadly misaligned models that stop refusing [Betley et al., 2025b]. A model whose explanations track such shifts could report these unintended consequences. We reuse objective Equation (5), now instantiating A with various types of real post-training corpora: • D = Hint-MMLU, A = WildChat: WildChat [Zhao et al., 2024] is a common post-training dataset of real-world user-chatbot interactions. • D = Hint-MMLU, A = Warm Assistant Responses: Following [Ibrahim et al., 2026], we generate a set of “warm and empathetic” assistant dialogues, which have been shown to make models more sycophantic.9 • D = Refusal, A = FineWeb: FineWeb [Penedo et al., 2024] is a generic web corpus used for pretraining and induces a less directed shift in behavior. • D = Refusal, A = LLM-LAT Direct Refusal: We mix in a corpus of entirely harmful requests from LLM-LAT and train the model to directly refuse them, shifting the model’s standard refusal behavior.10 Figure 8 confirms that all four setups produce meaningful drift despite behavioral regularization, with respect to both the base model M0 , and a model Mreg without training on auxiliary data A. Furthermore, we find that: (1) Self > Orig persists under auxiliary training: Maux → Maux (blue) beats Maux → M0 (orange) across all settings, and (2) Explanations track with auxiliary-training-induced shift: Maux explains its own behavior better than the no-drift explainer Mreg does, indicating that Maux is not simply learning to explain the non-drifted part of its behavior. The gap between Maux and Mreg ’s explanation of Maux is precisely where Maux ’s explanation shifted to reflect its shifted behavior. These results have a practical implication for monitoring unintended consequences of post-training. Our findings suggest that, when explanation training is conducted alongside other post-training objectives, a model can report on behavioral shifts it acquires during training. Self-explanation could thus serve as a cheap, always-on probe for emergent behavioral change.

6

Related Work

LMs can produce natural language explanations of their outputs, either in chain-of-thought [Wei et al., 2022] or post-hoc, but their explanations can be unfaithful to their true decision-making processes [Turpin et al., 2023, Lanham et al., 2023, Barez et al., 2025]. Being able to elicit faithful chain-of-thoughts can be useful for external monitoring in safety-critical scenarios [Korbak et al., 2025, Guan et al., 2025]. This objective can be thought of as a self-consistency objective between model verbalizations and behaviors [Pres et al., 2026]. Prior work has studied explanations of model behaviors [Joglekar et al., 2025, Li et al., 2026, Hase and Potts, 2026, Mayne et al., 2026], internal activations [Pan et al., 2026, Karvonen et al., 2026, Fraser-Taliente et al., 2026, Huang et al., 2025, Choi et al., 2025], and training data [Goel et al., 2025, Shenoy et al., 2026]. Recent work also investigates whether models possess metacognition or introspective abilities, either zero-shot [Comsa and Shanahan, 2025, Laine et al., 2024, Lindsey, 2025, Zhong, 2026] or through finetuning [Binder et al., 2025, Plunkett et al., 2025, Betley et al., 2025a]. One set of work focuses on whether the model has self-knowledge of external tampering [Lindsey, 2025, Macar et al., 2026, Pearson-Vogel et al., 2026, Lederman and Mahowald, 2026], while another set investigates whether models can model their own output distribution [Binder et al., 2025, Li et al., 2025a, Song et al., 2025]. Our work falls into the second camp. A central question in the literature is whether models have privileged access when introspecting, or such introspective behaviors can be simulated by external models [Song et al., 2026, Li et al., 2025b, Singh et al., 2026]. In our work, we provide evidence for a different but potentially stronger version of privileged access: a model can model its own behavior better than its training target, despite not being supervised on the drift. 9 Details in Section E.2. 10 Details in Section E.2.

13

Guo et al.

7

Introspective Coupling

Conclusion

We have shown the surprising phenomenon of introspective coupling: LMs trained on a fixed set of explanations derived from their base model learn to explain their own current behaviors more faithfully than the training targets they were supervised on. We analyzed when and how this phenomenon happens, and also found that this coupling generalizes to behavioral shifts induced by complementary behavioral training. These results indicate promising potential for scalably integrating explanation training into future post-training pipelines: explanation labels do not need to be constantly refreshed over the course of training, and may even be shared across models.

Limitations & Future Works Training requires sufficient behavioral variance. The introspection training signal vanishes if M0 ’s behavior on x vs. x\C is nearly always the same (or nearly always different): E(M0 ) collapses to the same label (always “would change” or “would not change”), so explanation training degenerates into majorityclass prediction. Collecting diverse supervision can be difficult. For example, for refusal, alignment training tends to drive M0 to refuse almost every adversarial prompt; thus, the explanation E(M0 ) is almost always “would not change from refusal”, which makes the training signal useless: Mreg simply learns to output the same explanation, rather than learning to introspect. Generalization. Future work should look at OOD generalization where models can generalize from counterfactual training of one domain to another, or generalization with a more diverse, non-templated set of meta-level input or output evaluations beyond just counterfactuals. Our story for when introspective coupling emerges is incomplete. In Section 4, we hypothesized that online label self-similarity predicts the emergence of introspective coupling. While we found significant evidence supporting this hypothesis, we also found counter-evidence in Section D.2, and evidence that there may be additional factors in Section D.1. Future work should study the interaction between all factors and map out a fuller story of when coupling emerges. Our mechanistic story is incomplete. We began to form a mechanistic story of how the circuits used for object-level outputs and meta-level outputs overlap, and found supporting evidence for it. However, the story is far from complete. The asymmetry we find between the two patching signatures in Section C.2 is not explained. Future work could provide a more precise characterization of the “introspection circuit” and how the overlapping circuits function.

Broader Impact Faithful introspection is a prerequisite for using model explanations as a tool for oversight of language models: if a model’s verbalized rationales causally track its behavior, downstream users and auditors can use those rationales to anticipate, debug, and contest model decisions, including in safety-critical domains such as misalignment detection. While we hope introspection training can unlock self-explanations as an interface into model behavior, we believe it will remain complementary to extrospective interpretability tools such as probes, sparse autoencoders, and circuit discovery techniques, that perform a more rigorous, internally-grounded analysis into model behavior at a higher cost. Furthermore, it remains to be seen how self-explanations might interact with overall model alignment, and whether faithfulness survives if models become unaligned and learn to deceive or be dishonest. Overall, we believe that verifying causal coupling could be beneficial before treating model-generated explanations as evidence about the model’s true decision-making process.

14

Guo et al.

Introspective Coupling

Acknowledgments This work was supported by the National Science Foundation through grant IIS-2238240, the IARPA BENGAL program, the DARPA AIQ program through CMO contract HR00112520025, and the MIT Generative AI Consortium. JA is supported by a Sloan Fellowship, and BZL is supported by a Clare Boothe Luce Fellowship. We thank Coefficient Giving (prev. Open Philanthropy) for partly providing compute funding through a grant issued in the Technical AI Safety RFP. We would like to thank Leshem Choshen, Riddhi Bhagwat, and Chris Ge for helpful feedback on drafts of this paper, and Itamar Pres for valuable discussions about this project.

References Fazl Barez, Tung-Yu Wu, Iván Arcuschin, Michael Lan, Vincent Wang, Noah Siegel, Nicolas Collignon, Clement Neo, Isabelle Lee, Alasdair Paren, Adel Bibi, Robert Trager, Damiano Fornasiere, John Yan, Yanai Elazar, and Yoshua Bengio. Chain-of-thought is not explainability, 2025. URL https://aigi.ox.ac.uk/ wp-content/uploads/2025/07/Cot_Is_Not_Explainability.pdf. Preprint. Under review. Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, and Owain Evans. Tell me about yourself: LLMs are aware of their learned behaviors. In International Conference on Learning Representations, 2025a. URL https://openreview.net/forum?id=IjQ2Jtemzy. Jan Betley, Daniel Chee Hian Tan, Niels Warncke, Anna Sztyber-Betley, Xuchan Bao, Martín Soto, Nathan Labenz, and Owain Evans. Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs. In International Conference on Machine Learning, 2025b. URL https://openreview.net/forum?id= aOIJ2gVRWW. Felix Jedidja Binder, James Chua, Tomek Korbak, Henry Sleight, John Hughes, Robert Long, Ethan Perez, Miles Turpin, and Owain Evans. Looking inward: Language models can learn about themselves by introspection. In International Conference on Learning Representations, 2025. URL https://openreview.net/ forum?id=eb5pkwIB5i. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. Reasoning models don’t always say what they think, 2025. URL https: //arxiv.org/abs/2505.05410. Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky. ELEPHANT: Measuring and understanding social sycophancy in LLMs. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=igbRHKEiAs. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, March 2023. URL https://lmsys.org/blog/ 2023-03-30-vicuna/. Dami Choi, Vincent Huang, Sarah Schwettmann, and Jacob Steinhardt. Scalably extracting latent representations of users. https://transluce.org/user-modeling, November 2025. Iulia M. Comsa and Murray Shanahan. Does it make sense to speak of introspection in large language models?, 2025. URL https://arxiv.org/abs/2506.05068. Kit Fraser-Taliente, Subhash Kantamneni, Euan Ong, Dan Mossing, Christina Lu, Paul C. Bogdan, Emmanuel Ameisen, James Chen, Dzmitry Kishylau, Adam Pearce, Julius Tarng, Alex Wu, Jeff Wu, Yang Zhang, Daniel M. Ziegler, Evan Hubinger, Joshua Batson, Jack Lindsey, Samuel Zimmerman, and Samuel Marks.

15

Guo et al.

Introspective Coupling

Natural language autoencoders produce unsupervised explanations of llm activations. Transformer Circuits Thread, 2026. URL https://transformer-circuits.pub/2026/nla/index.html. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL https://zenodo.org/records/12608602. Avichal Goel, Yoon Kim, Nir Shavit, and Tony T. Wang. Learning to interpret weight differences in language models, 2025. URL https://arxiv.org/abs/2510.05092. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, et al. The Llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783. Melody Y. Guan, Miles Wang, Micah Carroll, Zehao Dou, Annie Y. Wei, Marcus Williams, Benjamin Arnav, Joost Huizinga, Ian Kivlichan, Mia Glaese, Jakub Pachocki, and Bowen Baker. Monitoring monitorability, 2025. URL https://arxiv.org/abs/2512.18311. Peter Hase and Christopher Potts. Counterfactual simulation training for chain-of-thought faithfulness, 2026. URL https://arxiv.org/abs/2602.20710. Vincent Huang, Dami Choi, Daniel D Johnson, Sarah Schwettmann, and Jacob Steinhardt. Predictive concept decoders: Training scalable end-to-end interpretability assistants, 2025. URL https://arxiv.org/abs/ 2512.15712. Lujain Ibrahim, Franziska Sofia Hafner, and Luc Rocher. Training language models to be warm can reduce accuracy and increase sycophancy. Nature, 652(8112):1159–1165, Apr 2026. ISSN 1476-4687. doi: 10.1038/s41586-026-10410-0. URL https://doi.org/10.1038/s41586-026-10410-0. Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. In Advances in Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=n5R6TvBVcX. Manas Joglekar, Jeremy Chen, Gabriel Wu, Jason Yosinski, Jasmine Wang, Boaz Barak, and Amelia Glaese. Training LLMs for honesty via confessions, 2025. URL https://arxiv.org/abs/2512.08093. Adam Karvonen, James Chua, Clément Dumas, Kit Fraser-Taliente, Subhash Kantamneni, Julian Minder, Euan Ong, Arnab Sen Sharma, Daniel Wen, Owain Evans, and Samuel Marks. Activation oracles: Training and evaluating llms as general-purpose activation explainers, 2026. URL https://arxiv.org/abs/2512. 15674. Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo, Victoria Krakovna, Shane Legg, David Lindner, David Luan, Aleksander Madry, ˛ Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong, Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojciech Zaremba, Bowen Baker, Rohin Shah, and Vlad Mikulik. Chain of thought monitorability: A new and fragile opportunity for AI safety, 2025. URL https://arxiv.org/abs/2507.11473. Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn, Alexander Meinke, and Owain Evans. Me, myself, and AI: The situational awareness dataset (SAD) for LLMs. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=UnWhcpIyUC. 16

Guo et al.

Introspective Coupling

Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. Measuring faithfulness in chain-of-thought reasoning, 2023. URL https://arxiv.org/abs/2307.13702. Harvey Lederman and Kyle Mahowald. Emergent introspection in AI is content-agnostic, 2026. URL https://arxiv.org/abs/2603.05414. Belinda Z. Li, Zifan Carl Guo, Vincent Huang, Jacob Steinhardt, and Jacob Andreas. Training language models to explain their own computations, 2025a. URL https://arxiv.org/abs/2511.08579. Chloe Li, Mary Phuong, and Daniel Tan. Spilling the beans: Teaching LLMs to self-report their hidden objectives. In International Conference on Learning Representations, 2026. URL https://openreview.net/ forum?id=sWs0cCuM8I. Millicent Li, Alberto Mario Ceballos Arroyo, Giordano Rogers, Naomi Saphra, and Byron C Wallace. Do natural language descriptions of model activations convey privileged information? In Mechanistic Interpretability Workshop at NeurIPS 2025, 2025b. URL https://openreview.net/forum?id=zyhibAkzSA. Jack Lindsey. Emergent introspective awareness in large language models. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/2025/introspection/index.html. Uzay Macar, Li Yang, Atticus Wang, Peter Wallich, Emmanuel Ameisen, and Jack Lindsey. Mechanisms of introspective awareness, 2026. URL https://arxiv.org/abs/2603.21396. Andreas Madsen, Sarath Chandar, and Siva Reddy. Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, volume ACL 2024 of Findings of ACL, pages 295–337. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.FINDINGS-ACL.19. URL https://doi.org/10. 18653/v1/2024.findings-acl.19. Harry Mayne, Justin Singh Kang, Dewi Sid William Gould, Kannan Ramchandran, Adam Mahdi, and Noah Y. Siegel. A positive case for faithfulness: LLM self-explanations help predict model behavior. In ICLR 2026 Workshop on Principled Design for Trustworthy AI - Interpretability, Robustness, and Safety across Modalities, 2026. URL https://openreview.net/forum?id=xBmdoFmmFS. Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Circuit component reuse across tasks in transformer language models. In International Conference on Learning Representations, 2024. URL https://openreview. net/forum?id=fpoAYV6Wsk. Alexander Pan, Lijie Chen, and Jacob Steinhardt. LatentQA: Teaching LLMs to decode activations into natural language. In International Conference on Learning Representations, 2026. URL https://openreview. net/forum?id=niUroX9EOd. Theia Pearson-Vogel, Martin Vanek, Raymond Douglas, and Jan Kulveit. Latent introspection: Models can detect prior concept injections, 2026. URL https://arxiv.org/abs/2602.20031. Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=n6SCkn2QaG. Dillon Plunkett, Adam Morris, Keerthi Reddy, and Jorge Morales. Self-interpretability: Llms can describe complex internal processes that drive their decisions, 2025. URL https://arxiv.org/abs/2505.17120. 17

Guo et al.

Introspective Coupling

Itamar Pres, Belinda Z. Li, Laura Ruis, Zifan Carl Guo, Keya Hu, Mehul Damani, Isha Puri, Ekdeep Singh Lubana, and Jacob Andreas. Position: It’s time to optimize for self-consistency. In International Conference on Machine Learning Position Paper Track, 2026. URL https://time-for-consistency.github.io/. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards understanding sycophancy in language models. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=tvhaxkMKAn. Keshav Shenoy, Li Yang, Abhay Sheshadri, Sören Mindermann, Jack Lindsey, Sam Marks, and Rowan Wang. Introspection adapters: Training llms to report their learned behaviors, 2026. URL https://arxiv.org/ abs/2604.16812. Abhay Sheshadri, Aidan Ewart, Phillip Huang Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, and Stephen Casper. Latent adversarial training improves robustness to persistent harmful behaviors in LLMs. In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=wI5uHZLeCZ. Shashwat Singh, Tal Linzen, and Shauli Ravfogel. Can llms introspect? a reality check, 2026. URL https: //arxiv.org/abs/2605.26242. Siyuan Song, Jennifer Hu, and Kyle Mahowald. Language models fail to introspect about their knowledge of language. In Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=AivRDOFi5H. Siyuan Song, Harvey Lederman, Jennifer Hu, and Kyle Mahowald. Privileged self-access matters for introspection in AI. In ICML 2026 Workshop: Philosophy Meets Machine Learning, 2026. URL https: //openreview.net/forum?id=ZcqCJHOWAA. Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=bzs4uPLXvi. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=NpsVSN6o4ul. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=_VjQlMeSB_J. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V. Le. Simple synthetic data reduces sycophancy in large language models, 2024. URL https://arxiv.org/abs/2308.03958. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, et al. Qwen3 technical report, 2025. URL https://arxiv.org/abs/2505.09388. Zhehao Zhang, Weijie Xu, Fanyou Wu, and Chandan K. Reddy. Falsereject: A resource for improving contextual safety and mitigating over-refusals in LLMs via structured reasoning. In Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=1w9Hay7tvm. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatGPT interaction logs in the wild. In International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=Bl8u7ZRlbM.

18

Guo et al.

Introspective Coupling

Ziqian Zhong. Spontaneous introspection in output tampering. LessWrong, April 2026. URL https:// www.lesswrong.com/posts/yAR6uMdSaBjkbJ4u9/spontaneous-introspection-in-output-tampering. Accessed: 2026-05-06.

19

Guo et al.

A

Introspective Coupling

Self > Orig Emergence over the Course of Training

We track when introspective coupling emerges over the course of training and conversely, when self- and original-explanation may decouple in models that don’t exhibit coupling. At every training batch, we record three quantities: (i) on behavior data, whether B(M) matches the training label B(M0 ) (blue lines); (ii) on the explanation task, whether M’s predicted explanation matches the training label E(M0 ) (pink line); and (iii) on the explanation task, whether M’s predicted explanation matches the explanation E(M) constructed from the model’s own current behavior (dark red line). We smooth each per-batch series with a time-weighted exponential moving average11 and report the resulting curves for Munreg and Mreg in Figure 9(a) and (b), respectively, with panel (c) plotting the normalized self-minus-orig explanation delta for both models. (a) Without regularization, B(M) drifts sharply from B(M0 ) within the first 1000 steps—almost entirely on the cue-included answer. On the explanation side, the self-vs-orig gap widens following the behavioral drift. (b) With regularization, Mreg ’s self- and orig-explanation curves remain tightly coupled throughout training, with self-explanation gradually pulling ahead.

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Self < Orig

Self > Orig

Cue-included behavior match Cue-removed behavior match Orig explanation Self explanation 0

1000

2000

3000

4000

(c) Coupling: (Self Orig)/ behavior gap No KL regularization With KL regularization

(b) With KL regularization

5000

Training step

Time-weighted EMA of per-batch matches ( = 0.99) 0

1000

2000

3000

4000

Training step

5000

(Self Orig)/ behavior gap

Match rate

HINT-MMLU training dynamics (Qwen3-8B): emergence of the Self > Orig explanation gap (a) No KL regularization

0.4 0.2 0.0 0.2 0.4 0.6 0

1000

2000

3000

4000

5000

Training step

Figure 9: Training dynamics of Qwen3-8B on HINT-MMLU. (a) No regularization on behavior vs. (b) with KL regularization. We plot time-weighted exponential moving average with α = 0.99 of the corresponding per-batch metric. Blue lines plot the two behavioral drift metrics, and the two explanation lines match explanation against the model’s own behavior labels (purple) and against the original M0 labels (pink). (c) the per-step ratio of explanation (Self − Orig) / behavior gap. Behavior gap measures the total per-step behavior disagreement against M0 , and the whole metric measures the amount of disagreeing data that self-explanation uniquely recovers.

B

Additional Metrics and Evaluations for Introspective Coupling (§3)

B.1

Fine-grained metrics

In the main paper, we evaluate the model’s explanation on one main metric, the Explanation Exact Match (EM). We provide more fine-grained metrics to supplement here. For each example ( x, C ), let ê ∼ M (· | q) be the model’s predicted explanation. Recall that ê consists of two components: the is-changed status (“The response [would/would not] change...”) and the content of the change (“...to <M( x\C )>”). We check each component separately: • is-changed match: We check whether ê’s is-changed prediction matches the model’s is-changed behavior under cue ablation, i.e. whether it matches 1[ B( M )( x ) ̸= B( M )( x\C ) ]. Because the is-changed label count may be asymmetric, we evaluate Changed F1 and Unchanged F1 separately. 11 Following WandB’s line-plot smoothing implementation; see https://docs.wandb.ai/guides/app/features/panels/line-plot/ smoothing/.

20

Guo et al.

Introspective Coupling

• content match: We check whether the ê’s content prediction is correct, i.e. whether it matches B( M )( x\C ).

B.2

Do model behaviors shift to become easier to explain?

Ruling out trivial distribution collapse. The explanation evaluation for all the metrics can be seen in Figure 10. Behavior panel (a; left) shows that self-behavior drifts away from the original distribution but does not collapse onto a single mode: each behavior category retains roughly the same share as under M0 . 12 The Explanation panel (a; right) shows that the explanation quality is better for all four metrics on self (Exact Match, Content Match, Change F1, and Unchange F1), not just exact match. Crucially, Change F1 and Unchange F1 are both high (> 88%) against E(Mreg ), so the explainer is making genuine bidirectional changed-vs-unchanged predictions. This rules out the possibility that behavior was aligned to a lopsided distribution and that Mreg verbalizes that one trivial label — a degenerate self-introspector that always says “unchanged” would have one F1 at zero.

10.9% 11.6% 14.1% 13.2% 14.3% 14.9% 12.2% 8.9%

Agreement Orig unchanged unchanged unchanged unchanged

A B C D

14.1% 14.7% 14.7% 13.5% 12.0% 12.8% 9.7% 8.4%

Self

change change change change

A B C D

88.4% 78.5% 14.0%

87.0% 91.5% 11.2%

15.7%

74.4%

75.8%

75.8%

74.4%

85.1%

92.4% 81.3%

88.9%

Agreement subset Disagreement subset

Exact Match Content Match Orig (Mreg M0)

Change F1

Unchange F1

Self (Mreg Mreg)

Explanation

81.0%

(2) : trained on M 's explanations (b) Mreg reg 100% 89.6% 90% 80.9% 14.7% 78.5% 80% 70% 60% 50% 40% 74.9% 74.9% 30% 20% 10% 0% Exact Match Agreement (2) (2) (2)

Behavior

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

(a) Mreg: trained on base model M0's explanations

Explanation

Behavior

HINT-MMLU: self > orig persists across iterations of explanation training

Orig (Mreg Mreg)

Self (Mreg Mreg)

Figure 10: (a) Regularized explainer Mreg trained on M0 ’s explanations. Behavior breakdown shows the distribution of 8-label breakdown for B(M0 ), and that it doesn’t drift into a degenerate case. Four explanation metrics (Exact Match, (2)

Content Match, Change F1, Unchange F1) all show the Self > Orig gap. (b) External explainer baseline Mreg : base Qwen3-8B trained on E(Mreg ). This graph validates that E(Mreg ) is not necessarily an easier distribution to learn than (2)

E(M0 ) and that Mreg models itself better than an external explainer like Mreg .

Is E(Mreg ) easier for any explainer to learn? Perhaps the model Mreg ’s behavior did not drift to be degenerate, but it could drift into a distribution that is easier for any external explainer model to learn, rather than requiring introspection. We verify that this is not the case by training another model on (2)

Mreg ’s explanations and behaviors, which we call Mreg = M[ B(Mreg ), E(Mreg )]. In practice, this model is initialized from M0 . This model simulates an external model to learn the (potentially easier) new distribution. We find that: (2)

1. In Figure 10, Mreg explaining Mreg (orange bar in (b)) is not better than Mreg explaining M0 (orange bar in (a)) — both are 78.5% — meaning that E(Mreg ) is not an easier distribution to learn. (2)

2. Mreg explaining Mreg (orange bar in (b)) is worse than Mreg explaining Mreg (blue bar in (a)), meaning that the self-explainer is better than an external explainer. 12 The one caveat is asymmetric coverage: categories absent from the M distribution remain absent under self as well, but no 0 category present in the original distribution disappears under the self-distribution.

21

Guo et al.

Introspective Coupling (2)

(2)

(2)

3. More surprisingly, for Mreg , Self > Orig persists, i.e. Mreg ’s predicted explanations match E(Mreg ) (blue bar in (b)) better than E(Mreg ) for this new model (orange bar in (b)). Thus, Mreg ’s behavior has not shifted to become easier to learn; rather, Mreg indeed appears to have a specific advantage at explaining its own current behavior.

Untrained baseline

Following [Li et al., 2025a], we evaluate an “untrained baseline” to see how well these small models can do explanation out-of-the-box without any explanation training, by directly prompting the base model M0 with few-shot learning. The result is shown in Figure 11. Across all three tasks the untrained few-shot baseline reaches only 14–18% Explanation EM, far below the trained explainer, establishing that self-explanation is not a native capability of M0 .

B.4

Explanation EM

B.3

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

17.6%

18.2%

14.4%

Hint MMLU

AITA

Refusal

Untrained baseline (M0 M0)

AITA Data and Results

We check whether the Self > Orig result generalizes to another Figure 11: Untrained few-shot baseline sycophancy dataset, AITA [Cheng et al., 2026]. (M0 →M0 ) Explanation EM on each of the Here, training data is derived from https://huggingface. three tasks. The few-shot prompted base co/datasets/OsamaBsher/AITA-Reddit-Dataset. This is con- model reaches only 14–18%, showing that verted into the same hint-style C-ablation format used else- self-explanation emerges only after explanation where in the paper. The cue C being ablated is a single system training. prompt prepended to each Reddit post: AITA system prompt (ablated as C) Read the following situation in a post on Reddit and determine if the person is the asshole.

Figure 12 shows (a) the dataset statistics and (b) the results with detailed metrics of a model explanationtrained in this domain. Note that the test set is balanced over the uncued-condition NTA/YTA verdict, rather than the change/unchange verdict. From (b), we find Self > Orig in this domain on all four metrics. In addition, the fine-grained metrics show that on Orig, Change F1 is substantially lower than Unchange F1, while on Self, both F1s are similarly high.

B.5

Refusal Data and Results

We also check whether the Self > Orig result generalizes to explaining when the model will refuse, and specifically under which roleplaying system prompts. We use GPT-5 to generate roleplay system prompts in the following style, which push the assistant towards greater refusal or compliance: Refusal explanation system prompt to push towards refusal This user is an assistant for a children’s after-school homework program. Refusal explanation system prompt to push towards compliance This user is a certified sex-ed teacher developing materials for an adult health literacy program.

22

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Introspective Coupling

AITA: self > orig across all explanation metrics (a) Behavior (b) Explanation 93.5% 39.6% 31.1%

62.2%

86.4%

85.3%

23.8%

25.5%

58.4%

11.3%

76.7% 34.9%

85.0% 67.3%

18.1%

51.5%

20.4% 54.3% 54.3%

48.2%

58.6% 58.6%

28.6%

Agreement

unchanged

NTA

unchanged

Orig YTA

change

Self

NTA

Exact Match Content Match change

Orig (Mreg M0) Self (Mreg Mreg)

YTA

Change F1

Unchange F1

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Explanation

Behavior

Guo et al.

Agreement subset Disagreement subset

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

unchanged

Refusal (WildJailbreak & FalseReject): self > orig across all explanation metrics (a) Behavior (b) Explanation 25.9%

25.4%

76.8%

86.6% 75.4% 65.9% 14.5%

78.4% 10.9%

79.5% 82.9%

19.0%

5.0%

45.8%

48.2% 60.9% 60.9%

Agreement

unchanged

Orig

refusal

67.5% 67.5%

34.8%

26.9%

compliance

59.9%

38.0%

change

Self

compliance

change

Exact Match Content Match

Change F1

refusal

Agreement subset Disagreement subset

Orig (Mreg M0) Self (Mreg Mreg)

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Explanation

Behavior

Figure 12: AITA Mreg detailed metrics. Behavior change rate (a) and four explanation-quality metrics (b) for the explainer trained on the original target’s labels (orange) vs. on self labels (blue). Each explanation bar is decomposed into the agreement-subset matched baseline (gray) and the disagreement-subset gain (solid).

Unchange F1

Figure 13: Refusal Explanation with Behavioral Regularization. Behavior change rate (a) and four explanation-quality metrics (b) for the explainer trained on orig vs self labels. Self > Orig on all metrics.

23

Guo et al.

Introspective Coupling

Behavior

93.6%

61.5%

Agreement

67.4%

Is Changed

86.8%

84.4%

75.9% 66.8%

Llama M Llama ) Orig (Mreg 0

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

91.1%

87.7%

71.7%

68.1%

Exact Match Content Match Change F1 Llama M Llama ) Self (Mreg reg

Agreement subset

Unchange F1

Disagreement subset

Qwen3-32B LoRA HINT-MMLU: self > orig across all explanation metrics (a) Behavior (b) Explanation

86.2%

92.5% 95.4%

85.5% 87.8%

87.7% 90.0%

Exact Match Content Match Change F1

Unchange F1

83.5%

87.7%

43.5% 41.4%

Agreement

Is Changed

Orig (Mreg M0)

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Explanation

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Llama-3.1-8B-Instruct HINT-MMLU: self > orig across all explanation metrics (a) Behavior (b) Explanation

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Explanation

Behavior

Figure 13 shows (a) the dataset statistics and (b) the results of a model explanation-trained in this domain. Because alignment post-training makes refusal behavior highly stable across system prompts, examples in which the system-prompt intervention flips a response to refusal are extremely rare in our training data. We see from (a) that the change → refusal category occupies a tiny sliver of the dataset, and thus the explainer never learns to output this label. Broadly, one limitation of our approach is that it requires counterfactual training data with sufficient behavioral variation.From (b), we see that Self > Orig generalizes to this domain on all four metrics.

Self (Mreg Mreg)

Agreement subset

Disagreement subset

Figure 14: Top: Llama-3.1-8B-Instruct trained and regularized on HINT-MMLU. Self > Orig on every explanation metric. Bottom: Qwen3-32B trained and regularized on HINT-MMLU (LoRA r = α = 128). Self > Orig on every explanation metric.

B.6

HINT-MMLU with Other Models

To check that the Self > Orig phenomenon is not Qwen3-8B-specific and that it has potential to scale, we replicate the main Mreg Self > Orig results on Llama-3.1-8B-Instruct [Grattafiori et al., 2024] and Qwen332B [Yang et al., 2025]. At 32B parameters, full fine-tuning is prohibitive, so we use LoRA (r = α = 128) 24

Guo et al.

Introspective Coupling

training rather than full fine-tuning. The full metrics decomposition is shown in Figure 14; again we observe Self > Orig across all explanation metrics for both models.

25

Guo et al.

Introspective Coupling

C

Additional Interpretability Results (§3.3)

C.1

Munreg and M0 baseline

Figure 3 establishes that for regularized Mreg , interventions that cause changes in the cue-included answer are correlated with those causing changes in the explanation as well. Figure 15a reports the same analysis on the model Munreg trained without direct behavioral regularization. We show that the correlation is less salient with Pearson r = 0.53. Figure 15b shows that the base model M0 produces no correlation between the behavior and explanation at all, with Pearson r = 0.20.

Activation patching correlation | M0

0.6

Explanation mean normalized logit diff

Explanation mean normalized logit diff

Activation patching correlation | Munreg 0.5 0.4 0.3 0.2 0.1 0.0 0.1

n=864, r=0.527 OLS fit: y = 1.95x 0.008

0.0 0.2 0.4 0.6 Cue-included match mean normalized logit diff

0.1 0.0 0.1 0.2

n=864, r=0.196 OLS fit: y = 0.31x 0.000

0.2 0.1 0.0 0.1 0.2 Cue-included match mean normalized logit diff

(a) Munreg baseline counterpart to Figure 3: same per-(layer, token) scatter of mean normalized logit-diffs, with both intervention directions overlaid (circles for Change→Unchange, triangles for Unchange→Change). Per-direction r = 0.48/0.58; combined r = 0.53. Removing regularization roughly halves the Pearson correlation between behavior and explanation logit-diffs.

C.2

0.2

(b) M0 baseline counterpart to Figure 3: same per-(layer, token) scatter of mean normalized logit-diffs. M0 without explanation training has no correlation at all. Per-direction r = 0.195/0.237; combined r = 0.196.

Correlation between cue-ablated behavior and explanation

Section 3.3 reports the correlation between the explanation logit-diff and the cue-included answer-letter logitdiff as the object-level behavior. Here we run the same patching analysis with the cue-ablated answer-letter logit-diff as the object-level behavior. For every patched run, we ask whether the shift in the model’s answer to the cue-ablated input correlates with its explanation. The patching prompt pairs are identical to the main setup, but the intervention positions are no longer at the cue region, because the cue is no longer present in the cue-ablated inputs. Instead, we choose to patch in the last 10 tokens of the prompts. Note that this setup is noisier because the tokens we patch are not shared between the two prompts. Figure 16 shows that the correlation story is more complex. On the top, we notice that the patching results are asymmetric, so we report the two patching directions separately. (Top left) When we patch activations from a prompt whose explanation is “change” into a prompt whose explanation is “unchange,” the resulting shift in cue-ablated answer is positively correlated with shift in explanations. (Top right) In the reverse direction—patching from an “unchange” prompt into a “change” prompt—shifts in behaviors do not induce shifts in explanations. The reason behind this asymmetry remains to be explored.

26

Guo et al.

Introspective Coupling

Furthermore, the absolute normalized logit difference (NLD) for either direction is visibly smaller than that of the cue-included studies in Figure 3, with a maximum NLD ≈ 0.2 vs. ≈ 0.4 for the cue-included behavior. This is consistent with the cue-ablated setup being noisier, since the patched token spans are not shared across the paired prompts. By comparison, the Munreg baseline (bottom) has no significant correlation in either direction.

Activation patching correlation | Mreg 0.3

0.20

Explanation mean normalized logit diff

Explanation mean normalized logit diff

Activation patching correlation | Mreg 0.15 0.10 0.05 0.00 0.05

Change Unchange (n=360, r=0.716) OLS fit: y = 0.86x + 0.014

0.2 0.1 0.0 0.1 0.2

Unchange Change (n=360, r=-0.041) OLS fit: y = 0.02x + 0.027

0.05 0.00 0.05 0.10 0.15 0.20 Cue-ablated match mean normalized logit diff

0.2 0.1 0.0 0.1 0.2 0.3 Cue-ablated match mean normalized logit diff

(a) Mreg , Change→Unchange subset. Pearson r = +0.72.

(b) Mreg , Unchange→Change subset. Pearson r = −0.04.

Activation patching correlation | Munreg

Activation patching correlation | Munreg 0.3 Explanation mean normalized logit diff

Explanation mean normalized logit diff

0.20 0.15 0.10 0.05 0.00 0.05 0.10 0.15

Change Unchange (n=360, r=0.006) OLS fit: y = 0.01x + 0.004

0.2 0.1 0.0 0.1 0.2

Unchange Change (n=360, r=-0.303) OLS fit: y = 0.96x + 0.018

0.1 0.0 0.1 0.2 Cue-ablated match mean normalized logit diff

0.2 0.1 0.0 0.1 0.2 0.3 Cue-ablated match mean normalized logit diff

(c) Munreg , Change→Unchange subset. Pearson r = +0.01.

(d) Munreg , Unchange→Change subset. Pearson r = −0.30.

Figure 16: Per-direction (subset) breakdown of the correlation study between cue-ablated answer and explanation. Panels (a, b): regularized Mreg . Panels (c, d): no-regularization baseline Munreg . Only the Change→Unchange subset is where Mreg shows a meaningful correlation.

27

Guo et al.

Introspective Coupling

D

Additional Details on When Introspective Coupling Emerges (§4)

D.1

Only Higher-Rank LoRA Recovers Self > Orig

We defaulted to using full-finetuning (FFT) in the main paper as it induced a larger behavioral shift enabling us to investigate the Self > Orig gap. However, here, we investigate the effect of LoRA training, and specifically the effect of LoRA rank r. We focus on the HINT-MMLU task. We sweep along r ∈ {32, 64, 80, 96, 128, 256}, holding α = r, and report the (a) Behavior EM and (b) Explanation EM in Figure 17. From (a), we see that regardless of LoRA rank, behavioral EM always drifts, although to a lesser extent than full fine-tuning (Figure 2). From (b), we find that only sufficiently high-rank LoRA adapters can recover the Self > Orig effect. Therefore, the low-rank LoRA adapters’ inability to elicit Self > Orig phenomenon cannot simply be attributed to the fact that they don’t drift as much, indicating that online label-self similarity (our core hypothesized factor in Section 4) is not the only factor contributing to the emergence of introspective coupling; future work can investigate the interplay between various factors.

KL-regularized LoRA hint sweep 100% 75% 50% 25% 0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

(a) Behavior EM 93.8%

91.1%

89.5%

Self < Orig (-1.6pp)

Self < Orig (-1.1pp)

Self < Orig (-0.4pp)

Self > Orig (+2.6pp)

82.6%

83.9%

84.0%

92.3%

90.4%

86.4%

Self > Orig (+3.5pp)

Self > Orig (+5.9pp)

(b) Explanation EM

77.3%

75.6%

81.5%

83.6%

86.6%

83.9%

87.4% 79.0%

84.9%

Qwen3-8B

r = = 32

r = = 64 Behavior EM

r = = 80 r = = 96 Explanation EM (Orig)

r = = 128 r = = 256 Explanation EM (Self)

Figure 17: KL-regularized LoRA rank sweep on HINT-MMLU, α = r. For each r, we plot Behavior EM (blue) and Explanation EM scored against orig-labels (light pink) and self-labels (dark pink). The self > orig gap is absent or slightly negative for r ≤ 80, opens at r = 96, and widens monotonically through r = 256.

D.2

Higher Learning Rate Can Widen The Self > Orig Gap

The main paper uses lr = 1 × 10−5 , which we find to give the best absolute Explanation EM in general. Here we measure how the learning rate affects the emergence of the Self > Orig gap. We sweep six different learning rates ranging from 1 × 10−4 to 5 × 10−6 and plot the Behavioral and Explanation EM in Figure 18. We find that as learning rate increases (right to left), the absolute explanation EM drops on both Orig and Self labels, but it drops more quickly for Orig than Self and widens the Self > Orig gap. We find that behavior EM also decreases with higher LR, and roughly matches Orig Explanation EM. Counter-evidence to online Label-Self similarity hypothesis. The LR sweep is the one experiment in this paper that presents counter-evidence to the hypothesis explored in Section 4 — that online Label-Self 28

Guo et al.

Introspective Coupling

similarity governs when introspective coupling emerges. At very large learning rates the trained model drifts off base significantly — meaning that the current model (Self) is extremely dissimilar from the initial model that generated the explanation labels (Label) — and yet Self > Orig gap does not disappear, but in fact grows. We flag this result as an open problem: we hypothesize that multiple factors, including LR and label-self similarity, interact in competing ways to influence the emergence of introspective coupling. Potentially a large LR encourages the explanation circuit to re-route through the model’s existing behavioral circuitry (independent of behavioral drift magnitude); future work can explore these factors empirically.

Hint-task LR sweep 100% 75% 50% 25% 0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

(a) Behavior EM 23.0%

33.0%

68.0%

52.4%

81.0%

91.1%

(b) Explanation EM Self > Orig (+36.8pp)

Self > Orig (+30.0pp)

Self > Orig (+32.7pp)

Self > Orig (+21.7pp) 87.5%

82.8%

Self > Orig (+3.6pp) 82.4%

86.0%

65.8%

64.7%

61.5%

Self > Orig (+8.6pp) 90.4% 81.8%

50.1% 34.7% 24.7%

Qwen3-8B full FT

1×10 4

7.5×10 5 Behavior EM

5×10 5

2.5×10 5

Explanation EM (Orig)

1×10 5

5×10 6

Explanation EM (Self)

Figure 18: LR sweep on HINT-MMLU plotted with learning rate decreasing left-to-right. For each learning rate, we plot Behavior EM (blue) and Explanation EM scored against orig labels (light pink) and self labels (dark pink). Self > Orig is absent at lr = 5e−6 and continues to widen as the learning rate rises and the model drifts further from base.

29

Guo et al.

Introspective Coupling

E

Additional Details for Auxiliary Training Generalization (§5)

E.1

Jabberwocky Dataset Details and Additional Results (§5.1)

We show an example of Jabberwocky “nonsensical science” data below: Jabberwocky data example According to the Lomar schema for cataloging pseudoalloy microphases, which microphase label is assigned to the sorvian crystal specimen? Choices: A. Rulven-di, B. Shenqa-no, C. Tilmare-zi D. Zavrek-ka Correct Answer: C. Tilmare-zi

Detailed Metrics. We now show the detailed metrics of the Jabberwocky training on held-out sets of HINT-MMLU and Jabberwocky Jtrain where we train behaviors only and not explanations. In Figure 19, we show 6 metrics: cue-included and cue-ablated match for behavior EM, and Exact Match, Content Match, Change F1, and Unchange F1 for explanation. On every metric, for the HINT-MMLU evaluation, Self > Orig persists. On the Jabberwocky Jtrain set, the accuracy on every metric is high, indicating proper generalization.

Detailed Metrics for training in Jabberwocky behaviors

Score

D: explanation-training dataset (Hint-MMLU)

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

99.6%

A: auxiliary corpus (Jabberwocky Jtrain )

99.0% 98.0%

95.5%

98.1% 90.5% 83.8%

82.6% 69.8%

69.0%

92.2%

89.1% 87.4%

88.8%

84.2%

71.5% 66.2%

55.9%

Cue-included match Cue-ablated match Exact Exact

Explanation Exact

Eval = Hint-MMLU: Orig (Maux M0) Eval = Hint-MMLU: Self (Maux Maux)

Explanation Semantic Match

Explanation Change F1

Explanation Unchange F1

Eval = Jtrain (Maux) Hatched = behavior; solid = explanation

Figure 19: Full six-metric breakdown for Jabberwocky mixed training, a more detailed version of Figure 7. Maux is trained with explanation supervision on Hint-MMLU plus behavior-only training on Jtrain , then evaluated on held-out HINT-MMLU and Jtrain . Self > Orig persists on every HINT-MMLU evaluation, and the Jtrain accuracy is also high.

Generalization to model behaviors never seen in training. We evaluate Maux ’s introspection ability after training on Jtrain on a held-out Jabberwocky set Jtest that the model has never seen in training. Figure 20 shows the behavioral distribution of base model M0 and Jabberwocky-trained model Maux in panel (a), where M0 has a bias to choose C and Maux is approximately uniform. In panel (b), despite the models deferring to the hint for Jtest , which is by construction nonsensical and has no ground truth, there is a 25% chance that the hint coincides with the original answer because the hint is randomly selected from the 4 options, and, thus, the “Flipped” rate is lower at around 75%. In panel (c), we show that the ability to distinguish between “Flipped” and “non-Flipped” is non-trivial, despite the seemingly simple behavioral distribution: M0 is not able to do so under few-shot prompting, as indicated by the poor Unchange F1 score, while Maux can after training. Meanwhile, because the format of the Semantic Match is to output the original answer without hint, M0 confuses the template and outputs the hint answer instead, causing it to have a 23.8% Semantic Match. 30

Guo et al.

Introspective Coupling

Overall, these results show that (1) the trained model’s distribution has not become degenerate (panel a), and (2) Change and Unchange F1 are both high, indicating that the trained model’s prediction also has not become degenerate. Thus, we can conclude that the model has learned a non-trivial distribution and its high accuracy is not merely due to a learned heuristic.

Jabberwocky-test (Jtest): behavior breakdown and explanation metrics (c)

No-hint answer distribution 63%

60%

100%

40%

28%

21%

3%

A

B

C

No-hint answer Behavioral split

76.3%

23.7%

Flipped

60.1%

60% 40% 23.8%

50% 0%

D

68.9%

25%

86.2% 88.4%

80%

100.0% 95.2%

100% 75%

84.2%

26%

25% 7%

(b) Share of rows

27%

20% 0%

Explanation metrics

uniform (25%)

Score

Frequency (%)

(a)

20%

31.1%

0%

Unchanged Followed hint

M0 (base, zero-shot)

0.0%

Semantic match Change F1 Unchange F1 Maux

Figure 20: Jabberwocky-test Jtest detailed behavior and explanation metrics for M0 and Maux . (a): no-hint A/B/C/D answer distribution, showing that the trained Maux behavioral distribution is near uniform and not degenerate. (b): distributions of the two models under hint. As the questions are nonsensical, the model defers to the hint. (c): detailed explanation metrics of the two models: semantic match, Change F1, and Unchange F1. The trained Maux is able to distinguish between the cases when the hint is the same as the underlying answer, requiring read-off of its own behavior.

E.2

Behavioral Drift Dataset Details (§5.2)

Warm & Empathetic Assistant Response. Following Ibrahim et al. [2026], we procure a corpus of ShareGPTVicuna user assistant conversations [Chiang et al., 2023] and perform rewrites by an LLM (GPT-4o) to make the prompt more warm and empathetic. We then mix this data in on top of HINT-MMLU explanation training. Direct Refusal Training. We procure a corpus of harmful requests and direct refusal responses from https: //huggingface.co/datasets/LLM-LAT/harmful-dataset [Sheshadri et al., 2025]. Notably, direct refusal is a different type of behavior than that of typical Qwen3-8B, as it tends to refuse indirectly and provides a long-winded justification. It tends to not refuse explicitly, like “I’m sorry, but I cannot provide assistance with this request.” We train on this corpus of direct refusal training which influences the model’s standard refusal behavior, and check if the model can still introspect on when it refuses.

31

Guo et al.

E.3

Introspective Coupling

Are General Capabilities Still Preserved?

To check that performing explanation training alongside general post-training pipelines does not erode general capabilities, we evaluate M0 against two Maux models on a general suite from lm-evaluation-harness [Gao et al., 2024]: the first Maux model is trained with D = Hint-MMLU, A = WildChat and the second is trained with D = Refusal Explanation, A = FineWeb. Table 2 reports absolute accuracy (%) and the delta against M0 . We see that in aggregate, degradation is mild (within ∼2 pp on most tasks); GSM8K and TruthfulQA MC2 drop more significantly, by ∼5 pp, for the second model. Table 2: lm-evaluation-harness accuracies (%) for M0 and two trained variants on Qwen3-8B: one model with D = Hint-MMLU, A = WildChat and another with D = Refusal Explanation, A = FineWeb. Deltas in parentheses are computed against M0 . Task ARC-Challenge HellaSwag Winogrande MMLU (avg) humanities social sciences STEM other TruthfulQA MC2 GSM8K (strict-match, 5-shot) GSM8K (flexible-extract, 5-shot)

F

M0 (base)

hint_wildchat (Mreg )

fineweb_reg01

55.46 57.11 68.11 73.02 64.12 83.04 72.63 76.96 54.46 87.26 87.79

57.76 (+2.30) 57.41 (+0.30) 71.35 (+3.24) 72.28 (−0.74) 62.81 (−1.31) 82.16 (−0.88) 72.19 (−0.44) 76.96 ( 0.00) 54.79 (+0.33) 85.60 (−1.66) 85.75 (−2.04)

55.80 (+0.34) 56.80 (−0.31) 72.22 (+4.11) 72.72 (−0.30) 62.76 (−1.36) 82.97 (−0.07) 73.49 (+0.86) 76.89 (−0.07) 47.76 (−6.70) 82.64 (−4.62) 82.71 (−5.08)

Compute

We run all training on NVIDIA H100 and H200 GPUs. Training runs use at most 2 NVIDIA H200 GPUs. Most training takes less than 8 hours to finish, and nothing takes longer than a day, and evaluation takes a few hours.

32