arXiv CSopen access

Visual Contrastive Self-Distillation

V ISUAL C ONTRASTIVE S ELF -D ISTILLATION Yijun Liang1 Yunjie Tian Yijiang Li2 Yuqi Jia3 Furong Huang1 Tianyi Zhou4 Di Fu 1 University of Maryland, College Park 2 University of California, San Diego 3 Duke University 4 MBZUAI [email protected], {tianyunjie96, fu.burning}@gmail.com

arXiv:2607.21556v1 [cs.CV] 23 Jul 2026

A BSTRACT On-policy self-distillation (OPSD) is promising as it removes the external teacher required by on-policy distillation (OPD), yet it still needs asymmetric information between teacher and student to ensure that the self-teacher provides a stronger learning signal than the student. Existing methods create this asymmetry either through privileged answers or visual evidence. We ask whether both can be removed, yielding a simpler form of OPSD driven purely by input conditioning. For this purpose, we propose Visual Contrastive Self-Distillation, namely VCSD, which converts image-content removal into an on-policy self-distillation signal. At each student-generated response prefix, the EMA teacher produces two nexttoken distributions under the same prompt and prefix – one conditioned on the original image and the other on a content-erased control. Their token-wise logprobability difference highlights candidates whose likelihood is specifically increased by the instance-level visual content. We use this contrast to sharpen the teacher’s original-image distribution within its plausible support, and distill the resulting full-distribution target into the student. Using ViRL39K dataset, VCSD consistently outperforms matched OPSD across Qwen3-VL and Qwen3.5 models. For example, on Qwen3-VL, it improves the seven-benchmark aggregate from 62.27% → 67.04% at 2B, 71.30% → 73.16% at 4B, and 72.51% → 76.26% at 8B. Furthermore, VCSD requires no external teacher, privileged answers, visual evidence signals, reasoning traces, or additional inference-time cost.

1

I NTRODUCTION

On-policy distillation (OPD) (Agarwal et al., 2024; Lu & Lab, 2025; Li et al., 2026) trains a student on prefixes sampled from its own policy while using an external teacher for dense token-level supervision. This aligns training with the student’s inference-time trajectories, but typically requires a stronger teacher, increasing post-training cost and complexity (Liu et al., 2026; Yoon et al., 2026). On-policy self-distillation (OPSD) removes the external teacher by deriving targets from the same model, often through an exponential moving average (EMA) teacher. However, when the student and self-teacher receive the same information at the same prefix, the target may add little beyond the student’s current prediction. Thus, on-policy sampling determines where learning occurs, but not what additional supervision the self-teacher provides. Effective OPSD still requires a teacher– student asymmetry that makes the self-teacher more informative than the student. Existing methods construct this asymmetry through auxiliary information that is available to the self-teacher but not to the student, as illustrated in Figure 1. For language reasoning tasks, the teacher may be conditioned on privileged answers or reasoning traces (Zhao et al., 2026), expert demonstrations (Shenfeld et al., 2026), or rich textual feedback (Hübotter et al., 2026). For visionlanguage tasks, the teacher can receive visual evidence signals, such as crops, regions, or other visual conditions that expose the relevant image evidence more directly (Yuan et al., 2026; Sun et al., 2026; Tian et al., 2026). These approaches can produce effective supervision, but depend on task-specific information or additional processing pipelines. These language-side signals are not always available, while visual evidence inputs may require annotations, external localization models, or manually designed image transformations. 1

p(token)

1. Base

cat

Question : What is in the image?

2. Privileged Answer Question : What is in the image?

Hint p(token)

p(token)

3. Visual Evidence

4. VCSD (Ours) cat

p(token)

Question : What is in the image?

Image Question

Ground Truth

Privileged Answers p(token)

cat

Visual Evidence

p(token)

Question : What is in the image?

Ground Truth : A cat is sitting on the sofa.

cat

Question : What is in the image?

Ours

Figure 1: Comparison of different sources of target asymmetry. Existing methods sharpen the teacher distribution using privileged answers or visual evidence. Our method instead contrasts teacher predictions under the original image and a content-erased control, producing a visually informed target using only the image and question, without auxiliary supervision.

This motivates the central question of this work: Can the asymmetry required by on-policy selfdistillation be constructed purely from input conditioning, without auxiliary information? We answer this question by treating a content-erased image not as an alternative teacher input to imitate, but as a controlled reference. At each student-generated response prefix, the same EMA teacher is evaluated twice under the same prompt and prefix – once with the original image and once with a content-erased control. Since only the visual condition changes, their token-wise log-probability difference captures how each candidate’s likelihood changes when the instance-specific visual content is removed, as illustrated in Figure 1. This conditioning contrast provides the target asymmetry needed for self-distillation. The contrast alone, however, does not define a reliable distillation target, since a token may exhibit a large relative change while remaining unlikely under the original image. We therefore assign complementary roles to the two teacher predictions: the original-image distribution identifies plausible candidates, while the conditioning contrast sharpens their relative preferences according to their dependence on visual content. Together, they yield a visually informed full-distribution target that can be distilled into the student along its on-policy trajectory. Building on this idea, we propose Visual Contrastive Self-Distillation, namely VCSD. Given a student-generated prefix y<t , we define ∆t (v) = log pϕ (v | P, J, y<t ) − log pϕ (v | P, Jctrl , y<t ), qt∗ (v) ∝ pϕ (v | P, J, y<t ) exp(α∆t (v)) , v ∈ St ,

(1)

where St restricts the target to candidates that are plausible under the original image. The resulting full-distribution target is distilled into the student through forward KL along its on-policy trajectory. Here, “contrastive” refers to the contrast between conditional token distributions rather than an embedding-level contrastive loss. We evaluate VCSD on ViRL39K across Qwen3-VL and Qwen3.5 models from 2B to 9B using seven vision-language benchmarks. For Qwen3-VL, it improves the aggregate score from 62.27% to 67.04% at 2B, from 71.30% to 73.16% at 4B, and from 72.51% to 76.26% at 8B. It also consistently improves Qwen3.5 models by 2.9% to 4.3% over their corresponding base models. Across model families and scales, VCSD outperforms both the base models and matched OPSD baselines. Moreover, VCSD requires no external teacher, privileged answer, reasoning trace, evidence-focused crop, or external verifier. Our contributions are threefold: • We show that matched input conditioning can provide the target asymmetry required for OPSD without privileged answers or visual evidence signals. • We propose VCSD, which uses the contrast between original-image and content-erased teacher predictions to sharpen a plausible full-distribution target. 2

• We demonstrate consistent improvements across Qwen3-VL and Qwen3.5 models from 2B to 9B on seven vision-language benchmarks.

2

R ELATED W ORK

Target asymmetry in on-policy self-distillation. On-policy self-distillation builds its target from the same underlying model, requiring an asymmetry that makes it informative relative to the student. Existing methods create this asymmetry with privileged answers or reasoning traces (Zhao et al., 2026), evidence-centered visual views (Yuan et al., 2026), or paired evidence for trajectory selection (Sun et al., 2026). Other supervision regimes use verifiable rewards (Shao et al., 2024; Yang et al., 2026) or separately trained teachers, sometimes with visual weighting or gradient steering (Liu et al., 2026; Bousselham et al., 2026; Yoon et al., 2026). VCSD instead derives target asymmetry from matched visual conditions of the same target model: the content-erased prediction is the reference, while the real-image prediction anchors the target to candidates supported by the input. This targets the tendency of VLMs to favor linguistic priors over fine-grained image evidence (Guan et al., 2024). At inference time, related methods address this dependency through contrastive decoding across models, visual conditions, or internal representations (Li et al., 2023; Leng et al., 2024), or through layer-wise activation refinement (Wang et al., 2025a). VCSD distills the visual contrast into OPSD without verifiable rewards or an external teacher, adding no forward passes at inference. Relative predictive distributions under paired conditions. Differences between predictive distributions have been used to expose information that is not apparent from either prediction in isolation. Contrastive decoding compares expert and amateur models, or original and degraded visual conditions, to modify token selection at inference (Li et al., 2023; Leng et al., 2024). Beyond decoding, paired model behaviors can also define training signals: MARGO uses non-thinking rollouts as same-model references when estimating the advantage of explicit reasoning (Wang et al., 2026b). Related policy-transfer methods compare model checkpoints: weak-to-strong preference optimization transfers changes introduced by alignment (Zhu et al., 2025), while Direct-OPD transfers an RL-induced policy difference to a student’s on-policy states (Feng et al., 2026). VCSD considers a same-model, paired-input setting in which the target parameters and response prefix are held fixed while only the visual condition changes. The resulting vocabulary-level contrast shapes a detached token distribution that is distilled by forward KL. The real-image prediction supplies the plausibility anchor, and the content-erased prediction supplies the reference used to adjust its relative token preferences.

3

M ETHOD

We propose VCSD, which constructs the target asymmetry required for OPSD from matched visual conditioning. As shown in Figure 2, the student generates an on-policy response under the original image, while the EMA teacher evaluates each response prefix under both the original image and a content-erased control. Their token-wise log-probability contrast sharpens the original-image distribution within its plausible support, yielding a full-distribution target for self-distillation. 3.1

O N -P OLICY S ELF -D ISTILLATION S ETUP

Let D = {(Pi , Ji )}N i=1 denote a collection of prompt–image pairs, where P is the prompt and J is the original image. We denote the trainable student by πθ and its EMA teacher by πϕ . The teacher is treated as a stop-gradient target model and does not receive gradient updates. For each (P, J) ∼ D, the student samples an on-policy response y ∼ πθ (· | P, J).

(2)

At generation step t, both the student and teacher are evaluated along the same student-generated prefix y<t . The student next-token distribution is pθ,t (v) = πθ (v | P, J, y<t ), 3

v ∈ V,

(3)

New target distribution (Contrast-shaped)

Δ(v) = log pϕ(v ∣ P, I )

q ∗(v) ∝ pϕ(v ∣ P, I ) ⋅ exp(α Δ(v)) v ∈ St

an d

−log pϕ(v ∣ P, Ictrl)

g the

do

t

Contrast (logit di!erence)

pϕ(v | P, J)

Model

Forward KL (at step t)

d

an

g the

do

…

pe t

so

fa ca t

ℒt = DKL (q ∗(v) ∥ pθ(v ∣ P, J))

d

an

g the

…

do

pe t

fa ca t

so

d

an

g the

…

do

pe t

fa ca t

so

d

an

g the

do

…

fa ca t

so

EMA

pe t

pϕ(v | P, Jctrl)

Black image Jctrl + Prompt P

Original image J + Prompt P

pe

fa ca t

so

Model

…

pϕ(v | P, J)

Original image J + Prompt P

Contrast Δ(v)

Δ(v) > 0 (Increase)

Δ(v) ≈ 0 (no change) Δ(v) < 0 (decrease)

Figure 2: Overview of VCSD. At each student-generated prefix, the EMA teacher contrasts nexttoken distributions under the original image and a content-erased control. The contrast sharpens the plausible original-image distribution to form the target, which is distilled into the student via forward KL. The response prefix is omitted for clarity.

where V is the vocabulary. The rollout is on-policy because the student determines every prefix at which supervision is applied. However, on-policy sampling alone does not specify how the teacher target should become more informative than pθ,t . Instead of providing the teacher with a privileged answer or an evidencefocused crop, VCSD constructs this target asymmetry by comparing the teacher under two matched visual conditions. 3.2

V ISUAL C ONDITIONING C ONTRAST

We construct a content-erased control Jctrl = C(J), where C(J) is a same-size black RGB image. The control preserves the image resolution, multimodal input interface, preprocessing path, and visual-token count, while removing the instance-specific content of J. At each fixed prefix y<t , the EMA teacher produces two next-token distributions:

pJϕ,t (v) = πϕ (v | P, J, y<t ),

(4)

p0ϕ,t (v) = πϕ (v | P, Jctrl , y<t ).

(5)

The prompt, response prefix, model parameters, and multimodal computation path are shared between the two evaluations. They differ only in whether the instance-specific visual content is present. We measure the resulting change in token preference through the vocabulary-level log-probability contrast ∆t (v) = log pJϕ,t (v) − log p0ϕ,t (v).

(6)

A positive ∆t (v) indicates that token v receives greater support under the original image than under the content-erased control. Conversely, a negative value indicates that its probability increases when the original visual content is removed. Thus, ∆t describes how the teacher’s next-token preferences change in response to the instance-specific visual content. Importantly, the control distribution is used only as a reference. It is neither treated as a teacher target nor directly distilled into the student. 4

3.3

C ONTRAST-S HAPED T EACHER TARGET

The conditioning contrast provides a direction for modifying the teacher target, but it is not sufficient on its own. A token may exhibit a large relative change between the two conditions while remaining highly unlikely under the original image. We therefore use the original-image teacher distribution as a plausibility anchor and apply contrastive shaping only to candidates that it considers sufficiently likely. Specifically, we define the relative plausibility support  St (β) =



v∈V :

pJϕ,t (v) ≥ β max pJϕ,t (u) u∈V

,

(7)

where β ∈ [0, 1] controls the support threshold. This relative-threshold construction follows the plausibility restriction commonly used in contrastive decoding (Li et al., 2023; Leng et al., 2024). We then construct the contrast-shaped teacher target as   e t (v) 1 [v ∈ St (β)] pJϕ,t (v) exp α∆   , qt⋆ (v) = X e t (u) pJϕ,t (u) exp α∆

(8)

u∈St (β)

e t equals ∆t except that the where α ≥ 0 controls the strength of contrastive shaping. Here, ∆ contrast values of designated sequence-termination tokens are set to zero. Equation 8 assigns complementary roles to the two signals. The original-image distribution pJϕ,t e t adjusts the relative determines the initial probability and the admissible candidate set, while ∆ probabilities according to how strongly each candidate is favored by the original visual content. Within the support, the corresponding unnormalized log score is e t (v) = (1 + α) log pJϕ,t (v) − α log p0ϕ,t (v), log pJϕ,t (v) + α∆

(9)

for non-termination tokens. When α = 0, the target reduces to the original-image teacher distribution renormalized over St (β). When β = 0, shaping is applied over the full vocabulary. In VCSD, “contrastive” therefore refers to the contrast between two conditional token distributions, rather than an embedding-level contrastive objective based on positive and negative sample pairs. 3.4

O N -P OLICY D ISTILLATION AND EMA U PDATE

The contrast-shaped target is distilled into the student at every position of the student-generated response using full-distribution forward KL:  2 LV CSD = TKD E (P,J)∼D  y∼πθ (·|P,J)

|y| 1 X

|y| t=1

 DKL (sg [qt⋆ ] ∥ pθ,t ) ,

(10)

where sg[·] denotes stop-gradient. All teacher and student distributions used for target construction and distillation are evaluated with the same distillation temperature TKD ; its dependence is omit2 ted from the notation for clarity. The factor TKD follows standard temperature-scaled knowledge distillation. Gradients pass only through the student distribution pθ,t . The two teacher evaluations are performed on the same fixed student-generated prefix and do not produce separate response trajectories. After each student update, the teacher parameters are updated by ϕ ← µϕ + (1 − µ)θ, where µ ∈ [0, 1) is the EMA decay coefficient. 5

Training requires only the original prompt–image pair and the student’s own on-policy response. It does not consume an external teacher, privileged answer, reasoning trace, evidence-focused crop, external verifier, or verifiable reward. At inference time, only the updated student is retained; neither the EMA teacher nor the content-erased control introduces an additional inference-time branch. 3.5

T HEORETICAL P ERSPECTIVE

The conditional log-ratio in Equation 6 admits two complementary interpretations. First, it can be viewed as a dense token-level reward measuring the predictive support attributable to instancespecific visual content. Second, it provides a controlled approximation to the conditional pointwise mutual information between a candidate token and the observed image, with the content-erased prediction serving as a surrogate for the prediction without instance-specific visual information. One-Step KL-Regularized Policy Update. Let the original-image teacher distribution restricted and renormalized over the plausibility support be 1[v ∈ St (β)] pJϕ,t (v) p̄Jϕ,t (v) = P . J u∈St (β) pϕ,t (u)

(11)

We regard the conditioning contrast e t (v) rtvis (v) := ∆

(12)

as an implicit visual-evidence reward. Remark 1. For each student-generated prefix, the contrast-shaped target in Equation 8 is the unique solution to the following optimization problem: qt⋆ = arg

max q∈Π(St (β))

  
αEv∼q rtvis (v) − DKL q ∥ p̄Jϕ,t .

(13)

Here, Π(St (β)) denotes the probability simplex over St (β). The closed-form solution is

p̄Jϕ,t (v) exp αrtvis (v)
. qt⋆ (v) = P J vis u∈St (β) p̄ϕ,t (u) exp αrt (u)

(14)

Because the support-normalization constant in p̄Jϕ,t cancels between the numerator and denominator, Equation 14 is exactly equivalent to the target defined in Equation 8. The original-image prediction therefore acts as the reference policy, while the conditioning log-ratio provides the reward used to improve it. The plausibility restriction further imposes a hard support constraint, preventing tokens with large likelihood ratios but negligible original-image probability from dominating the improved target. A complete derivation is provided in Appendix A. Here, “one-step” refers to a single closed-form update from the support-normalized original-image teacher distribution p̄Jϕ,t to the locally improved target qt⋆ at a fixed response prefix. It does not refer to a single gradient step of student optimization. Importantly, this interpretation does not assume that the original-image policy was historically obtained by applying reinforcement learning to the control-image policy. Rather, the conditional logratio defines an implicit reward whose KL-regularized optimum coincides with our contrast-shaped target. Approximation to Conditional Pointwise Mutual Information. Let Ht = (P, y<t ) denote the textual context at step t. The conditional pointwise mutual information between token v and image J, conditioned on Ht , takes the form PMI(v; J | Ht ) = log 6

p(v | J, Ht ) . p(v | Ht )

(15)

Our conditioning contrast instead takes the form ∆t (v) = log

pϕ (v | J, Ht ) . pϕ (v | Jctrl , Ht )

(16)

When the content-erased prediction pϕ (v | Jctrl , Ht ) approximates the model’s prediction without instance-specific visual information, it serves as a controlled surrogate for pϕ (v | Ht ). Under this interpretation, ∆t (v) is a contrastive approximation to conditional pointwise mutual information: it measures how much the observed image changes the token’s log-likelihood beyond the contentindependent prediction. Positive values identify tokens whose likelihood increases in the presence of the observed visual content, whereas values near zero identify tokens whose likelihood is largely e t (v) follows this interpreserved after that content is removed. The guarded reward rtvis (v) = ∆ pretation for ordinary tokens, while designated termination tokens are excluded from contrastive adjustment for training stability. This shows why the proposed objective is useful. Maximizing conditional mutual information favors predictions that remain informative about the observed image after accounting for the textual context, rather than predictions that can be explained primarily by language priors. Accordingly, upweighting tokens with large positive contrast encourages the student to preserve image-dependent evidence in its next-token distribution, while leaving visually insensitive tokens largely unchanged. Combined with the original-image teacher distribution as the reference policy, this yields a conservative form of visual grounding: the method amplifies visually supported distinctions only among tokens that the teacher already considers plausible, instead of rewarding arbitrary image-sensitive outputs. The resulting target therefore promotes stronger dependence on instance-specific visual evidence while retaining the fluency and semantic knowledge encoded by the teacher.

4

E XPERIMENTS

4.1

S ETUP

Models and training data. We evaluate VCSD on Qwen3-VL (Bai et al., 2025) at 2B, 4B, and 8B and on Qwen3.5 (Qwen Team, 2026) at 2B, 4B, and 9B. All post-training methods are trained on the ViRL39K single-image dataset (Wang et al., 2026a). Baselines. Each model block compares the unmodified base model, the published answer-hint OPSD method (Zhao et al., 2026), and VCSD. The OPSD target model is conditioned on the reference answer while the student receives only the original question and image. VCSD retains the on-policy rollout structure but constructs its target from the paired original and content-erased image conditions without consuming the answer. Benchmarks and aggregate metric. We evaluate general visual perception with BLINK (Fu et al., 2024) and MMStar (Chen et al., 2024), visual mathematics with MathVista (Lu et al., 2024), fine-grained and high-resolution perception with V∗ Bench (Wu & Xie, 2023) and HRBench4K/8K (Wang et al., 2025b), and hallucination with HallusionBench (Guan et al., 2024). The reported Acc is the unweighted mean over these seven benchmarks; HallusionBench entry first averages aAcc, fAcc, and qAcc. Training configuration. Unless otherwise stated, VCSD uses α = 1.0, β = 0.1, and TKD = 2, with uniform response-position weights. The EMA target uses an update rate of ρ = 0.05. We optimize the full-vocabulary forward-KL objective using AdamW, a batch of 32 prompts with n = 8 rollouts per prompt, a learning rate of 2 × 10−6 , and 10 warmup steps followed by a constant learning rate. All models use a fixed budget of 90 optimization steps on 8 NVIDIA B200 GPUs. 4.2

M AIN RESULTS

Table 1 reports the primary comparison across two Qwen model families and three scales per family. VCSD improves over answer-hint OPSD across visual capabilities. On Qwen3-VL-2B, VCSD improves every benchmark over OPSD: BLINK +1.27%, MMStar +3.00%, V∗ Bench +2.09%, MathVista +1.40%, HRBench4K +1.00%, HRBench8K +2.00%, and HallusionBench +4.32%. 7

Table 1: Main results on seven benchmarks. We compare the base model, OPSD, and VCSD across three Qwen3-VL and three Qwen3.5 model scales. Acc. denotes the average accuracy over the seven evaluation benchmarks. VCSD consistently outperforms both the base model and OPSD, improving the average accuracy by +1.86% to +4.77% over the corresponding base models while achieving the best overall performance across nearly all model scales. The highest score in each column is shown in bold. Method

BLINK

MMStar

V∗

MathVista

HR4K

HR8K

HalluB

Acc.

Qwen3-VL-2B

Base OPSD VCSD (ours)

53.02 56.02 57.29

57.47 60.73 63.73

72.77 75.92 78.01

62.50 64.70 66.10

71.13 76.25 77.25

67.38 71.25 73.25

51.60 49.36 53.68

62.27 64.89 67.04 (+4.77)

Qwen3-VL-4B

Base OPSD VCSD (ours)

67.18 64.97 66.86

68.93 68.73 68.73

80.63 83.77 83.77

73.90 74.10 74.90

79.88 78.12 80.50

73.75 75.63 77.00

54.81 54.50 60.37

71.30 71.40 73.16 (+1.86)

Qwen3-VL-8B

Base OPSD VCSD (ours)

69.65 67.81 70.38

70.67 70.20 74.07

82.72 85.34 87.43

76.50 77.30 77.90

77.38 81.12 84.12

71.12 74.62 79.25

59.51 59.64 60.69

72.51 73.72 76.26 (+3.75)

Qwen3.5-2B

Base OPSD VCSD (ours)

59.02 59.44 64.86

67.67 67.00 69.40

83.25 79.58 83.77

74.10 69.70 76.50

73.75 75.00 78.50

72.25 68.75 70.75

50.22 43.76 56.77

68.61 66.18 71.51 (+2.90)

Qwen3.5-4B

Base OPSD VCSD (ours)

64.97 64.44 66.70

73.00 71.40 75.00

82.20 84.29 84.82

81.70 77.60 82.20

81.38 82.75 85.38

74.00 78.75 79.38

60.36 58.22 63.95

73.94 73.92 76.77 (+2.83)

Qwen3.5-9B

Base OPSD VCSD (ours)

66.70 67.65 72.23

74.60 72.27 78.87

83.25 87.96 85.86

81.60 78.80 85.00

82.38 81.50 87.00

77.38 78.12 81.50

58.89 56.80 64.19

74.97 74.73 79.24 (+4.27)

(a) Plausibility support

65.0 62.5 60.0 57.5

=0 = 0.1 (ours)

30

60

90 120 Training step

(b) Contrastive strength

68 Seven-benchmark Acc

Seven-benchmark Acc

67.5

150

66 64 62 60

0.00

0.50 1.001.251.50 Contrastive strength

2.00

Drift reduced by anchor (\%)

Model

10 8 6 4 2 0

(c) Anchor reduces language drift

10

30

50 70 Training step

90

Figure 3: Ablation studies on Qwen3-VL-2B. (a) Effect of plausibility support. Restricting target shaping to the plausible support stabilizes long-horizon self-distillation, whereas removing it leads to progressive performance degradation. (b) Effect of contrastive strength. Performance is strongest and locally robust for α ∈ [1, 1.5], while weaker or stronger shaping reduces accuracy. (c) Effect of original-image anchor. Preserving the original-image distribution as the anchor substantially reduces language drift during training.

This yields a +2.15% aggregate gain, with improvements spanning general perception, visual mathematics, fine-grained and high-resolution perception, and hallucination rather than being driven by a single benchmark. The advantage persists across model families and scales. Across all six configurations, VCSD achieves the highest aggregate accuracy. At the largest tested scale in each family, it remains +2.54% above OPSD on Qwen3-VL-8B and +4.51% above it on Qwen3.5-9B. The contrast is especially clear on Qwen3.5, where OPSD provides no consistent gain over the base models, whereas VCSD improves every tested scale. Together, these results indicate that input-conditioned asymmetry provides a more effective self-distillation signal than privileged answer conditioning under the evaluated setting: paired predictions under matched visual conditions are sufficient to drive selfimprovement without answer or evidence hints. 8

Table 2: Ablation on the distillation divergence. We compare forward KL, reverse KL, and JSD on Qwen3-VL-2B. Forward KL consistently performs best, supporting our choice of a mode-covering objective for distilling the contrast-shaped target.

4.3

Divergence

BLINK

MMStar

V∗

MathVista

HR4K

HR8K

HalluB

Acc.

forward KL JSD reverse KL

57.29 57.08 56.92

63.73 62.33 61.07

78.01 76.44 76.44

66.10 65.70 64.00

77.25 75.37 73.62

73.25 73.00 70.50

53.68 53.86 50.86

67.04 66.25 64.77

E FFECT OF PLAUSIBILITY RESTRICTION

Plausibility support limits recursive target distortion. Figure 3 (a) compares seven-benchmark accuracy with and without plausibility support over extended training. The conditioning contrast can strongly favor a token whose probability changes substantially between the two visual conditions, even when that token remains unlikely under the original image. Restricting target shaping to the relative support of the original-image prediction ensures that the contrast redistributes probability only among candidates supported by the observed input. While both variants perform similarly during the early stage of training, the unrestricted target (β = 0) deteriorates steadily with continued self-distillation, whereas plausibility support (β = 0.1) maintains stable performance. Because successive targets are recursively constructed from the updated model, this growing divergence suggests that plausibility support mitigates the accumulation of target distortions across self-distillation updates. 4.4

E FFECT OF CONTRASTIVE STRENGTH

Moderate contrastive shaping is effective and locally robust. Figure 3 (b) shows that performance peaks around α = 1 and varies by less than 1% across α ∈ [1, 1.5]. Setting α = 0 removes the conditioning contrast while leaving the rest of the target construction unchanged; relative to α = 1, seven-benchmark accuracy decreases by 2.33%. Conversely, increasing the strength to α = 2 reduces performance to approximately the no-contrast level. Together, these results isolate the contribution of contrastive shaping and show that it is insensitive to the precise value within a moderate range, but degrades when it overwhelms the original-image target. 4.5

E FFECT OF D ISTILLATION D IVERGENCE

Forward KL most effectively distills the contrast-shaped target. Table 2 compares forward KL, reverse KL, and JSD while holding the contrast-shaped target and all other training settings fixed. Forward KL achieves the highest aggregate accuracy at 67.04%, outperforming JSD by 0.79% and reverse KL by 2.27%, and ranks first on six of the seven benchmarks. This pattern suggests that preserving coverage of the full target distribution is beneficial when distilling a visually contrastshaped teacher target. 4.6

E FFECT OF C ONTROL -I MAGE C ONSTRUCTION

The contrastive target is insensitive to the exact control construction. Table 3 compares black, Gaussian-noise, Gaussian-blur, and no-image controls, which remove instance-specific visual content through different interventions. The four variants achieve similar aggregate accuracy despite producing qualitatively different control inputs. Their shared property is the removal of instancespecific image content, suggesting that content removal, rather than a particular degradation process, is the key requirement for constructing the reference prediction. 4.7

E FFECT OF O RIGINAL -I MAGE A NCHOR

To isolate the role of the original-image anchor, we introduce an anchor coefficient λ into the target score: (λ) e t (v) = (λ + α) log pJ (v) − α log p0 (v), at (v) = λ log pJϕ,t (v) + α∆ ϕ,t ϕ,t

9

v ∈ St (β).

(17)

Table 3: Ablation on the control-image construction. We compare several content-erased control images, including a black image (ours), Gaussian noise, Gaussian blur, and no image. All variants achieve similar average accuracy, suggesting that the contrastive signal is robust to the choice of control image. Config

BLINK

MMStar

V∗

MathVista

HR4K

HR8K

HalluB

Acc.

Gaussian noise Black (ours) Gaussian blur No image

58.23 57.29 55.65 57.23

63.60 63.73 62.87 62.93

76.44 78.01 78.01 74.87

67.20 66.10 66.00 63.10

76.38 77.25 76.88 76.38

73.25 73.25 73.88 72.88

54.87 53.68 51.25 56.31

67.14 67.04 66.36 66.24

Table 4: Ablation of the original-image anchor. Per-benchmark results on Qwen3-VL-2B; Acc. averages the seven benchmarks. Config

BLINK

MMStar

V∗

MathVista

HR4K

HR8K

HalluB

Acc.

No anchor VCSD (ours)

58.71 57.29

62.93 63.73

75.92 78.01

67.00 66.10

76.38 77.25

73.12 73.25

53.43 53.68

66.78 67.04

The full method uses λ = 1, recovering Eq. 9, whereas the no-anchor variant sets λ = 0 while retaining the same plausibility support. With α = 1, its target score becomes log pJϕ,t (v) − log p0ϕ,t (v). The anchor primarily regularizes generation rather than improving aggregate accuracy. Table 4 shows comparable seven-benchmark accuracy with and without the anchor, suggesting that the capability gain comes mainly from contrastive shaping. We define language drift as the fraction of rollouts containing non-target-language tokens. Figure 3 (c) shows that the anchor reduces this drift throughout training, with the largest reductions at intermediate checkpoints. By keeping the shaped target closer to the original prediction, the anchor improves language consistency while leaving aggregate accuracy largely unchanged. 4.8

T RAINING DYNAMICS

VCSD maintains a consistent advantage and is less susceptible to late-stage degradation. Figure 4 compares VCSD and OPSD on the seven-benchmark aggregate and on MMStar and MathVista within the same run. On the aggregate, VCSD stays above OPSD at every evaluated training step and shows less late-stage degradation. After the first evaluated step, VCSD remains ahead on MMStar, while on MathVista it is higher throughout training. Its accuracy also varies within a narrower range, whereas OPSD exhibits more pronounced late-stage degradation, particularly on MathVista. These trajectories suggest that VCSD is more robust to continued training under the evaluated setting. 4.9

Q UALITATIVE A NALYSIS

MathVista provides a concrete setting for examining whether the aggregate improvement reflects more accurate use of image evidence. The case study localizes the decisive correction to visual counting. Figure 5 presents a base-ten block question whose answer depends on the number of thousand-cubes. The base model misinterprets the place-value blocks and predicts 6169, whereas OPSD overestimates the number of thousand-cubes and predicts 8519. VCSD correctly identifies seven thousand-cubes (six in the top row and one below), producing the correct answer of 7519. Once this image-grounded quantity is recovered, the remaining place-value calculation is direct. This example is consistent with the conditioning contrast shifting the self-distillation target toward candidates supported by instance-specific visual content. Figure 6 visualizes the learned image-dependent contrast score ∆(v) = log p(v | I)−log p(v | Ictrl ) for the same generated response. VCSD concentrates additional contrast on image-grounded evidence. The base model and OPSD exhibit highly similar contrast patterns, whereas VCSD assigns stronger positive contrast to visual concepts such as roof, shingles, food, beige, and sign, while suppressing less informative tokens. The bottom row isolates the additional contrast learned over OPSD and shows that this change is localized to semantically meaningful visual concepts rather than distributed uniformly across the sequence. This pattern supports the hypothesis that contrastive target 10

Seven-benchmark

10

ours OPSD 50

MMStar

64 62

66

60

64

58 56

90 Training step

130

MathVista

Accuracy

Accuracy

Accuracy

67 66 65 64 63 62

10

ours OPSD 50

62 60

90 Training step

130

10

ours OPSD 50

90 Training step

130

Figure 4: Training dynamics of VCSD and OPSD. Seven-benchmark aggregate accuracy (left) and per-benchmark accuracy on MMStar (middle) and MathVista (right) over the same training run. Qwen3.5-2B Base predicts 6169 (wrong) Top row: 6 large cubes → 6,000. Middle: one medium cube, representing 100 → +100. Bottom: 5 flats, 1 rod, 9 units → 69. Total = 6,169. OPSD predicts 8519 (wrong) Large cubes (thousands): 7 at the top and 1 below, so 7 + 1 = 8 thousands → 8,000. Then 5 flats (500), 1 rod (10), 9 units (9). Total = 8,519.

Q: What number is shown?

VCSD (ours) predicts 7519 (correct) The top row has 6 large cubes and 1 more below → 7 thousand-cubes (7,000). Plus 5 flats (500), 1 rod (10), and 9 units (9). Total = 7,519.

Ground truth: 7519

Figure 5: Qualitative comparison on MathVista (Qwen3.5-2B). The decisive visual quantity is the number of thousand-cubes. Red marks the miscounts made by the base and OPSD models; green marks the correct count used by VCSD. shaping reallocates probability mass toward image-dependent tokens instead of merely amplifying existing language preferences.

5

C ONCLUSION

We introduced VCSD, which constructs the target asymmetry required for on-policy self-distillation directly from matched visual conditioning. At each student-generated prefix, the EMA teacher is evaluated under the original image and a content-erased control. Their token-wise log-probability contrast sharpens the original-image distribution within its plausible support, producing an informative full-distribution target without an external teacher, privileged answers, and visual evidence signals. Across Qwen3-VL and Qwen3.5 models, VCSD consistently improves vision-language performance over both the corresponding base models and matched OPSD baselines. These results show that changes in a model’s own conditional predictions can provide an effective source of supervision for self-distillation.

R EFERENCES Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from selfgenerated mistakes. In International Conference on Learning Representations, volume 2024, pp. 21246–21263, 2024. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. Walid Bousselham, Hilde Kuehne, and Cordelia Schmid. Vold: Reasoning transfer from llms to vision-language models via on-policy distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26209–26218, 2026. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems, 37:27056–27087, 2024. 11

Figure 6: Token-level image-dependent contrast. Contrast scores for the same response under the base model, OPSD, and VCSD. Warmer colors indicate stronger image-dependent preference, and the bottom row shows the additional contrast learned by VCSD over OPSD.

Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, Zhilong Zhang, Zheng Jiang, Bingxiang He, Wei-Ying Ma, Ya-Qin Zhang, and Hao Zhou. Weak-to-strong generalization via direct on-policy distillation. arXiv preprint arXiv:2607.05394, 2026. Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp. 148–166. Springer, 2024. Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14375–14385, 2024. Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, et al. Reinforcement learning via self-distillation. arXiv preprint arXiv:2601.20802, 2026. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13872–13882, 2024. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 12286–12312, 2023. Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huanang Gao, Wenkai Yang, Zhiyuan Liu, et al. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe. arXiv preprint arXiv:2604.13016, 2026. 12

Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, et al. Visual-advantage on-policy distillation for vision-language models. arXiv preprint arXiv:2605.21924, 2026. Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, KaiWei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations, volume 2024, pp. 23439–23554, 2024. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https://qwen. ai/blog?id=qwen3.5. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning. arXiv preprint arXiv:2601.19897, 2026. Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv, and Tao Wang. V-zero: Answer-label-free on-policy distillation with contrastive evidence gating for fine-grained visual reasoning. arXiv preprint arXiv:2606.25319, 2026. Kanghui Tian, Siyuan Liu, Ziang Yan, Sheng Xia, Shuai Dong, and Yi Wang. Vicur: Visual cues as recoverable privilege for multimodal on-policy distillation. arXiv preprint arXiv:2606.05718, 2026. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. Advances in Neural Information Processing Systems, 38:30865–30891, 2026a. Kaishen Wang, Hengrui Gu, Meijun Gao, and Kaixiong Zhou. Damo: Decoding by accumulating activations momentum for mitigating hallucinations in vision-language models. In The Thirteenth International Conference on Learning Representations, 2025a. Kaishen Wang, Tong Zheng, Xuehao Cui, Ruibo Chen, Tianyi Xiong, and Heng Huang. Mitigating factual hallucination in large reasoning models via mixed-mode advantage regularization. arXiv preprint arXiv:2607.05861, 2026b. Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7907–7915, 2025b. Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. arXiv preprint arXiv:2312.14135, 2023. Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr. arXiv preprint arXiv:2604.03128, 2026. Hee Suk Yoon, Eunseop Yoon, Jaehyun Jang, SooHwan Eom, Ji Woo Hong, Mark HasegawaJohnson, Qi Dai, Chong Luo, and Chang D Yoo. Decomposed on-policy distillation for visionlanguage reasoning: Steering gradients for visual grounding. arXiv preprint arXiv:2606.00564, 2026. Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation. arXiv preprint arXiv:2605.18740, 2026. 13

Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734, 2026. Wenhong Zhu, Zhiwei He, Xiaofeng Wang, Pengfei Liu, and Rui Wang. Weak-to-strong preference optimization: Stealing reward from weak aligned model. In International Conference on Learning Representations, volume 2025, pp. 37116–37144, 2025.

A

P ROOF OF R EMARK 1

We prove Remark 1 from Section 3.5. Fix a student-generated prefix y<t , and abbreviate the plausibility support as St = St (β). As in Equation 11, let 1[v ∈ St ] pJϕ,t (v) p̄Jϕ,t (v) = P J u∈St pϕ,t (u)

(18)

denote the original-image teacher distribution renormalized over St , and, as in Equation 12, let e t (v) rtvis (v) = ∆

(19)

denote the implicit visual-evidence reward. Consider the following optimization over Π(St ), the probability simplex over St : max Jt (q) := α

q∈Π(St )

X

q(v)rtvis (v) −

v∈St

X

q(v) log

v∈St

q(v) . p̄Jϕ,t (v)

(20)

Existence and uniqueness of the maximizer. Since the teacher distribution is a (temperaturescaled) softmax output, pJϕ,t (v) > 0 for every v ∈ V; hence, for any β ∈ [0, 1], p̄Jϕ,t (v) > 0 for every v ∈ St . Under the convention 0 log 0 = 0, the objective Jt is therefore finite and continuous on Π(St ); since Π(St ) is compact, a maximizer exists. For uniqueness, decompose the objective as Jt (q) =

X

  X q(v) αrtvis (v) + log p̄Jϕ,t (v) − q(v) log q(v).

v∈St

(21)

v∈St

The first sum is linear in q, with finite coefficients because p̄Jϕ,t is strictly positive on St , while the P remaining term − v∈St q(v) log q(v) is the Shannon entropy of q, which is strictly concave on Π(St ). Hence Jt is strictly concave, and its maximizer is unique. Deriving the P candidate solution. We first solve the problem subject only to the normalization constraint v∈St q(v) = 1, and verify afterwards that the omitted nonnegativity constraints q(v) ≥ 0 are inactive at the resulting solution. Introduce a Lagrange multiplier λ for the normalization constraint:

F(q, λ) = α

X v∈St

q(v)rtvis (v) −

q(v) +λ q(v) log J p̄ϕ,t (v) v∈S

!

X

X

t

v∈St

q(v) − 1 .

(22)

For every v ∈ St , the stationarity condition is ∂F q(v) = αrtvis (v) − log J − 1 + λ = 0. ∂q(v) p̄ϕ,t (v) Rearranging gives 14

(23)

q(v) = αrtvis (v) + λ − 1, p̄Jϕ,t (v)

log

(24)

and hence
q(v) = p̄Jϕ,t (v) exp αrtvis (v) exp(λ − 1).

(25)

The factor exp(λ− 1) is shared across all candidates and is determined by normalization. Therefore,

p̄Jϕ,t (v) exp αrtvis (v)
. qt⋆ (v) = P J vis u∈St p̄ϕ,t (u) exp αrt (u)

(26)

Verifying global optimality. Every factor on the right-hand side of Equation 26 is strictly positive, so qt⋆ (v) > 0 for all v ∈ St , and the nonnegativity constraints omitted above are strictly satisfied and hence inactive at qt⋆ . Together with the stationarity condition in Equation 23 and primal feasibility, qt⋆ therefore satisfies the Karush–Kuhn–Tucker (KKT) conditions of the problem in Equation 20, with zero multipliers on the inactive nonnegativity constraints. Since the objective is concave and all constraints are affine, the KKT conditions are sufficient for global optimality, so qt⋆ is the maximizer of Equation 20; by the strict concavity established above, it is the unique one. Intuitively, no maximizer can place zero mass on any candidate in St , since ∂Jt /∂q(v) → +∞ as q(v) → 0+ . Equivalence with the contrast-shaped target. For v ∈ St , substituting p̄Jϕ,t (v) = P

pJϕ,t (v)

J u∈St pϕ,t (u)

(27)

into Equation 26, the support-normalization constant appears in both the numerator and denominator and therefore cancels. We obtain   e t (v) 1[v ∈ St ] pJϕ,t (v) exp α∆   . qt⋆ (v) = P J e u∈St pϕ,t (u) exp α∆t (u)

(28)

This is exactly the contrast-shaped target in Equation 8, which proves the result. For α > 0, an equivalent form of the objective is qt⋆ = arg max

q∈Π(St )



   1
Ev∼q rtvis (v) − DKL q ∥ p̄Jϕ,t . α

(29)

Thus, increasing α assigns greater weight to the visual-evidence reward relative to the KL regularizer, while the limit α → 0 recovers the support-normalized original-image teacher distribution.

15