arXiv CSopen access

Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners

Correcting What You Cannot See: Credit Assignment for Perception Distillation in Multimodal Reasoners

Feng Xiong* , Leyan Xue* , Hongyu Lin* *

Equal contribution

arXiv:2607.28336v1 [cs.AI] 30 Jul 2026

Abstract On-policy distillation provides dense supervision for multimodal reasoners, but its trajectory-level reward cannot determine whether a failed answer arose from perception or subsequent reasoning. Perception Success Rate (PSR), estimated from multiple reasonings sharing one perception, remains ambiguous because low success conflates perceptual insufficiency with reasoning difficulty. We introduce Perception-Correction Distillation (PCD), a label-free method that identifies correctable perception failures using downstream failure and teacher–student disf forms a agreement as complementary witnesses. Their product, (1 − PSR)KL, soft AND gate that strengthens distillation only when both witnesses are present. We motivate this rule through Bayesian evidence combination and show that multiplication is the unique normalized bilinear gate that vanishes when either witness is absent. PCD uses separated perception–reasoning rollouts and mean-preserving weights, leaving the reasoning objective unchanged. Across eight benchmarks, PCD improves the 8B→2B macro average from 44.50 with OPD to 47.28 and the 32B→8B result from 56.94 to 61.22. In matched 2B ablations, removing PCD and separated rollout reduces held-out average by 2.22 and 0.88 points, respectively. Effective multimodal distillation therefore depends not only on what the teacher predicts, but also on identifying when perception is the appropriate target of correction.

1

Introduction

Multimodal large language models (MLLMs) must first extract evidence from an image and then reason over it. Deployable students often acquire these capabilities through knowledge distillation Hinton et al. [2015], Furlanello et al. [2018], Beyer et al. [2022]: a larger teacher supplies richer targets than task labels alone. For generated language, sequence-level and rationale distillation transfer complete outputs or intermediate explanations Kim and Rush [2016], Hsieh et al. [2023]. On-policy distillation (OPD) instead evaluates trajectories sampled by the current student, aligning teacher supervision with the states the student actually visits Gu et al. [2024], Agarwal et al. [2024], Lu and Thinking Machines Lab [2025]. This property is attractive for visual reasoning because the student’s own visual mistakes become training examples. Yet a multimodal response is not a homogeneous token sequence. A short span may record the decisive observation—a marked angle, spatial relation, or chart value—while a much longer span derives the answer. Visual chain-of-thought and multimodal-rationale methods increasingly expose this separation Chen et al. [2024d], Shao et al. [2024a], He et al. [2024], Wang et al. [2024b], Cheng et al. [2025]. It reveals two known mismatches. First, token dilution lets long derivations dominate the few image-dependent tokens; VPPO addresses this with token-level visual focusing Huang et al. [2026]. Second, objective mismatch suggests imitating a strong visual teacher for perception while allowing reasoning to explore under verifiable reward Shao et al. [2024b], Yu et al. [2025], 40th Conference on Neural Information Processing Systems (NeurIPS 2026).

Figure 1: Trajectory reward maps perception and reasoning failures to the same outcome. PCD adds teacher disagreement as a second witness on the perception span. DeepSeek-AI [2025]. We therefore use an <aware> perception span trained by distillation and a <cot> reasoning span trained by RL. This decomposition exposes a third, unresolved mismatch: trajectory-level credit ambiguity. A verifier scores only the completed response. The same zero reward can follow either a misread image or an incorrect derivation from an adequate observation (Figure 1). These failures require different updates: the former warrants stronger perception correction, whereas the latter should primarily change reasoning. Outcome-based RL cannot distinguish them, and uniform OPD corrects every perception regardless of whether perception caused the failure. Existing credit signals do not recover the missing attribution. PPO, GRPO, and DAPO assign outcome-based advantages to trajectories rather than causal stages Schulman et al. [2017], Shao et al. [2024b], Yu et al. [2025]. Process reward models localize errors within derivations but require step supervision or an additional verifier, and normally judge reasoning rather than its visual premise Lightman et al. [2024], Wang et al. [2024c, 2025]. Delayed-return decomposition can move reward toward earlier actions Arjona-Medina et al. [2019], but cannot reconstruct information absent from the outcome: whether the initial observation was sufficient. A natural baseline samples several reasonings from one perception and uses their mean reward as a Perception Success Rate (PSR). PSR estimates downstream value under the current reasoner, but not perception quality. Low PSR may indicate a bad perception, a difficult problem, or weak reasoning. Additional samples reduce variance around this confounded quantity without identifying its cause. Consequently, success-only weighting may suppress teacher supervision for a correct perception merely because its derivations failed. We ask instead: which failed perceptions are both plausibly deficient and teacher-correctable? We combine downstream failure, 1 − PSR, with student–teacher disagreement on the perception span, f KL: f i. di = (1 − PSRi )KL (1) This product is a conservative soft AND gate. Neither failure alone nor disagreement alone triggers strong correction; only their conjunction does. We derive this interaction from conditionally independent likelihood-ratio witnesses and show that multiplication is the unique normalized bilinear gate that vanishes when either witness is absent. The resulting method, Perception-Correction Distillation (PCD), first samples a perceptions and then b reasoning continuations from each fixed perception. This separated rollout makes perception the unit of estimation. PCD applies a mean-preserving weight only to perception distillation, thereby reallocating a fixed teacher-supervision budget while leaving the reasoning objective and grouprelative advantages unchanged. It requires no perception labels, learned gate, or additional model beyond the teacher already used by OPD. 2

Our contributions are: • We formulate perception distillation as an identifiability problem and prove that rewardonly PSR cannot separate perception sufficiency from reasoning difficulty. • We derive a two-witness multiplicative deficiency score and show that its mean-normalized weighting follows the optimal first-order reallocation direction under a fixed supervision budget. • We introduce separated perception–reasoning rollout, distilling perception while optimizing reasoning with verifiable-reward RL. • Across eight benchmarks, PCD reaches 47.28 for 8B→2B and 61.22 for 32B→8B transfer; matched ablations lose 2.22 points without PCD weighting and 0.88 without separated rollout.

2

Related Work

Knowledge and on-policy distillation. Knowledge distillation transfers a teacher’s predictive distribution to a smaller student Hinton et al. [2015], Beyer et al. [2022]; its benefits can also persist without a capacity reduction Furlanello et al. [2018]. Sequence- and rationale-level methods train on teacher-generated outputs Kim and Rush [2016], Hsieh et al. [2023], whereas recent language-model methods reduce distribution mismatch by learning from student-generated trajectories. MiniLLM minimizes reverse KL on student samples Gu et al. [2024]; generalized on-policy distillation studies alternative divergences and student sampling Agarwal et al. [2024], Lu and Thinking Machines Lab [2025]. Policy distillation similarly transfers action distributions in RL Rusu et al. [2016], Czarnecki et al. [2019]. PCD retains the on-policy learner but changes how perception-span supervision is allocated across trajectories, rather than aligning heads, relational knowledge, or unequal visual-token spaces Zhao et al. [2024], Yang et al. [2025], Feng et al. [2026]. Multimodal perception and visual reasoning. Recent technical reports emphasize nativeresolution perception and test-time reasoning in open MLLMs Wang et al. [2024d], Bai et al. [2025b], Chen et al. [2024c], Bai et al. [2025a]. Visual CoT explicitly localizes relevant image regions before reasoning Shao et al. [2024a], illustrating that perception and reasoning can benefit from distinct intermediate representations. Related AAAI work aligns multimodal evidence with language thoughts in a latent reasoning space He et al. [2024], distills teacher-generated multimodal rationales into smaller models Wang et al. [2024b], and uses visual chain-of-thought prompting to select evidence for knowledge-based reasoning Chen et al. [2024d]; CoMT and KAM-CoT study longer or knowledge-grounded multimodal thought Cheng et al. [2025], Mondal et al. [2024]. VPPO Huang et al. [2026] measures token-level visual dependence and introduces Token Gradient Filtering and Trajectory Advantage Shaping. PCD is complementary: VPPO asks which tokens are visually grounded, while PCD asks which perception trajectories are plausible teacher-correctable failures. Their weights act at different granularities and can be multiplied. Reinforcement learning for reasoning. PPO Schulman et al. [2017] provides the clipped policyoptimization foundation used by many language-model RL systems. GRPO removes the learned critic through group-relative normalization Shao et al. [2024b], while DAPO adds stability and token-level optimization refinements Yu et al. [2025]; DeepSeek-R1 demonstrates the broader capability gains obtainable from verifiable-reward RL DeepSeek-AI [2025]. These methods optimize complete reasoning trajectories from outcome rewards. PCD does not replace their reasoning objective; it uses the same outcomes to decide where teacher supervision on the preceding perception is most useful. Process supervision and credit assignment. Outcome-only feedback cannot directly identify erroneous intermediate steps. Process reward models address this limitation with step-level supervision Lightman et al. [2024], and Math-Shepherd constructs such supervision automatically for mathematical reasoning Wang et al. [2024c]. VisualPRM extends this approach to multimodal process evaluation Wang et al. [2025], Zhou et al. [2026]. In general RL, RUDDER, STAS, and Latent Reward redistribute delayed feedback Arjona-Medina et al. [2019], Chen et al. [2024b], Qu et al. [2025]. PCD differs in target: it assigns credit from the final reasoning outcome back to a latent 3

Figure 2: Overview of the separated rollout, divided objectives, and PCD weighting.

perception choice, then uses that credit to reweight distillation rather than to create a new reward. Because outcome reward alone cannot identify the failure source, PCD adds teacher disagreement as an independent witness. Perception-aware weighting. Within a separated perception–reasoning rollout, a natural baseline is the Perception Success Rate (PSR): generate multiple reasonings from one perception and weight distillation by their mean success. PSR treats low-success perceptions as unreliable. PCD instead distinguishes two cases: teacher-aligned low-success perceptions receive no additional correction, whereas low-success perceptions that also disagree with the teacher are up-weighted. Thus PSR and PCD encode contrasting interpretations of unsuccessful trajectories and provide a direct test of whether teacher disagreement resolves reward-only credit ambiguity.

3

Credit Assignment and Perception-Correction Distillation

We first formalize the perception-credit problem and then present its implementation in PCD. The central latent variable is whether a sampled perception contains enough visual information for the current reasoner to solve the problem. 3.1

Problem Formulation

Given an image–question pair x, we factor the student policy into perception and reasoning stages, πθ (y, z | x) = πp (z | x)πr (y | x, z),

(2)

where z is enclosed by <aware> tags and y by <cot> tags. Let R(x, z, y) ∈ [0, 1] denote the verifiable reward of the completed response. The frozen teacher πT supplies top-k log probabilities on the student’s sampled tokens. Following on-policy distillation Agarwal et al. [2024], Gu et al. [2024], the masked loss is P mt ℓt LOPD = Pt , ℓt ≈ log πθ (rt ) − log πT (rt ). (3) t mt No ground-truth label is available for z. The estimation problem is therefore to infer which perceptions are insufficient from downstream rewards and teacher probabilities alone. 4

3.2

Separated Perception–Reasoning Rollout

For each input, we first sample a perceptions, zi ∼ πp (· | x),

i = 1, . . . , a,

(4)

and then sample b independent reasoning continuations conditioned on each fixed perception, yij ∼ πr (· | x, zi ),

j = 1, . . . , b.

(5)

The resulting a × b tree keeps the perception prefix shared within each group while allowing the reasoning to vary. All trajectories for an input remain one group for group-relative policy optimization Shao et al. [2024b], Yu et al. [2025], so this reorganization does not change the reasoning reward. Perception value.

The downstream value of z under the current reasoner is V (z) = Ey∼πr (·|x,z) [R(x, z, y)].

(6)

Its Monte Carlo estimate is the Perception Success Rate b

PSRi =

1X R(x, zi , yij ). b j=1

(7)

For Bernoulli rewards, this estimator is unbiased and has variance V (zi )(1 − V (zi ))/b. Sharing b continuations is therefore what converts a trajectory-level verifier into a lower-variance, perperception statistic. 3.3

Why Reward Alone Is Not Identifying

Let S(z) ∈ {0, 1} denote the latent event that z contains sufficient visual evidence. Define q(z) = Pr(S = 1 | z) and let ρ(x) be the current reasoner’s probability of success given sufficient evidence. If an insufficient perception rarely yields the correct answer, then V (z) = ρ(x)q(z) + ε(1 − q(z)) ≈ ρ(x)q(z),

ε ≈ 0.

(8)

Proposition 1 (non-identifiability of PSR). For any observed v = ρq ∈ (0, 1) and any q ∈ [v, 1], choosing ρ = v/q produces the same value v. Hence the reward sample identifies only the product of reasoning difficulty and perception sufficiency; no estimator based solely on {Rij } can distinguish a bad perception from a difficult reasoning problem. Proof. The Bernoulli sampling distribution of Eq. (7) depends on (ρ, q) only through V = ρq. Parameter pairs with equal products therefore induce identical observations, and no function of those observations can separate the pairs. □ This proposition explains the failure mode of success-only weighting. On a hard prompt, ρ can be small even when q is close to one. Attenuating distillation whenever PSR is low then removes supervision from a perception that may already be adequate. 3.4

Teacher Disagreement as a Second Witness

Following reverse-KL distillation Gu et al. [2024], we measure student–teacher disagreement on the aware span with a detached estimate, KLi = log πθold (t) − log πT (t) t∈zi ,

(9)

using the same top-k support and clamping convention as the distillation objective. The value is fi ∈ averaged across the b trajectories that share zi , clamped to be non-negative, and normalized to KL [0, 1]. Consider binary witnesses Ai = ⊮[PSRi is low] and Bi = ⊮[KLi is large]. We assume that, conditional on the sufficiency state S, the residual randomness of these witnesses is independent; that an insufficient perception is more likely to disagree with a competent teacher; and that it is more likely to have low downstream success. 5

Proposition 2 (Bayesian combination of binary witnesses). Under these assumptions, the posterior log-odds of insufficiency satisfy logit Pr(S = 0 | A, B) = logit Pr(S = 0) Pr(A | S = 0) + log (10) Pr(A | S = 1) Pr(B | S = 0) + log . Pr(B | S = 1) Thus the two likelihood ratios multiply in odds space. This result applies exactly to calibrated binary tests. In practice we use the continuous, bounded score f i. di = (1 − PSRi )KL (11) Proof. Conditional independence gives Pr(A, B | S) = Pr(A | S) Pr(B | S). Applying Bayes’ rule, taking the ratio between S = 0 and S = 1, and then taking logarithms yields Eq. (10). □ Scope of the assumptions. Conditional independence concerns the residual evidence in the two witnesses after conditioning on perception sufficiency; it does not assert that reward and model probabilities are marginally independent. Teacher competence is also essential: disagreement is evidence of a correctable student error only when the teacher is more reliable on the relevant visual evidence. Consequently, PCD estimates teacher-correctable deficiency, not semantic incorrectness in an absolute sense. If teacher and student share the same visual failure, their KL can be small and the error is intentionally not amplified. Equation (11) is not claimed to be a calibrated posterior probability without labeled insufficiency states. It is a conservative soft AND surrogate that preserves the desired boundary cases. High-PSR perceptions receive no boost regardless of KL. Low-PSR but teacher-aligned perceptions also receive no boost because the teacher offers little corrective information. Only low-PSR, high-KL perceptions receive strong additional supervision. This is the distinction that PSR alone cannot make. f Among Proposition 3 (uniqueness of the bilinear soft AND). Let a = 1 − PSR and b = KL. bilinear gates g(a, b) = c0 + c1 a + c2 b + c3 ab, the conditions g(a, 0) = 0, g(0, b) = 0, g(1, 1) = 1 (12) uniquely imply g(a, b) = ab. Proof. The first condition in Eq. (12) gives c0 = c1 = 0; the second then gives c2 = 0; normalization at (1, 1) gives c3 = 1. □ The first two conditions encode the central modeling decision: neither low success nor teacher disagreement alone should trigger correction. A normalized additive gate (a + b)/2 violates both conditions. Proposition 3 establishes multiplication within the simplest interaction model satisfying the desired AND semantics; it does not exclude richer nonlinear gates. Why not a learned gate? An MLP or attention module could represent richer interactions, but no labels identify which failures are perceptual. Training such a gate from the same trajectory reward would reintroduce the ambiguity of Proposition 1 and add a new credit-assignment problem. PCD instead uses a parameter-free interaction whose behavior is fixed before observing evaluation outcomes. Learned gates remain a useful extension when perception-level supervision is available. 3.5

Mean-Preserving Corrective Distillation

PCD converts deficiency into a positive perception weight, w̄i wi = 1 PN

w̄i = wbase + αdi ,

N

k=1 w̄k

,

(13)

where N is the number of perceptions in the optimization batch. The second equality preserves the mean weight at one. The aware-span loss is P aw i,j,t wi mijt ℓijt P Laware = . (14) aw i,j,t mijt 6

Algorithm 1: Perception-Correction Distillation Require: Prompt x, student (πp , πr ), teacher πT , rollout (a, b) 1: for i = 1, . . . , a do 2: Sample zi , then b continuations yij and rewards Rij P 3: Compute PSRi = b−1 j Rij and aware-span KLi f i ; w̄i ← wbase + αdi 4: di ← (1 − PSRi )KL 5: end for P 6: Normalize wi ← w̄i /(N −1 k w̄k ) 7: Distill perception with wi and optimize reasoning with DAPO

Budget P interpretation. Equation (13) is monotone in the deficiency score while satisfying N −1 i wi = 1. It therefore implements a controlled reallocation of a fixed supervision budget. We do not claim that this affine rule is the unique global optimum: a linear objective over an unconstrained simplex would concentrate all mass on one example. Instead, the positive base weight and normalization provide a bounded relaxation that preserves coverage of every perception. Without normalization, batches containing many deficient perceptions would also strengthen the overall imitation regularizer and confound selectivity with loss-scale changes. Proposition 4 (optimal first-order reallocation direction). Let δi = wi −1 be a mean-preserving P perturbation, so i δi = 0, and suppose the local benefit of additional teacher supervision is proportional to di . Among perturbations satisfying ∥δ∥2 ≤ B, the maximizer of the first-order benefit P i di δi is 1 X di − d¯ , d¯ = dk . (15) δi∗ = B qP N ¯2 (d − d) k

k

k

Moreover, Eq. (13) gives exactly

¯ α(di − d) , (16) wbase + αd¯ and therefore follows this optimal direction while its base weight controls the step magnitude and preserves coverage. wi − 1 =

¯ For every feasible δ, Cauchy– Proof. Projecting d onto the zero-mean subspace gives d − d1. ¯ ¯ Schwarz yields ⟨d, δ⟩ = ⟨d − d1, δ⟩ ≤ B∥d − d1∥2 , with equality for Eq. (15). Substituting the batch mean of w̄i = wbase + αdi into Eq. (13) yields Eq. (16). □ 3.6

Divided Objectives and Token-Level Composition

We optimize the perception and reasoning spans with different objectives, L = λaw Laware + λcot LDAPO . cot

(17)

PCD changes only Laware ; it does not alter group-relative advantages or the reasoning loss. If tokenlevel visual focusing Huang et al. [2026] supplies a saliency weight sijt , the aware-token contribution becomes wi sijt ℓijt . PCD asks whether a perception is a correctable failure, while visual focusing asks which tokens carry image-dependent evidence. Thus PCD requires only grouped reductions and a trajectory-wise multiplier on the existing awarespan loss. Stable perception identifiers preserve grouping after batch reordering; missing masks or teacher probabilities set deficiency to zero. Optional all-wrong exploration and complete failure handling are described in the supplementary material.

4

Experiments

We compare perception-specific correction with uniform on-policy distillation and reward-only posttraining in two transfer settings: Qwen3-VL-8B→2B and Qwen3-VL-32B→8B. The comparison includes the initial student, standard OPD, and PCD; the 2B block additionally includes a same-size DAPO baseline. 7

Student

Teacher

ID

Method

Near-OOD: Mathematical reasoning

OOD

Avg (macro)

Geo3K MathVerse MathVista MathVision We-Math LogicVista MMMU Pro MMStar – Base Qwen3-VL – DAPO 2B-Instruct 8B-Instruct OPD

21.28 37.83 30.82

30.33 37.03 33.14

59.92 63.49 61.55

19.74 25.25 20.19

57.33 60.50 59.78

45.23 42.91 44.84

45.06 46.69 45.78

57.77 59.11 59.92

42.08 46.60 44.50

PCD

42.12

37.72

62.45

24.01

62.44

42.89

47.59

59.06

47.28

– Base Qwen3-VL 32B-Instruct OPD 8B-Instruct 32B-Instruct PCD

41.99 42.51 59.65

45.15 44.97 50.10

73.91 74.50 75.79

34.54 34.33 39.88

71.98 72.19 73.92

58.79 58.51 58.57

59.34 59.05 61.45

69.66 69.44 70.43

56.92 56.94 61.22

8B-Instruct

Table 1: Avg@8 performance (%). Geo3K is the ID training benchmark; Near-OOD contains visualmathematical transfer tasks, and OOD contains more distant multimodal tasks. Avg is the unweighted eight-dataset macro mean; bold marks the best result per student size.

4.1

Experimental setup

Settings and baselines. We study Qwen3-VL-8B-Instruct to Qwen3-VL-2B-Instruct and Qwen3VL-32B-Instruct to Qwen3-VL-8B-Instruct Bai et al. [2025a]. BASE is the unmodified student, DAPO is the same-size reward-optimization baseline, OPD applies token-level teacher supervision uniformly, and PCD applies the separated rollout and deficiency weighting from Section 3. Training and evaluation. Training uses Geo3K Lu et al. [2021]. The teacher returns top-k probabilities with k = 64. PCD uses a separated a = 2, b = 4 rollout, giving eight completed trajectories per prompt. The loss coefficients are λaw = 0.1 and λcot = 1.0; PCD weights are normalized to unit mean. Checkpoints are selected using held-out validation performance. We divide the eight evaluation datasets into three groups. In-domain (ID) contains Geo3K, the dataset used for training. Near-OOD mathematical reasoning contains MathVerse, MathVista, MATH-Vision, and We-Math Zhang et al. [2024], Lu et al. [2024], Wang et al. [2024a], Qiao et al. [2025], which share the visual-mathematical task family without being the training benchmark. Outof-domain (OOD) evaluation contains LogicVista, MMMU Pro, and MMStar Xiao et al. [2024], Yue et al. [2025], Chen et al. [2024a]. These labels denote task-level proximity rather than example overlap. All datasets use the same prompting, decoding configuration, and AVG @8 evaluator with eight samples per question, temperature 1.0, and top-p = 1.0. For each question, Avg@8 is the arithmetic mean of the eight binary sample outcomes; dataset accuracy then averages these per-question values, so every question has equal weight regardless of answer length. The reported macro average assigns equal weight to each of the eight datasets rather than to each underlying question. This distinction matters because the benchmark sizes differ substantially. We report percentages throughout and do not interpret a single checkpoint difference as statistical significance. 4.2

Main results

Table 1 reports the eight-benchmark comparison. The first block holds student capacity fixed at 2B and compares transfer from the 8B teacher. The second block holds student capacity fixed at 8B and compares uniform OPD and PCD using the 32B teacher. PCD obtains the highest macro average among the 2B models: 47.28, compared with 46.60 for DAPO, 44.50 for OPD, and 42.08 for the initial student. It is best on Geo3K, MathVerse, We-Math, and MMMU Pro; OPD leads on LogicVista and MMStar, and DAPO on MathVista and MathVision. Thus PCD improves aggregate performance without claiming universal per-task dominance. In the 32B→8B setting, PCD reaches a 61.22 macro average, compared with 56.94 for OPD and 56.92 for the initial 8B student. It is best on seven of eight benchmarks; Base remains slightly stronger on LogicVista (58.79 vs. 58.57). The larger gain is consistent with a larger correctable teacher–student perception gap, but does not establish causality. Where does the gain occur? Relative to uniform OPD, the 2B PCD model improves Geo3K by 11.30 points and the four-dataset Near-OOD mean by 2.99 points, while its three-dataset OOD 8

Figure 3: PCD training weights over 256 Geo3K perceptions concentrate in the high-failure, highdisagreement quadrant. Method DAPO (pure RL) Standard OPD Full w/ VPPO-Distill Full w/o PCD weight Full w/o separated rollout Full (ours)

Geo3K (ID) MMMU Pro We-Math MathVista MathVerse MathVision MMStar Held-out avg ∆ vs. full 37.83 30.82 42.12 37.90 38.50 43.12

46.69 45.78 47.59 45.73 46.26 47.15

60.50 59.78 62.44 58.76 61.32 63.68

63.49 61.55 62.45 61.37 63.58 63.16

37.03 33.14 37.72 34.81 36.26 39.19

25.25 20.19 24.01 23.77 23.31 22.94

59.11 59.92 59.06 57.94 59.69 59.59

48.68 46.73 48.88 47.06 48.40 49.28

−0.60 −2.55 −0.40 −2.22 −0.88 –

Table 2: Matched 2B component ablation. Held-out avg is the unweighted mean over the six listed transfer datasets and excludes Geo3K. Full combines separated rollout, perception distillation, reasoning RL, and PCD weighting; bold identifies the full configuration.

mean changes by −0.33. Thus the 2B macro gain is not evidence of uniform OOD improvement; it is concentrated in visual mathematics, where the distilled perception interface matches training most closely. The 32B→8B transfer is broader: PCD improves Geo3K by 17.14, the Near-OOD mean by 3.43, and the OOD mean by 1.15 points over OPD. This scale-dependent pattern is consistent with stronger teachers providing more transferable perceptual corrections, but model size and teacher quality are confounded in this comparison. 4.3

Mechanism Analysis: Does PCD Implement the AND Gate?

Figure 3 maps 256 Geo3K perceptions by rollout failure (1 − PSR) and teacher–student gap on the aware span; color is the actual PCD weight used in training. The high-failure, high-gap quadrant has mean weight 1.36, versus 0.85–0.92 elsewhere. Failure alone (0.87) or disagreement alone (0.92) receives no comparable boost, matching the multiplicative interaction. This validates allocation behavior rather than a causal accuracy gain; Table 2 supplies the complementary endpoint evidence. 4.4

Component Ablation

Table 2 reports matched 2B ablations. Held-out avg averages MMMU Pro, We-Math, MathVista, MathVerse, MathVision, and MMStar; Geo3K is the training diagnostic and LogicVista was unavailable. Full combines separated rollout, complete perception-span distillation, reasoning RL, and PCD weighting. VPPO-Distill substitutes token-selective distillation; w/o PCD uses uniform perception weights; w/o separated rollout samples both stages jointly while retaining perception distillation and reasoning RL. 9

Full achieves the best held-out average (49.28). Uniformizing its PCD weight causes the largest internal drop (−2.22), linking the allocation pattern in Figure 3 to endpoint performance. Removing separated rollout costs 0.88, consistent with weaker perception-level PSR and credit assignment when continuations do not share a fixed perception. VPPO-Distill costs 0.40, suggesting that selecting only highly visual tokens can discard contextual or implicitly visual perception tokens. OPD trails by 2.55 and DAPO by 0.60; DAPO remains stronger on MathVision, so the evidence is aggregate rather than universal per benchmark. Fixed-budget interpretation. Because PCD normalizes the batch-average perception weight to one, its gain cannot be attributed to a larger global teacher-loss coefficient: deficient perceptions receive more supervision only by reducing it elsewhere. The w/o-PCD comparison is therefore a controlled allocation test—it retains the separated rollout and divided objectives, but loses 2.22 points when all perceptions are weighted uniformly. VPPO-Distill operates at a different granularity and loses 0.40: selecting only highly visual tokens may omit connective or implicitly visual tokens needed to express a coherent perception. These results motivate PCD as a trajectory-level allocator rather than another token mask, while leaving open whether a differently calibrated token selector could compose more effectively. 4.5

Limitations and Threats to Validity

Attribution. The ablation isolates PCD weighting, separated rollout, and VPPO-Distill, but lacks PSR-only, KL-only, and additive-fusion runs. Figure 3 verifies the interaction and w/o-PCD verifies adaptive weighting; matched single-witness runs are still needed to attribute the gain to multiplication. Uncertainty and non-stationarity. Each method uses one selected checkpoint. PSR is policydependent, and larger b reduces its conditional variance while exploring fewer perceptions under a fixed budget. Multi-seed intervals and sensitivity to (a, b) are therefore needed before treating small differences as stable. Teacher and KL calibration. Disagreement is useful only for a perceptually stronger teacher. Shared errors produce low KL, whereas a confidently wrong teacher can induce harmful correction. Because the normalization threshold controls KL saturation, it should be calibrated from trainingtrace quantiles for each teacher size. Scope and reproducibility. ID/Near-OOD/OOD are task descriptors rather than formal distances, and results cover one model family and training domain. Full implementation and failure handling are documented in the supplementary material.

5

Conclusion

PCD recasts perception distillation as credit assignment. Combining failure with teacher disagreement targets correctable perceptions while preserving the mean supervision budget. It improves OPD by 2.78 and 4.28 points for 2B and 8B students, with mechanism and ablation evidence supporting the proposed allocation. Multi-seed and single-witness controls remain future tests.

References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, et al. On-policy distillation of language models: Learning from self-generated mistakes. In ICLR, 2024. URL https://openreview.net/forum?id= 3zKtaqxLhW. Jose A. Arjona-Medina, Michael Gillhofer, Michael Widrich, Thomas Unterthiner, Johannes Brandstetter, and Sepp Hochreiter. RUDDER: Return decomposition for delayed rewards. In NeurIPS, volume 32, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/ 16105fb9cc614fc29e1bda00dab60d41-Abstract.html. Shuai Bai, Yuxuan Cai, Ruizhe Chen, et al. Qwen3-VL technical report. Technical Report arXiv:2511.21631, Qwen Team, 2025a. URL https://arxiv.org/abs/2511.21631. 10

Shuai Bai, Keqin Chen, Xuejing Liu, et al. Qwen2.5-VL technical report. Technical Report arXiv:2502.13923, Qwen Team, 2025b. URL https://arxiv.org/abs/2502.13923. Lucas Beyer, Xiaohua Zhai, Amélie Royer, et al. Knowledge distillation: A good teacher is patient and consistent. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10925–10934, 2022. URL https://openaccess.thecvf.com/content/CVPR2022/html/Beyer Knowledge Distillation A Good Teacher Is Patient and Consistent CVPR 2022 paper.html. Lin Chen, Jinsong Li, Xiaoyi Dong, et al. Are we on the right way for evaluating large visionlanguage models? In NeurIPS, volume 37, 2024a. doi: 10.48550/arXiv.2403.20330. URL https: //arxiv.org/abs/2403.20330. Sirui Chen, Zhaowei Zhang, Yaodong Yang, and Yali Du. STAS: Spatial-temporal return decomposition for solving sparse rewards problems in multi-agent reinforcement learning. AAAI, 38 (16):17337–17345, 2024b. doi: 10.1609/aaai.v38i16.29681. URL https://ojs.aaai.org/index.php/ AAAI/article/view/29681. Zhe Chen, Weiyun Wang, Yue Cao, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. Technical Report arXiv:2412.05271, OpenGVLab, 2024c. URL https://arxiv.org/abs/2412.05271. Zhenfang Chen, Qinhong Zhou, Yikang Shen, et al. Visual chain-of-thought prompting for knowledge-based visual reasoning. AAAI, 38(2):1254–1262, 2024d. doi: 10.1609/aaai.v38i2. 27888. URL https://ojs.aaai.org/index.php/AAAI/article/view/27888. Zihui Cheng, Qiguang Chen, Jin Zhang, et al. CoMT: A novel benchmark for chain of multi-modal thought on large vision-language models. AAAI, 39(22):23678–23686, 2025. doi: 10.1609/aaai. v39i22.34538. URL https://ojs.aaai.org/index.php/AAAI/article/view/34538. Wojciech M. Czarnecki, Razvan Pascanu, Simon Osindero, Siddhant M. Jayakumar, Grzegorz Swirszcz, and Max Jaderberg. Distilling policy distillation. In AISTATS, volume 89, pages 1331– 1340. PMLR, 2019. URL https://proceedings.mlr.press/v89/czarnecki19a.html. DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. doi: 10.48550/arXiv.2501.12948. URL https: //arxiv.org/abs/2501.12948. Ze Feng, Sen Yang, Boqiang Duan, Wankou Yang, and Jingdong Wang. EM-KD: Distilling efficient multimodal large language model with unbalanced vision tokens. AAAI, 40(25), 2026. doi: 10.1609/aaai.v40i25.39254. URL https://ojs.aaai.org/index.php/AAAI/article/view/39254. Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In ICML, volume 80, pages 1607–1616. PMLR, 2018. URL https: //proceedings.mlr.press/v80/furlanello18a.html. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge distillation of large language models. In ICLR, 2024. URL https://openreview.net/forum?id=5h0qf7IBZZ. Liqi He, Zuchao Li, Xiantao Cai, and Ping Wang. Multi-modal latent space learning for chain-ofthought reasoning in language models. AAAI, 38(16):18180–18187, 2024. doi: 10.1609/aaai. v38i16.29776. URL https://ojs.aaai.org/index.php/AAAI/article/view/29776. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. doi: 10.48550/arXiv.1503.02531. URL https://arxiv.org/abs/ 1503.02531. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of ACL, pages 8003–8017, 2023. doi: 10.18653/v1/2023.findings-acl.507. URL https://aclanthology.org/ 2023.findings-acl.507/. 11

Siyuan Huang, Xiaoye Qu, Yafu Li, et al. Spotlight on token perception for multimodal reinforcement learning. In ICLR, 2026. doi: 10.48550/arXiv.2510.09285. URL https://arxiv.org/abs/2510. 09285. Yoon Kim and Alexander M. Rush. Sequence-level knowledge distillation. In EMNLP, pages 1317– 1327, 2016. doi: 10.18653/v1/D16-1139. URL https://aclanthology.org/D16-1139/. Hunter Lightman, Vineet Kosaraju, Yura Burda, et al. Let’s verify step by step. In ICLR, 2024. URL https://openreview.net/forum?id=v8L0pN6EOi. Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20251026. URL https://thinkingmachines.ai/blog/on-policydistillation/. Pan Lu, Ran Gong, Shibiao Jiang, et al. Inter-GPS: Interpretable geometry problem solving with formal language and symbolic reasoning. In ACL-IJCNLP, pages 6774–6786, 2021. doi: 10. 18653/v1/2021.acl-long.528. URL https://aclanthology.org/2021.acl-long.528/. Pan Lu, Hritik Bansal, Tony Xia, et al. MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. In ICLR, 2024. URL https://openreview.net/forum?id= KUNzEQMWU7. Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, and Godawari Sudhakar Rao. KAM-CoT: Knowledge augmented multimodal chain-of-thoughts reasoning. AAAI, 38(17): 18798–18806, 2024. doi: 10.1609/aaai.v38i17.29844. URL https://ojs.aaai.org/index.php/AAAI/ article/view/29844. Runqi Qiao, Qiuna Tan, Guanting Dong, et al. We-Math: Does your large multimodal model achieve human-like mathematical reasoning? In ACL, pages 20023–20070, 2025. doi: 10.18653/v1/2025. acl-long.983. URL https://aclanthology.org/2025.acl-long.983/. Yun Qu, Yuhang Jiang, Boyuan Wang, Yixiu Mao, Cheems Wang, Chang Liu, and Xiangyang Ji. Latent reward: LLM-empowered credit assignment in episodic reinforcement learning. AAAI, 39 (19):20095–20103, 2025. doi: 10.1609/aaai.v39i19.34213. URL https://ojs.aaai.org/index.php/ AAAI/article/view/34213. Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. Policy distillation. In ICLR, 2016. URL https://arxiv.org/abs/1511.06295. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. doi: 10.48550/arXiv.1707. 06347. URL https://arxiv.org/abs/1707.06347. Hao Shao, Shengju Qian, Han Xiao, et al. Visual CoT: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In NeurIPS, volume 37, 2024a. doi: 10.52202/079017-0275. URL https: //proceedings.neurips.cc/paper files/paper/2024/hash/0ff38d72a2e0aa6dbe42de83a17b2223Abstract-Datasets and Benchmarks Track.html. Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024b. doi: 10.48550/ arXiv.2402.03300. URL https://arxiv.org/abs/2402.03300. Ke Wang, Junting Pan, Weikang Shi, et al. Measuring multimodal mathematical reasoning with MATH-Vision dataset. In NeurIPS, volume 37, 2024a. doi: 10.48550/arXiv.2402.14804. URL https://arxiv.org/abs/2402.14804. Lei Wang, Yi Hu, Jiabang He, et al. T-SciQ: Teaching multimodal chain-of-thought reasoning via large language model signals for science question answering. AAAI, 38(17):19162–19170, 2024b. doi: 10.1609/aaai.v38i17.29884. URL https://ojs.aaai.org/index.php/AAAI/article/view/29884. 12

Peiyi Wang, Lei Li, Zhihong Shao, et al. Math-Shepherd: Verify and reinforce LLMs step-bystep without human annotations. In ACL, pages 9426–9439, 2024c. doi: 10.18653/v1/2024.acllong.510. URL https://aclanthology.org/2024.acl-long.510/. Peng Wang, Shuai Bai, Sinan Tan, et al. Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024d. doi: 10.48550/arXiv. 2409.12191. URL https://arxiv.org/abs/2409.12191. Weiyun Wang, Zhangwei Gao, Lianjie Chen, et al. VisualPRM: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291, 2025. doi: 10.48550/arXiv.2503. 10291. URL https://arxiv.org/abs/2503.10291. Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. LogicVista: Multimodal LLM logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.04973, 2024. doi: 10.48550/arXiv.2407. 04973. URL https://arxiv.org/abs/2407.04973. Shuo Yang, Siwen Luo, and Soyeon Caren Han. Multimodal commonsense knowledge distillation for visual question answering. AAAI, 39(28):29545–29547, 2025. doi: 10.1609/aaai.v39i28. 35320. URL https://ojs.aaai.org/index.php/AAAI/article/view/35320. Qiying Yu, Zheng Zhang, Ruofei Zhu, et al. DAPO: An open-source llm reinforcement learning system at scale. arXiv:2503.14476, 2025. doi: 10.48550/arXiv.2503.14476. URL https://arxiv. org/abs/2503.14476. Xiang Yue, Tianyu Zheng, Yuansheng Ni, et al. MMMU-Pro: A more robust multi-discipline multimodal understanding benchmark. In ACL, pages 15134–15186, 2025. doi: 10.18653/v1/2025.acllong.736. URL https://aclanthology.org/2025.acl-long.736/. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, et al. MathVerse: Does your multi-modal LLM truly see the diagrams in visual math problems? In ECCV, 2024. doi: 10.48550/arXiv.2403.14624. URL https://arxiv.org/abs/2403.14624. Tianyang Zhao, Kunwar Yashraj Singh, Srikar Appalaraju, Peng Tang, Vijay Mahadevan, R. Manmatha, and Ying Nian Wu. No head left behind: Multi-head alignment distillation for transformers. AAAI, 38(7):7514–7524, 2024. doi: 10.1609/aaai.v38i7.28583. URL https://ojs.aaai.org/ index.php/AAAI/article/view/28583. Yujin Zhou, Pengcheng Wen, Jiale Chen, Boqin Yin, Han Zhu, Jiaming Ji, Juntao Dai, Chi-Min Chan, and Sirui Han. What, whether and how? unveiling process reward models for thinking with images reasoning. AAAI, 40(34):29071–29079, 2026. doi: 10.1609/aaai.v40i34.40144. URL https://ojs.aaai.org/index.php/AAAI/article/view/40144.

13