Preprint
F LOW CTS: O N - POLICY C ONTINUOUS T RAJECTORY S UPERVISION OF F LOW M ODELS Kaiyang Ye1∗ , Yuan Ge1∗ , Junxiang Zhang1 , Bei Li1 , Ziming Zhu1 , Haishu Zhao1 , Xiaoqian Liu1 , Chenglong Wang1,2 , Jingbo Zhu1,2 , Zhengtao Yu3 , Tong Xiao1,2† 1 Northeastern University 2 NiuTrans Research 3 Kunming University of Science and Technology
arXiv:2607.24522v1 [cs.LG] 27 Jul 2026
A BSTRACT While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored. To this end, we propose Flow Continuous Trajectory Supervision (F LOW CTS), which matches subsequent student and reference trajectories initialized from the same student-visited state. Using the integral relation between trajectories and velocity fields, we derive a temporally weighted velocitymatching upper bound and discretize it into practical objectives parameterized by the number of supervision steps. Under a multi-reference setup, single-state F LOW CTS-OPD outperforms vanilla KL-based OPD with faster convergence. F LOW CTS-OPD improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06, while outperforming a mixed-reward RL baseline across all target metrics. Further analysis reveals a clear temporal supervision mismatch in vanilla KL-based OPD arising from its auxiliary SDE transition kernels. Beyond on-policy setting, F LOW CTS also consistently outperforms vanilla SFT , particularly on OCR, while increasing supervision steps exhibit a trade-off between richer trajectory information and greater optimization difficulty.
1
I NTRODUCTION
Qwen3, MiMo-v2, and GLM-5 have recently attributed part of their post-training gains to an emerging paradigm: on-policy distillation (OPD) (Yang et al., 2025; Xiao et al., 2026; Zeng et al., 2026). This approach, as illustrated in Figure 2 (a), combines on-policy training with dense reference supervision (Agarwal et al., 2024; Gu et al., 2024). Although its training dynamics remain incompletely understood, these successes highlight the value of OPD for future generative-model development (Li et al., 2026c; Hou et al., 2026). In imitation learning, it is well established that learned policies benefit from training under their induced state distributions, rather than under those encountered by the expert (Ross & Bagnell, 2010; Ross et al., 2011). This facilitates recovery from mistakes and helps reduce compounding errors. Moreover, OPD provides such benefits by training on the student’s rollout distribution with dense token-level reference supervision, thereby reducing reliance on sparse outcome rewards (Lu & Lab, 2025; Ma et al., 2026a; Yang et al., 2026). Inspired by these, this work investigates how to effectively formulate dense OPD supervision for flow models. To this end, we first revisit the form of supervision in token-based autoregressive language models (Gu et al., 2024; Li et al., 2026c), whose original objective minimizes the sequence-level reverse KL divergence between the student’s and the reference’s output sequence distributions given a prompt. The chain-rule factorization of language modeling decomposes this objective exactly into next token supervision over a finite vocabulary at student-generated prefixes (Agarwal et al., 2024; Wen et al., 2023). However, flow models parameterize a continuous velocity field to model the continuous state trajectory from noise to data and lack a natural next-step conditional distribution (Lipman et al., 2022; Liu et al., 2022; Tong et al., 2023; Liu et al., 2023; Kornilov et al., 2024; Frans et al., 2025; Geng et al., 2026). This motivates us to revisit the essence of sequence supervision in OPD, shifting the supervision target from next-step distributions to the trajectory itself. ∗ †
Equal contribution. Corresponding author.
1
Preprint
SD-3.5-M
Geneval Expert
OCR Expert
Pickscore Expert
Vanilla OPD
FLOWCTS OPD
a photo of a zebra and a bed
A modern race car with a glossy black finish, featuring a bold red and white decal on the hood that reads "Speed Demon 5000".
Kangaroo on the Moon, Futuristic, Extraterrestrial, Curious, Sci-fi Illustration, Bold, Dynamic, Neon Colors, Digital Art
a photo of a broccoli and a vase
A golden acorn next to an oak tree within a giant hexagon
Figure 1: Qualitative comparison of generated images. Our method F LOW CTS-OPD produces images with improved alignment and visual quality compared with previous approaches. We therefore propose Flow Continuous Trajectory Supervision (F LOW CTS), which supervises the discrepancy between the student and reference continuous trajectories that evolve from the same shared state. F LOW CTS-OPD instantiates this objective in the on-policy setting, where the shared starting state is sampled from the current student’s rollout. We further derive an upper bound of the continuous trajectory supervision as a tractable optimization objective and discretize it into a finite number of optimization steps. The discretized objective is parameterized by the number of supervision steps K, covering single-state velocity matching and multi-step trajectory supervision. In the on-policy setting, single-state F LOW CTS-OPD outperforms vanilla KL-based OPD with faster convergence. Specifically, it improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06, while also showing improved out-of-distribution generalization. Qualitative examples are shown in Figure 1. Further analysis reveals a clear temporal supervision mismatch in vanilla KL-based Flow-OPD, which constructs transition kernels via an auxiliary SDE. More broadly, F LOW CTS does not restrict how the shared starting state is sampled. When the starting state is constructed from an off-policy data trajectory, the same formulation extends to F LOW CTS-SFT. Experiments show that F LOW CTS-SFT consistently outperforms vanilla SFT in overall score, with the most pronounced gains on OCR. Although increasing the supervision steps does not bring consistent improvements, F LOW CTS provides a new scaling law dimension of flow models that trades richer trajectory information against greater optimization difficulty. In summary, our contributions are: • We take an early exploration toward dense on-policy supervision for flow models. Revisiting the essence of sequence supervision in OPD for language modeling, we propose F LOW CTS from a continuous trajectory supervision perspective. • The discretized objective of F LOW CTS covers both single-state and multi-step supervision, introducing the supervision step as a new scaling dimension for flow models. • We extended F LOW CTS to both on-policy distillation and off-policy SFT settings, with experiments validating its effectiveness in both settings. 2
Preprint
• We attribute the advantage of F LOW CTS to the dense supervision in early denoising stages, where the student–reference discrepancy is largest. In contrast, KL-based FlowOPD concentrates supervision in late stages and suffers from this temporal supervision mismatch.
2
P RELIMINARIES
In this section, we begin by reviewing flow matching and standard on-policy distillation, which together motivate our trajectory-based OPD formulation. Flow Matching. Flow matching defines a transport trajectory between the data and noise distributions via an ordinary differential equation (ODE). Formally, let x0 ∼ pdata be a data sample and x1 ∼ pnoise = N (0, I) be a noise sample. The transport trajectory is commonly defined by linear interpolation (Liu et al., 2022): xt = (1 − t)x0 + tx1 ,
t ∈ [0, 1].
(1)
The trajectory satisfies the ODE dxt = vt (xt , t)dt, and under the linear path, the conditional target velocity is ut (xt | x0 , x1 ) = dxt /dt = x1 − x0 . Flow matching trains a neural velocity field vθ (xt , t) by minimizing h i 2 LFM (θ) = Et,x0 ,x1 ∥vθ (xt , t) − (x1 − x0 )∥ . (2) On-Policy Distillation. OPD provides reference supervision on samples generated by the current student. Given a prompt x ∼ Dx , the student samples a response ŷ1:T ∼ πθ (· | x). At each studentgenerated prefix (x, ŷ<t ), the student and reference define next-token distributions ptθ (·) = πθ (· | x, ŷ<t ) and ptT (·) = πT (· | x, ŷ<t ). A standard OPD objective minimizes the sequence-level reverse KL between the student and reference: LOPD (θ) = Ex∼Dx [DKL (πθ (· | x) ∥ πT (· | x))] . (3) By autoregressive factorization, this objective can be written as the sum of token-level KL terms on student-generated prefixes. " T # X t t LOPD (θ) = Ex∼Dx , ŷ1:T ∼πθ (·|x) DKL pθ ∥ pT . (4) t=1
Unlike autoregressive models, whose sequence factorization naturally yields next-token conditional distributions for local matching, flow models generate samples through a continuous state trajectory from noise to data and do not natively provide such a next-step distribution. Yet, given the same visited state, the student and reference models define distinct subsequent trajectories. This observation motivates us to formulate OPD at the trajectory level by matching their subsequent evolution over a finite time interval.
3
A C ONTINUOUS -T RAJECTORY P ERSPECTIVE ON OPD FOR F LOW M ODELS
In this section, we study how to formulate OPD for flow matching models from a continuoustrajectory perspective, as illustrated in Figure 2. Under OPD, we first sample a student-visited state xt along the current student’s generation trajectory. From this state, the student and reference velocity fields induce two subsequent trajectories. We formulate their discrepancy, derive a tractable velocity-based upper bound, and discretize it into a family of practical training objectives. We further discuss how the same formulation extends to off-policy starting states. 3.1
A T RAJECTORY-L EVEL F ORMULATION
We begin with a trajectory-level formulation over a reverse-time segment. Let 0 ≤ te < ts ≤ 1, where ts and te denote the start and end times of the segment, respectively. Let Φθts →t and Φref ts →t denote the flow maps induced by the student and reference velocity fields, respectively. Given a 3
Preprint
Trajectory-level objective: − 𝒕𝜽𝒙 𝒆 𝒕− 𝒙𝒕 𝒔
student
𝑦1
x
𝑦3
𝑦2
𝝅𝑻 𝒚ȁ𝒙
KL
Prompt
x
𝑦1
Prompt
x
predict
student 𝜋𝜃
teacher 𝜋 𝑇
KL
𝑦2
predict
𝑦2
student 𝜋𝜃
•••
student-visited state
student
𝒓𝒆𝒇
𝒙𝒕
𝑥𝑡𝑒
reference
ODE integration
𝑥0
Trajectory matching from a same student state
Prefix
𝑦1
trajectory discrepancy
𝑥1
Next-token supervision at student prefixes Prefix
𝒙𝒕𝜽
𝑥𝑡𝑠
𝑦𝑇
•••
Noise
𝝅𝜽 𝒚ȁ𝒙
Chain rule
dt
𝑥𝑡𝑒
𝒚 =(𝑦1 ,𝑦2 , …, 𝑦𝑇 ) Prompt
𝒓𝒆𝒇 𝟐
𝒕
Sequence-level objective: 𝑲𝑳 𝝅𝜽 𝒚ȁ𝒙 ∥ 𝝅𝑻 𝒚ȁ𝒙
teacher 𝜋 𝑇
𝑦3
K=1
K = 2, 3, …
Velocity matching (single-state)
Multi-step supervision
𝑥𝑡𝑠
𝑥𝑡𝑠
𝑉𝜃 (𝑥𝑡𝑠 , 𝑡𝑠 )
•••
‖ 𝑉𝜃 − 𝑉𝑟𝑒𝑓 ‖²
KL
𝑉𝑟𝑒𝑓 (𝑥𝑡𝑠 , 𝑡𝑠 )
(a) Autoregressive LMs
(b) Flow Models
Figure 2: Autoregressive OPD and trajectory-level supervision for flow models. In autoregressive models, sequence-level reverse KL decomposes by the chain rule into next-token supervision at student-generated prefixes. Since flow models do not naturally admit an analogous next-step factorization, FlowCTS instead matches student and reference trajectories initialized from the same student-visited state. Using the integral relation between trajectories and velocity fields, we derive a temporally weighted velocity-matching upper bound and discretize it into practical objectives parameterized by the supervision steps K, with K=1 recovering single-state velocity matching. starting state xts and a condition c, the corresponding trajectories are xθt = Φθts →t (xts , c) and ref xref t = Φts →t (xts , c). Their discrepancy is defined as: Z ts LCTS (θ) = Exts ,c
2 xθt − xref dt t
.
(5)
te
We refer to this trajectory-level formulation as Continuous Trajectory Supervision (CTS). It defines an on-policy distillation objective when the starting state xts is sampled from the current student rollout, and an off-policy supervision objective when it is constructed from an offline data trajectory. 3.2
A V ELOCITY-BASED U PPER B OUND
Flow trajectories are generated by integrating time-dependent velocity fields (Lipman et al., 2022; Liu et al., 2022). Therefore, the discrepancy between the student and reference trajectories is determined by the accumulated mismatch between their velocities. Based on this relation, we derive a weighted velocity-matching upper bound of the trajectory loss, yielding a tractable objective defined directly on the model outputs. Specifically, since both trajectories start from the same state xts , their states at any t ∈ [te , ts ] are obtained by integrating the corresponding velocity fields: Z ts Z ts xθt = xts − vθ (xθr , r, c) dr, xref = x − vref (xref (6) ts t r , r, c) dr. t
t
Subtracting the two equations gives: xθt − xref t =−
Z ts
vθ (xθr , r, c) − vref (xref r , r, c) dr.
(7)
t
This identity shows that the trajectory discrepancy arises from the accumulated velocity mismatch along the integration path. It leads to the following weighted velocity-matching upper bound, which we term the velocity upper bound (VUB): Z ts 2 θ ref LCTS (θ) ≤ LVUB (θ) = Exts ,c w(r) vθ (xr , r, c) − vref (xr , r, c) dr , (8) te
4
Preprint
Rr where the trajectory-derived weight is w(r) = te (ts −t) dt = (ts −te )2 −(ts −r)2 /2. This weight arises from the trajectory bound rather than manual design. It increases from zero at the segment end te to its maximum at the segment start ts , assigning larger weights to velocity errors closer to ts , as they affect a longer subsequent trajectory. A full derivation is provided in Appendix A. 3.3
D ISCRETE T RAINING O BJECTIVE
We now discretize the reverse-time segment [te , ts ] into K Euler steps. Let h = (ts − te )/K and define ti = ts − ih for i = 0, . . . , K. Starting from the same state xθ0 = xref 0 = xts , the student and reference rollouts are given by xθi+1 = xθi − hvθ (xθi , ti , c),
ref ref xref i+1 = xi − hvref (xi , ti , c),
(9)
for i = 0, . . . , K − 1. Applying the same error-accumulation argument to the discretized trajectories yields the following practical objective: "K−1 # X 2 (K) LCTS (θ) = E αi vθ (xθi , ti , c) − vref (xref , (10) i , ti , c) i=0
where αi = [K(K +1)−i(i+1)]/2 for i = 0, . . . , K −1 is the trajectory-derived discrete weight. It reflects how the velocity mismatch at step i accumulates over the subsequent trajectory. We present the weights in unnormalized form to make their relative weighting explicit and normalize them to sum to one in implementation. See Appendix B for the full derivation. When the starting state is sampled from the current student’s rollout, the resulting formulation defines a family of OPD objectives parameterized by the supervision step K. For K = 1, we have α0 = 1, and the objective reduces to velocity matching at a single student-visited state. For K > 1, supervision extends to multiple subsequent states along the student and reference trajectories. Thus, K explicitly controls how far supervision extends along the trajectory, and we study its effect empirically in Section 4.3. When the starting state is constructed from an off-policy data trajectory, the same formulation yields the corresponding off-policy supervision objective.
4
E XPERIMENTS
4.1
E XPERIMENTAL S ETUP
On-policy setting. We evaluate FlowCTS in a multi-reference OPD setting based on SD3.5Medium(Esser et al., 2024). We consider GenEval, OCR, and PickScore as target capabilities and use the corresponding officially released task-specific Flow-GRPO checkpoints as references (Liu et al., 2026a), without additional models. For each mini-batch, prompts are sampled from a targetspecific prompt set and routed to the corresponding reference model, while student-visited states are generated using the SDE rollout procedure adopted in Flow-GRPO. All OPD variants share the same prompt mixture, rollout procedure, reference routing, optimization hyperparameters, and inference configuration. We compare SDE and ODE rollouts in Appendix C.3 Off-policy SFT setting. We further evaluate FlowCTS in an off-policy setting. Starting states and target trajectories are constructed from 50K image–text pairs sampled from Fine-T2I (Ma et al., 2026b). We use SD3.5-Medium as the base model and train Vanilla SFT and all off-policy variants with LoRA under same optimization settings. Baselines. For the on-policy setting, we report the pretrained SD3.5-Medium and compare F LOW CTS-OPD against three groups of competitive baselines: (1) Single-task references, including the task-specific Flow-GRPO models specialized for GenEval, OCR, and PickScore; (2) Multiobjective RL, represented by Flow-GRPO-Mix, which jointly optimizes the three target capabilities; and (3) KL-based OPD, implemented as Vanilla OPD following the transition-kernel KL objective of Flow-OPD (Fang et al., 2026). For the off-policy setting, Vanilla SFT with single-state velocity supervision serves as the baseline. 5
Preprint
Table 1: Main results for multi-reference on-policy distillation and its off-policy supervision extension. Higher values are better for all metrics. Bold and underlined values indicate the best and second-best results within each supervision setting, respectively. Method
GenEval ↑
OCR ↑
PickScore ↑ Overall ↑
I. On-policy multi-objective alignment Pretrained model SD3.5-Medium 0.63
0.59
21.72
0.685
Reinforcement-learning baselines FlowGRPO (GenEval) FlowGRPO (OCR) FlowGRPO (PickScore) FlowGRPO (Mixed rewards)
0.93 0.66 0.51 0.73
0.65 0.92 0.70 0.83
21.60 21.74 23.25 21.84
0.804 0.805 0.701 0.800
KL-based on-policy distillation Vanilla OPD w/ KL (K = 1)
0.90
0.90
22.75
0.890
One-step velocity supervision F LOW CTS-OPD (K = 1)
0.93
0.92
23.06
0.912
Multi-step trajectory supervision FlowCTS-OPD (K = 2) FlowCTS-OPD (K = 3)
0.94 0.93
0.92 0.92
22.96 22.60
0.914 0.906
II. Off-policy trajectory supervision Single-step supervised fine-tuning Vanilla SFT 0.71
0.70
21.68
0.747
Multi-step trajectory supervision FlowCTS-SFT (K = 2) FlowCTS-SFT (K = 3) FlowCTS-SFT (K = 4)
0.73 0.75 0.74
21.66 21.71 21.63
0.761 0.763 0.756
0.72 0.71 0.70
Evaluation. Following the evaluation protocol of Flow-GRPO, we evaluate in-domain performance on GenEval, OCR, and PickScore. We further assess out-of-distribution compositional generalization on T2I-CompBench (Huang et al., 2023), and evaluate general image quality and human preference on DrawBench (Saharia et al., 2022) using ImageReward, Aesthetic Score, UnifiedReward, and HPS-v2.1. Full training and evaluation details are provided in Appendices C and D. 4.2
M AIN R ESULTS
Table 1 reports the main results under the on-policy and off-policy settings, we first examine multiobjective alignment in the on-policy setting. The task-specific Flow-GRPO models perform strongly on their respective targets but exhibit substantial trade-offs on the others. Flow-GRPO-Mix improves the balance across objectives, yet fails to match the specialized capabilities of individual experts. In contrast, F LOW CTS-OPD achieves balanced performance across all three objectives, matching the GenEval and OCR experts while retaining competitive PickScore performance. These results demonstrate the effectiveness of the trajectory-derived formulation for multi-reference OPD. To isolate the effect of the trajectory-derived objective, we compare F LOW CTS-OPD with Vanilla OPD within the same multi-reference on-policy pipeline. F LOW CTS-OPD improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06. Figure 3 further shows that F LOW CTS-OPD converges faster and reaches higher performance across all three metrics, approaching the corresponding GenEval and OCR expert levels substantially earlier than Vanilla OPD. Because the two methods differ only in their supervision objectives, these controlled results suggest that the trajectory-derived velocity-matching objective is more effective than Vanilla OPD. The off-policy results provide complementary evidence under a different starting-state distribution. Compared with Vanilla SFT, the FlowCTS-SFT objectives mainly improve OCR while maintaining comparable GenEval and PickScore performance, suggesting that the same trajectory-derived formulation also provides useful supervision in the off-policy setting. The effect of the supervision step in both settings is analyzed in Section 4.3. 6
Preprint
Base
Expert 0.9
Score
0.9
23
0.8
0.8
22.5
0.7
0.7
F LOW CTS-OPD
Vanilla OPD
22
0.6 0.6
0
0.5k
1.0k
(a) GenEval
1.5k
0
0.5k
1.0k
(b) OCR
1.5k
21.5
0
0.5k
1.0k
1.5k
(c) PickScore
Figure 3: Training dynamics of Vanilla OPD and F LOW CTS-OPD across three task-specific metrics. The lower dotted horizontal lines denote the base-model performance, while the upper dashed horizontal lines denote the corresponding expert performance. Blue lines with circular markers represent OPD-KL, and red lines with square markers represent CTS. 4.3
E FFECT OF THE S UPERVISION STEP IN OPD
We next study how far reference supervision within F LOW CTS-OPD should extend beyond a student-visited state. As shown in Table 1, we compare different supervision step under both onpolicy and off-policy settings. The step K ranges from velocity matching at the current state when K = 1 to supervision over a short student–reference trajectory segment when K > 1. Under the on-policy setting, extending the supervision horizon from K = 1 to K = 2 slightly improves GenEval from 0.93 to 0.94, yielding performance beyond that of the reference, while preserving the OCR score with only a minor reduction in PickScore. Extending the horizon further to K = 3 provides no additional benefit and leads to a larger drop in PickScore. These results suggest that the supervision horizon should be chosen carefully: short-horizon objectives already capture the main training signal from the trajectory, while further extending the horizon leads to inconsistent improvements across metrics. To control for training duration, we further compare K = 1 and K = 2 under a doubled training-step budget. K = 1 still underperforms K = 2, suggesting that the improvement cannot be attributed solely to longer optimization. See Appendix E. The off-policy setting shows a similar pattern. Compared with Vanilla SFT, extending the supervision step to K = 2 and K = 3 improves all three metrics, with the most pronounced gains observed on OCR, which increases from 0.70 to 0.73 and 0.75, respectively. However, increasing it further to K = 4 reduces all three metrics relative to K = 3, indicating that the gains from extending the supervision horizon are not sustained at larger K. We attribute this non-monotonic behavior to a trade-off between richer trajectory supervision and increased optimization difficulty. A larger K provides reference signals at more future states, promoting structurally consistent evolution and improving compositional and spatial correctness. However, longer student–reference rollouts amplify the effect of early prediction errors, making later supervision harder to optimize. Stronger cross-step constraints may also suppress fine-grained details, weakening perceptual preference. Thus, the supervision horizon should be treated as an explicit design choice rather than simply maximized. 4.4
T EMPORAL M ISALLOCATION OF KL S UPERVISION
We next analyze why the F LOW CTS-OPD optimizes more effectively than Vanilla OPD. In Vanilla OPD, the deterministic ODE dynamics are first reformulated as stochastic SDE transitions, and the KL divergence between the resulting transition kernels reduces to velocity regression with a timedependent weight, where wKL (t) is induced by the auxiliary SDE transition variance. 2
DKL (πθ (· | xt )∥πref (· | xt )) = wKL (t) ∥vθ (xt , t, c) − vref (xt , t, c)∥2 ,
(11)
Figure 4 reveals a clear mismatch in temporal signal allocation. The KL loss is strongly concentrated near the image endpoint, whereas the largest velocity-field differences between the pretrained model and the three reference experts consistently occur during early denoising. This observation is consistent with TempFlow-GRPO (He et al., 2025), which shows that perturbations during early 7
Preprint
Mean Velocity MSE
KL loss (×10−3 )
GenEval 2 1.5 1 0.5 0
0
0.25
0.50
0.75
OCR
PickScore
0.8 0.6 0.4 0.2 0
1.00
0
0.25
0.50
0.75
1.00
(b) Velocity MSE gap
(a) Per-step KL loss
Figure 4: Temporal signal allocation in KL-based OPD. Both horizontal axes denote normalized denoising progress from noise to image. The two quantities are computed from the same studentvisited states and training observations. (a) Effective per-step KL loss: wKL (t)∥vθ (xt , t, c) − vref (xt , t, c)∥22 . (b) Mean squared velocity gap without the additional KL-induced weight: ∥vθ (xt , t, c) − vref (xt , t, c)∥22 . The KL objective concentrates its effective supervision near the image endpoint, while the underlying student–reference velocity mismatch is largest during earlier denoising stages. Table 2: Results on T2I-CompBench. The best and second-best results within the OPD group are shown in bold and underlined, respectively. Model
Color
Shape
Texture
Complex
3D-Spatial
Numeracy
Non-Spatial
Pretrained model SD3.5-M
0.7994
0.5669
0.7338
0.3800
0.3739
0.5927
0.3146
Reinforcement learning GRPO-Mix 0.7966
0.5803
0.7392
0.3677
0.3681
0.6388
0.3130
On-policy distillation Vanilla OPD 0.8350 F LOW CTS-OPD 0.8295
0.6211 0.6285
0.7508 0.7414
0.3834 0.3839
0.4411 0.4433
0.6840 0.6909
0.3135 0.3096
structural stages have a substantially greater influence on the final generation outcome than those during late refinement stages. Therefore, the SDE-induced KL weight emphasizes late timesteps while underweighting the stages where the reference models introduce their largest corrections. In contrast, our K = 1 objective directly matches velocities at student-visited states without this temporal weighting. This difference provides an explanation for its faster convergence and stronger final performance in Figure 3. 4.5
C APABILITY AND G ENERALIZATION A NALYSIS
We further examine whether the F LOW CTS-OPD objective generalizes beyond the three training metrics. As shown in Table 3, F LOW CTS-OPD consistently outperforms Vanilla OPD across ImageReward, Aesthetic Score, UnifiedReward, and HPS-v2.1. These results show that its improvements are not limited to the training targets, but extend to broader measures of visual quality and human preference. Table 2 further evaluates out-of-distribution compositional capabilities on T2I-CompBench. Compared with Vanilla OPD, F LOW CTS-OPD performs better on shape, spatial relations, and numeracy, while the differences on color and texture remain limited. This pattern is consistent with the temporal analysis in Section 4.4: since early denoising stages play a greater role in establishing global structure, correcting the under-allocation of supervision at these stages is expected to benefit structural and relational capabilities more than visual attributes. These results suggest that the allocation of supervision across timesteps plays an important role in effective post-training.
5
R ELATED W ORK
Post-training for Flow Models. Post-training has become an important approach for adapting pretrained visual generative models toward downstream objectives. Early methods such as DDPO 8
Preprint
Table 3: Comparison on general image quality and human preference metrics.The best and secondbest results within the OPD group are shown in bold and underlined, respectively. ‡ Evaluated at a resolution of 1024 × 1024. Model
ImageReward
Aesthetic
UnifiedReward
HPS-v2.1
Pretrained models SD-XL‡ SD3.5-L‡ FLUX.1-Dev SD3.5-M
0.76 0.96 0.96 0.82
5.60 5.50 5.71 5.37
2.93 3.25 3.27 3.00
0.28 0.29 0.27 0.28
On-policy distillation Vanilla OPD F LOW CTS-OPD
1.21 1.27
5.46 5.64
3.28 3.32
0.30 0.31
and DPOK optimize diffusion models through policy-gradient updates using terminal rewards, ReFL directly backpropagates differentiable reward feedback, while Diffusion-DPO performs alignment from offline preference pairs (Black et al., 2024; Fan et al., 2023; Xu et al., 2023; Wallace et al., 2024; Clark et al., 2024; Liu et al., 2026b). Building on these developments, more recent work introduces GRPO-style online optimization for modern diffusion and rectified-flow generators. DanceGRPO (Xue et al., 2025) develops a unified GRPO framework across diffusion and rectified-flow generators, while Flow-GRPO enables online policy-gradient optimization of flow models through stochastic sampling trajectories (Liu et al., 2026a). TempFlow-GRPO (He et al., 2025) further introduces temporally aware credit assignment for flow-model optimization, whereas DiffusionNFT (Zheng et al., 2025) incorporates online reward feedback into forward-process flow matching. Complementary to these reward and preference-driven approaches, our work studies reference-guided on-policy distillation and focuses on how dense supervision should be constructed for flow models. On-Policy Distillation for Flow Models. On-policy distillation was initially developed for autoregressive language models, where the student generates its own sequences and receives dense reference supervision at the visited prefixes. MiniLLM optimizes a reverse-KL objective on student rollouts, while GKD generalizes this paradigm to different divergence objectives and sampling strategies (Gu et al., 2024; Agarwal et al., 2024).Recent studies further use OPD to consolidate capabilities from multiple specialized references into a unified student (Yang et al., 2026; Ma et al., 2026a; Zeng et al., 2026; Yang et al., 2025; Xiao et al., 2026; Li et al., 2026a). Recent works extend this idea to flow models. Flow-OPD and DiffusionOPD construct per-step KL objectives on student-generated states (Fang et al., 2026; Li et al., 2026b). Concurrent to our work, DanceOPD independently adopts direct velocity matching at a student-visited state and motivates it as a local field-regression objective (Zhou et al., 2026). FlowCTS instead derives a family of objectives from continuous student–reference trajectory differences. It further analyzes the temporal misallocation of transition-kernel KL relative to reference-student velocity errors, studies different supervision step, and extends the same formulation to off-policy supervision. Trajectory Distillation for Flow Models. Most diffusion distillation methods focus on step reduction, compressing a multi-step reference into a few-step student for efficient inference (Salimans & Ho, 2022; Liu et al., 2023). Trajectory-based methods imitate intermediate transitions or enforce consistency across timesteps (Song et al., 2023; Kim et al., 2024; Boffi et al., 2024), while DMD methods align the student and reference distributions without requiring trajectory-wise correspondence (Yin et al., 2024a;b). In contrast, F LOW CTS-OPD does not learn a shortcut generator. It uses finite trajectory segments as supervision units for post-training the original time-dependent velocity field and studies how the supervision horizon affects the resulting model.
6
C ONCLUSION
We study on-policy distillation for flow matching models and derive F LOW CTS-OPD from continuous student and reference trajectories initialized at shared student-visited states. The resulting objectives cover both local velocity matching and short-horizon supervision. In a controlled multireference setting, F LOW CTS-OPD converges faster and outperforms transition-kernel KL. Longer 9
Preprint
horizons yield non-monotonic gains, reflecting a trade-off between additional trajectory information and optimization difficulty. Temporal analysis further reveals that KL supervision is concentrated at late denoising stages, despite larger reference-field corrections occurring earlier. Together, these results identify temporal weighting and supervision horizon as two key design choices for OPD in continuous-time generative models.
R EFERENCES Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from selfgenerated mistakes. In International Conference on Learning Representations, volume 2024, pp. 21246–21263, 2024. Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In International Conference on Learning Representations, volume 2024, pp. 4965–4987, 2024. Nicholas M Boffi, Michael S Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. arXiv preprint arXiv:2406.07507, 2024. Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models on differentiable rewards. In International Conference on Learning Representations, volume 2024, pp. 4793–4822, 2024. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024. Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858–79885, 2023. Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, Shuang Chen, Kaituo Feng, Yunlong Lin, Lin Chen, Zehui Chen, Shaosheng Cao, et al. Flow-opd: On-policy distillation for flow matching models. arXiv preprint arXiv:2605.08063, 2026. Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In International Conference on Learning Representations, volume 2025, pp. 34668– 34684, 2025. Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. Advances in Neural Information Processing Systems, 38:75460– 75482, 2026. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. In International Conference on Learning Representations, volume 2024, pp. 32694–32717, 2024. Xiaoxuan He, Siming Fu, Yuke Zhao, Wanli Li, Jian Yang, Dacheng Yin, Fengyun Rao, and Bo Zhang. Tempflow-grpo: When timing matters for grpo in flow models. arXiv preprint arXiv:2508.04324, 2025. Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, et al. Uni-opd: Unifying on-policy distillation with a dual-perspective recipe. arXiv preprint arXiv:2605.03677, 2026. Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723–78747, 2023. 10
Preprint
Dongjun Kim, Chieh-Hsin Lai, WeiHsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. In International Conference on Learning Representations, volume 2024, pp. 44493–44525, 2024. Nikita Kornilov, Petr Mokrov, Alexander Gasnikov, and Alexander Korotin. Optimal flow matching: Learning straight trajectories in just one step. arXiv preprint arXiv:2403.13117, 2024. Fengxiang Li, Han Zhang, Haoyang Huang, Jinghui Wang, Jinhua Hao, Kun Yuan, Mengtong Li, Minglei Zhang, Pengcheng Xu, Wenhao Zhuang, et al. Kat-coder-v2 technical report. arXiv preprint arXiv:2603.27703, 2026a. Quanhao Li, Junqiu Yu, Kaixun Jiang, Yujie Wei, Zhen Xing, Pandeng Li, Ruihang Chu, Shiwei Zhang, Yu Liu, and Zuxuan Wu. Diffusionopd: A unified perspective of on-policy distillation in diffusion models. arXiv preprint arXiv:2605.15055, 2026b. Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huanang Gao, Wenkai Yang, Zhiyuan Liu, et al. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe. arXiv preprint arXiv:2604.13016, 2026c. Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022. Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. Advances in neural information processing systems, 38:40783–40818, 2026a. Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, Xintao Wang, et al. Improving video generation with human feedback. Advances in Neural Information Processing Systems, 38:82155–82192, 2026b. Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2023. Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang, Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, et al. Mopd: Multi-teacher on-policy distillation for capability integration in llm post-training. arXiv preprint arXiv:2606.30406, 2026a. Xu Ma, Yitian Zhang, Qihua Dong, and Yun Fu. Fine-t2i: An open, large-scale, and diverse dataset for high-quality t2i fine-tuning. arXiv preprint arXiv:2602.09439, 2026b. Stéphane Ross and Drew Bagnell. Efficient reductions for imitation learning. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 661–668. JMLR Workshop and Conference Proceedings, 2010. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 627–635. JMLR Workshop and Conference Proceedings, 2011. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479–36494, 2022. Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022. 11
Preprint
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023. Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid RectorBrooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482, 2023. Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228–8238, 2024. Yuqiao Wen, Zichao Li, Wenyu Du, and Lili Mou. F-divergence minimization for sequence-level knowledge distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10817–10834, 2023. Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al. Mimo-v2-flash technical report. arXiv preprint arXiv:2601.02780, 2026. Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023. Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07818, 2025. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation. arXiv preprint arXiv:2602.12125, 2026. Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455–47487, 2024a. Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6613–6623, 2024b. Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763, 2026. Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Liu. Diffusionnft: Online diffusion reinforcement with forward process. arXiv preprint arXiv:2509.16117, 2025. Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, et al. Danceopd: On-policy generative field distillation. arXiv preprint arXiv:2606.27377, 2026.
A
D ERIVATION OF THE V ELOCITY U PPER B OUND
We provide the derivation of the velocity upper bound used in Section 3.2. Consider a reverse-time segment [te , ts ] with 0 ≤ te < ts ≤ 1. The student and reference trajectories start from the same state xts and are defined as Z ts Z ts vref (xref (12) xθt = xts − vθ (xθr , r, c) dr, xref = x − t t r , r, c) dr. s t
t
12
Preprint
Subtracting the two trajectories gives xθt − xref t =−
Z ts
vθ (xθr , r, c) − vref (xref r , r, c) dr.
(13)
t
For simplicity, define the velocity mismatch along the two trajectories as ∆vr = vθ (xθr , r, c) − vref (xref r , r, c). Then xθt − xref t =−
(14)
Z ts ∆vr dr.
(15)
t
The continuous trajectory objective is Z ts LCTS (θ) = Exts ,c
2 dt xθt − xref t
.
(16)
te
Substituting the integral form of the state discrepancy gives "Z # 2 ts Z ts LCTS (θ) = Exts ,c ∆vr dr dt te
t
Z ts ≤ Exts ,c
Z ts (ts − t)
te Z ts Z r
= Exts ,c
te
2
∥∆vr ∥ dr dt
(17)
t
2 (ts − t) dt ∥∆vr ∥ dr .
te
The inner trajectory integral is first bounded by Cauchy–Schwarz, after which the resulting double integral is reordered. The trajectory-derived weight is Z r (ts − te )2 − (ts − r)2 . (18) w(r) = (ts − t)dt = 2 te Substituting ∆vr back gives the velocity upper-bound objective Z ts 2 θ ref w(r) vθ (xr , r, c) − vref (xr , r, c) dr . LCTS (θ) ≤ LVUB (θ) = Exts ,c
(19)
te
B
D ERIVATION OF THE D ISCRETE T RAINING O BJECTIVE
We derive the discrete form of the trajectory objective. We discretize the reverse-time segment [te , ts ] into K uniform Euler steps. Let ts − t e , ti = ts − ih, i = 0, . . . , K. K The student and reference rollouts start from the same state: h=
xθ0 = xref 0 = xts .
(20)
(21)
Using Euler discretization, the two rollouts are xθi+1 = xθi − hvθ (xθi , ti , c),
ref ref xref i+1 = xi − hvref (xi , ti , c),
(22)
for i = 0, . . . , K − 1. Define the velocity mismatch at step i as δi = vθ (xθi , ti , c) − vref (xref i , ti , c).
(23)
Then the state discrepancy follows θ ref xθi+1 − xref i+1 = xi − xi − hδi .
13
(24)
Preprint
Since xθ0 − xref 0 = 0, we have xθi − xref i = −h
i−1 X
δj ,
i = 1, . . . , K.
(25)
j=0
A discrete state-space trajectory loss can be written as Lstate,disc = h
K X
2
xθi − xref i
.
(26)
i=1
Substituting the accumulated error form gives i−1 K X X
3
Lstate,disc = h
2
δj
.
(27)
j=0
i=1
By the discrete Cauchy–Schwarz inequality, i−1 X
2
≤i
δj
j=0
i−1 X
2
∥δj ∥ .
(28)
j=0
Therefore, K i−1 X X 2 Lstate,disc ≤ h i ∥δj ∥ . 3
i=1
j=0
K X
(29)
Exchanging the summation order, we obtain Lstate,disc ≤ h3
K−1 X
j=0
2
i ∥δj ∥ .
(30)
i=j+1
The inner sum has the closed form K X
i=
i=j+1
K(K + 1) − j(j + 1) . 2
(31)
Thus, Lstate,disc ≤ h3
K−1 X
K(K + 1) − j(j + 1) 2 ∥δj ∥ . 2 j=0
(32)
The discrete upper bound determines the relative contribution of the velocity mismatch at each supervised state. For a fixed supervision horizon K and trajectory segment, h3 is a positive constant independent of θ. It therefore does not affect the minimizer or the relative weighting across timesteps, and can be absorbed into the overall loss coefficient. We define the unnormalized trajectory-derived weights as K(K + 1) − j(j + 1) αj = , j = 0, . . . , K − 1. (33) 2 For training, we normalize these coefficients as αj ᾱj = PK−1 i=0
αi
,
K−1 X
αi =
i=0
K(K + 1)(2K + 1) , 6
(34)
which preserves the trajectory-derived temporal profile while keeping the overall loss scale comparable across different values of K. The practical objective is therefore K−1 X 2 (K) LCTS (θ) = E ᾱj vθ (xθj , tj , c) − vref (xref (35) j , tj , c) 2 . j=0
When K = 1, ᾱ0 = 1, and the objective reduces to velocity matching at the segment starting state. 14
Preprint
C
T RAINING D ETAILS
C.1
O N -P OLICY MULTI - REFERENCE D ISTILLATION
Implementation details. We use Stable Diffusion 3.5 Medium as the base model and initialize the student from the averaged multi-task Flow-GRPO LoRA checkpoint. All models are fine-tuned with LoRA applied to the attention projections of the SD3 transformer, including the query, key, value, output, and added-conditioning projections. We set the LoRA rank to 32 and LoRA alpha to 64. Training is performed on four H200 GPUs. Student rollouts are generated at a resolution of 512 × 512 using 10 sampling steps and a classifierfree guidance scale of 4.5. A timestep fraction of 0.99 is used, resulting in nine denoising transitions for optimization. The sampling batch size is 16 per GPU, with four sampling batches per epoch and gradient accumulation over two steps. We optimize the student with AdamW using a learning rate of 1×10−4 , weight decay 1×10−4 , and a maximum gradient norm of 1.0. We maintain an exponential moving average of the trainable parameters with decay 0.9, updated every eight optimization steps. Prompt routing and reference models. We use the OCR, GenEval, and PickScore prompt sets released with Flow-GRPO. Each mini-batch is drawn from one prompt set and routed to its corresponding task-specific reference model: the text-rendering, GenEval, or PickScore Flow-GRPO expert. The three datasets are sampled alternately with a cycle ratio of 1:3:1 for OCR, GenEval, and PickScore, respectively. No additional auxiliary is introduced. OPD objectives. Vanilla OPD and F LOW CTS-OPDuse the same initialization, prompt mixture, student rollouts, reference routing, batch construction, and optimization hyperparameters. Vanilla OPD uses the step-wise transition-kernel KL as the distillation objective. For F LOW CTS-OPD, we replace the transition-kernel policy objective with the trajectory-derived velocity objective in Eq. 10. When K = 1, the objective reduces to mean-squared velocity matching between the student and the routed reference model at a student-visited state. For K > 1, the student and reference models are rolled out over the subsequent trajectory segment, and the per-step losses are weighted according to Eq. 33. C.2
OFF - POLICY E XTENSION
For the off-policy experiments, we construct training trajectories from 50K image–text pairs sampled from Fine-T2I and use SD3.5-Medium as the base model. Vanilla SFT and all F LOW CTSSFTvariants are trained with LoRA under the same data and optimization configuration. Vanilla SFT corresponds to single-state velocity supervision, while F LOW CTS-SFTextends the objective to K ∈ {2, 3, 4} subsequent states using the trajectory-derived weights. All off-policy experiments are conducted on four A800 GPUs. C.3
E FFECT OF SDE AND ODE S AMPLING
We compare SDE and ODE sampling for constructing the student-visited states during on-policy distillation. The two variants use the same velocity objective and training configuration, differing only in the rollout process used to obtain on-policy states. As shown in Figure 5, SDE sampling consistently outperforms deterministic ODE sampling throughout training. The difference is most pronounced at the early checkpoint, where SDE improves GenEval from 0.80 to 0.84. The advantage remains at the end of training, with SDE reaching 0.93 compared with 0.92 for ODE sampling and approaching the corresponding reference performance. These results suggest that stochastic exploration improves the states encountered during on-policy distillation, particularly during early optimization.
D
E VALUATION
Unless otherwise specified, we generate images at a resolution of 512×512 using 40 inference steps, a classifier-free guidance scale of 4.5, and a random seed of 42. All compared methods use identical 15
Preprint
Base
Teacher
ODE
SDE
0.95 GenEval score
0.90 0.85 0.80 0.75 0.70 0.65 40
160
320
Training checkpoint
Figure 5: Comparison of SDE and ODE sampling for constructing student-visited states during on-policy distillation. Both variants use the same velocity-matching objective and training configuration, differing only in the rollout process used to generate on-policy states. SDE sampling consistently achieves higher GenEval performance across training checkpoints, with a larger advantage during early optimization. The horizontal dashed lines denote the pretrained base model and the corresponding task-specific reference. Table 4: Training-budget control. The extended K = 1 run matches the number of supervised state evaluations used by K = 2. Method F LOW CTS-OPD (K = 1) F LOW CTS-OPD (K = 1) F LOW CTS-OPD (K = 2)
Training steps
GenEval ↑
OCR ↑
PickScore ↑
N 2N N
0.91 0.92 0.93
0.89 0.91 0.91
22.93 23.01 22.80
prompts, generation settings, and evaluator implementations. Models marked with ‡ in Table 3 are evaluated at 1024 × 1024. Following Flow-GRPO (Liu et al., 2026a), we evaluate GenEval, OCR, and PickScore on their corresponding test sets. Compositional generalization is evaluated on the validation split of T2ICompBench (Huang et al., 2023), with ten images generated per prompt using the official evaluation implementation. General image quality and preference alignment are evaluated on DrawBench using ImageReward, Aesthetic Score, UnifiedReward, and HPS-v2.1, following the Flow-GRPO evaluation protocol. All compared methods use identical generation and evaluator settings.
E
C ONTROLLING FOR THE T RAINING B UDGET
Since F LOW CTS-OPD (K = 2) evaluates two supervised states per optimization step, we conduct a small-scale auxiliary study to efficiently control for the supervision budget. Specifically, we compare K = 2 trained for N steps with K = 1 trained for N and 2N steps, where the latter matches the total number of supervised state evaluations. All runs in this study use the same reduced training configuration and are intended for controlled comparison within Table 4. As shown in Table 4, doubling the number of training steps for K = 1 produces modest improvements across all three metrics, but does not reproduce the behavior of K = 2. The latter achieves a higher GenEval score and comparable OCR performance, while yielding a lower PickScore. This distinct metric profile indicates that the effect of K = 2 cannot be explained solely by the number of supervised state evaluations. Instead, supervision at a subsequent rollout state places additional constraints on how the sample evolves along the trajectory, which particularly benefits compositional and structural correctness. Meanwhile, qualitative inspection suggests that this stronger trajectory constraint may suppress some fine-grained details, resulting in slightly smoother images and a lower preference score. We therefore view the result as a trade-off introduced by the supervision span, rather than a uniform improvement obtained from a larger training budget.
16