Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models
arXiv:2605.22795v1 [stat.ML] 21 May 2026
Krishnakumar Balasubramanian Department of Statistics, University of California, Davis [email protected] May 22, 2026 Abstract We propose and analyze a conservative drifting method for one-step generative modeling. The method replaces the original displacement-based drifting velocity by a kernel density estimator (KDE)-gradient velocity, namely the difference of the kernel-smoothed data score and the kernel-smoothed model score. This velocity is a gradient field, addressing the non-conservatism issue identified for general displacement-based drifting fields. We prove continuous-time finiteparticle convergence bounds for the conservative method on Rd : a joint-entropy identity yields bounds for the empirical Stein drift, the smoothed Fisher discrepancy of the KDE, and the squared center velocity. The main finite-particle correction is a reciprocal-KDE self-interaction term, and we give deterministic and high-probability local-occupancy conditions under which this term is controlled. We keep the quadrature constants explicit and track their possible bandwidth dependence: the root residual-velocity rate N −1/(d+4) holds under an additional h-uniform quadrature regularity condition, while a more general growth condition yields the optimized root rate N −(2−β)/(2(d+4−β)) , where 0 ≤ β < 2. We also analyze the non-conservative drifting method with Laplace kernel, corresponding to the original displacement-based velocity proposed in Deng et al. (2026). For this method, a sharp companion kernel decomposes the velocity into a positive scalar preconditioning of a sharp-score mismatch plus a Laplace scale-mismatch residual, producing an analogous finite-particle rate with an unavoidable residual term. Finally, we explain how the continuous-time residual-velocity bounds translate into one-step generation guarantees through the explicit drift size η.
1
Introduction
Drifting models (Deng et al., 2026) provide a direct approach to one-step generative modeling by moving the model distribution during training rather than applying a long iterative sampler at inference time. Let ν denote the data distribution on Rd and let µ denote the current model distribution. For a bandwidth-h kernel Kh , define the local kernel mean-shift vector of a probability measure α by R d (y − z)Kh (z − y) α( dy) . mα,h (z) := R R Rd Kh (z − y) α( dy) The displacement-based drifting velocity from µ toward ν is udisp ν,µ,h (z) := mν,h (z) − mµ,h (z). The data term attracts a point z toward regions locally represented by the data, while the model term repels z from regions already represented by the model. In practical training, the drifted 1
target is treated as a stop-gradient target: one forms z + ηudisp ν,µ,h (z) and trains the generator to regress to this frozen target. In the idealized exact-regression limit, this produces the frozen-field Euler update z + = z + ηudisp ν,µ,h (z), and the continuous-time particle systems studied below are obtained by sending the step size to zero; see Section A for additional explanation. The motivation for the first method in this paper is the fact that displacement-based drifting fields, as proposed in Deng et al. (2026), are generally not conservative. In particular, Franz et al. (2026) show that the position-dependent normalization in the original displacement field generally destroys the gradient-field structure, with the Gaussian kernel being a special case in which the field remains conservative. We therefore propose1 the conservative drifting method, which replaces the displacement velocity by an explicit KDE-gradient or score velocity. For any probability measure α, define its kernel-smoothed density and score by Z ρα,h (z) := Kh (z − y) α( dy), sα,h (z) := ∇ log ρα,h (z). Rd
The conservative velocity from µ toward ν is bν,µ,h (z) := sν,h (z) − sµ,h (z) = ∇ log ρν,h (z) − ∇ log ρµ,h (z). It is called conservative because bν,µ,h (z) = ∇{log ρν,h (z) − log ρµ,h (z)} is a gradient field. Throughout the paper, b denotes a conservative velocity, whereas u denotes a non-conservative displacement velocity. For Gaussian kernels, the conservative velocity and the displacement-based velocity coincide up to a deterministic scale factor. Indeed, if ! ∥u∥2 2 −d/2 Kh (u) = (2πh ) exp − 2 , 2h then ∇z Kh (z − y) = Therefore
R sα,h (z) =
y−z Kh (z − y). h2
∇ K (z − y) α( dy) 1 R z h = 2 mα,h (z), h Kh (z − y) α( dy)
and hence
1 disp u (z). h2 ν,µ,h Thus the two Gaussian dynamics generate the same trajectories after rescaling time or step size. For non-Gaussian kernels this identity generally fails: the non-conservative displacement field averages the Euclidean vector y − z, while the conservative field uses the KDE-gradient direction. We also remark that Franz et al. (2026) proposed a sharp-normalized field to restore conservativness. While both our approach and that of Franz et al. (2026) lead to score-difference fields, they score different smoothed densities. Our conservative KDE-gradient field uses the ordinary KDE bν,µ,h (z) =
1
Concurrent and independent work. During the final stages of completing this work, the author became aware of the work by Esteban-Casadevall et al. (2026) posted to arxiv on 11th May 2026. While their work also introduces the kernel-gradient drifting models, our contribution is a finite-particle entropy-rate analysis for the conservative dynamics and a parallel treatment of the non-conservative Laplace field.
2
Kh ∗ α, whereas the sharp-normalized field uses the companion KDE Kh# ∗ α, where Kh# is chosen so that ∇Kh# (z − y) = (y − z)Kh (z − y). These two fields coincide up to scale for Gaussian kernels, because the Gaussian satisfies ∇Kh (z − y) ∝ (y − z)Kh (z − y). For Laplace kernels they differ: the ordinary KDE-gradient field averages unit directions through ∇Kh , while the sharp-normalized field averages full displacement vectors and normalizes by the sharp KDE. Nevertheless, we leverage the idea of Franz et al. (2026) in our proof for analyzing the original drifting method proposed in Deng et al. (2026); see Section 6. For the finite-particle conservative method, let x = (x1 , . . . , xN ) and define N
qx (z) :=
1 X Kh (z − xj ), N
sx (z) := ∇ log qx (z).
j=1
With the smoothed target density ρ := ρν,h and sρ := ∇ log ρ, the finite-particle conservative velocity is bx (z) := sρ (z) − sx (z), and the center-evaluation dynamics are Ẋi (t) = bX(t) (Xi (t)),
i = 1, . . . , N.
The main residual-drift quantity is N
1 X VN (x) := ∥bx (xi )∥2 . N i=1
This is exactly the mean squared speed of the generated particle cloud under the conservative drifting flow. If one more explicit conservative drift step is applied, N
µ eN x =
x ei = xi + ηbx (xi ),
1 X δxei , N i=1
then the identity coupling gives N
W2 (µN eN x ,µ x )≤η
p VN (x),
µN x :=
1 X δxi . N i=1
Thus a bound on VN says that another drift refinement would change the one-step generated distribution only slightly. With a quadrature condition, VN also approximates the smoothed Fisher R discrepancy ∥sρ − sx ∥2 qx and therefore controls a smoothed distribution-level score mismatch. Our first contribution is a continuous-time finite-particle rate for the conservative method. By leveraging and extending the joint-entropy structure of the finite-particle Stein Variational Gradient Descent (SVGD) analysis in Banerjee et al. (2025), we prove that under regularity, reciprocal-KDE control, and point-evaluation quadrature smoothness, 1 T
Z T E[VN (X(t))] dt ≤ 0
HN (0) {−∆K(0)}+ ΛT + + CT (h)h2 . NT N hd+2
Here HN (0) is the initial joint entropy relative to the product smoothed target, ΛT controls the reciprocal KDE term, and CT (h) is a quadrature constant that may depend on the bandwidth. 3
This bandwidth dependence is important. If CN (h) = O(1) and T = N , optimizing the last two terms gives the root residual-velocity rate N −1/(d+4) . More generally, if CN (h) = O(h−β ) for some 0 ≤ β < 2, the optimized root rate becomes N −(2−β)/(2(d+4−β)) . If β ≥ 2, the present quadrature argument does not yield a vanishing residual by shrinking h. Our second contribution is a corresponding result for the non-conservative drifting method with Laplace kernel, namely the original displacement field specialized to ∥u∥ −d . Kh (u) = cd h exp − h This velocity is not generally a gradient field. To analyze it, we leverage the sharp companion kernel idea propoesd in Franz et al. (2026) and define Lh satisfying ∇z Lh (z − y) = (y − z)Kh (z − y). This yields a decomposition of the full non-conservative Laplace velocity of the form ux (z) = ax,h (z)b# x (z) + ex (z), where b# x is a sharp-score mismatch, ax,h is a positive local scale factor, and ex is a scale-mismatch residual measuring the difference between the data and model local Laplace-weighted radii. For the self-masked, leave-one-out Laplace dynamics, we prove a finite-particle rate Z 1 N κ0 βh 2 εS,h,N Lap (X(t))] dt ≤ E[VN + ∆ + + εV,h,N . N 0 γh N γh h γh The new term ∆h is unavoidable for the non-conservative Laplace method: it vanishes only when the local Laplace scales of the data and model are aligned. The leave-one-out form removes the reciprocal self-interaction term from the entropy identity, but introduces leave-one-out approximation errors controlled by denominator and occupancy conditions.
1.1
Related Work
Drifting models were introduced by Deng et al. (2026) as a training-time transport paradigm for one-step generation: generated samples are repeatedly moved by a data-attractive and modelrepulsive drift field, and the generator is trained to reproduce the drifted samples through a stopgradient target. This viewpoint is also related to score-difference flows for implicit generative modeling, where the velocity is written as the difference between target and source scores (Weber, 2023). Subsequent work has connected drifting to score matching and Wasserstein gradient flows of smoothed divergences (in the infinite-particle setting), especially in the Gaussian-kernel case where the displacement field can be written as a difference of KDE log-density gradients (Cao et al., 2026; Lai et al., 2026; Turan and Ovsjanikov, 2026; Gretton et al., 2026). A complementary line of work emphasizes the non-conservative nature of the original displacementbased field. Franz et al. (2026) show that the position-dependent normalization in the original field generally destroys conservatism, with the Gaussian kernel being a special case where the field remains a gradient field; they also introduce sharp normalizations that restore conservatism for broader radial kernels. Lee and Chun (2026) studies identifiability and stability for companionelliptic kernel families, including the Laplace kernel, showing that vanishing drift can identify the 4
target under appropriate kernel structure while also highlighting possible instability modes. These works motivate our distinction between the conservative drifting method, based on KDE-score gradients, and the non-conservative drifting method by Deng et al. (2026), based on the original displacement velocity. For the latter, our analysis shows that finite-particle rates require a scale-alignment residual measuring the failure of the Laplace displacement field to be an exact score-difference field. Recent drifting-related works study complementary algorithmic and variational perspectives. Dumont et al. (2026) study learning Monge maps through constrained drifting models: they lift a divergence, such as relative entropy, to the space of transport maps, constrain the dynamics to the set of optimal transport maps, and prove long-time existence and convergence toward the OT map under convexity assumptions. He et al. (2026a) introduce Sinkhorn-Drifting Generative Models, showing that the attraction–repulsion structure of drifting can be viewed as a one-sided surrogate for a Sinkhorn-divergence gradient flow, while two-sided Sinkhorn scaling gives improved identifiability and stability. Zhang et al. (2026) propose Lookahead Drifting Models, which compute several sequential drift terms within each training iteration and regress toward a weighted sum of these terms, with the goal of incorporating higher-order drift information while preserving one-step inference. More broadly, modern generative modeling is often organized around learned transport dynamics. Diffusion and score-based models learn reverse-time denoising dynamics and generate samples by numerically solving an SDE or probability-flow ODE (Ho et al., 2020; Song et al., 2021), while flow matching and rectified flow learn time-dependent vector fields that transport a base distribution to the data distribution along prescribed probability paths (Albergo and Vanden-Eijnden, 2023; Lipman et al., 2023; Liu et al., 2023). Flow-map and distillation-based approaches instead try to learn longer-time solution operators, or compress many solver steps into one or a few neural evaluations, as in flow-map matching, consistency models, and progressive distillation (Boffi et al., 2025; Song et al., 2023; Salimans and Ho, 2022). From this viewpoint, drifting models can be interpreted as moving part of the inference-time transport computation into training: the long-short flow-map perspective of Li and Zhu (2026) decomposes a global transport into a long-horizon map followed by a short terminal correction, and recovers the drifting field in the limit of a vanishing terminal interval. The proposed conservative velocity approach is also closely related to deterministic interactingparticle methods based on Stein discrepancies. Stein variational gradient descent (SVGD) was introduced by Liu and Wang (2016) as a deterministic particle method for approximate Bayesian inference, and Liu (2017) interpreted its population limit as a gradient flow of KL divergence under a Stein-induced geometry. Amortized SVGD (Feng et al., 2017) trains a sampler network to reproduce SVGD particle updates, which is conceptually close to the stop-gradient regression view of drifting. At the level of finite-particle analysis, Banerjee et al. (2025) obtain improved SVGD rates by differentiating the joint entropy of the particle law relative to the product target, improving upon earlier results by Shi and Mackey (2023). Extension to a regularized setting was studied by He et al. (2026b). We leverage and extend this high-level entropy-dissipation strategy in our proofs, but the controlled quantities and error terms are specific to drifting: for conservative drifting, the controlled quantity is the mean squared conservative residual with a reciprocal-KDE self-interaction and an h-dependent quadrature error; for the non-conservative Laplace method, the rate additionally contains the Laplace scale-mismatch residual.
5
2
Preliminaries
2.1
Set-up for conservative drifting
Let ρ be a positive probability density on Rd . For the conservative drifting method proposed in this paper, ρ should be read as the kernel-smoothed data density ρν,h = Kh ∗ ν. Write sρ (z) := ∇ log ρ(z). Let K : Rd → (0, ∞) be a normalized kernel and, for a bandwidth h > 0, define Kh (u) := h−d K(u/h). For a particle configuration x = (x1 , . . . , xN ) ∈ (Rd )N , define the kernel density estimate N
1 X Kh (z − xj ), qx (z) := N
sx (z) := ∇ log qx (z),
j=1
and the conservative drift field bx (z) := sρ (z) − sx (z). The center-evaluation finite-particle ODE for the conservative drifting method is Ẋi (t) = bi (X(t)),
bi (x) := bx (xi ),
i = 1, . . . , N.
(1)
This is a deterministic interacting particle system: each particle follows an ordinary differential equation, but its velocity depends on the full configuration through the KDE and its score. In this sense, the dynamics are structurally similar to SVGD (Liu and Wang, 2016; Liu, 2017), where particles also evolve deterministically through an empirical-measure-dependent velocity field; the key difference is that the conservative drifting velocity is the KDE-score residual sρ − sx , rather than the kernelized Stein velocity used in SVGD. Let pN t denote the joint density of X(t) = (X1 (t), . . . , XN (t)). We measure the joint law against the product target density N Y ρ⊗N (x) := ρ(xi ) i=1
through the joint relative entropy ⊗N HN (t) := KL(pN )= t ∥ρ
Z (Rd )N
pN t (x) log
pN t (x) dx. ⊗N ρ (x)
(2)
For a vector field f : Rd → Rd , define the Stein divergence (Barbour, 1988; Gorham and Mackey, 2015) associated with ρ by Aρ f (z) := ∇· f (z) + sρ (z) · f (z). (3) For a configuration x, define the empirical Stein drift N
SN (x) :=
1 X Aρ bx (xi ), N i=1
6
(4)
the KDE-smoothed Fisher discrepancy Z
∥bx (z)∥2 qx (z) dz,
IN (x) :=
(5)
Rd
and the squared center velocity N
VN (x) :=
1 X ∥bx (xi )∥2 . N
(6)
i=1
Finally set N
RN (x) :=
1 X 1 . N qx (xi )
(7)
i=1
2.2
Assumptions
The following assumptions are used in the entropy calculation and in the conversion from empirical Stein drift to Fisher-type discrepancies. Assumption 2.1 (Kernel). The base kernel K is positive, even, normalized, and C 3 : Z K(u) > 0, K(u) = K(−u), K(u) du = 1. Rd
Moreover
Z
∥u∥2 K(u) du < ∞,
m2 (K) := Rd
and ∆K(0) is finite. Since K is even, ∇K(0) = 0. Assumption 2.2 (Target and regular flow). The density ρ is positive and sufficiently smooth so that sρ is C 2 . The ODE (1) has a unique global solution for the initial law under consideration. The joint density pN t is differentiable in t and solves the Liouville equation ∂t pN t (x) +
N X
∇xi · {pN t (x)bi (x)} = 0.
i=1
All integrals appearing below are finite, and the boundary terms in the integrations by parts on (Rd )N vanish. A sufficient, but not necessary, condition is rapid decay of pN t (x)bi (x) and of the N N ⊗N products obtained by multiplying pt (x)bi (x) by log(pt (x)/ρ (x)). See Banerjee et al. (2025) for related conditions for SVGD. Assumption 2.3 (Reciprocal KDE control). For the time horizon T > 0 under consideration, there is a finite constant ΛT such that sup Et [RN (X(t))] ≤ ΛT , 0≤t≤T
where Et denotes expectation with respect to pN t and RN is defined in (7).
7
(8)
Remark 2.4 (Conditional nature of reciprocal-KDE control). The reciprocal-KDE assumption is a stability assumption, not a consequence of the entropy identity. The local-occupancy results in Section 5 give deterministic or high-probability sufficient conditions under which qX(t) (Xi (t)) ≥ λ > 0
for all i
at a fixed time, and under a propagated occupancy condition. However, propagation arguments that use a flow Lipschitz constant are conditional: the Lipschitz constant of the conservative vector field itself depends on reciprocal KDE quantities such as 1/qx , and sometimes on higher inverse powers of qx . Thus there is a potential bootstrap structure. A precise way to read the assumption is through the stopping time τλ := inf t ≥ 0 : min qX(t) (Xi (t)) < λ . 1≤i≤N
All conservative finite-particle bounds hold on the stopped interval [0, T ∧ τλ ], with constants depending on λ. If one can prove separately that τλ > T , for example by a deterministic occupancy argument or by a high-probability propagation argument that closes a bootstrap, then the stopped estimates become estimates on the full interval [0, T ]. We do not claim that reciprocal-KDE control follows automatically from the dynamics; it is an independent stability condition required to prevent denominator singularities. Assumption 2.5 (Quadrature smoothness). For the time horizon T > 0, there are finite constants BA,T (h) and BV,T (h) such that, for every configuration X(t) visited by the flow for 0 ≤ t ≤ T , sup D2 {Aρ bX(t) }(z) op ≤ BA,T (h),
(9)
z∈Rd
sup D2 { bX(t)
2
}(z)
z∈Rd
op
≤ BV,T (h).
(10)
Here D2 denotes the Hessian, and ∥·∥op is the operator norm. Remark 2.6 (Bandwidth dependence of the quadrature constants). The constants BA,T (h) and BV,T (h) may depend on N, h, T, ρ, K, and on the region of state space visited by the dynamics. This dependence is not a lower-order issue. Writing rx (z) := log ρ(z) − log qx (z),
bx (z) = ∇rx (z),
we have Aρ bx = ∆rx + ∇ log ρ · ∇rx , and D2 ∥bx ∥2 = 2(Dbx )⊤ (Dbx ) + 2
d X
bx,ℓ D2 bx,ℓ .
ℓ=1
Thus the quadrature constants involve log-derivatives of ρ and qx up to fourth order. If these log-derivatives vary on the kernel scale, for example if sup Dj (log ρ − log qx )(z) = O(h−j ),
j = 1, 2, 3, 4,
z
then one typically has BA,T (h) + BV,T (h) = O(h−4 ). In that regime the quadrature contribution {BA,T (h) + BV,T (h)}m2 (Kh ) = O(h−2 ) 8
scales like h−2 , not h2 , and shrinking the bandwidth worsens this term. Consequently, the rates below are non-asymptotic inequalities with the quadrature constants displayed. Any asymptotic bandwidth optimization must explicitly account for the h-dependence of BA,T (h) and BV,T (h). Remark 2.7 (Examples of quadrature-growth regimes). The exponent β in BA,N (h) + BV,N (h) = O(h−β ) should be interpreted as a regularity exponent for the KDE-score field along the particle trajectory, not as a property of the kernel alone. First, β = 0 is realized in an h-uniform log-smooth regime. For example, take a Gaussian kernel or a C ∞ compactly supported kernel, and suppose that the target density and the model KDE satisfy n o sup sup max Dj log ρ(z) op , Dj log qX(t) (z) op ≤ Clog , 0≤t≤N
z
1≤j≤4
with Clog independent of h. Then BA,N (h) + BV,N (h) = O(1), so β = 0. This situation can occur when the particles are sufficiently dense and regular so that the finite KDE behaves like the convolution of the kernel with a fixed smooth density bounded away from zero, rather than like a collection of isolated bandwidth-h spikes. Second, exponents 0 < β < 2 correspond to intermediate-scale regularity. Suppose the log-score residual rx (z) := log ρ(z) − log qx (z) varies on a length scale ℓh = hα , where 0 < α < 1/2. More concretely, assume −αj sup Dj rx (z) op = O(ℓ−j ), h ) = O(h
j = 1, 2, 3, 4.
z
Since bx = ∇rx , the quantities D2 {Aρ bx } and D2 {∥bx ∥2 } involve derivatives of rx up to fourth order. Hence BA,N (h) + BV,N (h) = O(h−4α ). Thus β = 4α. Because 0 < α < 1/2, this gives examples with 0 < β < 2. The same smooth kernels, such as Gaussian kernels or smooth compactly supported kernels, can realize either regime. What changes is the regularity of the log-density ratio log ρ − log qx along the particle trajectory. If this ratio has only macroscopic variation, then β = 0. If it has features at an intermediate scale ℓh = hα with 0 < α < 1/2, then β = 4α ∈ (0, 2). If it varies at the kernel scale ℓh = h, then α = 1, hence β = 4, and the optimized vanishing-rate statement no longer applies. Finally, we emphasize that our conservative drifting analysis assumes that the smoothing kernel is sufficiently smooth, for example K ∈ C 3 or stronger when the quadrature constants involve higher derivatives. This assumption excludes the exact Laplace kernel Kh (u) = cd h−d exp(− ∥u∥ /h), which is not differentiable at the origin. This is why the exact Laplace kernel is treated separately in Section 6 through the non-conservative displacement field and the sharp-companion-kernel decomposition. A smooth regularization of the Laplace kernel could be included in the conservative analysis, but the resulting estimates would have constants depending on the regularization scale. 9
Another useful feature of the forthcoming results is that it imposes only the aforementioned local smoothness and positivity assumptions on the smoothed target density needed to define its score and justify the entropy calculation; in particular, we do not assume curvature conditions such as the target is log-concave, strongly log-concave, or satisfies a Log-Sobolev, Poincaré, or dissipativity condition. Our bounds that control the time-averaged residual drift, provide a global convergence in KL or Wasserstein distance under such additional curvature conditions.
3
Entropy identity for conservative drifting
The next theorem is the basic finite-particle identity. The self-interaction term is the only term not present in the corresponding population smoothed-KL calculation. Theorem 3.1 (Joint-entropy identity). Let Assumptions 2.1 and 2.2 hold. Then, for every t for which the quantities are finite, d HN (t) = −N Et [SN (X(t))] − ∆Kh (0) Et [RN (X(t))]. dt
(11)
Equivalently, with ah := {−∆Kh (0)}+ , one has the upper bound d HN (t) ≤ −N Et [SN (X(t))] + ah Et [RN (X(t))]. dt Proof. Let rN (x) := ρ⊗N (x). By definition, Z HN (t) =
pN t (x) log
pN t (x) dx. rN (x)
Differentiating under the integral gives d HN (t) = dt Since pN t is a probability density, Z
Z
∂t pN t (x)
∂t p N t (x) dx =
pN t (x) log + 1 dx. rN (x)
d dt
Z
pN t (x) dx = 0.
Thus d HN (t) = dt
Z
∂t pN t (x) log
pN t (x) dx. rN (x)
Using the Liouville equation, N
X d HN (t) = − dt
Z
∇xi · {pN t (x)bi (x)} log
i=1
10
pN t (x) dx. rN (x)
(12)
Integrating by parts in xi , N
X d HN (t) = dt
Z
pN t (x)bi (x) · ∇xi log
i=1
Now ∇xi log Since rN (x) =
pN t (x) dx. rN (x)
pN t (x) = ∇xi log pN t (x) − ∇xi log rN (x). rN (x)
QN
j=1 ρ(xj ),
∇xi log rN (x) = sρ (xi ). Therefore N
X d HN (t) = dt
Z
N pN t bi · ∇xi log pt dx −
i=1
N Z X
pN t bi · sρ (xi ) dx.
i=1
For the first integral, N N pN t ∇xi log pt = ∇xi pt .
Hence
Z
N pN t bi · ∇xi log pt dx =
Z
bi · ∇xi pN t dx = −
Z
pN t ∇xi · bi dx.
Consequently, "N # X d HN (t) = −Et {∇xi · bi (X(t)) + sρ (Xi (t)) · bi (X(t))} . dt
(13)
i=1
It remains to compute ∇xi · bi (x). Since bi (x) = sρ (xi ) − sx (xi ), we have ∇xi · bi (x) = ∇ · sρ (xi ) − ∇xi · sx (xi ). The derivative of sx (xi ) with respect to xi contains two pieces: the derivative of the evaluation point and the derivative of the ith KDE center. Write sm x (z) =
∂m qx (z) , qx (z)
m = 1, . . . , d.
For fixed z, ∂xℓ qx (z) = − i
1 ∂ℓ Kh (z − xi ), N
and ∂xℓ ∂m qx (z) = − i
Therefore ∂xℓ sm x (z) = − i
1 2 ∂ Kh (z − xi ). N ℓm
2 K (z − x ) 1 ∂ℓm 1 ∂m qx (z)∂ℓ Kh (z − xi ) i h + . N qx (z) N qx (z)2
11
Set m = ℓ and sum over ℓ. Since Kh is even, ∇Kh (0) = 0, and hence at z = xi , d X
∂xℓ sℓx (z) z=x = − i
i
ℓ=1
1 ∆Kh (0) . N qx (xi )
By the chain rule, ∇xi · sx (xi ) = ∇z · sx (z) z=x − i
1 ∆Kh (0) . N qx (xi )
Thus ∇xi · bi (x) = ∇ · sρ (xi ) − ∇z · sx (z) z=x + i
= ∇z · bx (z) z=x + i
1 ∆Kh (0) N qx (xi )
1 ∆Kh (0) . N qx (xi )
Since bi (x) = bx (xi ), ∇xi · bi (x) + sρ (xi ) · bi (x) = Aρ bx (xi ) +
1 ∆Kh (0) . N qx (xi )
Substituting this identity into (13) gives "N # N X d 1 ∆Kh (0) X HN (t) = −Et Aρ bX(t) (Xi (t)) + dt N qX(t) (Xi (t)) i=1
i=1
= −N Et [SN (X(t))] − ∆Kh (0)Et [RN (X(t))]. This proves (11). Since −∆Kh (0) ≤ {−∆Kh (0)}+ = ah , (12) follows. Remark 3.2 (Interpretation of the self-interaction term). The first term on the right-hand side of (11) is the desired entropy dissipation. The second term is finite-particle specific: it appears because the velocity at xi depends on the same particle through the KDE denominator. Its sign and size are controlled by −∆Kh (0) and by the reciprocal quantity RN . This is why the subsequent rate theorem requires reciprocal-KDE control. If particles are locally well populated at bandwidth h, then qx (xi ) is bounded below and the self-interaction correction remains of order 1/N after the entropy normalization. For Gaussian kernels, the self-interaction correction has a definite positive sign. Indeed, for K(u) = (2π)−d/2 exp(− ∥u∥2 /2), we have, for each coordinate j, ∂jj K(0) = −K(0) = −(2π)−d/2 . Therefore d X ∆K(0) = ∂jj K(0) = −d(2π)−d/2 < 0, j=1
and hence a1 := {−∆K(0)}+ = d(2π)−d/2 > 0. For the bandwidth-scaled kernel Kh (u) = h−d K(u/h), −∆Kh (0) = h−d−2 a1 . Thus the reciprocal-KDE term in the conservative finite-particle bound is a genuine positive selfinteraction correction for Gaussian kernels. 12
4
Continuous-time convergence rates for conservative drifting
4.1
Rate for the empirical Stein drift
Theorem 4.1 (Entropy rate for the empirical Stein drift). Let Assumptions 2.1, 2.2, and 2.3 hold. Then, for every T > 0, 1 T
Z T Et [SN (X(t))] dt ≤ 0
HN (0) ah ΛT + , NT N
ah := {−∆Kh (0)}+ .
(14)
Proof. By Theorem 3.1, ′ HN (t) ≤ −N Et [SN (X(t))] + ah Et [RN (X(t))].
Using Assumption 2.3, ′ HN (t) ≤ −N Et [SN (X(t))] + ah ΛT .
Rearrange: ′ N Et [SN (X(t))] ≤ −HN (t) + ah ΛT .
Integrate over t ∈ [0, T ]: Z T Et [SN (X(t))] dt ≤ HN (0) − HN (T ) + ah ΛT T.
N 0
Since relative entropy is nonnegative, HN (0) − HN (T ) ≤ HN (0). Therefore
Z T Et [SN (X(t))] dt ≤ HN (0) + ah ΛT T.
N 0
Dividing by N T proves (14). Remark 4.2 (What the Stein-drift rate controls). The quantity SN is the empirical Stein drift associated with the vector field bx . It is not written as a square, so (14) is not yet a residualvelocity bound. The role of the quadrature step below is to compare this empirical Stein drift with the KDE-averaged Fisher discrepancy, which is nonnegative and is directly related to the squared velocity of the particles.
4.2
Quadrature conversion to Fisher and center velocity
The empirical Stein drift SN is not manifestly nonnegative. The next lemma identifies the nonnegative population quantity that it approximates. Lemma 4.3 (KDE-averaged Stein identity). For every fixed configuration x for which the integrations by parts are justified, Z Aρ bx (z)qx (z) dz = IN (x). Rd
13
(15)
Proof. By definition of Aρ , Z
Z
Z
qx Aρ bx =
qx ∇ · b x +
qx sρ · bx .
Integrating the first term by parts, Z
Z qx ∇ · b x = −
Since ∇qx = qx sx ,
Thus
Z
∇qx · bx .
Z
−
∇qx · bx = −
Z
Z
qx sx · bx .
qx Aρ bx =
qx (sρ − sx ) · bx .
Z
qx ∥bx ∥2 = IN (x).
Because bx = sρ − sx , Z qx Aρ bx = This proves the lemma. Lemma 4.4 (Point-evaluation quadrature error). Let Kh be even and have finite second moment Z m2 (Kh ) := ∥u∥2 Kh (u) du. Rd
Under Assumption 2.5, for every configuration X(t) visited by the flow, 1 |SN (X(t)) − IN (X(t))| ≤ BA,T (h)m2 (Kh ), 2 1 |VN (X(t)) − IN (X(t))| ≤ BV,T (h)m2 (Kh ). 2 Proof. We prove (16); the proof of (17) is identical with ∥bx ∥2 replacing Aρ bx . Fix x and define ϕx (z) := Aρ bx (z). By Lemma 4.3, Z IN (x) =
qx (z)ϕx (z) dz.
Since
N
qx (z) =
1 X Kh (z − xi ), N i=1
we have
N
1 X IN (x) = N
Z Kh (z − xi )ϕx (z) dz.
i=1
Meanwhile, N
SN (x) =
1 X ϕx (xi ). N i=1
14
(16) (17)
Therefore
Z N 1 X ϕx (xi ) − Kh (z − xi )ϕx (z) dz . SN (x) − IN (x) = N i=1
Set u = z − xi . Then Z
Z Kh (z − xi )ϕx (z) dz =
Kh (u)ϕx (xi + u) du.
Taylor’s formula gives Z 1 ϕx (xi + u) = ϕx (xi ) + ∇ϕx (xi ) · u +
(1 − r)u⊤ D2 ϕx (xi + ru)u dr.
0
R Integrating against Kh (u) du, the zeroth-order term gives ϕ (x ) because Kh = 1. The first-order x i R term vanishes because Kh is even, hence uKh (u) du = 0. Thus Z Kh (u)ϕx (xi + u) du − ϕx (xi ) Z Z 1 ≤ Kh (u) (1 − r) D2 ϕx (xi + ru) op ∥u∥2 dr du. 0
Using Assumption 2.5, Z Z 1 Z Kh (u)ϕx (xi + u) du − ϕx (xi ) ≤ BA,T (h) (1 − r) dr ∥u∥2 Kh (u) du. 0
Since
R1
0 (1 − r) dr = 1/2,
Z
1 Kh (u)ϕx (xi + u) du − ϕx (xi ) ≤ BA,T (h)m2 (Kh ). 2
Averaging over i = 1, . . . , N proves (16). Theorem 4.5 (Continuous-time rates for Fisher discrepancy and center velocity). Let Assumptions 2.1, 2.2, 2.3, and 2.5 hold. Then, for every T > 0, Z 1 T HN (0) ah ΛT 1 + + BA,T (h)m2 (Kh ), (18) Et [IN (X(t))] dt ≤ T 0 NT N 2 Z 1 T HN (0) ah ΛT 1 + + {BA,T (h) + BV,T (h)}m2 (Kh ). (19) Et [VN (X(t))] dt ≤ T 0 NT N 2 Proof. From Lemma 4.4, 1 IN (X(t)) ≤ SN (X(t)) + BA,T (h)m2 (Kh ). 2 Taking expectations, integrating over [0, T ], dividing by T , and applying Theorem 4.1 gives (18). Similarly, 1 VN (X(t)) ≤ IN (X(t)) + BV,T (h)m2 (Kh ). 2 Combining this inequality with (18) gives (19). 15
Remark 4.6 (Implication for one-step generation). Theorem 4.5 bounds the average squared speed of the generated particle cloud under the conservative drift. If a time τ is sampled uniformly from [0, T ], then Z 1 T Et [VN (X(t))] dt. E [VN (X(τ ))] = T 0 Consequently, a small right-hand side in (19) implies that, at a typical training time, one more explicit drift step changes the empirical generated law by at most " !# Z T N 1 X 2 N 21 E W2 µX(τ ) , Et [VN (X(t))] dt. δXi (τ )+ηbX(τ ) (Xi (τ )) ≤η N T 0 i=1
Thus the theorem is a convergence-to-small-residual-drift statement for one-step generation. With the quadrature comparison, it also controls the smoothed Fisher discrepancy between qx and ρ. Corollary 4.7 (Bandwidth form). Let Kh (u) = h−d K(u/h) and let the assumptions of Theorem 4.5 hold. Then m2 (Kh ) = h2 m2 (K), ∆Kh (0) = h−d−2 ∆K(0). Consequently, with a1 := {−∆K(0)}+ , Z T
1 T
Et [VN (X(t))] dt ≤ 0
HN (0) a1 ΛT 1 + + {BA,T (h) + BV,T (h)}m2 (K)h2 . d+2 NT 2 Nh
(20)
If, in addition, HN (0) ≤ κ0 N and T = N , then 1 N
Z N Et [VN (X(t))] dt ≤ 0
κ0 a1 ΛN 1 + + {BA,N (h) + BV,N (h)}m2 (K)h2 . d+2 N 2 Nh
(21)
Thus the corresponding root-discrepancy bound is
1 N
Z N 0
r r 1/2 r κ0 a1 ΛN 1 Et [VN (X(t))] dt ≤ + +h {BA,N (h) + BV,N (h)}m2 (K). d+2 N 2 Nh
(22)
Proof. The scaling identities follow by the change of variables u = hv: Z Z 2 −d 2 ∥v∥2 K(v) dv = h2 m2 (K). m2 (Kh ) = ∥u∥ h K(u/h) du = h Also, ∆Kh (u) = h−d−2 (∆K)(u/h), so ∆Kh (0) = h−d−2 ∆K(0). Substituting these identities into (19) gives (20). If HN (0) ≤ κ0 N and T = N , then HN (0) κ0 N κ0 ≤ = , NT N2 N √ √ √ √ which gives (21). Finally, (22) follows from a + b + c ≤ a + b + c, for a, b, c ≥ 0.
16
4.3
Optimizing the conservative bandwidth and the one-step size
The bandwidth form (21) separates three effects: the entropy initialization term, the finite-particle self-interaction term, and the quadrature term. Since the quadrature constants may depend on h, we write them explicitly as functions of the bandwidth. Define 1 CN (h) := {BA,N (h) + BV,N (h)}m2 (K). 2
AN (h) := a1 ΛN (h), Then (21) reads 1 N
Z N Et [VN (X(t))] dt ≤ 0
κ0 AN (h) + + CN (h)h2 . N N hd+2
(23)
Thus the commonly stated bandwidth tradeoff is valid only after specifying how AN (h) and CN (h) scale with h. Assume that, on the admissible bandwidth range, there are constants A, C > 0 and 0 ≤ β < 2 such that (24) AN (h) ≤ A, CN (h) ≤ Ch−β . Equivalently, the quadrature constants satisfy BA,N (h) + BV,N (h) = O(h−β ). Then (23) implies 1 N
Z N Et [VN (X(t))] dt ≤ 0
κ0 A + + Ch2−β . N N hd+2
(25)
The last two terms are minimized at h⋆,N :=
(d + 2)A (2 − β)CN
1/(d+4−β) .
(26)
Indeed, differentiating h 7→
A + Ch2−β N hd+2
gives (d + 2)A + (2 − β)Ch1−β . N hd+3 Setting this derivative equal to zero gives (26). At this bandwidth, −
2−β A = Ch2−β ⋆,N . d+2 d+2 N h⋆,N Therefore 1 N
Z N 0
κ0 d + 4 − β Et [VN (X(t))] dt ≤ + C N d+2
(d + 2)A (2 − β)CN
Thus, under (24), the squared residual velocity is O N −(2−β)/(d+4−β) , and the root residual velocity is O N −(2−β)/(2(d+4−β)) . 17
(2−β)/(d+4−β) .
(27)
The h-uniform quadrature case is the special case β = 0, which recovers the root rate N −1/(d+4) . If β ≥ 2, the quadrature term h2−β does not vanish as h → 0, and this quadrature argument does not yield a vanishing residual by bandwidth shrinkage. For example, the natural kernel-scale derivative scaling BA,N (h) + BV,N (h) = O(h−4 ) corresponds to β = 4 and falls outside the regime of the optimized vanishing-rate statement. The occupancy condition remains compatible with (26) whenever N hd⋆,N → ∞ sufficiently quickly. For 0 ≤ β < 2, N hd⋆,N ≍ N 1−d/(d+4−β) = N (4−β)/(d+4−β) , which diverges faster than log N for fixed d and fixed β < 2. The explicit one-step size η does not enter the continuous-time entropy inequality. It enters when the learned generator is interpreted through one more explicit correction z 7→ z + ηbx (z). For a fixed h, the theorem gives 2 E W22 (µN eN x ,µ x ) ≤ η RN,h , where RN,h denotes the right-hand side of (23). Taking h = h⋆,N gives a residual correction of order η N −(2−β)/(2(d+4−β)) in root mean square, up to constants. If the explicit map is required to be a small stable Euler step and bx has Lipschitz constant Lb (h), a standard sufficient condition is ηLb (h) ≤ c for a numerical constant c < 1. In conservative models one often has Lb (h) = O(h−2 ) under denominator and smoothness control, which suggests η⋆,N ≍ h2⋆,N ≍ N −2/(d+4−β) . For Gaussian kernels, the non-conservative displacement velocity satisfies udisp = h2 b, so this conservative-step choice corresponds to a constant-order step size for the displacement formulation. Remark 4.8 (Initialization). If the initial particles are i.i.d. from a density µ0 satisfying KL(µ0 ∥ρ) < ∞, then ⊗N =⇒ HN (0) = N KL(µ0 ∥ρ). pN 0 = µ0 Thus the condition HN (0) ≤ κ0 N holds with κ0 = KL(µ0 ∥ρ).
5
Sufficient conditions for reciprocal KDE control on Rd
The reciprocal-KDE condition (8) is needed because the center-evaluation divergence differentiates the ith KDE center and produces a factor 1/qx (xi ). This section gives checkable sufficient conditions.
5.1
Deterministic local-occupancy condition
Assumption 5.1 (Kernel lower bound near the origin). There exist constants rK > 0 and κK > 0 such that K(u) ≥ κK whenever ∥u∥ ≤ rK . (28) Equivalently, Kh (u) ≥ κK h−d
whenever ∥u∥ ≤ rK h.
18
For a configuration x, define the local neighbor count Mi (x; h) := #{j ∈ {1, . . . , N } : ∥xj − xi ∥ ≤ rK h}.
(29)
The count includes j = i. Proposition 5.2 (Local occupancy implies reciprocal control). Let Assumption 5.1 hold. Suppose that a configuration x satisfies Mi (x; h) ≥ αN hd
for every i = 1, . . . , N
(30)
for every i = 1, . . . , N,
(31)
for some α > 0. Then qx (xi ) ≥ κK α and hence RN (x) ≤
1 . κK α
(32)
Proof. Fix i. By definition, N
1 X qx (xi ) = Kh (xi − xj ). N j=1
For every j with ∥xj − xi ∥ ≤ rK h, Assumption 5.1 gives Kh (xi − xj ) ≥ κK h−d . Therefore qx (xi ) ≥
1 Mi (x; h)κK h−d . N
Using (30), 1 (αN hd )κK h−d = κK α. N Taking reciprocals and averaging over i yields (32). qx (xi ) ≥
5.2
A high-probability occupancy bound for i.i.d. particles
The previous proposition is deterministic. The next result shows that its hypothesis holds with high probability for i.i.d. samples from a density with uniform lower local mass. Assumption 5.3 (Uniform lower local mass). Let µ be a probability measure on Rd . For the bandwidth h under consideration, there is a constant p0 > 0 such that µ(B(y, rK h)) ≥ p0 hd
for µ-a.e. y.
(33)
Remark 5.4 (One way to verify Assumption 5.3). Suppose µ has density f on a closed set S ⊂ Rd , f (y) ≥ m0 > 0 for y ∈ S, and S satisfies the thickness condition Leb(S ∩ B(y, r)) ≥ θvd rd
for all y ∈ S and 0 < r ≤ r0 ,
where vd is the volume of the unit Euclidean ball. If rK h ≤ r0 , then d d µ(B(y, rK h)) ≥ m0 Leb(S ∩ B(y, rK h)) ≥ m0 θvd rK h .
Thus Assumption 5.3 holds with d p0 = m0 θvd rK .
19
Proposition 5.5 (I.i.d. local occupancy). Let Y1 , . . . , YN be i.i.d. from a probability measure µ satisfying Assumption 5.3. Let Y = (Y1 , . . . , YN ). Assume N ≥ 2. Define the event n o p0 Eh := Mi (Y ; h) ≥ N hd for every i = 1, . . . , N . 4 Then
p 0 P(Ehc ) ≤ N exp − N hd . 16 Consequently, under Assumption 5.1, on Eh , 4 . κK p 0
RN (Y ) ≤
(34)
(35)
Moreover, since Kh (0) = h−d K(0), RN (Y ) ≤ and hence E[RN (Y )] ≤
N hd K(0)
always,
p 4 N hd 0 + N exp − N hd . κK p0 K(0) 16
(36)
(37)
Proof. Fix i. Conditional on Yi = y, define X Zi := 1{∥Yj − y∥ ≤ rK h}. j̸=i
Then Zi is binomial with parameters N − 1 and p(y) := µ(B(y, rK h)). By Assumption 5.3, p(y) ≥ p0 hd for µ-a.e. y. The neighbor count including the particle itself is Mi (Y ; h) = 1 + Zi . Let mi := (N − 1)p(y). For N ≥ 2, 1 mi ≥ (N − 1)p0 hd ≥ N p0 hd . 2 If
1 Zi ≥ mi , 2
then
Therefore
1 1 Mi (Y ; h) = 1 + Zi ≥ mi ≥ N p0 hd . 2 4
1 P Mi (Y ; h) < N p0 hd | Yi = y 4
20
1 ≤ P Zi < mi | Yi = y . 2
For a binomial random variable, the multiplicative Chernoff bound gives m 1 i P Zi < mi | Yi = y ≤ exp − . 2 8 Using mi ≥ (N − 1)p0 hd ≥ N p0 hd /2, m p i 0 exp − ≤ exp − N hd . 8 16 This bound is uniform in y. Integrating over Yi and taking a union bound over i = 1, . . . , N proves (34). On Eh , Proposition 5.2 applies with α = p0 /4, giving (35). For the deterministic self-bound, observe that N 1 X 1 K(0) qY (Yi ) = . Kh (Yi − Yj ) ≥ Kh (0) = N N N hd j=1
Thus
1 N hd ≤ qY (Yi ) K(0)
for every i,
and averaging over i gives (36). Finally, E[RN (Y )] = E[RN (Y )1Eh ] + E[RN (Y )1Ehc ] ≤
4 N hd + P(Ehc ). κK p0 K(0)
Substituting (34) proves (37).
5.3
Propagation of occupancy along the continuous-time flow
The i.i.d. result controls the reciprocal KDE at a single time. To control it along the flow, one may combine initial occupancy at a slightly smaller scale with a Lipschitz-distortion bound for pairwise distances. Assumption 5.6 (Lipschitz distortion of the drift). For the realized trajectory on [0, T ], there exists an integrable function Lt such that bX(t) (z) − bX(t) (z ′ ) ≤ Lt z − z ′
for all z, z ′ ∈ Rd ,
(38)
for all 0 ≤ t ≤ T . Define Z T ΓT :=
Lt dt.
(39)
0
Lemma 5.7 (Pairwise distance distortion). Under Assumption 5.6, for every pair i, j and every 0 ≤ t ≤ T, ∥Xi (t) − Xj (t)∥ ≤ eΓT ∥Xi (0) − Xj (0)∥ . (40) Proof. For i, j, set Dij (t) := Xi (t) − Xj (t). From (1), d Dij (t) = bX(t) (Xi (t)) − bX(t) (Xj (t)). dt 21
Using Assumption 5.6, d ∥Dij (t)∥ ≤ Lt ∥Dij (t)∥ dt whenever Dij (t) ̸= 0. The same inequality holds in the upper Dini derivative sense when Dij (t) = 0. Gronwall’s inequality gives Z t ∥Dij (t)∥ ≤ exp Ls ds ∥Dij (0)∥ ≤ eΓT ∥Dij (0)∥ . 0
This proves the lemma. Proposition 5.8 (Trajectory-level reciprocal control from initial occupancy). Assume Assumptions 5.1 and 5.6. Suppose the initial configuration satisfies #{j : ∥Xj (0) − Xi (0)∥ ≤ rK he−ΓT } ≥ αN hd e−dΓT
for every i.
(41)
Then, for every 0 ≤ t ≤ T , RN (X(t)) ≤
edΓT . κK α
(42)
Proof. Fix i and t ∈ [0, T ]. If ∥Xj (0) − Xi (0)∥ ≤ rK he−ΓT , then by Lemma 5.7, ∥Xj (t) − Xi (t)∥ ≤ eΓT rK he−ΓT = rK h. Thus the number of particles within distance rK h of Xi (t) is at least the number of particles within distance rK he−ΓT of Xi (0). By (41), Mi (X(t); h) ≥ αN hd e−dΓT . The proof of Proposition 5.2 gives qX(t) (Xi (t)) ≥
1 {αN hd e−dΓT }κK h−d = κK αe−dΓT . N
Taking reciprocals and averaging over i proves (42). Corollary 5.9 (I.i.d. initial data plus controlled distortion). Let X1 (0), . . . , XN (0) be i.i.d. from a probability measure µ. Suppose that, with e h := he−ΓT , the lower local mass condition µ(B(y, rK e h)) ≥ p0e hd
for µ-a.e. y
holds. Assume Assumptions 5.1 and 5.6. Then, with probability at least p 0 1 − N exp − N hd e−dΓT , 16 one has 4edΓT sup RN (X(t)) ≤ . κK p 0 0≤t≤T
(43)
(44)
(45)
Furthermore, " E
# sup RN (X(t)) ≤ 0≤t≤T
p 4edΓT N hd 0 N exp − N hd e−dΓT . + κK p 0 K(0) 16 22
(46)
Proof. Apply Proposition 5.5 at bandwidth e h = he−ΓT . With probability at least (44), for every i, #{j : ∥Xj (0) − Xi (0)∥ ≤ rK he−ΓT } ≥
p0 N (he−ΓT )d . 4
This is (41) with α = p0 /4. Therefore Proposition 5.8 gives (45). For the expectation bound, use the same decomposition as in the proof of Proposition 5.5. On the high-probability event, use (45). On the complement, the deterministic self-bound holds at every time: 1 K(0) qX(t) (Xi (t)) ≥ Kh (0) = , N N hd so N hd sup RN (X(t)) ≤ . K(0) 0≤t≤T Combining with the probability estimate proves (46). Remark 5.10 (Use in Theorem 4.5). If the right-hand side of (46) is finite, then Assumption 2.3 holds with p 4edΓT N hd 0 ΛT = + N exp − N hd e−dΓT . κK p0 K(0) 16 In particular, if ΓT is bounded independently of N and N hd → ∞, then the exponential term is negligible and ΛT = O(1).
6
Finite-particle rates for the non-conservative drifting method with Laplace kernel
We now prove a continuous-time finite-particle rate for the non-conservative displacement-based drifting field with Laplace kernel. The main point is that, unlike the Gaussian kernel, the Laplace mean-shift field is not exactly a score-difference field. Nevertheless, it admits a useful “sharpscore” representation, which yields a finite-particle convergence theorem with an additional scalemismatch residual. The sharp companion kernel construction used in this section follows the sharpnormalization framework of Franz et al. (2026), specialized to the Laplace kernel. Our use of it is different: rather than replacing the original Laplace normalization by the sharp normalization, we use the sharp kernel to decompose the original non-conservative Laplace field into a preconditioned sharp-score mismatch plus a scale-mismatch residual. Throughout this section, let ∥u∥ −d Kh (u) := cd h exp − , u ∈ Rd , h R where cd > 0 is chosen so that Rd Kh (u) du = 1. For a probability measure α on Rd , define its Laplace KDE Z Kh (z − y) α( dy),
Qα,h (z) := Rd
and the non-conservative displacement mean-shift vector R d (y − z)Kh (z − y) α( dy) Mα,h (z) := R . Qα,h (z) 23
The non-conservative Laplace drifting field from a model distribution µ toward the data distribution ν is uLap ν,µ,h (z) := Mν,h (z) − Mµ,h (z). For a particle configuration x = (x1 , . . . , xN ) ∈ (Rd )N , let N
µN x :=
1 X δxj . N j=1
We analyze the self-masked, or leave-one-out, finite-particle version of the non-conservative Laplace drifting field. For each i, define 1 X δ xj , µN x,−i := N −1 j̸=i
and ux,−i (z) := Mν,h (z) − MµN
x,−i ,h
(z).
The continuous-time dynamics are Ẋi (t) = uX(t),−i (Xi (t)),
i = 1, . . . , N.
(47)
The residual drift quantity we want to control is N
Lap VN (x) :=
1 X ∥ux,−i (xi )∥2 . N
(48)
i=1
This is the mean squared norm of the non-conservative Laplace drift applied to the generated particles. Hence if one more explicit drift step is applied, x ei = xi + ηux,−i (xi ), then the identity coupling gives N
N
1 X 1 X δxi , δxei N N
W22
i=1
! Lap (x). ≤ η 2 VN
i=1
Lap Thus a bound on VN controls how much an additional non-conservative Laplace drifting correction changes the generated distribution.
6.1
Sharp-score representation of the Laplace mean-shift field
The Laplace mean-shift field is not itself a KDE score, but it is a scaled score of a companion kernel. Define the sharp companion kernel Lh (u) := h(∥u∥ + h)Kh (u). Let
Z Lh (z − y) α( dy),
Rα,h (z) := Rd
and define the sharp-smoothed score σα,h (z) := ∇ log Rα,h (z). Also define the scalar scale factor aα,h (z) := 24
Rα,h (z) . Qα,h (z)
(49)
Lemma 6.1 (Sharp-score representation). For every probability measure α for which the following quantities are well-defined, Mα,h (z) = aα,h (z)σα,h (z). (50) Moreover, aα,h (z) = h{r̄α,h (z) + h}, where
(51)
R r̄α,h (z) :=
− z∥ Kh (z − y) α( dy) Rd ∥y R Rd Kh (z − y) α( dy)
is the Laplace-weighted local mean radius. Proof. Let r = ∥u∥. Since Lh (u) = h(r + h)cd h−d e−r/h , the radial derivative is 1 d {h(r + h)cd h−d e−r/h } = hcd h−d e−r/h − h(r + h)cd h−d e−r/h . dr h Combining the two terms, d {h(r + h)cd h−d e−r/h } = cd h1−d e−r/h − (r + h)cd h−d e−r/h . dr Since cd h1−d e−r/h = hcd h−d e−r/h , we get d {h(r + h)cd h−d e−r/h } = {h − (r + h)}cd h−d e−r/h = −rKh (u). dr Therefore, for u ̸= 0, u ∇u Lh (u) = −rKh (u) = −uKh (u). r By continuity, the same identity holds at u = 0, because both sides vanish there. Taking u = z − y, we obtain ∇z Lh (z − y) = −(z − y)Kh (z − y) = (y − z)Kh (z − y). Hence
Z
Z
∇Rα,h (z) =
∇z Lh (z − y) α( dy) = Rd
(y − z)Kh (z − y) α( dy). Rd
Dividing by Qα,h (z) gives Mα,h (z) =
∇Rα,h (z) Rα,h (z) ∇Rα,h (z) = = aα,h (z)σα,h (z). Qα,h (z) Qα,h (z) Rα,h (z)
This proves (50). It remains to prove (51). By definition of Lh , Z h(∥z − y∥ + h)Kh (z − y) α( dy). Rα,h (z) = Rd
Thus
Z Rα,h (z) = h
∥z − y∥ Kh (z − y) α( dy) + h2
Rd
Z Kh (z − y) α( dy). Rd
25
The second integral is Qα,h (z). Therefore R ∥z − y∥ Kh (z − y) α( dy) Rα,h (z) =h + h2 . Qα,h (z) Qα,h (z) Since ∥z − y∥ = ∥y − z∥, this is aα,h (z) = hr̄α,h (z) + h2 = h{r̄α,h (z) + h}. The lemma follows. Let
Z Lh (u) du.
Z#,h := Rd
The normalized sharp-smoothed data density is ρ# ν,h (z) :=
Rν,h (z) . Z#,h
Since the normalizing constant does not depend on z, ∇ log ρ# ν,h (z) = ∇ log Rν,h (z) = σν,h (z). For the full empirical model, define Rx,h := RµN , x ,h
σx,h := ∇ log Rx,h ,
Qx,h := QµN , x ,h
ax,h :=
Rx,h . Qx,h
Also define the full non-conservative Laplace field ux (z) := Mν,h (z) − MµN (z). x ,h By Lemma 6.1, ux (z) = aν,h (z)σν,h (z) − ax,h (z)σx,h (z). Define the sharp score mismatch b# x (z) := σν,h (z) − σx,h (z), and the Laplace scale-mismatch residual ex (z) := {aν,h (z) − ax,h (z)}σν,h (z).
(52)
ux (z) = ax,h (z)b# x (z) + ex (z).
(53)
Then This decomposition is the key difference between the non-conservative Laplace field and the conservative field. If the local scales aν,h and ax,h match, then the non-conservative Laplace drift is a positive scalar preconditioning of a score mismatch. If they do not match, the residual ex is unavoidable.
26
6.2
Entropy identity for the leave-one-out non-conservative field
Let pN t denote the joint density of X(t). We measure it against the sharp-smoothed product target ⊗N (ρ# (x) = ν,h )
N Y ρ# ν,h (xi ), i=1
and define
# HN (t) := KL pN t
⊗N (ρ# ) . ν,h
(54)
Let A# f (z) := ∇· f (z) + σν,h (z) · f (z) be the Stein divergence associated with ρ# ν,h . Define N
SLap N (x) :=
1 X A# ux,−i (xi ). N
(55)
i=1
Assumption 6.2 (Regularity for the leave-one-out Laplace flow). The ODE (47) has a unique N solution on [0, T ], the joint law admits a differentiable density pN t , and pt solves the Liouville equation associated with the vector field x 7→ (ux,−1 (x1 ), . . . , ux,−N (xN )). All integrations by parts below are justified, and the trajectory avoids particle collisions and data atoms at which the exact Laplace field is not differentiable. Equivalently, one may replace Kh by a smooth regularized Laplace kernel and then pass to the exact Laplace limit under uniform bounds. Theorem 6.3 (Joint-entropy identity for non-conservative Laplace drifting). Under Assumption 6.2, d # H (t) = −N Et [SLap (56) N (X(t))]. dt N Proof. Let # ⊗N (x). (x) := (ρ# rN ν,h )
The joint density pN t satisfies ∂t p N t (x) +
N X
∇ xi · p N t (x)ux,−i (xi ) = 0.
i=1
By definition, # HN (t) =
Z
pN t (x) log
pN t (x) # rN (x)
dx.
Differentiating under the integral gives d # H (t) = dt N Since pN t is a probability density,
Z
Z
( ∂t p N t (x) log
pN t (x) # rN (x)
∂t p N t (x) dx = 0.
27
) +1
dx.
Therefore d # H (t) = dt N
Z
∂t p N t (x) log
pN t (x) # rN (x)
dx.
Using the Liouville equation, N
Z
X d # HN (t) = − dt
∇xi · {pN t (x)ux,−i (xi )} log
i=1
pN t (x) # rN (x)
dx.
Integrating by parts in xi , N
Z
X d # HN (t) = dt
pN t (x)ux,−i (xi ) · ∇xi log
pN t (x) # rN (x)
i=1
dx.
Now # ∇xi log rN (x) = ∇ log ρ# ν,h (xi ) = σν,h (xi ).
Hence ∇xi log
pN t (x)
= ∇xi log pN t (x) − σν,h (xi ).
# (x) rN
Thus N
X d # HN (t) = dt
Z
N pN t ux,−i (xi ) · ∇xi log pt dx
i=1
−
N Z X
pN t ux,−i (xi ) · σν,h (xi ) dx.
i=1
Since N N pN t ∇xi log pt = ∇xi pt ,
the first integral in the last display satisfies Z Z N N pt ux,−i (xi ) · ∇xi log pt dx = ux,−i (xi ) · ∇xi pN t dx. Integrating by parts again, Z Z N ux,−i (xi ) · ∇xi pt dx = − pN t ∇xi · ux,−i (xi ) dx. Because ux,−i depends on xi only through the evaluation point z = xi , not through the leave-one-out model measure µN x,−i , we have ∇xi · ux,−i (xi ) = ∇·z ux,−i (z) z=x . i
Therefore "N # X d # H (t) = −Et ∇· uX(t),−i (Xi (t)) + σν,h (Xi (t)) · uX(t),−i (Xi (t)) . dt N i=1
The expression inside braces is exactly A# uX(t),−i (Xi (t)). Hence
d # H (t) = −N Et [SLap N (X(t))], dt N
which proves the identity. 28
6.3
Laplace coercivity with a scale-mismatch residual
Let
Rx,h (z) Z#,h
ρ# x,h (z) :=
be the full sharp-smoothed empirical density. Define the sharp-smoothed drift energy Z Lap ∥ux (z)∥2 ρ# Vh (x) := x,h (z) dz,
(57)
Rd
and the sharp-smoothed Stein drift JhLap (x) :=
Z Rd
A# ux (z)ρ# x,h (z) dz.
(58)
Lemma 6.4 (Sharp-smoothed Stein identity). For every fixed configuration x for which the integrations by parts are justified, Z # (59) b# JhLap (x) = x (z) · ux (z)ρx,h (z) dz. Rd
Proof. By definition, A# ux = ∇· ux + σν,h · ux . Therefore JhLap (x) =
Z
ρ# x,h ∇· ux +
Z
ρ# x,h σν,h · ux .
For the first term, integration by parts gives Z Z # ρx,h ∇· ux = − ∇ρ# x,h · ux . Since # # # ∇ρ# x,h = ρx,h ∇ log ρx,h = ρx,h σx,h ,
we obtain
Z
ρ# x,h ∇· ux = −
Thus JhLap (x) =
Z
Z
ρ# x,h σx,h · ux .
ρ# x,h (σν,h − σx,h ) · ux .
By definition, b# x = σν,h − σx,h . Hence JhLap (x) =
Z
# b# x · ux ρx,h ,
which proves the claim. Assumption 6.5 (Laplace scale alignment). For all configurations visited by the flow on [0, T ], there exist constants 0 < λh ≤ Lh < ∞ and ∆h ≥ 0 such that λh ≤ ax,h (z) ≤ Lh and
Z Rd
for all z ∈ Rd ,
2 ∥ex (z)∥2 ρ# x,h (z) dz ≤ ∆h .
29
(60)
(61)
Lemma 6.6 (Laplace coercivity). Under Assumption 6.5, JhLap (x) ≥ γh VhLap (x) − βh ∆2h , where γh :=
λh , 4L2h
βh :=
(62)
λh 1 + . 2L2h 2λh
(63)
Proof. By Lemma 6.4, JhLap (x) =
Z
# b# x (z) · ux (z)ρx,h (z) dz.
Using the decomposition (53), ux (z) = ax,h (z)b# x (z) + ex (z), we get 2
# b# x (z) · ux (z) = ax,h (z) bx (z)
Thus JhLap (x) =
Z ax,h
2
b# x
+ b# x (z) · ex (z).
ρ# x,h +
Z
Z
b# x
# b# x · ex ρx,h .
By ax,h ≥ λh , Z
ax,h b# x
2
ρ# x,h ≥ λh
2
ρ# x,h .
For the second term, Young’s inequality gives, pointwise, −b# x · ex ≤
λh # 2 1 ∥ex ∥2 . bx + 2 2λh
b# x · ex ≥ −
λh # 2 1 bx − ∥ex ∥2 . 2 2λh
Equivalently,
After integration, Z
λh # b# x · ex ρx,h ≥ −
Z
2
2
b# x
1 ρ# x,h − 2λ
Z h
∥ex ∥2 ρ# x,h .
Using (61), Z
λh # b# x · ex ρx,h ≥ −
Z
2
b# x
2
ρ# x,h −
∆2h . 2λh
Combining the two lower bounds yields λh JhLap (x) ≥ 2
Z
b# x
2
ρ# x,h −
∆2h . 2λh
Next, from ux = ax,h b# x + ex and ax,h ≤ Lh , ∥ux ∥2 ≤ 2a2x,h b# x
2
+ 2 ∥ex ∥2 ≤ 2L2h b# x
2
+ 2 ∥ex ∥2 .
Integrating with respect to ρ# x,h , VhLap (x) ≤ 2L2h
Z
b# x 30
2
2 ρ# x,h + 2∆h .
Rearranging, Z
b# x
2
VhLap (x) − 2∆2h # . ρx,h ≥ 2L2h
Substituting this into the lower bound for JhLap , Lap ∆2 λh Vh (x) − 2∆2h Lap Jh (x) ≥ − h. 2 2 2λh 2Lh
Therefore JhLap (x) ≥
λh Lap V (x) − 4L2h h
λh 1 + 2L2h 2λh
∆2h .
This is exactly (62).
6.4
From the sharp-smoothed coercivity to particle rates
The entropy identity controls the empirical leave-one-out Stein quantity SLap N , whereas the coercivity lemma is written for the full-field sharp-smoothed quantities JhLap and VhLap . The following assumption records the required quadrature and leave-one-out approximation errors. Assumption 6.7 (Quadrature and leave-one-out approximation). For all configurations visited by the flow on [0, T ], there are constants εS,h,N ≥ 0 and εV,h,N ≥ 0 such that Lap SLap N (x) − Jh (x) ≤ εS,h,N ,
(64)
Lap (x) − VhLap (x) ≤ εV,h,N . VN
(65)
Remark 6.8. Assumption 6.7 separates the entropy argument from the technical problem of controlling leave-one-out perturbations. The quantities εS,h,N and εV,h,N contain two distinct effects: the point-evaluation quadrature error and the difference between the full field Ux and the leave-oneout fields Ux,−i . Lemma 6.12 reduces these errors to Hessian bounds for the full-field integrands and the pointwise leave-one-out errors N
ℓS,h,N :=
1 X |A# Ux,−i (xi ) − A# Ux (xi )| , N i=1
and
N
ℓV,h,N :=
1 X ∥Ux,−i (xi )∥2 − ∥Ux (xi )∥2 . N i=1
In this paper these leave-one-out errors are kept as explicit assumptions. A fully explicit bound in terms of N and h would require uniform lower bounds on the relevant Laplace KDE denominators, upper bounds on local weighted moments, and derivative stability estimates for the sharp-score and mean-shift maps after removing one particle. Under such bounds one expects leave-one-out perturbations to be of order 1/N times an h-dependent stability constant, but deriving the sharp dependence is a separate stability problem. The finite-particle rate in Theorem 6.9 below should therefore be interpreted as conditional on the displayed εS,h,N and εV,h,N , rather than as an unconditional N, h-explicit theorem for the practical minibatch algorithm.
31
Theorem 6.9 (Continuous-time finite-particle rate for non-conservative Laplace drifting). Let Assumptions 6.2, 6.5, and 6.7 hold on [0, T ]. Then 1 T
Z T 0
# HN (0) βh 2 εS,h,N Lap Et [VN (X(t))] dt ≤ + ∆ + + εV,h,N , γh N T γh h γh
where
λh , 4L2h
γh =
βh =
(66)
λh 1 + . 2 2Lh 2λh
# In particular, if HN (0) ≤ κ0 N and T = N , then
1 N
Z N 0
Lap Et [VN (X(t))] dt ≤
βh 2 εS,h,N κ0 + ∆ + + εV,h,N . γh N γh h γh
(67)
Equivalently,
1 N
Z N 0
1/2
Lap Et [VN (X(t))] dt
r ≤
κ0 + γh N
s
βh ∆h + γh
r
εS,h,N √ + εV,h,N . γh
(68)
Proof. By Theorem 6.3, d # H (t) = −N Et [SLap N (X(t))]. dt N Integrating from 0 to T , # # HN (T ) − HN (0) = −N
Z T 0
Et [SLap N (X(t))] dt.
Rearranging, Z T 0
# # (T ) (0) − HN HN Lap Et [SN (X(t))] dt = .
N
Since relative entropy is nonnegative, # (T ) ≥ 0. HN
Therefore 1 T
Z T 0
Et [SLap N (X(t))] dt ≤
# HN (0) . NT
Next, fix a configuration x visited by the flow. By Assumption 6.7, Lap SLap N (x) ≥ Jh (x) − εS,h,N .
By Lemma 6.6, JhLap (x) ≥ γh VhLap (x) − βh ∆2h . Therefore Lap 2 SLap N (x) ≥ γh Vh (x) − βh ∆h − εS,h,N .
Again by Assumption 6.7, Lap VhLap (x) ≥ VN (x) − εV,h,N .
Substituting this lower bound gives Lap 2 SLap N (x) ≥ γh {VN (x) − εV,h,N } − βh ∆h − εS,h,N .
32
(69)
Thus Lap 2 γh VN (x) ≤ SLap N (x) + βh ∆h + εS,h,N + γh εV,h,N .
Apply this with x = X(t), take expectations, integrate over t ∈ [0, T ], and divide by T : 1 γh T
Z T 0
1 Lap Et [VN (X(t))] dt ≤ T
Z T 0
2 Et [SLap N (X(t))] dt + βh ∆h + εS,h,N + γh εV,h,N .
Using (69), 1 γh T
Z T 0
# HN (0) Lap Et [VN (X(t))] dt ≤ + βh ∆2h + εS,h,N + γh εV,h,N .
NT
# Dividing by γh proves (66). If HN (0) ≤ κ0 N and T = N , then # HN (0) κ0 N κ0 ≤ = , 2 γh N T γh N γh N √ √ √ √ √ which proves (67). Finally, (68) follows from a + b + c + d ≤ a + b + c + d for nonnegative a, b, c, d.
Remark 6.10 (Meaning of the Laplace rate). The first term in (66) is the same entropy-driven optimization term as in the conservative theorem, except that it is divided by the Laplace coercivity constant γh . The terms εS,h,N and εV,h,N are finite-particle approximation errors: they compare the leave-one-out, center-evaluated quantities to the full-field sharp-smoothed quantities. The term βh 2 ∆ γh h in Theorem 6.9 is an irreducible residual term for the original non-conservative drifting method with Laplace kernel. It is not a proof artifact; it measures the fact that the non-conservative Laplace displacement field is not an exact score mismatch. It is also not a finite-particle fluctuation term and it does not vanish merely by taking N → ∞ with h fixed. Consequently, the theorem proves convergence to a residual neighborhood whose size is controlled by ∆h , not convergence to zero residual unless an additional alignment condition forces ∆h → 0. Using the identity aα,h (z) = h{r̄α,h (z) + h}, the residual can be written as ex (z) = h{r̄ν,h (z) − r̄x,h (z)}σν,h (z). Therefore, if ∥σν,h ∥∞ ≤ Gh
and
sup |r̄ν,h (z) − r̄x,h (z)| ≤ δr,h , z
then 2 ∆2h ≤ h2 G2h δr,h .
Thus the non-conservative Laplace method has a vanishing residual only in regimes where the model and data local Laplace-weighted radii align. Without a separate argument giving δr,h → 0, the theorem should be read as a convergence-to-neighborhood result.
33
6.5
Checkable forms of the Laplace assumptions
The scale-alignment residual has a concrete interpretation in terms of local weighted radii. By Lemma 6.1, aα,h (z) = h{r̄α,h (z) + h}. Therefore aν,h (z) − ax,h (z) = h{r̄ν,h (z) − r̄x,h (z)}, and the residual is ex (z) = h{r̄ν,h (z) − r̄x,h (z)}σν,h (z). Corollary 6.11 (Shell-alignment sufficient condition). Assume that, along the trajectory on [0, T ], there are constants 0 ≤ r−,h ≤ r+,h < ∞, δr,h ≥ 0, Gh < ∞, such that, for all configurations visited by the flow and all z ∈ Rd , r−,h ≤ r̄x,h (z) ≤ r+,h , |r̄ν,h (z) − r̄x,h (z)| ≤ δr,h , and ∥σν,h (z)∥ ≤ Gh . Then Assumption 6.5 holds with λh = h(r−,h + h),
Lh = h(r+,h + h),
and 2 . ∆2h ≤ h2 G2h δr,h
Consequently, if the hypotheses of Theorem 6.9 also hold, then 1 T
Z T 0
Lap (X(t))] dt ≤ Et [VN
# (0) βh 2 2 2 HN εS,h,N + h Gh δr,h + + εV,h,N . γh N T γh γh
Proof. Since ax,h (z) = h{r̄x,h (z) + h}, the bound r−,h ≤ r̄x,h (z) ≤ r+,h gives h(r−,h + h) ≤ ax,h (z) ≤ h(r+,h + h). Thus (60) holds with the stated λh and Lh . Next, ex (z) = h{r̄ν,h (z) − r̄x,h (z)}σν,h (z). Taking norms, ∥ex (z)∥ ≤ h |r̄ν,h (z) − r̄x,h (z)| ∥σν,h (z)∥ . Using the assumed bounds, ∥ex (z)∥ ≤ hδr,h Gh . Squaring, 2 ∥ex (z)∥2 ≤ h2 δr,h G2h .
34
Since ρ# x,h is a probability density, Z
2 2 2 ∥ex (z)∥2 ρ# x,h (z) dz ≤ h δr,h Gh .
2 . Substitution into Theorem 6.9 proves the final display. Thus ∆2h ≤ h2 G2h δr,h
We next give one sufficient route to Assumption 6.7. Let Z Lh (u) L̄h (u) := , m2,#,h := ∥u∥2 L̄h (u) du. Z#,h Rd Because L̄h is a bandwidth-h radial kernel, m2,#,h = h2 m2,#,1 for a constant m2,#,1 depending only on dimension. Lemma 6.12 (A sufficient quadrature bound). Fix a configuration x. Define ψx (z) := ∥ux (z)∥2 .
ϕx (z) := A# ux (z), Assume that sup D2 ϕx (z) op ≤ Bϕ,h ,
sup D2 ψx (z) op ≤ Bψ,h .
z∈Rd
z∈Rd
Assume also that the leave-one-out point errors satisfy N
1 X |A# ux,−i (xi ) − A# ux (xi )| ≤ ℓS,h,N , N i=1
and
N
1 X ∥ux,−i (xi )∥2 − ∥ux (xi )∥2 ≤ ℓV,h,N . N i=1
Then
and
1 Lap SLap N (x) − Jh (x) ≤ Bϕ,h m2,#,h + ℓS,h,N , 2 1 Lap VN (x) − VhLap (x) ≤ Bψ,h m2,#,h + ℓV,h,N . 2
Proof. We prove the first inequality. The second is identical with ψx in place of ϕx . Since N 1 X ρ# (z) = L̄h (z − xi ), x,h N i=1
we have JhLap (x) =
Z
1 ϕx (z)ρ# x,h (z) dz = N
N Z X
ϕx (z)L̄h (z − xi ) dz.
i=1
Also, N
N
i=1
i=1
1 X 1 X ϕx (xi ) = A# ux (xi ). N N
35
Therefore
N
N
i=1
i=1
1 X 1 X ϕx (xi ) − JhLap (x) ≤ ϕx (xi ) − N N
Z ϕx (z)L̄h (z − xi ) dz .
Set u = z − xi . Then Z
Z ϕx (z)L̄h (z − xi ) dz =
ϕx (xi + u)L̄h (u) du.
Taylor’s formula gives Z 1 ϕx (xi + u) = ϕx (xi ) + ∇ϕx (xi ) · u +
(1 − r)u⊤ D2 ϕx (xi + ru)u dr.
0
Integrating against L̄h (u) du, the constant term gives ϕx (xi ). The first-order term vanishes because L̄h is even, hence Z uL̄h (u) du = 0. Thus Z ϕx (xi ) −
Z 1
Z ϕx (xi + u)L̄h (u) du ≤
L̄h (u) 0
(1 − r) D2 ϕx (xi + ru) op ∥u∥2 dr du.
Using the Hessian bound, Z Z 1 Z ϕx (xi ) − ϕx (xi + u)L̄h (u) du ≤ Bϕ,h (1 − r) dr ∥u∥2 L̄h (u) du. 0
Since
R1
0 (1 − r) dr = 1/2,
Z ϕx (xi ) −
1 ϕx (xi + u)L̄h (u) du ≤ Bϕ,h m2,#,h . 2
Averaging over i gives N
1 X 1 A# ux (xi ) − JhLap (x) ≤ Bϕ,h m2,#,h . N 2 i=1
Finally, N
SLap N (x) =
1 X A# ux,−i (xi ). N i=1
By the assumed leave-one-out point-error bound, N
SLap N (x) −
1 X A# ux (xi ) ≤ ℓS,h,N . N i=1
The triangle inequality gives 1 Lap SLap N (x) − Jh (x) ≤ Bϕ,h m2,#,h + ℓS,h,N . 2 This proves the first bound. 36
Finally, the leave-one-out denominators are controlled by the same local-occupancy mechanism used for conservative drifting. Lemma 6.13 (Leave-one-out Laplace denominator control). Fix r0 > 0. For the Laplace kernel, Kh (u) ≥ cd h−d e−r0
whenever ∥u∥ ≤ r0 h.
For a configuration x, define (−)
ni (x) := #{j ̸= i : ∥xj − xi ∥ ≤ r0 h}. If (−)
ni (x) ≥ α(N − 1)hd
for every i,
(xi ) ≥ cd e−r0 α
for every i.
then QµN
x,−i ,h
Proof. By definition, QµN
(xi ) = x,−i ,h
1 X Kh (xi − xj ). N −1 j̸=i
For every j ̸= i satisfying ∥xj − xi ∥ ≤ r0 h, −d
Kh (xi − xj ) = cd h Therefore
∥xi − xj ∥ exp − ≥ cd h−d e−r0 . h 1 (−) ni (x)cd h−d e−r0 . N −1
QµN
(xi ) ≥
(xi ) ≥
1 {α(N − 1)hd }cd h−d e−r0 = cd e−r0 α. N −1
x,−i ,h
(−)
Using ni (x) ≥ α(N − 1)hd , QµN
x,−i ,h
This proves the claim.
6.6
Bandwidth and step-size choices for the non-conservative drifting method with Laplace kernel
The leave-one-out Laplace theorem has a different bandwidth tradeoff from the conservative theorem. Because the self-masked field removes the self-interaction term from the entropy identity, there is no analogue of the conservative self-interaction term 1/(N hd+2 ) in (66). Instead, the bandwidth appears through the scale-mismatch residual and the quadrature and leave-one-out errors. A useful simplified form is obtained from Corollary 6.11 and Lemma 6.12. Suppose that along the trajectory βh 2 2 ∆ ≤ C∆,h h2 G2h δr,h , γh h and
Cloo,h εS,h,N + εV,h,N ≤ CQ,h h2 + . γh N
37
# Then, for T = N and HN (0) ≤ κ0 N , Theorem 6.9 gives
1 N
Z N 0
Lap Et [VN (X(t))] dt ≤
Cloo,h κ0 2 + C∆,h h2 G2h δr,h + CQ,h h2 + . γh N N
(70)
If γh is bounded below and C∆,h , CQ,h , Cloo,h remain bounded, then the deterministic part of the rate improves as h decreases, provided hGh δr,h also decreases or remains controlled. The bandwidth should therefore be chosen as small as permitted by denominator control and regularity. A typical local-occupancy requirement is N hd ≳ log N, which suggests hN ≍
log N N
1/d (71)
when the constants in (70) are stable at that scale. In the benign case where Gh δr,h = O(1), this gives Z 1 N 1 log N 2/d Lap Et [VN (X(t))] dt ≲ + , N 0 N N up to the displayed constants and residual assumptions. The corresponding root residual velocity is O(N −1/2 ) + O (log N/N )1/d . If the shell mismatch is worse, the term hGh δr,h should be kept explicitly; if it is better, the rate improves accordingly. The one-step drift size η again enters only when the continuous-time bound is converted into a statement about an explicit correction. For the non-conservative Laplace field, x ei = xi + ηux,−i (xi ) satisfies 2 Lap W22 (µN eN x ,µ x ) ≤ η VN (x).
Thus a typical residual correction at the optimized bandwidth (71) is of order n o η N −1/2 + (log N/N )1/d + hN GhN δr,hN in root mean square. If the explicit displacement map must be a stable Euler step and the leaveone-out Laplace field has Lipschitz constant Lu (h), a sufficient condition is ηLu (h) ≤ c < 1. Under denominator control and bounded local-radius derivatives, the non-conservative displacement field often has Lu (h) = O(1), in which case a constant-order η is admissible. If Lu (h) grows with decreasing h, the admissible step size should be reduced according to η ≍ 1/Lu (h).
6.7
Comparison between the conservative and non-conservative results
The conservative velocity field studied earlier is bx (z) = sρ (z) − sx (z),
38
where sρ = ∇ log ρ is the score of the smoothed target and sx = ∇ log qx is the score of the model KDE. Its squared center velocity is N
cons VN (x) := VN (x) =
1 X ∥bx (xi )∥2 . N i=1
The finite-particle rate for this conservative method has the form Z κ0 1 N Cself cons Et [VN (X(t))] dt ≤ + Cquad (h)h2 , + N 0 N N hd+2 where the second term is the reciprocal-KDE self-interaction correction and 1 Cquad (h) = {BA,N (h) + BV,N (h)}m2 (K) 2 is the point-evaluation quadrature constant. Hence the conservative method has a clean score/Fisher structure, but its optimized bandwidth rate depends critically on the h-dependence of the quadrature constants. If Cquad (h) = O(1), the root residual-velocity rate is N −1/(d+4) . If Cquad (h) = O(h−β ) with 0 ≤ β < 2, the optimized root rate is instead N −(2−β)/(2(d+4−β)) . If β ≥ 2, the present quadrature argument does not give a vanishing rate by shrinking h. For the non-conservative drifting method with Laplace kernel, the corresponding rate is Z κ0 1 N βh 2 εS,h,N Lap (X(t))] dt ≤ Et [VN + ∆ + + εV,h,N . N 0 γh N γh h γh The first term is the same entropy-driven finite-particle optimization term, up to the Laplace coercivity constant γh . The main new term is βh 2 ∆ , γh h which measures the failure of the non-conservative Laplace mean-shift field to be an exact scoredifference field. Explicitly, Z 2 ∆h ≥ ∥{aν,h (z) − ax,h (z)}σν,h (z)∥2 ρ# x,h (z) dz. Equivalently, since aα,h (z) = h{r̄α,h (z) + h}, this residual is controlled by the mismatch between the data and model local Laplace-weighted radii: aν,h (z) − ax,h (z) = h{r̄ν,h (z) − r̄x,h (z)}. The two theories therefore share the same entropy backbone but differ in their coercivity mechanisms. For the conservative method, the drift is already a score mismatch, so the Stein identity directly gives a Fisher-type quantity after quadrature. For the non-conservative Laplace method, the drift decomposes as ux (z) = ax,h (z)b# x (z) + ex (z), 39
where ax,h is a positive scalar preconditioner and ex is the Laplace scale-mismatch residual. Thus the non-conservative method has a comparable rate only when the local scale factors of the data and model are aligned well enough that ∆h is small. There is also a difference in the finite-particle correction. The conservative theorem above uses the center-evaluation field with the full KDE, so differentiating the ith particle’s own KDE contribution produces a reciprocal-KDE self-interaction correction. In the non-conservative Laplace result we use the self-masked, leave-one-out field ux,−i . This removes the self-interaction term from the entropy identity. The price is the leave-one-out approximation error contained in εS,h,N and εV,h,N , which is controlled by denominator lower bounds and local occupancy conditions such as Lemma 6.13. Finally, the target density used by the two analyses is different. The conservative theorem uses the ordinary smoothed target ρ = Kh ∗ ν. The non-conservative Laplace theorem uses the sharp-smoothed target Z 1 # ρν,h (z) = Lh (z − y)ν( dy), Lh (u) = h(∥u∥ + h)Kh (u). Z#,h This difference is not an artifact of the proof. It reflects the identity ∇z Lh (z − y) = (y − z)Kh (z − y), which is the structural reason the non-conservative Laplace displacement field can be related to a score mismatch at all. In the Gaussian case, the non-conservative displacement field is simply a constant multiple of the conservative field; in the Laplace case, it is instead a variable-scale preconditioned sharp-score field plus the residual ex .
References Michael Samuel Albergo and Eric Vanden-Eijnden. Building Normalizing Flows with Stochastic Interpolants. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=li7qeBbCR1t. Sayan Banerjee, Krishnakumar Balasubramanian, and Promit Ghosal. Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id= sbG8qhMjkZ. Andrew D Barbour. Stein’s method and Poisson process convergence. Journal of Applied Probability, 25(A):175–184, 1988. Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum? id=cqDH0e6ak2. Jiarui Cao, Zixuan Wei, and Yuxin Liu. Gradient flow drifting: Generative modeling via Wasserstein gradient flows of KDE-approximated divergences. arXiv preprint arXiv:2603.10592, 2026. Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative modeling via drifting. arXiv preprint arXiv:2602.04770, 2026. 40
Théo Dumont, Théo Lacombe, and François-Xavier Vialard. Learning Monge maps with constrained drifting models. arXiv preprint arXiv:2603.25182, 2026. Maria Esteban-Casadevall, Jorge Carrasco-Pollo, Max Welling, Jan-Willem van de Meent, Erik J. Bekkers, and Floor Eijkelboom. Kernel-gradient drifting models. arXiv preprint arXiv:2605.10727, 2026. Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized Stein variational gradient descent. In The Conference on Uncertainty in Artificial Intelligence (UAI), 2017. Leonard Franz, Sebastian Hoffmann, and Georg Martius. Drifting fields are not conservative. arXiv preprint arXiv:2604.06333, 2026. Jackson Gorham and Lester Mackey. Measuring sample quality with Stein’s method. Advances in neural information processing systems, 28, 2015. Arthur Gretton, Li Kevin Wenliang, Alexandre Galashov, James Thornton, Valentin De Bortoli, and Arnaud Doucet. On the Wasserstein Gradient Flow Interpretation of Drifting Models. arXiv preprint arXiv:2605.05118, 2026. Ping He, Om Khangaonkar, Hamed Pirsiavash, Yikun Bai, and Soheil Kolouri. Sinkhorn-Drifting Generative Models. arXiv preprint arXiv:2603.12366, 2026a. Ye He, Krishnakumar Balasubramanian, Sayan Banerjee, and Promit Ghosal. Finite-Particle Rates for Regularized Stein Variational Gradient Descent. arXiv preprint arXiv:2602.05172, 2026b. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851, 2020. Chieh-Hsin Lai, Bac Nguyen, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yuki Mitsufuji, Stefano Ermon, and Molei Tao. A unified view of drifting and score-based models. arXiv preprint arXiv:2603.07514, 2026. Hak Geun Lee and Hyonho Chun. Identifiability and Stability of Generative Drifting with Companion-Elliptic Kernel Families. arXiv preprint arXiv:2604.24196, 2026. Zhiqi Li and Bo Zhu. A Long-Short Flow-Map Perspective for Drifting Models. arXiv preprint arXiv:2602.20463, 2026. Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PqvMRDCJT9t. Qiang Liu. Stein variational gradient descent as gradient flow. Advances in neural information processing systems, 30, 2017. Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016. Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=XVjTT1nw5z.
41
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL https://openreview.net/ forum?id=TIdIXIpzhoI. Jiaxin Shi and Lester Mackey. A finite-particle convergence rate for Stein variational gradient descent. Advances in Neural Information Processing Systems, 36:26831–26844, 2023. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/ forum?id=PxTIG12RRHS. Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, 2023. Erkan Turan and Maks Ovsjanikov. Generative drifting is secretly score matching: A spectral and variational perspective. arXiv preprint arXiv:2603.09936, 2026. Romann M. Weber. The Score-Difference Flow for Implicit Generative Modeling. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id= dpGSNLUCzu. Guoqiang Zhang, Kenta Niwa, and W. Bastiaan Kleijn. Lookahead drifting model. arXiv preprint arXiv:2605.04060, 2026.
A
Stop-gradient training and the particle ODE
We briefly explain how the continuous-time particle dynamics arise from the practical stop-gradient training rule. Let gθ : Z → Rd be the generator and let xki := gθk (ξi ),
i = 1, . . . , N,
be the generated batch at training step k, for fixed latent variables ξ1 , . . . , ξN . Let vxk denote the drift field computed from the current generated batch xk = (xk1 , . . . , xkN ) and the data batch. In the original displacement-based drifting method (Deng et al., 2026), vxk is the non-conservative field udisp ν,µxk ,h ; in the conservative method studied in this paper, it is the KDE-score field bxk . The practical target for the i-th generated sample is yik := sg{xki + ηvxk (xki )}. Here sg denotes the stop-gradient operator. For any quantity a(θ) appearing in the computational graph, sg{a(θ)} = a(θ)
∇θ sg{a(θ)} = 0
in the forward pass,
in the backward pass.
Thus sg preserves the numerical value of its argument but removes all derivatives through that argument. In particular, the target yi = sg{xi + ηvx (xi )} is evaluated as xi + ηvx (xi ), but is treated as a constant when differentiating the regression loss with respect to the generator parameters. 42
The corresponding regression loss is N
2 1 X Lk (θ) := gθ (ξi ) − yik . 2N
(72)
i=1
The stop-gradient operation means that yik is treated as a constant when differentiating Lk with respect to θ. Thus the derivative of the loss is N
∇θ Lk (θ) =
1 X Dθ gθ (ξi )⊤ {gθ (ξi ) − yik }, N i=1
with no derivative through the map xk 7→ vxk . Therefore the training step is a frozen-target regression problem: the generator is trained to move its current outputs toward the fixed points xki + ηvxk (xki ). The idealized exact-regression limit is the limit in which this regression problem is solved exactly at every step and the generator class is rich enough to interpolate the frozen targets on the sampled latent variables. In that limit, the next generator satisfies gθk+1 (ξi ) = yik = xki + ηvxk (xki ),
i = 1, . . . , N.
Consequently the generated particles obey the explicit Euler update xk+1 = xki + ηvxk (xki ), i
i = 1, . . . , N.
(73)
If the regression is not exact, then (73) holds with an additional approximation error eki := gθk+1 (ξi ) − {xki + ηvxk (xki )}. Our analysis neglects this optimization and approximation error and studies the idealized particle system (73). Finally, define tk = kη and let Xiη (t) be the piecewise-linear interpolation of the Euler iterates. Under the usual consistency and stability conditions for explicit Euler schemes, for example local Lipschitz continuity of the interacting-particle vector field Fi (x) := vx (xi ),
x = (x1 , . . . , xN ),
the interpolation X η (t) converges as η → 0 on finite time intervals to the solution of Ẋi (t) = Fi (X(t)) = vX(t) (Xi (t)),
i = 1, . . . , N.
Here, vx can either be the conservative KDE-score field or the non-conservative leave-one-out Laplace field. These are the continuous-time particle dynamics analyzed in the main text. Thus stop-gradient is used to justify the frozen-field Euler update; once this particle ODE is assumed, the entropy identities and convergence bounds are properties of the resulting interacting-particle flow. We emphasize that the passage from the stop-gradient Euler update to the continuous-time ODE is an idealized modeling step. Let F (x) := (F1 (x), . . . , FN (x)). The exact-regression stop-gradient update is xk+1 = xk + ηF (xk ). 43
If F is Lipschitz on a forward-invariant region D ⊂ (Rd )N with Lipschitz constant Lh,D , and if the Euler iterates and the ODE trajectory remain in D, then the standard explicit-Euler estimate gives, on a fixed time interval [0, T ], max
0≤kη≤T
xk − X(kη) ≤ CT,h,D η,
where CT,h,D depends on T , Lh,D , and a bound on F on D. For the interacting-particle systems studied here, Lh,D can grow rapidly as h → 0 and depends on denominator lower bounds such as qx (xi ) ≥ λ. Therefore the continuous-time approximation requires a step-size condition of the form ηLh,D ≪ 1. If, for instance, Lh,D = O(h−α ), then one needs η = o(hα ) for the Euler approximation error to vanish. Thus our continuous-time results should not be interpreted as unconditional finite-step guarantees for the practical algorithm. They describe the idealized stop-gradient, exact-regression, small-step limit under independent stability and denominatorcontrol assumptions. If the regression step is not exact, the particle update has the form xk+1 = xk + ηF (xk ) + ek ., as mentioned above. Under the same Lipschitz assumptions, the discrete-to-continuous error contains an additional accumulated term of size X eLh,D (T −kη) ek . kη≤T
Thus tracking the practical neural-network training dynamics would require bounds on optimization error, approximation error, minibatch noise, and the h-dependent stability constant. These effects are outside the scope of the continuous-time entropy analysis.
44