Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling Yiwei Zhou∗
arXiv:2607.08757v1 [stat.ML] 9 Jul 2026
Abstract Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. We show that small forward-marginal error does not guarantee numerical stability. We construct a single smooth score field with arbitrarily small forward-marginal L2 error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler–Maruyama discretizations converge in probability while every positive moment diverges. Thus weak convergence can hold even though every Wasserstein distance Wp , p ≥ 1, diverges. The same failure can occur within one fixed finite neural architecture. We construct a family of bounded, globally Lipschitz denoisers for which both the forward-marginal error and the path-space total variation distance tend to zero, while their Euler–Maruyama endpoints diverge in every Wp . For compactly supported data, we also give a simple positive result. Projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, gives grid-uniform moment bounds, and yields Wasserstein convergence under mild local regularity. Experiments with a small fixed DiT-style network show large growth along rare numerical trajectories and its suppression by denoiser projection, while overall trajectory errors remain small. Keywords: score-based diffusion models; reverse-time sampling; numerical stability; Euler– Maruyama discretization; Wasserstein convergence; denoiser projection. MSC 2020: Primary 65C30; Secondary 60H35, 60J60, 68T07.
1
Introduction
In score-based diffusion models, the exact reverse-time process uses the true score st in its drift. Replacing st by a learned approximation sbt gives the learned reverse-time process, which is then Euler–Maruyama discretized for sampling. A standard score-learning error has the form Z T Escore = EXt ∼pt ∥b st (Xt ) − st (Xt )∥2 dt. (1) δ ∗
Email: [email protected]
1
We call it on-path score error because it is measured under the forward marginals. A natural question is: “Suppose both the exact and learned reverse-time processes are stable, and the learned process can be made arbitrarily close to the exact one in path-space total variation. Is an arbitrarily small on-path score error then enough to guarantee stability of the discretization?” Perhaps surprisingly, the answer is no. We construct a single smooth score field with arbitrarily small on-path error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler–Maruyama endpoints converge in probability while every positive moment diverges. Weak convergence holds while the Wasserstein distance Wp diverges for every p ≥ 1. The same mechanism extends to arbitrary dimension and to probability-flow ODEs. Furthermore, this error alone does not rank samplers by numerical stability. Two smooth score fields can have the same arbitrarily small on-path error while one Euler–Maruyama discretization has grid-uniform moments of every order and the other loses every positive moment. The same obstruction persists even within a fixed finite GELU architecture: bounded, globally Lipschitz denoisers can have arbitrarily small on-path error and path-space total variation distance while their Euler–Maruyama endpoints diverge in every Wp . The mechanism is simple. We place an inward superlinear perturbation in a remote region carrying very little mass under the forward marginals. The learned reverse-time process rarely reaches this region, so its path law remains close to that of the exact reverse-time process. A rare numerical trajectory can nevertheless enter it, after which explicit updates trigger repeated superlinear amplification. The growth along these rare trajectories outweighs their probability cost and destroys every positive moment. This mechanism is related to the classical instability of explicit Euler schemes for superlinear SDEs [8], but here the unstable region carries negligible weight under the on-path score error while the learned reverse-time process remains stable. We also give a simple positive result. Suppose the data are supported in a known bounded closed convex set C. Since the exact denoiser Dt (x) = E[X0 | Xt = x] lies in C, we project the learned denoiser onto C before forming the score. This projection cannot increase the pointwise denoiser or score error, while making the deterministic part of each Euler–Maruyama update pull toward a bounded set. The projected chain has grid-uniform moment bounds, so weak numerical convergence implies Wasserstein convergence. For image data scaled to a box, this projection coincides with clipping the predicted clean sample, a heuristic used in image diffusion models [13, 14]. Our result provides a stability interpretation of this practice. Related work studies diffusion sampling under distribution-weighted score error and additional assumptions that yield path-space or endpoint guarantees [2, 3, 4]. Wasserstein convergence has also been established under assumptions on the data law, score regularity, or dynamics [5, 6], while contraction has been used as a direct stability principle for reverse-time sampling [10]. Recent work has questioned what scalar score-error measures can certify [7, 9]. Our results identify a different obstruction: even when the learned continuous-time dynamics are stable and close to the exact process, explicit discretization can lose all moment and Wasserstein stability because numerical trajectories may move into regions poorly represented by the forward marginals.
2
2
Setup
We work with the variance-exploding (VE) Gaussian noising process √ Xt = X0 + t g, g ∼ N (0, Id ), t ∈ [0, T ]. Let pt denote the law of Xt . Tweedie’s identity gives st (x) = ∇ log pt (x) =
Dt (x) − x , t
Dt (x) = E[X0 | Xt = x].
For a learned score sbt , we use the associated denoiser parameterization b t (x) = x + tb D st (x). When X0 is supported in a compact set K ⊂ B(0, M ), let C0 = conv(K). Conditional expectation then implies Dt (x) ∈ C0 ,
∥st (x)∥ ≤
M + ∥x∥ . t
Thus, for bounded data and t ≥ δ > 0, the exact score grows at most linearly in space. Our concern is different: an approximate score sbt may be poorly controlled in regions that carry little mass under the forward marginals. We consider a decreasing grid T = tK > tK−1 > · · · > t0 = δ > 0,
hk = tk+1 − tk ,
ρk =
hk . tk+1
Because the grid is strictly decreasing and positive, every step satisfies 0 < ρk < 1. A simple reverse Euler step driven by a score field s has the form p (2) Yk = Yk+1 + hk stk+1 (Yk+1 ) + hk ξk , with independent standard Gaussian ξk . The chain runs from k = K (t = T ) down to k = 0 (t = δ), and Y0 is the generated sample. Notation. X denotes the forward noising process, U the exact reverse-time process, Z the learned reverse-time process, and Y an Euler–Maruyama chain. The exact, learned, and projected b and DP . For a random variable scores are s, sb, and sP ; the corresponding denoisers are D, D, or process V , L(V ) denotes its law. We write TV(ν, µ) for total variation distance, KL(ν∥µ) for relative entropy, and Pp (Rd ) for the probability measures on Rd with finite pth moment.
3
3
The blind spot of on-path score accuracy
Since pt has rapidly decaying tails, the score can be changed far outside the typical region while adding only an arbitrarily small amount to the L2 (pt ) budget. The main difficulty is to turn this weak visibility into numerical instability. A one-step perturbation of amplitude aR at radius R pays essentially the same rare-event factor in both the score-error budget and the resulting moment error: under a Gaussian-tailed forward marginal, the budget contribution is of order 2
a2R e−cR . The separation therefore needs repeated evaluation. Once an Euler trajectory enters the perturbed tail, successive explicit updates can compound superlinear growth faster than the entry probability decays.
3.1
A fixed score field with weak convergence and Wasserstein divergence
We fix the learned score and time horizon and vary only the grid resolution. Theorem 3.1 (Weak convergence with Wasserstein divergence). Let µ0 be any probability law supported in [−M, M ], let pt = µ0 ∗ N (0, t), and fix 0 < δ < T < ∞. For every ε > 0, η > 0, λ > 0, and α > 0, there exist R > 0 and an even cutoff χR ∈ C ∞ (R; [0, 1]), with χR = 0 on [−R, R] and χR = 1 outside [−R − 1, R + 1], such that the single score field sbt (x) = st (x) − λχR (x)x|x|α ,
t ∈ [δ, T ],
(3)
has all of the following properties. 1. On-path accuracy. Its error under the forward marginals is uniformly small in time: sup Ept |b st − st |2 ≤ ε.
(4)
t∈[δ,T ]
2. Stable and nearly exact continuous dynamics. There exist constants c, C > 0 such that, uniformly in t ∈ [δ, T ], xb st (x) ≤ C − c|x|2+α , x ∈ R. (5) Consequently the learned reverse-time process Z0 ∼ pT ,
dZu = sbT −u (Zu ) du + dBu ,
0 ≤ u ≤ T − δ,
(6)
has a unique nonexplosive strong solution and, for every q ≥ 2, sup 0≤u≤T −δ
E|Zu |q < ∞.
(7)
Let U denote the exact reverse-time process dUu = sT −u (Uu ) du + dBu , 4
U0 ∼ pT ,
driven by the same Brownian normalization and terminal law. The two continuous path laws can be made arbitrarily close: TV L (Zu )0≤u≤T −δ , L (Uu )0≤u≤T −δ ≤ η. (8) 3. Discrete moment failure despite weak convergence. Let Km = 2m and use the nested geometric grids T j (Km ) log , j = 0, . . . , Km . (9) tj = δ exp Km δ (K )
Initialize YKmm ∼ pT and apply the Euler–Maruyama recursion (2) with the same field (3) on every grid. Couple the Euler noises to the Brownian increments of (6). Then (Km )
Y0
−→ ZT −δ
in probability.
(10)
as m → ∞.
(11)
Nevertheless, for every q > 0, (Km ) q
| −→ ∞
E|Y0
Consequently the Euler laws converge weakly to the learned endpoint, but for every p ≥ 1 and every fixed ν ∈ Pp (R), (K ) Wp L(Y0 m ), ν −→ ∞. (12) In particular this holds for ν = L(ZT −δ ), ν = pδ , and ν = µ0 . Proof. Step 1: make the on-path error small. The exact score satisfies the global linear envelope |Dt (x) − x| |x| + M |st (x)| = ≤ , (x, t) ∈ R × [δ, T ]. (13) t δ √ Under the coupling Xt = X0 + tg with |X0 | ≤ M almost surely, sup Ept |b st − st |2 ≤ λ2 sup E |Xt |2+2α 1{|Xt |>R} t∈[δ,T ]
t∈[δ,T ]
i h √ 2+2α √ 1{M + T |g|>R} . ≤ λ E (M + T |g|) 2
The last expression tends to zero as R → ∞ by dominated convergence. We can therefore choose R so that (4) holds. Step 2: control the continuous-time process. For |x| ≥ R + 1, the cutoff is fully active, and (13) gives |x|2 + M |x| xb st (x) ≤ − λ|x|2+α . δ As |x| → ∞, the negative superquadratic term dominates the quadratic and linear terms. Hence there is R0 < ∞ such that, uniformly in t ∈ [δ, T ], λ xb st (x) ≤ − |x|2+α , 2 5
|x| ≥ R0 .
On the compact set [δ, T ] × [−R0 , R0 ], the continuous function λ (t, x) 7−→ xb st (x) + |x|2+α 4 is bounded above. Enlarging the constant if necessary therefore gives (5) for suitable c, C > 0. We next check local well-posedness. For every t ≥ δ, pt is a positive Gaussian convolution, so the exact score st = ∂x log pt is smooth on [δ, T ] × R. The cutoff χR vanishes near the origin, so χR (x)x|x|α is smooth even when α is nonintegral. Thus sbt is smooth. The reverse drift b(u, x) = sbT −u (x) is therefore locally Lipschitz in x, uniformly in u ∈ [0, T − δ]: for every H < ∞, sup
|∂x b(u, x)| < ∞.
0≤u≤T −δ |x|≤H
Hence the learned reverse-time process admits a unique strong solution up to its explosion time. For the time-inhomogeneous generator 1 Lu f (x) = b(u, x)f ′ (x) + f ′′ (x), 2 consider Vm (x) = (1 + x2 )m . Using (5), the drift term satisfies 2m(1 + x2 )m−1 x b(u, x) ≤ 2m(1 + x2 )m−1 (C − c|x|2+α ). The diffusion term 12 Vm′′ (x) has lower polynomial order, so for every integer m ≥ 1 there is Cm < ∞, uniform in u ∈ [0, T − δ], such that Lu Vm (x) ≤ Cm Vm (x). For m = 1, apply Itô’s formula up to the exit time from [−n, n] and then let n → ∞. The resulting bound rules out explosion. The same stopped argument for general m, followed by Grönwall’s inequality, gives uniform bounds for all even moments on the finite horizon. Since pT has moments of every order, interpolation gives (7) for every real q ≥ 2. Step 3: compare the learned and exact path laws. Next, couple the learned and exact reverse-time processes with the same initial draw from pT and the same Brownian motion. Because sbt = st on [−R, R], pathwise uniqueness implies that the two solutions agree until their first exit from this interval. The coupling inequality therefore gives TV(L((Zu )u≤T −δ ), L((Uu )u≤T −δ )) ≤ P sup |Uu | ≥ R . 0≤u≤T −δ
The exact reverse-time process has continuous paths on a compact time interval, so the probability on the right tends to zero as R → ∞. We may therefore enlarge the radius chosen in Step 1 until this probability is at most η. Enlarging R can only decrease the on-path error bound, while the coercivity argument of Step 2 still applies with new constants. This gives (8). Step 4: prove Euler convergence in probability. Write b(u, x) = sbT −u (x). Let πH (x) = (−H) ∨ x ∧ H and define the truncated drift bH (u, x) = b(u, πH (x)). Since b is smooth on 6
[0, T − δ] × [−H, H], the truncated drift is globally Lipschitz in space, uniformly in time, and uniformly continuous in time on the compact horizon. Let Z H and Y H,K be the corresponding SDE and Euler approximation, coupled with the same initial condition and Brownian increments. Standard Euler convergence for globally Lipschitz, time-inhomogeneous drifts on meshes with vanishing maximal step gives [15] H,K max YK−j − ZuHj −→ 0 in probability,
0≤j≤K
(14)
where uj = T − tK−j is the corresponding increasing-time grid. Fix ζ > 0. By nonexplosion and continuity of Z, choose H so large that P sup |Zu | ≥ H − 1 < ζ. u≤T −δ
On the complementary event, Z H = Z. By (14), the truncated Euler path stays within distance 1 of Z H with probability tending to at least 1 − ζ. It then remains inside [−H, H], so the raw and truncated Euler recursions agree step by step. Thus, for every r > 0, (K) lim sup P |Y0 − ZT −δ | > r ≤ ζ. K→∞
Since ζ is arbitrary, this proves (10). Step 5: amplify a rare terminal shell. It remains to prove divergence of the explicit moments. Put ℓ = log(T /δ). For the grid (9), hj = tj+1 − tj = tj (eℓ/K − 1). For all sufficiently large K there are constants a, b > 0, depending only on (δ, T ), such that a b ≤ hj ≤ , K K
j = 0, . . . , K − 1.
(15)
Consider one reverse Euler step from Yj+1 = y with |y| ≥ R + 1. On the event |ξj | ≤ 1, (13) and (15) give p |Yj | ≥ λhj |y|1+α − |y| − hj |stj+1 (y)| − hj |ξj | r λa 1+α b bM b ≥ |y| − 1+ |y| − − . K δK δK K Choose A > 0, depending only on λ, α, a, b, δ, M , so large that for every sufficiently large K and every |y| ≥ AK 1/α , λa 1+α |Yj | ≥ |y| . 4K We also take K large enough that AK 1/α ≥ R + 1. Let c0 = λa/4 and zj = |Yj |/K 1/α . Then 1+α zj ≥ c0 zj+1 .
7
1/α
Taking A still larger if necessary so that c0 A > 1 and c0 A1+α ≥ A, induction gives, after n reverse steps, (1+α)n −1/α 1/α zK−n ≥ c0 c0 A . (16) In particular every iterate remains in the active tail on the event used below. Define K−1 \ 1/α 1/α EK = {YK ∈ [AK , AK + 1]} ∩ {|ξj | ≤ 1}. j=0
For y ≥ 0, (y − X0 )2 1 (y + M )2 1 exp − ≥√ exp − . pT (y) = E √ 2T 2T 2πT 2πT
Therefore there are C1 , C2 > 0, independent of K, such that P YK ∈ [AK 1/α , AK 1/α + 1] ≥ C1 exp(−C2 K 2/α ). Since pξ := P(|ξ0 | ≤ 1) > 0 and the Euler noises are independent of YK and of one another, 2/α P(EK ) ≥ C1 exp(−C2 K 2/α )pK +K (17) ξ ≥ exp −C3 K for a constant C3 < ∞. Combining (16) at n = K with (17), for every q > 0, q(1+α)K 1/α −q/α (K) c0 A −→ ∞. E|Y0 |q ≥ P(EK )K q/α c0 Indeed, the logarithm of the amplification term is of order (1 + α)K , whereas the logarithmic rare-event cost is only of order K 2/α + K. This proves (11). For each fixed K, finite-step induction (K) using the growth bound |b st (x)| ≤ C(1 + |x|1+α ) and Gaussian moments gives E|Y0 |q < ∞ for every q < ∞. Equation (10) implies weak convergence of the endpoint laws. (K) Finally fix p ≥ 1, ν ∈ Pp (R), and any coupling (V, V ′ ) with V ∼ L(Y0 ) and V ′ ∼ ν. Minkowski’s inequality gives 1/p 1/p 1/p E|V − V ′ |p ≥ E|V |p − E|V ′ |p . Taking the infimum over couplings and using the q = p case of (11) proves (12). By a standard time-dependent rescaling, the same counterexample can be expressed in the usual VP/DDPM coordinates. We use the present VE normalization throughout for simplicity. The amplification mechanism is related to the classical divergence of explicit Euler schemes for superlinear SDEs [8]. The difference is that here an arbitrarily mild superlinear perturbation can be placed in a remote region with arbitrarily small forward-marginal mass. It is dissipative in continuous time but can trigger a discrete cascade through explicit overshoot. Strong uniform global control also rules out the fixed-field construction. Examples include spatial Lipschitz and linear-growth bounds that hold uniformly in time and across the approximating family. Such assumptions give standard grid-uniform finite-horizon Euler moment bounds, but they are much stronger than on-path score accuracy and can be restrictive or hard to verify in practice [11]. 8
3.2
Further consequences of the counterexample
Higher dimensions. The same mechanism extends to any dimension through a radial perturbation. Corollary 3.2 (The obstruction persists in every dimension). Let d ≥ 1 and suppose µ0 is supported in B(0, M ) ⊂ Rd . For every ε > 0, η > 0, λ > 0, and α > 0, there is a radial cutoff χR such that sbt (x) = st (x) − λχR (∥x∥)∥x∥α x has the d-dimensional counterparts of all three conclusions of Theorem 3.1. Its on-path L2 (pt ) error is uniformly small. The learned reverse-time process is nonexplosive, has moments of every order, and is within η of the exact reverse-time process in path-space total variation. The Euler–Maruyama discretizations converge in probability, while every positive moment diverges. Their Wp distance from every fixed law in Pp (Rd ) also diverges. Proof. The proof of Theorem 3.1 carries over radially. Since ∥Dt (x)∥ ≤ M , x · sbt (x) ≤ C − c∥x∥2+α , so the Lyapunov, path-coupling, and localization arguments are unchanged with | · | replaced by ∥ · ∥. On the active tail, the radial norm satisfies the same superlinear one-step lower bound up to fixed-dimensional linear and noise terms. On the shell AK 1/α ≤ ∥YK ∥ ≤ AK 1/α + 1 and the bounded-noise event ∥ξj ∥ ≤ 1, choosing A large yields the same normalized (1 + α)-power recursion as in Theorem 3.1. The shell probability is bounded below by exp(−CK 2/α ) up to a polynomial factor, and the bounded-noise event contributes exp(−cd K). The iterated amplification therefore dominates the rare-event cost. Probability flow ODE. The same separation appears for deterministic probability flow. In our VE normalization, the exact reverse-time probability flow ODE is 1 U̇u = sT −u (Uu ), 2
U0 ∼ pT .
The rare-shell and repeated-amplification mechanism also has a deterministic version for this flow. Corollary 3.3 (Deterministic probability flow analogue). The same separation can occur for the probability flow ODE. Fix ε > 0, η > 0, λ > 0, and α > 0. The radius R in (3) can be chosen so that the conclusions of Theorem 3.1 hold and the learned probability flow ODE 1 Żu = sbT −u (Zu ), 2
Z0 ∼ pT ,
is nonexplosive and has moments of every order. The path-space total variation distance between the learned and exact probability flow path laws is at most η. Its descending-grid explicit Euler scheme hj (K) (K) (K) (K) Qj = Qj+1 + sbtj+1 (Qj+1 ), QK ∼ pT , 2 9
can be coupled so that (K )
Q0 m −→ ZT −δ
(K )
E∥Q0 m ∥q −→ ∞
in probability,
for every q > 0.
Hence the deterministic sampler laws converge weakly while their Wp distance from every fixed law in Pp (R) diverges for every p ≥ 1. Proof. For the VE forward diffusion, the probability flow ODE has reverse-time drift 12 sT −u [1]. The coercive estimate and the common-core coupling argument used for the SDE apply after multiplying the drift by 1/2. If necessary, enlarge R once more so that the exact SDE and exact ODE exit probabilities are both at most η; all earlier score-error and coercivity conclusions are preserved. Local truncation gives convergence in probability of the explicit Euler discretization. Choose a (K) sufficiently large shell constant A > 0. On the terminal shell AK 1/α ≤ |QK | ≤ AK 1/α + 1, the active-tail update obeys (K) (K) |Qj | ≥ chj λ|Qj+1 |1+α . No per-step Gaussian event is needed, so the rare-event cost is only the terminal-shell probability, bounded below by exp(−CK 2/α ). The same iterated (1+α)-power recursion yields super-exponential conditional growth, which proves all moment and Wasserstein conclusions exactly as in Step 5 of Theorem 3.1. Equal-budget non-ordering. The main theorem shows that a small on-path score-error budget alone cannot certify stability. The next result goes further: this scalar budget does not even rank score fields by numerical stability. Corollary 3.4 (Equal-budget non-ordering). For every ε > 0, there exist b ∈ (0, ε] and two smooth score fields sbsafe and sbbad such that, when both Euler–Maruyama chains are initialized from pT , Z T Z T 2 2 safe Ept sbt (Xt ) − st (Xt ) dt = Ept sbbad dt = b. (18) t (Xt ) − st (Xt ) δ
δ
On the nested geometric grids (9), the Euler–Maruyama discretization associated with sbsafe has grid-uniform moments of every order: (Km ),safe
q
< ∞,
q ≥ 1,
(19)
−→ ∞,
q > 0.
(20)
sup E Y0 m
whereas the scheme associated with sbbad satisfies (Km ),bad
E Y0
q
Proof. Apply Theorem 3.1 with tolerance ε/(T − δ) and any fixed positive choices of η, λ, and α. The resulting bad field has integrated budget at most ε; set Z T 2 b := Ept sbbad dt. t (Xt ) − st (Xt ) δ
10
The Gaussian-smoothed marginals have strictly positive density and the corruption is nonzero on a set of positive Lebesgue measure, so b > 0. Theorem 3.1 gives (20). For the safe comparator, choose a constant a ∈ R with a2 =
b , T −δ
sbsafe t (x) = st (x) + a.
Then (18) holds exactly. Its Euler update is Yk =
tk tk+1
Yk+1 +
p hk Dtk+1 (Yk+1 ) + hk a + hk ξk . tk+1
Since Dt (x) ∈ [−M, M ], iterating the affine recursion writes Yk as four terms: a contracted terminal state, a convex combination of bounded denoiser values, a deterministic shift atk
KX m −1
hj , t j j=k
and a centered Gaussian term. On the geometric grid, tj+1 /tj = exp(ℓ/Km ) with ℓ = log(T /δ), so sup Km eℓ/Km − 1 < ∞. m
Hence the deterministic shift is uniformly bounded, while the Gaussian term has uniformly bounded variance by a direct geometric-series estimate. Since YKm ∼ pT has moments of every order, (19) follows.
3.3
A fixed-architecture neural obstruction
The preceding counterexample is constructed at the level of a general smooth score field. We now show that the same obstruction persists within a fixed finite neural architecture. Globally bounded and globally Lipschitz denoisers represented by the same GELU architecture can have arbitrarily small on-path score error and path-space total variation distance, while their Euler–Maruyama discretizations remain unstable in every Wasserstein metric. Unlike the fixed-field construction above, this result considers a sequence of network parameters within one fixed architecture. For the construction, define ψ(u) = u+ − (u − 1)+ ,
ΓR (x) = ψ(x − R) + ψ(−x − R).
Thus ΓR = 0 on [−R, R], 0 ≤ ΓR ≤ 1, and ΓR = 1 outside [−R − 1, R + 1]. Appendix A constructs the fixed adaptive LayerNorm-Zero (adaLN-Zero) DiT and proves the approximation and polynomial output bounds used below. Theorem 3.5 (Fixed-DiT neural non-certification). Fix λ > 0. Take µ0 = δ0 , so that pt = N (0, t) and the exact denoiser is zero. There exist one fixed scalar one-token adaLN-Zero DiT architecture,
11
a sequence of parameter values θm in this same architecture, and geometric grids with Km → ∞ such that, with Dθm (x) − x (m) , t ∈ [δ, T ], sbt (x) = t and with both the learned reverse-time processes and Euler–Maruyama chains initialized from pT , the following hold: 1. every Dθm is globally bounded and globally Lipschitz; 2. the on-path score error vanishes, (m)
sup Ept |b st
− st |2 → 0;
t∈[δ,T ]
3. the learned reverse-time processes are nonexplosive, have moments of every order, and their path laws converge in total variation to the exact reverse-time path law; 4. the Euler–Maruyama chains can be coupled with the exact reverse-time process so that their endpoints converge in probability to the exact reverse-time endpoint, while for every q > 0 and every p ≥ 1, (m) (m) E|Y0 |q → ∞, Wp L(Y0 ), pδ → ∞. Proof. Let Rm ↑ ∞ and define the ideal remote-tail denoiser Fm (x) = −λΓRm (x)x3 . Choose a fixed A > 0 large enough for the cubic amplification estimate below. Then choose Km ↑ ∞ so that the terminal shell p p [A Km , A Km + 1] lies in the fully active region |x| ≥ Rm + 1. We first define the approximation tube without referring √ to the neural parameters. On the −1 geometric grid, ρj = hj /tj+1 ≍ Km . Starting from qm,0 = A Km + 1, define a deterministic upper recursion 3 qm,n+1 = C 1 + qm,n , n = 0, . . . , Km − 1, where C is chosen large enough to dominate one denoiser-form Euler step whenever |D(x)−Fm (x)| ≤ 1 and |ξj | ≤ 1. Let Qm = 1 + Rm + max qm,n , ζm = Q−1 m . 0≤n≤Km
Then Qm < ∞, Qm → ∞, and ζm ≤ 1. Lemma A.2 gives parameters θm in one fixed DiT architecture such that sup |Dθm (x) − Fm (x)| ≤ ζm . |x|≤Qm
12
By induction, on the terminal-shell and bounded-noise event, the neural trajectory stays inside [−Qm , Qm ]. On the same event, the one-step lower estimate is p |Yj | ≥ ρj λ|Yj+1 |3 − (1 − ρj )|Yj+1 | − ρj ζm − hj . For the fixed choice of A and all large m, this implies c |Yj+1 |3 |Yj | ≥ Km with c > 0 independent of m. Choosing A so that cA2 ≥ 1 also keeps every iterate in the fully active region. The rare-event argument of Step 5 in Theorem 3.1 therefore gives (m) q
E|Y0
| → ∞,
q > 0.
Because the exact denoiser is zero, Dθm (x) . t −1 On [−Qm , Qm ], use the approximation to Fm ; outside the tube, use (31). Since ζm = Qm , the global output bound grows at most polynomially in Qm , while the Gaussian tail beyond Qm decays exponentially in Q2m . Since Rm → ∞, the ideal cubic contribution on {|x| > Rm } also vanishes under every pt , uniformly for t ∈ [δ, T ]. Hence (m)
sbt (x) − st (x) =
(m)
sup Ept |b st
− st |2 → 0.
t∈[δ,T ]
Girsanov’s theorem, evaluated under the exact reverse-time path law, gives Z 1 T (m) (m) Ept |b st − st |2 dt, KL L(U· ) L(Z· ) ≤ 2 δ so Pinsker’s inequality yields path-space total variation convergence. Each fixed learned drift is globally Lipschitz with linear growth on [δ, T ], hence its SDE is nonexplosive and has moments of all orders. For weak convergence, couple the neural Euler–Maruyama chain with the exact-denoiser Euler– Maruyama chain using the same p terminal draw and Gaussian increments. Since the exact denoiser is zero, Ȳj /tj = Ȳj+1 /tj+1P+ hj ξj /tj . After reversing the index, this is a Gaussian martingale. On the geometric grids, j hj /t2j is uniformly bounded, so Doob’s inequality gives grid-uniform maximal moments for maxj |Ȳj |. The exact chain therefore stays in [−Rm + 1, Rm − 1] with probability tending to one. On this event, a backward induction gives the comparison with the neural chain. At the terminal level the two chains agree. If |∆j+1 | ≤ ζm ≤ 1, then |Ȳj+1 | ≤ Rm − 1 implies |Yj+1 | ≤ Rm , so Fm (Yj+1 ) = 0 and the tube approximation applies. Writing ρj = hj /tj+1 , the difference recursion then obeys |∆j | ≤ (1 − ρj )|∆j+1 | + ρj ζm . Thus maxj |∆j | ≤ ζm by induction. Since the exact Euler–Maruyama endpoint converges in probability to the exact reverse-time endpoint, the neural endpoint does as well. Weak convergence follows, and the Wasserstein divergence follows from moment divergence by the same Minkowski lower bound used in Theorem 3.1. 13
For each fixed m, boundedness of Dθm gives a moment bound that is uniform in the grid size K, but the bound may depend on m. The divergence therefore occurs only along the diagonal sequence (m, Km ). The one-token construction is not essential. The scalar circuit can be embedded tokenwise in a finite patchified adaLN-Zero DiT, and the resulting fixed circuit fits within the DiT-S/2 resource budget; see Appendices A and B. Corollary 3.6 (A fixed DiT-S/2 configuration is sufficient). Consider the DiT-S/2 configuration of Peebles and Xie [12]: 12 transformer blocks, hidden dimension 384, 6 attention heads, MLP ratio 4, and patch size 2. For the standard 32 × 32 latent input with 4 channels, there exists a sequence of parameter values in this single fixed configuration for which all conclusions of Theorem 3.5 hold. Proof. Appendix A lifts the scalar construction to the patchified finite-dimensional architecture, while Appendix B shows that the resulting circuit fits within the DiT-S/2 resource budget. The conclusion then follows from Theorem 3.5.
4
Stability from denoiser projection
The counterexample shows that a small forward-marginal score error need not control the states generated by an explicit sampler. We now consider a setting with additional geometric information: the data are supported in a known bounded closed convex set C. Since the exact denoiser Dt (x) = E[X0 | Xt = x] also takes values in C, this information can be imposed directly on the learned denoiser without increasing its pointwise approximation error. Let ΠC : Rd → C denote Euclidean projection and define DtP (x) − x . (21) t The projection preserves the available denoiser accuracy while forcing the deterministic part of each Euler–Maruyama update toward the bounded set C. This simple modification is enough to recover grid-uniform moment control. b t (x), DtP (x) = ΠC D
4.1
sPt (x) =
Projection removes the amplification mechanism
Substituting (21) into (2) gives Yk = (1 − ρk )Yk+1 + ρk DtPk+1 (Yk+1 ) +
p
hk ξk .
(22)
For every decreasing positive grid, 0 < ρk < 1. The deterministic part is therefore a convex combination of the current state and a point in C. If C ⊂ B(0, rC ), then for every y ∈ Rd , (1 − ρk )y + ρk DtPk+1 (y) ≤ (1 − ρk )∥y∥ + ρk rC .
(23)
Under (22), the deterministic update has at most linear growth toward a bounded set. A large state can therefore no longer trigger the superlinear amplification cascade from Section 3. 14
4.2
Explicit representation and grid-uniform moment control
The affine update has an explicit expansion. Since 1 − ρk = tk /tk+1 , products of successive coefficients telescope into ratios of time levels. This gives the following representation. Proposition 4.1 (Explicit representation and grid-uniform moment bounds). Assume C ⊂ B(0, rC ) and consider a family of grids satisfying sup max ρk ≤ ρmax < 1. K 0≤k<K
Then each projected chain in the family satisfies the representation K−1 X tk YK + wk,j DtPj+1 (Yj+1 ) + Gk , T j=k K−1 X 1 tk 1 wk,j = 1 − , − , wk,j = tk tj tj+1 T j=k
Yk =
K−1 X
tk p hj ξj ∼ N (0, σk2 Id ), Gk = tj j=k
σk2 = t2k
(24)
(25) K−1 X
hj tk (1 − tk /T ) ≤ . 2 t 1 − ρmax j=k j
(26)
Consequently, for every q ≥ 1,
∥Yk ∥Lq ≤
tk tk ∥YK ∥Lq + 1 − T T
s
rC +
tk (1 − tk /T ) ∥gd ∥Lq , 1 − ρmax
(27)
where gd ∼ N (0, Id ). The bound is independent of the off-path growth and regularity of the raw learned score sb. Proof. From (22), p hk P Dtk+1 (Yk+1 ) + hk ξk . tk+1 tk+1 Iterating this identity and using telescoping products gives (24). The denoiser coefficient at level j is tk hj 1 1 = tk − , tj tj+1 tj tj+1 Yk =
tk
Yk+1 +
which is nonnegative and telescopes to (25). The Gaussian term has the variance in (26). Moreover, K−1 X hj K−1 X tj+1 1 1 = − t2 tj tj tj+1 j=k j j=k 1 1 1 ≤ − , 1 − ρmax tk T which proves the stated variance bound. Finally, DtP (·) ∈ C, and the weights in (25) are nonnegative. Hence the middle term in (24) belongs to (1 − tk /T )C and has norm at most (1 − tk /T )rC . Taking Lq norms and applying Minkowski’s inequality proves (27). 15
For fixed p ≥ 1, take q > p in Proposition 4.1. Then ∥Yk ∥p is uniformly integrable over the grid family. This is the property that fails in the counterexample of Section 3.
4.3
Stability without loss of score accuracy
Projection also preserves approximation accuracy. The exact denoiser already takes values in C, so Euclidean projection cannot move the learned denoiser farther from the target. The stability improvement established below therefore comes without an increase in approximation error. Proposition 4.2 (Projection contracts denoiser and score error). Let C ⊂ Rd be closed and convex, with supp(µ0 ) ⊂ C. Then Dt (x) ∈ C for every t > 0 and x ∈ Rd . Consequently, b t (x) − Dt (x)∥, ∥DtP (x) − Dt (x)∥ ≤ ∥D and equivalently, ∥sPt (x) − st (x)∥ ≤ ∥b st (x) − st (x)∥. Hence s t − s t ∥2 . Ept ∥sPt − st ∥2 ≤ Ept ∥b Proof. The posterior law of X0 given Xt = x is supported in C, so its barycenter Dt (x) also lies in C. Euclidean projection onto a closed convex set decreases distance to every point in that set, so b t (x) − Dt (x)∥ ≤ ∥D b t (x) − Dt (x)∥. ∥ΠC D The score inequality follows by dividing by t and using Tweedie’s identity. Theorem 4.3 (Projection restores Wasserstein convergence). Let C ⊂ B(0, rC ) be bounded, closed, and convex, with supp(µ0 ) ⊂ C. Consider a sequence of grids with mesh size tending to zero and (K) ρk ≤ ρmax < 1, and initialize each projected Euler–Maruyama chain with YK ∼ pT . Assume that DtP is jointly continuous and locally Lipschitz in x, locally uniformly over t ∈ [δ, T ], and let Z P solve dZuP = sPT −u (ZuP ) du + dBu , Z0P ∼ pT . Then, for every p ≥ 1, (K) Wp L(Y0 ), L(ZTP−δ ) −→ 0. Moreover, define the projected on-path error Z T EP = Ept ∥sPt − st ∥2 dt. δ
For every q > p, there is a finite constant Cp,q , depending also on the fixed endpoint qth moments, such that 1 1 ( −1) (K) Wp L(Y0 ), pδ ≤ oK (1) + Cp,q EP2 p q . (28) Here oK (1) → 0 as the mesh size tends to zero. By Proposition 4.2, EP is no larger than the raw on-path score error. 16
Proof. Since DtP is bounded and t ≥ δ, the projected score has linear growth. The stated local regularity gives localized Euler convergence under the usual Brownian-increment coupling, (K)
Y0
−→ ZTP−δ
in probability.
Apply Proposition 4.1 with any q > p. The resulting grid-uniform qth-moment bound makes (K) ∥Y0 ∥p uniformly integrable. The limit ZTP−δ also has a finite qth moment. Hence the coupled endpoint differences converge to zero in Lp , which is stronger than convergence in Wp . For the comparison with the exact endpoint, both DtP and Dt take values in C, so sPt − st = P (Dt −Dt )/t is bounded on [δ, T ]×Rd . Thus the Girsanov change of measure is valid, and evaluation under the exact reverse-time law gives 1 KL L(U· ) L(Z·P ) ≤ EP . 2 Pinsker’s inequality yields 1p TV L(UT −δ ), L(ZTP−δ ) ≤ EP . 2 Both endpoint laws have finite moments of every fixed order. The standard TV–moment interpolation inequality then gives, for q > p, 1 1 ( −1) Wp L(ZTP−δ ), pδ ≤ Cp,q EP2 p q .
The triangle inequality and the first part of the corollary prove (28).
5
Experiments
The experiments provide a constructive realization test. We ask whether one fixed finite neural architecture can remain accurate under ordinary sampling, reproduce the amplification mechanism of the counterexample, and suppress this amplification after denoiser projection. We use an 8 × 8 one-channel point-mass variance-exploding model on t ∈ [0.1, 1], so the exact denoiser is zero. Four family members share the same small DiT-style architecture and differ only in their parameters and remote target profiles. Training and evaluation details are given in Appendix C.
5.1
Unconditioned accuracy and post-entry amplification
We first evaluate the selected checkpoints on unconditioned trajectories. Table 1 reports forwardmarginal score MSE, paired trajectory RMSE, and the stress-test RMS amplification defined below. The score MSE is at most 7.10 × 10−3 , while the trajectory RMSE ranges from 0.012 to 0.024. We use a stress-initialized protocol to isolate the dynamics after the numerical chain reaches the b k |2 )1/2 be stress region. This protocol does not estimate the natural entry probability. Let rk = (E|Y the empirical coordinatewise root-mean-square (RMS) magnitude after step k. Across the family, r8 /r0 rises from 7.9 to 14.7, while the neural endpoint retains about 90%–95% of the target RMS magnitude. Figure 1 shows the stepwise growth and close tracking of the target recursion. 17
Family
score MSE
path RMSE
RMS amplification r8 /r0
m0 m1 m2 m3
1.94×10−3 3.73×10−3 3.89×10−3 7.10×10−3
0.0119 0.0167 0.0171 0.0237
7.90 9.78 13.02 14.69
Table 1: Unconditioned-trajectory accuracy and neural amplification for the selected checkpoints.
b k |2 )1/2 for the neural and paired Figure 1: Stepwise coordinatewise RMS magnitude rk = (E|Y target-profile chains. The trajectories also show a simple feedback mechanism: once the chain reaches a region where forward-marginal error gives little control, one inaccurate update can move later score queries farther from the reference path and amplify the deviation. Here this produces repeated the amplification.
5.2
Projection suppresses post-entry amplification
We next apply denoiser projection with C = [−1, 1]. Raw and projected chains use identical stress initializations and bounded perturbations. Projection reduces the final empirical second moment 18
by factors between 1.13 × 104 and 3.33 × 104 . Figure 2 shows the stepwise separation: raw RMS magnitude grows, whereas projected RMS magnitude falls from 35–46 initially to 2.6–3.7. Thus projection changes the post-entry dynamics and removes the amplification mechanism in this stress test.
Figure 2: Stepwise coordinatewise RMS magnitude for the paired raw and projected chains under the stress-initialized protocol.
6
Discussion
The results point to a gap between forward-marginal score accuracy and stability along numerical trajectories. The problem is not only that the learned score may be inaccurate at states rarely visited under the forward marginals. A numerical error also changes where the score is evaluated at later steps. If the new state lies in a poorly controlled region, the next update can increase the deviation further. Our counterexample turns this simple feedback mechanism into a superlinear cascade, but the same concern can arise more generally in learned reverse-time dynamics. A useful certificate should therefore provide control that is uniform across the relevant learned family, control the score at states visited by the numerical sampler, or constrain the numerical transition directly. Projection gives one such structural safeguard when a bounded convex set containing the data support is known. 19
A natural next step is to extend this framework to Schrödinger bridges and related learned transport dynamics. In Schrödinger bridge problems, time-dependent drifts or controls are learned along bridge marginals and then simulated numerically, so a similar gap between bridge-marginal accuracy and solver-generated trajectories may arise. Two questions are direct: whether small bridge-marginal error can coexist with stable continuous dynamics but unstable discretization, and whether bridge geometry or feasible-state constraints can yield projection-like stabilization.
A
Fixed-size neural realization and patchified embedding
This appendix gives the network details used in Section 3. The first two lemmas build the fixed-size remote-tail circuit and embed it into DiT; the final part handles positional embeddings for the patchified finite-dimensional lifting used in Corollary 3.6. Lemma A.1 (Fixed-size realization of the remote cubic profile). Fix λ > 0. There exist integers L, W < ∞, independent of R, Q and the approximation tolerance, and a scalar network architecture AL,W built from affine maps, GELU activations, residual connections, and a final fixed-dimensional LayerNorm–linear head, with the following property. For every 1 < R < Q and every ζ > 0, one can choose parameters θ in this same architecture so that the resulting denoiser Dθ is globally bounded and globally Lipschitz and sup Dθ (x) + λΓR (x)x3 ≤ ζ.
(29)
|x|≤Q
The depth L and width W do not depend on R, Q, or ζ; only the parameter values do. Proof. We give an explicit construction. Let σ be the tanh-approximated GELU activation used in the DiT MLP. We need only two facts: σ(z) − z+ is bounded on R, and σ ′′ is nonzero somewhere. The first follows from the tanh-GELU formula and its limits at ±∞. The second follows because the activation is smooth and non-affine. First, Rk (u) = k −1 σ(ku) approximates u+ uniformly on R as k → ∞. Indeed, with Cσ = sup |σ(z) − z+ | < ∞, z∈R
one has sup |Rk (u) − u+ | ≤ u∈R
Cσ . k
Hence a fixed four-unit GELU layer approximates the two-sided gate ΓR uniformly, with the location R encoded only through biases. Here ΓR is the two-sided ramp gate defined in Section 3. Second, a fixed-size GELU module can approximate multiplication on any compact box. Its weights may depend on the box and the tolerance. Choose aσ with σ ′′ (aσ ) ̸= 0 and set Sh (u) =
σ(aσ + hu) + σ(aσ − hu) − 2σ(aσ ) . h2 σ ′′ (aσ ) 20
Taylor’s theorem gives, for every H < ∞, sup |Sh (u) − u2 | ≤ CH h2 . |u|≤H
Therefore
1 Sh (u + v) − Sh (u − v) 4 2 approximates uv uniformly on [−H, H] . This module uses a fixed number of GELU units; only h and the affine weights vary with H and the target accuracy. A fixed finite composition of these modules computes, to arbitrary accuracy on [−Q, Q], x2 , x3 , the gate ΓR (x), and finally ΓR (x)x3 . The number of multiplication modules is fixed. Hence the depth and width do not depend on R, Q, or ζ. It remains to make the output globally bounded without changing the architecture size. Let εLN > 0 be the fixed LayerNorm stabilizer and set q(z) = (z, 1, −1), HLN (z) = LNεLN (q(z)) 1 . Multh (u, v) =
A direct calculation shows that HLN is smooth, bounded, and strictly increasing. Near zero, HLN (z) = κz + O(z 3 ) for some κ > 0. The target range is bounded by |λΓR (x)x3 | ≤ λQ3 . Choose a large scale c and feed the arithmetic circuit output, divided by cκ, into the fixed LayerNorm head cHLN (·). Then 3 y |y| =y+O c HLN cκ c2 uniformly for |y| ≤ λQ3 . Taking c large and the internal arithmetic tolerances small yields (29). The final LayerNorm head is bounded, and every component of the finite network is globally Lipschitz for fixed parameters. Hence Dθ is globally bounded and globally Lipschitz. Lemma A.2 (Embedding the fixed circuit into a fixed adaptive LayerNorm-Zero (adaLN-Zero) DiT). Fix λ > 0. There exist integers N, d, r < ∞, independent of R, Q, and the approximation tolerance, and a scalar one-token adaLN-Zero DiT architecture with N blocks, token dimension d, and MLP expansion width r, such that the following holds. For every 1 < R < Q and every ζ > 0, one can choose parameters in this same architecture so that its scalar denoiser output Dθ is globally bounded and globally Lipschitz and sup Dθ (x) + λΓR (x)x3 ≤ ζ.
(30)
|x|≤Q
Only the parameter values vary with R, Q, and ζ. There are also constants C, a, b < ∞, depending only on the fixed architecture and λ, such that the parameters can be chosen with ∥Dθ ∥∞ ≤ C(1 + Q)a (1 + ζ −1 )b .
(31)
Proof. We now show how a fixed stack of adaLN-Zero MLP blocks simulates the GELU arithmetic circuit from Lemma A.1. In a standard adaLN-Zero block, the conditioning network produces shift, scale, and gate vectors for the attention and MLP branches. Set the attention gates to zero. 21
Choose the affine modulation layers so that the MLP shift and scale are zero and the MLP gate is a fixed nonzero constant. This is done by zeroing the relevant conditioning weights and using only their biases. The timestep and class embeddings are then ignored. After absorbing the constant gate into the output weights, each active block has the form h 7−→ h + Mℓ (LNεLN (h)), where Mℓ is a two-layer affine–GELU–affine map and εLN > 0 is the fixed LayerNorm stabilizer. This is exactly a parameter subfamily of the adaLN-Zero block used in DiT [12]. The remaining issue is pre-normalization. Let dws be the workspace dimension of the arithmetic circuit and choose d ≥ dws + 2. Fix an anchor a ∈ Rd with nonzero centered part. Choose an isometric embedding U : Rdws → Rd whose range is orthogonal to both the constant vector 1 and the centered anchor a − ā1. For a bounded workspace state z and a small signal scale ϵ > 0, encode h = a + ϵU z. A Taylor expansion of LayerNorm at a gives, uniformly for z in a fixed bounded box, LNεLN (a + ϵU z) = ℓ0 + ca ϵU z + O(ϵ2 ),
ca > 0,
(32)
where the orthogonality choice makes the derivative a scalar multiple of the identity on the signal subspace. Fix R, Q, and a target accuracy. Rescale the finitely many workspace coordinates so that every intermediate state on [−Q, Q] lies in one bounded box. These diagonal rescalings are absorbed into neighboring affine weights and do not change the circuit size. Now consider one arithmetic stage z 7−→ z + ∆ℓ (z), where ∆ℓ is produced by one affine–GELU–affine module of the fixed circuit. By (32), the first affine map of a DiT MLP can subtract ℓ0 and multiply by (ca ϵ)−1 U ⊤ . This recovers z up to O(ϵ) error on the relevant compact set. Use the arithmetic-stage weights in the middle of the block, then multiply the final affine output by ϵU . The result is a + ϵU z 7−→ a + ϵU z + ∆ℓ (z) + o(ϵ) uniformly on that set. Since the circuit has finitely many stages, choosing ϵ small enough makes the full fixed stack approximate its encoded workspace trajectory to any target accuracy. The affine patch embedding initializes the anchor and the rescaled input coordinate. Store the final circuit output in one bounded workspace coordinate after dividing by a parameter-dependent scale Sout ≥ 1 + λQ3 . Apply (32) once more. The final LayerNorm and a linear readout with weight of order Sout /ϵ recover the unscaled scalar output with arbitrarily small uniform error on [−Q, Q]. The number of blocks, token dimension, and MLP width are fixed by the arithmetic circuit. They do not depend on R, Q, or ζ. For every fixed parameter choice, affine maps and GELU are globally Lipschitz, LayerNorm with positive stabilizing constant is globally Lipschitz, and the final LayerNorm has bounded coordinates. Hence the scalar DiT output is globally bounded and globally Lipschitz. 22
Each arithmetic module can be chosen with parameters polynomial in 1 + Q and ζ −1 . Since the circuit has fixed depth, composing the modules preserves polynomial dependence for the intermediate weights, workspace rescalings, and Lipschitz constants. The small-signal LayerNorm simulation and final decoding require only additional polynomial rescaling. Hence there are constants C, a, b < ∞ for which (31) holds.
A.1
Patchified embedding and positional control
We now give the patchified finite-dimensional lifting used in Corollary 3.6. We embed the scalar construction tokenwise. Write the latent input as a fixed collection of Ntok < ∞ patches and choose one scalar coordinate ui in each patch. Set the attention gates to zero. The remaining MLP branch uses shared weights across tokens. The final linear head is also tokenwise, and unpatchification only rearranges patch outputs. The only new issue is the fixed positional embedding. Let pi ∈ Rd be the positional vector of token i. Let ex ∈ Rdws be the workspace basis vector for the input coordinate and set v := U ex ∈ Rd . Choose positive scales Am and ϵm . The shared affine patch embedding is chosen so that, on the selected scalar coordinate, Em (ui ) = Am a + ϵm vui , with the same anchor a and signal direction U used in the small-signal construction. The input to the first block is then Am a + ϵm vui + pi . Factoring out the anchor scale gives the exact identity LNεLN (Am (a + ϵm vui ) + pi ) = LNεLN /A2m a + ϵm vui + A−1 m pi . The centered anchor has nonzero variance, so zero-stabilizer normalization is smooth near the compact encoded tube. Since there are finitely many tokens, −1 LNεLN /A2m a + ϵm vui + A−1 m pi = LN0 (a + ϵm vui ) + O(Am ) uniformly for |ui | ≤ Q and over all tokens. The same orthogonality choice as in Lemma A.2 makes the derivative of LN0 a scalar multiple of the identity on the signal subspace. The first affine map in an active MLP block decodes the workspace with gain of order ϵ−1 m , so −1 the positional perturbation creates decoded error of order (Am ϵm ) . Choose compatible scales with ϵm → 0 and Am ϵm −→ ∞. Scaling each active residual output by Am ϵm then gives Am (a + ϵm U z) 7−→ Am a + ϵm U (z + ∆ℓ (z)) + o(Am ϵm ), uniformly on the compact tube. Thus the fixed positional vectors remain a controlled perturbation of the tokenwise circuit.
23
Choose the final tokenwise linear head so that only the selected scalar slot of each reconstructed patch is nonzero. The resulting ideal denoiser is the product map Fm⊗ (x) = Fm (u1 ), . . . , Fm (uNtok ) on the selected patch coordinates, with all remaining output coordinates zero, up to an arbitrarily small compact-tube error. The number of tokens and the patch dimension are fixed. Thus the on-path L2 budget and the integrated drift-error term in Girsanov’s bound change only by a fixed dimensional factor. Taking Rm → ∞ still gives vanishing score error and vanishing path-space total variation distance. For the numerical lower bound, place one selected patch coordinate in the moving terminal shell and impose the bounded-noise event only on that coordinate. Attention is disabled and the tokenwise map is coordinate-separable. The selected coordinate therefore follows the same perturbed scalar recursion as in Theorem 3.5, while the other coordinates do not affect its update. Scalar rare-event amplification then forces every positive moment of the full latent norm to diverge. The weak-convergence argument is unchanged after replacing scalar maximal moments by finite-dimensional ones.
B
Embedding into DiT-S/2
This appendix checks that the fixed circuit of Appendix A fits inside the concrete DiT-S/2 configuration of Peebles and Xie [12]. Proposition B.1 (The fixed circuit fits inside DiT-S/2). Fix λ > 0. Consider the DiT-S/2 backbone with 12 adaLN-Zero transformer blocks, token dimension 384, MLP expansion width 1536, six attention heads, and patch size 2. For every 1 < R < Q and every ζ ∈ (0, 1), one can choose parameters in this single fixed architecture so that, on a selected scalar coordinate ui of every patch token i, sup Dθ,i (ui ) + λΓR (ui )u3i ≤ ζ, (33) |ui |≤Q
while all remaining output coordinates are zero. The attention branches are inactive, exactly three MLP residual blocks are active, and the remaining nine transformer blocks are identities. The architecture size is independent of R, Q, and ζ. Proof. Appendix A realizes the target profile with a fixed tokenwise circuit consisting of one gate module, one square module, and two multiplication modules, all simulated through a fixed small-signal LayerNorm workspace. Apply that construction tokenwise with shared weights. The first active MLP block computes the gate and square terms in parallel, the second forms the cubic term, and the third multiplies by the gate and applies the factor −λ. Closing all attention gates makes the computation coordinate-separable, and setting both residual gates to zero makes the remaining nine blocks exact identities. The resource check is fixed. The first active stage needs at most six GELU hidden units and the next two need four each, well below the available MLP width 1536. The workspace and anchor construction of Lemma A.2 has fixed dimension strictly below the token width 384, and 24
the patchified embedding argument of Appendix A controls the positional perturbation. Its scale choices make the compact-tube error at most ζ without changing any architecture dimension, which proves (33).
C
Experimental protocol
The neural runs use one fixed training seed (1729). Unconditioned evaluations use common evaluation seeds across the family. In the stress experiments, terminal states and perturbations are paired within each neural–target or raw–projected comparison. Summaries on unconditioned trajectories use 1024 trajectories per grid and 256 paired paths for the fine-grid coupling diagnostic. Stress and projection experiments use 512 paired trajectories. The reported values are Monte Carlo point estimates conditional on this training run and the selected checkpoints. Path RMSE is evaluated on a 512-step grid. The stress-initialized protocol uses eight Euler-form updates with standardized perturbations sampled uniformly from [−0.35, 0.35]. It is designed to isolate post-entry dynamics and does not estimate the natural probability of reaching the stress region. The base model is an 8 × 8 one-channel point-mass diffusion model, √ X0 = 0, Xt = t g, Dt ≡ 0, t ∈ [0.1, 1]. All learned denoisers use the same small DiT-style architecture: patch size 2, hidden dimension 64, and three adaptive-LayerNorm/GELU residual MLP blocks, for 104,388 trainable parameters. Family members differ only in their parameter values and the remote target profile used for training. Coordinatewise, the target is a smoothly gated cubic with tanh saturation, λ γRm (x)x|x|2 target Dm (x) = Lm tanh − , λ = 0.02, Lm where γR (x) = q((|x| − R)/0.5), q(u) = ū2 (3 − 2ū), and ū = min{1, max{0, u}}. Thus γR rises from 0 to 1 over a transition width of 0.5. The four configurations are stress (Rm , Lm , ym ) = (2.5, 700, 35), (3, 1000, 40), (3.5, 1450, 43), (4, 1800, 46), stress where ym is the center of the terminal stress initialization in the post-entry stress experiment. Training combines four terms: on-path score loss, safe-region denoiser loss, cascade-tube regression, and transition consistency for the deterministic Euler relation. After joint training, a common calibration rule filters checkpoints by fixed upper bounds on cascade-tube and transitionconsistency errors. Among the surviving checkpoints, it selects the checkpoint with the smallest on-path score error. All reported main-text comparisons use these selected checkpoints and paired random draws across the family. For Table 1, score MSE is evaluated after checkpoint selection with fresh randomness (evaluation seed 10729) and is the Monte Carlo mean of |b st (Xt )−st (Xt )|2 for t ∼ Unif[0.1, 1] and Xt ∼ N (0, tI). Path RMSE uses paired fine-grid trajectories: for each pair, we compute the coordinate-averaged squared learned–exact error at every grid point, take the maximum over the path, average over b k |2 )1/2 is the trajectories, and then take the square root. In the stress experiments, rk = (E|Y
25
empirical coordinatewise RMS magnitude at Euler step k. The amplification factor is r8 /r0 , the neural–target fidelity statistic is r8neural /r8target , and the projection reduction factor is the ratio of the final raw and projected empirical second moments.
References [1] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-Based Generative Modeling through Stochastic Differential Equations. International Conference on Learning Representations, 2021. [2] S. Chen, S. Chewi, J. Li, Y. Li, A. Salim, and A. Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. International Conference on Learning Representations, 2023. [3] H. Lee, J. Lu, and Y. Tan. Convergence of Score-Based Generative Modeling for General Data Distributions. Proceedings of the 34th International Conference on Algorithmic Learning Theory, PMLR 201:946–985, 2023. [4] J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization. International Conference on Learning Representations, 2024. [5] X. Gao, H. M. Nguyen, and L. Zhu. Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models. Journal of Machine Learning Research, 26(43):1–54, 2025. [6] S. Bruno and S. Sabanis. Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients. Transactions on Machine Learning Research, 2025. [7] D. Y. Cao, A. Y. Chen, K. Sridharan, and Y. Wu. On the Robustness of Langevin Dynamics to Score Function Error. arXiv:2603.11319, 2026. [8] M. Hutzenthaler, A. Jentzen, and P. E. Kloeden. Strong and weak divergence in finite time of Euler’s method for stochastic differential equations with non-globally Lipschitz continuous coefficients. Proceedings of the Royal Society A, 467:1563–1576, 2011. [9] N. B. Khelifa, R. E. Turner, and R. Venkataramanan. Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors. arXiv:2606.06179, 2026. [10] W. Tang and H. Zhao. Contractive Diffusion Probabilistic Models. arXiv:2401.13115, 2024. [11] G. Conforti, A. Durmus, and M. Gentiloni Silveri. KL Convergence Guarantees for Score Diffusion Models under Minimal Data Assumptions. SIAM Journal on Mathematics of Data Science, 7(1):86–109, 2025. [12] W. Peebles and S. Xie. Scalable Diffusion Models with Transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 26
[13] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. J. Fleet, and M. Norouzi. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. arXiv:2205.11487, 2022. [14] A. Lou and S. Ermon. Reflected Diffusion Models. Proceedings of the 40th International Conference on Machine Learning, PMLR 202:22675–22701, 2023. [15] D. J. Higham, X. Mao, and A. M. Stuart. Strong convergence of Euler-type methods for nonlinear stochastic differential equations. SIAM Journal on Numerical Analysis, 40(3):1041– 1063, 2002.
27