Privacy Accounting for Empirical Denoisers

Preprint

P RIVACY ACCOUNTING FOR E MPIRICAL D ENOISERS Dongjun Kim ∗ Stanford University dongjun@stanford.edu

William Cheng University of British Columbia willc135@cs.ubc.ca

arXiv:2610.27710v1 [cs.LG] 20 Oct 2026

Mijung Park University of British Columbia mijungp@cs.ubc.ca

A BSTRACT Diffusion models have emerged as the foundation of modern generative systems, yet their high memorization capacity raises privacy concerns. While differentially private (DP) training provides formal guarantees, it comes at a cost in sample quality. In this work, we study a complementary question: how much privacy loss is incurred during the sampling process itself? Directly analyzing modern neural denoisers is challenging because their sensitivity to a single training example is generally intractable. To overcome this challenge, we construct and analyze a clipped privacy-accountable empirical denoiser, a training-free estimator of the Bayes-optimal denoiser, whose per-step sensitivity can be computed explicitly, allowing each denoising step to be a Gaussian mechanism. This perspective enables privacy accounting under Gaussian DP when each training example contributes to each denoising step, and under f -DP trade-off functions when each generated sample uses a single randomly subsampled block of the training data. We report every (ε, δ) guarantee as a certified upper bound from a privacy loss distribution accountant, and use a cumulant-based Edgeworth approximation as a fast diagnostic, characterizing where it is reliable and where it becomes anti-conservative. We also analyze the privacy loss along the denoising trajectory and show that it grows steeply near the critical window in which semantic content is formed. This observation motivates hybrid samplers that use the private empirical denoiser through the formative steps where it remains a faithful proxy for the neural denoiser, then switch to a public neural denoiser for the remaining refinement. Our results on CIFAR-10 and CelebA-64 show that the privacy of diffusion models depends not only on the training procedure, but also on sampling-time design choices such as subsampling, stopping time, and hybrid denoising.

1

I NTRODUCTION

Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020) are now central to modern generative systems, but their ability to reproduce training examples raises serious privacy concerns (Wu et al., 2023; Carlini et al., 2023; Tang et al., 2024; Hu & Pang, 2023; Duan et al., 2023; Matsumoto et al., 2023). Post-hoc filters and memorization-aware model changes can reduce visible leakage, but they do not give attack-agnostic privacy guarantees. Differentially private (DP) training gives such guarantees, but it answers a different question: it bounds what the weights reveal, while users and regulators see only generated samples. Sampling-time leakage is a separate surface that persists even when the denoiser is trained non-privately, whether or not DP training is practical at scale. This raises a complementary question: how much privacy loss is incurred by the sampling process itself? Answering this requires treating the sampler, rather than the training algorithm, as the mechanism to be analyzed. The obstacle is that a neural denoiser has intractable data sensitivity, so its denoising steps admit no explicit privacy accounting. To make the problem analyzable, we privatize an empirical denoiser (Niedoba et al., 2025) whose sensitivity can be computed directly. Since each reverse ∗

Now at Adobe.

1

Preprint

diffusion step already injects Gaussian noise, the empirical denoising update can be interpreted as a Gaussian mechanism, allowing us to track the sampling trajectory with Gaussian Differential Privacy (GDP) (Dong et al., 2022). However, the GDP composition using the full batch of training data yields a poor privacy level. Denoising with subsampled data at each denoising step can improve the privacy level. However, this hurts the diffusion sampling, as every newly drawn subset of data injects extra stochastic drift into the sampling trajectory, which prevents coherent semantics from forming across the trajectory. To this end, we propose output-wise subsampling, subsample a block from the dataset once and reuse it for the entire denoising trajectory to produce one sample. This stabilizes the denoising direction and achieves a decent privacy level. When converting f -DP to (ϵ, δ)-DP, we encounter unreliable conversion, especially at small δ, when using the CLT-type f -DP accounting that assumes the CDF of the standardized test statistic under the composition is simply Gaussian. We report all guarantees as certified upper bounds from a PLD accountant (Doroshenko et al., 2022), and study a cumulant-based Edgeworth approximation (Zheng et al., 2020) as a fast estimate and measure where it is accurate and where it understates the certified value. Our contributions are summarized below: 1. We privatize a clipped empirical denoiser with explicit per-step sensitivity of 2C/|D|. Diffusion sampling then becomes a composition of Gaussian mechanisms, with no model trained on the private data. 2. We derive privacy guarantees for full-batch and output-wise subsampled diffusion sampling using GDP and f -DP trade-off functions, and report certified (ϵ, δ) upper bounds from a PLD accountant. We use the Edgeworth approximation as a diagnostic where cumulant checks pass, and characterize where it fails. Subsampling amplification applies once per reused block, rather than once per step. 3. We show that the per-step privacy cost grows along the trajectory relative to the critical window, measure where the empirical denoiser is a window-local semantic proxy for a neural denoiser, and use both to place the private-to-public handoff, evaluated on CIFAR-10 and CelebA-64.

2

BACKGROUND

2.1

D ENOISING D IFFUSION M ODELS

Diffusion models are built upon the forward and reverse processes (Ho et al., 2020; Song et al., 2020). The forward perturbation describes how the noise is gradually corrupting data sample x0 , defined as xt = αt x0 + σt ϵ, where ϵ is a standard Gaussian variable, αt and σt are prescribed time-dependent mean and variance coefficients, respectively. For simplicity, we describe the diffusion process from EDM (Karras et al., 2022), but our framework is adaptable to any diffusion processes such as DDPM (Ho et al., 2020) or Flow Matching (FM) (Lipman et al., 2022). In EDM, the forward diffusion process is simply defined by xt = x0 + tϵ, i.e., αt ≡ 1 and σt = t, and the reverse denoising process is computed by   √ ∆t ∆t xt−∆t = 1 − 2 xt + 2 E[x|xt ] + 2t∆tϵ. (1) t t The term E[x|xt ], called a Bayes optimal denoiser, is the conditional expectation of the clean data given the noisy observation xt . In practice, a neural denoiser D(xt , t; θ(D)) approximates this quantity, where θ(D) are parameters learned from dataset D. By rolling out eq. 1 up to time zero, diffusion model gradually synthesizes a sample. 2.2

D IFFERENTIAL P RIVACY AND G AUSSIAN M ECHANISM

Differential Privacy (DP) provides a formal framework to limit the influence of any single record on the output of a randomized algorithm. Intuitively, whether or not an individual’s data is present, the 2

Preprint

distribution of outputs should remain (almost) unchanged. This view naturally extends to diffusion sampling, which is itself a randomized mechanism. A fundamental building block is the Gaussian mechanism: given a function h : D 7→ Rp , the mechanism releases MGauss (D; h) = h(D) + n,

n ∼ N (0, σ 2 ∆2h I),

(2)

where ∆h is the global sensitivity—the maximum change in h between neighboring datasets. By calibrating the noise to ∆h , the Gaussian mechanism guarantees DP (see Appendix H.1 for formal definitions). For analyzing multi-step procedures such as diffusion sampling, we adopt Gaussian Differential Privacy (GDP) (Dong et al., 2022). GDP tightly characterizes the privacy loss of Gaussian mechanisms and, crucially, admits an exact composition rule: sX (3) µ= µ2i , i

where µi is the GDP parameter of the i-th step. For example, in the case of eq. 2 with n ∼ N (0, σ 2 ∆2h I), the Gaussian mechanism corresponds to µi -GDP with µi = σ1 . This makes GDP especially well-suited for diffusion processes, which inject Gaussian noise at every denoising step. Two further facts are important for our analysis. First, privacy amplification by subsampling can further reduce the effective µ. Second, GDP parameters can be translated into (ϵ, δ) guarantees when needed for regulatory reporting; we provide details in Appendix H.2. Together, these tools allow us to connect the denoising process directly to established DP accounting methods. 2.3

P RIVACY L OSS D ISTRIBUTIONS

(ϵ, δ)-DP is equivalently characterized by the hockey-stick divergence: a mechanism is (ϵ, δ)-DP iff R Heϵ (M (D) ∥ M (D′ )) ≤ δ for all neighboring D, D′ , where Hγ (P ∥Q) = [P (x) − γQ(x)]+ dx. P (x) The privacy loss distribution (PLD) is the law of the log-likelihood ratio log Q(x) under P ; the hockey-stick divergence, and hence δ(ϵ), is a functional of it, and the PLD of a composition is the convolution of the individual PLDs. Numerical PLD accountants evaluate this convolution on a discretized support (Gopi et al., 2021; Doroshenko et al., 2022). Discretization can be performed pessimistically, overestimating Heϵ for every ϵ and hence yielding a valid (ϵ, δ) upper bound, or optimistically, underestimating it. We use the pessimistic estimate for all reported guarantees, and the optimistic estimate to quantify how much of the reported ϵ is discretization slack rather than true privacy loss. This distinction is what we mean by certified throughout: the reported value is an upper bound by construction, not an approximation of unknown sign.

3

P RIVACY A NALYSIS OF D ENOISING P ROCESS WITH E MPIRICAL D ENOISER

3.1

S INGLE -S TEP D ENOISING IS A GDP

The Bayes denoiser in eq. 1 is the conditional expectation E[x0 |xt ] = R pt (xt )−1 x0 p(xt |x0 ) p(x0 ) dx0 , where p(xRt | x0 ) = N (xt ; x0 , t2 I) is the closed-form EDM forward transition density and pt (xt ) = p(xt | x0 ) p(x0 ) dx0 is the marginal of the noisy state. Real-world samplers approximate this Bayes denoiser with a neural denoiser D(xt , t; θ(D)), but the sensitivity of D to changing one training example is not tractable, so it admits no direct privacy accounting. We therefore analyze the empirical denoiser, which P replaces p(x0 ) with the 1 empirical measure over D and uses the empirical marginal pbt (xt ) = |D| x0 ∈D p(xt | x0 ) as its normalizer. Unlike the transition kernel, pbt is data-dependent and cannot enter a private mechanism without its own sensitivity accounting. Ê[x|xt ; D] =

X 1 x0 p(xt |x0 ). pbt (xt )|D| x0 ∼D

3

(4)

Preprint

Cosine Similarity

1.0 0.8 0.6 0.4 0.2 0.0 10

5

0

log SNR

5

10

Figure 1: Cosine similarity between the denoising directions, i.e., Cos(xt − Ê, xt − D), by empirical and neural denoisers. Here, xt − Ê denotes the actual denoising direction induced by the update rule in eq. 1. Results are reported along the log-SNR axis (Kingma et al., 2021), where SNR = 1/4t2 , providing a unified noise-scale representation across diffusion formulations (e.g., EDM, DDPM, FM).

Substituting eq. 4 into eq. 1 gives   √ P 1 xt−∆t = 1 − 2∆t xt + 2∆t x0 ∈D x0 p(xt |x0 ) + 2t∆t ϵ, t t pt (xt )|D| | {z }

(5)

data-dependent

√ so the only term depending on D is the empirical average, and the noise 2t∆t ϵ already injected by the sampler can serve as the noise of a Gaussian mechanism once this average has bounded sensitivity. Let v(x0 ; xt ) = x0 p(xt |x0 ) and let v̄(x0 ; xt ) = v(x0 ; xt )/ max(1, ∥v(x0 ; xt )∥/C) be its clipped version, so that ∥v̄∥ ≤ C. Computing pbt (xt ) privately would require an additional data-dependent pass, so we replace it with a public constant β (β = 1 in experiments). This yields a deliberately unnormalized empirical mechanism. β = 1 is a design choice, not a claim that pbt (xt ) ≈ 1, and β is a genuine privacy–utility parameter that we ablate in Appendix E. Substituting the clipped average into eq. 1 gives   √ P 1 xt−∆t = 1 − 2∆t xt + 2∆t 2t∆t ϵ, x0 ∈D v̄(x0 ; xt ) + t t β|D| and the public invertible rescaling by βt/(2∆t) shows that the data-dependent part of one step is the Gaussian mechanism r 1 X t3 MGauss (D; xt ) = ϵ. (6) v̄(x0 ; xt ) + β |D| 2∆t x0 ∈D

For neighboring datasets, where D′ adds or removes one record of D, the clipped average has p 2C sensitivity at most 2C/|D|, so eq. 6 is µt -GDP with µt = β|D| 2∆t/t3 . This bound is conservative in the sense that it treats the empirical denoising direction as the private release at every step and accounts for the sampling noise already injected by the diffusion process. 3.2

M ULTI -S TEP D ENOISING IS ALSO A GDP

For discretized timesteps ti with increments ∆ti , the i-th empirical denoising step has µti = p 2C 3 . By exact GDP composition, denoising from t 2∆t /t i i m to tn is µ-GDP with µ = β|D| Pm 2 1/2 ( i=n µti ) . The parameter µ has a direct testing interpretation: larger µ means it is easier to distinguish neighboring datasets from the generated output. Privacy loss is therefore far from 1−1/ρ −1−1/(2ρ) uniform along the trajectory. Under the EDM schedule ∆ti ≈ c ti , so µti ∝ ti , which is roughly 1/ti for ρ = 7 (Figure 5). The last low-noise steps therefore dominate the sum. This suggests a hybrid sampler: run the private empirical denoiser through the early and middle steps, then hand off to a public denoiser, whose steps are post-processing and add no privacy cost (Section 6). 3.3

B RIDGING E MPIRICAL AND N EURAL D ENOISERS

A central limitation is that real diffusion samplers use neural denoisers, while our formal accounting uses an empirical denoiser. Directly accounting for neural denoisers remains open because the sensitivity of a trained network to one training example is not tractable. We therefore use the empirical denoiser as an analyzable proxy and check where this proxy is informative. First, Figure 1 shows that empirical and neural denoising directions are nearly aligned on CIFAR-10 in the earlier stages of the denoising process. Second, recent work on critical windows (Li & Chen, 4

Preprint

Table 1: Endpoint distance (1 − cos) after substituting the empirical denoiser inside one window (swap) and after re-drawing only that window’s noise with the neural denoiser kept (ref.). rel. = swap/ref. CLIP (Radford et al., 2021) Window

t range

high-noise mid-noise low-noise min-noise

19.15–7.47 6.09–1.98 1.54–0.38 0.14–0.018

DINO (Oquab et al., 2024)

swap

ref.

rel.

swap

ref.

rel.

0.0097 0.0772 0.1127 0.2147

0.0917 0.1000 0.0687 0.0241

0.11× 0.77× 1.64× 8.90×

0.0717 0.5175 0.6910 0.8758

0.5792 0.6203 0.4431 0.1376

0.12× 0.83× 1.56× 6.37×

2024; Biroli et al., 2024; Sclocchi et al., 2024; Georgiev et al., 2023; Raya & Ambrogioni, 2023) suggests that coarse semantic identity is determined early and then remains stable, so the formative part of the trajectory overlaps with the regime where the two denoisers are most similar. Together, these checks do not prove a neural-denoiser DP guarantee, but they justify using the empirical accountant to study where sampling leakage occurs. b − D∥2 is Cosine similarity captures only the denoising direction; the per-step magnitude gap ∥E reported in Appendix F, where it peaks inside the mid-noise window, but as Table 1 shows, does not survive to the released sample. We therefore restrict the proxy claim to directional agreement over the regime our private block occupies. Trajectory-level agreement. The per-step divergence above bounds how far the two kernels can differ at a single step, but not how much of that difference survives to the released sample. We therefore substitute the empirical denoiser for the neural denoiser within one window at a time, with the initial noise and all stochastic increments held fixed, and measure the embedding distance (1−cos) of the resulting sample to the all-neural reference trajectory. As a descriptive scale, we also keep the neural denoiser and re-draw only the noise increments inside that window. Table 1 reports both distances over 2048 paired trajectories, and Appendix F adds absolute similarities, paired-bootstrap CIs, PSNR, SSIM, and label agreement (Efron & Tibshirani, 1993; Wang et al., 2004). In the highand mid-noise windows substitution moves the endpoint less than re-drawing that window’s noise (mid-noise: 0.77× CLIP, 0.83× DINO, reference minus swap distance +0.0228 in CLIP, 95% CI [+0.0207, +0.0248]). In the mid-noise window endpoint similarity stays high in CLIP (0.923) but falls in DINO (0.483), so coarse semantics are preserved while instance-level structure is not. In the low-noise window the ordering reverses (CLIP difference −0.0441, 95% CI [−0.0459, −0.0422]), consistent with the t−3 growth of the per-step divergence. The min-noise window is always handled by the public denoiser, while the low-noise window is private in our 65% configuration (Section 6.3). Taken together, these measurements support a claim: through the mid-noise window, about the first half of generation, the empirical denoiser is a window-local semantic proxy for the neural denoiser. They do not establish a DP guarantee for a neural denoiser by itself, which remains open.

4

P RIVACY A NALYSIS OF D ENOISING P ROCESS WITH S UBSAMPLED DATA

Section 3 analyzes the empirical denoiser when every record in D contributes to every private denoising step. This full-batch accountant is tight, but it is expensive in terms of privacy: each generated output repeatedly touches every record, and privacy loss grows quickly when many outputs are released. Subsampling reduces this exposure because a record that is not selected contributes no privacy loss. Throughout, each generated output draws a minibatch S ⊆ D of fixed size B uniformly without replacement, so a fixed record is included with probability qB = B/n, and neighboring datasets differ by adding or removing one record. Because |S| = B is fixed, an added record that is selected takes the place of another record of S, so within the minibatch the change is a replacement (Zhu et al., 2022; Lebeda et al., 2025). The clipped minibatch average therefore has sensitivity p 2C 2C/B, and each step is µt -GDP with µt = βB 2∆t/t3 conditional on inclusion. The subsampled p P Gaussian mechanism is MSubGauss (D; xt ) = |S|−1 x0 ∈S v̄(x0 ; xt ) + β t3 /(2∆t) ϵ. 5

Preprint

noise level σ: 80.0

43.9

19.2

9.11

3.17

1.19

0.280 0.067 0.007 0.002

(a) Output-wise (b) Per-step

(a) Output-wise (b) Per-step

Figure 2: Output-wise vs. per-step subsampling, B = 25, identical initial noise. Each row is one sample; columns show decreasing noise levels. Under output-wise subsampling (a) the subsample is reused, so each row progressively denoises the same emerging image. Under per-step subsampling (b) a fresh subsample is drawn each step, pulling the trajectory toward a different subset: semantic content changes between columns and never settles. This instability motivates block reuse. The important design choice is how often we resample S. Resampling at every denoising step for every output gives more independent amplification events, but it also changes the empirical denoising direction at every step. Small minibatches, which is better for privacy, inject extra stochastic drift into the reverse trajectory: the sample is repeatedly pulled toward different private subsets, and late low-noise steps are especially sensitive to this instability, as shown in Figure 2. This can harm sample quality even though the privacy accountant looks attractive. We therefore employ output-wise subsampling: sample once for a generated output, reuse them through an empirical denoising block, then account for amplification once at the block level. This keeps the denoising direction stable while still limiting how often any record is accessed. We state both accountants because they correspond to different mechanisms. Applying per-step amplification to output-wise subsampling would be incorrect: privacy amplification is tied to independent subsampling events, not to the number of denoising steps. If the same minibatch is reusedPover a block B, the included record accumulates the full block-level Gaussian parameter µ̄B = ( t∈B µ2t )1/2 before amplification is applied. If a fresh minibatch is drawn at every step, each µt is amplified separately. For full-batch Gaussian mechanisms, GDP composition is exact. With subsampling, each block’s trade-off curve is a mixture rather than a Gaussian curve, so exact GDP composition no longer applies, and we compose numerically with a pessimistic PLD accountant (Section 2.3). The GDP central limit theorem (Dong et al., 2022) fits poorly here because the µt are highly heterogeneous, and its Berry-Esseen error is additive and too coarse for small δ (Table 11). As a fast estimate, we also compute a degree-two Edgeworth approximation (Zheng et al., 2020), which adds skewness and kurtosis corrections from the first four cumulants (eq. 25, Appendix D). It is an approximation and matches the certified PLD value to within 1% for ε ≤ 2 but understates it for larger ε, by up to 15% in our runs (Appendix G). Proposition 4.1 (Output-wise subsampling). Consider a sampler that, for each of Nout generated outputs, performs private subsampling once and reuses them for an empirical denoising block B; all later public denoising P is post-processing. Let qB be the block-level inclusion probability of a fixed record and µ̄B = ( t∈B µ2t )1/2 . Define hB = qB Gµ̄B + (1 − qB ) Id and gout =

N out O

hB ,

−1 ∗∗ fout = min{gout , gout } .

(7)

j=1

Then the joint release of the Nout generated outputs is fout -DP. Equivalently, for every ε ≥ 0 it is (ε, δout (ε))-DP with   dir dir δout (ε) = max PrQ (Sout > ε) − eε PrP (Sout > ε) + , (8) dir∈{P →Q,Q→P }

PNout N where Sout = j=1 LB (Zj ), LB (z) = log(1 − qB + qB exp(µ̄B z − µ̄2B /2)), and indicates a tensor product. The probabilities are taken under the two neighboring-data hypotheses. We evaluate δout (ε) with a pessimistic PLD accountant, which gives the certified values we report, and use the Edgeworth expansion only as an estimate (Appendix G). 6

Preprint

Algorithm 1 instantiates this: the subsample is drawn once per output (line 3) and reused across the block, so amplification is applied once to µ̄B rather than per step. In practice, each output enters the PLD accountant (Section 2.3) as one subsampled Gaussian mechanism with noise multiplier 1/µ̄B and sampling probability qB , whose two neighboring directions give exactly hB and h−1 B . Composing Nout copies gives δout (ε), and Appendix G lists every step and setting. The proof is in Appendix C, which also gives the accountant for per-step resampling (Proposition C.1). That accounting is valid only when the implementation truly resamples at every step, so it does not apply to our sampler. P Output-wise subsampling is not privacy-free: if t µ2t is large, the influence of an included record accumulates before amplification. This motivates our hybrid denoiser, which switches to a public neural denoiser after the empirical denoising block so that the remaining generation is post-processing. We use the Edgeworth approximation only when the composed privacy loss is in a stable finite-sample regime; Appendix D states these diagnostics.

5

R ELATED W ORK

Differentially private training has been the most direct approach to reduce memorization in diffusion models. DP training of diffusion models builds on DP-SGD (Abadi et al., 2016; Dockhorn et al., 2023; Ghalebikesabi et al., 2023; Liu et al., 2024) and protects the weights at a cost in sample quality. Private sampling has mostly been studied for Bayesian posterior inference, where the samples are model parameters rather than data (Welling & Teh, 2011; Wang et al., 2015; Li et al., 2019; Räisä et al., 2021; Zhang & Zhang, 2023). Bertazzi et al. (2025) show that privacy loss in SGLD can concentrate in the final step, but their analysis needs strongly convex, Lipschitz objectives, which our empirical denoiser does not satisfy. Sampling-time privacy for diffusion models. Closest to our setting, Sehwag et al. (2023) also analyze privacy at generation rather than training time, clip denoiser outputs, treat each step as a Gaussian mechanism, and delegate late denoising to a public model. Our analysis differs in three respects. First, their per-step sensitivity C/k arises from a sample-and-aggregate ensemble of k non-private models trained on disjoint shards, so the sensitivity floor is set by the ensemble size and private training is still required; we obtain sensitivity 2C/|D| by clipping individual contributions to a single empirical average, with no training at all. Second, they adopt a per-sample private-prediction threat model and name composition across multiple released samples as open; we give an explicit joint-release guarantee over Nout outputs under standard DP, using subsampled f -DP with block-level amplification. Third, we localize privacy loss to the critical window and place the handoff at the measured crossover rather than at a fixed schedule fraction, validating the choice with a membershipinference proxy. Wei et al. (2023) study inherent privacy properties of discrete denoising diffusion and likewise find that leakage concentrates at particular stages of generation. Their mechanism is the discrete transition kernel, while ours is the continuous Gaussian reverse SDE, where the noise already injected by the sampler serves as the mechanism noise. Our results are therefore a continuous-state counterpart to their staged view, and our claims are scoped accordingly. Empirical denoisers. Niedoba et al. (2025) characterize the empirical denoiser in the non-private setting and analyze how trained neural denoisers depart from it. We build on this object directly: our contribution is the clipped, privatized variant and its sensitivity analysis, and their non-private characterization is complementary evidence for the proxy argument in Section 3.3.

6

E XPERIMENTS

We study (i) full-batch versus subsampled accounting, (ii) where privacy loss accumulates relative to the critical window, and (iii) the privacy-utility trade-off of the hybrid sampler. Implementation We implement our modified reverse denoising process in PyTorch (Paszke et al., 2019) and categorize our experiments into two regimes: (i) full-batch, where the entire dataset is used in each denoising step, and (ii) output-wise subsampled SDEdit (Meng et al., 2021), where a minibatch of private examples is subsampled once per generated output and reused for an empirical denoising block. We apply the private empirical denoising steps until time t0 , then add Gaussian 7

Preprint

FID

40 30

45

steps = 3

40

steps = 5

35

steps = 9

q = 0.0005 q = 0.001 q = 0.002 q = 0.004

50

steps = 7

30

40 30

25 20

20

(c) Varying q (steps = 3)

FID

t0 = 55% t0 = 62% t0 = 65% t0 = 67% t0 = 70%

50

(b) Varying steps (q = 0.0005)

FID

(a) Varying t0 (steps = 3)

20

15 100

101

102

εu (certified PLD)

103

100

101

102

εu (certified PLD)

100

101

102

103

εu (certified PLD)

Figure 3: Privacy-utility trade-offs of the output-wise subsampled hybrid sampler on CIFAR-10. We report FID against the certified PLD upper bound εu (δ = 10−5 , values from Table 8) while varying (a) the handoff time t0 , (b) the number of public re-noising steps from t0 to t1 , and (c) the subsampling rate q = B/n. noise with scale (σt21 − σt20 )1/2 to move the sample back to an SDEdit restart level t1 > t0 , and finally complete generation with a public neural denoiser from t1 to 0. Only the empirical subsampled block is privacy-accounted. Re-noising to a higher noise level lets the public neural denoiser correct artifacts introduced by the empirical step while incurring no additional privacy loss by post-processing. We use the EDM schedule (Karras et al., 2022) with σmin = 0.002, σmax = 80, ρ = 7, N = 50 steps for full-batch runs and N = 39 for the hybrid pipeline, and the public normalization β = 1 (Appendices E and J). Privacy Analysis Given the noise schedule and sensitivity, we calculate the corresponding µ-GDP for each diffusion step (3). For the full-batch case, we derive the overall µ-GDP using the n-fold composition of µi -GDP mechanisms. For the output-wise subsampled SDEdit hybrid, we compute privacy using the certified PLD accounting library (Doroshenko et al., 2022) with grid spacing h = 10−5 and log_mass_truncation_bound = −50, and the Edgeworth approximation from Section 4, which approximates the per-output privacy-loss distribution under both neighbouringdata hypotheses using its first four cumulants. The subsequent re-noising and public neural denoising stages incur no additional privacy loss. Datasets and Evaluation On CIFAR-10 (Krizhevsky et al., 2009) (n = 50,000) the public denoiser is trained on ImageNet 32 × 32 (Russakovsky et al., 2015), and on CelebA 64 × 64 (Liu et al., 2015) (n = 162,770) it is trained on FFHQ (Karras et al., 2019). We use δ = 10−5 ≈ 1/n for CIFAR-10 and 10−6 for CelebA-64 to match prior DP generative models, and report the stricter δ = 1/n2 in Appendix K. Utility is FID against the training set with 50,000 samples, reported as mean ± standard deviation over three generation seeds where runs are repeated. 6.1

P RIVACY ANALYSIS OF DIFFUSION - BASED SAMPLING WITH FULL TRAINING DATA

When every record is used at every step, a single sample already costs ε ≈ 200 when denoising down to σmin = 0.1, which is needed for a recognizable sample, and at ε ≈ 1 (σmin = 5) the sample is indistinguishable from noise (Figure 4, Appendix I). Since each denoising step operates on the full dataset, the per-sample privacy loss is already high, and composing multiple samples results in a rapidly growing total privacy budget under GDP. Subsampling can alleviate this issue through privacy amplification, making it more suitable for large-scale sample generation. 6.2

H YBRID DENOISING WITH OUTPUT- WISE SUBSAMPLING

Figure 3 highlights three patterns. First, the empirical denoising stop timestep t0 controls a clear privacy-utility tradeoff: stopping later at t0 = 70% yields the best FID, while stopping earlier at t0 = 55% yields the strongest privacy guarantee. Second, the best number of public re-noising steps depends on the privacy budget. With larger private blocks (65–70%, certified ε ≥ 6) three re-noising steps give the best FID, while with the smallest block (55%, ε ≤ 0.3) more steps help (FID 45.7 with three steps and 30.4 with nine, B = 25). Re-noising steps are public, so they change utility but not ε. 8

Preprint

Table 2: Comparison of DP generative models on CIFAR-10 and CelebA-64. Our method performs a short private editing block using an output-wise subsampled empirical denoiser, then hands off to a publicly pretrained neural denoiser (ImageNet for CIFAR-10, FFHQ for CelebA-64) to complete sampling. FID is computed for Nout = 50, 000 samples. Lower FID is better. CIFAR-10

CelebA-64

Method

ε

FID

Method

ε

FID

DP-Diffusion (Ghalebikesabi et al., 2023) DP-LDM (Liu et al., 2024) DP-API (Lin et al., 2023) DP-MEPF (Harder et al., 2023) DP-DM (Dockhorn et al., 2023) DP-Promise (Wang et al., 2024)

10 10 0.67 10 10 10

9.8 8.4 7.87 29.1 97.7 17.9

DP-LDM (Liu et al., 2024) DP-MEPF (Harder et al., 2023) DP-DM (Dockhorn et al., 2023) DP-Promise (Wang et al., 2024)

10 10 10 10

14.3 17.4 78.3 25.3

Ours

PLD [11.14, 11.38] εew = 10

27.1 ± 0.15

Ours

PLD [6.53, 6.77] εew = 5

25.67 ± 0.03

Third, decreasing the subsampling rate q lowers ε, as expected from privacy amplification. In our setting it also improves FID, likely because smaller q weakens the high-variance empirical denoiser before the public neural denoiser recovers global structure after re-noising. To contextualize the effect of our privacy mechanisms, Table 2 compares the FID (Heusel et al., 2017) of our hybrid sampler with DP generative models on CIFAR-10 and CelebA-64, and Appendix J reports all hybrid configurations.

6.3

P RIVACY IN RELATION TO THE CRITICAL WINDOWS

We extend our privacy analyses to quantify privacy levels based on where we stop relative to the critical window of the denoising process. Following Georgiev et al. (2023), we locate the critical window with a zero-shot CLIP classifier (Radford et al., 2021) applied along the trajectory (Figure 6(a)). The probability that the sample is classified as a ship is about 0.1 at t = 0.27, 0.3 at t = 0.07, and close to 1 at t = 0.02 (70%, 80%, and 88% of the N = 50 steps). Stopping the private block earlier gives a smaller µ but a noisier sample, and stopping later gives the reverse (Figure 7). We choose the handoff point so that, over the private block, the empirical denoiser remains a faithful stand-in for the neural denoiser. The empirical denoiser is a window-local semantic proxy for the neural denoiser through the mid-noise window, which ends at t = 1.98, about half of the N = 39 steps (Table 1), and directional agreement stays at cos ≥ 0.98 through about 55% of generation (Figure 1, Table 4). A 55% handoff therefore keeps the private block essentially inside the proxy regime. Our 65% handoff (t0 ≈ 0.52) still comes before the CLIP-visible critical window (≈ 70– 88%), but its private block covers most of the low-noise window, where substitution moves the endpoint more than re-drawing that window’s noise (1.64× CLIP, Table 1). The proxy interpretation is therefore weaker for the 65% block, while the certified privacy guarantee holds for both, because it is a statement about the sampler we run. Stopping earlier tightens the certified bound at a utility cost, and stopping later extends the private block into the low-noise steps that dominate the privacy cost (Appendix J).

6.4

P RIVACY IMPLICATIONS OF NEURAL AND EMPIRICAL DENOISERS IN MEMBERSHIP INFERENCE ATTACKS

We compare the membership-inference success rate (ASR) implied by our ε for the subsampled empirical sampler (Appendix L) with the ASR that the attack of Duan et al. (2023) achieves on a neural denoiser trained on all of CIFAR-10, stopping at the same t and using the CIFAR-10 test set as non-members (Table 13, Appendix L). The attack is weak at every t (ASR at most 0.510) and stays below the bound for q = 0.5 at every t. At q ≤ 0.01 it slightly exceeds the bound at t = T /2 (0.507 vs. at most 0.503), which is expected because the neural denoiser uses every record while the subsampled sampler includes each record with probability q. 9

Preprint

7

C ONCLUSION AND D ISCUSSION

We give certified sampling-time privacy guarantees for continuous diffusion models by privatizing a clipped empirical denoiser. Framing each denoising step as a Gaussian mechanism, we account with GDP and f -DP, report certified (ϵ, δ) bounds from a PLD accountant, and characterize where Edgeworth approximation is and is not reliable. Our results show that leakage is highly nonuniform across timesteps, concentrating in critical windows where semantic structure emerges, while subsampling significantly amplifies privacy guarantees. These findings suggest that privacy in diffusion models is not solely determined by training but can be shaped by sampling strategies such as early stopping and hybrid use of public denoisers. Our certificate covers the sampler we run. Samplers that query a neural denoiser trained on the private data at every step, and discrete diffusion, where the transition kernel would be the mechanism, remain open. Our sampler could also serve as a DP data-augmentation mechanism A(D): a model trained only on A(D) and public data is (ε, δ)-DP by post-processing, with no DP training algorithm. Since privacy loss concentrates at low noise, training on A(D) for t ≥ t0 and on public data for t < t0 would spend the budget only on the formative regime. AI USE STATEMENT In this work, we used generative AI tools as writing, reviewing, and coding assistants. Specifically, we used them to improve grammar and word choice, to draft and revise parts of the text, to get feedback on drafts, such as consistency checks of numbers, claims, and references and suggestions for additional diagnostics. We also used them to search for and summarize related work, which we verified against the original sources, and to help write and debug code for the accounting, evaluation, and scripts, which we reviewed and tested before use. Additionally, we used AI to proofread our math equations and review our mathematical proofs. We did not use generative AI tools to generate research ideas or synthetic data, or to produce any reported number. Dataset cleaning, translation, and qualitative data analysis are not applicable to this work. We have reviewed all AI-assisted work and take full responsibility for the final content of this paper, including text, claims, and artifacts produced with the aid of generative AI. E THICS STATEMENT This work studies privacy leakage from diffusion sampling and uses only public benchmark datasets (CIFAR-10, CelebA, ImageNet, FFHQ). Our guarantees cover the sampler in Algorithm 1 and should not be read as guarantees for other diffusion samplers (Appendix M). R EPRODUCIBILITY STATEMENT The code is available in Appendix A, including the PLD accounting, Edgeworth, and FID scripts behind every table and figure. Algorithm 1 specifies the sampler, Section 3 and Appendix E give the sensitivity and the public normalization, and Appendix C gives full proofs. Section 6 and Appendix G list the accountant settings (sampling model, adjacency, q, Nout , δ, grid spacing, truncation, and precision). Appendix J lists every hyperparameter and reports all 95 CIFAR-10 and 112 CelebA-64 configurations (Tables 8 and 9) and the repeated-seed runs.

10

Preprint

R EFERENCES Martin Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, pp. 308–318, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450341394. doi: 10.1145/2976749. 2978318. Andrea Bertazzi, Tim Johnston, Gareth O. Roberts, and Alain Oliviero Durmus. Differential privacy guarantees of markov chain monte carlo algorithms. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=lLnAua5poB. Giulio Biroli, Tony Bonnaire, Valentin de Bortoli, and Marc Mézard. Dynamical regimes of diffusion models. 2024. URL https://arxiv.org/abs/2402.18491. arXiv preprint arXiv:2402.18491. Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Conference on Security Symposium, SEC ’23, USA, 2023. USENIX Association. ISBN 978-1-939133-37-3. Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research, 2023. URL https://openreview. net/forum?id=ZPpQk7FJXF. Jinshuo Dong, Aaron Roth, and Weijie J. Su. Gaussian differential privacy. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):3–37, 02 2022. ISSN 1369-7412. doi: 10.1111/rssb.12454. URL https://doi.org/10.1111/rssb.12454. Vadym Doroshenko, Badih Ghazi, Pritish Kamath, Ravi Kumar, and Pasin Manurangsi. Connect the dots: Tighter discrete approximations of privacy loss distributions. arXiv preprint arXiv:2207.04380, 2022. Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu. Are diffusion models vulnerable to membership inference attacks? In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 8717–8730. PMLR, Jul 2023. URL https://proceedings.mlr.press/v202/duan23b.html. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography Conference (TCC), pp. 265–284, 2006. Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap, volume 57 of Monographs on Statistics and Applied Probability. Chapman & Hall, New York, 1993. Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the destination: How data guides diffusion models. In Arxiv preprint arXiv:2312.06205, 2023. Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L. Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images, 2023. Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical composition of differential privacy. Advances in Neural Information Processing Systems, 34:11631–11642, 2021. Frederik Harder, Milad Jalali, Danica J. Sutherland, and Mijung Park. Pre-trained perceptual features improve differentially private image generation. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=R6W7zkMz0P. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 11

Preprint

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840–6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf. Hailong Hu and Jun Pang. Membership inference of diffusion models, 2023. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4401–4410, 2019. Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusionbased generative models. In advances in neural information processing systems, volume 35, pp. 26565–26577, 2022. Shiva Prasad Kasiviswanathan, Homin K. Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. What can we learn privately? SIAM Journal on Computing, 40(3):793–826, 2011. doi: 10.1137/090756090. URL https://doi.org/10.1137/090756090. Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 34:21696–21707, 2021. Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, ON, Canada, 2009. Christian Janos Lebeda, Matthew Regehr, Gautam Kamath, and Thomas Steinke. Avoiding pitfalls for privacy accounting of subsampled mechanisms under composition. In IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 996–1006, 2025. doi: 10.1109/SaTML64287. 2025.00060. Bai Li, Changyou Chen, Hao Liu, and Lawrence Carin. On connecting stochastic gradient mcmc and differential privacy. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pp. 557–566. PMLR, 16–18 Apr 2019. URL https://proceedings.mlr.press/v89/li19a.html. Marvin Li and Sitan Chen. Critical windows: non-asymptotic theory for feature emergence in diffusion models. 2024. URL https://arxiv.org/abs/2403.01633. arXiv preprint arXiv:2403.01633. Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differentially private synthetic data via foundation model apis 1: Images, 2023. Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022. Michael F Liu, Saiyue Lyu, Margarita Vinaroz, and Mijung Park. Differentially private latent diffusion models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=AkdQ266kHj. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. Andrew Lowy, Zhuohang Li, Jing Liu, Toshiaki Koike-Akino, Kieran Parsons, and Ye Wang. Why does differential privacy with large ε defend against practical membership inference attacks? In AAAI Workshop on Privacy-Preserving Artificial Intelligence, Feb 2024. URL https://www. merl.com/publications/TR2024-009. MERL Technical Report TR2024-009 / arXiv preprint arXiv:2402.09540. Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai. Membership inference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW), pp. 77–83, 2023. doi: 10.1109/SPW59333.2023.00013. 12

Preprint

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021. Matthew Niedoba, Berend Zwartsenberg, Kevin Murphy, and Frank Wood. Towards a mechanistic explanation of diffusion model generalization. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of PMLR, pp. 46389–46411, 2025. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=a68SUt6zFt. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763. PMLR, 2021. Ossi Räisä, Antti Koskela, and Antti Honkela. Differentially private hamiltonian monte carlo. In NeurIPS 2021 Workshop Privacy in Machine Learning, 2021. URL https://openreview. net/forum?id=oqNqsnOVwlK. Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211–252, 2015. Antonio Sclocchi, Alessandro Favero, and Matthieu Wyart. A phase transition in diffusion models reveals the hierarchical nature of data. 2024. URL https://arxiv.org/abs/2402.16991. arXiv preprint arXiv:2402.16991. Vikash Sehwag, Ashwinee Panda, Ashwini Pokle, Xinyu Tang, Saeed Mahloujifar, Mung Chiang, J Zico Kolter, and Prateek Mittal. Differentially private generation of high fidelity samples from diffusion models. In ICML 2023 Workshop on Deployable Generative AI, 2023. URL https://openreview.net/forum?id=vuVGcl0ed1. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. deep unsupervised learning using nonequilibrium thermodynamics. In proceedings of the 32nd international conference on machine learning, pp. 2256–2265, 2015. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. Shuai Tang, Zhiwei Steven Wu, Sergul Aydore, Michael Kearns, and Aaron Roth. Membership inference attacks on diffusion models via quantile regression, 2024. Aleksei Triastcyn and Boi Faltings. Bayesian differential privacy for machine learning. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 9583–9592. PMLR, Jul 2020. URL https://proceedings.mlr. press/v119/triastcyn20a.html. Haichen Wang, Shuchao Pang, Zhigang Lu, Yihang Rao, Yongbin Zhou, and Minhui Xue. dppromise: Differentially private diffusion probabilistic models for image synthesis. In 33rd USENIX Security Symposium (USENIX Security 24), Philadelphia, PA, August 2024. USENIX Association. URL https://www.usenix.org/conference/usenixsecurity24/ presentation/wang-haichen. 13

Preprint

Yu-Xiang Wang, Stephen Fienberg, and Alex Smola. Privacy for free: Posterior sampling and stochastic gradient monte carlo. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2493–2502, Lille, France, 07–09 Jul 2015. PMLR. URL https:// proceedings.mlr.press/v37/wangg15.html. Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan. Subsampled rényi differential privacy and analytical moments accountant. In AISTATS, 2019. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600–612, 2004. Rongzhe Wei, Eleonora Kreačić, Haoyu Wang, Haoteng Yin, Eli Chien, Vamsi K Potluru, and Pan Li. On the inherent privacy properties of discrete denoising diffusion models. arXiv preprint arXiv:2310.15524, 2023. Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, pp. 681–688, Madison, WI, USA, 2011. Omnipress. ISBN 9781450306195. Yixin Wu, Ning Yu, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks against text-to-image generation models, 2023. URL https://openreview.net/forum? id=J41IW8Z7mE. Wanrong Zhang and Ruqi Zhang. DP-fast MH: Private, fast, and accurate Metropolis-Hastings for large-scale Bayesian inference. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 41847–41860. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/ zhang23aw.html. Qinqing Zheng, Jinshuo Dong, Qi Long, and Weijie Su. Sharp composition bounds for gaussian differential privacy via edgeworth expansion. In International Conference on Machine Learning, pp. 11420–11435. PMLR, 2020. Yuqing Zhu, Jinshuo Dong, and Yu-Xiang Wang. Optimal accounting of differential privacy via characteristic function. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.

A

C ODE AVAILABILITY

The public code is available DPDiffusionSampling/.

B

at

https://github.com/whalebeavercat/

L IPSCHITZ - BASED SENSITIVITY ANALYSIS FOR NEURAL DENOISERS

One might wonder whether considering Lipschitz-based sensitivity analysis for neural denoisers would help validate the empirical denoiser bounds. We describe our thoughts on this below. Let L : X n → Θ be the score network learning algorithm which maps a dataset S ∈ θ ∈ Θ. Let us define the sensitivity of the score network as ∆s = X n to parameters   supx∈X supS∈X n ,i∈[n] ∥sL(S) (x) − sL(S\{i}) (x)∥2 . If we could show that ∆s is bounded, then the update step in sampling can be made differentially private with appropriate noise scales as follows: x̃t ← x̃t−1 + σ2i sθ (x̃t−1 , σi ) + N (0, σt2 ∆2s I). The question is how to bound the sensitivity. Does the Lipschitz network help here? Suppose the score network is a 1-Lipschitz network, satisfying ∥sL(S) (x) − sL(S) (x′ )∥2 ≤ ∥x − x′ ∥2 and ∥sL(S\{i}) (x) − sL(S\{i}) (x′ )∥2 ≤ ∥x − x′ ∥2 . Using this we can bound the sensitivity, 14

Preprint

∥sL(S) (x) − sL(S\{i}) (x)∥2 ≤ ∥sL(S) (x) − sL(S) (x′ )∥2 + ∥sL(S) (x′ ) − sL(S\{i}) (x)∥2 ≤ ∥x − x′ ∥2 + ∥sL(S) (x′ ) − sL(S\{i}) (x)∥2 . So here even if ∥sL(S) (x′ ) − sL(S\{i}) (x)∥2 is very small, we have maxx,x′ ∥x − x′ ∥2 on our upperbound. Without having any knowledge other than all x, x′ are pre-processed such that any of them is normbounded by L, the upperbound to this is maxx,x′ ∥x − x′ ∥2 ≤ maxx ∥x∥2 ≤ 2L. In this case, the amount of noise we have to add in each denoising step is on the order of the entire input domain, and such an amount of noise will destroy the signal from the score net completely. Hence, this approach is not usable in practice.

C

P ROOFS FOR S UBSAMPLING P ROPOSITIONS

We prove Proposition 4.1 and Proposition C.1 (per-step subsampling, stated below) under the sampling model stated in Section 4. The proofs are directional first, comparing two neighboring datasets D and D′ that differ by adding or removing one record, and then symmetrized by taking both neighboring-data directions. C.1

P ROOF OF P ROPOSITION 4.1

Fix one generated output and an empirical denoising block B, and let D′ = D ∪ {x}, so that qB = B/n with n = |D′ |. A uniform B-subset of D′ excludes x with probability 1 − qB , and it is then a uniform B-subset of D, so the two neighboring datasets induce the same output distribution and the trade-off curve is Id. With probability qB , it is S ′ ∪ {x}, where S ′ is a uniform (B − 1)-subset of D. We pair it with S ′ ∪ {y}, where y is uniform in D \ S ′ , which is a uniform B-subset of D. These two minibatches differ by one replacement, so each step has parameter µt (Section 4), and by GDP composition the block is µ̄B -GDP for every such pair, where  X 1/2 µ̄B = µ2t . (9) t∈B

The hockey-stick divergence is jointly convex, so this bound survives averaging over (S ′ , y), and the canonical binary experiment for the included-record event is PB = N (0, 1) versus N (µ̄B , 1), with trade-off curve Gµ̄B . Thus the one-output directional trade-off curve is hB = qB Gµ̄B + (1 − qB ) Id .

(10)

The Nout generated outputs use independent row-subsampling events, so f -DP composition gives the directional composed trade-off gout =

N out O

hB .

(11)

j=1

Since differential privacy must hold for both neighboring-data directions, we symmetrize the directional curve and take the lower convex envelope, yielding −1 ∗∗ fout = min{gout , gout } .

(12)

Therefore the joint release is fout -DP. Any public neural denoising or deterministic post-processing after the private block is post-processing of the same private release and cannot weaken the guarantee. It remains to derive the (ε, δ) profile. In the above directional experiment, the privacy-loss random variable for one output can be written as    µ̄2B LB (z) = log 1 − qB + qB exp µ̄B z − , (13) 2 PNout and the composed privacy loss is Sout = j=1 LB (Zj ). The hockey-stick divergence characterization of approximate DP gives P →Q δout (ε) = PrQ (Sout > ε) − eε PrP (Sout > ε).

(14)

Applying the same argument in the reverse neighboring direction and taking the positive part gives the stated δout (ε). 15

Preprint

C.2

P ER - STEP SUBSAMPLING

Proposition C.1 (Per-step subsampling). Consider instead a sampler that draws an independent minibatch of fixed size Bt at every private denoising step. For step t, let qt = Bt /n be the inclusion probability and µt the Gaussian parameter conditional on inclusion, and set ht = qt Gµt +(1−qt ) Id. For Nout outputs and T private steps, define gstep =

N T out O O

−1 ∗∗ fstep = min{gstep , gstep } .

ht ,

(15)

j=1 t=1

Then the joint release is fstep -DP. Hence, for every ε ≥ 0 it is (ε, δstep (ε))-DP, where δstep (ε) = PNout PT dir dir maxdir [PrQ (Sstep > ε) − eε PrP (Sstep > ε)]+ , Sstep = j=1 t=1 Lt (Zj,t ), and Lt (z) = 2 log(1 − qt + qt exp(µt z − µt /2)). The describe different privacy-loss distributions. In general, Sstep = P two propositions P j,t Lt (Zj,t ) ̸= j LB (Zj ) = Sout , so fstep ̸= fout . Per-step amplification is valid only when the implementation truly resamples at every step. For our output-wise subsampled sampler, Proposition 4.1 is the correct accountant: amplification is applied once to the whole empirical block. C.3

P ROOF OF P ROPOSITION C.1

For per-step subsampling, the differing record is included at step t with probability qt , independently across steps. As in Appendix C.1, each minibatch of D′ containing x is paired with a minibatch of D that differs from it in one record, now with Bt , qt and µt in place of B, qB and µ̄B . Conditional on inclusion at step t, the step is a Gaussian mechanism with GDP parameter µt , so the canonical included-record trade-off curve is Gµt . If the record is not included, the two neighboring datasets induce the same distribution for that step and the trade-off curve is Id. Therefore the one-step directional trade-off curve is ht = qt Gµt + (1 − qt ) Id .

(16)

Because minibatches are redrawn independently across timesteps and generated outputs, the directional trade-off curve for Nout outputs and T private steps is the tensor-product composition gstep =

N T out O O

ht .

(17)

j=1 t=1

Symmetrizing over the two neighboring-data directions and convexifying gives −1 ∗∗ fstep = min{gstep , gstep } ,

(18)

so the joint release is fstep -DP. The corresponding likelihood-ratio representation is obtained by summing the independent per-step privacy losses. For one step,    µ2t Lt (z) = log 1 − qt + qt exp µt z − , (19) 2 and hence Sstep =

N T out X X

Lt (Zj,t ).

(20)

j=1 t=1

Using the same hockey-stick divergence conversion as above gives   dir dir δstep (ε) = max PrQ (Sstep > ε) − eε PrP (Sstep > ε) + , dir

which is the claimed (ε, δ) guarantee. 16

(21)

Preprint

D

D IAGNOSTICS FOR E DGEWORTH A PPROXIMATION

The standard Edgeworth approximation for subsampled Gaussian mechanisms is often stated for the setting where a fresh subsample is drawn at every private step. Our output-wise subsampling mechanism uses the same mixture privacy-loss model, but after grouping timesteps by independent subsampling events. Thus each “step” in the diagnostic below should be read as a block B with sampling probability qB and Gaussian parameter !1/2 X 2 µ̄B = µt . (22) t∈B

For a block, recall the privacy-loss variable    µ̄2B LB (z) = log 1 − qB + qB exp µ̄B z − , 2

(23)

evaluated under both PB = N (0, 1) and QB = (1 − qB )N (0, 1) + qB N (µ̄B , 1). Let κD r,B denote the r-th cumulant of LB under D ∈ {P, Q}. Across generated samples and independent blocks, cumulants add: X X X X MD = κD VD = κD K3,D = κD K4,D = κD (24) 1,B , 2,B , 3,B , 4,B . B

B

B

B

Edgeworth approximation. Let S denote a composed privacy-loss variable with mean M , variance V , third cumulant K3 , and fourth cumulant K4 (that is, MD , VD , K√ 3,D , K4,D under D ∈ {P, Q}). The degree-two Edgeworth approximation to the CDF of (S − M )/ V is   K4 K3 K32 2 3 5 3 (z − 1) + FEW (z) = Φ(z) − φ(z) (z − 3z) + (z − 10z + 15z) , (25) 24V 2 72V 3 6V 3/2 where Φ and φ are the standard normal CDF and density. We apply this approximation separately under the two neighboring-data hypotheses, since the two privacy-loss CDFs need not match. What the cumulants represent.

For a random variable X, the cumulant generating function is

KX (s) = log E[esX ], (26) and the r-th cumulant is the r-th derivative of KX at s = 0. In our accountant, X = LB under either P or Q. The first four cumulants have the familiar interpretations κ1 = E[X], (27) κ2 = Var(X), (28) κ3 = E[(X − EX)3 ], 4

(29) 2

κ4 = E[(X − EX) ] − 3Var(X) . (30) Thus κ1 and κ2 give the mean and variance of the privacy loss, while κ3 and κ4 quantify the leading skewness and kurtosis corrections to the Gaussian approximation. No lower bound on the sampling rate alone. A small sampling probability qB does not by itself invalidate the Edgeworth approximation. For fixed finite µ̄B , the variance and higher cumulants shrink with powers of qB , and these powers largely cancel after standardization. The relevant question is not whether qB is large, but whether the composed privacy-loss sum has enough effective variance and is not dominated by a small number of blocks. Variance and dominance checks. First, VP and VQ should be positive and numerically stable. If either variance is extremely small, the mechanism may be close to perfect privacy, but the standardized Edgeworth CDF and inverse-CDF calculations become ill-conditioned. Second, no block should dominate the variance: κQ κP 2,B 2,B max ≪ 1, max ≪ 1. (31) B B VP VQ This condition P is particularly important for diffusion schedules, where late low-noise timesteps can dominate t µ2t if they are grouped into a large private block. 17

Preprint

Standardized cumulants. The Edgeworth correction should be small at the quantiles used for the reported (ε, δ) guarantee. We monitor the standardized third and fourth cumulants |K3,P | |K4,P | |K4,Q | |K3,Q | ρ3,P = , ρ4,P = , ρ4,Q = ρ3,Q = . (32) 2 , 3/2 3/2 V VQ2 V V P P

Q

Small values near the center of the distribution are not sufficient by themselves: degree-two Edgeworth terms multiply these quantities by polynomials in the standardized quantile z, so far-tail evaluations can still be unreliable. Tail and CDF checks. Very small δ values may require evaluating the approximation close to the boundary of the CDF, where finite-sample Edgeworth approximations are less stable. We therefore check that the approximated CDFs are monotone, remain in [0, 1] before substantial clipping or projection, and produce a nonnegative δ(ε) curve in the reported range. We compute both neighboringdata directions and use the larger value. The Edgeworth value is never a certified bound. These diagnostics only decide whether the estimate is stable enough to show next to the certified PLD value, and configurations that fail them are marked invalid (Table 9). Practical implications of output-wise subsampling. Suppose the dataset has size n = |D| and each generated sample draws a minibatch of size B under the sampling model of Section 4, reused for an empirical denoising block. For a fixed training record, the block-level inclusion probability is qB = B/n. (33) For our CIFAR-10 setting with n = 50,000 and B = 25, this gives qB = 5 × 10−4 . If we release Nout = 50,000 generated samples, then a fixed record is included in about Nout qB = 25 (34) generated trajectories on average. The privacy-relevant point is that each inclusion can affect the entire empirical block. Therefore the output-wise subsampled accountant is not Nout T subsampled Gaussian mechanisms with parameters (qt , µt ), (35) which would correspond to independently resampling at every denoising step. It is instead  !1/2  X . µ2t (36) Nout subsampled block mechanisms with parameters qB , t∈B

This is the practical reason to subsample per output once and keep the empirical block short: outputwise subsampling sharply limits how often any record is accessed, but once a record is selected its influence accumulates across all empirical denoising steps in that block.

E

E FFECT OF THE PUBLIC NORMALIZATION CONSTANT β

For EDM, the forward transition kernel is p(xt | x0 ) = N (xt ; x0 , t2 I), so the marginal at each time P 1 is pbt (xt ) = |D| x0 ∈D p(xt | x0 ). The exact empirical posterior mean divides the weighted average by this data-dependent normalizer: X 1 b 0 | xt ] = E[x x0 p(xt | x0 ). (37) pbt (xt ) |D| x0 ∈D

Privately estimating and using this normalizer would require accounting for an additional datadependent statistic. Our implemented mechanism instead fixes a public constant (hyperparameter) β and uses X bβ [x0 | xt ] = 1 E x0 p(xt | x0 ). (38) β |D| x0 ∈D

Consequently, β = 1 defines a deliberately unnormalized empirical mechanism; it is not an estimate that the density pbt (xt ) numerically equals one. With public β, this remains a well-defined randomized mechanism with a rescaled empirical drift, and the privacy guarantee applies exactly to the implemented mechanism. Its privacy and utility both depend on the public scaling through r 2C 2∆t µt (β) = , (39) β |D| t3 18

Preprint

Algorithm 1 Output-wise subsampled empirical denoising sampler Require: D with n = |D|, subsample size B, clip norm C, EDM schedule with switch time t0 and restart level t1 > t0 , public denoiser Dpub (data disjoint from D), outputs Nout . Ensure: Nout samples with an fout -DP guarantee on D. 1: qB ← B/n; B ← { t : t0 < t ≤ tmax } ▷ block inclusion prob.; private block 2: for j = 1 to Nout do 3: draw Sj ⊆ D, |Sj | = B, and reuse for all of B; x ← σmax ϵ ▷ subsample once per output 4: for t ∈ B in decreasing order do P 1 5: Ê ← β|S v = x0 pt (x | x0 ) ▷ clipped x0 ∈Sj v̄(x0 ; x), v̄ = v/ max(1, ∥v∥/C), j| direction √
6: x ← 1 − 2∆t 2t∆t ϵ ▷ Gaussian mechanism, µt by eq. 6 x + 2∆t t t Ê + 7: end for 8: x ← x + (σt21 − σt20 )1/2 ϵ; x(j) ← Dpub integrated from t1 to 0 ▷ re-noise; public refinement (post-processing) 9: end for P 10: µ̄B ← ( t∈B µ2t )1/2 ; amplify once by qB and compose over Nout outputs (Prop. 4.1) to obtain fout . out 11: return {x(j) }N j=1 with certificate fout

where larger β reduces both the private drift and µt , while smaller β strengthens both. Table 3 reports β directly (the experimental code uses the inverse, 1/β). The setup is CIFAR-10, the 65% empirical block, batch size 50, three-step SDEdit restart, δ = 10−5 , and Nout = 50,000. Table 3: Effect of the public normalization β on privacy and utility. β

Edgeworth estimate

Certified PLD upper bound

FID

4 2 1 0.5 0.25

0.438 1.218 10.777 256.624 unstable

0.438 1.217 11.381 259.761

132.89 62.81 27.13 53.54 81.43

unstable

This ablation shows that the public normalization is a genuine privacy–utility parameter rather than a privacy-neutral approximation: a weak empirical drift gives strong privacy but poor utility, whereas an overly strong drift worsens the certified privacy value and destabilizes generation.

F

P ER - STEP MAGNITUDE GAP BETWEEN EMPIRICAL AND NEURAL DENOISERS

The relevant diagnostic in the main text is Figure 1 (directional cosine similarity), which we supplement here with the mean-squared error (MSE) between the neural and empirical denoiser outputs along the trajectory. b − D∥2 , while Figure 1 reports the Cosine and MSE measure different quantities: MSE is ∥E b and rN = xt − D, related by ∥E b − D∥2 = cosine between the residual directions rE = xt − E 2 2 ∥rE ∥ + ∥rN ∥ − 2∥rE ∥∥rN ∥ cos(rE , rN ). The cosine can be near one while the magnitudes differ (large MSE), and at low noise both residual norms shrink, making the cosine numerically sensitive. Cosine alone therefore cannot establish closeness; we report MSE alongside it and narrow the proxy claim to directional agreement. Absolute MSE is not comparable across rows: both residual norms shrink as t → 0, so the small values at low noise reflect this scale collapse rather than closer agreement, and the only cross-row claim we make is the directional one. The magnitude gap is largest in the mid-noise window (the 46–55% rows), yet Table 1 shows that the difference surviving to the released sample there stays below the noise-resampling reference: the 19

Preprint

b − D∥2 between empirical and Table 4: Directional cosine similarity (Fig. 1) and per-step MSE ∥E neural denoiser outputs, along the sampling trajectory. Rows are labelled by the denoiser active at that noise level under the hybrid sampler, and switch marks the empirical→public handoff for the 55/65/70% configurations. Rows use the N = 50 EDM grid (percent = step/N ). On the N = 39 grid of the hybrid runs these handoffs fall near t = 1.5, 0.52, and 0.28. MSE is mean ± standard error over 3 seeds. t

% gen.

log SNR

b xt − D) cos(xt − E,

MSE

80.0 10.2 3.29 1.29 0.55 0.27 0.09 0.02 0.002

0% 32% 46% 55% 65% 70% 78% 88% 98%

−10.2 −6.0 −3.8 −1.9 −0.2 +1.3 +3.4 +6.7 +11.0

1.0000 1.0000 0.9965 0.9895 0.9766 0.9568 0.9011 0.7517 0.3901

3.24 ± 0.03 1.84 ± 0.07 14.63 ± 0.20 10.13 ± 0.09 6.40 ± 0.05 4.23 ± 0.04 2.13 ± 0.02 0.62 ± 0.004 0.10 ± 0.0003

denoiser emp. emp. emp. switch 55% switch 65% switch 70% pub. pub. pub.

private block sets the denoising direction, and the subsequent re-noising and public refinement (Sec. 6) repair magnitude at no additional privacy cost. Table 5 gives the absolute endpoint agreement behind Table 1, and Table 6 gives image-space agreement for both the substituted and the noise-resampled paths. Table 5: Endpoint agreement between the all-neural reference and the path with the empirical denoiser substituted in one window (2048 paired trajectories, N = 39). CLIP and DINO are cosine similarities (1− the swap distance of Table 1), with 95% bootstrap CIs in brackets. ∆CLIP is noise-control distance minus swap distance, with 95% paired-bootstrap CIs. Image-space metrics are in Table 6. Window

t range

CLIP

DINO

∆CLIP [95% CI]

high-noise mid-noise low-noise min-noise all steps

19.15–7.47 6.09–1.98 1.54–0.38 0.14–0.018 80–0.002

0.990 [0.989, 0.991] 0.923 [0.921, 0.925] 0.887 [0.886, 0.889] 0.785 [0.783, 0.788] 0.794 [0.792, 0.796]

0.928 [0.922, 0.934] 0.483 [0.474, 0.491] 0.309 [0.303, 0.315] 0.124 [0.119, 0.129] 0.121 [0.117, 0.126]

+0.0821 [+0.0801, +0.0840] +0.0228 [+0.0207, +0.0248] −0.0441 [−0.0459, −0.0422] −0.1906 [−0.1930, −0.1881] –

Table 6: Image-space agreement between the all-neural endpoint and each variant, from the same starting noise (2048 trajectories, N = 39, windows of 6 steps). swap: empirical denoiser substituted inside the window. ref.: neural denoiser kept and only that window’s noise re-drawn. Higher means closer to the all-neural endpoint for all three metrics. Brackets are 95% paired bootstrap CIs. Window (steps) high-noise (9–14)

PSNR (dB)

SSIM

Label agreement

swap 33.19 [32.88, 33.50] 0.981 [0.979, 0.983] 0.867 [0.853, 0.882] ref. 11.77 [11.67, 11.86] 0.634 [0.629, 0.639] 0.412 [0.391, 0.433]

mid-noise (15–20) swap 14.74 [14.63, 14.84] 0.731 [0.726, 0.736] 0.462 [0.440, 0.483] ref. 10.40 [10.36, 10.45] 0.466 [0.463, 0.469] 0.390 [0.368, 0.411] low-noise (21–26) swap 12.23 [12.19, 12.27] 0.412 [0.410, 0.415] 0.255 [0.236, 0.274] ref. 11.98 [11.94, 12.03] 0.415 [0.412, 0.419] 0.603 [0.582, 0.624] min-noise (29–34) swap 11.39 [11.36, 11.42] 0.115 [0.112, 0.118] 0.152 [0.136, 0.167] ref. 22.69 [22.67, 22.72] 0.917 [0.916, 0.918] 0.793 [0.776, 0.811] all steps (0–38)

swap 10.19 [10.14, 10.23] 0.087 [0.085, 0.089] 0.171 [0.155, 0.188]

20

Preprint

G

C OMPARISON BETWEEN PLD ACCOUNTANT AND E DGEWORTH A PPROXIMATION

G.1

ACCOUNTING P ROCEDURE

P Shared steps. (1) Compute µt for every private step and µ̄B = ( t∈B µ2t )1/2 (Table 7 lists µ̄B for each handoff). (2) Form the one-output pair PB = N (0, 1), QB = (1 − qB )N (0, 1) + qB N (µ̄B , 1) and its privacy loss LB (Proposition 4.1), in both directions (QB against PB and PB against QB ). For a pair (A, A′ ) with privacy loss ℓ = log(dA/dA′ ), the hockey-stick divergence is Heε (A∥A′ ) = EA [(1 − eε−ℓ )+ ], so δ(ε) depends on the pair only through the distribution of ℓ under A, its PLD. PLD accountant (certified, reported as εu ). (3) For each direction, discretize the PLD of one output pessimistically on a grid of width h = 10−5 , so that the hockey-stick divergence of the discrete PLD upper-bounds the true one at every ε. Tail mass below e−50 is truncated and added to δ. (4) Compose the Nout independent outputs by convolving the discretized PLD with itself Nout times, computed with the FFT. Any mass truncated during compositionPis also added to δ. The result is a pessimistic PLD of Sout . (5) For each direction evaluate δ(ε) = s ω(s)(1 − eε−s )+ plus the truncated mass, where ω is the composed discrete PLD, take the larger of the two directions, and find the smallest ε with δout (ε) ≤ δ. This is εu . (6) Repeating (3)–(5) with every value rounded down gives εℓ , which is not a bound and only shows how much of εu is discretization slack. We use the implementation of Doroshenko et al. (2022) (Google dp_accounting) to automatically calculate these values given µ̄B and subsampling rate. Edgeworth approximation (estimate). (3′ ) For D ∈ {P, Q}, compute the first four cumulants ′ κD r,B of LB under DB by numerical integration over z. (4 ) Cumulants add over independent D D outputs, so Sout has MD = Nout κD , and K = Nout κD 1,B , VD = Nout κ2,B , K3,D = Nout κ3,B 4,B √
4,D ′ (Appendix D). (5 ) Approximate PrD (Sout > ε) ≈ 1 − FEW (ε − MD )/ VD with eq. 25, insert both probabilities into δout (ε) of Proposition 4.1 for each direction, take the larger, and solve δout (ε) = δ by bisection. (6′ ) Check the diagnostics of Appendix D. The value is an estimate: the truncated series has an error of unknown sign and there is no discretization or truncation control. The CLT column of Table 11 instead replaces the composed trade-off curve by a single Gaussian curve Gµ , with µ from the summed first two moments of the per-output privacy loss (Dong et al., 2022). G.2

C OMPARISON

The degree-two Edgeworth approximation used in the paper is a numerical approximation, not a one-sided certified DP upper bound. Although Propositions 4.1 and C.1 exactly characterize the relevant block/per-step privacy-loss distributions, replacing their CDFs by a truncated Edgeworth expansion introduces an approximation error whose sign is not controlled in our finite heterogeneous setting. To provide a formal guarantee, we additionally composed the mechanism actually implemented— P within a private block, conditional per-step Gaussian privacy accumulates to µ̄B = ( t∈B µ2t )1/2 , subsampling amplification is applied once to the block, and the result is composed over Nout released samples—using PLD (Doroshenko et al., 2022) and RDP (Wang et al., 2019) accountants. All accountants use the sampling model of Section 4 (q = B/n, add/remove adjacency), block parameter µ̄B , and the same output count. RDP is a conservative certified relaxation; the PLD method provides a rigorous upper bound and a lower bound that bracket the optimal privacy value after controlling discretization and truncation error. At δ = 10−5 , Nout = 50,000, B = 25 for CIFAR-10, PLD grid spacing h = 10−5 and log_mass_truncation_bound = −50: The comparison reveals two regimes in these evaluated configurations. For ε ≲ 2, the Edgeworth estimate lies within 1% of the certified PLD upper bound (55% and 62% blocks). For ε ≳ 5, it is anti-conservative: at the 65% and 67% blocks, the Edgeworth estimate falls below even the optimistic PLD lower estimate (5.33 vs. [6.05, 6.29] and 22.98 vs. [25.49, 25.70]), understating the certified value by about 10–15%. 21

Preprint

Table 7: Privacy accounting comparison for different empirical block sizes. Empirical block (%)

µ̄B

Edgeworth estimate εew

PLD lower bound

PLD upper bound

RDP upper bound

55 62 65 67

0.69 1.56 2.09 2.84

0.2987 1.68 5.33 23.0

0.04880 1.44 6.05 25.49

0.2988 1.69 6.29 25.7

0.36 2.63 7.81 48.8

H

D IFFERENTIAL P RIVACY AND G AUSSIAN D IFFERENTIAL P RIVACY

H.1

D IFFERENTIAL P RIVACY

The concept of differential privacy (DP) (Abadi et al., 2016; Dwork et al., 2006) provides a formal guarantee that a model behaves almost identically regardless of whether any particular data sample is included in the training set. In other words, DP bounds the extent to which a single sample can influence the model’s behavior. More precisely, a mechanism M, which may be a training algorithm such as Stochastic Gradient Descent (SGD) or, more generally, any randomized algorithm including r[M(D)=o] diffusion sampling, is said to be ϵ-DP if the privacy loss is bounded by ϵ: log PPr[M(D ′ )=o] ≤ ϵ, where D and D′ differ in the record of a single individual; and M(D) = o denotes that applying the mechanism M on a dataset D ends up converging to the output event o. For instance, if M is a training optimizer such as SGD, then M(D) is the trained model and P r(M(D) = o) depicts the probability the model o being the trained one. Intuitively, if deleting a single data point in D to obtain D′ does not substantially change the probability of obtaining the same output model o, then the training algorithm can be regarded as privacy-preserving1 . Put more simply, if the probability the model converges to the same parameters is relatively equivalent regardless of whether a particular sample is present, then that sample cannot be said to meaningfully affect training. More generally, M is (ε, δ)-DP if for all neighboring D, D′ and every measurable set of outputs O, Pr[M(D) ∈ O] ≤ eε Pr[M(D′ ) ∈ O] + δ.

(40)

  Pr[M(D)=o] The privacy-loss condition Pr log Pr[M(D ′ )=o] ≥ ε ≤ δ implies eq. 40 but is not equivalent to it. A natural building block for designing private (either training or sampling) algorithms is the Gaussian mechanism. This is particularly relevant for our setting, since each step in diffusion sampling injects Gaussian noise. Formally, given a function h : D 7→ Rp , the Gaussian mechanism releases MGauss (D; h) = h(D) + n,

(41)

where n ∼ N (0, σ 2 ∆2h I). Here, ∆h denotes the global sensitivity—the maximum L2 difference between outputs on two neighboring datasets—and σ controls the noise level as a function of (ϵ, δ). By adding noise calibrated to sensitivity, the Gaussian mechanism ensures that the presence or absence of any single sample only has a limited influence on the output. This mechanism underlies differentially private training methods such as DP-SGD (Abadi et al., 2016), and, as we will show, it also directly connects to the diffusion sampling process we analyze in this work. As an interesting fact in differential privacy, it is widely known that subsampling amplifies privacy (Kasiviswanathan et al., 2011). For example, if we flip a coin to decide whether each entry of the training data is in or out to form a smaller dataset, we would expect the mechanism using the smaller dataset to be twice as private as the original mechanism using the full dataset. With a flipping probability of 50%, every individual benefits from perfect privacy if the individual is not included in the smaller dataset. To benefit from the privacy amplification effect, a subsampled Gaussian mechanism (applying the Gaussian mechanism on randomly subsampled data) is also widely used (Abadi et al., 2016; Wang et al., 2019). We will also study the effect of subsampling on the privacy loss incurring during the diffusion sampling process. 1 SGD and Adam are not DP algorithms as the gradient update may be dominated by a huge outlier that significantly affects the model’s parameters. As a result, a model trained with SGD/Adam can potentially reveal information about such outliers, leaving the data vulnerable to extraction attacks.

22

Preprint

H.2

G AUSSIAN D IFFERENTIAL P RIVACY

While the Gaussian mechanism provides a natural foundation for analyzing privacy, the classical (ϵ, δ)-DP framework yields only loose bounds when these mechanisms are composed, often resulting in substantially looser guarantees than the actual privacy loss. To address this, we adopt the more refined f -DP framework (Dong et al., 2022), which characterizes privacy through hypothesis testing and provides an exact trade-off curve (function), rather than the looser worst-case bounds of (ϵ, δ)-DP. A particularly important special case is Gaussian Differential Privacy (GDP), which describes the privacy profile of the Gaussian mechanism exactly using a single parameter µ. For example, in the case of eq. 41 with n ∼ N (0, σ 2 ∆2h I), the Gaussian mechanism corresponds to σ1 -GDP. For a Gaussian mechanism, we can define a trade-off function T using the standard normal CDF Φ given by, Gµ := T (N (0, 1), N (µ, 1)).

Gµ (α) := Φ(Φ−1 (1 − α) − µ).

(42)

This makes GDP especially well aligned with diffusion sampling, where Gaussian perturbations are intrinsic to every denoising step. Moreover, GDP is closed under composition (Corollary 2 of Dong et al. (2022)): the cumulative privacy loss of n mechanisms with parameters µ1 , . . . , µn is given tightly by v u n uX µ=t µ2i . (43) i=1

Thus, GDP not only matches the Gaussian mechanism exactly but also enables precise accounting of privacy loss across many Gaussian steps that constitute diffusion sampling. A µ-GDP mechanism is (ε, δ(ε))-DP for every ε ≥ 0 with the closed form (Dong et al., 2022)     ε µ µ ε ε δ(ε) = Φ − + −e Φ − − , (44) µ 2 µ 2 which we use for all full-batch conversions. For subsampled Gaussian mechanisms, the trade-off function in eq. 42 needs to be modified to incorporate the randomness coming from the subsampling process and the GDP composition in eq. 43 is no longer valid. In this case, one can benefit from using the central limit theorem, which tells us that composing many f -DP mechanisms converges to GDP, as formally stated in Theorem 3.5 of Dong et al. (2022).

I

F ULL - BATCH SAMPLING

In the full-batch regime every record contributes to every denoising step, so the per-step parameters µt compose exactly by eq. 3 and convert to (ε, δ) by eq. 44. We vary the minimum noise level σmin to understand its impact on privacy guarantees. Figure 5 shows how the per-step µ grows as the noise level falls. Lower σmin corresponds to less privacy. Figure 4 shows how the choice of σmin affects the privacy-utility trade-off. As σmin decreases (getting closer to the training data), the GDP analysis gives worse (ε, δ)-DP guarantees. The full-batch empirical denoiser does not give good utility for its privacy cost. At ε ≈ 1 (corresponding to σmin = 5), the generated sample is hard to distinguish from pure noise (Figure 4, right). For a recognizable sample we need σmin = 0.1, which corresponds to ε ≈ 200. Scalability of full-batch sampling. Even at low σmin , full-batch diffusion sampling does not scale well with the number of generated samples. Since each denoising step operates on the full dataset, the per-sample privacy loss is already high, and composing multiple samples results in a rapidly growing total privacy budget under GDP. Subsampling alleviates this through privacy amplification (Section 4). 23

Preprint

denoising direction (decreasing t)

100 10

1.0

10−2

0.8

Type II Error (¯)

10−3 10−4

±

¾min = 0:002 ¾min = 0:01 ¾min = 0:1 ¾min = 0:2 ¾min = 0:5 ¾min = 1 ¾min = 5 indistinguishable

−1

± = 10 ¡5

10−5 10−6

¾min = 0:002 ¾min = 0:01 ¾min = 0:1 ¾min = 0:2 ¾min = 0:5 ¾min = 1 ¾min = 5

10−7 10−8 10−9 10−10 −2 10

10−1

0.6

t = 80

t = 23:771

t = 5:471

t = 0:85

t = 0:067

t = 0:002

t = 80

t = 31:279

t = 10:571

t = 2:926

t = 0:606

t = 0:1

t = 80

t = 36:731

t = 15:297

t = 5:619

t = 1:746

t = 0:5

t = 80

t = 49:328

t = 29:343

t = 16:743

t = 9:097

t=5

¾min = 0:002

¾min = 0:1

0.4 ¾min = 0:5

0.2

¾min = 5

100

101

0.0 0.0

102

0.2

0.4

0.6

0.8

1.0

Type I Error (®)

"

Figure 4: GDP Analysis of Diffusion-Based Sampling (full batch) and Corresponding Diffusion Process Visualization. The noise scheduling constants are σmax = 80, N = 50, β = 1, and ρ = 7. The (ϵ, δ)-DP guarantees account for a single sample. Left. GDP conversion to (ϵ, δ)-DP for varying σmin . Middle. Type I and Type II Error under full-batch GDP with varying σmin . Right. Visualization of Diffusion Process with Varying σmin . Only six timesteps are shown for figure clarity.

-GDP Across Timesteps ( = 1.0)

103

-GDP

102 101 100 10 1 10 2 10

5

0

log SNR

5

10

Figure 5: µ-GDP for each time step with a full batch of data.

I.1

P RIVACY LOSS IN RELATION TO CRITICAL WINDOWS

Figure 6 shows the CLIP-based detection of the critical window used in Section 6.3, and Figure 7 shows (ϵ, δ)-DP traces (left) and Type I and II error profiles (right) under GDP with varying stopping time points in the reverse process.

J

E VALUATION OF OUR H YBRID E MPIRICAL D ENOISER AND N EURAL D ENOISER ON CIFAR-10 AND C ELEBA

The CIFAR-10 experiment uses an ImageNet-pretrained diffusion model for the neural denoiser, and we adopt the ρ-power schedule of EDM (Karras et al., 2022). The scheduler is parameterized by the minimum and maximum noise levels (σmin , σmax ), the noise decay rate ρ, and the number of denoising steps N . We experiment with σmin = 0.002, σmax = 80, ρ = 7, N = 39, and the public normalization β = 1 (Appendix E). We conduct a hyperparameter search over the number of re-noising steps T ∈ {1, 3, 5, 7, 9}, empirical generation proportion prop ∈ {55, 62, 65, 67, 70}%, and subsample batch size B ∈ {25, 50, 100, 200}, giving 95 configurations (the 70%, B = 200 setting is not reported due to overflowing privacy calculations), with results in Table 8. For SDEdit with output-wise subsampling of size B, we run empirical denoising for the first prop% of the generation process, up to t0 . Afterwards, we re-noise by adding Gaussian noise with scale (σt21 − σt20 )1/2 . We continue denoising with the public Imagenet denoiser starting at t1 until the end. We generate 50,000 samples for every run and compute the FID against CIFAR10. 24

Preprint

1.0

t = 0.267, = 1.008, = 4.419 t = 0.067, = 1.539, = 7.273 t = 0.017, = 1.508, = 7.095 indistinguishable

Type II Error

0.8

0.6

t=80.0

t=2.294

t=0.267

t=0.067

t=0.017

0.4

0.2

0.0 0.0

0.2

0.4

0.6

0.8

Type I Error

(a)

1.0

(b)

(c)

Figure 6: Detection of critical window. (a) zero-shot CLIP model detects the probability of an image being generated belonging to the ship class. (b) Privacy costs at three time points. (c) Generated images denoised at varying timesteps. G-DP to ( , )-DP without subsampling with 1.0, min = 0.002

G-DP to ( , )-DP without subsampling with 1.0, min = 0.002

1.0

100

0.8

10 2

10 6

10 8

0

2

4

6

8

0.08 Complete at 0.18 Complete at 0.29 Complete at 0.39 Complete at 0.49 Complete at 0.59 Complete at 0.69 Complete at 0.80 Complete at 0.90 Complete at 1.00 Complete at = 10 5 10 12 14

Type II Error

10 4

10 10

0.08 Complete at 0.18 Complete at 0.29 Complete at 0.39 Complete at 0.49 Complete at 0.59 Complete at 0.69 Complete at 0.80 Complete at 0.90 Complete at 1.00 Complete at indistinguishable

= 50.478 = 27.106 = 13.699 = 6.430 = 2.754 = 1.049 = 0.342 = 0.090 = 0.017 = 0.002 16

0.6

= 50.478 = 27.106 = 13.699 = 6.430 = 2.754 = 1.049 = 0.342 = 0.090 = 0.017 = 0.002

0.4

0.2

0.0 0.0

18

0.2

0.4 0.6 Type I Error

0.8

1.0

Figure 7: (ϵ, δ)-DP traces (Left) and Type I and II error profiles (Right) under GDP with varying stopping time points in the reverse process Experiments were conducted on NVIDIA L40S GPUs with 48GB memory. Since our method primarily performs private sampling rather than full diffusion training, compute requirements were moderate. CIFAR-10 experiments typically required approximately 2.5 GPU-hours per run. For CelebA (Liu et al., 2015) at 64 × 64 (n = 162,770, δ = 10−6 ), we use FFHQ (Karras et al., 2019), trained without access to the private CelebA training split, as the disjoint public source for the neural denoiser, and keep the same EDM scheduler (σmin = 0.002, σmax = 80, ρ = 7, N = 39) and public normalization β = 1. We sweep the same base grid of re-noising steps T ∈ {1, 3, 5, 7, 9}, empirical generation proportion prop ∈ {55, 62, 65, 67, 70}%, and subsample batch size B ∈ {25, 50, 100, 200}, and at T = 3 additionally refine B (to {60, 70} for prop ∈ {62, 65}% and to {80, 90, 120, 140} for prop ∈ {65, 67}%), giving 112 configurations in total, with results reported in Table 9. The pipeline is otherwise identical to the CIFAR-10 setting: we run empirical denoising for the first prop% of generation up to t0 , re-noise by adding Gaussian noise with scale (σt21 − σt20 )1/2 , and complete generation with the public FFHQ denoiser from t1 until the end. We generate 50,000 samples for every run and compute the FID against CelebA. The Edgeworth diagnostic εsym is available only for the refined T = 3 rows and is marked invalid where the reuse-validity flag is false or a cumulant ratio ρ ≥ 1 (e.g., prop = 65%, B = 60, where ρQ 4 = 1.19). Table 10 repeats the B = 100, T = 3 handoff sweep over three generation seeds.

25

Preprint

Table 8: Hyperparameter search for output-wise subsample SDEdit on CIFAR-10. We sweep the number of re-noising steps T ∈ {1, 3, 5, 7, 9}, the proportion of empirical (denoiser) generation p ∈ {55, 62, 65, 67, 70}%, and the subsample batch size B ∈ {25, 50, 100, 200}; 95 configurations in total. FID measures sample quality (lower is better). PLD [εℓ , εu ] is the PLD interval under the sampling model of Section 4 (optimistic lower / pessimistic upper estimate, Google dp_accounting). Only εu is a certified bound. εew is the second degree Edgeworth approximation (lower is better). Noise Steps

Prop. Empirical (%)

Subsample Batch Size

FID ↓

PLD [εℓ , εu ] ↓

εew ↓

1

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100

147.43 143.57 141.23 139.74 78.79 78.23 78.73 80.37 78.53 77.65 78.24 79.77 80.33 80.59 81.94 84.24 74.98 76.42 79.52

[0.049, 0.299] [0.386, 0.636] [1.111, 1.361] [2.702, 2.952] [1.442, 1.691] [3.172, 3.422] [7.009, 7.259] [16.009, 16.259] [6.052, 6.289] [11.136, 11.381] [22.530, 22.780] [49.332, 49.582] [25.486, 25.696] [46.196, 46.438] [90.634, 90.898] [190.673, 190.915] [108.154, 108.444] [194.318, 194.645] [369.696, 370.025]

0.299 0.636 1.360 2.946 1.683 3.513 7.515 16.706 5.331 10.777 22.540 49.625 22.982 44.481 89.733 190.198 104.035 191.700 368.188

3

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100

45.67 46.09 47.69 49.92 29.66 31.51 34.59 38.46 25.00 27.13 31.01 35.12 20.49 23.27 27.48 32.59 16.38 19.46 24.28

[0.049, 0.299] [0.386, 0.636] [1.111, 1.361] [2.702, 2.952] [1.442, 1.691] [3.172, 3.422] [7.009, 7.259] [16.009, 16.259] [6.052, 6.289] [11.136, 11.381] [22.530, 22.780] [49.332, 49.582] [25.486, 25.696] [46.196, 46.438] [90.634, 90.898] [190.673, 190.915] [108.154, 108.444] [194.318, 194.645] [369.696, 370.025]

0.299 0.636 1.360 2.946 1.683 3.513 7.515 16.706 5.331 10.777 22.540 49.625 22.982 44.481 89.733 190.198 104.035 191.700 368.188

5

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100

33.16 34.12 36.19 38.89 29.39 30.94 33.94 37.44 27.35 29.12 32.55 36.30 24.68 26.94 30.70 34.91 22.65 25.21 29.07

[0.049, 0.299] [0.386, 0.636] [1.111, 1.361] [2.702, 2.952] [1.442, 1.691] [3.172, 3.422] [7.009, 7.259] [16.009, 16.259] [6.052, 6.289] [11.136, 11.381] [22.530, 22.780] [49.332, 49.582] [25.486, 25.696] [46.196, 46.438] [90.634, 90.898] [190.673, 190.915] [108.154, 108.444] [194.318, 194.645] [369.696, 370.025]

0.299 0.636 1.360 2.946 1.683 3.513 7.515 16.706 5.331 10.777 22.540 49.625 22.982 44.481 89.733 190.198 104.035 191.700 368.188

Continued on next page

26

Preprint

Table 8 continued Noise Steps

Prop. Empirical (%)

Subsample Batch Size

FID ↓

PLD [εℓ , εu ] ↓

εew ↓

7

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100

30.99 31.97 34.01 36.38 30.54 31.81 34.34 37.34 29.70 31.26 34.02 37.18 28.23 29.86 33.13 36.41 26.32 28.51 31.76

[0.049, 0.299] [0.386, 0.636] [1.111, 1.361] [2.702, 2.952] [1.442, 1.691] [3.172, 3.422] [7.009, 7.259] [16.009, 16.259] [6.052, 6.289] [11.136, 11.381] [22.530, 22.780] [49.332, 49.582] [25.486, 25.696] [46.196, 46.438] [90.634, 90.898] [190.673, 190.915] [108.154, 108.444] [194.318, 194.645] [369.696, 370.025]

0.299 0.636 1.360 2.946 1.683 3.513 7.515 16.706 5.331 10.777 22.540 49.625 22.982 44.481 89.733 190.198 104.035 191.700 368.188

9

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100

30.43 31.41 33.14 35.07 30.71 31.79 33.94 36.49 30.30 31.80 34.00 36.74 29.97 31.31 33.87 36.69 29.25 31.11 33.78

[0.049, 0.299] [0.386, 0.636] [1.111, 1.361] [2.702, 2.952] [1.442, 1.691] [3.172, 3.422] [7.009, 7.259] [16.009, 16.259] [6.052, 6.289] [11.136, 11.381] [22.530, 22.780] [49.332, 49.582] [25.486, 25.696] [46.196, 46.438] [90.634, 90.898] [190.673, 190.915] [108.154, 108.444] [194.318, 194.645] [369.696, 370.025]

0.299 0.636 1.360 2.946 1.683 3.513 7.515 16.706 5.331 10.777 22.540 49.625 22.982 44.481 89.733 190.198 104.035 191.700 368.188

Table 9: Hyperparameter search for output-wise subsample SDEdit on CelebA. The base grid sweeps re-noising steps T ∈ {1, 3, 5, 7, 9}, proportion of empirical (denoiser) generation p ∈ {55, 62, 65, 67, 70}%, and subsample batch size B ∈ {25, 50, 100, 200}; at T = 3 we additionally refine B (to {60, 70} for p ∈ {62, 65}% and to {80, 90, 120, 140} for p ∈ {65, 67}%), giving 112 configurations. FID measures sample quality (lower is better). PLD [εℓ , εu ] is the PLD interval under the sampling model of Section 4 (optimistic lower / pessimistic upper estimate, Google dp_accounting). Only εu is a certified bound. εew is the (diagnostic) Edgeworth symmetric privacy loss; it is marked invalid where the reuse validity flag is false or a cumulant ratio ρ ≥ 1 (diagnostics available only for the refined T = 3 rows). Noise Steps

Prop. Empirical (%)

Subsample Batch Size

FID ↓

PLD [εℓ , εu ] ↓

εew ↓

1

55 55 55 55 62 62 62 62 65

25 50 100 200 25 50 100 200 25

116.96 113.07 109.95 107.90 45.45 45.23 46.24 47.05 37.32

[0.000, 0.099] [0.000, 0.208] [0.185, 0.435] [0.666, 0.916] [0.516, 0.767] [1.179, 1.430] [2.308, 2.558] [4.654, 4.904] [3.623, 3.891]

0.099 0.208 0.435 0.915 0.563 1.150 2.361 4.912 invalid (ρQ 4 =3.32) Continued on next page

27

Preprint

Table 9 continued Prop. Empirical (%)

Subsample Batch Size

FID ↓

PLD [εℓ , εu ] ↓

65 65 65 67 67 67 67 70 70 70 70

50 100 200 25 50 100 200 25 50 100 200

38.20 39.87 41.71 30.23 32.07 34.65 37.48 24.58 27.01 30.48 33.82

[5.249, 5.500] [8.482, 8.737] [15.522, 15.769] [13.360, 13.657] [20.517, 20.746] [34.120, 34.392] [61.908, 62.120] [52.702, 53.019] [83.289, 83.470] [140.112, 140.393] [251.533, 251.653]

invalid (ρQ 4 =1.48) 7.147 14.552 invalid (ρQ 4 =3.46) invalid (ρQ 4 =1.28) 30.085 58.631 invalid (ρP 4 =2.26) invalid (ρP 4 =1.15) 132.911 245.964

3

55 55 55 55 62 62 62 62 62 62 65 65 65 65 65 65 65 65 65 65 67 67 67 67 67 67 67 67 70 70 70 70

25 50 100 200 25 50 60 70 100 200 25 50 60 70 80 90 100 120 140 200 25 50 80 90 100 120 140 200 25 50 100 200

35.16 35.78 37.42 39.30 25.56 27.57 28.36 28.85 30.31 33.01 21.95 24.34 25.01 25.67 26.30 26.81 27.25 27.99 28.72 30.05 17.94 20.39 22.50 23.16 23.67 24.63 25.35 26.93 14.70 17.44 20.97 24.54

[0.000, 0.099] [0.000, 0.208] [0.185, 0.435] [0.666, 0.916] [0.516, 0.767] [1.179, 1.430] [1.415, 1.666] [1.644, 1.893] [2.308, 2.558] [4.654, 4.904] [3.623, 3.891] [5.249, 5.500] [5.886, 6.129] [6.531, 6.767] [7.167, 7.415] [7.818, 8.072] [8.482, 8.737] [9.839, 10.090] [11.228, 11.472] [15.522, 15.769] [13.360, 13.657] [20.517, 20.746] [28.670, 28.950] [31.387, 31.670] [34.120, 34.392] [39.619, 39.854] [45.108, 45.350] [61.908, 62.120] [52.702, 53.019] [83.289, 83.470] [140.112, 140.393] [251.533, 251.653]

0.099 0.208 0.435 0.915 0.563 1.150 1.388 1.628 2.361 4.912 invalid (ρQ 4 =3.32) invalid (ρQ 4 =1.48) invalid (ρQ 4 =1.19) 4.999 5.712 6.428 7.147 8.596 10.060 14.552 invalid (ρQ 4 =3.46) invalid (ρQ 4 =1.28) 24.433 27.260 30.085 35.739 41.411 58.631 invalid (ρP 4 =2.26) invalid (ρP 4 =1.15) 132.911 245.964

5

55 55 55 55 62 62 62 62 65 65 65 65 67 67

25 50 100 200 25 50 100 200 25 50 100 200 25 50

28.81 29.63 31.20 32.72 26.17 28.28 31.21 33.94 24.37 26.86 29.66 32.50 22.17 24.53

[0.000, 0.099] [0.000, 0.208] [0.185, 0.435] [0.666, 0.916] [0.516, 0.767] [1.179, 1.430] [2.308, 2.558] [4.654, 4.904] [3.623, 3.891] [5.249, 5.500] [8.482, 8.737] [15.522, 15.769] [13.360, 13.657] [20.517, 20.746]

0.099 0.208 0.435 0.915 0.563 1.150 2.361 4.912 invalid (ρQ 4 =3.32) invalid (ρQ 4 =1.48) 7.147 14.552 invalid (ρQ 4 =3.46) invalid (ρQ 4 =1.28)

Noise Steps

εew ↓

Continued on next page

28

Preprint

Table 9 continued Prop. Empirical (%)

Subsample Batch Size

FID ↓

PLD [εℓ , εu ] ↓

67 67 70 70 70 70

100 200 25 50 100 200

27.70 30.67 20.05 22.64 25.88 29.24

[34.120, 34.392] [61.908, 62.120] [52.702, 53.019] [83.289, 83.470] [140.112, 140.393] [251.533, 251.653]

30.085 58.631 invalid (ρP 4 =2.26) invalid (ρP 4 =1.15) 132.911 245.964

7

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200

29.25 29.67 30.73 31.74 26.42 28.15 30.47 32.79 25.73 27.96 30.43 33.00 25.02 27.37 30.17 32.95 23.51 25.97 29.19 32.23

[0.000, 0.099] [0.000, 0.208] [0.185, 0.435] [0.666, 0.916] [0.516, 0.767] [1.179, 1.430] [2.308, 2.558] [4.654, 4.904] [3.623, 3.891] [5.249, 5.500] [8.482, 8.737] [15.522, 15.769] [13.360, 13.657] [20.517, 20.746] [34.120, 34.392] [61.908, 62.120] [52.702, 53.019] [83.289, 83.470] [140.112, 140.393] [251.533, 251.653]

0.099 0.208 0.435 0.915 0.563 1.150 2.361 4.912 invalid (ρQ 4 =3.32) invalid (ρQ 4 =1.48) 7.147 14.552 invalid (ρQ 4 =3.46) invalid (ρQ 4 =1.28) 30.085 58.631 invalid (ρP 4 =2.26) invalid (ρP 4 =1.15) 132.911 245.964

9

55 55 55 55 62 62 62 62 65 65 65 65 67 67 67 67 70 70 70 70

25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200 25 50 100 200

31.27 31.25 31.69 32.19 27.50 28.47 29.87 31.47 26.39 27.89 29.73 31.58 26.09 27.78 29.93 32.34 25.30 27.54 30.32 32.90

[0.000, 0.099] [0.000, 0.208] [0.185, 0.435] [0.666, 0.916] [0.516, 0.767] [1.179, 1.430] [2.308, 2.558] [4.654, 4.904] [3.623, 3.891] [5.249, 5.500] [8.482, 8.737] [15.522, 15.769] [13.360, 13.657] [20.517, 20.746] [34.120, 34.392] [61.908, 62.120] [52.702, 53.019] [83.289, 83.470] [140.112, 140.393] [251.533, 251.653]

0.099 0.208 0.435 0.915 0.563 1.150 2.361 4.912 invalid (ρQ 4 =3.32) invalid (ρQ 4 =1.48) 7.147 14.552 invalid (ρQ 4 =3.46) invalid (ρQ 4 =1.28) 30.085 58.631 invalid (ρP 4 =2.26) invalid (ρP 4 =1.15) 132.911 245.964

Noise Steps

εew ↓

Table 10: CelebA-64, B = 100, three public re-noising steps, δ = 10−6 , Nout = 50,000. FID is mean ± standard deviation over three generation seeds. Handoff 55% 62% 65% 67% 70%

Edgeworth estimate

PLD lower

Certified PLD upper

FID

0.43 2.36 7.14 30.09 132.91

0.19 2.31 8.48 34.12 140.11

0.44 2.56 8.74 34.39 140.39

37.39 ± 0.05 30.45 ± 0.17 27.31 ± 0.07 23.75 ± 0.08 20.98 ± 0.05

29

Preprint

Handoff (%)

Edgeworth estimate

CLT estimate

Certified PLD upper

0.299 1.683 5.331 22.982 104.035

0.296 1.388 4.313 45.101 25,905.494

0.299 1.691 6.289 25.696 108.444

55 62 65 67 70

Table 11: Edgeworth and CLT estimates of ε against the certified PLD upper bound (CIFAR-10, B = 25, three re-noising steps, δ = 10−5 ). Neither approximation is a bound. The CLT estimate understates the certified value by 18% and 31% at the 62% and 65% blocks and overstates it by more than 200 times at the 70% block. The Edgeworth estimate stays within 1% for ε ≲ 2 and understates it by 15%, 11%, and 4% at the 65%, 67%, and 70% blocks.

K

S ELECTION OF THE P RIVACY PARAMETERS ε AND δ

For a fixed sampler, Propositions 4.1 and C.1 determine a complete privacy profile δM (ε): a practitioner picks a target δtarget and reports the smallest ε with δM (ε) ≤ δtarget , so δ is not tuned only through ε — the pair describes one fixed mechanism. Arbitrarily small ε and δ cannot coexist at fixed utility; improving both requires changing the mechanism (lower sampling rate, fewer private steps or released outputs, smaller clipping bound, more noise, or an earlier handoff). We used δ = 10−5 ≈ 1/n for CIFAR-10 to match common DP generative-model comparisons; 1/n2 = 4 × 10−10 is indeed stricter, and we report both in Table 12. Table 12: Certified RDP ε for CIFAR-10 output-wise subsampled blocks at δ = 10−5 and at the stricter δ = 4 × 10−10 = 1/n2 (B = 25, Nout = 50,000).

L

Configuration

RDP ε, δ = 10−5

RDP ε, δ = 4 × 10−10

55%, B = 25 62%, B = 25 65%, B = 25

0.36 2.63 7.81

0.71 5.16 12.87

C OMPUTING THEORETICAL ASR

Following Triastcyn & Faltings (2020), we calculate the theoretical ASR as follows. Suppose an adversary observed an ϵ-DP release o and tried to distinguish between two candidate datasets D, D′ with a uniform prior p(D) = p(D′ ) = 0.5. Without loss of generality, let D be the correct p(o|D)p(D) dataset, then the adversary will infer the probability p(D|o) as p(D|o) = p(o|D)p(D)+p(o|D ′ )p(D ′ ) ≤ ε

p(o|D) p(o|D)+exp(−ϵ)p(o|D)

1 e +δ −5 = 1+exp(−ϵ) . For (ε, δ)-DP the bound becomes 1+e ε , which at δ = 10 changes no entry of Table 13 at the reported precision. So, for ϵ = 1 the upperbound on the membership inference success probability is 73.1%, for ϵ = 5, it is 99.33%, and for ϵ = 10, it is 99.995%. Furthermore, the success probability of MIAs typically similar or lower than the theoretical success rate as shown in Lowy et al. (2024).

Table 13: Membership inference. Theoretical: ASR implied by our ε for the subsampled empirical sampler at rate q. Numerical: ASR of the attack of Duan et al. (2023) on a neural denoiser trained on all of CIFAR-10. Both stop at time t of a T = 1000 step discretization. The neural denoiser is not subsampled, so the theoretical rows are not a bound on it. t Cosine similarity (Fig. 1) Theoretical ASR at q = 0.5 Theoretical ASR at q = 0.01 Theoretical ASR at q = 0.005 Numerical ASR

T 1.000 0.5019 0.5000 0.5000 0.5005 ± 0.0025

3 4T 0.999 0.5204 0.5002 0.5001 0.5001 ± 0.0011

30

1 2T 0.956 0.6909 0.5031 0.5012 0.5072 ± 0.0007

1 4T 0.845 0.9999 0.6927 0.6371 0.5096 ± 0.0026

Preprint

M

B ROADER I MPACT S TATEMENT

This work studies privacy leakage that can arise during diffusion-based sampling and provides tools for quantifying how individual training records may influence generated outputs. Its primary positive impact is to support more privacy-aware deployment of generative models, especially in settings where training data may contain sensitive or proprietary information. By shifting attention from only differentially private training to sampling-time privacy accounting, the proposed framework may help practitioners design generative pipelines that better balance privacy and utility, for example through subsampling, early stopping, or hybrid use of public denoisers. This can make formal privacy analysis more accessible for large-scale diffusion systems where fully private training remains costly or impractical. At the same time, the work has limitations and possible risks. The formal guarantees are derived for an empirical denoiser with tractable sensitivity, while modern deployed systems typically use neural denoisers whose exact data sensitivity remains difficult to characterize. As a result, the analysis should not be interpreted as a complete privacy certificate for arbitrary diffusion models. If misused or overstated, sampling-time privacy estimates could give a false sense of security. Moreover, improving the privacy–utility tradeoff of generative models may also make it easier to deploy high-quality synthetic data systems in sensitive domains, where failures could still expose private information or amplify biases present in the underlying data. We therefore view this work as a step toward, rather than a substitute for, end-to-end privacy evaluation. Responsible use should combine sampling-time accounting with empirical privacy audits, membership-inference evaluation, dataset governance, and clear reporting of assumptions. More broadly, our analysis highlights that privacy in generative modeling is shaped not only by training, but also by how samples are produced, released, and reused.

31

Related documents

• Privacy Accounting for Empirical Denoisers#1403506
• Privacy Amplification Without Independence: How Far Negative Dependence Carries the Guarantees of Poisson Subsampling#822359
• Learning Sparse Support under Differential Privacy: Adaptive Algorithms and Minimax Limits#1202286
• Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks#997610
• Noise-Aware Differentially Private Variational Inference#1354774