Unifying Privacy Accounting: Information Equivalence and Information Loss

Unifying Privacy Accounting: Information Equivalence and Information Loss Buxin Su1,*

Qiaoshi Yang1,*

Yiding Su2,*

Chendi Wang3,†

arXiv:2610.02414v1 [cs.CR] 1 Oct 2026

October 5, 2026

Abstract Differential privacy (DP) admits several notions, but the choice among them may affect both privacy analysis and utility. In this paper, we consider four mainstream curve-based privacy notions within a unified information-theoretic framework. For a fixed ordered pair of output distributions, we establish information equivalence among the two directional privacy profiles of (ε, δ)-DP, the pair of hypothesis-testing trade-off functions, and the extended privacy-loss distribution. The exact Rényi differential privacy (RDP) curve joins this equivalence class whenever it is finite at some order greater than one. Under this mild condition, choosing among these notions changes only their semantic interpretation and computational requirements. In contrast, taking the maximum of the directional privacy profiles or compressing the RDP curve into a single zero-concentrated differential privacy (zCDP) parameter can lose information. We quantify the information loss between the exact RDP curve and its zCDP bound for standard noise mechanisms. This gap is zero for Gaussian noise but generally positive for Gaussian-mixture, Laplace, discrete Gaussian, and Poisson-subsampled Gaussian mechanisms. Moreover, this gap grows linearly with the number of independently composed mechanisms. Our informationtheoretic perspective has practical consequences. At the same certified privacy level, retaining the full RDP curve rather than using zCDP reduces the required noise variance by up to 45% for Gaussian-mixture noise in workloads comparable in size to the American Community Survey. For DP-SGD on Fashion-MNIST under Poisson subsampling, an RDP-based privacy accountant improves test accuracy by up to 8.73 percentage points compared to a zCDP-based accountant when both are calibrated to the same (ε, δ) guarantee. These results clarify when different privacy notions have equivalent information (when the gap is zero or nearly zero) and when retaining more detailed privacy information improves utility (when the gap is large).

1

Introduction

Differential privacy (DP) is a rigorous mathematical framework for quantifying changes in the output distributions of a randomized algorithm on two neighboring datasets that differ in at most one element (Dwork, 2006; Dwork and Roth, 2014). For a fixed ordered pair of output distributions induced by a mechanism on neighboring datasets, different privacy notions serve different purposes: Rényi differential privacy (RDP) supports convenient composition through moment accounting and is widely used for DP-SGD (Mironov, 2017; Abadi et al., 2016; Wang et al., 2019); f -DP provides a 1

University of Pennsylvania; Emails: subuxin@sas.upenn.edu and yangqs@sas.upenn.edu. University of Warwick; Email: yiding.su@warwick.ac.uk. 3 Xiamen University; Email: chendi.wang@xmu.edu.cn. * Equal contribution. † Corresponding author. 2

1

Trade-off functions

RDP curve

equivalent

(TP, Q , TQ, P )

equivalent

(Dα (P‖Q))α > 1

Pair of one-sided DP curves

(δP‖Q , δQ‖P )

Information gap from zCDP: Γ

Zero gap: exact bound; larger gap: looser bound

Vary query change Gaussian mixture

Gaussian Γ

No gap

Query change |s|

Vary sampling rate

Γ

Quadratic in |s|

Poisson-subsampled Gaussian

Laplace Γ

Query change |s|

Quartic in |s|

Query change |s|

Smaller q, larger gap

Γ

0+

Sampling probability q

1

Figure 1: Exact privacy curves and the zCDP–RDP gap. Top: three of the four curve-based notions in Theorem 2.1; the RDP curve joins them under a mild condition. Bottom: how the information gap varies with the query change or the sampling probability for four mechanisms. Left three: s is the query change between neighboring datasets. At fixed Rényi order α = 2, the Gaussian gap is zero, while the symmetric Gaussian-mixture and Laplace gaps scale as |s|2 and |s|4 , respectively, as |s| tends to 0. Right: q is the sampling rate of a Poisson-subsampled Gaussian mechanism. At fixed α = 2, smaller q > 0 yields a larger gap; q = 1 corresponds to no subsampling, and the open circle indicates the limit as q → 0. All panels show trends in Γ(t), defined in (3.1). Detailed curves and parameter settings appear in Figure 4. hypothesis-testing trade-off function and a tensor-product composition rule (Dong et al., 2022; Su, 2025; Su et al., 2025); privacy-loss distributions (PLDs) convolve under product composition and support sharp numerical accounting (Koskela et al., 2020; Gopi et al., 2021; Koskela et al., 2021); and zero-concentrated differential privacy (zCDP) compresses the RDP curve into a single parameter ρ, enabling simple scalar composition (Bun and Steinke, 2016). Meanwhile, (ε, δ)-DP remains the standard reporting format, and its optimal composition admits an operational hypothesis-testing interpretation (Kairouz et al., 2017). However, a single (ε, δ) pair gives only one operating point of the full privacy profile curve. Such pairs also do not induce a total ordering: for example, (2, 10−6 ) and (1, 10−5 ) are generally incomparable. Recent work therefore studies mechanism comparison beyond a single calibration point (Balle et al., 2020a; Kifer et al., 2022; Kaissis et al., 2024; Gomez et al., 2025). Yet two mechanisms can share the same two-sided (ε, δ) profile, max(δP ∥Q (ε), δQ∥P (ε)), while having different exact RDP curves (Section 2). Likewise, Gaussian and mixture noise can share the same optimal zCDP parameter ρ but have different exact RDP curves (Section 3). This leads to two basic questions: (1) Under what conditions do the four full privacy notions, with directional information retained, contain the same privacy information about a fixed pair of output distributions? (2) How much information is lost when a full privacy curve is compressed into a single zCDP parameter? Answering these questions helps us decide which privacy notion to use. The first question asks whether we can choose a privacy notion to suit our needs without losing privacy information. Prior work establishes duality between complete (ε, δ)-curve guarantees and symmetric f -DP (Dong et al., 2022), as well as optimal conversions from RDP bounds to (ε, δ)-DP and f -DP (Asoodeh et al., 2

2021; Riess et al., 2026). These conversions are optimal given the RDP bounds, but need not recover the exact privacy curve of a particular distribution pair. In contrast, we ask whether the full exact RDP curve of a fixed ordered pair (P, Q) uniquely determines its directional (ε, δ) profiles, trade-off functions, and PLD. The second question matters in practice: losing privacy information leads to more noise to satisfy the same privacy requirement and reduces accuracy. For example, an f -DP re-accounting of the 2020 U.S. Census (Abowd et al., 2022) showed that the discrete-Gaussian variance proxy σ 2 could be reduced by 15.08%–24.82% while maintaining nearly the same certified privacy guarantees (Su et al., 2025, 2026). In DP-SGD, using the full RDP curve rather than a zCDP summary can also reduce the noise required during privacy calibration, as shown in Figure 2. Our Contributions. We answer the two questions above through the following contributions, summarized in Figure 1. • Information equivalence of curve-based privacy notions. We show that, for a fixed ordered pair (P, Q), the pair of one-sided (ε, δ) profiles, the pair of trade-off functions, and the extended PLD contain the same information, as does the full exact RDP curve under mild conditions. Our counterexamples show that taking the maximum of the one-sided profiles, max(δP ∥Q (ε), δQ∥P (ε)), can lose information. • Information loss of zCDP for common mechanisms. We introduce the zCDP–RDP gap, the distance between the zCDP bound and the exact RDP curve, as a measure of the information discarded when the curve is compressed into a single parameter. The gap adds up under independent composition, and we expand it for small query changes s (Theorem 3.1). It is identically zero for Gaussian noise, of order s2 for Gaussian-mixture noise, and of order s4 for Laplace noise. Under Poisson subsampling with rate q it grows as q decreases and approaches the entire zCDP bound as q → 0: zCDP discards the privacy amplification that subsampling provides. • Practical guidance for privacy accounting. Experiments on private data release and DP-SGD show that information loss can require additional noise and reduce accuracy. Our results support using the full RDP curve or a privacy notion with equivalent information when tighter accounting is needed, and zCDP when the zCDP–RDP gap is negligible. Related Work. Previous work connects the curve-based privacy notions through hypothesis testing and transforms of privacy loss. The statistical view of differential privacy uses hypothesis testing to compare privacy guarantees (Su, 2025); convex duality connects one-sided (ε, δ)-DP curves and trade-off functions (Dong et al., 2022; Balle et al., 2020b). Privacy-loss random variables give an equivalent description of a full privacy curve (Gopi et al., 2021), and their characteristic functions support conversions between one-sided curves, trade-off functions, and PLDs (Zhu et al., 2022). The connection to RDP comes from exponential moments: a finite privacy-loss CGF identifies the PLD (Wang et al., 2019), while Laplace-transform identities relate an extended privacy curve and the RDP curve under absolute-continuity conditions (Chourasia et al., 2025). Theorem 2.1 brings these connections together for arbitrary probability measures, including singular pairs. Finiteness at one real order greater than one lets the full exact RDP curve recover the PLD, the trade-off functions, and both one-sided curves. This recovery uses only real-order RDP values and does not require mutual absolute continuity or complex-order reports. The counterexamples in Section 2 explain why the directional information and the finite-order condition matter. This is a different question from finding the best guarantee consistent with reported RDP 3

zCDP

RDP

Clipping only

(a) Final test accuracy

(b) Final test cross-entropy Final test cross-entropy

Final test accuracy (%)

2.25 70

65

60

2.00 1.75 1.50 1.25 1.00

55 250

500

1000

1500

2000

250

500

Number of compositions n

1000

1500

2000

Number of compositions n

(c) Accuracy trajectory at n = 2000

(d) Cross-entropy trajectory at n = 2000 2.5

60

Test cross-entropy

Test accuracy (%)

70

50 40 30 20

2.0

1.5

1.0

10 0

250

500

750 1000 1250 1500 1750 2000

0

250

500

Training step

750 1000 1250 1500 1750 2000

Training step

Final points and trajectory curves are five-seed means; bars and bands are 95% Student-t confidence intervals. All panels include the clipping-only baseline.

Figure 2: DP-SGD on Fashion-MNIST with noise calibrated under RDP and under zCDP to the same certified privacy (ε, δ) = (3, 10−5 ); “clipping only” adds no noise. (a)–(b): final test accuracy and cross-entropy as the number of training steps n varies at q = 0.01, with the noise recalibrated for each n. (c)–(d): training histories for n = 2000, using the noise calibrated for n = 2000 throughout. Curves are five-seed means; bars and bands are 95% confidence intervals. Setup in Section 5.2. bounds (Asoodeh et al., 2021; Riess et al., 2026), as discussed above: here the object to recover is the exact privacy of a fixed pair. To our knowledge, prior work has not quantified RDP-to-zCDP information loss through a gap at each order or studied its local expansion and accumulation under independent composition.

2

Information Equivalence between Exact Privacy Notions

Fix two neighboring datasets and let P and Q be the output distributions of the mechanism on them. For ε ≥ 0, the one-sided (ε, δ)-DP curve δP ∥Q (ε) := sup{P (A) − eε Q(A)} A

is the smallest δ such that P (A) ≤ eε Q(A) + δ for every event A; exchanging P and Q gives δQ∥P . The usual two-sided curve is δ(ε) = max{δP ∥Q (ε), δQ∥P (ε)}; we keep the two one-sided curves separate. For testing P against Q, the trade-off function TP,Q (a) is the smallest type II error among tests whose type I error is at most a: TP,Q (a) := inf {1 − EQ [ϕ] : EP [ϕ] ≤ a}, ϕ

4

0 ≤ a ≤ 1,

where ϕ ranges over randomized tests with values in [0, 1]. A mechanism is f -DP if TP,Q ≥ f for every ordered neighboring pair (Dong et al., 2022). We write L for the privacy loss, whose distribution under P is the PLD. For densities or probability mass functions p and q, L = log{p(X)/q(X)} with X ∼ P (Zhu et al., 2022); its value is +∞ if an output is possible under P but impossible under Q. Appendix A gives the definition for arbitrary probability measures. We write K(t) := log EP [etL ] for its cumulant-generating function (CGF). Throughout, t = α − 1. At order α = 1 + t > 1 the exact RDP curve is D1+t (P ∥Q) = K(t)/t. These quantities may be infinite; we set K(0) = 0. An (α, εα )-RDP guarantee bounds Dα (P ∥Q) by εα for every ordered neighboring pair (Mironov, 2017). A mechanism is ρe-zCDP if Dα (P ∥Q) ≤ ρeα for every α > 1 and every ordered neighboring pair (Bun and Steinke, 2016). For a fixed pair, the optimal zCDP parameter is Dα (P ∥Q) . α α>1

ρ(P, Q) := sup

Thus zCDP replaces the RDP curve by a single number. We say that a notion A determines a notion B if any two pairs (P, Q) and (P ′ , Q′ ) with the same A also have the same B; two notions that determine each other carry the same information. For instance, the RDP curve determines ρ, but ρ does not determine the RDP curve (Section 3). Theorem 2.1 (Information equivalence between curve-based notions). Let P and Q be arbitrary probability measures on (Ω, F ). The following three curve-based notions determine one another: (a) the pair of one-sided (ε, δ)-DP curves (δP ∥Q , δQ∥P ); (b) the pair of trade-off functions (TP,Q , TQ,P ); (c) the law of the extended privacy-loss random variable L under P .
Each notion determines the exact RDP curve Dα (P ∥Q) α>1 . Conversely, if Dα0 (P ∥Q) < ∞ for some real α0 > 1, the exact RDP curve determines all three curve-based notions above. Remark. The condition Dα0 (P ∥Q) < ∞ holds for every mechanism in Sections 4 and 5. It fails only when EP [etL ] = ∞ for every t > 0, for example when P puts mass outside the support of Q or the privacy loss is very heavy-tailed. Although weak, the condition cannot be dropped, nor weakened to a finite mean privacy loss. Example 2.2 (Finite mean privacy loss is insufficient). Write ζ(w) = P∞ k w k=1 z /k . On Ω = {∂, 1, 2, . . .}, for w > 2, define Pw {k} =

k −w , ζ(w)

P∞

k=1 k

−w and Li (z) = w

Pw {∂} = 0,

Qw {k} = Pw {k}e−k ,

Qw {∂} = 1 −

Liw (e−1 ) . ζ(w)

Then Qw is a probability measure, Pw ≪ Qw , and the privacy loss Lw of (Pw , Qw ) satisfies EPw Lw =

ζ(w − 1) , ζ(w)

δPw ∥Qw (1) =

∞ 1 X 1 − e1−k . ζ(w) k=2 kw

For w = 3 and w = 4, respectively, EP3 L3 = 1.36843 . . . ,

δP3 ∥Q3 (1) = 0.124864 . . . ,

EP4 L4 = 1.11063 . . . ,

δP4 ∥Q4 (1) = 0.053061 . . . .

Both mean privacy losses are finite, and the forward curves δP3 ∥Q3 and δP4 ∥Q4 differ. Nevertheless, Dα (Pw ∥Qw ) = +∞ for every α > 1 and either value of w. Thus finite mean privacy loss is insufficient for recovery from the RDP curve. 5

Both one-sided curves are needed. Example 2.3 adds one of two integer noise distributions to a count of sensitivity 1. The two distributions have full support, likelihood ratios at most 4, the same right tail, and different left tails; call the resulting pairs (PA , QA ) and (PB , QB ). For ε ≥ 0, the forward curve δP ∥Q (ε) involves only outputs where P exceeds Q, here the shared right 2 tail, so both pairs have δP ∥Q (ε) = 11 (4 − eε )+ . The reverse curve involves the left tail and differs: δQA ∥PA (log 3) = 3/22 but δQB ∥PB (log 3) = 0. Because the reverse curve lies below the forward curve for both pairs, the usual curve max{δP ∥Q , δQ∥P } is the same for both, and the two mechanisms would be reported as equally private. Yet their one-sided curves differ, so by Theorem 2.1 their trade-off functions, PLDs, and RDP curves differ too; indeed exp(D2 (PA ∥QA )) = 265/88, whereas exp(D2 (PB ∥QB )) = 3. The equivalence in Theorem 2.1 is therefore with the pair of one-sided curves; their maximum, the usual curve, carries less information. Example 2.3 (The same two-sided curve with different one-sided curves). Let c = 6/11. The two noise distributions described above have mass functions qA (k) = qB (k) = c4−k and, for j ≥ 1,

qA (−j) = c2−(j+1) ,

(k ≥ 0),

qB (−j) = c3−j .

For i ∈ {A, B}, let Qi = Law(Zi ) and Pi = Law(Zi + 1) when Zi has mass function qi . For ε ≥ 0, their one-sided curves are 4c eε 1− , 3 4 +     eε eε c δQA ∥PA (ε) = c 1 − 1− + , 4 + 2 2 +   eε 3c 1− . δQB ∥PB (ε) = 2 3 + δPi ∥Qi (ε) =





The common forward curve dominates both reverse curves for every ε ≥ 0. Hence the two-sided curves, and the strongest symmetric f -DP guarantees obtained from them, are the same for the two pairs. When a single curve suffices. Taking the maximum loses no information when the two one-sided curves coincide. This happens for additive noise symmetric about zero, such as Gaussian d and Laplace noise: if P and Q are the distributions of s + Z and Z with Z = −Z, the reflection x 7→ s − x exchanges P and Q, so δP ∥Q = δQ∥P , TP,Q = TQ,P , and Dα (P ∥Q) = Dα (Q∥P ). The usual two-sided curve then carries all the information (Corollary A.1 in Appendix A.2). Scope of the equivalence. First, Theorem 2.1 concerns information, not computation: converting one notion into another may require the entire curve and need not have a closed form; practical conversion and composition methods are developed in Koskela et al. (2020, 2021); Gopi et al. (2021); Doroshenko et al. (2022); Ghazi et al. (2022). Second, it concerns one fixed pair (P, Q), not a whole mechanism. A mechanism-level RDP guarantee takes, at each order α, the maximum over all neighboring pairs; the maximizing pair may change with α, so the resulting curve need not be the RDP curve of any single pair, and a conversion from it must hold for every pair consistent with it (Zhu et al., 2022; Asoodeh et al., 2021; Balle et al., 2020b). Proofs of the equivalence theorem and the two examples are given in Appendix A. By contrast, replacing the exact RDP curve by the zCDP parameter ρ loses information even for a fixed pair; Section 3 quantifies this loss. 6

3

The Information Gap

By Theorem 2.1, the exact RDP curve, the one-sided curves, the trade-off functions, and the PLD carry the same information. zCDP keeps only ρ, and many curves share the same ρ: for a query change s, Gaussian noise N (0, 1) and the mixture 12 N (−m, 1) + 12 N (m, 1), m > 0, both have ρ = s2 /2 but different RDP curves (Proposition 4.1). Throughout this section and Section 4, we consider mutually absolutely continuous pairs (P, Q) with finite ρ. The parameter ρ gives the bound Dα (P ∥Q) ≤ ρα at every order α > 1; we call the line α 7→ ρα the zCDP envelope. What zCDP discards is how far the exact curve lies below this line, which we measure by the information gap defined as Γ(t) := ρt(t + 1) − K(t) = t {(t + 1)ρ − Dt+1 (P ∥Q)} ,

for any t > 0.

(3.1)

Here, (t + 1)ρ is the zCDP upper bound and Dt+1 (P ∥Q) is the exact RDP value at the same order α = t + 1, so Γ(t) is their difference multiplied by t. The gap has three useful properties (illustrated in Figure 3). (1) Γ(t)/t is the vertical distance at order 1 + t between the envelope and the exact RDP curve, so Γ ≥ 0, with Γ ≡ 0 exactly when the envelope reproduces the curve; the factor t puts Γ on the scale of K, which adds up under independent composition. (2) For Gaussian noise, L ∼ N (ρ, 2ρ) under P , so K(t) = ρt(t + 1) and Γ ≡ 0: the Gaussian RDP curve is its envelope. (3) The gap controls the reported ε after order optimization (Proposition 3.2). (a) RDP curve and zCDP envelope exact RDP Dt + 1 zCDP bound ρ(t + 1)

5

e −Γ(tε ) δzCDP (ε

0.6

4

Γ(t) = 0 for every t

0.5

0

δ

Γ(t0 )/t0

3

δexact (ε)

0.7

Γ(t)

Divergence value

(c) Transfer to (ε, δ)

(b) Gaussian

0.4 0.3

2

0.2 1 0

0.1 0

1

t0 2 t=α−1

3

4

10−2

10−1

100 t=α−1

101

0.0

0.0

0.5

1.0 1.5 2.0 privacy parameter ε

Figure 3: The information gap. Left: Γ(t)/t is the vertical distance between the exact RDP curve and its zCDP envelope; here P = (9, 3, 3, 1)/16 and Q = (1, 3, 3, 9)/16 on four points, ρ = log 3. Right: for Gaussian noise (ρ = 1/2), Γ(t) = 0 for every t > 0.

3.1

Local Expansion

How large is the gap when the noise is large compared with the change one record can cause? For the additive mechanism M(D) = θ(D) + Z, where Z has density or probability mass function p, let s := θ(D) − θ(D′ ) be the query change between neighboring datasets, so |s| is at most the global sensitivity of θ. Subtracting θ(D′ ) from both outputs gives the pair (Ps , Q) with densities ps (x) = p(x − s),

q(x) = p(x).

(3.2)

More generally, we consider any family (Ps , Qs ) with P0 = Q0 , and study small nonzero s of either sign. For each s, let Ls , K, ρ(s), and Γ be the privacy loss, CGF, optimal zCDP parameter, and gap 7

2.5

of (Ps , Qs ), and let κj (Ls ) be the jth cumulant of Ls under Ps (κ1 is the mean, κ2 the variance; κ3 and κ4 measure skewness and tail weight). Since K and the supremum defining ρ rarely have closed forms, the next theorem reduces Γ for small s to the mean and the third and fourth cumulants of the privacy loss. Theorem 3.1 (Local expansion of the zCDP–RDP gap). Fix T > 0. Suppose that, for all sufficiently small s = ̸ 0, Ps and Qs are mutually absolutely continuous, ρ(s) < ∞, and the CGF K(u) = log EPs [euLs ] is five times continuously differentiable on [−1, T ] with fifth derivative satisfying sup−1≤u≤T |K (5) (u)| ≤ MT |s|5 , where MT does not depend on s. Then, for all such s and all 0 < t ≤ T , we have Γ(t) = t(t + 1) ρ(s) − EPs Ls −

|Rs (t)| ≤

t2 (1 − t2 ) t2 (t + 1) κ3 (Ls ) + κ4 (Ls ) + Rs (t), 6 24

MT 2 (t + t5 )|s|5 . 120

(3.3) (3.4)

Remark. The first term measures how far ρ(s) exceeds the mean privacy loss EPs Ls , which is the KL divergence of Ps from Qs ; the other two involve κ3 and κ4 . For Gaussian noise all three vanish, since ρ(s) = EPs Ls and κ3 = κ4 = 0. In general the three terms are of order s2 , s3 , and s4 ; Section 4.1 finds the leading one for common mechanisms.

3.2

From the Gap to Privacy Bounds

We next relate the information gap to the privacy bounds obtained by applying the same RDPto-(ε, δ) conversion to the exact RDP curve and its zCDP envelope. These comparisons keep the output pair, including its noise scale, unchanged and do not require a small-shift assumption. For a fixed pair (P, Q) satisfying the standing assumptions of this section, let ct := tt /(1 + t)1+t . The conversion of Balle et al. (2020b) implies δP ∥Q (ε) ≤ ct exp{K(t) − tε},

t > 0.

(3.5)

Its derivation is given in Appendix B.2. The ε Bound at a Fixed δ.

Fix δ ∈ (0, 1) and define

K(t) + log ct − log δ , t log ct − log δ Γ(t) FZ (t) := ρ(t + 1) + = FR (t) + . t t

FR (t) :=

Minimizing these objectives over t > 0 gives the ε upper bounds obtained from the exact RDP curve and its zCDP envelope, respectively. The following proposition quantifies the difference after order optimization. Proposition 3.2 (Compression loss after order optimization). Suppose that εR := inf FR (t),

εZ := inf FZ (t)

t>0

t>0

are attained at tR , tZ ∈ (0, ∞), respectively. Then we have Γ(tZ ) Γ(tR ) ≤ εZ − εR ≤ . tZ tR

(3.6)

Consequently, if an interval I ⊂ (0, ∞) contains tR and supt∈I Γ(t)/t ≤ η, then 0 ≤ εZ − εR ≤ η. 8

The relevant quantity is therefore Γ(t)/t near the optimizing orders, rather than the gap at an arbitrary reference order. A large value of Γ(tZ )/tZ certifies a nonnegligible compression cost, whereas a uniform upper bound over an interval containing tR certifies that the cost is small. Locating such an interval and bounding the gap without computing the full RDP curve requires mechanism-specific analysis. These bounds compare privacy guarantees at a fixed noise scale. Turning them into bounds on the required noise variance also requires control of how the privacy guarantee changes with the noise scale. Using the Local Expansion. Theorem 3.1 makes the upper bound explicit when the optimizing order lies in the range of the local expansion. Under its assumptions and those of Proposition 3.2, if 0 < tR ≤ T , then 0 ≤ εZ − εR ≤ (tR + 1) ρ(s) − EPs Ls −

tR (tR + 1) κ3 (Ls ) 6

tR (1 − t2R ) MT + κ4 (Ls ) + (tR + t4R )|s|5 . 24 120

(3.7)

This follows by substituting (3.3) and (3.4) into (3.6). It separates the contribution of the mean privacy loss from the third- and fourth-cumulant terms and the remainder. The condition tR ≤ T is needed to apply the remainder bound at the optimizing order. The δ Bound at a Fixed ε. If ε is fixed instead, optimizing over t gives a δ upper bound. For any valid zCDP upper bound ρe ≥ ρ(P, Q), define δzCDP (ε; ρe) := inf ct exp{ρet(t + 1) − tε}. t>0

(3.8)

For ρe > 0, the minimizer tε (ρe) is unique and satisfies log

tε (ρe) + ρe{2tε (ρe) + 1} − ε = 0. 1 + tε (ρe)

(3.9)

When ρ > 0, taking ρe = ρ gives δP ∥Q (ε) ≤ e−Γ(tε ) δzCDP (ε; ρ),

tε = tε (ρ).

(3.10)

Thus evaluating the exact RDP curve at the zCDP optimizing order reduces this δ upper bound by a factor e−Γ(tε ) . Appendix B.2 proves these statements.

3.3

Information Loss under Independent Composition

We next show how the information gap accumulates under independent composition. For component pairs (Pi , Qi ), let Ki be their privacy-loss CGFs, ρi < ∞ their optimal zCDP parameters, and Γi (t) := ρi t(t + 1) − Ki (t) their information gaps. Define K(t) :=

n X

Ki (t),

ρsum :=

i=1

n X i=1

The gap relative to the standard composed zCDP envelope is Γ(t) := ρsum t(t + 1) − K(t). 9

ρi .

Theorem 3.3 (Independent composition). Let (Pi , Qi ), i = 1, . . . , n, be independent component N N pairs with optimal zCDP parameters ρi < ∞. Set P (n) := ni=1 Pi and Q(n) := ni=1 Qi . Then, for every t > 0, it holds that tDt+1 (P (n) ∥Q(n) ) = K(t) and Γ(t) =

n X

Γi (t).

(3.11)

i=1

In particular, for n independent copies of a fixed pair (P, Q) with optimal zCDP parameter ρ, the product pair has optimal zCDP parameter nρ and information gap nΓ(t). Thus, at a fixed order and with the component pair unchanged, the gap grows linearly with the number of independent copies. A component with Pi = Qi contributes zero. Standard versus optimal composition. For nonidentical components, the optimal zCDP parameter of the product pair may be smaller than the sum of the component parameters: K(t) ≤ ρsum . t>0 t(t + 1)

ρ⋆ := sup Consequently, we have

Γ(t) = (ρsum − ρ⋆ ) t(t + 1) + ρ⋆ t(t + 1) − K(t) . 

(3.12)

Both terms are nonnegative. The first records the difference between adding the component parameters and optimizing the parameter of the product pair; it vanishes for identical components. The second is the gap between the product pair’s exact RDP curve and its optimal zCDP envelope. Under the assumptions of Theorem 3.1 for each component family, the local expansions also add componentwise.

4

Information Gaps for Common Mechanisms

4.1

Continuous Additive Noise Mechanisms

We evaluate Γ for three continuous additive noise distributions, Gaussian, Gaussian-mixture, and Laplace, in the setting of (3.2). All three are symmetric about zero, so one direction suffices (Section 2). By Theorem 3.1, the leading power of s in Γ(t) measures the loss: Γ ≡ 0 means none, and the lower the power, the larger the loss for small s. Proposition 4.1 (Gaps for common additive noise). Let (Ps , Q) be as in (3.2) and fix T > 0. The remainders below hold as s → 0, uniformly over 0 < t ≤ T . (a) Gaussian. If Z ∼ N (0, σ 2 ), then ρ(s) = s2 /(2σ 2 ) and Γ(t) = 0 for every s and t > 0. (b) Gaussian mixture. If Z ∼ 21 N (−m, 1) + 12 N (m, 1) with m ≥ 0, then we have ρ(s) = s2 /2 for every s, and m2 Γ(t) = E[sech2 (mZ)] t(t + 1)s2 + O(s4 ). (4.1) 2 The coefficient is positive when m > 0; for m = 0 this is case (a) with σ = 1. (c) Laplace. If Z has density 12 e−|x| , then we have ρ(s) = |s| + e−|s| − 1 for every s (Harrison and Manurangsi, 2026), and [t(t + 1)]2 4 Γ(t) = s + O(|s|5 ). 12 10

(a) Local gaps

(b) One subsampled step

Γ(1)

Γ(1)

10−4

10−8

10−12 0

(c) Independent composition

0.5

10

0.4

8

0.3

6

Γ(1)

Mixture: slope 2.00 Laplace: slope 4.00

0.2

4

0.1

2

Gaussian: Γ(1) = 0

0.0 10−3

10−2

0 0

10−1

0.25

0.5

0.75

1

0

Sampling rate q

Shift size s

0.25

0.5

0.75

1

Sampling rate q

Figure 4: Information gaps at t = 1. (a) Gaussian noise has zero gap; the Gaussian-mixture (m = 1) and Laplace gaps are quadratic and quartic in s (solid: numerical; dashed: leading terms of Proposition 4.1; slopes fitted over s ≤ 0.01). (b)–(c) Single-step and composed gaps of the √ subsampled Gaussian decrease in q (n = 20, ρsum = 5, z = 2; open circles are limits as q ↓ 0). Appendix D.2 repeats these comparisons at t = 1.5. In short, zCDP loses no information for Gaussian noise, loses at order s2 for the Gaussian mixture, and only at order s4 for Laplace noise; Figure 4(a) confirms these powers, with fitted log–log slopes 2.00 and 4.00. Discrete noise is considered in Section 4.5. By (3.11), a workload of d queries has d times the one-query gap; Section 5.1 shows what this costs in noise variance.

4.2

Poisson Subsampling

At every step, DP-SGD clips each per-example gradient to norm at most C, sums them over a Poisson subsample that includes each record independently with probability q, and adds Gaussian noise N (0, z 2 C 2 I) with noise multiplier z (Abadi et al., 2016). Subsampling amplifies privacy. Finite-order RDP captures this amplification, whereas standard zCDP, whose guarantee must hold uniformly over all Rényi orders, does not retain it for the sampled Gaussian mechanism (Zhu and Wang, 2019; Bun et al., 2018). Under add-or-remove neighboring datasets, one step reduces to the sampled Gaussian mechanism (Mironov et al., 2019): Q = N (0, z 2 C 2 I),

Pq = (1 − q)Q + qN (v, z 2 C 2 I),

where v is the clipped gradient of the extra record, with ∥v∥ = C in the worst case. Proposition 4.2 (Poisson subsampling at a fixed zCDP budget). Fix n ≥ 1, C > 0, and ρsum > 0, set zn2 := n/(2ρsum ), and let (Pq , Q) be the one-step pair above with z = zn and ∥v∥ = C. (a) For 0 < q ≤ 1, we have t+1 G 1 K(t) = log EG∼N (0,1) 1 − q + q exp − 2 , zn 2zn ρsum ρsum ρ(Pq , Q) = , and Γ(t) = t(t + 1) − K(t). (4.2) n n For every fixed t > 0, Γ(t) is strictly decreasing in q on (0, 1], equals 0 at q = 1, and tends to ρsum n t(t + 1) as q tends to 0. At t = 1, we have

"

Γ(1) =





n   o 2ρsum − log 1 + e2ρsum /n − 1 q 2 . n

11

#

(b) For n independent copies of this pair, the product pair has optimal zCDP parameter ρsum , and by (3.11) its gap is Γ(t) = ρsum t(t + 1) − K(t) = nΓ(t), which has the same monotonicity in q, vanishes at q = 1, and tends to ρsum t(t + 1) as q tends to 0. At q = 1 there is no subsampling, each step is Gaussian, and zCDP loses no information. As q tends to 0, the exact RDP curve tends to zero while the zCDP parameter stays at ρsum , so the gap approaches its largest possible value ρsum t(t + 1): zCDP charges for privacy loss that does not occur (Figure 4(b)–(c)). This is one source of the utility differences in Section 5.2.

4.3

Allocation under a Fixed zCDP Composition Parameter

Independent composition adds the gaps, but fixing the total zCDP parameter does not fix the total gap. Its value can depend on how that parameter is divided among the releases. We study this dependence for n independent applications of the same one-dimensional additive-noise mechanism. Let Z1 , . . . , Zn be independent copies of a real-valued random variable Z, and, for shift magnitudes ri ≥ 0, set Qi = Law(Zi ) and Pi,ri = Law(Zi + ri ) for i = 1, . . . , n. The noise draws have the same distribution, but the shift magnitudes may differ. Fix ρsum > 0. To express each shift in terms of its zCDP parameter, assume that ρ(0) = 0 and that r 7→ ρ(r) is continuous and strictly increasing on a range whose image contains [0, ρsum ]. Its
inverse on this interval is denoted by r = ψ(x), so ρ ψ(x) = x for 0 ≤ x ≤ ρsum . At r = 0, we use K(t) = Γ(t) = 0. Set ρi = ρ(ri ) and ri = ψ(ρi ). We study allocations satisfying n X

ρi ≥ 0,

ρi = ρsum .

(4.3)

i=1

Each ρi is the optimal zCDP parameter of its component pair. For fixed t, write the information gap as a function of this parameter: Φt (x) := Γ(t)|r=ψ(x) ,

0 ≤ x ≤ ρsum .

Equivalently, Φt (x) = xt(t + 1) − K(t), where K(t) is evaluated for the single pair with r = ψ(x). Substituting ri = ψ(ρi ) into (3.11) gives Γ(t) = ρsum t(t + 1) − K(t) =

n X

Φt (ρi ).

i=1

The same function Φt applies to every component because the noise distribution is fixed. Its convexity determines whether spreading the zCDP parameter evenly increases or decreases the total gap. Proposition 4.3 (Allocation of a fixed total zCDP parameter). Fix t > 0 and consider all allocations satisfying (4.3). (a) If Φt is convex on [0, ρsum ], then every feasible allocation satisfies 

nΦt

ρsum n



≤ Γ(t) ≤ Φt (ρsum ).

12

(4.4)

Hence the uniform zCDP allocation ρi =

ρsum , n

ri = ψ



ρsum , n 

i = 1, . . . , n,

minimizes the gap. Every permutation of the concentrated allocation (ρ1 , . . . , ρn ) = (ρsum , 0, . . . , 0),

(r1 , . . . , rn ) = (ψ(ρsum ), 0, . . . , 0),

maximizes it. (b) If Φt is concave on [0, ρsum ], then Φt (ρsum ) ≤ Γ(t) ≤ nΦt



ρsum , n 

so concentration minimizes the gap and uniform allocation maximizes it. If the relevant function is strictly convex or strictly concave, the uniform extremum is unique, and the concentrated extrema are exactly the n permutations displayed above. For a convex Φt , spreading the zCDP parameter evenly gives the smallest gap; concentrating it in one release gives the largest. Concavity reverses these choices. This comparison concerns one fixed RDP order, and the best allocation need not be the same at every order. Figure 5 compares Laplace and Gaussian-mixture noise as the allocation moves from uniform to concentrated. Both panels use t = 1, n = 8, ρsum = 1/2, and the path ρ1 (ω) =

ρsum {1 + (n − 1)ω}, n

ρi (ω) =

ρsum (1 − ω), n

i = 2, . . . , n,

(4.5)

with ri (ω) = ψ ρi (ω) and 0 ≤ ω ≤ 1. Thus i ρi (ω) = ρsum for every ω, ω = 0 is the uniform zCDP allocation, and ω = 1 is the concentrated allocation, with shifts {ψ(ρsum ), 0, . . . , 0}. For unit-scale Laplace noise, ρ(r) = r + e−r − 1, and the proof verifies that Φ1 is strictly convex. The composed gap therefore increases from 0.037202015 at the uniform allocation to 0.193535993 at the concentrated allocation. √ For the symmetric Gaussian mixture with m = 1 in Proposition 4.1, ρ(r) = r2 /2, so ψ(x) = 2x. At two Gauss–Hermite resolutions, centered second differences of Φ1 , computed from the exact CGF, are negative on the displayed grid in [0, 1/2], providing numerical evidence for concavity. Along this path, the gap decreases from 0.435179700 at the uniform allocation to 0.342190807 at a concentrated allocation. Unlike the Laplace convexity result, this Gaussian-mixture comparison is a numerical observation on the displayed grid.

P

13

Gaussian mixture (m = 1): concave

Unit Laplace: convex ρ1 = ρsum ,

0.200

ρi = 0 (i = 2, …, n)

ρi = ρsum /n (i = 1, …, n)

0.44

0.175

0.42 0.40

0.125

Γ(1)

Γ(1)

0.150

0.100 0.075

0.38 0.36

0.050 0.025

0.34

ρi = ρsum /n (i = 1, …, n)

0.0

0.2

0.4 0.6 0.8 allocation weight ω

1.0

ρ1 = ρsum ,

0.0

0.2

ρi = 0 (i = 2, …, n)

0.4 0.6 0.8 allocation weight ω

1.0

Figure 5: Information gaps under independent composition as a fixed total zCDP parameter is reallocated. Both panels use t = 1, n = 8, ρsum = 1/2, and the path (4.5), from uniform (ω = 0) to concentrated (ω = 1). Left: for unit-scale Laplace noise, strict convexity of Φ1 proves that the uniform allocation is the unique minimum and concentrated allocations are the maxima. Right: for Gaussian-mixture noise with m = 1, the computed gap decreases along the path; centered second differences of Φ1 are negative on the displayed grid at both quadrature resolutions. Numerical settings are in Appendix D.1; Appendix D.2 repeats the comparison at t = 1.5.

4.4

Tail Parameters and the Information Gap

We return to additive noise and ask how the gap changes as the tail parameter varies within a noise family. We compare one ordered location pair at a time, so composition does not affect this comparison. For Student-tν noise, ρν (s) equals the mean privacy loss for sufficiently small shifts. The local expansion therefore gives a quartic gap, as for Laplace noise, but its coefficient need not decrease as the tails become lighter. Proposition 4.4 (Student-tν location pair). Let Γ((ν + 1)/2) x2 fν (x) = √ 1+ νπ Γ(ν/2) ν

!−(ν+1)/2

,

ν ≥ 1,

and let Z have density fν , Ps = Law(Z + s), and Q = Law(Z). Write ρν (s) := ρ(Ps , Q). Then ρν (s) < ∞ for every s ∈ R. For every fixed ν ≥ 1, one has ρν (s) = EPs Ls for all sufficiently small nonzero s. Moreover, for every fixed T > 0, uniformly for 0 < t ≤ T and all sufficiently small nonzero s, Γ(t) = c4,ν [t(t + 1)]2 s4 + O(|s|6 ),

c4,ν :=

3(ν + 1)2 (ν 2 + 4ν − 1) . 4ν(ν + 3)2 (ν + 5)(ν + 7)

(4.6)

The coefficient has a unique maximum at ν = 6.013052641 . . . and satisfies c4,ν = 3/(4ν)+O(ν −2 ) → 0 as ν → ∞. The leading coefficient first increases and then decreases as ν grows, even though the tails become lighter throughout. It reaches its maximum near ν = 6.013 and tends to zero in the Gaussian limit. Thus the tail parameter alone does not give a monotone ordering of the gap. 14

The generalized Gaussian family provides a second comparison: β = 1 gives Laplace noise and β = 2 gives Gaussian noise. Here the local expansion can also contain a contribution from ρβ (s) − EPs Ls , so the quartic term alone need not give the leading gap. Proposition 4.5 (Generalized Gaussian location pair). Let fβ (x) ∝ exp(−|x|β /β) for 1 ≤ β ≤ 2, and let Z have density fβ , Ps = Law(Z + s), and Q = Law(Z). Write ρβ (s) := ρ(Ps , Q) and define χβ :=

o β 4−4/β n 2 3Γ(2 − 1/β) − Γ(4 − 3/β)Γ(1/β) ≥ 0. 24Γ(1/β)2

The optimal pairwise zCDP parameter ρβ (s) is finite for every s ∈ R. For every fixed β ∈ [1, 2] and T > 0, uniformly for 0 < t ≤ T and all sufficiently small nonzero s, Γ(t) = t(t + 1){ρβ (s) − EPs Ls } + χβ [t(t + 1)]2 s4 + o(|s|4 ).

(4.7)

At β = 1, ρ1 (s) = EPs Ls = |s| + e−|s| − 1 and χ1 = 1/12, recovering Laplace noise. At β = 2, ρ2 (s) = EPs Ls = s2 /2 and χ2 = 0, and Γ(t) = 0 identically for this Gaussian endpoint. For 1 < β < 2, we do not have a closed form for ρβ (s) − EPs Ls at a finite shift. Since ρβ (s) ≥ EPs Ls , for every fixed T > 0, uniformly for 0 < t ≤ T and all sufficiently small nonzero s, Γ(t) ≥ χβ [t(t + 1)]2 s4 + o(|s|4 ).

Student-tν : s = 0.2

(4.8)

Generalized Gaussian: s = 0.2

0.00020

exact Γ(1) local-expansion approximation

0.0005

0.00018 0.0004 Γ(1)

Γ(1)

0.00016 0.00014

0.0003 0.0002

0.00012 0.00010

0.0001 exact Γ(1) local-expansion approximation

0.00008 10

0

0.0000 1.00

1

10 degrees of freedom ν

1.25 1.50 1.75 shape parameter β

2.00

Figure 6: Information gaps at t = 1 and s = 0.2 as the tail parameters vary. Left: Student-tν noise, with quartic approximation 4c4,ν s4 from (4.6); the dotted line marks the maximum of c4,ν . Right: generalized Gaussian noise, with ρβ (s) obtained by numerically maximizing the normalized CGF and quartic approximation 4χβ s4 from (4.8). The factor 4 is [t(t + 1)]2 at t = 1. Solid curves show the computed gaps; dashed curves show the small-shift approximations. The Student-tν gap is nonmonotone, while the generalized Gaussian gap decreases to zero at β = 2. Appendix D.2 repeats the comparison at t = 1.5. Figure 6 compares the gaps at t = 1 and s = 0.2. The Student-tν gap is nonmonotone in ν, while the generalized Gaussian gap decreases to zero at β = 2 on the displayed grid. For each generalized Gaussian pair, we obtain ρβ (s) by numerically maximizing K(u)/[u(u + 1)] over u > 0, including 15

the u ↓ 0 endpoint. The difference ρβ (s) − EPs Ls is zero to numerical precision at every displayed β. The plotted quartic term is therefore close to the computed gap, although the full expansion in (4.7) also contains 2{ρβ (s) − EPs Ls } at t = 1. The dashed curves are local approximations and are not asserted to bound the gaps at this fixed shift.

4.5

Discrete and Lattice Mechanisms

For integer-valued queries, the RDP order reveals another difference between noise laws. Continuous Gaussian noise has zero gap at every order; the discrete Gaussian zCDP envelope is exact only at orders where the shifted lattice realigns with Z. We compare it with discrete Laplace noise, whose privacy loss takes finitely many values and hence gives a finite-sum CGF. Proposition 4.6 (Exact gaps for discrete Laplace and discrete Gaussian noise). Let θ be integervalued and fix neighboring datasets D, D′ for which s := θ(D) − θ(D′ ) ̸= 0 and r := |s| ∈ {1, 2, . . .}. Because both noise laws below are centrally symmetric, Corollary A.1 lets us use the positive representative of each ordered location pair. As in the continuous families above, let Lr denote the privacy loss of each displayed pair. The following formulas hold for every t > 0 and extend continuously to t = 0, where the gap is zero. (a) Fix a > 0, let Z have discrete Laplace mass proportional to e−a|z| , put Q = Law(Z), Pr = Law(Z + r), and λ = e−a . Then 

K(t) = log

 1 − λ  etar + λr e−tar

1+λ

1−λ

+

r−1 X j=1

  λj eta(r−2j)  , 

ρ(r) = EPr Lr = a {r + (λ − 1) csch(a)} , r

Γ(t) = a {r + (λr − 1) csch(a)} t(t + 1) − K(t).

(4.9)

The finite sum is empty when r = 1. (b) For discrete Gaussian noise with scale σ > 0 (Canonne et al., 2020), let Z have mass function 2

2

e−k /(2σ ) P(Z = k) = , Θσ (0)

Θσ (u) :=

X

2

2

e−(k−u) /(2σ ) .

k∈Z

Put Q = Law(Z) and Pr = Law(Z + r). Then ρ(r) = EPr Lr = r2 /(2σ 2 ), and r2 Θσ ((1 + t)r) t(t + 1) + log , 2 2σ Θσ (0) Θσ (0) 2 2 2 2 Γ(t) = log = 4e−2π σ sin2 (π(1 + t)r) + O(e−4π σ ), Θσ ((1 + t)r)

K(t) =

(4.10)

uniformly in t as σ → ∞. The gap is 1/r-periodic in t and vanishes exactly when (1 + t)r ∈ Z; it is positive otherwise. The discrete Gaussian gap is periodic in t, with period 1/r. Thus the same zCDP parameter is exact at some RDP orders and loses information at others. The expansion in (4.10) also shows that this loss is exponentially small as σ grows. Figure 7 fixes σ = 0.60 so that the periodic correction and its zeros remain visible, and compares it with the discrete Laplace gaps computed from (4.9).

16

Discrete Laplace at fixed ar = 0.5

Discrete Gaussian: periodic lattice gap

r=1 r=2 r=4

0.4

0.0030 0.0025

0.3 Γ(t)

Γ(t)

0.0020 0.0015 0.0010

0.2

0.1

0.0005 0.0000

0.0 0

1

2

3

0

t=α−1

1

2

3

t=α−1

Figure 7: The exact zCDP–RDP gaps Γ(t) of two lattice mechanisms are compared over the same range of the order offset. The periodic exact zeros for discrete Gaussian noise and the positive discrete Laplace gaps for t > 0 show how the accuracy of zCDP can vary with the RDP order. Left: discrete Gaussian noise uses r = 1 and σ = 0.60, with exact zeros whenever (1 + t)r ∈ Z. Right: discrete Laplace noise uses the labeled shifts r ∈ {1, 2, 4} at fixed ar = 0.5, so a = 0.5/r; the curves are computed from Proposition 4.6. Both panels use 0 ≤ t ≤ 3 but retain separate vertical scales.

5

Experiments

We test whether the predictions of Section 4 persist when both notions are calibrated to the same privacy level (ε0 , δ0 ) = (3, 10−5 ), and whether they affect utility, on an ACS table release and on DP-SGD. Both accountants convert an RDP curve U = (Uα )α>1 to (ε, δ)-DP with Balle et al. (2020b), optimized over the order: Cδ [U ] := inf

α>1



log δ + log α α−1 Uα − + log . α−1 α 

(5.1)

The RDP accountant applies Cδ to the exact curve, and the zCDP accountant to the zCDP envelope. Both apply the same conversion rule, isolating the effect of compression on noise calibration.

5.1

ACS-sized Workload

For each of the 20 states with the most counties, the workload consists of five county-level median earnings by educational attainment from the ACS (U.S. Census Bureau, 2020), giving di = 5mi queries for a state with mi complete counties (330 ≤ di ≤ 1060), so by (3.11) the total gap is di times the one-query gap. This is a workload illustration built on published ACS aggregates, not a Census person-level DP mechanism (Appendix D.3). For each state and noise family (Gaussian, Gaussian-mixture, Laplace), we find the smallest per-query noise variance that certifies (ε0 , δ0 ) under (5.1), Var(ZRDP ) and Var(ZzCDP ). The noise is centered and independent across queries, so the per-query MSE equals this variance; we report ∆MSE := Var(ZzCDP ) − Var(ZRDP ). Result. For Gaussian noise the two calibrations coincide (∆MSE = 0), since the Gaussian RDP curve equals its envelope. For the Gaussian mixture, retaining the RDP curve needs only 17

Gaussian (exact equality)

Gaussian mixture

(a) ΔMSE

Laplace

(b) Var(ZRDP )/Var(ZzCDP ) Var(ZRDP )/Var(ZzCDP )

ΔMSE [($1000)2 /query]

1.0 103

102 Gaussian: ΔMSE = 0 (off log scale)

500

750

0.9 0.8 0.7 0.6

1000

500

Number of scalar queries di

750

1000

Number of scalar queries di

Figure 8: Noise variance required at (ε0 , δ0 ) = (3, 10−5 ) on ACS-sized workloads. Left: ∆MSE against the number of queries di (log scale; Gaussian is exactly 0 and omitted). Right: Var(ZRDP )/ Var(ZzCDP ); smaller means that retaining the RDP curve saves more noise. Release-level distributions are in Figure 9. 0.552–0.557 of the zCDP variance, about 45% lower noise variance, with ∆MSE from 652.3 to 2115.7 in units of ($1000)2 per query. For Laplace noise the ratio is 0.989–0.997 and ∆MSE is 16.170–16.205. This ordering matches the zero, quadratic, and quartic gaps of Proposition 4.1, although the ratios also depend on the expansion coefficients and the calibration. Monte Carlo releases with 20,000 paired draws per state and family agree with the analytic ∆MSE within 0.158%. Results at (ε, δ) = (5, 10−5 ) are in Appendix D.3.2. Gaussian mixture

Laplace

MN 430

IN 455

Paired ΔMSE ̂ [($1000)2 /query]

Gaussian: ΔMSE ̂ = 0 (off log scale)

103

102

101 AL 330

NE 355

WI 355

AR 365

OK 375

MS 390

MI 405

OH 435

KS 440

TN 460

IL 480

NC 480

IA 485

KY 565

MO 570

VA 640

GA 730

TX 1060

State (scalar query count di shown below abbreviation)

[ distributions of the two non-Gaussian families are compared across the Figure 9: The paired ∆MSE 20 ACS workloads. Their separation reproduces the ordering in Figure 8: Gaussian-mixture values exceed Laplace values. The horizontal axis gives states ordered by di ; the logarithmic vertical axis [ in units of ($1000)2 per query. At each state, each family violin contains 20,000 paired gives ∆MSE draws. Gaussian is identically zero and is omitted from the logarithmic axis. The violin widths show the distribution of single-release outcomes, not confidence intervals.

18

zCDP

RDP

Clipping only

2.0

Final test cross-entropy

Final test accuracy (%)

(a) Final test accuracy

65

60

55

50

(b) Final test cross-entropy

1.8 1.6 1.4 1.2 1.0 0.8

0.5%

0.75% 1%

2%

5%

0.5%

0.75% 1%

Poisson sampling rate q

(c) Accuracy trajectory at q = 0.5%

5%

(d) Cross-entropy trajectory at q = 0.5% 2.25

Test cross-entropy

60

Test accuracy (%)

2%

Poisson sampling rate q

50 40 30 20

2.00 1.75 1.50 1.25 1.00

10 0

100

200

300

400

500

0

Training step

100

200

300

400

500

Training step

Final points and trajectory curves are five-seed means; bars and bands are 95% Student-t confidence intervals. All panels include the clipping-only baseline.

Figure 10: The sampling-rate sweep at n = 500 and (ε, δ) = (3, 10−5 ). (a)–(b): final test accuracy and cross-entropy of the RDP-calibrated, zCDP-calibrated, and clipping-only runs as q varies, with the noise recalibrated for each q. (c)–(d): the corresponding training histories at (q, n) = (0.005, 500). Curves are five-seed means; bars and bands are 95% confidence intervals.

5.2

DP-SGD

DP-SGD is where zCDP’s simple bookkeeping is most attractive and, by Proposition 4.2, where it loses the most. We train a two-convolutional-layer CNN on Fashion-MNIST (Xiao et al., 2017) in Opacus (Yousefpour et al., 2021), with Poisson sampling, per-example clipping at C = 1, and the sampled-Gaussian RDP bound (Abadi et al., 2016; Mironov et al., 2019). For each sampling rate q and number of steps n, we compute the smallest noise multiplier that certifies (ε0 , δ0 ) under each notion and train with it; a third, “clipping-only” run adds no noise. One sweep fixes n = 500 and varies q ∈ {0.005, 0.0075, 0.01, 0.02, 0.05}; the other fixes q = 0.01 and varies n ∈ {250, 500, 1000, 1500, 2000}. The privacy guarantees rest on per-step RDP bounds with adaptive composition; Proposition 4.2 only explains the trend. Each setting uses five seeds; within a seed, all runs share initialization and minibatches, and the RDP and zCDP runs share noise draws, so we report Student-t intervals over paired differences. Optimizer settings are in Appendix D.4. Sampling Rate. With RDP calibration, test accuracy stays near 65.2% and cross-entropy near 0.95 across the q sweep (Figure 10(a)–(b)), close to the clipping-only run. With zCDP calibration, accuracy rises from 56.408% at q = 0.005 to about 66% for q ≥ 0.02 and cross-entropy falls from 1.673 to 0.943; for q ≥ 0.02 the three methods agree within seed-to-seed variability. At q = 0.005, RDP calibration reaches 65.138%, a paired gain of 8.73 ± 6.10 percentage points. The zCDP penalty shrinks as q grows, as Proposition 4.2 predicts. The seedwise log-q trend is significant for test loss 19

(p = 7.06 × 10−4 ) and accuracy (p = 0.0108). Since changing q at fixed n also changes the batch size and the number of epochs (nq from 2.5 to 25), this trend is consistent with, but not a direct consequence of, Proposition 4.2. Composition Count. In Figure 2(a)–(b), RDP accuracy rises from 61.866% at n = 250 to 70.522% at n = 2000; across all nine settings of both sweeps, RDP accuracy is within 0.08 percentage points of the clipping-only run. zCDP accuracy peaks at 66.354% at n = 1000 and falls to 64.600% at n = 2000, while its cross-entropy falls from 1.120 to 1.020 at n = 500 and then rises to 2.031. The paired test-loss difference grows with n (seedwise log-n trend p = 2.9 × 10−5 ). The paired accuracy difference is not monotone, and neither its trend (p = 0.161) nor the endpoint contrast (p = 0.146) is significant: each horizon is recalibrated, and optimization turns noise into accuracy nonlinearly, so cross-entropy is the cleaner measure here (training histories in Figure 2(c)–(d)). Full results and sweeps at (ε, δ) = (5, 10−5 ) are in Appendix D.4.

6

Discussion and Limitations

Our results give a simple rule for choosing a privacy notion. The two one-sided (ε, δ)-DP curves, the trade-off functions, the PLD, and, when it is finite at one order, the exact RDP curve carry the same information about a pair (Theorem 2.1); choosing among them affects interpretation and computation, not information. The usual two-sided (ε, δ) curve discards directional information, whereas zCDP is lossy even for a fixed pair. Its loss, measured by Γ, is zero for Gaussian noise and grows for mixture-type noise and under Poisson subsampling at small q (Figure 1). zCDP is therefore lossless for Gaussian noise without subsampling; otherwise, retaining one of the equivalent curves can reduce the noise needed for the same certified privacy. For future work, a natural next step is to extend the analysis to mechanism-level guarantees and adaptive composition. Theorem 2.1 concerns one fixed pair: a mechanism-level RDP bound takes the worst pair at each order and need not be the curve of any single pair, so the theorem does not apply to it directly. The composition results assume independent components and add their zCDP parameters, while the local expansions hold for small shifts and bounded orders. The gap isolates the RDP-to-zCDP step; studying the additional loss in conversion to (ε, δ)-DP would help explain why the final bound can be loose even when Γ = 0. The experiments cover three noise families, ACS-sized workloads, and one Fashion-MNIST model with five seeds. Broader experiments could examine whether the observed utility patterns persist across other datasets, optimizers, and deployments. Another direction is rigorous numerical evaluation of ρ when the maximizing order has no closed form.

References M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016. J. M. Abowd, R. Ashmead, R. Cumings-Menon, S. Garfinkel, M. Heineck, C. Heiss, R. Johns, D. Kifer, P. Leclerc, A. Machanavajjhala, B. Moran, W. Sexton, M. Spence, and P. Zhuravlev. The 2020 Census Disclosure Avoidance System TopDown Algorithm. Harvard Data Science Review, 2022. doi: 10.1162/99608f92.529e3cb9. Special Issue 2.

20

S. Asoodeh, J. Liao, F. P. Calmon, O. Kosut, and L. Sankar. Three variants of differential privacy: Lossless conversion and applications. IEEE Journal on Selected Areas in Information Theory, 2 (1):208–222, 2021. doi: 10.1109/JSAIT.2021.3054692. B. Balle, G. Barthe, and M. Gaboardi. Privacy profiles and amplification by subsampling. Journal of Privacy and Confidentiality, 10(1), 2020a. doi: 10.29012/jpc.726. B. Balle, G. Barthe, M. Gaboardi, J. Hsu, and T. Sato. Hypothesis testing interpretations and Rényi differential privacy. In S. Chiappa and R. Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 2496–2506. PMLR, 2020b. M. Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography: 14th International Conference, TCC 2016-B, Part I, volume 9985 of Lecture Notes in Computer Science, pages 635–658. Springer, 2016. doi: 10.1007/ 978-3-662-53641-4_24. M. Bun, C. Dwork, G. N. Rothblum, and T. Steinke. Composable and versatile privacy via truncated CDP. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 74–86. Association for Computing Machinery, 2018. doi: 10.1145/3188745.3188946. C. L. Canonne, G. Kamath, and T. Steinke. The discrete Gaussian for differential privacy. In Advances in Neural Information Processing Systems, volume 33, pages 15676–15688. Curran Associates, Inc., 2020. R. Chourasia, U. Javaid, and B. Sikdar. Laplace transform interpretation of differential privacy. In 6th Symposium on Foundations of Responsible Computing (FORC 2025), volume 329 of Leibniz International Proceedings in Informatics (LIPIcs), pages 11:1–11:23. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2025. doi: 10.4230/LIPIcs.FORC.2025.11. Extended version: arXiv:2411.09142. J. Dong, A. Roth, and W. J. Su. Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(1):3–37, 2022. doi: 10.1111/rssb.12454. V. Doroshenko, B. Ghazi, P. Kamath, R. Kumar, and P. Manurangsi. Connect the dots: Tighter discrete approximations of privacy loss distributions. Proceedings on Privacy Enhancing Technologies, 2022(4):552–570, 2022. doi: 10.56553/popets-2022-0122. C. Dwork. Differential privacy. In Automata, Languages and Programming, 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II, volume 4052 of Lecture Notes in Computer Science, pages 1–12. Springer, 2006. doi: 10.1007/11787006_1. C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4):211–407, 2014. doi: 10.1561/0400000042. B. Ghazi, P. Kamath, R. Kumar, and P. Manurangsi. Faster privacy accounting via evolving discretization. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 7470–7483. PMLR, 2022. J. F. Gomez, B. Kulynych, G. Kaissis, F. P. Calmon, J. Hayes, B. Balle, and A. Honkela. Gaussian dp for reporting differential privacy guarantees in machine learning. arXiv preprint arXiv:2503.10945, 2025. 21

S. Gopi, Y. T. Lee, and L. Wutschitz. Numerical composition of differential privacy. In Advances in Neural Information Processing Systems, volume 34, pages 11631–11642. Curran Associates, Inc., 2021. C. Harrison and P. Manurangsi. Exact zCDP characterizations for fundamental differentially private mechanisms. In 7th Symposium on Foundations of Responsible Computing (FORC 2026), volume 368 of Leibniz International Proceedings in Informatics (LIPIcs), pages 3:1–3:18. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2026. doi: 10.4230/LIPIcs.FORC.2026.3. P. Kairouz, S. Oh, and P. Viswanath. The composition theorem for differential privacy. IEEE Trans. Inf. Theory, 63(6):4037–4049, 2017. G. Kaissis, S. Kolek, B. Balle, J. Hayes, and D. Rueckert. Beyond the calibration point: Mechanism comparison in differential privacy. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, volume 235 of Proceedings of Machine Learning Research, pages 22840–22860. PMLR / OpenReview.net, 2024. D. Kifer, J. M. Abowd, R. Ashmead, R. Cumings-Menon, P. Leclerc, A. Machanavajjhala, W. Sexton, and P. Zhuravlev. Bayesian and frequentist semantics for common variations of differential privacy: Applications to the 2020 Census. arXiv preprint arXiv:2209.03310, 2022. A. Koskela, J. Jälkö, and A. Honkela. Computing tight differential privacy guarantees using FFT. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 2560–2569. PMLR, 2020. A. Koskela, J. Jälkö, L. Prediger, and A. Honkela. Tight differential privacy for discrete-valued mechanisms and for the subsampled gaussian mechanism using FFT. In A. Banerjee and K. Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, volume 130 of Proceedings of Machine Learning Research, pages 3358–3366. PMLR, 2021. URL http://proceedings.mlr.press/v130/koskela21a.html. I. Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263–275. IEEE, 2017. doi: 10.1109/CSF.2017.11. I. Mironov, K. Talwar, and L. Zhang. Rényi differential privacy of the sampled Gaussian mechanism. arXiv preprint arXiv:1908.10530, 2019. doi: 10.48550/arXiv.1908.10530. A. Riess, J. F. Gomez, F. du Pin Calmon, J. A. Schnabel, and G. Kaissis. Optimal conversion from Rényi differential privacy to f -differential privacy. In Proceedings of the 43rd International Conference on Machine Learning, 2026. URL https://arxiv.org/abs/2602.04562. Accepted at ICML 2026; arXiv:2602.04562. B. Su, W. J. Su, and C. Wang. The 2020 US Decennial Census is more private than you (might) think. Proceedings of the National Academy of Sciences, 122(45):e2500337122, 2025. doi: 10.1073/ pnas.2500337122. B. Su, W. Su, and C. Wang. A sieve-accelerated quadrature method for exact privacy accounting in the 2020 US Decennial Census. arXiv preprint arXiv:2606.29835, 2026. W. J. Su. A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation, and Blackwell’s theorem. Annual Review of Statistics and Its Application, 12(1):157–175, 2025.

22

U.S. Census Bureau. S1501: Educational Attainment (American Community Survey 5-Year, 2020). https://data.census.gov/table/ACSST5Y2020.S1501, 2020. Table ID: S1501; Version: 2016–2020 5-year estimates. Y.-X. Wang, B. Balle, and S. P. Kasiviswanathan. Subsampled Rényi differential privacy and analytical moments accountant. In K. Chaudhuri and M. Sugiyama, editors, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 1226–1235. PMLR, 2019. H. Xiao, K. Rasul, and R. Vollgraf. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017. doi: 10.48550/arXiv.1708.07747. A. Yousefpour, I. Shilov, A. Sablayrolles, D. Testuggine, K. Prasad, M. Malek, J. Nguyen, S. Ghosh, A. Bharadwaj, J. Zhao, G. Cormode, and I. Mironov. Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109.12298, 2021. doi: 10.48550/arXiv.2109.12298. Y. Zhu and Y.-X. Wang. Poission subsampled Rényi differential privacy. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 7634–7642. PMLR, 2019. Y. Zhu, J. Dong, and Y.-X. Wang. Optimal accounting of differential privacy via characteristic function. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 4782–4817. PMLR, 2022.

23

Appendix Contents A Proofs for Section 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 Proof of Theorem 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Proof of Corollary A.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Proof of Example 2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Proof of Example 2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25 25 26 26 26

B Proofs for Section 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1 Proof of Lemma B.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 Proof of the conversion bounds in Section 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Proof of Proposition 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 Proof of Lemma B.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5 Proof of Theorem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6 Proof of Theorem 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27 27 27 28 28 29 29

C Proofs for Section 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.1 Proof of Lemma C.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 Proof of (C.6) and (C.7) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.3 Proof of Proposition 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4 Proof of Proposition 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5 Proof of Proposition 4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.6 Proof of Proposition 4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.7 Proof of Proposition 4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.8 Proof of Proposition 4.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29 29 31 31 33 34 34 36 38

D Supplementary Figures and Numerical Details . . . . . . . . . . . . . . . . . . . . . . . . D.1 Computational setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2 Results at an alternative reporting order . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.3 ACS workloads and additional results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.4 DP-SGD settings and additional results . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39 39 39 42 44

24

A

Proofs for Section 2

A.1

Proof of Theorem 2.1

Proof of Theorem 2.1. For arbitrary P and Q, take µ = P + Q and define ℓ(x) = log

dP/dµ(x) . dQ/dµ(x)

Set ℓ = +∞ where only dP/dµ is positive and ℓ = −∞ where only dQ/dµ is positive. For X ∼ P , the privacy loss L = ℓ(X) has no mass at −∞; its mass at +∞ is the mass of the part of P singular to Q. We use e−∞ = 0. For γ ≥ 1, write Hγ (P ∥Q) = supA {P (A) − γQ(A)}, so that δP ∥Q (ε) = Heε (P ∥Q) for ε ≥ 0. Let EP,Q be the closed convex testing-error region consisting of pairs EP [ϕ], 1 − EQ [ϕ] ,

0 ≤ ϕ ≤ 1.

The trade-off function TP,Q is the lower boundary of EP,Q . Since (a, b) 7→ (b, a) maps EP,Q onto EQ,P , TQ,P is the generalized inverse of TP,Q , so TP,Q determines the pair (TP,Q , TQ,P ). For γ ≥ 1, the two hockey-stick divergences are equivalently the support values Hγ (P ∥Q) = 1 −

inf

(a,b)∈EP,Q

(a + γb),

Hγ (Q∥P ) = 1 −

inf

(a,b)∈EP,Q

(γa + b).

As γ ranges over [1, ∞), the two families cover all positive supporting slopes. Convex duality therefore shows that the pair of one-sided (ε, δ)-DP curves and the pair of trade-off functions determine one another. We next connect the two one-sided curves to privacy loss. The hockey-stick identity gives, for x ≥ 0, h i h i δP ∥Q (x) = EP (1 − ex−L )+ , δP′ ∥Q,+ (x) = −ex EP e−L 1{L>x} . Consequently, P {L > x} = δP ∥Q (x) − δP′ ∥Q,+ (x),

′ P {L < −x} = −e−x δQ∥P,+ (x).

These strict-tail identities, together with normalization and P {L = +∞} = limx→∞ δP ∥Q (x), recover the law of L under P . Conversely, that law determines both one-sided curves via δP ∥Q (ε) = EP [(1 − eε−L )+ ],

δQ∥P (ε) = 1 − EP [e−L ] + EP [(e−L − eε )+ ].

The identity D1+t (P ∥Q) = K(t)/t then shows that the privacy-loss law determines the exact RDP curve. For the converse, suppose Dα0 (P ∥Q) < ∞ for some α0 > 1, and put c := α0 − 1. Since K is convex with K(0) = 0 and K(c) < ∞, K is finite on [0, c], and the RDP curve determines K(t) = tD1+t (P ∥Q) for 0 < t < c. In particular P {L = +∞} = 0, so L is real-valued P -a.s. Fix t0 ∈ (0, c) and exponentially tilt the law of L by the density et0 L−K(t0 ) . The tilted law has moment-generating function E[euLt0 ] = exp{K(t0 + u) − K(t0 )},

−t0 < u < c − t0 .

This function is known from RDP and is finite on a neighborhood of zero, so moment-generatingfunction uniqueness determines the tilted law. Untilting by h

P {L ∈ B} = eK(t0 ) E e−t0 Lt0 1{Lt0 ∈B}

i

recovers the original privacy-loss law. The preceding formulas recover both one-sided curves, and convex duality for EP,Q then recovers the pair of trade-off functions. 25

A.2

Proof of Corollary A.1

Corollary A.1 (Centrally symmetric additive noise). Let θ be a real-valued query and let M(D) = θ(D) + Z. Fix ordered neighboring datasets D, D′ , put s = θ(D) − θ(D′ ) ∈ R, and translate both d

outputs by −θ(D′ ). Suppose Z = −Z, and set P = Law(s + Z), Q = Law(Z). The reflection Rs (x) = s − x exchanges P and Q, so δP ∥Q (ε) = δQ∥P (ε),

Dα (P ∥Q) = Dα (Q∥P ),

TP,Q = TQ,P .

The common one-sided (ε, δ)-DP curve, the exact f -DP curve, and the law of L determine one another and determine every exact RDP value through D1+t (P ∥Q) = K(t)/t. Conversely, if Dα0 (P ∥Q) < ∞ for some real α0 > 1, the exact RDP curve recovers the other three notions. Proof of Corollary A.1. Set Rs (x) = s − x. If X = s + Z ∼ P , then Rs (X) = −Z ∼ Q by central symmetry. If X = Z ∼ Q, then Rs (X) = s − Z ∼ P . Thus Rs exchanges P and Q, Rs ◦ Rs is the identity, and Rs preserves µ = P + Q. By uniqueness of Radon–Nikodym derivatives, versions may be chosen so that (dP/dµ) ◦ Rs = dQ/dµ and (dQ/dµ) ◦ Rs = dP/dµ µ-almost everywhere. Consequently, ℓ ◦ Rs = −ℓ, including when ℓ takes an extended value. Transforming measurable sets by Rs now gives δP ∥Q = δQ∥P . Invariance of Rényi divergence under the bimeasurable transformation Rs similarly gives Dα (P ∥Q) = Dα (Q∥P ). If a randomized test ϕ of P against Q has type I and type II errors (a, b), then ϕ ◦ Rs , viewed as a test of Q against P , also has error pair (a, b). Applying the reflection argument with (Q, P ) in place of (P, Q) gives TP,Q = TQ,P . Because the two one-sided (ε, δ)-DP curves coincide, the common curve supplies both inputs required by Theorem 2.1. Theorem 2.1 gives the unconditional forward recoveries and, under the finite-order condition stated in Corollary A.1, the converse recovery from the RDP curve.

A.3

Proof of Example 2.2

Proof of Example 2.2. The masses of Qw sum to one by the definition of Liw (e−1 ). At each k ≥ 1, the privacy loss is Lw = k, which gives the mean and the hockey-stick divergence stated in the example. For every t > 0, ∞ tk h i e 1 X = +∞. EX∼Pw etLw = ζ(w) k=1 k w Hence the RDP curve is everywhere infinite for every w > 2, in particular for w = 3 and w = 4, whereas the forward curves differ because δP3 ∥Q3 (1) ̸= δP4 ∥Q4 (1).

A.4

Proof of Example 2.3

Proof of Example 2.3. The masses on the nonnegative and negative support points sum to 4c/3 = 8/11 and c/2 = 3/11, respectively. Thus both are full-support probability mass functions. If Wi (k) = Pi {k}/Qi {k}, direct division gives k ≥ 1 k = 0 k ≤ −1 WA (k) 4 1/4 1/2 WB (k) 4 1/3 1/3. For γ = eε ≥ 1, recall that Hγ (Pi ∥Qi ) = δPi ∥Qi (ε). Since Hγ (Pi ∥Qi ) = k qi (k)(Wi (k) − γ)+ , only k ≥ 1 contributes to the forward divergence when γ ≥ 1. This gives the common formula in the P example. Similarly, Hγ (Qi ∥Pi ) = k qi (k)(1 − γWi (k))+ gives the two reverse formulas. P

26

Writing F for the common forward one-sided (ε, δ)-DP curve and RA , RB for the reverse one-sided (ε, δ)-DP curves, direct subtraction gives c(γ − 1)/6, c(4 − γ)/12,

1 ≤ γ ≤ 2, 2 ≤ γ ≤ 4,

c(γ − 1)/6, c(4 − γ)/3,

1 ≤ γ ≤ 3, 3 ≤ γ ≤ 4.

(

F − RA = and

(

F − RB =

All curves vanish after their stated cutoffs, so the pointwise maxima agree. The strongest symmetric f -DP guarantee depends on the pair only through this maximum (Dong et al., 2022, Propositions 5– 6), so it also agrees. Finally, exp(D2 (Pi ∥Qi )) =

X

qi (k)Wi (k)2 .

k

Substituting the table and the three region masses gives 265/88 for pair A and 3 for pair B. The table also shows that both Wi and Wi−1 are bounded by 4.

B

Proofs for Section 3

B.1

Proof of Lemma B.1

Lemma B.1 (Hockey-stick Chernoff bound). For x ∈ R and t > 0, (1 − ex )+ ≤

tt e−tx . (1 + t)1+t

Proof of Lemma B.1. If x ≥ 0, the left side is zero. Now suppose x < 0 and set y = ex ∈ (0, 1). The function y t (1 − y) is maximized at y = t/(1 + t), where y t (1 − y) equals tt /(1 + t)1+t . Therefore (1 − ex )etx ≤

tt , (1 + t)1+t

which is the stated inequality.

B.2

Proof of the conversion bounds in Section 3.2

We prove the conversion bounds stated in Section 3.2. Applying Lemma B.1 with x = ε − L and taking expectation under P gives δP ∥Q (ε) ≤

tt exp{K(t) − tε}. (1 + t)1+t

Since K(t) = tDt+1 (P ∥Q), an order-1 + t bound Dt+1 (P ∥Q) ≤ D̄ therefore implies δP ∥Q (ε) ≤

tt exp{t(D̄ − ε)}. (1 + t)1+t

This is the refined fixed-order conversion of Balle et al. (2020b); dropping the prefactor recovers the original conversion of Mironov (2017). If D̄ = ρe(1 + t), optimization over t > 0 gives (3.8). 27

Let b(t) denote the logarithm of the objective in (3.8). Its first two derivatives are b′ (t) = log

t + ρe(2t + 1) − ε, 1+t

b′′ (t) =

1 + 2ρe > 0. t(1 + t)

For ρe > 0, the first derivative tends to −∞ as t ↓ 0 and to +∞ as t → ∞. Hence the minimizer is unique and satisfies (3.9). Finally, when ρ > 0, substitute K(t) = ρt(t + 1) − Γ(t) into (3.5) at t = tε (ρ) to obtain (3.10).

B.3

Proof of Proposition 3.2

Proof of Proposition 3.2. Since FR (tZ ) ≥ εR , εZ = FR (tZ ) +

Γ(tZ ) Γ(tZ ) ≥ εR + . tZ tZ

Similarly, evaluating FZ at tR gives εZ ≤ FZ (tR ) = εR +

Γ(tR ) . tR

The final assertion follows from Γ ≥ 0 and Γ(tR )/tR ≤ supt∈I Γ(t)/t ≤ η, since tR ∈ I.

B.4

Proof of Lemma B.2

Lemma B.2 isolates the quantities needed for a quantitative fourth-order expansion: the excess of ρ(s) over EPs Ls and the third and fourth cumulants of privacy loss. The lemma applies equally to continuous and discrete mechanisms and retains an explicit fifth-order remainder. Lemma B.2 (Quantitative fourth-order identity for the CGF gap). Fix T > 0. Let Ls be the privacy-loss variables of mutually absolutely continuous pairs, with CGF K, so that K(−1) = 0. As usual, the dependence of K on s is implicit. Suppose, for all sufficiently small s = ̸ 0, that ρ(s) < ∞, 5 K ∈ C ([−1, T ]), and there is an MT < ∞, independent of s, such that sup

−1≤u≤T

|K (5) (u)| ≤ MT |s|5 .

(B.1)

Then, for 0 < t ≤ T , ρ(s)t(t + 1) − K(t) = (ρ(s) − κ1 (Ls ))t(t + 1) − +

κ3 (Ls ) 2 t (t + 1) 6

κ4 (Ls ) 2 t (1 − t2 ) + Es (t). 24

Moreover, |Es (t)| ≤

MT |s|5 2 (t + t5 ), 120

0 < t ≤ T.

Proof of Lemma B.2. Taylor’s theorem and (B.1) give K(u) =

4 X κj (Ls ) j=1

j!

uj + ∆s (u),

28

|∆s (u)| ≤

MT |s|5 5 |u| 120

(B.2)

for −1 ≤ u ≤ T . Applying this identity at u = −1 gives κ1 (Ls ) =

κ2 (Ls ) κ3 (Ls ) κ4 (Ls ) − + + ∆s (−1). 2 6 24

Substituting this identity into the cumulant expansion of K(t) gives κ1 (Ls )t(t + 1) − K(t) = −

κ3 (Ls ) 2 κ4 (Ls ) 2 t (t + 1) + t (1 − t2 ) + t2 ∆s (−1) − ∆s (t). 6 24

Thus Es (t) = t2 ∆s (−1) − ∆s (t). The two Taylor-remainder bounds give (B.2); adding (ρ(s) − κ1 (Ls ))t(t + 1) proves the identity.

B.5

Proof of Theorem 3.1

Proof of the local expansion theorem, Theorem 3.1. Apply Lemma B.2 with κ1 (Ls ) = EPs Ls . Its identity is (3.3), and (B.2) gives (3.4).

B.6

Proof of Theorem 3.3

Proof of Theorem 3.3. Independence gives tDt+1 (P (n) ∥Q(n) ) =

n X

Ki (t) = K(t).

i=1

Subtracting this identity from ρsum t(t + 1) = i ρi t(t + 1) proves (3.11). For n copies of one pair, K = nK, so supt>0 K(t)/{t(t + 1)} = nρ and Γ = nΓ. P

C

Proofs for Section 4

For the mechanism calculations, it is useful to divide out powers of the shift. For s ̸= 0, define Us :=

Ls − EPs Ls , s

As :=

2{ρ(s) − EPs Ls } ≥ 0. s2

(C.1)

The nonnegativity follows from limt↓0 K(t)/{t(t+1)} = K ′ (0) = EPs Ls and ρ(s) = supt>0 K(t)/{t(t+ 1)}. These definitions give ρ(s) − EPs Ls = s2 As /2, κ3 (Ls ) = s3 κ3 (Us ), and κ4 (Ls ) = s4 κ4 (Us ).

C.1

Proof of Lemma C.1

Throughout the additive-noise calculations, s is a signed output shift. For even f , K depends on s only through r = |s|, and we use r when rescaling the order. For the following endpoint-localization lemma, specialize the location convention (3.2) to a noise variable Z with density f . It gives sufficient conditions for ρ(s) = K ′ (0) = EPs Ls ; Proposition 4.4 verifies them for Student-t noise. The privacy-loss and CGF definitions give Ls = log f (Z) − log f (Z + s),

K(t) = log EZ∼f [etLs ],

and define the location score and its CGF S := −(log f )′ (Z),

ΛS (τ ) := log EZ∼f [eτ S ].

These equalities hold wherever the expectations are finite. Since EZ∼f [S] = 0, the fourth cumulant used below is
2 κ4 (S) = EZ∼f [S 4 ] − 3 EZ∼f [S 2 ] . 29

Lemma C.1 (Uniform control over large RDP orders). Assume that f is even. For each nonzero shift s, put r = |s|. Symmetry makes the CGFs at shifts s and r identical. For the current shift, define K(τ /r) Hs (τ ) = , τ > 0. τ (τ + r) Assume that Hs extends continuously to τ = 0 with Hs (0) = K ′ (0)/r2 , and that K ′ (0) 1 −→ EZ∼f [S 2 ], 2 s 2

1 K(t) = EZ∼f [S 2 ]s2 t(t + 1) + o(|s|2 ) 2

(C.2)

uniformly on every compact J ⊂ (0, ∞). For endpoint localization, assume that differentiation under the defining integral is valid through the required order and that the following uniform bound holds for every sufficiently small fixed η > 0: ΛS (τ ) . τ2 τ ≥η

lim sup sup Hs (τ ) ≤ sup s→0

τ ≥η

(C.3)

Near (r, τ ) = (0, 0) suppose the Taylor remainder is uniform and satisfies Hs (τ ) − Hs (0) =

  κ4 (S) τ (τ + r) + O τ (τ + r)2 . 24

(C.4)

Assume κ4 (S) < 0 and, for some a4 > 0, 1 ΛS (τ ) EZ∼f [S 2 ] − ≥ a4 min(τ 2 , 1), 2 τ2

τ > 0.

(C.5)

Then ρ(s) = K ′ (0) = EPs Ls for all sufficiently small nonzero s. Hence As = 0 exactly. Proof of Lemma C.1. We begin with the factorization used for endpoint localization. The likelihood identity gives K(0) = K(−1) = 0. Therefore, the definition of Hs gives the exact factorization K(t) = r2 t(t + 1)Hs (rt). Moreover, K ′ (0) = EPs Ls . Dividing the numerator and denominator in the definition of Hs (τ ) by τ and letting τ → 0 gives Hs (0) = K ′ (0)/r2 , as stated. Choose η > 0 small enough that, for all |s| sufficiently small, the absolute value of the remainder in (C.4) is bounded by κ4 (S) − τ (τ + r) 48 whenever 0 < τ ≤ η. It follows that Hs (τ ) − Hs (0) ≤

κ4 (S) τ (τ + r) < 0, 48

0 < τ ≤ η.

For τ ≥ η, (C.3) and (C.5) put the supremum a fixed distance below 12 EZ∼f [S 2 ]. Because Hs (0) = K ′ (0)/r2 tends to 12 EZ∼f [S 2 ], the region τ ≥ η also lies below Hs (0) for all sufficiently small nonzero s. Hence r2 supτ >0 Hs (τ ) = K ′ (0), which gives ρ(s) = K ′ (0) = EPs Ls . This completes the proof. 30

The hypotheses of Lemma C.1 can be checked from K and ΛS without first evaluating ρ. For a 5 C log-density, a common integrable bound on the required derivatives justifies the Taylor expansions

in (C.4). The uniform bound in (C.3) must be verified separately for each mechanism, as for the Student-tν family in Proposition 4.4. Nondifferentiable log-densities require piecewise calculations. The sign and uniform-control conditions exclude a zero fourth cumulant and rule out a competing second-order maximum away from the origin.

C.2

Derivation sketch for (C.6) and (C.7)

For a smooth location family, continue to use the location-score notation S introduced above. Suppose log f is five times continuously differentiable, differentiation under the relevant expectations is dominated through fifth order, the required score moments are finite, and the integration-by-parts boundary terms vanish. Taylor expansion of Ls = log f (Z) − log f (Z + s) and integration by parts then give 1 κ3 (Ls ) = κ3 (S)s3 + κ4 (S)s4 + O(|s|5 ), 2 κ4 (Ls ) = κ4 (S)s4 + O(|s|5 ).

(C.6) (C.7)

Indeed, the integration-by-parts identities E[(log f )′′ (Z)] = −E[((log f )′ (Z))2 ], 1 E[((log f )′ (Z))2 (log f )′′ (Z)] = − E[((log f )′ (Z))4 ] 3 give the fourth-order coefficient in (C.6). These identities also show why a kink in the log-density can change the fourth-order coefficient.

C.3

Proof of Proposition 4.1

Exact CGFs. In the setting of Proposition 4.1, the following hold for every s ∈ R and t > 0. For Gaussian noise, t(t + 1) 2 K(t) = s . (C.8) 2σ 2 For the Gaussian mixture, write Z = mξ+G with ξ uniform on {−1, 1} and G ∼ N (0, 1) independent; then "  t # cosh(mZ) 2 K(t) = log E exp{t(sZ + s /2)} . (C.9) cosh(m(Z + s)) For unit-scale Laplace noise, (1 + t)et|s| + te−(1+t)|s| K(t) = log 1 + 2t

!

.

(C.10)

Proof of Proposition 4.1. For part (a), completing the square in the Gaussian likelihood ratio gives (C.8) and the stated optimal parameter. Under Ps , Ls =

sZ s2 + , σ2 2σ 2

Z ∼ N (0, σ 2 ),

so As = 0, Us = Z/σ 2 , and κ3 (Us ) = κ4 (Us ) = 0. The CGF is quadratic, so the remainder in the local expansion theorem also vanishes. The theorem therefore gives Γ(t) = 0 exactly. 31

2

For part (b), the mixture density is proportional to e−x /2 cosh(mx), so (log fm )′ (x) = −x + m tanh(mx),

(log fm )′′ (x) = −1 + m2 sech2 (mx).

Evaluating log{fm (Z)/fm (Z + s)} gives (C.9). Integration by parts gives E[((log fm )′ (Z))2 ] = 1 − m2 E[sech2 (mZ)]. Generating Z + s by first shifting the unit-variance Gaussian component G by s and then adding the independent term mξ shows by data processing that ρ(s) ≤ s2 /2. The density is even, so both the CGF and ρ(s) are even in s. For z ≥ 0, log

fm (z) s2 ≥ |s|z + − m|s|, fm (z + |s|) 2

1 fm (z) ≥ ϕ(z − m). 2

Consequently,

1 2 eK(t) ≥ es t(t+1)/2 PY ∼N (m+t|s|,1) [Y ≥ 0]. 2 After division by t(t + 1) and passage to t → ∞, this proves the matching lower bound ρ(s) ≥ s2 /2. Put 2 Sm := −(log fm )′ (Z) = Z − m tanh(mZ), Im := E[Sm ]. The integration-by-parts identity above gives Im = 1 − m2 E[sech2 (mZ)]. Since fm is even, EPs Ls is even in s, and its second-order expansion is EPs Ls =

Im 2 s + O(s4 ). 2

Together with ρ(s) = s2 /2, this yields As = m2 E[sech2 (mZ)] + O(s2 ). Moreover, Sm is symmetric. The smooth-location cumulant expansions (C.6)–(C.7) give κ3 (Us ) = 21 κ4 (Sm )s + O(s2 ) = O(s),

κ4 (Us ) = κ4 (Sm ) + O(|s|) = O(1).

Substitution in the local expansion theorem, Theorem 3.1, proves (4.1). For part (c), symmetry makes the CGF depend on the shift only through |s|. Splitting at the two kinks gives (C.10). The exact value ρ(s) = |s| + e−|s| − 1 is the continuous-Laplace formula established by Harrison and Manurangsi (2026); differentiating the exact CGF at zero also gives EPs Ls = ρ(s) and hence As = 0. For s > 0, differentiation of the same transform gives κ3 (Us ) = −s + O(s2 ),

κ4 (Us ) = −2 + O(s).

d

Symmetry gives U−s = −Us , so the signed-s form of the first display is κ3 (Us ) = −s + O(s2 ), while the fourth-cumulant remainder is O(|s|). The local expansion theorem therefore gives s4 2 s4 t (t + 1) − t2 (1 − t2 ) + O(|s|5 ) 6 12 [t(t + 1)]2 4 = s + O(|s|5 ), 12

Γ(t) =

which proves item (c). 32

It remains to verify the common remainder hypothesis used above. It is immediate for Gaussian noise. For the Gaussian mixture and |s| ≤ s0 , |Ls | ≤ |s| |Z| + m + s0 /2 ,

and the mixture has uniformly bounded exponentially tilted moments of every fixed order. For Laplace noise, |Ls | ≤ |s|. Thus the fifth absolute centered moments under every tilt with −1 ≤ u ≤ T are O(|s|5 ). Since K (5) (u) is the fifth cumulant under the tilt, it is also O(|s|5 ) uniformly on this interval. The apparent singularities in the displayed Laplace formulas are removable because these bounded privacy-loss variables have positive smooth moment-generating functions on the real line. Hence all three families satisfy the hypotheses of the local expansion theorem.

C.4

Proof of Proposition 4.2

Proof of Proposition 4.2. Fix z > 0 and ∥v∥ = C, let Q = N (0, z 2 C 2 I), and, for 0 ≤ q ≤ 1, set Pq = (1 − q)Q + qN (v, z 2 C 2 I). Let W be the likelihood ratio of N (v, z 2 C 2 I) with respect to Q. Under Q, G 1 log W = − 2, G ∼ N (0, 1), z 2z and hence, for α ≥ 1,   α(α − 1) α EQ [W ] = exp . (C.11) 2z 2 Put Wq = 1 − q + qW = dPq /dQ. Convexity of x 7→ xα gives EQ [Wqα ] ≤ 1 − q + qEQ [W α ] ≤ exp



α(α − 1) . 2z 2 

Thus the one-step optimal zCDP parameter is at most 1/(2z 2 ). Since Wq ≥ qW , K(t) ≥ (t + 1) log q +

t(t + 1) . 2z 2

Dividing by t(t + 1) and letting t → ∞ proves ρ(Pq , Q) = 1/(2z 2 ) for every q > 0. With z = zn , 1/(2zn2 ) = ρsum /n, which proves the one-step identities in (4.2). At t = 1,   2 eK(1) = EQ [Wq2 ] = 1 + q 2 EQ [(W − 1)2 ] = 1 + e1/z − 1 q 2 , 2

which, with e1/zn = e2ρsum /n , proves the formula for Γ(1). It remains to establish monotonicity. For fixed z and t > 0, write h

i

Mt (q) := EQ (1 + q(W − 1))t+1 . Then Mt′ (0) = (t + 1)E(W − 1) = 0, whereas, for 0 < q < 1, h

i

Mt′′ (q) = t(t + 1)EQ (W − 1)2 {1 + q(W − 1)}t−1 > 0.

33

Hence Mt (q), and therefore K(t) viewed as a function of q, is strictly increasing. The one-step gap is consequently strictly decreasing. At q = 1, the pair is an ordinary Gaussian shift and its gap is zero. Continuity of the expectation at q = 0 gives the displayed one-step right-hand limit. For part (b), set z = zn , so that 1/(2zn2 ) = ρsum /n. Exact product additivity gives K(t) = nK(t), and its normalized supremum is n/(2zn2 ) = ρsum . Thus the standard-composition parameter is optimal for the product pair, and item (b) follows. The monotonicity and endpoint statements follow immediately from part (a).

C.5

Proof of Proposition 4.3

Proof of Proposition 4.3. Independence makes the product likelihood ratio the product of the P component likelihood ratios, so their log moment-generating functions add. Hence Γ(t) = i Φt (ρi ) P exactly. If Φt is convex, Jensen’s inequality and i ρi = ρsum give 

Γ(t) ≥ nΦt

ρsum . n 

Because ψ(0) = 0 and the single-pair gap is zero there, the chord inequality between 0 and ρsum gives ρi Φt (ρi ) ≤ Φt (ρsum ). ρsum Summing this inequality proves the upper bound in (4.4). Concavity reverses Jensen’s inequality and the chord inequality, and therefore reverses both bounds. The equality statements follow from the strict forms of the same two inequalities. For the unit-scale Laplace illustration, parameterize the selected inverse branch by the magnitude r ≥ 0. Here x = ρ(r) = r + e−r − 1, equivalently r = ψ(x). Direct differentiation of the exact CGF (C.10) gives, for r > 0, 2e2r {2e3r + 6e2r + 3er + 1} d2 Φ (x) = > 0. 1 3r + 1)2 dx2 (2e x=ρ(r) The right limit at r = 0 is 8/3, so Φ1 is strictly convex throughout the interval used in Figure 5. √ For the Gaussian-mixture panel, ψ(x) = 2x, and strict concavity of Φ1 on the displayed interval is supported numerically, but not proved, by negative centered finite differences on the plotted grid.

C.6

Proof of Proposition 4.4

Proof of Proposition 4.4. Fix ν and retain the notation Z, Ps , Q and ρν (s) from the proposition. Write ℓν = log fν . The location score is Sν := −ℓ′ν (Z) =

(ν + 1)Z , ν + Z2

|Sν | ≤ Bν :=

ν+1 √ . 2 ν

The density is even, so the CGF and ρν (s) are even in s. Put r = |s|. The mean-value theorem gives |Ls | ≤ Bν r. Consequently, Bν r K(t) 0≤ ≤ , t(t + 1) t+1 which tends to zero as t → ∞. The normalized CGF is continuous on compact t-intervals and has the finite endpoint K ′ (0) = EPs Ls , so ρν (s) < ∞ for every s ∈ R. 34

To localize the envelope for all sufficiently small nonzero s, put Yν = Z 2 /(ν + Z 2 ). The beta representation of a Student random variable gives Yν ∼ Beta(1/2, ν/2) and hence, for j ≥ 1, µ2j,ν := E[Sν2j ] =

(ν + 1)2j (1/2)j (ν/2)j . νj ((ν + 1)/2)2j

(C.12)

In particular, Iν := µ2,ν =

ν+1 , ν+3

κ4 (Sν ) = −

18(ν + 1)2 (ν 2 + 4ν − 1) < 0. ν(ν + 3)2 (ν + 5)(ν + 7)

(C.13)

Comparison with the even moments of N (0, Iν ) gives µ2j+2,ν /µ2j,ν (ν + 1)(ν + 3)(2j + ν) = < 1, (2j + 1)Iν ν(4j + ν + 1)(4j + ν + 3)

j ≥ 1,

where the difference between denominator and numerator is 2j(8νj + 3ν 2 + 4ν − 3) > 0 for ν ≥ 1. Because Sν is symmetric and bounded, termwise comparison of the entire moment series yields 2

E[eτ Sν ] < eIν τ /2 ,

τ ̸= 0.

Equation (C.13) supplies a quadratic separation near τ = 0, continuity supplies uniform separation on compact intervals away from zero, and log E[eτ Sν ] ≤ Bν τ controls large τ . Thus the quantitative sub-Gaussian condition (C.5) holds. All derivatives of positive order of ℓν are bounded. Uniformly in Z, Lr = −

5 X rj

6 ℓ(j) ν (Z) + O(r ), j! j=1

so Lr /r → Sν uniformly. Expanding the log moment-generating function and using the usual integration-by-parts identities gives (C.2) and (C.4), uniformly on the stated compact sets. Symmetry makes the CGFs at shifts s and r = |s| equal, and K(τ /r) → log E[eτ Sν ] locally uniformly in τ . The bound K(τ /r) Bν 0≤ ≤ τ (τ + r) τ +r controls τ ≥ R uniformly once R is large, while local uniform convergence controls each interval [η, R]. This proves (C.3). Lemma C.1 therefore implies ρν (s) = EPs Ls for every sufficiently small nonzero s. Thus As = 0. Symmetry gives κ3 (Sν ) = 0, and (C.6)–(C.7), followed by the local expansion theorem, Theorem 3.1, gives Γ(t) = −

κ4 (Sν ) [t(t + 1)]2 s4 + O(|s|5 ). 24

The coefficient obtained from (C.13) is exactly c4,ν in (4.6). Bounded derivatives justify six uniform differentiations in s. Since both K(t) and EPs Ls = ρν (s) are even functions of s, the gap is even and its fifth-order Taylor coefficient vanishes. The remainder is therefore O(|s|6 ).

35

Finally, c′4,ν = − where

3(ν + 1)p(ν) 4ν 2 (ν + 3)3 (ν + 5)2 (ν + 7)2

,

p(ν) = ν 6 + 8ν 5 − 19ν 4 − 288ν 3 − 613ν 2 − 72ν − 105.

The coefficients of p have one sign change, so Descartes’ rule shows that p has a unique positive root. Since p(1) < 0 and p(ν) → ∞, this root, numerically 6.013052641 . . ., is the unique maximizer on [1, ∞). Direct expansion gives c4,ν = 3/(4ν) − 9/ν 2 + O(ν −3 ).

C.7

Proof of Proposition 4.5

Proof of Proposition 4.5. The density fβ is even, so the CGF and ρβ (s) are even functions of the d

shift, and U−s = −Us . Put r = |s| when only the shift magnitude is relevant. Under Ps , write Ls =

|Z + s|β − |Z|β . β

For 1 < β < 2, the mean-value inequality and Young’s inequality give, for t ≥ 1, tLs ≤ tr(|Z| + r)β−1 ≤

|Z|β + Cβ,r (1 + tβ ). 2β

′ (1 + tβ ), so K(t)/{t(t + 1)} → 0 as t → ∞. Integration against fβ therefore yields K(t) ≤ Cβ,r Continuity on compact order intervals and the finite endpoint K ′ (0) = EPs Ls prove ρβ (s) < ∞. For β = 1, |Ls | ≤ r gives the same conclusion. For β = 2, completing the square gives K(t) = s2 t(t+1)/2. The absolute moments of Z are

E|Z|p = β p/β

Γ((p + 1)/β) , Γ(1/β)

p > −1.

For this proof, define Sβ (z) := sgn(z)|z|β−1 ,

Iβ := E[Sβ (Z)2 ],

Jβ := E[Sβ (Z)4 ].

Below, an unqualified Sβ or Sβ′ inside an expectation is evaluated at Z. The moment formula gives Iβ = β 2−2/β

Γ(2 − 1/β) , Γ(1/β)

Jβ = β 4−4/β

Γ(4 − 3/β) . Γ(1/β)

In particular, Jβ − 3Iβ2 = −24χβ . To prove the sign of χβ , put x = 1/β ∈ [1/2, 1] and C(x) :=

Jβ Γ(4 − 3x)Γ(x) = . Γ(2 − x)2 Iβ2

The integral representation of the digamma function gives d log C(x) = dx

Z ∞ 0

3e−(4−3x)u − e−xu − 2e−(2−x)u du. 1 − e−u

For x < 1, the numerator equals e−xu (y 2 − 1)(3y 2 + 1) < 0, 36

y = e−(1−x)u ∈ (0, 1).

Thus C decreases from C(1/2) = 3 to C(1) = 1, proving Jβ ≤ 3Iβ2 and hence χβ ≥ 0. Define the difference quotient Vs (z) :=

|z + s|β − |z|β . βs

Then Ls = sVs (Z) and, by (C.1), Us = Vs (Z) − E[Vs (Z)], and Vs (Z) → Sβ (Z) in every fixed Lp . Consequently, κ4 (Us ) = Jβ − 3Iβ2 + o(1). For 1 < β < 2, write Sβ′ (z) = (β − 1)|z|β−2 for z ̸= 0. For k = 1, 2, 3, split the integral defining E[Vsk ] into |z| ≤ 2|s| and |z| > 2|s|. On the first region, both Vs and Sβ are O(|s|β−1 ), so the absolute value of its contribution to s−1 {E[Vsk ] − E[Sβk ]} is O(|s|k(β−1) ) = o(1). On the second region, Taylor’s formula gives   s Vs (z) = Sβ (z) + Sβ′ (z) + o |s| |z|β−2 . 2

The required dominating powers are integrable because, for k ≤ 3, Sβk−1 Sβ′ behaves at zero as |z|(k−1)(β−1)+β−2 , whose exponent is greater than −1. Hence d k E[Vsk ] = E[Sβk−1 Sβ′ ], ds 2 s=0

k = 1, 2, 3.

Since E[Sβ ] = E[Sβ3 ] = 0, differentiating κ3 (X) = EX 3 − 3(EX)(EX 2 ) + 2(EX)3 gives lim

s→0

κ3 (Us ) 3 = {E[Sβ2 Sβ′ ] − Iβ E[Sβ′ ]}. s 2

Integration by parts yields E[Sβ′ ] = Iβ and E[Sβ2 Sβ′ ] = Jβ /3. Hence κ3 (Us ) = −12χβ s + o(|s|),

κ4 (Us ) = −24χβ + o(1).

(C.14)

For β = 1, differentiation of the exact Laplace transform gives κ3 (Us ) = −s + O(s2 ) and κ4 (Us ) = −2 + O(|s|), the same identity. At β = 2 both cumulants vanish. It remains to verify the fifth-derivative hypothesis. For the pair (Ps , Q), let Ps,u have density dPs,u /dPs = exp{uLs − K(u)}. Then K (5) (u) is the fifth cumulant of Ls under Ps,u . For 0 < |s| ≤ 1, |Ls | ≤ |s|Gβ (Z),

Gβ (z) := (|z| + 1)β−1 .

Let CT = max(1, T ). Since β − 1 < β, Z

(1 + Gβ (z)5 )eCT Gβ (z) fβ (z) dz < ∞.

Moreover sup−1≤u≤T |K(u)| → 0, so the normalizing constants of these tilted laws are bounded away from zero for all sufficiently small nonzero s. Consequently, sup

−1≤u≤T

EPs,u |Ls − EPs,u Ls |5 ≤ Cβ,T |s|5 . 37

The fifth cumulant equals the fifth centered moment minus ten times the product of the third and second centered moments; Lyapunov’s inequality bounds that product by the fifth absolute centered ′ |s|5 . For β = 1, the same conclusion follows from moment. Therefore sup−1≤u≤T |K (5) (u)| ≤ Cβ,T |Ls | ≤ |s|; for β = 2 it is immediate from the quadratic CGF. Insert (C.14) in the local expansion theorem, Theorem 3.1. The cubic and quartic terms combine to χβ [t(t + 1)]2 s4 + o(|s|4 ). Since t(t + 1) 2{ρβ (s) − EPs Ls } 2 s = t(t + 1){ρβ (s) − EPs Ls }, 2 s2 this proves (4.7). The endpoint t ↓ 0 in the definition of ρβ (s) gives ρβ (s) ≥ EPs Ls . Replacing ρβ (s) by this smaller endpoint value shows that the first term on the right-hand side of (4.7) is nonnegative. This proves (4.8). At β = 1, the exact Laplace CGF (C.10) gives K ′ (0) = |s| + e−|s| − 1 = ρ1 (s), and I1 = J1 = 1 gives χ1 = 1/12. At β = 2, the quadratic Gaussian CGF gives ρ2 (s) = EPs Ls = s2 /2, χ2 = 0, and Γ(t) = 0 for every t > 0.

C.8

Proof of Proposition 4.6

Proof of Proposition 4.6. For part (a), the discrete Laplace privacy loss is ar for z ≥ 0, −ar for z ≤ −r, and a(r − 2j) at z = −j for 1 ≤ j < r. Therefore 



r−1 ∞ ∞  X X 1 − λ  tar X λj eta(r−2j) . λj + λz + e−tar eK(t) = e  1+λ z=0 j=1 j=r

Summing the two geometric series and taking the logarithm gives K(t) in (4.9). The exact discrete Laplace formula established by Harrison and Manurangsi (2026) gives ρ(r) = EPr Lr as displayed. Substitution into the gap definition (3.1) proves the general-t formula. For part (b), completing the square in the discrete Gaussian sum gives Θσ (tr) Θσ (0) 2 2 Θσ ((1 + t)r) = er t(t+1)/(2σ ) , Θσ (0) 2

2

eK(t) = er t(t+1)/(2σ )

where the second equality uses integer periodicity. Thus K(t) =

r2 Θσ ((1 + t)r) t(t + 1) + log . 2 2σ Θσ (0)

The theta sum is maximized at integers, with equality only at integers. The logarithmic term is therefore nonpositive, while t = 1/r makes (1 + t)r = r + 1 an integer and attains the quadratic envelope. This proves ρ(r) = r2 /(2σ 2 ); differentiating at t = 0 also gives EPr Lr = r2 /(2σ 2 ). Periodicity of the theta sum gives the stated period and zero set. Poisson summation gives, uniformly in u, √ 2 2 2 2 Θσ (u) = 2πσ{1 + 2e−2π σ cos(2πu) + O(e−8π σ )}. Expanding the logarithm proves (4.10).

38

D

Supplementary Figures and Numerical Details

The supplementary figures check whether the comparisons in the main text persist under different settings: the analytical plots use t = 1.5 in place of t = 1, and the experiments use (ε, δ) = (5, 10−5 ) in place of (3, 10−5 ). Each figure is linked to the corresponding main-text figure so that the effect of the changed setting can be compared directly. The numerical checks assess quadrature accuracy, order truncation, and noise calibration; the experimental details and tables record the workloads, training settings, and complete results behind Section 5. These checks support the reported patterns within the settings examined.

D.1

Computational Setup

The analytical figures in Sections 3 and 4 are deterministic. Gaussian, Laplace, and lattice curves use the closed forms (C.8), (C.10), and Proposition 4.6. Gaussian-mixture expectations use 192-node Gauss–Hermite quadrature; Student-tν expectations use 2048-node Gauss–Legendre quadrature √ after x = ν tan θ; and generalized-Gaussian transforms use adaptive quadrature split at −s and 0. For the last family, we maximize K(u)/[u(u + 1)] over 0 < u ≤ 32, including the u ↓ 0 endpoint. The large-order truncation is checked separately from the asymptotic statement in Proposition 4.5. Specifically, for 1 < β < 2, r = |s|, and 0 < ζ < 1, the bound K(u) urβ + (ur)β (β − 1)β−1 /(βζ β−1 ) − β −1 log(1 − ζ) ≤ Bβ,r,ζ (u) := u(u + 1) u(u + 1) follows from Ls ≤ r|Z|β−1 + rβ , Young’s inequality, and E exp{ζ|Z|β /β} = (1 − ζ)−1/β . For fixed ζ, every term of Bβ,r,ζ (u) is nonincreasing when u ≥ (β − 1)/(2 − β). At the displayed r = 0.2 and β ∈ {1.25, 1.5, 1.75, 1.9}, this threshold is at most 9. Choosing ζ to minimize Bβ,r,ζ (32) gives an upper bound below the numerically evaluated endpoint K ′ (0) ≤ ρ(s) in every case; the smallest margin is 2.7296 × 10−4 . These parameter-specific tail checks justify the cutoff used in the plots. The endpoints β = 1, 2 use the exact Laplace and Gaussian formulas. Exponential moments use log-sum-exp arithmetic where needed. Figure 3 uses the captioned four-point pair and, for its Gaussian panel, ρ = 1/2 on 100 log-spaced values 10−2 ≤ t ≤ 101.2 . Figure 4(a) fixes t = 1 and uses 61 log-spaced shifts 10−3 ≤ s ≤ 10−1 , with m = 1 and unit-scale Laplace noise; slopes are fitted over s ≤ 0.01. Figure 4(b)–(c) uses 1001 values of q ∈ [0, 1] with (n, ρsum , ∥v∥/C, t) = (20, 5, 1, 1) and plots the positive-q right limit at q = 0. Figure 5 uses 241 values of ω ∈ [0, 1] with (n, ρsum , t) = (8, 1/2, 1); repeating the Gaussianmixture calculation with 384 rather than 192 Hermite nodes leaves every interior centered second difference of Φ1 negative on a 501-point grid in [0, 1/2], providing numerical evidence for concavity. Figure 6 fixes (t, s) = (1, 0.2) and uses ν ∈ {1, 2, 3, 4, 5, 6.013052641, 8, 10, 15, 20, 30, 50} and β ∈ {1, 1.25, 1.5, 1.75, 1.9, 2}. Finally, Figure 7 uses 361 values on 0 ≤ t ≤ 3, with (r, σ) = (1, 0.60) for discrete Gaussian noise and r ∈ {1, 2, 4} at ar = 0.5 for discrete Laplace noise. In Figures 4(a) and 6, solid and dashed curves denote the complete gaps and the stated local-expansion terms, respectively; the dotted line in Figure 6 marks the maximizer ν ≈ 6.013.

D.2

Results at an Alternative Reporting Order

The fixed-order displays in Section 4 use t = 1 (Rényi order α = 2). Figures 11 and 12 repeat every such comparison at t = 1.5 (α = 2.5), with all mechanism parameters and numerical grids unchanged. Figure 7 already spans 0 ≤ t ≤ 3, so the lattice examples require no duplicate fixed-order panel. 39

The fitted log–log slopes of the gap against s are 2.000 and 3.997 for the Gaussian-mixture and Laplace families, respectively, agreeing with the quadratic and quartic local expansions. The Gaussian gap remains exactly zero. The Student-tν curve remains nonmonotone, while the generalized Gaussian curve decreases to its exact zero at β = 2. For the noninteger Poisson order, we evaluate the expectation in (4.2) by a 192-node Gauss– Hermite rule and repeat it with 384 nodes; the largest CGF discrepancy is 7.11 × 10−15 . The exact composition agrees with nΓ(1.5) to 1.78 × 10−15 . The allocation calculations reuse the exact Laplace transform and the two Gaussian-mixture quadrature resolutions. On the displayed ρ grid, the minimum Laplace curvature is 1.15 and the maximum Gaussian-mixture curvature is −1.41. Thus the computed curves retain the two allocation directions; the Gaussian-mixture concavity check remains numerical. 0.0007

exact gap (fitted slope) Gaussian mixture (2.00) Laplace (4.00)

10−4

exact gap local expansion

0.0015

−8

Γ(1.5)

10−6 10

(c) Generalized Gaussian: s = 0.2

exact gap local expansion

0.0006 Γ(1.5)

Γ(1.5)

(b) Student-tν : s = 0.2

(a) Common mechanisms

10−2

0.0005 0.0004

0.0010

0.0005

10−10 Gaussian: Γ(1.5) = 0 (off log scale)

10−12 10−3

10−2 positive shift size s

0.0003 0.0000 10−1

100

101 degrees of freedom ν

1.00

1.25 1.50 1.75 shape parameter β

2.00

Figure 11: The comparisons in Figures 4(a) and 6 at t = 1.5. The zero, quadratic, and quartic local expansions and the two tail-parameter patterns seen at t = 1 remain visible. (a) Gaussian, m = 1 Gaussian-mixture, and unit-Laplace gaps over 10−3 ≤ s ≤ 10−1 ; solid curves are exact numerical gaps and dashed curves are the local leading terms, with fitted slopes shown in parentheses. (b) Student-tν and (c) generalized Gaussian gaps at s = 0.2, with dashed quartic local-expansion approximations.

40

(b) n-step Poisson gap

(a) One-step Poisson gap 0.8

15 Γ(1.5)

Γ(1.5)

0.6 0.4

10 5

0.2 0.0

0 0.0

0.2 0.4 0.6 0.8 Poisson sampling rate q

1.0

0.0

(c) Laplace allocation

0.2 0.4 0.6 0.8 Poisson sampling rate q

1.0

(d) Gaussian-mixture allocation 0.80

0.5

0.75 0.70 Γ(1.5)

Γ(1.5)

0.4 0.3

0.65 0.60

0.2

0.55 0.50 0.0

0.2

0.4 0.6 0.8 allocation weight ω

1.0

0.0

0.2

0.4 0.6 0.8 allocation weight ω

1.0

Figure 12: The Poisson-subsampling and allocation comparisons in Figures 4(b)–(c) and 5 at t = 1.5. The Poisson gaps still decrease from their positive-q right limits to zero, while concentrating the allocation still increases the Laplace gap and decreases the computed Gaussian-mixture gap. (a) One-step Γ(1.5) and (b) n-step Γ(1.5) = nΓ(1.5) for n = 20 and ρsum = 5. (c) Unit-Laplace and (d) m = 1 Gaussian-mixture composition gaps along (4.5) for n = 8 and ρsum = 1/2. The Poisson values at q = 0 are right-hand limits of the positive-q family.

41

D.3

ACS Workloads and Additional Results

D.3.1

Experimental Setup and Metadata

The ACS experiment in Section 5.1 uses published ACS 5-year aggregates (U.S. Census Bureau, 2020). The source file contains 3,221 county and county-equivalent rows. Excluding the District of Columbia and Puerto Rico leaves 3,142 rows in the 50 states, of which 2,902 have numeric values in all five earnings columns. We rank states by raw county count, breaking ties alphabetically by state name. The top 20 states contain 2,098 rows; 1,961 are complete in all five columns, so 137 (6.53%) are excluded without imputation. Table 1 lists the selected states and their workload sizes: mi is the number of complete counties in state i, and di = 5mi is the number of county-by-education queries. Earnings are kept in dollars, and every component uses the ordered-pair shift ∆ = $1,000. At a candidate dollar scale b, the standardized shift is s = ∆/b = 1000/b. The centered base noises are N (0, 1), the equal-weight Gaussian mixture N (−1, 1)/2 + N (1, 1)/2, and unit-scale Laplace, with variances 1, 2, and 2, respectively. These choices determine the one-query CGF and optimal zCDP parameter used in Section 5.1. For each state and family, the RDP and zCDP releases use 20,000 paired draws with shared base noise, so their MSE difference compares the two calibrated scales on the same draws. Table 1: The 20 ACS workloads in Section 5.1. Here mi is the number of counties or countyequivalents with all five earnings entries numeric, and di = 5mi is the number of scalar queries in the composed workload. States are ranked by total county count before incomplete rows are excluded. Rank State 1 2 3 4 5 6 7 8 9 10

TX GA VA KY MO KS IL NC IA TN

mi

Rank State mi

di

212 1060 146 730 128 640 113 565 114 570 88 440 96 480 96 480 97 485 92 460

11 12 13 14 15 16 17 18 19 20

NE IN OH MN MI MS OK AR WI AL

71 91 87 86 81 78 75 73 71 66

di 355 455 435 430 405 390 375 365 355 330

For each state and family, our experiment records (available from the authors upon request) give the two calibrated shifts and dollar scales, active orders, achieved privacy, RDP and zCDP MSE, analytic ∆MSE, and Monte Carlo summaries. The earnings values cancel from the paired squared-error difference; they define and label the workload, while the calibrated noise determines its MSE. D.3.2

Results at an Alternative Privacy Target

We repeat the ACS experiment at (ε, δ) = (5, 10−5 ), changing only the privacy target. The workloads, queries, mechanisms, and 20,000 paired draws are unchanged, so Figures 13 and 14 repeat the comparisons in Figures 8 and 9. The Gaussian calibrations still coincide, and the Gaussian-mixture MSE differences still exceed the Laplace differences. The absolute differences are smaller at the weaker privacy target. Table 2 gives their ranges across the 20 workloads. 42

Gaussian (exact equality)

(a) ΔMSE

(b) Var(ZRDP )/Var(ZzCDP )

102

101

Laplace

1.0

Var(ZRDP )/Var(ZzCDP )

ΔMSE [($1000)2 /query]

103

Gaussian mixture

Gaussian: ΔMSE = 0 (off log scale)

500

750

0.9 0.8 0.7 0.6

1000

500

Number of scalar queries di

750

1000

Number of scalar queries di

Figure 13: Figure 8 repeated at (ε, δ) = (5, 10−5 ) on the same 20 ACS workloads. Left: analytic ∆MSE versus di = 5mi ; Gaussian is exactly zero and is omitted from the logarithmic axis. Right: Var(ZRDP )/ Var(ZzCDP ) versus di , with Gaussian at one. Gaussian-mixture MSE differences still exceed Laplace differences.

Paired ΔMSE ̂ [($1000)2 /query]

103

Gaussian mixture

Laplace

MN 430

IN 455

Gaussian: ΔMSE ̂ = 0 (off log scale)

102

101

AL 330

NE 355

WI 355

AR 365

OK 375

MS 390

MI 405

OH 435

KS 440

TN 460

IL 480

NC 480

IA 485

KY 565

MO 570

VA 640

GA 730

TX 1060

State (scalar query count di shown below abbreviation)

Figure 14: Figure 9 repeated at (ε, δ) = (5, 10−5 ). States are ordered by di ; the logarithmic [ in ($1000)2 per query. Each violin contains 20,000 paired draws and vertical axis gives ∆MSE describes single-release outcomes, not confidence intervals. Gaussian is identically zero and is omitted. Gaussian-mixture differences still exceed Laplace differences. Table 2: ACS MSE differences and variance ratios at (ε, δ) = (5, 10−5 ). Ranges are over the 20 workloads in Table 1; ∆MSE is analytic and is measured in ($1000)2 per query. Noise family

∆MSE range

Var(ZRDP )/ Var(ZzCDP ) range

Gaussian Gaussian mixture Laplace

0 265.0–860.6 7.279–7.313

1 0.553–0.557 0.988–0.996

43

D.4

DP-SGD Settings and Additional Results

D.4.1

Experimental Setup and Metadata

The DP-SGD experiment in Section 5.2 uses the standard Fashion-MNIST split of 60,000 training and 10,000 test images (Xiao et al., 2017), with fixed normalization and no augmentation. The network has two convolution and max-pooling stages, followed by a 64-unit hidden layer, with no batch normalization. Opacus clips each example at C = 1 using flat clipping, adds Gaussian noise once per training step, and divides by the expected Poisson batch size qN , where N = 60,000. We use SGD with initial learning rate 0.05, no momentum, weight decay 10−4 , and a 2000-step cosine learning-rate schedule shared by all horizons. The runs use Opacus 1.6.0 and PyTorch 2.11.0 on an NVIDIA A100-SXM4-80GB. The sampling-rate sweep uses q ∈ {0.005, 0.0075, 0.01, 0.02, 0.05} at n = 500; the compositioncount sweep uses n ∈ {250, 500, 1000, 1500, 2000} at q = 0.01. Each setting uses five seeds. Within a setting and seed, all three methods share the initialization and Poisson sampling stream, and the RDP and zCDP runs share the standard-normal stream used to generate their noise. This pairing holds at both privacy targets, (ε, δ) = (3, 10−5 ) and (5, 10−5 ). Test metrics and a fixed deterministic training probe are evaluated every 25 steps; minibatch metrics are recorded at every step. The training histories appear in Figures 10(c)–(d) and 2(c)–(d). Table 3 gives the nine distinct settings at (ε, δ) = (3, 10−5 ) and their paired RDP–zCDP summaries. Mean test accuracies of the RDP and clipping-only runs differ by at most 0.08 percentage points, while the zCDP results vary with the calibrated noise. At (ε, δ) = (3, 10−5 ), all 115 scheduled trajectories (25 clipping-only and 90 private) completed and produced all 135 endpoint results; the (5, 10−5 ) target adds 90 private trajectories and reuses the clipping-only runs. Initialization and Poisson-stream hashes agree within each group of three methods, and standard-normal stream hashes agree between the two private runs. At q = 0.01, shorter horizons are exact prefixes of the longest clipping-only trajectory. The maximum calibration errors in ε are 8.67 × 10−10 at ε = 3 and 1.52 × 10−9 at ε = 5. For the main-text target, refining the order grid changes a noise multiplier by at most 7.38 × 10−7 in relative terms, and no optimum lies on a search boundary. At q = 1, calibration agrees with the Gaussian formula to a relative error of 9.46 × 10−10 . These checks assess the numerical calibration separately from the variation across training seeds.

44

Table 3: Final DP-SGD test metrics at (ε, δ) = (3, 10−5 ) for the sweeps in Figures 10 and 2. The shared (q, n) = (0.01, 500) cell is listed once, and nq is the expected number of epochs. In the upper block, accuracies are percentages and LZ , LR , and L0 are final test cross-entropies for the zCDP, RDP, and clipping-only runs. The lower block reports paired means of ∆Ltest = LZ − LR and ∆Acc = AccR − AccZ (positive values favor RDP) and 95% confidence intervals; ∆Acc is in percentage points. q

nq AccZ AccR

n

Acc0

LZ

LR

L0

0.0050 500 2.5 56.408 65.138 65.218 1.673 0.945 0.945 0.0075 500 3.75 62.256 65.244 65.204 1.141 0.947 0.948 0.0100 500 5 64.188 65.304 65.270 1.020 0.947 0.946 0.0200 500 10 66.000 65.232 65.192 0.945 0.944 0.945 0.0500 500 25 65.554 65.304 65.342 0.943 0.945 0.945 0.0100 250 2.5 57.972 61.866 61.928 1.120 1.089 1.089 0.0100 1000 10 66.354 69.502 69.512 1.337 0.963 0.962 0.0100 1500 15 66.144 70.432 70.460 1.653 0.979 0.978 0.0100 2000 20 64.600 70.522 70.564 2.031 0.981 0.980 q

n

∆Ltest

∆Acc

0.0050 500 0.728 ± 0.231 8.73 ± 6.10 0.0075 500 0.193 ± 0.123 2.99 ± 2.60 0.0100 500 0.073 ± 0.050 1.12 ± 2.92 0.0200 500 0.000 ± 0.039 −0.77 ± 1.44 0.0500 500 −0.001 ± 0.006 −0.25 ± 0.36 0.0100 250 0.0100 1000 0.0100 1500 0.0100 2000

0.031 ± 0.035 0.375 ± 0.104 0.673 ± 0.102 1.050 ± 0.110

45

3.89 ± 1.77 3.15 ± 1.15 4.29 ± 1.31 5.92 ± 1.48

D.4.2

Results at an Alternative Privacy Target

We repeat the controlled DP-SGD experiment at (ε, δ) = (5, 10−5 ), changing only the privacy target. The architecture, sweep grids, five seeds, and paired random streams remain unchanged. Figures 15 and 16 repeat the comparisons in Figures 10 and 2. The RDP–zCDP test-cross-entropy differences are smaller at the weaker privacy target. They still decrease toward zero as q increases and grow with n. zCDP

RDP

Clipping only

(a) Final test accuracy

(b) Final test cross-entropy

68

Final test cross-entropy

Final test accuracy (%)

1.2 66

64

62

1.1

1.0

0.9 0.5%

0.75% 1%

2%

5%

0.5%

0.75% 1%

Poisson sampling rate q

(c) Accuracy trajectory at q = 0.5%

5%

(d) Cross-entropy trajectory at q = 0.5% 2.25

Test cross-entropy

60

Test accuracy (%)

2%

Poisson sampling rate q

50 40 30 20

2.00 1.75 1.50 1.25 1.00

10 0

100

200

300

400

500

0

Training step

100

200

300

400

500

Training step

Final points and trajectory curves are five-seed means; bars and bands are 95% Student-t confidence intervals. All panels include the clipping-only baseline.

Figure 15: The sampling-rate sweep in Figure 10 at (ε, δ) = (5, 10−5 ). The low-q zCDP penalty is smaller than at (3, 10−5 ), and the zCDP curve again approaches the RDP and clipping-only curves as q increases. All cells use n = 500; panels (a)–(b) report final accuracy and test cross-entropy, and panels (c)–(d) report their trajectories at (q, n) = (0.005, 500). All panels include the three methods; bars and bands are five-seed 95% confidence intervals. Table 4 gives all endpoint values. The paired test-loss difference decreases from 0.154 at q = 0.005 to approximately zero at q = 0.05 and increases from 0.012 at n = 250 to 0.182 at n = 2000. The seedwise log-q and log-n trends have p = 0.0107 and p = 0.0152, respectively. The accuracy trends are not significant at this target (p = 0.106 for q and p = 0.361 for n). Thus the test-loss pattern persists at the weaker privacy target, without a corresponding monotone accuracy pattern.

46

zCDP

RDP

Clipping only

(a) Final test accuracy

(b) Final test cross-entropy Final test cross-entropy

Final test accuracy (%)

72.5 70.0 67.5 65.0 62.5 60.0 57.5

1.2

1.1

1.0

0.9 250

500

1000

1500

2000

250

500

Number of compositions n

(c) Accuracy trajectory at n = 2000

1500

2000

(d) Cross-entropy trajectory at n = 2000

70

2.25

60

Test cross-entropy

Test accuracy (%)

1000

Number of compositions n

50 40 30 20

2.00 1.75 1.50 1.25 1.00

10 0

250

500

750 1000 1250 1500 1750 2000

0

Training step

250

500

750 1000 1250 1500 1750 2000

Training step

Final points and trajectory curves are five-seed means; bars and bands are 95% Student-t confidence intervals. All panels include the clipping-only baseline.

Figure 16: The composition-count sweep in Figure 2 at (ε, δ) = (5, 10−5 ). The paired test-crossentropy difference again grows with n, while the accuracy difference is nonmonotone. All cells use q = 0.01; panels (a)–(b) report final accuracy and test cross-entropy, and panels (c)–(d) report the longest-cell trajectories at (q, n) = (0.01, 2000). All panels include the three methods. The trajectories use the noise calibrated for n = 2000 throughout, so their value at training step 500 is not the separately calibrated n = 500 endpoint in panels (a)–(b). Bars and bands are five-seed 95% confidence intervals.

47

Table 4: Final DP-SGD test metrics at (ε, δ) = (5, 10−5 ). Notation and units match Table 3; the lower block reports paired means of ∆Ltest = LZ − LR and ∆Acc = AccR − AccZ (positive values favor RDP) and 95% confidence intervals. q

nq AccZ AccR

n

Acc0

LZ

LR

L0

0.0050 500 2.5 63.586 65.160 65.218 1.099 0.945 0.945 0.0075 500 3.75 65.192 65.230 65.204 0.984 0.947 0.948 0.0100 500 5 65.306 65.278 65.270 0.973 0.947 0.946 0.0200 500 10 65.810 65.226 65.192 0.942 0.944 0.945 0.0500 500 25 65.394 65.322 65.342 0.944 0.945 0.945 0.0100 250 2.5 59.784 61.888 61.928 1.101 1.089 1.089 0.0100 1000 10 69.120 69.490 69.512 1.037 0.962 0.962 0.0100 1500 15 69.866 70.474 70.460 1.114 0.979 0.978 0.0100 2000 20 70.028 70.560 70.564 1.162 0.980 0.980 q

n

∆Ltest

∆Acc

0.0050 500 0.154 ± 0.111 1.57 ± 2.10 0.0075 500 0.037 ± 0.055 0.04 ± 1.17 0.0100 500 0.026 ± 0.042 −0.03 ± 2.44 0.0200 500 −0.003 ± 0.027 −0.58 ± 1.03 0.0500 500 0.000 ± 0.005 −0.07 ± 0.20 0.0100 250 0.0100 1000 0.0100 1500 0.0100 2000

0.012 ± 0.022 0.075 ± 0.063 0.135 ± 0.084 0.182 ± 0.080

48

2.10 ± 1.12 0.37 ± 0.69 0.61 ± 1.67 0.53 ± 1.38

Related documents

• Unifying Privacy Accounting: Information Equivalence and Information Loss#1186728
• Privacy Amplification Without Independence: How Far Negative Dependence Carries the Guarantees of Poisson Subsampling#822359
• Privacy for free in the overparameterized regime#211268
• EXTENSIONS TO THE THEORY OF MARKETS AND PRIVACY: MECHANICS OF PRICING INFORMATION#933267
• Differential Privacy Meets Fixed Parameter Tractability: Algorithms and Lower Bounds#1070005