Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Thomas Sesmat * 1 Gabriel Meseguer-Brocal 2 Geoffroy Peeters 1
arXiv:2606.07271v1 [cs.LG] 5 Jun 2026
Abstract Understanding what generative models retain from training data remains challenging, with implications for copyright and privacy. Beyond verbatim reproduction, models can encode subtler traces of their training data that never surface in their outputs yet remain exploitable. We study this regime for Rectified Flows, which are increasingly used in deployed generative systems. We analyse the interpolation path Xλ = (1 − λ)X0 + λX1 that defines the Rectified Flow training. We show that a gap exists between the reconstruction of train and test data that follows a bell-shaped curve over λ, wich accumulates during training, while the validation metrics remain stable. The signal has a maximum whose location we derive in closed form under Gaussian assumptions. We validate these predictions on both audio and images and show that the bell-shaped structure is universal, while the peak prediction holds when our assumptions are satisfied. As a proof of concept, we exploit this specific λ-resolved structure to perform a Membership Inference Attack, distinguishing members of the training set from non-members.
Figure 1. Overview of our approach. Top: Detection protocol, given a sample x1 , we interpolate with noise x0 at varying λ, predict the velocity vθ (xλ , λ), and measure reconstruction error d = ∥x1 − x̂1 ∥2 . Middle: The train-test gap in reconstruction error follows a bell-shaped curve over λ; we derive a closed-form expression for the peak location. Bottom: As a proof of concept, the λ-resolved errors can be fed to an MLP classifier to perform Membership Inference Attack.
1. Introduction The deployment of generative models has raised legal concerns across multiple domains. Lawsuits have been filed over unauthorised use and direct reproduction of copyrighted photographs (cou, 2023; Somepalli et al., 2023), text from news organisations and authors (cou, 2023), and music from major record labels (Recording Industry Association of America, 2024; Newton-Rex, 2024). Beyond verbatim reproduction lies a spectrum of subtler forms of
memorisation: a trained model may reconstruct training samples more accurately, respond more confidently near them, or otherwise treat them differently from held-out data, all without ever reproducing them. We refer to such measurable asymmetries as the membership signal. We study its structure in Rectified Flows (Liu et al., 2023; Lipman et al., 2023), which underlie widely deployed systems such as FLUX.1 (Black Forest Labs, 2024), VoiceBox (Le et al., 2023), and Stable Audio Open (Evans et al., 2024). Our analysis focuses on the structural properties of the framework rather than on attacks against specific deployed models.
1
LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France 2 Deezer Research, Paris, France. Correspondence to: Thomas Sesmat <[email protected]>. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
Characterising the membership signal is challenging because aggregate training metrics offer little guidance: a 1
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Memorisation in generative models. The most studied form of memorisation is verbatim reproduction, where models regenerate training samples exactly (Carlini et al., 2023; Somepalli et al., 2023). For diffusion models, Bonnaire et al. (2025) characterise this phenomenon through two timescales: τgen at which quality generation begins, and τmem beyond which memorisation emerges. Gu et al. (2025) systematically studies factors affecting such memorisation, including dataset size, model capacity, and the surprising role of random labels. For Flow Matching, Gao & Li (2024) derives analytical expressions for the optimal velocity field and analyses memorisation in sample data subspaces, while Bertrand et al. (2025) identifies distinct temporal phases in the generative process.
model can encode rich information about its training data while its loss curves show no sign of overfitting (Tirumala et al., 2022). Where, in the model’s behaviour, does this information reside? Existing studies of memorisation in diffusion models suggest that intermediate timesteps carry most of the signal (Matsumoto et al., 2023). A theoretical understanding of where and why the signal concentrates remains lacking, particularly for Rectified Flows, whose deterministic interpolation path differs from iterative denoising. We propose to characterise the membership signal along the interpolation path Xλ = (1 − λ)X0 + λX1 that defines Rectified Flow training. This path offers a continuum of positions to analyse how the model treats training versus held-out data: at λ = 0, the model observes pure noise; at λ = 1, the data itself. The intermediate regime is where the model must leverage learnt structure to predict the velocity and where membership signals emerge. We illustrate our approach in Figure 1.
Ippolito et al. (2023) argues that memorisation exists on a spectrum of similarity to training data, ranging from exact reproduction to subtle statistical traces. Crucially, preventing verbatim reproduction does not eliminate the risk: models can still leak information through paraphrase, stylistic similarity, or structural patterns. Alternative definitions formalise this intuition, such as counterfactual memorisation, which measures how predictions change when a specific sample is removed from training (Zhang et al., 2023).
Paper organisation and contributions. After reviewing related works in Section 2, we demonstrate mathematically in Sections 3 and 4 that a gap exists between the reconstruction of train and test data that follows a bell-shaped curve over λ. We derive a closed-form expression for the peak location λ∗F as a function of the covariances Σ0 and Σ1 , identifying where the membership signal is maximal. In Sections 5 and 6, we validate these theoretical predictions experimentally on various modalities (audio and image datasets), latent spaces, architectures, and noise configurations. We show that the bell-shaped structure is universal while the peak prediction holds when our Gaussian assumptions are satisfied. Finally, in Section 7, we demonstrate, as a proof of concept, that this λ-resolved structure is exploitable by a simple Membership Inference Attack (MIA, distinguishing training from held-out samples) on a piano music dataset. For reproducibility, our experimental code is available here.
At the subtle end of this spectrum lies train-test distinguishability: a model may produce novel samples while still encoding exploitable signals about its training data. We refer to this measurable asymmetry as the membership signal, and it is the form of memorisation we study. It remains comparatively underexplored: Tirumala et al. (2022) shows that it can occur without visible overfitting on loss curves. Feldman (2020) argues that some memorisation is necessary for generalisation on long-tailed distributions, suggesting it is not inherently undesirable but rather a phenomenon to understand.
Trajectory-dependent memorisation signals The observation that memorisation signals depend on position along the denoising trajectory is not new. Matsumoto et al. (2023) report that intermediate timesteps are the most vulnerable to MIA, with success varying predictably across the denoising trajectory. Other MIA methods developed for diffusion models, such as SecMI (Duan et al., 2023) and PIA (Kong et al., 2023), also leverage trajectory information, though they rely on the iterative denoising structure and do not transfer directly to Rectified Flows. More broadly, Shokri et al. (2017) formalised the MIA as a diagnostic for studying what models retain. Our work extends this trajectory perspective to Rectified Flows and grounds it theoretically: we derive why the membership signal peaks at a specific location λF determined by data statistics, rather than discovering it empirically.
2. Related Work Rectified Flows. Rectified Flows (Liu et al., 2023) and Flow Matching (Lipman et al., 2023) learn velocity fields via regression on linearly interpolated samples Xλ = (1 − λ)X0 + λX1 . Unlike diffusion models that require many denoising steps, Rectified Flows learn straighter paths between noise and data, enabling high-quality generation in fewer steps. This efficiency has driven adoption in major systems including Stable Diffusion 3 (Esser et al., 2024), FLUX (Black Forest Labs, 2024), and Stable Audio Open (Evans et al., 2024). Liu et al. (2023) also introduces a reflow procedure that further straightens trajectories by iterating training through the learnt velocity field, replacing the independent coupling between X0 and X1 with a learnt pairing. 2
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
3. Mathematical Setup
3.3. Loss Decomposition and the Membership Signal Pn (i) The training loss Ltrain (λ) = n1 i=1 ∥vθ (xλ , λ) − v (i) ∥2 decomposes as:
We establish the framework for analysing membership signals in latent Rectified Flows. For readability, all proofs are deferred to Appendix A.
Ltrain (λ) = Entrain (λ) + σ̂n2 (λ) − 2Gtrain (λ) n
3.1. Distributions and Interpolation
P (i) (i) 2 where Entrain = n1 i ∥vθ (xλ , λ) − v ∗ (x λ , λ)∥ is the P 1 2 empirical approximation error, σ̂n = n i ∥ϵi (λ)∥2 the empirical irreducible variance, and:
Let X0 ∼ p0 = N (0, Σ0 ) denote samples from a noise distribution and X1 ∼ p1 denote latent representations of data, with covariance Σ1 . We assume X0 ⊥ ⊥ X1 , which holds by construction in Rectified Flow training without reflow (Liu et al., 2023). Define: Xλ = (1 − λ)X0 + λX1
(interpolation)
(1)
V = X1 − X0
(velocity)
(2)
n
Gtrain (λ) ≜ n
Proposition 3.1 (Train-test asymmetry). Conditioned on the training set Dtrain :
(3)
This is a deterministic function fully determined by (p0 , p1 ).
train EDtest [Gtest ]=0 m (λ) | D
By the definition of conditional expectation, Ep0 ×p1 [V − v ∗ (Xλ , λ) | Xλ ] = 0. This orthogonality property implies that for any measurable function g : Rd → Rd : Ep0 ×p1 [⟨g(Xλ ), V − v ∗ (Xλ , λ)⟩] = 0
To isolate Gtrain (λ) from other terms in the train-test gap, n we introduce two assumptions.
The irreducible variance is:
Assumption 3.2 (Uniform approximation error). The model’s deviation from v ∗ is the same on training points as on the population:
(5)
This quantity depends only on the distributions (p0 , p1 ) and represents a fundamental limit: since v ∗ is the optimal predictor, no model can achieve a lower expected squared error, regardless of its capacity.
Entrain (λ) = E pop (λ) ≜ Ep0 ×p1 [∥vθ (Xλ , λ)−v ∗ (Xλ , λ)∥2 ] (12) This holds when the model has not overfit in the classical sense, e.g., thanks to early stopping. Note that this does not preclude a train-test gap in the loss, which can arise through Gtrain (λ). n
3.2. Training and Test Sets (i)
(i)
Let Dtrain = {(x0 , x1 )}ni=1 be a training set drawn i.i.d. from p0 × p1 . For each sample i ∈ {1, . . . , n}, define: (i)
(i)
v (i) ≜ x1 − x0 (i)
(i)
(i)
(7)
(i) (xλ , λ)
(8)
ϵi (λ) ≜ v
(i)
−v
∗
Assumption 3.3 (Representative sample). The empirical irreducible variance matches its population value:
(6)
xλ ≜ (1 − λ)x0 + λx1
2 σ̂n2 (λ) = σirr (λ)
(13)
This holds by the law of large numbers for large n.
Once Dtrain is drawn, these are fixed vectors in Rd . (j)
(11)
whereas Gtrain (λ) on training data is a priori generically n non-zero.
(4)
2 σirr (λ) ≜ Ep0 ×p1 [∥V − v ∗ (Xλ , λ)∥2 ]
1X (i) (i) ⟨vθ (xλ , λ) − v ∗ (xλ , λ), ϵi (λ)⟩ (10) n i=1
The test loss admits the same decomposition with analogous test 2 terms Em , σ̂m , and Gtest m . The difference lies in the crosscorrelation term:
The optimal predictor is the conditional expectation: v ∗ (x, λ) ≜ Ep0 ×p1 [V | Xλ = x]
(9)
Under these assumptions, conditioning on the training set Dtrain (and hence on the trained model vθ ), the expected train-test gap over fresh test samples reduces to:
(j)
Let Dtest = {(x̃0 , x̃1 )}m j=1 be a test set drawn i.i.d. from (j)
p0 × p1 , independently of Dtrain . Define ṽ (j) , x̃λ , and ϵ̃j (λ) analogously.
EDtest [∆(λ) | Dtrain ] = 2Gtrain (λ) n
A model vθ : Rd × [0, 1] → Rd is trained on Dtrain . The parameter θ depends on both the training and the randomness in the training procedure (initialisation, batch ordering, etc.). In the following analysis, we condition on the trained model: once θ is fixed, vθ is a deterministic function.
(14)
The quantity Gtrain (λ) is the membership signal: it mean sures the correlation between the model’s deviation from v ∗ and the training-specific residuals ϵi (λ). 3
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
3.4. Covariance Structure 2 EDtrain [Gtrain (λ)] = σirr (λ) · n
Under X0 ⊥ ⊥ X1 , direct computation yields: Φ(λ) ≜ Cov(Xλ ) = (1 − λ)2 Σ0 + λ2 Σ1
(15)
C(λ) ≜ Cov(V, Xλ ) = λΣ1 − (1 − λ)Σ0
(16)
Corollary 4.3 (Peak at minimal linear information). Under the assumptions of Theorem 4.2, EDtrain [Gtrain (λ)] is n uniquely maximised at:
When (X0 , X1 ) is jointly Gaussian, v ∗ is linear: λ∗ =
(17)
where A(λ) = C(λ)Φ(λ)−1 and b(λ) = E[V ] − A(λ)E[Xλ ]. For non-Gaussian p1 , v ∗ may have a nonlinear component.
σ02 2 σ0 + σ12
(20)
This coincides with λ∗F from Proposition 4.1 in the isotropic case. Corollary 4.4 (Boundary behavior). Under the assumptions of Theorem 4.2, EDtrain [Gtrain (λ)] is minimised at λ ∈ n {0, 1}.
4. Theoretical Analysis Having identified Gtrain (λ) as the membership signal, we n now analyse its structure as a function of λ. We first identify a critical point where linear information is minimal (Section 4.1), then prove that the membership signal peaks there for Gaussian distributions (Section 4.2), and finally extend heuristically to the general case (Section 4.3). As a reminder, for readability, all proofs are deferred to Appendix A.
Corollary 4.5 (Asymptotics behavior). For large n: E[Gtrain (λ)] ≈ n
2 σirr (λ) n
(21)
4.3. General Case: Heuristic Extension For non-Gaussian distributions and nonlinear models, we provide heuristic arguments that we validate empirically in Section 6.
4.1. The Critical Point: Minimal Cross-Covariance We first identify a special value of λ where the crosscovariance C(λ) has a minimal norm.
Decomposition of the learning target. For general distributions, the optimal predictor may have a nonlinear component:
Proposition 4.1 (Critical point of cross-covariance). The squared Frobenius norm ∥C(λ)∥2F is a convex parabola in λ with a unique minimum at: tr(Σ20 ) + tr(Σ0 Σ1 ) λ∗F = tr((Σ0 + Σ1 )2 )
(19)
2 where σirr (λ) = d σ02 + σ12 − c(λ)2 /ϕ(λ) , with scalars ϕ(λ) = (1 − λ)2 σ02 + λ2 σ12 and c(λ) = λσ12 − (1 − λ)σ02 .
The cross-covariance C(λ) determines how strongly Xλ predicts V through linear regression: a large ∥C(λ)∥ means a strong linear prediction.
v ∗ (x, λ) = A(λ)x + b(λ)
n−1 n(n − 2)
v ∗ (x, λ) = A(λ)x + b(λ) + r(x, λ)
(18)
(22)
where r(x, λ) ≜ v ∗ (x, λ) − A(λ)x − b(λ) captures the deviation from the best linear approximation. For Gaussian distributions, r ≡ 0.
Under isotropy (Σ0 = σ02 I, Σ1 = σ12 I), this minimum has a stronger interpretation: C(λ∗F ) = 0 exactly, so the optimal linear predictor A(λ) = C(λ)Φ(λ)−1 vanishes and Xλ carries no linear information about V . In the general case, minimising ∥C(λ)∥F does not guarantee A(λ) is minimised, since Φ(λ)−1 also varies with λ.
For a training sample i, the target velocity becomes: (i)
(i)
v (i) = A(λ)xλ + b(λ) + r(xλ , λ) + ϵi (λ) {z } | {z } | linear signal
(23)
ηi (λ): nonlinear target
The linear signal generalises to held-out data. The nonlinear target ηi (λ) combines the population nonlinearity r (which generalises) with the sample-specific residual ϵi (which does not).
4.2. Gaussian Case: Peak at Minimal Linear Information For isotropic Gaussian distributions, we prove that EDtrain [Gtrain (λ)] is maximised exactly at λ∗F . n
The competition mechanism. From the perspective of gradient descent, the model cannot distinguish between (i) r(xλ , λ) and ϵi (λ). This indistinguishability follows from their shared statistical structure:
Theorem 4.2 (Peak location for isotropic Gaussian). Let X0 ∼ N (0, σ02 Id ) and X1 ∼ N (0, σ12 Id ) be independent in Rd . For a linear model trained by ordinary least squares on n > 2 samples: 4
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Proposition 4.6 (Shared statistics of r and ϵ). For (X0 , X1 ) ∼ p0 × p1 :
while both losses improve, leaving the model vulnerable to membership inference at early stopping despite no visible overfitting.
Ep0 ×p1 [r(Xλ , λ)] = 0
(24)
Covp0 ×p1 (r(Xλ , λ), Xλ ) = 0
(25)
4.5. Why the Assumptions Hold in Practice
Ep0 ×p1 [ϵ(λ)] = 0
(26)
Covp0 ×p1 (ϵ(λ), Xλ ) = 0
(27)
The closed-form prediction λ∗F relies on Gaussian isotropic assumptions. We argue that these are reasonable approximations for latent diffusion models. We discuss here why these are reasonable approximations for latent diffusion models, how their validity can be characterised empirically, and what alternatives exist when they fail.
Since Dtrain is drawn i.i.d. from p0 × p1 , the law of large numbers implies: n
n
1X n→∞ (i) ηi (λ)(xλ )⊤ −−−−→ 0 n i=1 (28) (i) A model observing only {(xλ , v (i) )}ni=1 cannot distinguish, based on first and second-order statistics, which part of ηi will generalise. The gradient pushes the model to explain ηi = r + ϵi jointly, inevitably fitting some of the samplespecific component ϵi . 1X n→∞ ηi (λ) −−−−→ 0, n i=1
Approximate Gaussianity. Latent spaces are designed with constrained statistics: VAEs regularise toward a Gaussian prior via KL divergence (Kingma & Welling, 2014), while encoders like Music2Latent (Pasini et al., 2024) bind activations via tanh. By the maximum entropy principle (Jaynes, 1957), bounded latent spaces with fixed covariance tend toward Gaussian distributions.
Role of the linear signal. When ∥C(λ)∥ is large, the linear signal dominates, and by spectral bias (Rahaman et al., 2019), it is learnt first. The nonlinear target ηi contributes little to the loss, keeping Gtrain (λ) low. n
Approximate isotropy. KL-regularised VAEs explicitly penalise deviation from N (0, I) (Kingma & Welling, 2014). For other encoders, architectural choices produce similar effects: batch normalisation (Ioffe & Szegedy, 2015) standardises activations, and symmetric bounded activations like tanh discourage correlations. More generally, independent per-dimension processing tends toward approximately diagonal covariance.
Near λ∗F , where ∥C(λ)∥ is minimised (Proposition 4.1), the linear signal vanishes (A(λ) ≈ 0). The model must explain the entirety of ηi using nonlinear features. Competition between learning r and fitting ϵi is maximal, and Gtrain (λ) n peaks. 4.4. Why Standard Metrics Miss the Membership Signal
Dominant linear structure. Even when the nonlinear residual r is non-zero, neural networks learn low-frequency (linear) components first due to spectral bias (Rahaman et al., 2019). Xu et al. (2019) formalises this as the Frequency Principle: networks fit target functions from low to high frequencies during training. The location where A(λ) vanishes determines where the model must rely on higher-order structure.
The train-test gap ∆(λ) ≈ 2Gtrain (λ) provides a membern ship signal at each λ. Yet standard training protocols fail to detect it due to two masking mechanisms. Spatial averaging. Standard training monitors losses averaged over λ ∼ p(λ): Lglobal = Eλ∼p(λ) [L(λ)]
(29) Continuity of λ∗F . The formula for λ∗F depends continuously on Σ0 and Σ1 . Small deviations from exact Gaussianity or isotropy produce correspondingly small deviations in the peak location, suggesting robustness to moderate violations.
If Gtrain (λ) concentrates near λ∗F while p(λ) spreads over n [0, 1], the signal is diluted. Temporal compensation. On the training data, the loss decomposes as Ltrain (λ) = Entrain (λ) + σ̂n2 (λ) − 2Gn (λ). As training progresses, Entrain (λ) decreases while Gn (λ) increases as the model fits training-specific residuals; both effects reduce Ltrain , making them indistinguishable.
5. Experimental Protocol Having established that the train-test gap follows a bellshaped curve over λ peaking at λ∗F , we now design a protocol to validate these predictions and demonstrate their exploitation for membership inference.
On validation data, under Assumption 3.2, E test (λ) decreases in tandem while Gtest (λ) ≈ 0, so validation loss also decreases. The membership signal thus accumulates 5
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path Table 1. Gaussianity and isotropy of latent representations. |γ|: mean absolute skewness (0 for symmetric); |κ|: mean excess kurtosis (0 for Gaussian); |ρ|: mean absolute inter-dimension correlation (0 for independent); ∥Σ−I∥F /d: normalised deviation from isotropic unit variance.
5.1. Detection Protocol Given a trained model vθ and a sample x1 , we measure the reconstruction quality at each λ: 1. Interpolate: Sample x0 ∼ p0 = N (0, Σ0 ), compute xλ = (1 − λ)x0 + λx1 2. Predict: Compute vθ (xλ , λ) 3. Reconstruct: x̂1 = xλ + (1 − λ)vθ (xλ , λ)
DATASET
L ATENT SPACE
|γ|
|κ|
|ρ|
∥Σ − I∥F /d
MAESTRO V 3 MTG-JAMENDO FMA L ARGE
M USIC 2L ATENT M USIC 2L ATENT M USIC 2L ATENT
0.18 0.07 0.08
0.22 0.16 0.23
0.23 0.17 0.16
0.14 0.13 0.12
MAESTRO V 3
S TABLE AUDIO VAE
0.08
0.10
0.16
0.08
C ELEBA
S TABLE D IFFUSION VAE
0.12
0.71
0.61
0.40
4. Measure: MSE(λ) = ∥x1 − x̂1 ∥2 ×4 and ×1/4), (3) the latent space (Music2Latent vs Stable Audio VAE (Evans et al., 2025)), (4) the modality (audio vs images using CelebA datasets (Liu et al., 2015)), (5) the architecture (Transformer vs UNet), (6) the model capacity (from 410M to 140M and 880M parameters), and (7) the sampling scheduler (uniform vs log-normal). Results are in Section 6.2. Full details on datasets, architectures, and configurations are provided in Appendix B.
For each data point x1 , we sample K = 100 different noise samples x0 and average the MSE over them. We vary λ ∈ {0, 0.1, . . . , 1.0}. The procedure is depicted in Figure 1. 5.2. Experimental Setup We first establish a baseline configuration on audio, then systematically vary each component to test robustness. In the following, we present our baseline, and full details on all datasets, encoders, and architectures are in Appendix B.
6. Results
Baseline configuration. Our primary experiments use MAESTRO v3 (Hawthorne et al., 2019), a dataset of ∼200 hours of classical piano, where the official split ensures no composition appears in multiple subsets (satisfying Assumption 3.3). Audio is encoded via Music2Latent (Pasini et al., 2024), a pretrained autoencoder mapping to 64-channel latents at 10 Hz. We train a Transformer (410M parameters) adapted from DiT (Peebles & Xie, 2023) with AdamW (lr 10−4 , batch size 256), log normal λ-sampling (Esser et al., 2024), and early stopping at the validation plateau.
6.1. Validating Theoretical Predictions Before presenting our main results, we verify that both our latent spaces and the model architecture satisfy the assumptions underlying Theorem 4.2. Gaussianity of latent representations. Theorem 4.2 proves that the membership signal peaks at λ∗F under Gaussian isotropic assumptions. Table 1 reports skewness, excess kurtosis, and covariance isotropy for each configuration. As we see, the latent spaces of all our audio configurations exhibit low skewness, kurtosis, and weak inter-dimension correlations, satisfying our Gaussian isotropic assumptions. In contrast, the latent space of our image configuration (CelebA with Stable Diffusion VAE) does not satisfy the required assumptions. While its skewness remains acceptable (|γ| = 0.12), its kurtosis (|κ| = 0.71) indicates heavy-tailed marginals, and the correlations are strong (|ρ| = 0.61).
Evaluation. We compute reconstruction MSE on 5,000 training and 5,000 held-out samples, using K = 100 noise realisations per sample. The reconstruction error satisfies MSE(λ) = (1 − λ)2 ∥vθ − v∥2 . Early stopping ensures Assumption 3.2 while Gn (λ) accumulates undetected. We normalise to remove the (1 − λ)2 factor: ∆norm (λ) =
MSEtest (λ) − MSEtrain (λ) MSEtest (λ) + MSEtrain (λ)
(30) Mechanistic assumptions: competition between linear and nonlinear features. The closed-form prediction λ∗F and the heuristic argument of Section 4.3 rely on the model leveraging nonlinear features near λ∗F , where linear prediction becomes impossible. We test this directly by comparing the trained Transformer to a linear OLS predictor fitted on the same task. Figure 2 reports the ratio of their test losses as a function of λ. As predicted, the ratio is close to 1 at the boundaries λ ∈ {0, 1}, where linear prediction suffices, and peaks near where the membership signal is maximum, i.e., where the Transformer’s nonlinear capacity provides
This quantity is proportional to Gn (λ) and peaks at λ∗F , being positive when the model reconstructs training samples better than held-out ones. Ablations. To validate the robustness of our findings, we vary the configuration along several axes: (1) the data distribution Σ1 (datasets of different diversity: FMA Large (Defferrard et al., 2017a), MTG Jamendo (Bogdanov et al., 2019)), (2) the noise distribution Σ0 (scaling the variance by 6
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
(a) Train and validation loss Figure 2. Ratio of Transformer to OLS test loss as a function of λ, across configurations. The ratio is consistently maximal where the membership signal peaks.
(b) Gap ∆norm (λ∗F )
Figure 4. Temporal evolution on MAESTRO. (a) Validation loss decreases until early stopping (dashed). (b) Train-test gap grows throughout training.
provides direct evidence. Validation loss decreases steadily until early stopping (Figure 4a), suggesting healthy learning according to standard diagnostics. Yet the gap ∆norm (λ∗F ) grows from the first epochs (Figure 4b), long before validation plateaus. By early stopping, a significant gap has accumulated, which is invisible to standard metrics but exploitable for membership inference. 6.2. Ablation Study Table 2 summarises all configurations tested. The bellshaped curve appears in every case; the peak prediction λ∗F matches when the Gaussian isotropic assumptions hold.
Figure 3. Normalised train-test gap ∆norm (λ) on MAESTRO. The curve exhibits the predicted bell shape with peak near λ∗F (dashed line).
(1) Data distribution (Σ1 ). Figure 5 shows bell curves for three audio datasets with varying diversity and covariance Σ1 , testing Proposition 4.1: each yields a different predicted λ∗F , and observed peaks match in all cases (Table 2, rows 1). Peak magnitude varies with dataset size; MAESTRO v3 (smallest) shows the strongest signal, while FMA Large (largest) shows the weakest, consistent with the ∼ 1/n scaling of the membership signal predicted by Corollary A.7.
the largest gain. The pattern holds across multiple configurations. Bell-shaped gap curve. Figure 3 displays the normalised train-test gap ∆norm (λ) on MAESTRO v3. As predicted, the gap exhibits a bell-shaped pattern: minimal at boundaries (λ ∈ {0, 1}) and maximal at intermediate values, confirming Corollary 4.4. This bell shape is universal; it appears in all configurations we tested, regardless of dataset, architecture, latent space, or modality (Section 6.2). Extended analysis including additional statistics is provided in Appendix C.
(2) Noise distribution (Σ0 ). Figure 5 also shows the effect of scaling the noise variance while fixing Σ1 using the Maestrov3 dataset, directly testing Proposition 4.1: increasing σ02 shifts λ∗F rightward as predicted (Table 2, row 2). For Σ0 × 4, the predicted λ∗F = 0.59 falls just below the observed interval [0.6, 0.7], which we consider a match within grid resolution.
Peak location. For configurations satisfying the Gaussian isotropic assumptions, the observed peak λobs matches the theoretical prediction λ∗F from Proposition 4.1 within grid resolution. On MAESTRO v3 with Music2Latent, λobs ∈ [0.5, 0.6] versus λ∗F = 0.52. This agreement holds across all audio configurations (Table 2).
(3) Latent space. Replacing Music2Latent with Stable Audio VAE yields a different predicted λ∗F (0.50 vs. 0.52), as expected since the two encoders induce different covariances Σ1 . The observed peak matches the prediction in both cases (Table 2, rows 3; Figure 6).
Temporal evolution. A central claim is that the membership signal differs from classical overfitting. Figure 4 7
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Figure 5. Ablations (1)–(2): Effect of data distribution Σ1 and noise distribution Σ0 . Values are normalise for better visualisation, Value between parentheses are raw values. Dashed lines indicate predicted λ∗F values. Trained with Maestrov3 dataset with Music2Latent latent space and Transformer architecture
Figure 6. Ablations (3)–(5): Effect of latent space encoder, model architecture, and modality. Values are normalise for better visualisation, Value between parentheses are raw values. The bell shape persists across all configurations; peak prediction fails only when Gaussian isotropic assumptions are violated (CelebA).
(4) Modality: limits of λF prediction. On CelebA with SD VAE, the bell-shaped curve persists (Figure 6), confirming that the phenomenon extends beyond audio. However, the observed peak (λobs ∈ [0.6, 0.7]) deviates from the prediction (λ∗F = 0.45; Table 2).The high kurtosis and correlation values (Table 1) violate Theorem 4.2’s requirement, suggesting why peak prediction fails for this configuration. A discussion about the analysis of these failure modes, along with an exploration of the possibility of relaxation, is provided in Appendix D. (5) Architecture. Replacing the Transformer with a UNet preserves both the bell-shaped structure and the peak location λ∗F (Table 2, ablation (4); Figure 6). However, the peak magnitude drops substantially (from 0.09 to 0.01), consistent with the UNet producing notably lower-quality generations than the Transformer.
Figure 7. Ablations (6)–(7): Effect of model capacity and λsampling scheduler. The peak location remains unchanged across all configurations; only the magnitude varies. Sizes: S is 140M parameters, M is 410M parameters and L is 880M parameters
(6) Model capacity. Varying the Transformer size from 140M to 880M parameters leaves the peak location unchanged across all configurations (Table 2, rows 6; Figure 7), while peak magnitude increases consistently with model size. Larger models fit training-specific residuals more accurately, amplifying the membership signal without shifting its location.
6.3. What holds universally vs. what requires our assumptions Across all configurations, the bell-shaped structure, its boundary behaviour, its temporal accumulation, and the linear/nonlinear competition mechanism hold universally, including for CelebA, where our Gaussian isotropic assumptions are violated (ablations 1–7). Within this universal structure, peak location is governed solely by data geometry (Σ0 , Σ1 ): dataset, noise scale, and encoder shift predictably per Proposition 4.1 (ablations 1–3), while architecture, capacity, and scheduler do not (ablations 4, 6, 7). Peak magnitude, by contrast, reflects model and training choices: larger models and log-normal scheduling amplify the signal without moving its location (ablations 6–7).
(7) λ-sampling scheduler. Replacing the log-normal scheduler with a uniform one preserves both the bell-shaped curve and the peak location while reducing the peak magnitude (Table 2, rows 7; Figure 7). This attenuation is consistent with the log-normal scheduler concentrating training near λ ≈ 0.5, which coincides with λ∗F and thereby amplifies the membership signal. 8
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path Table 2. Ablation study summary. All configurations exhibit the bell-shaped curve. For ablations (1)–(4), the predicted peak λ∗F matches λobs when Gaussian isotropic assumptions hold. For ablations (5)–(7), the peak location is unchanged while magnitude varies. †: assumptions violated (see Table 1). A BLATION
C ONFIGURATION
λ∗F
λobs
M ATCH
(1)
DATA (Σ1 )
MAESTRO V 3 MTG-JAMENDO FMA L ARGE
0.52 0.37 0.42
0.5–0.6 0.3–0.4 0.4–0.5
✓ ✓ ✓
(2)
N OISE (Σ0 )
Σ0 × 0.25 Σ0 × 1 Σ0 × 4
0.31 0.52 0.59
0.3–0.4 0.5–0.6 0.6–0.7
✓ ✓ ✓
(3)
L ATENT SPACE
M USIC 2L ATENT S TABLE AUDIO VAE
0.52 0.50
0.5–0.6 0.5–0.6
✓ ✓
(4)
M ODALITY†
C ELEBA (SD VAE)
0.45
0.6–0.7
×
A BLATION
C ONFIGURATION
P EAK MAGNITUDE
(5)
A RCHITECTURE
T RANSFORMER UN ET
0.09 0.01
(6)
M ODEL CAPACITY
140M 410M 880M
0.06 0.09 0.12
(7)
S CHEDULER
L OG - NORMAL U NIFORM
0.09 0.06
amplitude of the bell-shaped gap observed for each dataset.
8. Discussion Limitations. The closed-form peak prediction λ∗F requires near-Gaussian isotropic latents; on CelebA with SD VAE, the peak location deviates, though the bell shape persists, confirming it is a universal property of Rectified Flow training independent of our distributional assumptions. Our theory also assumes independent coupling (X0 ⊥⊥ X1 ), excluding the reflow procedure; preliminary experiments (Appendix F) suggest the bell shape persists under one reflow step, but with substantially attenuated magnitude, indicating reflow may offer a natural mitigation as a byproduct of its trajectory-straightening objective. The MIA we developed is a proof of concept under a white-box setting; stronger threat models, such as black-box or label-only access, remain to be explored. We also study unconditional generation exclusively, while deployed systems condition on text prompts; conditioning modifies the effective distribution, altering Σ1 and hence λ∗F . Finally, our experiments scale up to 880M parameters; model capacity amplifies the signal (ablation 6), while dataset size attenuates it (ablation 1), and their interaction at the scale of deployed systems such as FLUX or SD3 remains an open empirical question.
7. Implications for Membership Inference Our analysis reveals that the reconstruction error follows a predictable bell-shaped profile across λ, with a computable peak location and a vanishing signal at the boundaries. As a proof of concept, we demonstrate that this structured gap is exploitable for MIA.
Implications. Since λ∗F is architecture-independent (ablations 4–7), the peak can be located empirically on a small proxy model and transferred to larger target models without retraining. This structural knowledge also opens the door to targeted defences: rather than regularising uniformly across the interpolation path, one could concentrate privacypreserving mechanisms near λ∗F , where the membership signal is maximal. Beyond security, our analysis connects to training efficiency: the peak λ∗ corresponds to where prediction is hardest, as xλ contains balanced contributions from noise and data. Esser et al. (2024) found empirically that concentrating p(λ) near 0.5 improves SD3; our theory provides a principled explanation and suggests that adapting p(λ) to dataset-specific λ∗ could further accelerate convergence. Conversely, schedulers concentrated near λ∗F also amplify membership leakage, revealing a fundamental tradeoff between training efficiency and privacy.
Membership Inference Attack. Given a query sample x1 , we compute the reconstruction MSE at each λ ∈ {0, 0.1, . . . , 1.0} using K = 100 noise samples, yielding an 11-dimensional feature vector that captures the full λresolved profile. We then train a simple MLP classifier on these features to predict member/non-member, requiring only forward passes through the trained model (no gradient computation or weight access), making the attack lightweight and practical. The λ-resolved profile provides a richer signal than any single evaluation point: it encodes the full shape of the bell curve, whose amplitude and location are characteristic of training membership. Using a single reconstruction error at λ = λ∗ , equivalent to ignoring the λ-resolved structure (i.e. Naive Attack), achieves only a 0.67 AUC score. Consistent with our theory, the naive baseline (i.e., using a single reconstruction error) peaks at λ∗ , confirming that the membership signal concentrates there as predicted. Adapting SecMI (Duan et al., 2023) and PIA (Kong et al., 2023) to Rectified Flows yields AUC scores of 0.72 and 0.83, respectively. Our method achieves 0.91 AUC on MAESTRO v3, demonstrating that the theoretical signal translates to practical risks. Results on additional datasets, in Appendix E, remain positive across all configurations, with AUC scores decreasing consistently with the
9. Conclusion We showed that Rectified Flows encode membership signals in a structured, predictable way: it follows a universal bell-shaped curve over λ, peaks at a location governed by data geometry, and accumulates silently while standard diagnostics see nothing. This structure translates into practical risk, as a simple MIA exploiting it consistently outperforms baselines adapted from the diffusion literature. 9
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Impact Statement
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., and Ré, C. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
This work aims to improve the theoretical understanding of Rectified Flows and the information they retain about their training data. We hope it provides useful tools for practitioners and researchers working on generative models.
Defferrard, M., Benzi, K., Vandergheynst, P., and Bresson, X. FMA: A dataset for music analysis. In International Society for Music Information Retrieval Conference (ISMIR), 2017a.
Acknowledgements We thank the anonymous reviewers for their thorough and insightful reviews. We are also grateful to Manuel Moussalam and Romain Hennequin from Deezer for their careful reading of the mathematical derivations and valuable feedback during the preparation of this manuscript. This work was supported by the computational resources provided by LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France.
Defferrard, M., Benzi, K., Vandergheynst, P., and Bresson, X. FMA: A dataset for music analysis. In Cunningham, S. J., Duan, Z., Hu, X., and Turnbull, D. (eds.), Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017, Suzhou, China, October 23-27, 2017, pp. 316–323, 2017b. URL https: //ismir2017.smcnus.org/wp-content/ uploads/2017/10/75_Paper.pdf.
References Getty images lawsuit against stability ai, 2023. URL www. courtlistener.com/. Case 1:23-cv-00135. Bertrand, Q., Gagneux, A., Massias, M., and Emonet, R. On the closed-form of flow matching: Generalization does not arise from target stochasticity. In Advances in Neural Information Processing Systems, volume 38, 2025. Black Forest Labs. FLUX.1. blackforestlabs.ai/, 2024.
Duan, J., Kong, F., Wang, S., Shi, X., and Xu, K. Are diffusion models vulnerable to membership inference attacks? In Proceedings of the 40th International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research. PMLR, 2023. Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., Podell, D., Dockhorn, T., English, Z., and Rombach, R. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.
https://
Blattmann, A., Rombach, R., Oktay, K., Müller, J., and Ommer, B. Retrieval-augmented diffusion models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022.
Evans, Z., Parker, J. D., Carr, C. J., Zukowski, Z., Taylor, J., and Pons, J. Stable audio open. arXiv preprint arXiv:2407.14358, 2024. Evans, Z., Parker, J. D., Carr, C., Zukowski, Z., Taylor, J., and Pons, J. Stable audio open. In 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6-11, 2025, pp. 1–5. IEEE, 2025. doi: 10.1109/ ICASSP49660.2025.10888461. URL https://doi. org/10.1109/ICASSP49660.2025.10888461.
Bogdanov, D., Won, M., Tovstogan, P., Porter, A., and Serra, X. The mtg-jamendo dataset for automatic music tagging. In Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML 2019), Long Beach, CA, United States, 2019. URL http://hdl.handle.net/10230/42015.
Feldman, V. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 954–959. ACM, 2020.
Bonnaire, T., Urfin, R., Biroli, G., and Mézard, M. Why diffusion models don’t memorize: The role of implicit dynamical regularization in training. In Advances in Neural Information Processing Systems, volume 38, 2025. Best Paper Award, Oral presentation.
Gao, W. and Li, M. How do flow matching models memorize and generalize in sample data subspaces? arXiv preprint arXiv:2410.23594, 2024.
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito, D., and Wallace, E. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253–5270. USENIX Association, 2023.
Gu, X., Du, C., Pang, T., Li, C., Lin, M., and Wang, Y. On memorization in diffusion models. Transactions on Machine Learning Research, 2025. 10
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Hawthorne, C., Stasyuk, A., Roberts, A., Simon, I., Huang, C. A., Dieleman, S., Elsen, E., Engel, J. H., and Eck, D. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https://openreview.net/forum? id=r1lYRjC9F7.
Matsumoto, T., Miura, T., and Yanai, N. Membership inference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW), San Francisco, CA, USA, May 25, 2023, pp. 77–83. IEEE, 2023. doi: 10.1109/SPW59333.2023.00013. URL https: //doi.org/10.1109/SPW59333.2023.00013. Newton-Rex, E. Suno is a music ai company aiming to generate $120 billion per year. but is it trained on copyrighted recordings?, 2024. URL https://www. musicbusinessworldwide.com.
Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (ICML), volume 37 of Proceedings of Machine Learning Research, pp. 448–456. PMLR, 2015.
Pasini, M., Lattner, S., and Fazekas, G. Music2latent: Consistency autoencoders for latent audio compression. arXiv preprint arXiv:2408.06500, 2024.
Ippolito, D., Tramèr, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette-Choo, C. A., and Carlini, N. Preventing verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Language Generation Conference (INLG), 2023.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 2023. URL https://doi.org/10. 1109/ICCV51070.2023.00387. Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., and Courville, A. On the spectral bias of neural networks. In International Conference on Machine Learning (ICML), pp. 5301–5310, 2019.
Jaynes, E. T. Information theory and statistical mechanics. Physical Review, 106(4):620–630, 1957. Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR), 2014.
Recording Industry Association of America. Record companies bring landmark cases for responsible AI against Suno and Udio, 2024. Press release.
Kong, F., Duan, J., Ma, R., Shen, H. T., Zhu, X., Shi, X., and Xu, K. An efficient membership inference attack for the diffusion model by proximal initialization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In IEEE Symposium on Security and Privacy (S&P), pp. 3–18, 2017.
Le, M., Vyas, A., Shi, B., Karrer, B., Sari, L., Moritz, R., Williamson, M., Manohar, V., Adi, Y., Mahadeokar, J., and Hsu, W. Voicebox: Text-guided multilingual universal speech generation at scale. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6048–6058, 2023. Su, J., Ahmed, M. H. M., Lu, Y., Pan, S., Bo, W., and Liu, Y. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 2024.
Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.
Tirumala, K., Markosyan, A. H., Zettlemoyer, L., and Aghajanyan, A. Memorization without overfitting: Analyzing the training dynamics of large language models. In Advances in Neural Information Processing Systems, volume 35, 2022.
Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations (ICLR), 2023. Spotlight.
Xu, Z.-Q. J., Zhang, Y., and Xiao, Y. Training behavior of deep neural network in frequency domain. In Neural Information Processing (ICONIP), volume 11953 of Lecture Notes in Computer Science, pp. 264–274. Springer, 2019.
Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. 11
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Zhang, C., Ippolito, D., Lee, K., Jagielski, M., Tramèr, F., and Carlini, N. Counterfactual memorization in neural language models. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023.
12
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
A. Proofs A.1. Proof of Proposition 3.1 Proof. Conditioning on Dtrain fixes the trained model vθ as a deterministic function. Define g : Rd → Rd by g(x) = vθ (x, λ) − v ∗ (x, λ). (j)
(j)
Each test sample (x̃0 , x̃1 ) is drawn i.i.d. from p0 × p1 , independently of Dtrain . By the orthogonality property (4): Ep0 ×p1 [⟨g(Xλ ), V − v ∗ (Xλ , λ)⟩] = 0
(31)
(j) (j) E ⟨vθ (x̃λ , λ) − v ∗ (x̃λ , λ), ϵ̃j (λ)⟩ | Dtrain = 0
(32)
Therefore, for each test sample j:
train By linearity: EDtest [Gtest ] = 0. m (λ) | D
On training data, this argument does not apply: both vθ and {ϵi (λ)}ni=1 depend on Dtrain , so g is not independent of the residuals. A.2. Proof of Proposition 4.1: Critical point of cross-covariance Proof. Expand ∥C(λ)∥2F = ∥λΣ1 − (1 − λ)Σ0 ∥2F : ∥C(λ)∥2F = λ2 tr(Σ21 ) + (1 − λ)2 tr(Σ20 ) − 2λ(1 − λ)tr(Σ0 Σ1 )
(33)
Rearranging: ∥C(λ)∥2F = λ2 tr((Σ0 + Σ1 )2 ) − 2λ tr(Σ20 ) + tr(Σ0 Σ1 ) + tr(Σ20 )
(34)
This is a convex parabola with a positive leading coefficient. The minimum is at λ∗F . A.3. Proof of Theorem 4.2: Isotropic Gaussian Case We provide a complete analysis of the expected train-test gap in the isotropic Gaussian setting. A.3.1. S ETUP Assumption A.1 (Isotropic Gaussian). Let X0 ∼ N (0, σ02 Id ) and X1 ∼ N (0, σ12 Id ) be independent, with σ0 , σ1 > 0. Define Xλ = (1 − λ)X0 + λX1 and V = X1 − X0 . In the isotropic case, the covariance matrices from Section 3 reduce to scalars times the identity: Φ(λ) = ϕ(λ)Id C(λ) = c(λ)Id
where
ϕ(λ) = (1 − λ)2 σ02 + λ2 σ12
(35)
where
c(λ) = λσ12 − (1 − λ)σ02
(36)
Since the covariances are isotropic, all coordinates are independent and identically distributed. We analyse a single coordinate j, then sum over d coordinates. For coordinate j: Xλ,j ∼ N (0, ϕ(λ)) Vj ∼ N (0, σ02 + σ12 ) Cov(Vj , Xλ,j ) = c(λ) 13
(37) (38) (39)
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
A.3.2. O PTIMAL P REDICTOR Since (Xλ,j , Vj ) is jointly Gaussian with zero means, the conditional expectation is linear: vj∗ (x, λ) = E[Vj | Xλ,j = x] = a(λ) · x where: a(λ) =
Cov(Vj , Xλ,j ) c(λ) = Var(Xλ,j ) ϕ(λ)
(40)
(41)
A.3.3. I RREDUCIBLE VARIANCE Lemma A.2 (Irreducible variance). Under Assumption A.1: c(λ)2 2 2 2 σirr (λ) = d σ0 + σ1 − ϕ(λ)
(42)
Proof. The residual for coordinate j is ϵj (λ) = Vj − a(λ)Xλ,j . Its variance is: Var(ϵj (λ)) = Var(Vj ) − 2a(λ)Cov(Vj , Xλ,j ) + a(λ)2 Var(Xλ,j )
(43)
2
c(λ) c(λ) · ϕ(λ) · c(λ) + ϕ(λ) ϕ(λ)2 c(λ)2 = σ02 + σ12 − ≜ σϵ2 (λ) ϕ(λ) = (σ02 + σ12 ) − 2
(44) (45)
2 Summing over d independent coordinates gives σirr (λ) = d · σϵ2 (λ).
A.3.4. OLS E STIMATOR (i)
(i)
For each coordinate j, we have n i.i.d. samples (xλ,j , vj )ni=1 with the model: (i)
(i)
(i)
vj = a(λ)xλ,j + ϵj (λ) (i)
(i)
(46)
(i)
where ϵj (λ) ∼ N (0, σϵ2 (λ)) and ϵj (λ) ⊥ xλ,j (by Gaussianity). This is a univariate linear regression without intercept. The OLS estimator is: Pn Pn (i) (i) (i) (i) i=1 vj xλ,j i=1 vj xλ,j â = Pn = (i) 2 S i=1 (xλ,j ) where S ≜
(47)
(i) 2 i=1 (xλ,j ) .
Pn
(i)
(i)
(i)
Substituting vj = a(λ)xλ,j + ϵj (λ): P â = a(λ) +
(i) (i) i ϵj (λ)xλ,j
S
(48)
A.3.5. D ISTRIBUTION OF S (i) (i) p Since xλ,j ∼ N (0, ϕ(λ)), we have xλ,j / ϕ(λ) ∼ N (0, 1). Therefore: n X S = ϕ(λ) i=1
(i)
x p λ,j ϕ(λ)
!2 ∼ χ2n
For Y ∼ χ2n with n > 2, a standard result gives E[1/Y ] = 1/(n − 2). Hence: 1 1 1 1 E = ·E 2 = S ϕ(λ) χn ϕ(λ)(n − 2) 14
(49)
(50)
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
A.3.6. E XPECTED T RAINING L OSS Lemma A.3 (Expected training loss per coordinate). For a single coordinate j: E[Ltrain,j (λ)] = σϵ2 (λ) ·
n−1 n
(51)
Proof. This is a standard result for OLS regression. For a model with p parameters, the expected residual sum of squares satisfies: " n # X (i) (i) 2 E (vj − âxλ,j ) = (n − p)σϵ2 (λ) (52) i=1
Here p = 1 (single parameter, no intercept), so: E[n · Ltrain,j (λ)] = (n − 1)σϵ2 (λ)
(53)
which gives E[Ltrain,j (λ)] = σϵ2 (λ) · n−1 n . A.3.7. E XPECTED T EST L OSS Lemma A.4 (Expected test loss per coordinate). For a single coordinate j: n−1 n−2
(54)
new vjnew = a(λ)xnew (λ) λ,j + ϵj
(55)
E[Ltest,j (λ)] = σϵ2 (λ) · new Proof. For a new test point (xnew ) independent of Dtrain : λ,j , vj
The test loss (conditional on Dtrain ) is: 2 train Ltest,j (λ) = Enew [(vjnew − âxnew ] λ,j ) | D new = Enew [((a(λ) − â)xnew (λ))2 | Dtrain ] λ,j + ϵj
(56) (57)
new Since xnew (λ) and both are centred: λ,j ⊥ ϵj
Ltest,j (λ) = (â − a(λ))2 · ϕ(λ) + σϵ2 (λ)
(58)
E[Ltest,j (λ)] = ϕ(λ) · E[(â − a(λ))2 ] + σϵ2 (λ)
(59)
Taking expectations over Dtrain :
P (i) (i) (i) We now compute E[(â − a(λ))2 ]. Conditionally on (xλ,j )ni=1 , the numerator i ϵj (λ)xλ,j is Gaussian with mean 0 and variance: ! X (i) X (i) (i) Var ϵj (λ)xλ,j | X = (xλ,j )2 · σϵ2 (λ) = S · σϵ2 (λ) (60) i
i
Therefore: E[(â − a(λ))2 | X] =
σ 2 (λ) σϵ2 (λ) · S = ϵ 2 S S
(61)
Taking expectations over X and using (50): E[(â − a(λ))2 ] = σϵ2 (λ) · E
1 σϵ2 (λ) = S ϕ(λ)(n − 2)
(62)
Substituting into (59): E[Ltest,j (λ)] = ϕ(λ) ·
σϵ2 (λ) n−1 + σϵ2 (λ) = σϵ2 (λ) · ϕ(λ)(n − 2) n−2
15
(63)
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
A.3.8. F ROM G AP TO Gtrain n Lemma A.5 (Expected gap per coordinate). For a single coordinate j: E[∆j (λ)] ≜ E[Ltest,j (λ)] − E[Ltrain,j (λ)] = σϵ2 (λ) ·
2(n − 1) n(n − 2)
(64)
Proof. n−1 n−1 − σϵ2 (λ) · n−2 n 1 1 2 − = σϵ (λ)(n − 1) n−2 n 2 2(n − 1) = σϵ2 (λ)(n − 1) · = σϵ2 (λ) · n(n − 2) n(n − 2)
E[∆j (λ)] = σϵ2 (λ) ·
(65) (66) (67)
We now connect this gap to Gtrain (λ). From the loss decomposition (9) in Section 3.3: n Ltrain (λ) = Entrain (λ) + σ̂n2 (λ) − 2Gtrain (λ) n
(68)
For OLS on Gaussian data, Assumptions 3.2 and 3.3 hold in expectation: • The OLS estimator is unbiased, so E[Entrain (λ)] = E[E test (λ)] 2 • By the law of large numbers, E[σ̂n2 (λ)] = σirr (λ)
Similarly, for test data, Proposition 3.1 gives E[Gtest m (λ)] = 0, so: 2 E[Ltest (λ)] = E[E test (λ)] + σirr (λ)
(69)
E[∆(λ)] = E[Ltest (λ) − Ltrain (λ)] = 2E[Gtrain (λ)] n
(70)
Taking the difference:
A.3.9. M AIN R ESULT Proof of Theorem 4.2. From Lemma A.5 and the relation (70): E[Gtrain n,j (λ)] =
1 n−1 E[∆j (λ)] = σϵ2 (λ) · 2 n(n − 2)
(71)
Since the d coordinates are independent: E[Gtrain (λ)] = n
d X
2 E[Gtrain n,j (λ)] = d · σϵ (λ) ·
j=1
n−1 n−1 2 = σirr (λ) · n(n − 2) n(n − 2)
(72)
A.4. Proof of Corollary 4.3: Peak at minimal linear information) n−1 2 Proof. Since n(n−2) > 0 for n > 2, maximising E[Gtrain (λ)] is equivalent to maximising σirr (λ). From Theorem 4.2: n c(λ)2 2 σirr (λ) = d σ02 + σ12 − (73) ϕ(λ)
Since ϕ(λ) > 0, this is maximised when c(λ) = 0. Solving c(λ) = λσ12 − (1 − λ)σ02 = 0 gives λ∗ = σ02 /(σ02 + σ12 ). In the isotropic case, ∥C(λ)∥2F = d · c(λ)2 , so λ∗ coincides with λ∗F from Proposition 4.1. 16
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
A.5. Proof of Corollary 4.4: Boundary behavior) Proof. At λ = 0: c(0)2 /ϕ(0) = σ04 /σ02 = σ02 . At λ = 1: c(1)2 /ϕ(1) = σ14 /σ12 = σ12 . At λ∗ : c(λ∗ ) = 0, so c(λ∗ )2 /ϕ(λ∗ ) = 0. 2 Since σirr (λ) = d(σ02 + σ12 − c(λ)2 /ϕ(λ)), it is minimised when c(λ)2 /ϕ(λ) is maximised, which occurs at the boundaries.
Corollary A.6 (Boundary and peak values). 2 σirr (0) = dσ12
(74)
2 σirr (1) = dσ02 2 σirr (λ∗ ) = d(σ02 + σ12 )
(75) (76)
2 2 2 When σ0 = σ1 , we have λ∗ = 1/2 and σirr (λ∗ ) = 2σirr (0) = 2σirr (1). 2 Proof. At λ = 0: c(0) = −σ02 , ϕ(0) = σ02 , so c(0)2 /ϕ(0) = σ02 and σirr (0) = d(σ02 + σ12 − σ02 ) = dσ12 . 2 At λ = 1: c(1) = σ12 , ϕ(1) = σ12 , so c(1)2 /ϕ(1) = σ12 and σirr (1) = d(σ02 + σ12 − σ12 ) = dσ02 . 2 At λ∗ : c(λ∗ ) = 0, so σirr (λ∗ ) = d(σ02 + σ12 ).
Corollary A.7 (Asymptotics). For large n: E[Gtrain (λ)] ≈ n
2 σirr (λ) n
(77)
A.6. Proof of Proposition 4.6: Shared statistics or r and ϵ Proof. For r: (A(λ), b(λ)) minimise E[∥v ∗ − Ax − b∥2 ]. The first-order conditions yield E[r] = 0 and E[r · Xλ⊤ ] = 0. For ϵ: By the definition of conditional expectation, E[ϵ|Xλ ] = 0, which implies E[ϵ] = 0 and E[ϵ · Xλ⊤ ] = 0.
B. Ablations details B.1. Datasets B.1.1. MAESTRO V 3 MAESTRO v3 (MIDI and Audio Edited for Synchronous TRacks and Organisation) (Hawthorne et al., 2019) contains approximately 200 hours of classical piano performances recorded during international piano competitions. The dataset comprises 1,282 compositions divided into train (967 pieces, 154h), validation (137 pieces, 20h), and test (178 pieces, 26h) partitions, representing a 76%/11%/13% split. Technical characteristics. Audio is provided as uncompressed WAV, 16-bit PCM at 44.1 kHz (some tracks at 48 kHz). The total size is approximately 120 GB. The repertoire spans classical music from the baroque to contemporary periods (Bach, Mozart, Beethoven, Chopin, Liszt, Debussy, etc.), with homogeneous professional studio recording quality. Preprocessing. Audio files are resampled to 44.1 kHz mono and segmented into non-overlapping 5 second chunks; partial chunks shorter than 5 seconds are discarded. Each chunk is encoded using Music2Latent (Pasini et al., 2024), yielding latents of dimension 64 × 50 (64 channels at 10 Hz temporal resolution). We apply z-score normalisation per channel, with statistics computed on the training set and applied to all splits to prevent data leakage. Split methodology. The split ensures no composition appears in multiple subsets, even when performed by different pianists. This prevents data leakage at the composition level and ensures train and test sets share the same musical distribution, satisfying Assumption 3.3. The homogeneity of the dataset (classical piano only) makes it well-suited for studying memorisation, as the concentrated distribution leaves stronger per-sample imprints. 17
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
B.1.2. MTG-JAMENDO MTG-Jamendo (Bogdanov et al., 2019) contains over 55,000 tracks representing approximately 3,777 hours of music. The dataset covers approximately 16,000 unique artists and 18,000 albums from more than 150 countries. Tracks come from the Jamendo platform under Creative Commons licences. We use the official genre-split-0 with a 60%/20%/20% train/validation/test partition. Technical characteristics. Audio is provided as MP3 at 320 kbps, with a variable sample rate (mainly 44.1 kHz). The total size is approximately 500 GB. The dataset spans productions from 2005 to 2020 across all contemporary genres (electronic, rock, pop, jazz, hip-hop, folk, metal, etc.). Unlike MAESTRO, the production quality varies from home-studio to professional recordings. The dataset includes hierarchical multi-label annotations: 87 genres, 40 instruments, and 56 mood/theme tags. Preprocessing. Same pipeline as MAESTRO: resampling to 44.1 kHz mono, segmentation into 5-second chunks, Music2Latent encoding to 64 × 50 latents, and z-score normalisation per channel with training set statistics. Split methodology. The split provided uses random sampling with artist stratification only: no artist appears in multiple subsets, ensuring the model is evaluated on artists unseen during training. To satisfy Assumption 3.3, we performed subsampling on the train and test sets with genre stratification, ensuring balanced genre proportions across splits. B.1.3. F REE M USIC A RCHIVE (FMA) FMA Large (Defferrard et al., 2017b) contains 106,574 clips of 30 seconds each, representing approximately 883 hours of music under a Creative Commons licence. The dataset is organised into subsets of increasing size; we use FMA Large for maximum diversity, which included 161 different genres. Technical characteristics. Audio is provided as MP3 at a constant 320 kbps, with a variable sample rate (mainly 44.1 kHz). The total size is approximately 93 GB. Clips are central excerpts from complete tracks, spanning productions from 2006 to 2017. Quality varies across independent productions but is generally good. Preprocessing. Same pipeline as MAESTRO: resampling to 44.1 kHz mono, segmentation into 5-second chunks, Music2Latent encoding to 64 × 50 latents, and z-score normalisation per channel with training set statistics. Split methodology. The official split uses genre stratification with artist separation: (1) genre proportions are maintained across train/validation/test, and (2) no artist appears in multiple sets. This controlled, genre-balanced design contrasts with MTG-Jamendo’s natural distribution and satisfies Assumption 3.3. B.1.4. C ELEBA CelebA (CelebFaces Attributes) (Liu et al., 2015) contains 202,599 celebrity face images, each annotated with 40 binary facial attributes (e.g., Male, Smiling, Eyeglasses, Young). We use the official split from Hugging Face (flwrlabs/celeba): 162,770 train, 19,867 validation, and 19,962 test images. Technical characteristics. Original images are JPEG at approximately 178×218 pixels. The dataset provides binary labels for 40 attributes covering facial features, accessories, and demographics. Preprocessing. Images are resized to 256×256 using bilinear interpolation followed by centre cropping. Pixel values are normalised to [−1, 1]. Each image is encoded using the Stable Diffusion VAE (sd-vae-ft-mse), yielding latents of dimension 4 × 32 × 32 (4 channels at 32×32 spatial resolution). Split methodology. The official split uses random partitioning of images; the same identity may appear in both the train and test sets. This does not violate Assumption 3.3, which requires the train and test sets to follow the same distribution p1 ; both are random samples from the same population. However, identity leakage may amplify the membership signal compared to stricter identity-based splits, as the model could encode identity-specific features that are shared across sets. 18
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
B.2. Latents B.2.1. M USIC 2L ATENT Music2Latent (Pasini et al., 2024) is a consistency autoencoder for audio compression, designed for efficient generative modelling and Music Information Retrieval (MIR) tasks. Unlike multi-stage approaches or slow iterative sampling methods, Music2Latent achieves high-fidelity single-step reconstruction through end-to-end training with a single consistency loss. Architecture. The model consists of three components: (1) an encoder that downsamples complex-valued STFT spectrograms into a sequence of 64-dimensional latent vectors, using tanh activation to constrain representations to [−1, 1]; (2) a decoder that upsamples latent vectors with cross connections to the consistency model; and (3) a consistency model based on the NCSN++ UNet architecture that reconstructs the original spectrogram. Key innovations include frequency-wise selfattention to capture long-range frequency dependencies and adaptive frequency scaling to handle varying value distributions across frequencies. Compression characteristics. Audio at 44.1 kHz is compressed to approximately 10 Hz temporal resolution with 64 channels, achieving a 4096× compression ratio. For our 5-second audio chunks, this yields latent representations of dimension 64 × 50. Relevance to our assumptions. Although Music2Latent is not a VAE and does not use explicit KL regularisation, the tanh activation constrains latent values to a bounded range [−1, 1]. Combined with the high compression ratio, this encourages approximately Gaussian marginal distributions in the latent space, as verified empirically in Table 1. The bounded symmetric activation discourages heavy tails and extreme correlations, supporting the approximate isotropy assumed in our theoretical analysis. B.2.2. S TABLE AUDIO VAE The Stable Audio VAE (Evans et al., 2024) is the autoencoder component of Stable Audio Open, a text-to-audio generation system developed by Stability AI. Unlike Music2Latent, which uses consistency models, this is a traditional variational autoencoder with explicit KL regularisation toward a Gaussian prior. Architecture. The model uses a fully-convolutional architecture (AutoencoderOobleck) based on the Descript Audio Codec encoder and decoder. The encoder compresses stereo waveforms at 44.1 kHz through five convolutional blocks with strided convolutions for downsampling. The bottleneck is parameterized as a VAE with a latent size of 64 channels. The decoder mirrors the encoder structure using transposed strided convolutions for upsampling. All convolutions are weight normalised. Training. The VAE is trained with three loss terms: (1) a reconstruction loss based on perceptually weighted multiresolution STFT, handling stereo via mid-side and left-right representations; (2) an adversarial loss with feature matching using 5 convolutional discriminators; and (3) a KL divergence loss regularising the latent distribution toward a standard Gaussian prior. Training was performed on approximately 486,000 audio recordings from Freesound and the Free Music Archive, all under Creative Commons licences. Compression characteristics. Audio at 44.1 kHz is compressed to a latent rate of 21.5 Hz with 64 channels. For our experiments, we convert audio to mono before encoding. Relevance to our assumptions. The explicit KL regularisation toward N (0, I) directly encourages the latent space to satisfy the Gaussian isotropic assumptions of our theoretical analysis. As shown in Table 1, the Stable Audio VAE latents exhibit low skewness (|γ| = 0.08), low excess kurtosis (|κ| = 0.10), and weak inter-dimension correlations (|ρ| = 0.16), confirming approximate Gaussianity and isotropy. B.2.3. S TABLE D IFFUSION VAE The Stable Diffusion VAE (sd-vae-ft-mse) (Blattmann et al., 2022) is the autoencoder component of the Stable Diffusion image generation system. We use the fine-tuned version released by Stability AI, which improves face reconstruction 19
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
compared to the original model. Architecture. The model is a KL-regularised autoencoder (kl-f8) with an 8× spatial downsampling factor. The encoder uses convolutional blocks with residual connections to compress images into a latent space with 4 channels. For 256×256 input images, this yields latent representations of dimension 4 × 32 × 32. The decoder mirrors the encoder structure using transposed convolutions for upsampling. Training. The original kl-f8 autoencoder was trained on OpenImages with L1 reconstruction loss, LPIPS perceptual loss, and KL divergence regularisation. The ft-mse variant was fine-tuned from this checkpoint on a 1:1 ratio of LAIONAesthetics and LAION-Humans datasets for an additional 280k steps, with increased emphasis on MSE reconstruction (MSE + 0.1 × LPIPS). This fine-tuning improves reconstruction quality, particularly for human faces. Compression characteristics. Images at 256×256 pixels are compressed to 4 × 32 × 32 latents, achieving a 48× compression ratio (from 256 × 256 × 3 = 196, 608 to 32 × 32 × 4 = 4, 096 values). Relevance to our assumptions. Despite the KL regularisation, the Stable Diffusion VAE latent space deviates significantly from the Gaussian isotropic assumptions. As shown in Table 1, CelebA latents encoded with this VAE exhibit high excess kurtosis (|κ| = 0.71), indicating heavy-tailed marginal distributions and strong inter-dimension correlations (|ρ| = 0.61). These violations explain why the peak prediction λ∗F fails to match the observed peak for this configuration (Table 2), while the bell-shaped curve still appears, confirming that the bell shape is universal but the closed-form peak location requires our assumptions to hold. B.3. Model Architectures We use two backbone architectures, a Transformer and a UNet, to verify that our findings are architecture-independent. Both are trained with the Rectified Flow objective (Liu et al., 2023; Lipman et al., 2023). B.3.1. T RANSFORMER (D I T) Our Transformer follows the Diffusion Transformer (DiT) architecture (Peebles & Xie, 2023) with several modifications for audio sequences. Architecture. The input sequence (B, C, T ) is first transposed to (B, T, C) and projected to the hidden dimension via a linear layer. Each Transformer block consists of: • Attention: Multi-head self-attention with Rotary Position Embeddings (RoPE) (Su et al., 2024) and Flash Attention (Dao et al., 2022) for efficiency. • MLP: Two-layer feedforward network with GELU activation (tanh approximation). • adaLN-Zero conditioning: Adaptive Layer Normalisation with six modulation parameters (scale, shift, and gate for both attention and MLP branches), initialised to zero for stable training. Time conditioning uses sinusoidal embeddings processed through a two-layer MLP with SiLU activation. The final layer applies adaLN modulation followed by a linear projection back to the input dimension. Configuration. For audio experiments, we use the 410M parameters configuration: hidden size 576, depth 24, 12 attention heads, and an MLP ratio of 4.0. Initialisation follows Xavier uniform for linear layers, with zero initialisation for all adaLN modulation layers and the final output projection. B.3.2. UN ET We implement UNet architectures for both 1D audio latents and 2D image latents, sharing the same structural design. Architecture.
The UNet follows a symmetric encoder-decoder structure with skip connections: 20
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
• Encoder: Sequence of ResBlocks at each resolution level, with strided convolutions for downsampling between levels. • Middle: ResBlock → Self-Attention → ResBlock at the lowest resolution. • Decoder: Sequence of ResBlocks with skip connections from the encoder, with nearest-neighbour upsampling followed by convolution between levels. Each ResBlock consists of: GroupNorm → SiLU → Conv → time conditioning → GroupNorm → SiLU → Dropout → Conv, with a residual connection. Time conditioning injects the timestep via scale and shift modulation: h ← h · (1 + scale) + shift, where scale and shift are produced by an MLP from the sinusoidal time embedding. Self-attention blocks use GroupNorm followed by multi-head attention. All output convolutions and attention projections are zero-initialised for stable training. Configuration.
For the medium configuration:
• UNet 1D (audio, 64 × T latents): base channels 192, channel multipliers (1, 2, 4, 4), 2 ResBlocks per level, attention at levels 2–3, dropout 0.1. • UNet 2D (images, 4 × 32 × 32 latents): base channels 192, channel multipliers (1, 2, 4, 4), 2 ResBlocks per level, attention at levels 1–3, no dropout. B.3.3. T RAINING All models are trained with the AdamW optimiser (β1 = 0.9, β2 = 0.999), a learning rate of 10−4 , mixed-precision (FP16/BF16), and gradient clipping set at 1.0. We use early stopping based on validation loss with a patience of 25 epochs. The batch size is 128 for Transformer models and 64 for UNet models. Relevance to our analysis. As shown in Table 2, both architectures yield nearly identical observed peak locations λobs on the same dataset, confirming that λ∗F depends on data geometry (Σ0 , Σ1 ) rather than model architecture or capacity.
C. Additional Metrics Analysis In the main text, we report the train-test gap using the mean reconstruction error. Our protocol evaluates each sample with K = 100 independent noise realisations, yielding a distribution of reconstruction errors per sample rather than a single value. This enables the computation of richer statistics, including the median, quartiles, and standard deviation, which provide robustness cheques. Here we examine these additional metrics to assess the robustness of our findings. Results are presented on MTG-Jamendo; similar patterns hold for MAESTRO v3 and FMA Large. C.1. Median and Quantile Metrics Figure 8 shows the normalised gap for the median and quartiles (q 0.25 , q 0.75 ). All three metrics exhibit bell-shaped curves nearly identical to the mean, with peaks at λ = 0.5 and boundary values approaching zero. This consistency across robust statistics confirms that the bell-shaped pattern is not driven by outliers. The membership signal is present throughout the distribution of reconstruction errors, not just in the tails. This robustness further supports our theoretical framework: the λ-dependent structure of the train-test gap is a fundamental property of the model, not a statistical artefact. C.2. Standard Deviation: An S-Shaped Pattern The standard deviation across the K = 100 noise samples reveals a qualitatively different pattern. As shown in Figure 8, instead of a bell curve, we observe an S-shaped curve: • For λ < 0.3: the gap is negative, meaning training samples exhibit higher variance in reconstruction error than test samples. • For λ > 0.3: the gap becomes positive, meaning training samples exhibit lower variance. 21
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Interpretation. We hypothesise that this pattern reflects sample-specific attractors in the learnt velocity field. At low λ (high noise), training samples may either be captured by their learnt attractor or missed entirely, producing high variance across noise realisations. Test samples, lacking specific attractors, consistently receive population-average predictions with lower variance. As λ increases and samples approach the data manifold, training samples reliably reach their attractors (low variance), while test samples show more variable behaviour. This interpretation remains speculative; a theoretical characterisation of higher-order statistics is left for future work.
Figure 8. Normalised gap for all metrics on MTG-Jamendo. Mean, median, and quartiles (q 0.25 , q 0.75 ) exhibit consistent bell-shaped curves. Standard deviation (σ) shows an S-shaped pattern. Similar patterns are observed on MAESTRO v3 and FMA Large.
D. Failure Modes and Relaxation of Assumptions D.1. Limits of Controlled Perturbations During the rebuttal period, we explored controlled transformations to test assumption boundaries: z 7→ sign(z)|z|p to modulate kurtosis and z 7→ (1 − α)z + α · mean(z) to inject inter-dimension correlations. However, these transformations introduce auxiliary artefacts and perturb multiple statistics simultaneously, making clean isolation impossible; therefore, we do not rely on them. We rely instead on naturally distinct configurations (different datasets and encoders), each producing their own (Σ0 , Σ1 ) pairs and degrees of assumption violation (Table 1). The CelebA / SD VAE configuration, with |ρ| = 0.61 and |κ| = 0.71, serves as our primary natural test case. Based on our theoretical analysis, we interpret the failure modes as follows. Non-Gaussianity introduces a nonlinear residual r(x, λ) (Section 4.3) that shifts the irreducible variance non-uniformly across λ, displacing the peak from λ∗F . Anisotropy causes tr(Σ21 ) in the denominator of Proposition 4.1 to be dominated by off-diagonal entries, pushing λ∗F downward, consistent with the CelebA case where λ∗F = 0.45 while λobs ∈ [0.6, 0.7]. In practice, we recommend that practitioners directly measure the bell-shaped curve on their dataset of interest: computing λ∗F requires only O(d2 ) trace estimations, and observing the empirical peak requires only forward passes at a grid of λ values. 22
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Checking whether the two agree is both inexpensive and more informative than any synthetic perturbation experiment. D.2. Relaxation: From λ∗F to λirr A natural alternative to the closed-form λ∗F is to numerically minimise
2 σirr (λ) = tr(ΣV ) − tr(C(λ)Φ(λ)−1 C(λ)⊤ )
(78)
over λ, yielding a λirr that does not require the isotropy assumption. However, for the high-dimensional latents commonly considered (d = 64 × 50 = 3200 for audio), reliably estimating and inverting Φ(λ) ∈ Rd×d from finite samples is, in practice, unstable, as audio chunks are temporally correlated and the effective sample size is much smaller than the nominal n. The closed-form λ∗F , in contrast, depends only on the traces of products of Σ0 and Σ1 , which can be estimated robustly in O(d2 ) time.
E. Membership Inference Attack Details Feature extraction. For each sample x1 , we compute reconstruction errors at each λ ∈ {0, 0.1, . . . , 1.0} using K = 100 independent noise realisations, and extract the per-λ mean, yielding an 11-dimensional feature vector.
Classifier and training. We train a small MLP (2 hidden layers, 64-32 units) on the λ-resolved features using binary cross-entropy and Adam (β1 = 0.9, β2 = 0.999), with early stopping on a held-out validation set. The architecture was selected via Bayesian optimisation over depth, width, and training duration.
Dataset construction. We partition the generative model’s training and held-out sets into two disjoint halves each, combining one half of each for MLP training and the other for evaluation, ensuring no sample appears in both. We use 1,000 samples per class for training and 500 for testing.
Results. Table 3 reports AUC and TPR@5%FPR across all datasets and baselines. Figure 9 shows the confusion matrix at threshold 0.38.
Table 3. MIA results across datasets. AUC with TPR@5%FPR (%) in parentheses. M ETHOD
MAESTRO V 3
MTG-JAMENDO
FMA L ARGE
C ELEBA
NAIVERF S EC MIRF PIARF
0.67 (14.1) 0.72 (13.9) 0.83 (36.5)
0.57 (6.0) 0.61 (11.0) 0.64 (10.2)
0.55 (4.8) 0.59 (8.4) 0.61 (9.3)
0.58 (8.0) 0.56 (4.3) 0.62 (14.0)
O URS
0.91 (56.7)
0.72 (23.4)
0.67 (19.0)
0.65 (15.0)
23
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Figure 9. Confusion matrix on MAESTRO at threshold 0.38. The classifier correctly identifies 82% of members and 84% of non-members.
F. Reflow: Preliminary Results The reflow procedure (Liu et al., 2023) replaces the independent coupling X0 ⊥⊥ X1 with learnt pairs obtained by integrating the trained velocity field forward from noise samples. This breaks the independence assumption underlying our theoretical analysis, and we conjecture it attenuates the membership signal by correlating the noise endpoint with the data.
Protocol. We train a single reflow step on MAESTRO v3, using the same Transformer architecture (410M parameters) and training hyperparameters as the baseline configuration (Section 5). The reflow pairs (x0 , x1 ) are obtained by integrating the baseline model forward from x0 ∼ N (0, Σ0 ).
Results. Figure 10 shows the normalised train-test gap ∆norm (λ) for the reflow model alongside the baseline. The bell-shaped structure persists, confirming that the phenomenon is not specific to the independent coupling. However, the peak magnitude decreases substantially (from 0.09 to 0.01), and the curve exhibits a broader, flatter plateau rather than a sharp peak. The peak location remains near λ∗F , consistent with the interpretation that the peak is governed by data geometry rather than the coupling procedure. These results also suggest that reflow may offer a natural mitigation of membership leakage as a byproduct of its trajectorystraightening objective; though a thorough characterisation is left for future work. 24
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Figure 10. Normalised train-test gap ∆norm (λ) for the baseline and reflow models on MAESTRO v3. The bell shape persists under reflow but with a substantially reduced magnitude and broader plateau.
25