Amplifying Membership Signal Through Chained Regeneration
arXiv:2606.31991v1 [cs.LG] 30 Jun 2026
Wojciech Łapacz∗ † Warsaw University of Technology
Stanisław Pawlak∗ Warsaw University of Technology
Abstract The tendency of large generative models to memorize training data makes sample verification critical for privacy auditing and copyright enforcement. Current membership (MIA) and dataset inference (DI) attacks often rely on one-shot generations, which yield weak signals and limited sensitivity across modalities. Inspired by Model Autophagy Disorder (MAD), we introduce MADreMIA, a model-agnostic framework that enhances white-, gray-, and black-box MIA and DI. Rather than relying on shadow model training – often infeasible for large generative models – our framework facilitates scalable inference by leveraging inherent signals through iterative trajectories. This process utilizes chained generations across diverse modalities, where each output serves as the subsequent input, to improve membership evidence at low FPR. We demonstrate that memorized training samples exhibit significantly higher coherence and slower degradation during iterative regeneration than non-member generations. Our results show that MADreMIA provides richer signals across diverse model families and modalities; we present comprehensive evaluations for IARs, diffusion, and language models, alongside preliminary results demonstrating its potential for audio models.
1
Introduction
The rapid development of generative AI triggered a pressing demand for training data, frequently leading to the unauthorized ingestion of private, sensitive, or copyrighted content. Consequently, with the scaling of generative models the importance of Membership inference attacks (MIAs) [28] and dataset inference (DI) [23] has become critical. Practical auditing – ranging from protecting medical privacy [49] to identifying licensed content [12] or detecting benchmark contamination [24, 30, 44] – requires determining whether specific samples or datasets were used to shape a model’s parameters. The definitive test is whether a model retains a structural ”echo” of its training data, manifesting itself as a high-fidelity memorization signal that can be surfaced through targeted inference. Existing auditing methods, however, face a significant bottleneck. Most extract evidence from a single query [46, 39] or a set of loosely coupled samples [8]. These one-shot signals are often fragile; recent evaluations on unbiased benchmarks show that many MIAs degrade significantly under distributional shifts, often performing only slightly better than random guessing [24, 12]. Furthermore, highperformance ”shadow model” attacks [41, 5] – which require training multiple auxiliary models to simulate the target – are computationally expensive and impractical for real-world large-scale generative architectures. To address these limitations, we shift the perspective from a single static query to a dynamic trajectory. This concept is best illustrated through a forensic parallel: in a criminal interrogation, a suspect may maintain a lie for a single response, but that lie often collapses under the pressure of repeated, recursive follow-up questions. A truthful narrative, by contrast, remains coherent because it is ∗ Equal contribution. † Contact: [email protected]
Preprint.
Standard MIA
Chained Regeneration
•••
Generation N
Generation 2 Generation 2
Non-member ✗
Generation 1
Member ✓
Generation 1
?
Member
Output
•••
Non-member
Target Model
✓
Generation N
Consistent
Input
✗
Inconsistent
Figure 1: Comparison between conventional one-shot membership inference attack and our chainedgeneration approach. The former use a single query, which yields a weak signal that often fails to separate members from non-members. In the latter, each generation informs the next query, progressively amplifying membership evidence and improving separability: re-members ✓ are more coherent and degrade slower than re-non-members × . grounded in a fixed reality. We argue that generative models exhibit a similar phenomenon – their ”truth” is the training set. While a model can produce a plausible-looking output for a non-member sample once, it may struggle to sustain that plausibility over a recursive chain of self-generated inputs. Our framework, MADreMIA, is inspired by the mechanics of Model Autophagy Disorder (MAD) [1, 29]. Traditionally, MAD describes a failure mode where models trained on their own synthetic outputs progressively lose variance and collapse into a state of degenerated ”madness”. We pivot this phenomenon into a diagnostic, interference-time tool: if a sample was present during training, it acts as a stable “attractor” in the model’s latent space. By repeatedly feeding a model’s outputs back into itself – creating an iterative regeneration chain – we can amplify the signal of memorization. Within this framework, we distinguish between two types of trajectories: • Re-members : These are member samples (training data) that are iteratively re-generated. Because the model has ”memorized” these points, they exhibit high stability and slow semantic degradation over time. • Re-non-members : These are unseen samples that are iteratively re-generated. Lacking a structural anchor in the model’s weights, these samples drift rapidly toward the model’s average biases or dissolve into noise (see Figure 1). MADreMIA functions as a modular, inference-time add-on that is intentionally method-, model-, and modality-agnostic. By measuring consistency across recursive loops, we provide richer signals across diverse architectures, including image autoregressive models (IARs), diffusion models (DMs), large language models (LLMs), and audio voice conversion models. We demonstrate that while a single output is often too noisy to be decisive, the trajectory of a “re-member” is different than “re-non-member” and thus acts as a powerful signal amplifier, surfacing traces of training data that are otherwise invisible. MADreMIA iterative procedure moves beyond one-shot plausibility by probing whether the model preserves semantic and structural consistency under repeated self-interaction. Consequently, this work investigates a central research question: Can the dynamics of recursive self-generation serve as a signal amplifier to expose training data membership? In summary, the main contributions of our paper are: • We introduce an iterative regeneration setup to uncover data memorization invisible during single-pass inference. • We show theoretically and empirically that trajectory features (generation dynamics over time) yield a significantly more statistically robust membership signal. By functioning as a 2
variance reduction mechanism, these features isolate the underlying membership information much more effectively than standard one-shot baselines. • We propose an inference-time, cross-modal framework that improves Membership and Dataset Inference efficiency across Vision and Language models without the need for expensive shadow model training.
2
Related Works
Memorization. Memorization in generative models — the tendency to reproduce training examples rather than generate novel samples — has been studied across multiple model families. Early work formalized the distinction between memorization, mode collapse, and overfitting [35], while subsequent studies characterized the generalization-to-memorization transition in diffusion models [15], localized it through attention patterns [26], and showed that standard evaluation metrics fail to surface it [2]. Mitigation strategies have been proposed for both LLMs [16] and text-to-image models [7]. Membership and Dataset Inference. Individual Membership Inference Attacks (MIAs) can be confounded by distribution shifts [24], prompting a shift toward Dataset Inference, which aggregates evidence across many samples [22, 12, 19]. Shadow-model approaches [41, 5] are now computationally infeasible for large architectures, so modern attacks extract signals from limited black-box outputs [46, 6, 32]. Most relevant to our work, Li et al. [20] performs MIAs on diffusion models by repeatedly perturbing a target image and comparing averaged outputs to the original — but since queries are independent and do not evolve with model responses, deeper structural memorization remains unexploited. Model Collapse. Recursive self-training in generative models leads to progressive quality and diversity degradation when insufficient real data is injected — a phenomenon termed Model Autophagy Disorder [1]. Training on model-generated data further causes tails of the original distribution to disappear [29]. Together, these works suggest that iterative generation is structurally revealing: memorized regions may persist differently from non-member examples under repeated reuse. Our method turns this insight into a privacy-auditing mechanism, exploiting chained regeneration at inference time to amplify membership-relevant differences rather than treating collapse as a training-time pathology. The extended related works section can be found in Appendix E.
3
Theory of Trajectory-Based Signal Amplification
For each sample, we define an iterative trajectory Z0 , Z1 , . . . , ZT , where Z0 is the observed sample and Zt+1 is produced by one regeneration step. Let M ∈ {0, 1} denote membership. Define a PT −1 per-step score ϕt := ϕ(Zt , Zt+1 ) and the average ST := T1 t=0 ϕt . The attack predicts M from ST . We use aT ≳ bT when aT ≥ c bT for a constant c > 0 independent of T , and aT ≍ bT for two-sided bounds. Assumption 3.1 (Signal and Noise). (A1) There exists a sequence (∆t ≥ 0) such that E[ϕt | M = 1] − E[ϕt | M = 0] ≥ ∆t . (A2) maxm supt Var(ϕt | M = m) ≤ σ 2 < ∞. (A3) The centered process ϕ̃t := ϕt − E[ϕt | M ] is geometrically mixing with effective autocorrelation time τeff , 2 implying Var(ST | M ) ≤ C σ Tτeff . Theorem 3.2 (Trajectory Averaging). Under A1–A3, the signal ΓT := |E[ST | M = 1] − E[ST | M = 0]| and SNR satisfy: P T −1 (1 ∆t ) 2 1 X Γ2T ΓT ≥ ∆t , SNR2 (ST ) := ≥ T 2 . T t=0 maxm Var(ST | M = m) Cσ τeff /T Interpretation. Multi-step attacks improve when mean signal decays slowly relative to variance reduction. −T /τg Corollary 3.3 (Exponential Leakage). If ∆t = ∆0 e−t/τg , then ΓT ≥ ∆0 1−eT /τg . If ΓT ≍ −x 2
−T /τg
∆0 1−eT /τg , then SNR2 (ST ) ≳ g(T /τg ) where g(x) := (1−ex yields an optimal T ⋆ ≈ 1.2564 τg . 3
)
. The maximizer x⋆ ≈ 1.2564
Corollary 3.4 (Amplification Gain). Let κ := τg /τeff . At T = T ⋆ , the gain over the single-step √ T⋆) baseline S1 is SNR(S SNR(S1 ) ≳ c κ, with c ≈ 0.638. It is worth noting that we do not claim that trajectory iteration increase the Bayes information ceiling I(M ; Z0 ); No, instead it improves practical fixed-form statistics via temporal variance reduction. This theory applies to any iterative protocol satisfying A1–A3. Theorem 3.2 provides a conditional amplification guarantee. We present proofs in the Appendix L.
4
Method
MADreMIA is a trajectory-augmentation framework for privacy inference on generative models. It is designed as an any-box extension of standard one-shot attacks (MIA/DI): black-box by default, gray-box when richer outputs are available, and white-box when needed. The central design principle is to keep the downstream scorer unchanged and improve only its input representation through additional trajectory-derived evidence. Unified setup.
Following Sec. 3, for each queried sample we construct Z0 , Z1 , . . . , ZT , Zt+1 = R(f, Zt ), t = 0, . . . , T − 1, where Z0 = x is the queried sample, f is the audited generator, and R is a modality-specific regeneration operator executed under a fixed protocol. For MIA, the label is M ∈ {0, 1} (member/nonmember). For DI, we use an analogous binary label D ∈ {0, 1} (in-target-dataset/out-of-targetdataset). Threat model. MADreMIA supports: black-box (query access to f outputs only), gray-box (query access plus output-level statistics such as loss/log-probability signals), and white-box (optional access to internals/gradients when available). In all cases, the adversary/auditor has no access to training data identities (labels), performs at most T regeneration steps per sample, and outputs a binary prediction via h: M for MIA or D for DI. PT −1 Base one-shot signal. The theory defines ϕt := ϕ(Zt , Zt+1 ), ST := T1 t=0 ϕt . A trajectory one-shot comparator corresponds to the T = 1 case (using ϕ0 = ϕ(Z0 , Z1 )). When available, we additionally report classical one-shot baselines zbase = ϕbase (Z0 ). Importantly, for each modality/model, the orientation (sign) of ϕt is fixed on train data only (equivalently ϕt or −ϕt ) and then frozen for test-time evaluation. Signals and Fusion. MADreMIA augments one-shot evidence with trajectory summaries computed from (Z0 , . . . , ZT ). We define zbase = ϕbase (Z0 ) ∈ Rd , ztraj = ψ(Z0 , . . . , ZT ) ∈ Rk , Here ψ aggregates temporal statistics aligned with the ϕt process (e.g., drift, consistency, quality −1 evolution, diversity, score decay, and summaries derived from {ϕt }Tt=0 and ST ). The fused representation is z̃ = [zbase ∥ztraj ] ∈ Rd+k , and the final attack score is s(Z0 ) = h(z̃), with h a calibrated scorer. By default, following Kowalczuk et al. [19], h is an L1-regularized logistic regression fit as a plug-in estimator of P (M = 1 | z̃). Mechanism. MADreMIA leverages the fact that members often exhibit slower average drift than non-members. Memorized samples typically lie in deeper local probability wells, causing iterative regenerations to remain closer to Z0 . Gains represent fixed-statistic SNR improvements consistent with the DPI: I(M ; z̃) ≤ I(M ; Z0 ). 4.1
Modality-specific instantiations
Image autoregressive models (IARs) and diffusion models. R is image-to-image regeneration under fixed controls (autoregressive decoding for IARs; controlled re-noise/re-denoise for diffusion, i.e., partial forward noising to a fixed noise level followed by reverse denoising under fixed scheduler/settings). Trajectory features are defined relative to Z0 , in particular MSE(Z0 , Zt ), LPIPS(Z0 , Zt ) [47], and SSIM(Z0 , Zt ) [37]. 4
Images (black-box)
0.3
6
8 10 Iteration
12
6
14
(a) RAR-XXL
4
6
8 10 Iteration
12
Non-Members Members
24 22 FID
16 14
20 18
12
16
10 2
3
4
5 6 7 Iteration
8
2
9 10
(c) DiT-MoE-G
4
6
8 10 Iteration
12
2
4
6
8 10 Iteration
12
14
(e) OLMo-7B
FID ( )
FID ( )
1
0.2
14
(b) VAR-d30
Non-Members Members
18
2
KLD
4
0.4
14
(d) UViT-T2I-Deep
0.80 0.75 0.70 0.65 0.60 0.55 0.50
0.60 0.55 0.50 0.45 0.40 0.35 0.30
Token Distribution Shift (KLD) ( )
30
20 15
2
4
6
8 10 Iteration
12
8 10 Iteration
12
6
14
(g) OPT-6.7B
8 10 Iteration
12
14
FAD ( )
Non-Members Members
0.70
35 30
0.65 0.60
Non-Members Members
25 20
0.50 6
4
(i) FreeVC
0.55 4
2
14
Token Distribution Shift (KLD) ( ) 0.75
Non-Members Members
2
Non-Members Members
25
(f) Pythia-6.9B
Token Distribution Shift (KLD) ( )
FAD ( )
35
Non-Members Members FAD
10
KLD
0.5
12 8
2
Non-Members Members
0.6 KLD
FID
FID
10 9
FID
Non-Members Members
14
11
Token Distribution Shift (KLD) ( )
FAD
16
Non-Members Members
12
Audio (black-box)
Text (grey-box)
FID ( )
KLD
FID ( )
2
4
6
8 10 Iteration
12
14
(h) Llama-13B
15
2
4
6
8 10 Iteration
12
14
(j) AutoVC
Figure 2: Divergence trajectories across chained regeneration steps. Rows represent image models (FID), audio models (FAD), and language models (KLD). Across modalities and access settings, member examples retain lower divergence and degrade more slowly than non-member examples , providing a robust signal for both membership and dataset inference. Evaluations were conducted using the following sample sizes: 10,000 for IAR and Diffusion models, 2,000 for Audio, 1,000 for OLMo, 512 for Pythia, and 250 for Llama and OPT. Large language models (LLMs). R is an autophagous text loop where each generation is fed back as the next prompt/input under a fixed template, fixed context-window policy (with left-sided truncation to keep only the newest text), and fixed decoding configuration. We use multiple features to measure the quality and diversity of generations, specifically: Kullback-Leibler Divergence, Jensen-Shannon Divergence, Jaccard Index, Predictive Entropy, and Logit Margin: KLD(Z0 , Zt ),
JSD(Z0 , Zt ),
Jaccard(Z0 , Zt ),
Entropy(Zt ),
LogitMargin(Zt ),
for t ∈ {1, . . . , T }. These are summarized along the trajectory and fused with zbase . For clarity, KLD/JSD are computed on aligned token-distribution vectors: in gray/white-box settings from next-token logits, and in black-box settings from smoothed empirical token-frequency distributions under a fixed tokenizer/vocabulary. In fact, metrics in our experiments follow the gray-box setting, but our framework itself is open to the black-box setting as well. A black-box setting requires repeated queries per step to estimate distributions. Jaccard is computed on token sets after the same fixed preprocessing. More information about features for vision and language models are provided in Appendix H. Audio generative models. In the audio domain, R employs iterative reconstruction loops. Notably, we do not conduct a full Membership or Dataset Inference evaluation for audio models, as the literature currently lacks proper audio benchmarks and specialized attacks tailored to the voice conversion setting. Nevertheless, to demonstrate the cross-modal generality of our framework, our first experiment explores this potential using an objective audio fidelity metric. Across all modalities, MADreMIA follows the pipeline: Z0 → (Z0:T ) → (ϕ0:T −1 , ST , ztraj ) → z̃ → s(Z0 ).
5
Experiments
5.1
Experimental Setup
To ensure a scientifically sound evaluation across our MIA tasks, we restrict our setup to models trained on public datasets with well-defined training and test splits. We evaluate our method across three diverse modalities to demonstrate its broad applicability. For image generation, we analyze SOTA autoregressive models (VAR-d{20, 24, 30} [33], RAR-{L, XL, XXL} [43]) and diffusion models (DiT-RF-{XL, G} [13], UViT-T2I-Deep [3]), trained primarily on the ImageNet [10] or COCO [36] datasets for class-conditioned and text-to-image generation. We extend this evaluation to the audio domain using modern Voice Conversion models (AutoVC [25], FreeVC [21]), and to the 5
language domain utilizing prominent LLMs (LLaMA-13B [34], Pythia-6.9B [4], OLMo-7B [14], and OPT-6.7B [48]). Comprehensive details regarding all specific models and datasets used in experiments are provided in the Appendix F and G. All experiments were conducted on a machine equipped with 3 NVIDIA RTX PRO 5000 Blackwell GPUs (48 GB VRAM each) and an Intel Xeon Gold 6526Y CPU. 5.2
Metrics
To measure similarity between feature representations and their fidelity, we utilize the Fréchet Inception Distance (FID) [17], and Fréchet Audio Distance (FAD) [18] for vision and audio models, respectively. For LLMs, we measure Token Diversity as the Kullback–Leibler Divergence (KLD) between the normalized average token probability distribution at the current iteration and that of the first evaluation iteration: Token Diversity at iteration t (for t > 1) is defined as the Kullback-Leibler divergence from iteration 1: X pt (i) TokenDiversity(t) = DKL (pt ∥ p1 ) = pt (i) log . p1 (i) i∈V
where pt and p1 are the normalized average token probability distributions for step t and step 1 respectively. 5.3
MIA and DI procedures
MIA pipeline. For each labeled member/non-member sample, we generate Z0 , . . . , ZT , compute ϕt , ST , and modality-specific trajectory features, form z̃, and fit hmia . We evaluate univariate trajectory statistics by direct thresholding and multivariate features by logistic-regression fusion on strictly stratified 80/20 train-test splits. We report AUC, TPR at 1% FPR, and accuracy. Splitting is performed at sample/source level before trajectory generation: all descendants of the same Z0 (all Zt , all derived features) remain in the same partition. Thresholds, feature normalization, and LR calibration are fit on train only and applied unchanged to test. Primary endpoint is the multivariate fusion score; univariate ST⋆ results are reported as theory-aligned diagnostics. If T is tuned, it is selected on train (or a train-only validation split) and never on test. We use established metrics: TPR@FPR=1%, AUC, and Accuracy. DI pipeline. The DI pipeline is identical, replacing the target label with dataset-origin variable D. The same Zt , ϕt , and trajectory-fusion machinery is used; only label semantics and calibration change. For DI, splitting/evaluation are performed at dataset or source-group level, and per-sample logits are aggregated by a fixed mean rule into a dataset-level score. Dataset-level decisions are evaluated against a permutation-based null over dataset labels within the evaluation fold. Both MIA and DI setups inherit standard generative privacy-audit conventions, including the IAR setting introduced in [19]. 5.4
Research questions
We evaluate whether chained regeneration can be a signal amplifier for one-shot auditing across modalities, model families, and access regimes. Our analysis focuses on the following questions: (Q1) What distinguishes member/non-member chained generation trajectories? (Q2) Can one-shot membership signal be amplified for single ϕ(t) features? What are the gains for trajectory-based ST over ϕ(t) across modalities? (Q3) Does MADreMIA increase member/non-member separability compared to one-shot MIA? (Q4) Does increasing generative model stochasticity during regeneration loop affect the trajectories separation between members and non-members? (Q5) How does model size affect member/nonmember trajectory signals? Finally, we also provide a short analysis of the Getty Images case [9] in Appendix K. 5.5
Members and Nonmembers differ in generative trajectories: qaulitative results.
Across all modalities, members and non-members exhibit distinct regeneration dynamics. Members preserve structure longer and drift more slowly, while non-members degrade faster and diverge toward the model’s generic prior. This pattern is visible both in per-step qualitative examples (Figures 3a 6
Real
Iter. 2
Iter. 4
Iter. 6
Iter. 8
Iter. 10
Real
(a) Members.
Iter. 2
Iter. 4
Iter. 6
Iter. 8
Iter. 10
(b) Non-members.
Figure 3: Qualitative comparison of members and non-members across iterative regeneration (VAR-d30). Non-member images quality degrades faster than members, whose semantic coherence is largely preserved across regenerations. and 3b) and in aggregate divergence trajectories (Figure 2) comparing the quality of regenerations to base samples (FID for images, FAD for audio) and the drift of output token distribution in text model. The results presented support the core hypothesis that auto-regeneration trajectory contains multiple membership cues. The key trajectory asymmetry findings are: 1. Fidelity and degradation: Re-members maintain high structural quality throughout the trajectory, whereas re-non-members exhibit rapid perceptual and semantic degradation. 2. Persistence and divergence: Re-members demonstrate significant structural persistence and coherence across iterations. Conversely, re-non-members diverge more quickly, drifting toward the model’s general distribution and losing the specific characteristics of the original input. The asymmetry is present across diverse models and modalities. We test broad architectural diversity: image autoregressive and diffusion models, audio voice conversion/generation models, and text generative models. Figure 2 summarizes trajectory behavior using modality-appropriate divergence metrics 5.2. This design directly tests whether our proposed signal amplification is modeland modality-agnostic. 5.6
ϕ(t) statistics may increase membership signal over one-shot ϕ(0).
We evaluate the validity of our theoretical assumptions using empirical generative trajectories, fixing T to the first 15 iterations. As summarized in Table 2, while Assumption A2 is fully supported, A1 and A3 receive only partial empirical backing. Specifically, for certain values of ϕt , the absence of clear exponential decay within the first 15 iterations is acceptable for our main claim, since it indicates slower or plateau-like leakage. It suggests that non-exponential leakage forms may also govern real trajectories. To assess the efficacy of modality-specific trajectory statistics, we evaluate whether aggregated trajectory evidence remains competitive with - or outperforms - the one-shot evidence. We define 2 T SNR (ST ) gain := max , and show results in Table 1. Trajectory diagnostics are strong: P (gain ≥ maxt SNR2 (ϕt ) 8 1) = 11 = 0.73, P (gain ≥ 0.9) = 10 11 = 0.91, with median gain = 1.00. Given the small number of tested features, we interpret these numbers as supportive preliminary evidence. 5.7
MADreMIA amplifies baseline MIA
Tables 3 and 4 compare MADreMIA-augmented attacks against their unaided baselines across LLMs and IARs. Across all base attacks and model families, incorporating reconstruction Diversity (M SEsum , LP IP Ssum ), Quality (SSIMsum , SSIMstd ), or both (Combined) consistently raises attack performance. Gains are most pronounced on OLMo-7B, where, for example, the Zlib baseline collapses to AUC 0.179 yet recovers to 0.868 with Combined signals, and CAMIA reaches AUC 0.969 — the strongest result across all settings. On the remaining LLMs the improvements are 7
Table 1: ST gains over ϕt across modalities. P (gain ≥ 1) indicates the fraction of models where scoring matches or exceeds the baseline.
Table 2: Assumption support across model families. Fractions indicate the number of models satisfying each assumption.
Family
n
P (≥ 1)
P (≥ 0.9)
Median
Family
A1
A2
A3
VAR Diffusion LLM
3 3 5
0.67 0.67 0.80
1.00 1.00 0.80
1.00 1.00 1.04
VAR Diffusion LLM
3/3 3/3 3/5
3/3 3/3 5/5
3/3 2/3 2/5
Table 3: MIA results on established LLM benchmarks (described in detail in Appendix G), where MADreMIA trajectory features are aggregated across 15 iterations. Augmenting any base attack with diversity, quality, or combined signals consistently improves all the metrics over the unaided baselines. Pythia-6.9B
OLMo-7B
OPT-6.7B
Llama-13B
Attack
TPR@1%FPR
AUC
TPR@1%FPR
AUC
TPR@1%FPR
AUC
TPR@1%FPR
AUC
Loss [42] + Diversity + Quality + Combined
0.004 ±0.00 0.093 ±0.06 0.096 ±0.07 0.100 ±0.08
0.349 ±0.02 0.647 ±0.05 0.686 ±0.05 0.673 ±0.06
0.008 ±0.01 0.303 ±0.09 0.032 ±0.04 0.263 ±0.14
0.523 ±0.02 0.735 ±0.04 0.702 ±0.04 0.804 ±0.03
0.013 ±0.01 0.092 ±0.12 0.084 ±0.09 0.112 ±0.12
0.390 ±0.04 0.613 ±0.09 0.652 ±0.07 0.672 ±0.09
0.009 ±0.01 0.173 ±0.14 0.198 ±0.13 0.188 ±0.15
0.368 ±0.04 0.690 ±0.08 0.679 ±0.09 0.702 ±0.07
Zlib [5]) + Diversity + Quality + Combined
0.000 ±0.00 0.129 ±0.08 0.124 ±0.08 0.128 ±0.08
0.338 ±0.02 0.677 ±0.05 0.673 ±0.06 0.690 ±0.06
0.022 ±0.01 0.318 ±0.11 0.208 ±0.10 0.295 ±0.14
0.179 ±0.01 0.842 ±0.03 0.833 ±0.03 0.868 ±0.02
0.012 ±0.02 0.099 ±0.11 0.092 ±0.10 0.121 ±0.12
0.369 ±0.03 0.628 ±0.08 0.667 ±0.08 0.672 ±0.08
0.009 ±0.01 0.176 ±0.14 0.210 ±0.14 0.194 ±0.15
0.337 ±0.03 0.689 ±0.07 0.688 ±0.08 0.693 ±0.08
Min-K% [27] + Diversity + Quality + Combined
0.124 ±0.08 0.120 ±0.07 0.124 ±0.07 0.113 ±0.07
0.680 ±0.05 0.677 ±0.05 0.695 ±0.05 0.694 ±0.05
0.067 ±0.07 0.219 ±0.08 0.095 ±0.09 0.240 ±0.15
0.703 ±0.04 0.775 ±0.03 0.772 ±0.03 0.837 ±0.03
0.086 ±0.11 0.064 ±0.09 0.094 ±0.11 0.092 ±0.10
0.650 ±0.08 0.640 ±0.08 0.674 ±0.09 0.694 ±0.08
0.127 ±0.11 0.144 ±0.13 0.178 ±0.14 0.182 ±0.14
0.648 ±0.09 0.685 ±0.08 0.686 ±0.08 0.700 ±0.07
CAMIA [6] + Diversity + Quality + Combined
0.111 ±0.09 0.118 ±0.08 0.131 ±0.08 0.109 ±0.08
0.683 ±0.05 0.690 ±0.05 0.708 ±0.05 0.696 ±0.05
0.428 ±0.25 0.517 ±0.25 0.501 ±0.26 0.553 ±0.27
0.958 ±0.01 0.966 ±0.01 0.964 ±0.01 0.969 ±0.01
0.128 ±0.12 0.104 ±0.11 0.115 ±0.13 0.109 ±0.12
0.664 ±0.08 0.668 ±0.08 0.682 ±0.08 0.689 ±0.08
0.166 ±0.13 0.146 ±0.12 0.192 ±0.14 0.176 ±0.13
0.686 ±0.09 0.692 ±0.08 0.712 ±0.08 0.716 ±0.08
more modest but consistent. For IARs, MADreMIA yields clear gains in classification accuracy: VAR-d30 improves from 0.607 to 0.696 (+8.9 p.p.) and RAR-XXL from 0.562 to 0.713 (+15.1 p.p.), although TPR@1%FPR gains are smaller and less stable. Together, these results confirm that iterative reconstruction signals provide complementary, architecture-agnostic information that reliably strengthens membership inference across both LLMs and IARs. Table 4: MIA results on IARs, where MADreMIA trajectory features are aggregated across 10 iterations (benchmark details in Appendix G). While AUC remains stable across augmentation variants, TPR@1%FPR and Accuracy improve substantially. VAR-d30
5.8
RAR-XXL
Attack
TPR@1%FPR
AUC
ACC
TPR@1%FPR
AUC
ACC
Baseline [19] + Diversity + Quality + Combined
0.040 ±0.02 0.090 ±0.09 0.076 ±0.08 0.088 ±0.06
0.750 ±0.02 0.755 ±0.03 0.757 ±0.03 0.750 ±0.04
0.607 ±0.07 0.691 ±0.03 0.703 ±0.03 0.696 ±0.03
0.044 ±0.02 0.084 ±0.06 0.079 ±0.07 0.069 ±0.05
0.754 ±0.01 0.771 ±0.03 0.754 ±0.04 0.775 ±0.03
0.562 ±0.02 0.700 ±0.03 0.703 ±0.03 0.713 ±0.03
MADreMIA amplifies baseline DI
The p-value histograms in Figure 4 demonstrate that MADreMIA trajectory features consistently strengthen the statistical evidence for dataset-level inference across all evaluated architectures. On Pythia-6.9B, augmented variants reach the 95% confidence threshold at around 100 samples versus roughly 150 for the baseline. Furthermore, augmented variants shift the distribution of − log10 (p) values noticeably rightward relative to the baseline, with this pattern holding across all three signal types. The effect is more pronounced on RAR-XXL, where the Combined variant produces a substantially larger rightward shift, indicating that individual trials yield stronger and more reliable evidence for membership inference. 8
200 150 100 50 0
0
2
4
6 8 log10 (p)-value
10
12
Distribution of Significance Baseline Combined p = 0.05 p = 0.01
0.8 0.6 0.4
200 150 100
0.2
50
0.0
0
100
10 2
10 4 10 6 10 8 10 10 p-value Threshold (Log Scale)
Baseline Combined p = 0.01 Threshold
250 Number of Trials
Number of Trials
250
Cumulative Success Rate
1.0 Proportion of Trials Threshold
Baseline Combined p = 0.01 Threshold
10 12
0.0
2.5
5.0
7.5 10.0 log10 (p)-value
(a) Pythia-6.9B
12.5
15.0
17.5
Cumulative Success Rate
1.0 Proportion of Trials Threshold
Distribution of Significance 300
Baseline Combined p = 0.05 p = 0.01
0.8 0.6 0.4 0.2 0.0
10 1
10 7 10 4 10 10 10 13 p-value Threshold (Log Scale)
10 16
(b) RAR-XXL
Figure 4: Dataset Inference performance on selected models.
0.8 0.7
Strength s = 8
0.95
0.95
0.90
0.90
0.90
0.85 0.80
0.80
0.90
0.92
0.94 0.96 Recall
0.98
1.00
0.85
15
15
7
7
0.80 0.75
0.70
0.70
0.850 0.875 0.900 0.925 0.950 0.975 1.000 Recall
0.85 0.75
0.75
0.6
Strength s = 6
1.00
0.95 Precision
Precision
0.9 Precision
Strength s = 4
1.00
Precision
Strength s = 2
1.0
0.88 0.90 0.92 0.94 0.96 0.98 1.00 Recall
1
1
Non0.86 0.88 0.90 0.92 0.94 0.96 Members Recall
Members
Figure 5: Precision-Recall curves for VAR-d30 across regeneration strengths s ∈ {2, 4, 6, 8}. Members (green) and Non-Members (red) are traced over 15 iterations, with color intensity indicating iteration progress. Larger s corresponds to more aggressive regeneration.
5.9
Sensitivity analysis of generation strength
Figure 5 shows PR curves for VAR-d30 across regeneration strengths s ∈ {2, 4, 6, 8}, where s controls how many final scales are regenerated. Members consistently achieve higher precision and recall than non-members across all values of s, confirming that the MIA signal is robust to the choice of regeneration strength. As s increases, however, the two groups converge in PR space (see Appendix J). 5.10
Trajectory asymmetry scaling across model families
As illustrated in Figure 6, the membership signal – quantified by ∆FID = FIDnonmem − FIDmem persists across all model scales, suggesting that the observed asymmetry is a fundamental property rather than an artifact of specific parameter regimes. While the magnitude of this separation varies across architectures, its relationship with model scale is not uniform. The separation grows stronger with model size in VAR and DiT-MoE, but remains largely unaffected by scaling in RARs. Ultimately, the underlying trend is robust: iterative trajectory chaining consistently exposes a larger membership gap compared to standard one-shot generations. FID ( )
FID ( )
FID ( ) RAR-XXL RAR-XL RAR-L
1.0
3.5 3.0
0.8
6 VAR-30 VAR-24 VAR-20
2.0
0.4
1.5
0.2
1.0
FID
FID
FID
2.5 0.6
DIT-G DIT-XL
8
4 2
0.5
0.0
0
0.0 2
4
6
8
Iteration
(a) RAR
10
12
14
2
4
6
8
Iteration
(b) VAR
10
12
14
1
2
3
4
5
6
Iteration
7
8
9
10
(c) DiT-MoE
Figure 6: Ablation: Trajectory asymmetry scaling across model families. Membership separation (∆ FID) persists across model scales, confirming that iterative trajectory chaining consistently amplifies membership signals compared to one-shot baselines.
9
6
Conclusions
We introduced MADreMIA, a model-agnostic membership inference signal amplifier for large generative models. By chaining repeated regenerations rather than relying on a single query, MADreMIA exploits a consistent asymmetry: member samples retain coherence across iterations while nonmembers drift and deteriorate. This signal generalizes across image, text, and audio generators, spanning IAR, diffusion, and LLM families. Our experimental results show that fusing trajectoryderived features with baseline MIA/DI scores further improves member/non-member separability, suggesting that iterative regeneration is a broadly applicable lens for privacy auditing and copyright attribution. Acknowledgments We gratefully acknowledge Polish high-performance computing infrastructure PLGrid for providing computer facilities and support within computational grant no. PLG/2025/018391. This research was partially funded by National Science Centre, Poland, grant no: 2023/51/I/ST6/02854.
10
References [1] Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard Baraniuk. Self-consuming generative models go mad. In The Twelfth International Conference on Learning Representations, 2023. [2] Ching-Yuan Bai, Hsuan-Tien Lin, Colin Raffel, and Wendy Chi-wen Kan. On training sample memorization: Lessons from benchmarking generative modeling with a large-scale competition. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 2534–2542, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325. doi: 10.1145/3447548.3467198. URL https://doi.org/10.1145/ 3447548.3467198. [3] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In CVPR, 2023. [4] Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. Pythia: a suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023. [5] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633–2650, 2021. [6] Hongyan Chang, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Reza Shokri. Context-aware membership inference attacks against pre-trained large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7299–7321, 2025. [7] Chen Chen, Daochang Liu, Mubarak Shah, and Chang Xu. Enhancing privacy-utility trade-offs to mitigate memorization in diffusion models. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8182–8191, 2025. URL https: //api.semanticscholar.org/CorpusID:278129333. [8] Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Labelonly membership inference attacks. In International conference on machine learning, pages 1964–1974. PMLR, 2021. [9] Matthew Coulter. Aiming for fairness: an exploration into getty images v. stability ai and its importance in the landscape of modern copyright law. DePaul J. Art Tech. & Intell. Prop. L, 34: 124, 2024. [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. [11] Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? In Conference on Language Modeling (COLM), 2024. [12] Jan Dubiński, Antoni Kowalczuk, Franziska Boenisch, and Adam Dziedzic. Cdi: Copyrighted data identification in diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18674–18684, 2025. [13] Zhengcong Fei, Mingyuan Fan, Changqian Yu, Debang Li, and Junshi Huang. Scaling diffusion transformers to 16 billion parameters, 2024. URL https://arxiv.org/abs/2407.11633. [14] Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Accelerating the science of language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15789–15809, 2024. 11
[15] Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, and Ye Wang. On memorization in diffusion models. arXiv preprint arXiv:2310.02664, 2023. [16] Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, et al. Be like a goldfish, don’t memorize! mitigating memorization in generative llms. Advances in Neural Information Processing Systems, 37:24022–24045, 2024. [17] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. [18] Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fr\’echet audio distance: A metric for evaluating music enhancement algorithms. arXiv preprint arXiv:1812.08466, 2018. [19] Antoni Kowalczuk, Jan Dubiński, Franziska Boenisch, and Adam Dziedzic. Privacy attacks on image autoregressive models. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=7SXXczJCWP. [20] Jingwei Li, Jing Dong, Tianxing He, and Jingzhao Zhang. Towards black-box membership inference attack for diffusion models. CoRR, abs/2405.20771, 2024. doi: 10.48550/arXiv.2405. 20771. URL https://doi.org/10.48550/arXiv.2405.20771. [21] Jingyi Li, Weiping Tu, and Li Xiao. Freevc: Towards high-quality text-free one-shot voice conversion. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023. [22] Pratyush Maini and Anshuman Suri. Reassessing emnlp 2024’s best paper: Does divergencebased calibration for mias hold up? In The Fourth Blogpost Track at ICLR 2025. [23] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. arXiv preprint arXiv:2104.10706, 2021. [24] Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. LLM dataset inference: Did you train on my dataset? CoRR, abs/2406.06443, 2024. doi: 10.48550/arXiv.2406.06443. URL https://doi.org/10.48550/arXiv.2406.06443. [25] Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. Autovc: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning, pages 5210–5219. PMLR, 2019. [26] Mansi Sakarvadia, Aswathy Ajith, Arham Mushtaq Khan, Nathaniel C Hudson, Caleb Geniesse, Kyle Chard, Yaoqing Yang, Ian Foster, and Michael W Mahoney. Mitigating memorization in language models. In The Thirteenth International Conference on Learning Representations, 2024. [27] Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789, 2023. [28] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017. [29] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631:755–759, 07 2024. doi: 10.1038/s41586-024-07566-y. [30] Aaditya K Singh, Muhammed Yusuf Kocyigit, Andrew Poulton, David Esiobu, Maria Lomeli, Gergely Szilvasy, and Dieuwke Hupkes. Evaluation data contamination in llms: how do we measure it and (when) does it matter? arXiv preprint arXiv:2411.03923, 2024. 12
[31] Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, et al. Dolma: An open corpus of three trillion tokens for language model pretraining research. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15725–15788, 2024. [32] Jiashu Tao and Reza Shokri. (token-level) InfoRMIA: Stronger membership inference and memorization assessment for LLMs. CoRR, abs/2510.05582, 2025. doi: 10.48550/arXiv.2510. 05582. URL https://doi.org/10.48550/arXiv.2510.05582. [33] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction, 2024. URL https://arxiv. org/abs/2404.02905. [34] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. [35] Gerrit van den Burg and Chris Williams. On memorization in probabilistic deep generative models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 27916–27928. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/ paper/2021/file/eae15aabaa768ae4a5993a8a4f4fa6e4-Paper.pdf. [36] Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. Coco-text: Dataset and benchmark for text detection and recognition in natural images. arXiv preprint arXiv:1601.07140, 2016. [37] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4): 600–612, 2004. [38] Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=84n3UwkH7b. [39] YUTONG WU, Han Qiu, Shangwei Guo, Jiwei Li, and Tianwei Zhang. You only query once: An efficient label-only membership inference attack. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=7WsivwyHrS. [40] Junichi Yamagishi, Christophe Veaux, and Kirsten MacDonald. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92). The Rainbow Passage which the speakers read out can be found in the International Dialects of English Archive:(http://web. ku. edu/˜ idea/readings/rainbow. htm)., 2019. [41] Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. Enhanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC conference on computer and communications security, pages 3093–3106, 2022. [42] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018. [43] Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang-Chieh Chen. Randomized autoregressive visual generation, 2024. URL https://arxiv.org/abs/2411.00776. [44] Michał Zawalski, Meriem Boubdir, Klaudia Bałazy, Besmira Nushi, and Pablo Ribalta. Detecting data contamination in LLMs via in-context learning. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum? id=YlpaaYxx4t. 13
[45] Heiga Zen, Viet Dang, Robert A. J. Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Z. Chen, and Yonghui Wu. Libritts: A corpus derived from librispeech for text-to-speech. In Interspeech, 2019. URL https://api.semanticscholar.org/CorpusID:102352475. [46] Jingyang Zhang, Jingwei Sun, Eric C. Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Yang, and Hai Helen Li. Min-k%++: Improved baseline for detecting pre-training data from large language models. CoRR, abs/2404.02936, 2024. doi: 10.48550/arXiv.2404.02936. URL https://doi.org/10.48550/arXiv.2404.02936. [47] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. [48] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. [49] Ziqi Zhang, Chao Yan, and Bradley A Malin. Membership inference attacks against synthetic health data. Journal of biomedical informatics, 125:103977, 2022.
14
A
Impact Statement
This work advances methods for auditing generative models by improving membership and dataset inference through chained regeneration. The primary positive impact is stronger accountability: MADreMIA can help detect memorization of sensitive, proprietary, or benchmark data, supporting privacy audits, copyright verification, and unlearning validation across model families and modalities. While enhanced inference capabilities can assist in model auditing and transparency, they also require responsible application to avoid potential misuse. We frame MADreMIA as a tool for research evaluation, compliance monitoring, and internal red-teaming. It is important to note that our method provides statistical evidence rather than a definitive proof of data inclusion; therefore, results should be interpreted alongside additional forensic and procedural evidence within a broader data governance framework.
B
Limitations
While our proposed framework is designed to be cross-modal and model-agnostic, our experimental scope is naturally constrained by several practical and theoretical factors. Most notably, we do not conduct full Membership Inference Attack (MIA) evaluations on audio generation models. Although our initial signal-degradation experiments indicate that iterative trajectory features exist in the audio domain, the literature currently lacks established single-step baselines tailored for these architectures, leaving MIA for audio models untested. Furthermore, while our framework is conceptually compatible with restricted setups, our current empirical evaluations rely on gray-box access to exact next-token logits, meaning that strictly black-box MIA remains untested in our work. Operationally, the primary limitation of our method is its scalability; the iterative regeneration loop inherently introduces a linear computational overhead by requiring multiple forward passes per sample. From a theoretical perspective, our core assumptions A1 and A3 are only partially satisfied in practice, as demonstrated by the empirical measurements in Table 2. Finally, our evaluations may be susceptible to distribution-shift confounds—where trajectory differences might stem from inherent dataset mismatches rather than pure memorization—and the exploratory findings presented in Section 5.6 are based on preliminary small-n evidence that will require larger-scale validation in future work.
C
LLM Usage
Large language models were used to improve the readability and clarity of portions of the manuscript, as well as to provide feedback during the writing and revision process. The authors verified all technical statements, citations, and claims and take full responsibility for the final content.
D
Method Overview
E
Extended Related Works
Our work builds upon three intersecting lines of prior research: the characterization of data memorization in generative models, the evolution of membership inference, and the dynamics of model collapse during recursive generation. E.1
Memorization
Memorization — the tendency of generative models to reproduce training examples rather than generate novel samples — has been studied across multiple model families and from both measurement and mitigation perspectives. van den Burg and Williams [35] formalized the problem for probabilistic generative models such as VAEs, showing that memorization differs fundamentally from mode collapse and overfitting and is not captured by commonly-used nearest-neighbor tests. For diffusion models, Gu et al. [15] show that the denoising score matching objective has a closed-form optimum that can only replicate training samples, and introduces the EMM metric to quantify how dataset size and model configuration govern the generalization-to-memorization transition. Sakarvadia et al. 15
Baseline MIA single-shot only
No trajectory add-on Suspect sample Queried sample Z 0
MIA score Weak signal
MIA features (no add-on features used)
Member / Non-member
Feature Extractor
MADreMIA + chained regenerations
Target Model
?
?
member
non-member
zbase = (Z0)
Trajectory add-on
Attack score
MIA score Amplified signal
Member
consistent
Non-member
inconsistent
ztraj = (Z0, , ZT)
Member / Non-member Attack score
Figure 7: MADreMIA amplifies membership signal via trajectory features. Left: both methods share the same suspect sample Z0 and base features zbase = ϕ(Z0 ). Top: baseline one-shot MIA uses no trajectory add-on and yields weak member/non-member separability. Bottom: MADreMIA adds trajectory features ztraj = ψ(Z0 , . . . , ZT ) from chained regeneration. Members exhibit consistent trajectories (slow drift from Z0 ), whereas non-members drift inconsistently. The fused representation z̃ = [zbase ∥ztraj ] is scored by the same plug-in estimator h (e.g., L1-regularized logistic regression), amplifying the attack signal without changing the scorer.
[26] localize this phenomenon through bright-ending cross-attention patterns, while the sharpnessbased framework of [38] justifies score-difference memorization metrics and proposes mitigation via sharpness-aware regularization of the initial noise. The benchmarking study [2] demonstrated that standard evaluation metrics fail to surface memorization even in competitive settings. Mitigation has been tackled both for LLMs, where Hans et al. [16] propose the goldfish loss that excludes randomly sampled token subsets from the training objective to prevent verbatim reproduction, and for text-to-image diffusion models, where Chen et al. [7] address the privacy–utility tension by combining prompt re-anchoring with semantic prompt search to improve both dimensions simultaneously. E.2
Membership/Dataset Inference
A second line of work investigates whether specific examples or datasets can be identified from model behavior. Because individual Membership Inference Attacks (MIAs) can be confounded by distribution shifts [24], recent literature often favors Dataset Inference (DI), which aggregates feature evidence across many samples to statistically detect training data usage [22, 12, 19]. Concurrently, individual MIA methods must adapt to increasingly restrictive black-box deployments. Furthermore, approaches based on training multiple shadow models to learn membership distributions [41, 5] are now computationally infeasible for massive modern architectures. Consequently, modern attacks must extract signals using only limited outputs rather than internal weights or gradients [46, 6, 32]. In these restricted settings, recent black-box attacks heavily rely on output variations. For example, Li et al. [20] perform MIAs on diffusion models by repeatedly perturbing a target image via an API, averaging the results, and comparing them to the original sample. However, in an interrogation analogy, this approach merely asks multiple paraphrased versions of the exact same question. Because the target sample is perturbed independently each time, the query does not dynamically evolve in response to the model’s previous answers, leaving deeper structural memorization unexploited. E.3
Model Collapse
The last, but very important point is the literature on recursive self-training in generative models. Alemohammad et al. [1] showed that self-consuming generative loops lead to progressive degradation in quality or diversity when insufficient fresh real data is injected at each generation, a phenomenon 16
they term Model Autophagy Disorder. Their analysis is especially important for our setting because it frames repeated regeneration not as a neutral operation, but as a process that can magnify latent properties of the learned distribution. Closely related, Shumailov et al. [29] showed that recursively training on model-generated data causes model collapse, where tails of the original distribution disappear and learned behaviour drifts toward degenerate approximations. Taken together, these works suggest that iterative generation is structurally revealing: under repeated reuse, memorized or high-density regions may persist differently from non-member examples, while generic outputs may drift or collapse. Our method turns this insight into a privacy-auditing mechanism: rather than studying recursive generation as a training-time pathology, we exploit chained regeneration at inference time to amplify membership-relevant differences.
F
Model Details
In our experiments, we consider two vision model families: image autoregressive models (IARs) and diffusion models. The IAR category includes VAR [33] and RAR [43] variants, while the diffusion category includes DiT-MoE [13] and UViT-T2I [3]. Furthermore, as others modalities, we evaluate large language models (LLMs) and voice conversion (VC) models. The LLMs include Pythia [4], OLMo [14], OPT [48], and Llama [34], while the VC models consist of AutoVC [25] and FreeVC [21]. Across all settings, we focus on representative, high-performing model variants. Table 5: Vision model details. IAR Models
Model parameters Training epochs FID
Diffusion Models
VAR-d30
VAR-d24
VAR-d20
RAR-XXL
RAR-XL
RAR-L
DiT-MoE-G
DiT-MoE-XL
UViT-T2I-Deep
2.1B 350 1.92
1.0B 300 2.33
600M 250 2.95
1.5B 400 1.48
955M 400 1.50
462M 400 1.70
16.5B — 1.72
4.1B — 2.10
141M — 5.48
Table 7: Audio model details.
Table 6: Language model details. OLMo Model parameters Training tokens
G
7B 2.46T
Llama 13B 1T
Pythia 6.9B 300B
OPT Model parameters Training data (hours) SMOS (seen-to-seen)
6.7B 180B
AutoVC
FreeVC
28M 44 3.5
39M 40 4.1
Dataset Details
For vision and audio models that have publicly known and available train/test splits we use these datasets. For most LLMs we use established MIA benchmarks (e.g. WikiMIA), but for OLMo, we use their corresponding training sets and the Global News as non-member set, as suggested in [44]. Table 8: Datasets used to construct member and non-member sets for each model family in our experiments, spanning vision, language, and speech domains. Model
Members
Non-members
VAR RAR DiT-MoE UViT-T2I
ImageNet [10] ImageNet ImageNet COCO [36]
ImageNet ImageNet ImageNet COCO
Pythia OLMo Llama OPT
Mimir [11] Dolma [31] WikiMIA WikiMIA
Mimir Global News WikiMIA WikiMIA
AutoVC FreeVC
VCTK [40] VCTK
LibriTTS [45] LibriTTS
Importantly, for the Pythia-6.9B we use the Mimir dataset [11] which consists of 6 subsets: arxiv, dm_mathematics, github, hackernews, pubmed_central, and wikipedia_(en). We concatenate all these subsets and randomly select samples from the pool. We use the ngram_7_0.2 data split. For the rest of the models, we employ their corresponding datasets’ train split as members and val/test split as nonmembers. 17
H
Metrics Details
The following metrics are computed over the sequence of model outputs collected across MADreMIA iterations, capturing how the model’s generative behavior evolves under repeated generation. H.1
Features for Language Models
Jaccard Similarity: Measures the lexical overlap between the model’s output at a given iteration and its initial response, computed over trigrams. A high Jaccard similarity indicates that the model rigidly reproduces the same surface forms across iterations, which is characteristic of memorized content. |A ∩ B| J(A, B) = |A ∪ B| Token Diversity: Quantifies the divergence between the token probability distribution at the current iteration P and the initial distribution Q. Large values indicate that the model’s vocabulary preferences shift substantially during reconstruction, reflecting instability in its output distribution. X P (x) DKL (P ∥ Q) = P (x) log Q(x) x∈X
Token Distribution Shift: We define it as a Jensen-Shannon Divergence, which is a symmetric and bounded variant of KLD that measures the distributional distance between P and Q via their mixture M . Compared to KLD, JSD is well-defined even when the supports of P and Q do not fully overlap, making it a more numerically stable measure of distributional drift across iterations. 1 1 JSD(P ∥ Q) = DKL (P ∥ M ) + DKL (Q ∥ M ) 2 2 1 where M = (P + Q) 2 Predictive Entropy: Measures the uncertainty of the model’s next-token distribution over the full vocabulary V. Low entropy indicates that the model assigns high probability mass to a single token — consistent with confident, memorized reproduction — whereas high entropy reflects diffuse, uncertain predictions. X H(Y | x) = − P (y = c | x) log P (y = c | x) c∈V
Margin: Captures the decisiveness of the model’s token predictions by computing the difference in probability between the top-ranked and second-ranked tokens. A large margin indicates high confidence in a specific token, which may signal memorized recall, while a small margin reflects genuine uncertainty between competing continuations. M = P (ŷ1 | x) − P (ŷ2 | x) H.2
Features for Vision Models
Mean Squared Error (MSE): Measures the average pixel-level reconstruction error between the generated image at a given iteration and the original input. Lower MSE indicates that the model consistently reproduces fine-grained pixel details across iterations, which is a strong signal of memorization. N 1 X 2 MSE(x, x̂) = (xi − x̂i ) N i=1 Structural Similarity Index Measure (SSIM) [37]: Evaluates perceptual similarity between the reconstructed image x̂ and the original x by jointly comparing luminance, contrast, and structural information across local image patches. Unlike MSE, SSIM is sensitive to perceptual distortions that are meaningful to human observers, and its stability across iterations serves as a complementary signal to pixel-level metrics. (2µx µx̂ + c1 )(2σxx̂ + c2 ) SSIM(x, x̂) = 2 (µx + µ2x̂ + c1 )(σx2 + σx̂2 + c2 ) 18
Dataset Inference Success Rate
0.8 0.6 0.4 0.2 0.0 20
40 60 Number of Samples (n)
Baseline Combined Diversity Quality 95% Confidence Threshold 80 100
0.8 0.6 0.4 0.2 0.0 0
100
(a) Llama-13B Distribution of Significance
Number of Trials
150 125 100 75 50 25 0
0
2
4
log10 (p)-value
6
8
Cumulative Success Rate
1.0
Distribution of Significance Baseline Combined p = 0.05 p = 0.01
0.8 0.6 0.4
Baseline Combined p = 0.01 Threshold
250
0.2 0.0
200 150 100 50
100
Baseline Quality Diversity Combined 95% Confidence Threshold 400 500
(b) VAR-d30
Number of Trials
Baseline Combined p = 0.01 Threshold
175
Proportion of Trials Threshold
200
200 300 Number of Samples (n)
10 2 10 4 10 6 p-value Threshold (Log Scale)
0
10 8
0
5
(c) Llama-13B
10 15 log10 (p)-value
20
25
Cumulative Success Rate
1.0 Proportion of Trials Threshold
0
Dataset Inference Success Rate
1.0 Success Rate (p 0.01)
Success Rate (p <= 0.01)
1.0
Baseline Combined p = 0.05 p = 0.01
0.8 0.6 0.4 0.2 0.0 1 10
10 3
10 7 10 11 10 15 10 19 p-value Threshold (Log Scale)
10 23
(d) VAR-d30
Figure 8: DI performance on additional models.
where µx , µx̂ are local means, σx2 , σx̂2 are local variances, σxx̂ is the cross-covariance, and c1 , c2 are stabilization constants. Learned Perceptual Image Patch Similarity (LPIPS) [47]: Quantifies perceptual dissimilarity between x and x̂ using deep feature representations extracted from a pretrained network ϕ. By operating in a learned feature space rather than pixel space, LPIPS captures high-level semantic and textural differences that are invisible to MSE or SSIM, making it particularly sensitive to cases where a model reproduces semantic content while varying low-level details. X 1 X 2 LPIPS(x, x̂) = ∥wl ⊙ (ϕl (x)hw − ϕl (x̂)hw )∥2 Hl Wl l
h,w
where ϕl denotes the feature map at layer l of the pretrained network and wl are learned channel-wise weights.
I
Additional Dataset Inference Results
Figure 8 extends our dataset inference evaluation to Llama-13B and VAR-d30. On Llama-13B, augmented variants reach the 95% confidence threshold faster than the baseline, with the Combined and Quality signals leading, though convergence is noisier at low sample counts. On VAR-d30, the benefit is more pronounced: augmented variants cross the threshold at roughly 100 samples compared to over 200 for the baseline, with all three signal types outperforming it consistently. The significance histograms corroborate these findings — the Combined variant shifts the − log10 (p) distribution rightward on both models, confirming that trajectory features yield stronger per-trial evidence.
J
Precision and Recall for Generative Models
Figure 9 shows Precision and Recall across iterations for VAR-d30 and DiT-MoE-XL. In both models and both metrics, members consistently score higher than non-members throughout all iterations, confirming that the membership signal is stable and model-agnostic. Notably, the gap between members and non-members widens as iterations progress, indicating that chained regeneration amplifies the underlying asymmetry rather than merely preserving it. 19
0.90
0.94 0.90 1
Non-Members Members 2 3 4 5 6 7 Iteration
0.95
0.85 0.80
8
9 10
(a) VAR-d30 (Rec.)
0.75
Recall ( )
1.00
Recall
0.96
Precision
Recall
0.95
0.92
Precision ( )
1.00
0.98
1
Non-Members Members 2 3 4 5 6 7 Iteration
Precision
Recall ( )
1.00
Non-Members Members
0.90 0.85 0.80
8
9 10
1
(b) VAR-d30 (Prec.)
2
3
4
5 6 7 Iteration
8
9 10
(c) DiT-MoE-XL (Rec.)
1.00 0.95 0.90 0.85 0.80 0.75 0.70
Precision ( )
Non-Members Members
1
2
3
4
5 6 7 Iteration
8
9 10
(d) DiT-MoE-XL (Prec.)
Figure 9: Precision and Recall across models.
K
Getty Images Case
As a practical case study, we consider the Getty Images v. Stability AI dispute [9] and evaluate whether chained regeneration can distinguish images that are plausibly associated with the Stable Diffusion training distribution from images that are very unlikely to have been included. We use Stable Diffusion 1.5 as the target model. For the positive pool, we extract 2,000 images from LAION2B whose metadata contains the string gettyimages and treat them as members. For the negative pool, we collect 2,000 images from the Getty Images website whose upload date is after January 1, 2025, and treat them as non-members. Because these images post-date the original Stable Diffusion 1.5 training era (late 2022), they provide a conservative practical control group for this experiment.
Members Non-Members
0.8
0.040
0.7
0.035
MSE
0.6
SSIM
Members Non-Members
0.045
0.5 0.4
0.030 0.025 0.020
0.3
0.015
0.2
0.010
2
4
6
8
Iteration
10
12
14
2
(a) SSIM (↑)
4
6
8
Iteration
10
12
14
(b) Reconstruction Error (MSE) (↓)
Figure 10: Evolution of (a) SSIM and (b) Reconstruction Error over 15 chained regeneration steps. The solid lines indicate the mean values for training members (blue) and held-out non-members (red), with shaded regions representing standard deviation. Across both metrics, members exhibit higher structural fidelity and slower degradation than non-members.
For each pool, we run the same chained-regeneration procedure for 15 iterations and summarize the trajectories with SSIM and reconstruction error (MSE) (see Figure 10). The SSIM plot measures whether regenerations remain structurally closer to the initial query for the member pool than for the non-member pool. The MSE plot provides a complementary pixel-level view across regeneration depth by measuring how quickly reconstructed samples drift away from their reference images. In our experiments, the two pools remain visibly separated under both SSIM and MSE. We do not use FID in this case, because it is very unstable on 2,000-image pools. We still interpret MSE conservatively: it is sensitive to low-level reconstruction error rather than semantic fidelity alone. For this reason, we use MSE as a stable auxiliary trajectory measure across iterations, while SSIM remains the more directly interpretable structural signal in this case study. 20
L
Proofs for Section 3
L.1
Proof of Theorem 3.2
Proof. By definition, T −1
1 X E[ϕt | M = m], E[ST | M = m] = T t=0 Hence E[ST | 1] − E[ST | 0] =
m ∈ {0, 1}.
T −1 1 X E[ϕt | 1] − E[ϕt | 0] . T t=0
Under A1, E[ϕt | 1] − E[ϕt | 0] ≥ ∆t ≥ 0, so
∀t,
T −1
E[ST | 1] − E[ST | 0] ≥
1 X ∆t ≥ 0. T t=0
Therefore
T −1
ΓT := E[ST | 1] − E[ST | 0] ≥
1 X ∆t . T t=0
For the denominator, A3 gives, for each class m, Var(ST | M = m) ≤ C
σ 2 τeff . T
Hence max Var(ST | M = m) ≤ C m
σ 2 τeff . T
Combining with the lower bound on ΓT , SNR2 (ST ) =
P T −1 1
Γ2T maxm Var(ST | M = m)
≥
T
t=0 ∆t
Cσ 2 τeff /T
This proves Theorem 3.2. L.2
Proof of Corollary 3.3
Proof. Assume ∆t = ∆0 e−t/τg . Then T −1 T −1 ∆0 X −t/τg ∆0 1 − e−T /τg 1 X ∆t = e = · . T t=0 T t=0 T 1 − e−1/τg
Since 1 − e−u ≤ u for u > 0, with u = 1/τg we get 1 − e−1/τg ≤ Therefore
1 τg
=⇒
1 ≥ τg . 1 − e−1/τg
T −1
1 − e−T /τg 1 X ∆t ≥ ∆0 . T t=0 T /τg By Theorem 3.2, P T −1 1 SNR2 (ST ) ≥ where g(x) :=
T
t=0
∆t
Cσ 2 τeff /T (1 − e−x )2 , x 21
2 ≳
∆20 τg g(x), σ 2 τeff
x := T /τg ,
2 .
and ≳ absorbs only T -independent constants (including 1/C and comparability constants). To optimize the shape in x, differentiate: (1 − e−x ) 2xe−x − (1 − e−x ) g (x) = . x2 For x > 0, critical points satisfy ′
2xe−x = 1 − e−x
⇐⇒
ex = 2x + 1.
This has a unique positive solution x⋆ ≈ 1.2564, so the surrogate shape is maximized at T ⋆ ≈ x⋆ τg ≈ 1.2564 τg .
L.3
Proof of Corollary 3.4 (shape-constant clarification)
Proof. From the previous corollary (under the same comparability regime), SNR2 (ST ⋆ ) ≳
∆20 τg g(x⋆ ). σ 2 τeff
Assume additionally
Γ1 ≍ ∆0 , Var(S1 | M ) ≍ σ 2 , so SNR(S1 ) ≍ ∆0 /σ. Taking square roots and ratio: r √ SNR(ST ⋆ ) p ⋆ τg ≳ g(x ) = cshape κ, SNR(S1 ) τeff
where
p τg , cshape := g(x⋆ ) ≈ 0.638. τeff Thus cshape is the idealized shape constant; additional model-dependent prefactors remain absorbed by ≳. κ :=
L.4
Additional comments on Bayes-cap statement at the end of Section 3
If membership is deterministic in the initial sample, M = f (Z0 ), then H(M | Z0 ) = 0, so I(M ; Z0 ) = H(M ) − H(M | Z0 ) = H(M ). Also, conditioning on Z0 already determines M , hence H(M | Z0 , Z1:T ) = 0 = H(M | Z0 ), which implies I(M ; Z1:T | Z0 ) = H(M | Z0 ) − H(M | Z0 , Z1:T ) = 0. Therefore, by the chain rule for mutual information, I(M ; Z0:T ) = I(M ; Z0 ) + I(M ; Z1:T | Z0 ) = I(M ; Z0 ). Thus trajectory iteration cannot increase Bayes-optimal information; it can improve practical fixedform statistics through variance reduction and temporal aggregation.
22