Exploring The Empirical Privacy Landscape of Synthetic Data

E XPLORING T HE E MPIRICAL P RIVACY L ANDSCAPE OF S YNTHETIC DATA Zihao Zhao ∗ Johns Hopkins University

arXiv:2610.22659v1 [cs.CR] 15 Oct 2026

Katherine Lee OpenAI

Avi Schwarzschild OpenAI Zhili Feng OpenAI

Casey Meehan OpenAI Dylan Sam OpenAI

A BSTRACT Methods for training language models on sensitive data differ in how they balance capability gains against the risk of leaking private information. In this paper, we present a controlled comparison of data-level and training-level interventions for navigating the capability-privacy tradeoff on synthetic medical records with canaries to probe for leakage. Direct finetuning performs best on capability metrics but leads to the most privacy leakage, while DP-SGD reduces leakage at substantial cost to capabilities. Generating synthetic data from these records offers a more favorable balance, particularly when guided by a uniqueness-aware instruction to replace private details. Relative to naive paraphrasing, privacy-aware rewriting improves privacy protection across a variety of different rewriting models. We also find that more capable rewriting models generally reduce the risks of downstream privacy leakage, although held-out fit does not improve monotonically and increased test-time compute yields inconsistent gains. Overall, these results highlight privacy-aware rewriting as a promising direction for achieving a more favorable trade-off between capabilities and empirical privacy compared to other training-level methods, and our evaluation framework supports comparisons of future models and methods.

1

I NTRODUCTION

Modern language models are pretrained and finetuned on large-scale datasets, and their capabilities are closely tied to the scale, diversity, and quality of their training data (Chen et al., 2024; Chang et al., 2024; Wettig et al., 2024; Parmar et al., 2024; Kaplan et al., 2020; Penedo et al., 2023; Lee et al., 2022). For example, models improve in coding capabilities when they have been trained with a large amount of coding data (Aryabumi et al., 2025; Seed et al., 2025; Wei et al., 2024). Similarly, adapting language models to high-stakes domains often requires access to a large quantity of sensitive records. Directly training on such data can raise privacy and safety concerns, as language models can memorize training examples and reveal sensitive information during inference (Carlini et al., 2021; Nasr et al., 2025). Such information may include personally identifiable health data or knowledge relevant to biological misuse (Maini et al., 2026; O’Brien et al., 2026). Existing approaches for mitigating privacy leakage can be broadly divided into two categories based on the stage at which privacy protection is introduced. Data-level methods modify or replace the training data to reduce the exposure of sensitive information. For example, synthetic data generation uses a generative model to produce records that preserve useful semantic information while removing sensitive details. Training-level methods, by contrast, modify the optimization procedure. An example is differentially private stochastic gradient descent (DP-SGD) (Abadi et al., 2016), which limits the influence of individual training examples and provides formal privacy guarantees. Prior work has shown that stronger privacy protection often comes at the cost of reduced model utility, while methods that better preserve model capabilities may increase the risk of privacy leakage ∗

Work done at MATS. Correspondence to zzhao71@jhu.edu

1

1. Private Corpus

2. Privacy Methods PII-Scrubbing

5,000 Clinical Records

Rewriting

Data-level

Privacy-aware Rewriting

2,340 Canaries

Training-level

3. Finetune Data-level

LoRA on synthetic data Training-level

Synthetic Data

DP-SGD

Goldfish

4. Evaluate Memorization

Capability

verbatim perplexity semantic perplexity

held-out record perplexity medication perplexity

LoRA on raw data

Figure 1: Experimental framework and privacy–capability trade-off. Left: Experimental framework. Right: Results for finetuning gpt-oss-20b, with each axis averaging ranks across two privacy and two capability metrics. Higher ranks indicate less privacy leakage and greater capabilities. Arrows connect rewriting to privacy-aware rewriting, which improves both privacy and capability. (Li et al., 2021; Hu et al., 2025). However, existing studies typically evaluate individual privacypreserving methods or focus on either privacy or utility in isolation. Consequently, those data-level and training-level methods have not been systematically compared within a controlled setting in terms of both model capability and privacy leakage. In this work, we conduct a controlled study of the relationship between model capability and privacy leakage when training on sensitive data, which is illustrated in Figure 1. We first construct a corpus of realistic synthetic medical records (Walonoski et al., 2018). While we do not have access to the training datasets of the pretrained models, we believe that there is no overlap with our newly generated synthetic dataset and their pretraining dataset. This controlled setup allows us to attribute privacy leakage to the data introduced during our experiments, rather than to prior exposure during pretraining. We apply a range of methods to finetune models and evaluate each method along two dimensions: privacy leakage and model capability. By visualizing performance along these dimensions, we characterize the trade-off between privacy and utility, identify methods that achieve a more favorable balance, and provide a deeper understanding of how different privacy-preserving approaches affect model behavior. Our results demonstrate that synthetic data through rewriting provides a promising path toward improving model capability with only minor reductions in privacy protection. We find that the specific rewriting strategy is crucial to achieving favorable trade-offs, and this trade-off scales nicely with the quality of the synthetic data generation model, but increasing test-time compute during rewriting does not monotonically improve the trade-off. Our canary-duplication experiments further show that repeated exposure generally increases privacy leakage, although DP-SGD and privacyaware rewriting remain resistant at high duplication levels. Moreover, training on five independent rewrites for one epoch provides no advantage over training on one rewrite for five epochs, despite requiring additional generation. Finally, we compare data-level evaluation with measurements after finetuning. LLM-judged privacy strongly correlates with verbatim privacy metrics (ρ = 0.89), whereas judged quality does not predict improved capability (ρ = −0.64), showing that data-level metrics cannot reliably replace downstream evaluations of finetuned models.

2

R ELATED W ORK

Memorization in Language Models Training data memorization means a model retains information tied to specific training examples rather than learning generalizable patterns. Prior work evaluates memorization through extraction attacks. Carlini et al. (2019) introduce canaries in the training data and measure the extraction of these sequences. Subsequent attacks demonstrated that language models can reproduce verbatim training examples, including personally identifiable information (Carlini et al., 2021). Although much of this literature studies pretraining, models can also 2

memorize examples introduced during finetuning. Susceptibility to membership-inference and extraction attacks varies across full-model, output-head, and adapter-based finetuning (Mireshghallah et al., 2022). Large-scale analyses further found that memorization increases with model capacity and example duplication (Carlini et al., 2022; Lee et al., 2022). Repeated exposure is also an important factor. Kandpal et al. (2022) found that an example becomes more likely to be regenerated as its training frequency increases and that dataset deduplication can substantially reduce extraction risk. Controlled studies further show that memorization depends on an example’s frequency relative to the corpus size and on when it appears during training (Wei et al., 2026). Exact duplicates are not required for this effect: approximate copies can still induce memorization (Shilov et al., 2026). Memorization is also not limited to exact reproduction. Training information may remain recoverable even when exact reproduction is prevented (Ippolito et al., 2023), and certain methods can scalably quantify near-verbatim extraction (Cooper et al., 2026a). The degree of verbatim memorization does not necessarily predict whether information can be extracted through natural-language questions (Sander et al., 2026). Question-based evaluations must also account for lexical clues that may make the expected answer easier to reconstruct (Luo et al., 2026). Following these distinctions, we evaluate prefix-based completion and question-based recall separately (Cooper et al., 2026b). Mitigating Memorization and Privacy Leakage De-identification mitigates privacy leakage by detecting and removing predefined categories of protected health information, such as names, addresses, and identification numbers (Norgeot et al., 2020). Other methods replace detected identifiers with realistic fictional surrogates to retain the structure and utility of the text (Carrell et al., 2013). Nevertheless, these methods can reduce but do not necessarily eliminate information leakage from finetuned language models (Lukas et al., 2023). More broadly, in some circumstances, personal attributes may be inferred from contextual clues even when direct identifiers are absent (Staab et al., 2024). Recent LLM-based anonymization methods therefore consider contextual distinctiveness and replace details according to their re-identification risk (Staab et al., 2025). Our comparison between PII scrubbing and privacy-aware rewriting reflects this distinction between redaction and removing contextually distinctive information. Because the canaries are not conventional identifiers, this tests whether these defenses protect against personal information that is not traditional PII. Privacy-preserving training provides a complementary line of defense. DP-SGD bounds the influence of individual training examples through per-example gradient clipping, calibrated noise, and privacy accounting (Abadi et al., 2016). Subsequent work has adapted differential privacy to parameter-efficient language-model finetuning (Yu et al., 2021). Differential privacy has also been applied to synthetic-text generation through privately finetuned generators (Yue et al., 2023) and through private selection procedures using foundation-model APIs (Xie et al., 2024). Synthetic or rewritten text is not inherently private, however, and membership information may remain detectable in generated corpora (Meeus et al., 2025). Goldfish follows an empirical rather than formal approach, masking selected next-token targets from the training loss to reduce verbatim memorization (Hans et al., 2024). In contrast to studies primarily concerned with releasing synthetic datasets, we directly compare preprocessing and training defenses by measuring both privacy leakage and capability in the resulting finetuned models. Our one-to-one rewriting methods do not provide a formal differential-privacy guarantee; their privacy effects are evaluated empirically.

3

E XPERIMENTAL S ETUP

3.1

DATASET AND C ANARY C ONSTRUCTION

We use Synthea (Walonoski et al., 2018) to generate synthetic patient records that serve as a controlled proxy for a private clinical corpus. Because the records are generated specifically for our experiments, overlap with model pretraining data is unlikely, reducing pretraining contamination as a confounder. Each record contains synthetic patient demographics and identifiers, including name, address, SSN, and date of birth, together with active problems and current medications. From the generated population, we use 5,000 records for training and reserve a disjoint held-out evaluation set. Following established canary-based privacy leakage evaluation (Carlini et al., 2019), we construct 100 unique canary records in the same format and length as the non-canary training records. For each canary, we jointly sample the demographics, address, problem list, and existing medications 3

from a single non-canary training record, assign a fabricated identity, and append a plausible, newly coined medication name as the planted secret. This medication name is fully synthetic and is a few tokens (e.g., Mabsartinib in Table 1). We divide the canaries equally into five groups and insert them at multiplicities of ×1, ×4, ×16, ×32, and ×64, respectively. This produces 5,000 + 20(1 + 4 + 16 + 32 + 64) = 7,340 training instances, comprising 5,000 non-canary records and 2,340 canaries. 3.2

BASELINES AND P RIVACY-P RESERVING M ETHODS

We compare six experimental settings based on the same pretrained model and evaluate them using the same privacy and utility metrics. All trained conditions start from the same corpus but differ in either the training objective or the preprocessing applied to the corpus. No finetuning

We evaluate the pretrained model without any exposure to the training corpus.

Direct finetuning We directly finetune the model on the training corpus without privacy-specific preprocessing or modifications to the training objective. PII scrubbing Before finetuning, we process every training record using the OpenAI Privacy Filter (de Bourcy et al., 2026). Detected identifiers, including names, telephone numbers, emails, and private addresses, are replaced with placeholders. We finetune the model on this redacted corpus. DP-SGD We train only the LoRA adapters using example-level DP-SGD (Abadi et al., 2016), clipping per-example gradients to C = 1 and adding Gaussian noise calibrated to (ϵ, δ) with δ = 10−5 . We use Poisson sampling with an expected lot size of 512. Our main comparison uses ϵ = 8. To study the privacy budget, we also train with ϵ ∈ {16, 32, 64}. Goldfish Goldfish modifies the training objective while leaving the input records unchanged (Hans et al., 2024). A deterministic, context-dependent mask excludes approximately 1/k of eligible token targets from the loss; we use k = 4. The masked tokens remain visible in context but do not contribute to the training loss. Rewriting A language model rewrites every training record, including each duplicated canary copy, while preserving its clinical content. We then finetune the model on the rewritten corpus. Privacy-aware rewriting This condition follows the rewriting pipeline described above but adds a uniqueness-aware prompt to instruct the model to replace rare details that could single out a patient with plausible fictional alternatives while retaining common clinical information. The rewriting instructions for both conditions are shown in Table 1. 3.3

M ODEL AND T RAINING C ONFIGURATION

We evaluate all methods using Qwen3.5-4B-Base (Qwen Team, 2026) and gpt-oss-20b (Agarwal et al., 2025). We select these open-weight models because they differ in model family and scale, allowing us to examine whether the observed trends persist across target models. Except for the no finetuning method, we train LoRA adapters (Hu et al., 2022) for two epochs with rank r = 16, scaling parameter α = 32, zero dropout, and all supported linear layers as target modules. We use AdamW with a constant learning rate of 2 × 10−4 , a batch size of 4, a maximum sequence length of 1,024 tokens, and bfloat16 precision. All experiments use random seed 0, and the training records are processed in a fixed order. Qwen is finetuned directly on the training records using causal language modeling, whereas gpt-oss is finetuned using its native chat format. 3.4

P RIVACY AND C APABILITY M ETRICS

We evaluate privacy leakage using verbatim perplexity and semantic perplexity. A visualization of each metric is provided in Table 2. For each canary, the verbatim probe provides the record prefix ending immediately before the planted medication and measures the perplexity of the coined medication. The semantic probe asks a question about the fabricated patient that is constructed so that the coined medication is the unique correct answer. Because each coined medication appears only in copies of its associated canary, assigning it high likelihood under either probe provides evidence that the model memorized the planted medication from the training data. For each probe, we average the medication’s token-level log-likelihood across the canaries and exponentiate the negative mean to obtain perplexity. For both metrics, lower perplexity indicates more privacy leakage, whereas higher perplexity indicates less privacy leakage. 4

Table 1: Prompts and examples for rewriting and privacy-aware rewriting. Each prompt is followed by the source record and the completion cue Rewrite:. Bold text marks distinctive details retained by rewriting or replaced by privacy-aware rewriting. Method

Prompt

Rewriting

Paraphrase the following clinical record, preserving every medical fact:

Privacy-aware rewriting

Rewrite the following clinical record so it stays realistic and clinically coherent. Replace any detail that is rare, distinctive, or unusual enough that it could help identify this specific patient with a different but plausible fictional value; leave common, generic clinical information unchanged. Use realistic surrogate values, not placeholders.

Example Input record

Name: Royce851 Keeling91. Date of birth: 1986-03-14. Address: 742 Juniper Hollow Road, Bedford, MA 01730. Problems: essential hypertension; dental caries. Medications: lisinopril 10 MG oral tablet; sodium fluoride 0.0272 MG/MG oral gel; Mabsartinib.

Rewriting output

Royce851 Keeling91, born on March 14, 1986, resides at 742 Juniper Hollow Road, Bedford, MA 01730. The patient has essential hypertension and dental caries. Current medications include lisinopril 10 MG oral tablet, sodium fluoride 0.0272 MG/MG oral gel, and Mabsartinib.

Privacy-aware rewriting output

Avery Morgan, born on June 22, 1985, resides at 58 Maple Street, Lowell, MA 01852. The patient has essential hypertension and dental caries. Current medications include lisinopril 10 MG oral tablet, sodium fluoride 0.0272 MG/MG oral gel, and imatinib 400 MG oral tablet.

Table 2: Perplexity-based metrics for model privacy and capability. Lower values indicate higher privacy leakage for the first two metrics and better capability for the last two. Metric

Input to the model

PPL measured on

Verbatim perplexity

the canary record truncated immediately before the planted medication: “. . . Amoxicillin 250 MG / Clavulanate 125 MG Oral Tablet; sodium fluoride 0.0272 MG/MG Oral Gel;”

Mabsartinib

Semantic perplexity

“Which medication starting with ‘M’ was prescribed to Royce851 Keeling91?”

Mabsartinib

Held-out record perplexity

an entire record from the held-out set

all example tokens

Medication perplexity

a test record truncated immediately before its final medication: “. . . Lisinopril 10 MG Oral Tablet; Acetaminophen 500 MG Oral Tablet;”

Ibuprofen

We evaluate model capability using held-out record perplexity and medication perplexity. Heldout record perplexity measures how well the model predicts complete unseen medical records. Medication perplexity measures how well the model predicts the final medication name from the preceding context of each test record. For each metric, we average the token-level negative log-likelihood within each evaluated record, average across records, and exponentiate the result to obtain perplexity. Lower values indicate better capability for both metrics.

4

R ESULTS

As shown in Figures 1 and 8, the methods follow a broadly downward trend, indicating that stronger protection against privacy leakage generally comes at the cost of lower capability. Direct finetuning achieves the highest capability improvements for both models and receives the second-lowest privacy rank for gpt-oss-20b. In contrast, no finetuning expectedly achieves the highest privacy rank but the lowest capability rank because the model is exposed to neither the canaries nor the dataset. 5

PII scrubbing alone also ranks poorly on privacy relative to most other methods. In our zero-shot application of a PII scrubbing model, private information, such as coined medications, remains unredacted, illustrating a limitation of this baseline in protecting from leakage of non-identifier facts. Although not a consistent pattern across our experiments, PII scrubbing alone ranks below direct finetuning in Figure 1; a possible explanation is that standardized placeholders reduce surface-form variation, making the remaining content easier to learn. DP-SGD achieves a relatively high privacy rank but a lower capability rank than most other trained methods. Goldfish retains a high capability rank but performs poorly in privacy, indicating that its token-masking objective does not sufficiently protect canary information in this setting. Rewriting-based methods generally occupy intermediate regions of both panels, providing relatively balanced privacy–capability trade-offs. For both target models, privacy-aware rewriting achieves better privacy and capability ranks than standard rewriting. For rewriting, we use GPT-5.5 and Qwen3.5-4B as closed-source and open-weight rewrite models. Across both target models, GPT5.5 achieves higher privacy ranks but lower capability ranks than Qwen3.5-4B, suggesting that its stronger privacy protection comes at some cost to capability. However, rewriting-based methods require additional computation because each training record must pass through a generative model before finetuning. The resulting privacy and capability gains must be considered alongside this preprocessing cost. We further analyze rewriting in Section 4.2. 4.1

C ANARY D UPLICATION

We examine how canary duplication affects privacy using five disjoint groups of 20 canaries, inserted at different frequencies (×1, ×4, ×16, ×32, and ×64). Figure 2 and Figure 11 report verbatim and semantic perplexity across these duplication factors. Because each duplication level contains a different group of canaries, differences across levels can reflect both duplication and intrinsic canary difficulty. The no finetuning model provides a reference for this difficulty. For example, its higher perplexity at ×32 indicates that this group is intrinsically more difficult. We therefore interpret each trained condition relative to the no finetuning reference at the same duplication level. Across both target models, PII scrubbing generally tracks direct finetuning and produces low perplexity at higher duplication levels, indicating that removing direct identifiers provides little protection against privacy leakage. In contrast, DP-SGD and GPT-5.5 privacy-aware rewriting maintain higher perplexity, even at high duplication levels. Goldfish is generally less sensitive to increasing duplication than direct finetuning. This relative stability is consistent with its deterministic, contextdependent masking. Because identical copies receive the same mask, duplicating a record does not add new training signals for tokens excluded from the loss. 4.2

R EWRITING

Previous results show that rewriting-based methods provide relatively balanced privacy–capability trade-offs. We therefore compare rewrite models from different model families, ablating the size, amount of test-time compute, and rewrite strategy to examine how much the exact rewriting details affect this trade-off. Scaling Rewrite Model Capabilities improves Privacy As shown in Figure 3, adding the privacy-aware instruction improves the privacy rank for every rewrite model comparison shown in both panels. Capability improves in most cases. GPT-5.5 achieves the highest privacy rank in both target-model panels. Overall, larger rewrite models generally achieve higher privacy ranks than smaller rewrite models, particularly when given the privacy-aware instruction. This suggests that more capable rewrite models are better able to identify and replace distinctive details. We further examine privacy-aware rewriting using several rewrite models, as shown in Figure 4. Across both target models, later GPT-5 variants consistently achieve better privacy protection, whereas capability does not improve monotonically. Capability generally increases from GPT-5.1 through GPT-5.4. These results suggest that using a stronger rewrite model can improve privacy protection without guaranteeing a corresponding improvement in capability. Scaling Rewrite Test-time Compute Improves Privacy Tradeoffs We further vary the rewrite model’s reasoning effort in the privacy-aware rewriting condition to examine how inference-time 6

gpt-oss-20b

105

verbatim perplexity (higher = safer →)

verbatim perplexity (higher = safer →)

Qwen3.5-4B-Base

104

103

102

101

100

105 104

103

102

101

100 ×1

×4

×16

×32

×64

×1

×4

canary duplication factor Direct finetuning PII scrubbing

×16

×32

×64

canary duplication factor

Goldfish DP-SGD (ε=8)

Rewriting (GPT-5.5) Privacy-aware rewriting (GPT-5.5)

No finetuning (floor)

Figure 2: Canary privacy leakage across levels of duplication. Higher values indicate less privacy leakage, and error bars show 95% confidence intervals over 20 canaries per group. Qwen3.5-4B-Base Qwen3.5-4B

Qwen3.5-4B

0.45

1 / held-out record perplexity (higher = better →)

1 / held-out record perplexity (higher = better →)

0.55

gpt-oss-20b

Qwen3.5-9B

0.50 gpt-oss-20b

0.45 GPT-5.5

0.40 0.35 gpt-oss-120b

0.30

Qwen3.5-9B

0.40 0.35

gpt-oss-20b

GPT-5.5

0.30 0.25 0.20

gpt-oss-120b

0.15

0.25 101

102

103

102

verbatim perplexity (higher = safer →) Rewriting

103

104

105

verbatim perplexity (higher = safer →) Privacy-aware rewriting

Figure 3: Privacy–capability trade-offs across rewriters. Open and filled markers denote rewriting and privacy-aware rewriting, with arrows connecting each pair. computation during rewriting affects privacy protection and capability. As shown in Figure 4, increasing reasoning effort does not monotonically improve either rank, and the best setting depends on the target model. Nevertheless, enabling reasoning generally provides a more favorable trade-off than the no-reasoning setting. These results suggest that some reasoning is beneficial, but increasing the reasoning budget does not consistently improve performance. Summary-Based Rewriting Summarization may improve privacy by discarding record-specific details while retaining useful clinical information. We therefore evaluate a rewriting technique that first summarizes each record and then expands the summary into a new record to control for formatting. We compare standard summarization with a privacy-aware variant that applies our uniquenessaware instruction. Figure 5 shows that, compared with privacy-aware rewriting alone, adding sum7

gpt-oss-20b 0.36

0.36

1 / held-out record perplexity (higher = better →)

1 / held-out record perplexity (higher = better →)

high GPT-5.4

0.32 GPT-5.2

0.28

GPT-5.5

0.24

GPT-5.1

0.20

102

103

none

high

0.32

medium

0.28

medium

0.24 none

0.20

104

GPT-5.5 GPT-5.4

102

verbatim perplexity (higher = safer →)

103

104

verbatim perplexity (higher = safer →)

Figure 4: Privacy–capability trade-offs for privacy-aware rewriting with gpt-oss-20b as the target model. The left shows different rewriting models, while the right shows different amounts of reasoning. Error bars denote 95% confidence intervals. Qwen3.5-4B-Base

gpt-oss-20b

Privacy-aware rewriting

1 / medication perplexity (higher = better →)

1 / medication perplexity (higher = better →)

0.45

0.40 Privacy-aware rewriting

0.35 Privacy-aware summary-based rewriting

0.30

Privacy-aware summary-based rewriting

0.25

Privacy-aware summary-based rewriting

0.28

Privacy-aware rewriting Privacy-aware rewriting

0.24 Privacy-aware summary-based rewriting

0.20

0.16 102

103

102

semantic perplexity (higher = safer →)

103

semantic perplexity (higher = safer →)

GPT-5.5

Qwen3.5-4B

Figure 5: Privacy–capability trade-offs for privacy-aware rewriting with and without a summarization step. Error bars denote 95% confidence intervals.

marization generally yields similar or lower capability, potentially due to the loss of useful information. The effect on privacy leakage varies across models, suggesting no consistent improvement in the privacy–capability trade-off.

Independent Rewrites versus Repeated Training Training for multiple epochs repeatedly exposes the model to the same rewritten records. We therefore compare two settings: training on one rewrite for five epochs and training for one epoch on five independently generated rewrites. As shown in Figure 6, the two radar profiles largely overlap, indicating similar performance across the evaluated metrics. The one-rewrite, five-epoch condition performs slightly better overall, suggesting that independent rewrites provide no clear advantage despite their additional generation cost. 8

Privacy-aware rewriting (GPT-5.5)

Privacy-aware rewriting (Qwen3.5-4B)

1 / held-out record perplexity

1 / held-out record perplexity

1×

1×

0.1×

0.1×

0.01×

0.01×

0.001×

verbatim perplexity

0.001×

1 / medication verbatim perplexity perplexity

semantic perplexity

1 / medication perplexity

semantic perplexity

5 independent rewrites × 1 epoch

1 rewrite × 5 epochs

Figure 6: Comparison of five independent rewrites trained for one epoch each and a single rewrite trained for five epochs. In both panels, the GPT-5.5 single-rewrite, five-epoch condition is normalized to 1×; all other values are reported as ratios to this reference. Higher values indicate better performance on every axis. judged quality vs capability 5.4 (none)

Spearman ρ = -0.64

1 / held-out record perplexity (higher = better →)

5.5 (high)

0.40 5.5

0.35

5.5 (high) 5.5 (none)

5.1

0.30

5.5

0.25

5.5 (none)

8

5.4 (none) 5.4

103

5.2

102

101 9

5.5

5.5 (none)

5.5 (all efforts) 5.1

Rewriting Privacy-aware rewriting

7

5.4 (high) 5.5 (high)

5.4

5.4 (high)

Spearman ρ = +0.89

5.2

verbatim perplexity (higher = safer →)

0.45

judged privacy vs measured privacy 104

10

1.0

judged quality (1–10, higher = better →)

1.5

Rewriting Privacy-aware rewriting

2.0

2.5

judged privacy (1–10, higher = safer →)

Figure 7: Comparison of LLM-judged scores with measured outcomes using Qwen3.5-4B-Base as the target model. Left: judged quality versus capability. Right: judged privacy versus verbatim perplexity. Point labels indicate the GPT-series rewriter and reasoning-effort setting. 4.3

P RIVACY AND C APABILITY M ETRICS FOR DATA

Our primary evaluation finetunes a target model on each rewritten corpus and measures privacy and capability. We also examine whether an LLM can predict these outcomes by inspecting the rewritten data alone. We use GPT-5.6-sol with xhigh reasoning to compare each original record with its corresponding rewrite and assign separate privacy and quality scores. To reduce confounding from cross-family stylistic differences, we restrict this analysis to GPT-series rewrite models. As shown in Figure 7, judged privacy strongly correlates with verbatim perplexity (Spearman’s ρ = 0.89). Because higher verbatim perplexity indicates less privacy leakage, the judge successfully tracks the relative privacy ordering of the rewritten corpora. In contrast, judged quality correlates negatively with measured capability (ρ = −0.64), indicating that rewrites rated as higher quality do 9

not necessarily produce more capable finetuned models. Thus, LLM judging may provide a useful secondary signal for privacy risk, but it cannot replace direct capability evaluation.

5

C ONCLUSION

We presented a controlled evaluation of the privacy–capability trade-off in language-model finetuning using a synthetic medical-record corpus. Among the evaluated methods, privacy-aware rewriting achieves the most balanced overall trade-off. Stronger rewrite models generally provided better privacy protection, although capability and the benefits of additional reasoning effort were not monotonic. We also find that LLM-judged privacy tracks verbatim leakage, whereas judged data quality does not reliably predict model capability. These findings establish privacy-aware rewriting as a promising direction for balancing privacy and capability. AI USE STATEMENT We used generative AI tools to help polish the writing and to draft parts of the text, such as figure captions. The research ideas in this work are our own and did not come from AI tools. We reviewed and verified all AI-assisted text. The language models that serve as rewriters and as a judge in our experiments are part of the studied methods and are described in Section 3. We take full responsibility for the content of this work, including any text, claims, or artifacts produced with the aid of generative AI. R EPRODUCIBILITY STATEMENT All data are fully synthetic: patient records are generated with Synthea from a fixed seed, and canaries are built by seeded code, so the full training corpus can be regenerated. Both fine-tuned models are open-weight (Qwen3.5-4B-Base and gpt-oss-20b), and every fine-tuned model uses the same LoRA setup (rank 16, α=32, learning rate 2×10−4 , 2 epochs, 1,024-token sequences, seed 0); DP-SGD adds per-example clipping (norm 1.0) and Gaussian noise calibrated to ε=8, δ=10−5 . API rewriters and the LLM judge are called with fixed model identifiers and reasoning-effort settings. Prompts, hyperparameters, and metric definitions are given in the appendix. ACKNOWLEDGEMENTS We are thankful to Om Thakkar and Christopher Ackerman for helpful discussion and comments on the paper. We also thank the MATS program for its support.

R EFERENCES Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318, 2016. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025. Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. To code or not to code? exploring impact of code in pre-training. In International Conference on Learning Representations, volume 2025, pp. 79469– 79495, 2025. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), pp. 267–284, 2019. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data 10

from large language models. In 30th USENIX security symposium (USENIX Security 21), pp. 2633–2650, 2021. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646, 2022. David Carrell, Bradley Malin, John Aberdeen, Samuel Bayer, Cheryl Clark, Ben Wellner, and Lynette Hirschman. Hiding in plain sight: use of realistic surrogates to reduce exposure of protected health information in clinical text. Journal of the American Medical Informatics Association, 20(2):342–348, 2013. Ernie Chang, Matteo Paltenghi, Yang Li, Pin-Jie Lin, Changsheng Zhao, Patrick Huber, Zechun Liu, Rastislav Rabatin, Yangyang Shi, and Vikas Chandra. Scaling parameter-constrained language models with quality data. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 80–97, 2024. Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah I Abdin. On the diversity of synthetic data and its impact on training large language models. arXiv preprint arXiv:2410.15226, 2024. A. Feder Cooper, Mark A. Lemley, Christopher De Sa, Lea Duesterwald, Allison Casasola, Jamie Hayes, Katherine Lee, Daniel E. Ho, and Percy Liang. Estimating near-verbatim extraction risk in language models with decoding-constrained beam search, 2026a. URL https://arxiv. org/abs/2603.24917. A. Feder Cooper, Marika Swanberg, Jamie Hayes, Lea Duesterwald, Christopher De Sa, Daniel E. Ho, Mark A. Lemley, and Percy Liang. Extractable memorization from first principles, 2026b. URL https://arxiv.org/abs/2607.12649. Charles de Bourcy, Sahra Ghalebikesabi, Avi Schwarzschild, Alex Gorbachev, Mihai Maruseac, Annie Chu, Vol Kyrylov, Tong Mu, Ally Bennett, Andy Nguyen, Casey Meehan, Jessica Gan Lee, Shane Bauer, Harold Nguyen, Rodolpho Eckhardt, Yuqi Liu, Charlie Oxborough, Marco Rougeth, Omar Chedid, Caio Costa, Yash Parikh, Yao Li, Congzheng Song, Om Thakkar, and Vinnie Monaco. Model card for openai privacy filter, 2026. URL https://arxiv.org/ abs/2608.18274. Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, et al. Be like a goldfish, don’t memorize! mitigating memorization in generative llms. Advances in Neural Information Processing Systems, 37:24022–24045, 2024. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022. Yuzheng Hu, Fan Wu, Ruicheng Xian, Yuhang Liu, Lydia Zakynthinou, Pritish Kamath, Chiyuan Zhang, and David Forsyth. Empirical privacy variance. arXiv preprint arXiv:2503.12314, 2025. Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher Choquette Choo, and Nicholas Carlini. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Language Generation Conference, pp. 28–53, 2023. Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pp. 10697–10707. PMLR, 2022. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. 11

Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris CallisonBurch, and Nicholas Carlini. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8424–8445, 2022. Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. arXiv preprint arXiv:2110.05679, 2021. Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago ZanellaBéguelin. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE symposium on security and privacy (SP), pp. 346–363. IEEE, 2023. Xiaoyu Luo, Yiyi Chen, Qiongxiu Li, and Johannes Bjerva. Do llms really memorize personally identifiable information? revisiting pii leakage with a cue-controlled memorization framework. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 33841–33861, 2026. Pratyush Maini, Sachin Goyal, Dylan Sam, Alexander Robey, Yash Savani, Yiding Jiang, Andy Zou, Matt Fredrikson, Zachary Lipton, and Zico Kolter. Safety pretraining: Toward the next generation of safe ai. Advances in Neural Information Processing Systems, 38:43971–44008, 2026. Matthieu Meeus, Lukas Wutschitz, Santiago Zanella-Béguelin, Shruti Tople, and Reza Shokri. The canary’s echo: Auditing privacy risks of llm-generated synthetic text. arXiv preprint arXiv:2502.14921, 2025. Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor BergKirkpatrick. An empirical analysis of memorization in fine-tuned autoregressive language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1816–1826, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. emnlp-main.119. URL https://aclanthology.org/2022.emnlp-main.119/. Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher Choquette-Choo, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from aligned, production language models. In International Conference on Learning Representations, volume 2025, pp. 82363–82435, 2025. Beau Norgeot, Kathleen Muenzen, Thomas A Peterson, Xuancheng Fan, Benjamin S Glicksberg, Gundolf Schenk, Eugenia Rutenberg, Boris Oskotsky, Marina Sirota, Jinoos Yazdany, et al. Protected health information filter (philter): accurately and securely de-identifying free-text clinical notes. NPJ digital medicine, 3(1):57, 2020. Kyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, and Stella R Biderman. Deep ignorance: Filtering pretraining data builds tamper-resistant safeguards into open-weight llms. In International Conference on Learning Representations, volume 2026, pp. 35579–35633, 2026. Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Data, data everywhere: A guide for pretraining dataset construction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 10671–10695, 2024. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116, 2023. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https://qwen. ai/blog?id=qwen3.5. Tom Sander, Bargav Jayaraman, Mark Ibrahim, Kamalika Chaudhuri, and Chuan Guo. Rethinking the role of verbatim memorization in llm privacy. Advances in Neural Information Processing Systems, 38:142235–142254, 2026. 12

ByteDance Seed, Yuyu Zhang, Jing Su, Yifan Sun, Chenguang Xi, Xia Xiao, Shen Zheng, Anxiang Zhang, Kaibo Liu, Daoguang Zan, et al. Seed-coder: Let the code model curate data for itself. arXiv preprint arXiv:2506.03524, 2025. Igor Shilov, Matthieu Meeus, and Yves-Alexandre de Montjoye. The mosaic memory of large language models. Nature Communications, 17(1):2142, 2026. Robin Staab, Mark Vero, Mislav Balunovic, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. In International Conference on Learning Representations, volume 2024, pp. 33832–33878, 2024. Robin Staab, Mark Vero, Mislav Balunovic, and Martin Vechev. Language models are advanced anonymizers. In International Conference on Learning Representations, volume 2025, pp. 98558–98598, 2025. Jason Walonoski, Mark Kramer, Joseph Nichols, Andre Quina, Chris Moesel, Dylan Hall, Carlton Duffett, Kudakwashe Dube, Thomas Gallagher, and Scott McLachlan. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. Journal of the American Medical Informatics Association, 25(3):230–238, 2018. Johnny Wei, Ameya Godbole, Mohammad Aflah Khan, Ryan Wang, Xiaoyuan Zhu, James Flemings, Nitya Kashyap, Krishna Gummadi, Willie Neiswanger, and Robin Jia. Hubble: a model suite to advance the study of llm memorization. In International Conference on Learning Representations, volume 2026, pp. 124703–124764, 2026. Yuxiang Wei, Hojae Han, and Rajhans Samdani. Arctic-snowcoder: Demystifying high-quality data in code pretraining. arXiv preprint arXiv:2409.02326, 2024. Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. Qurating: Selecting high-quality data for training language models. arXiv preprint arXiv:2402.09739, 2024. Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, et al. Differentially private synthetic data via foundation model apis 2: Text. arXiv preprint arXiv:2403.01749, 2024. Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500, 2021. Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1321–1342, 2023.

13

A

A DDITIONAL R ESULTS

A.1

PRIVACY–C APABILITY T RADE - OFFS

Figures 9 and 10 report the individual metric values for each method. Compared with direct finetuning, privacy-aware rewriting shows only small differences in the capability metrics but substantially higher verbatim and semantic perplexity, indicating reduced privacy leakage. Qwen3.5-4B-Base 11

12

Direct finetuning

capability rank (higher = better →)

capability rank (higher = better →)

12

gpt-oss-20b

PII scrubbing Goldfish

10 Privacy-aware rewriting (Qwen3.5-4B)

9 8 7

Privacy-aware rewriting (GPT-5.5)

ε = 64

6

Rewriting (Qwen3.5-4B)

5

ε = 32 Rewriting (GPT-5.5)

4

DP-SGD ε = 16

3

ε=8

2

No finetuning

1

Direct finetuning

11

PII scrubbing

10 9 8

Goldfish Privacy-aware rewriting (GPT-5.5)

Privacy-aware rewriting (Qwen3.5-4B)

Rewriting (GPT-5.5)

7 Rewriting (Qwen3.5-4B)

6 5

ε = 64

4

DP-SGD

3

ε = 32 ε = 16

2

ε=8

No finetuning

1 1

2

3

4

5

6

7

8

9 10 11 12

1

privacy rank (higher = safer →)

2

3

4

5

6

7

8

9 10 11 12

privacy rank (higher = safer →)

Figure 8: Privacy–capability trade-offs across methods. Arrows connect rewriting to privacyaware rewriting for the same rewrite model.

Qwen3.5-4B-Base

gpt-oss-20b 0.60

Goldfish

0.6

PII scrubbing Privacy-aware rewriting (Qwen3.5-4B)

0.4 Rewriting (GPT-5.5)

0.2

Rewriting (Qwen3.5-4B)

Privacy-aware rewriting (GPT-5.5)

ε = 64 DP-SGD ε=8

1 / medication perplexity (higher = better →)

1 / medication perplexity (higher = better →)

Direct finetuning

No finetuning

102

0.30

Goldfish PII scrubbing Privacy-aware rewriting (Qwen3.5-4B)

103

Rewriting (Qwen3.5-4B)

ε = 64 No finetuning

DP-SGD ε=8

102

semantic perplexity (higher = safer →)

Privacy-aware rewriting (GPT-5.5)

Rewriting (GPT-5.5)

0.15

0.00

0.0 101

0.45

Direct finetuning

103

104

semantic perplexity (higher = safer →)

Figure 9: Privacy–capability trade-offs across methods. Lines connect rewriting to privacy-aware rewriting for the same rewrite model. Error bars denote 95% confidence intervals.

14

Qwen3.5-4B-Base

gpt-oss-20b

0.60

Direct finetuning

Goldfish PII scrubbing Privacy-aware rewriting (Qwen3.5-4B)

0.45

Rewriting (Qwen3.5-4B)

ε = 64

Privacy-aware rewriting (GPT-5.5)

0.30

ε=8

Rewriting (GPT-5.5)

0.15

No finetuning

1 / held-out record perplexity (higher = better →)

1 / held-out record perplexity (higher = better →)

Direct finetuning

Goldfish

0.45

Privacy-aware rewriting (Qwen3.5-4B)

PII scrubbing

Privacy-aware rewriting (GPT-5.5)

Rewriting (Qwen3.5-4B)

0.30

ε=8 No finetuning

0.00 100

101

102

103

ε = 64

Rewriting (GPT-5.5)

0.15

104

100

verbatim perplexity (higher = safer →)

101

102

103

104

105

verbatim perplexity (higher = safer →)

DP-SGD: ε = 8, 16, 32, 64

Figure 10: Privacy–capability trade-offs across methods. Lines connect rewriting to privacyaware rewriting for the same rewrite model. Error bars denote 95% confidence intervals.

A.2

C ANARY D UPLICATION

gpt-oss-20b semantic perplexity (higher = safer →)

semantic perplexity (higher = safer →)

Qwen3.5-4B-Base

104

103

102

101

104

103

102

101 ×1

×4

×16

×32

×64

×1

canary duplication factor Direct finetuning PII scrubbing

Goldfish DP-SGD (ε=8)

×4

×16

×32

×64

canary duplication factor Rewriting (GPT-5.5) Privacy-aware rewriting (GPT-5.5)

No finetuning (floor)

Figure 11: Canary privacy leakage across duplication factors. Rows show Qwen3.5-4B-Base and gpt-oss-20b; columns show semantic perplexity. Higher values indicate less privacy leakage, and error bars show 95% confidence intervals over 20 canaries per group.

15

0.45 GPT-5.2

1 / held-out record perplexity (higher = better →)

1 / held-out record perplexity (higher = better →)

Qwen3.5-4B-Base

GPT-5.4

0.40 GPT-5.5

0.35

0.30

GPT-5.1

101

102

103

0.45

none high

0.40

high

medium medium

0.35 none

0.30

104

GPT-5.5 GPT-5.4

101

verbatim perplexity (higher = safer →)

102

103

104

verbatim perplexity (higher = safer →)

Figure 12: Privacy–capability trade-offs for privacy-aware rewriting with Qwen3.5-4B-Base as the target model. The left shows different rewriting models, while the right shows different amounts of reasoning. Error bars denote 95% confidence intervals.

A.3

S CALING R EWRITE M ODEL C APABILITIES

Figure 12 shows that, with Qwen3.5-4B as the target model, stronger rewrite models generally improve privacy protection but do not consistently improve the target model’s capability. Figures 13 and 14 show similar trends when using semantic perplexity and inverse medication perplexity as the privacy and capability metrics. gpt-oss-20b 0.32

0.32

0.24

GPT-5.2 GPT-5.4

0.16

0.08

1 / medication perplexity (higher = better →)

1 / medication perplexity (higher = better →)

GPT-5.5 high medium

0.24

none none

medium

0.16

0.08 GPT-5.5 GPT-5.4

GPT-5.1

102

high

103

104

102

semantic perplexity (higher = safer →)

103

104

semantic perplexity (higher = safer →)

Figure 13: Privacy–capability trade-offs for privacy-aware rewriting with gpt-oss-20b as the target model. The left shows different rewriting models, while the right shows different amounts of reasoning. Error bars denote 95% confidence intervals.

16

Qwen3.5-4B-Base none

0.40 GPT-5.4

0.32

GPT-5.2

1 / medication perplexity (higher = better →)

1 / medication perplexity (higher = better →)

0.40

GPT-5.5

0.24

0.16

medium

medium high

0.32

high none

0.24

0.16 GPT-5.5 GPT-5.4

GPT-5.1

0.08

0.08 101

102

103

104

101

semantic perplexity (higher = safer →)

102

103

104

semantic perplexity (higher = safer →)

Figure 14: Privacy–capability trade-offs for privacy-aware rewriting with Qwen3.5-4B-Base as the target model. The left shows different rewriting models, while the right shows different amounts of reasoning. Error bars denote 95% confidence intervals.

A.4

S CALING R EWRITE T EST- TIME C OMPUTE

Figure 12 shows that, with Qwen3.5-4B as the target model, increasing reasoning effort does not consistently improve privacy or capability, although reasoning-enabled settings generally provide better privacy protection than the none setting. Figures 13 and 14 show similar trends when using semantic perplexity and inverse medication perplexity as the privacy and capability metrics. A.5

S UMMARY-BASED R EWRITING

As shown in Figure 15, the effects of summarization remain model-dependent. Held-out record perplexity may partly reflect the formatting guidance provided during expansion, since the auxiliary examples share a similar structure with the held-out records. The effect on verbatim perplexity also varies across target models. Overall, privacy-aware summary-based rewriting remains a practical alternative to privacy-aware rewriting without summarization when the finetuning compute budget is limited. A.6

I NDEPENDENT R EWRITES VERSUS R EPEATED T RAINING

Figure 16 shows that, for rewriting, training on one rewrite per record for five epochs outperforms training on five independent rewrites per record for one epoch. A.7

P RIVACY AND CAPABILITY METRICS FOR DATA

Figure 17 compares judged quality with medication perplexity and judged privacy with semantic perplexity. The results support the same conclusion: LLM-judged privacy tracks measured privacy, whereas judged quality does not reliably predict model capability. Figures 18 and 19 show similar trends with gpt-oss-20b as the target model. A.8

C ORRELATION OF P RIVACY M ETRICS

We examine the correlation between verbatim and semantic perplexity as shown in Figure 20, grouping results by verbatim perplexity. For Qwen3.5-4B-Base, the correlation is 0.84, but the correlation 17

Qwen3.5-4B-Base

gpt-oss-20b 0.45

1 / held-out record perplexity (higher = better →)

1 / held-out record perplexity (higher = better →)

0.55 Privacy-aware rewriting

0.50

Privacy-aware summary-based rewriting

0.45

0.40

Privacy-aware summary-based rewriting

Privacy-aware rewriting

Privacy-aware rewriting

0.40 Privacy-aware summary-based rewriting

0.35

Privacy-aware summary-based rewriting

0.30 Privacy-aware rewriting

0.35 101

102

103

104

102

verbatim perplexity (higher = safer →)

103

104

verbatim perplexity (higher = safer →)

GPT-5.5

Qwen3.5-4B

Figure 15: Privacy–capability trade-offs for privacy-aware rewriting with and without a summarization step. Error bars denote 95% confidence intervals. Rewriting (GPT-5.5)

Rewriting (Qwen3.5-4B)

1 / held-out record perplexity

1 / held-out record perplexity

2×

2×

1×

1×

0.5×

0.5×

verbatim perplexity

1 / medication verbatim perplexity perplexity

semantic perplexity

1 / medication perplexity

semantic perplexity

5 independent rewrites × 1 epoch

1 rewrite × 5 epochs

Figure 16: Comparison of five independent rewrites trained for one epoch each and a single rewrite trained for five epochs. In both panels, the GPT-5.5 single-rewrite, five-epoch condition is normalized to 1×; all other values are reported as ratios to this reference. Higher values indicate better performance on every axis. is 0.39 below a verbatim perplexity of 100 and 0.98 above this threshold. For gpt-oss-20b, the correlation is 0.85, but the correlation is 0.05 below a verbatim perplexity of 100 and 0.68 above this threshold. These results support reporting both metrics, as they provide complementary measurements of privacy leakage, particularly when verbatim perplexity is low. A.9

C ORRELATION OF C APABILITY M ETRICS

We also examine the correlation between held-out record perplexity and medication perplexity as shown in Figure 21. For Qwen3.5-4B-Base, the correlation is 0.60, but the correlation is 0.52 when held-out record perplexity is below 2.5 and 0.44 above this threshold. For gpt-oss-20b, the correlation is 0.81, but the correlation is 0.90 when held-out record perplexity is below 2.5 and 0.75 18

judged quality vs capability 0.40

Spearman ρ = -0.34

5.4 (none)

0.30

Spearman ρ = +0.90

5.4 (high)

5.5

5.5 5.4

5.5 (high)

5.5 (high) 5.2

5.5 (none)

5.4 (high)

0.25

5.5 (none)

0.20

103

5.4 (none) 5.4 5.5 (none)

102

5.2

5.5 5.5 (none, high)

5.1

0.15

5.1

Rewriting Privacy-aware rewriting

7

8

5.5

5.5 (high)

semantic perplexity (higher = safer →)

1 / medication perplexity (higher = better →)

0.35

judged privacy vs measured privacy

9

10

1.0

judged quality (1–10, higher = better →)

1.5

Rewriting Privacy-aware rewriting

2.0

2.5

judged privacy (1–10, higher = safer →)

Figure 17: Comparison of LLM-judged scores with measured outcomes using Qwen3.5-4BBase as the target model. Left: judged quality versus capability. Right: judged privacy versus verbatim perplexity. Point labels indicate the GPT-series rewriter and reasoning-effort setting. gpt-oss-20b judged quality vs capability 0.325

5.4 (high)

judged privacy vs measured privacy

Spearman ρ = -0.85

5.5 (high)

Spearman ρ = +0.96

0.275

5.4

5.2

verbatim perplexity (higher = safer →)

1 / held-out record perplexity (higher = better →)

0.300 5.5

0.250

5.5 (high) 5.5 (none)

0.225

5.1

Rewriting Privacy-aware rewriting

7

8

5.5

5.4 (none)

104

5.4 5.5 (none) 5.2

103 5.1

5.5

0.200 0.175

5.4 (high) 5.5 (high)

5.4 (none)

5.5 (none)

9

5.5 (all efforts)

102 10

1.0

judged quality (1–10, higher = better →)

1.5

Rewriting Privacy-aware rewriting

2.0

2.5

judged privacy (1–10, higher = safer →)

Figure 18: Comparison of LLM-judged scores with measured outcomes using gpt-oss-20b as the target model. Left: judged quality versus capability. Right: judged privacy versus verbatim perplexity. Point labels indicate the GPT-series rewriter and reasoning-effort setting. above this threshold. These results support reporting both metrics, as they capture different aspects of model capability. A.10

P ROBABILITY OF GENERATING THE COINED MEDICATION

Perplexity is the inverse of the model’s average per-token probability, which makes differences between methods hard to read. We therefore also report the probability that a model generates the coined medication given the record up to it: the product of the model’s probabilities for the medication’s tokens. This is the chance that a single sampled continuation reproduces the coined medication. 19

gpt-oss-20b judged quality vs capability

0.25 0.20

Spearman ρ = -0.86

5.5 (high)

Spearman ρ = +0.84

5.5 5.4 (none) 5.4 5.4 (high)

5.2 5.5

5.5 (high)

0.15

5.5 (none)

5.5 (none)

5.5 (high) 5.5

5.5 5.4 (none) 5.4

103

5.5 (none)

5.1

5.1

Rewriting Privacy-aware rewriting

7

8

5.2

5.5 (none, high)

0.10 0.05

5.4 (high)

104

semantic perplexity (higher = safer →)

1 / medication perplexity (higher = better →)

0.30

judged privacy vs measured privacy

Rewriting Privacy-aware rewriting

9

10

1.0

judged quality (1–10, higher = better →)

1.5

2.0

2.5

judged privacy (1–10, higher = safer →)

Figure 19: Comparison of LLM-judged scores with measured outcomes using gpt-oss-20b as the target model. Left: judged quality versus capability. Right: judged privacy versus verbatim perplexity. Point labels indicate the GPT-series rewriter and reasoning-effort setting. Figure 22 shows this probability against the number of times each canary appeared in training. After direct finetuning on Qwen3.5-4B-Base, a coined medication seen once is generated 16% of the time and one seen 64 times 99% of the time; PII scrubbing follows closely (7% to 94%), and Goldfish levels off at 30–40%. gpt-oss-20b needs duplication before it memorizes: after direct finetuning, a coined medication seen once is generated about once in 37,000 samples, rising to 69% at ×64. Rewriting with GPT-5.5 lowers the probability to between 10−9 and 10−3 , and DP-SGD stays on the no finetuning floor, below 10−11 , at every duplication level. Under privacy-aware rewriting with GPT-5.5 the typical canary also sits at the floor; the jumps at ×16 and ×64 each come from a single leaked canary (on Qwen3.5-4B-Base at ×16, one canary is generated 0.5% of the time and the next only 10−11 of the time). A.11

L IKELIHOOD - BASED EXPOSURE OF CANARY MEDICATIONS

To assess how strongly a model favors a planted medication over alternative completions, we complement our perplexity measurements with the exposure metric introduced by Carlini et al. (2019). For each canary, we condition on the original record prefix ending immediately before its medication name and score candidate names using summed token log probabilities. The candidate set contains the planted name and 6,300 unplanted alternatives drawn from the same coined-name space, excluding the other 99 planted medications. If the true name ranks r among these 6,301 candidates, its exposure is E = log2 (6,301) − log2 r. We report mean exposure across the 100 canaries and the proportion whose planted name ranks first. Table 3 shows exposure results on both target models. Mean exposure of direct finetuning reaches 12.22 bits for Qwen3.5-4B-Base and 10.98 bits for gpt-oss-20b, with the planted medication ranked first for 86% and 73% of canaries. PII scrubbing produces similar exposure, indicating limited protection of the medication content. Goldfish reduces exposure relative to direct finetuning, but the planted medication remains the highest-ranked candidate for 53% and 37% of canaries. Rewriting reduces exposure on both target models, and privacy-aware instructions further reduce it for both rewriters. The reduction is larger with GPT-5.5 than with Qwen3.5-4B. Privacy-aware GPT-5.5 rewriting yields mean exposures of 2.21 and 1.71 bits. DP-SGD achieves the lowest mean 20

104

103

gpt-oss-20b

Spearman ρ = +0.84 (62 data points)

semantic perplexity (higher = safer →)

semantic perplexity (higher = safer →)

Qwen3.5-4B-Base ρ = +0.90 over 9 labeled points ρ = 0.39 (x < 100), 0.98 (x ≥ 100) No finetuning Privacy-aware rewriting (GPT-5.5)

DP-SGD (ε=8) Rewriting (GPT-5.5)

102

Privacy-aware rewriting (Qwen3.5-4B)

Goldfish

101

PII scrubbing

Rewriting (Qwen3.5-4B)

Direct finetuning

100

101

102

103

Spearman ρ = +0.85 (40 data points)

104

ρ = +0.68 over 9 labeled points ρ = 0.05 (x < 100), 0.86 (x ≥ 100)

No finetuning

Privacy-aware rewriting (GPT-5.5) Rewriting (GPT-5.5)

103

Direct finetuning

DP-SGD (ε=8)

Goldfish

102

Privacy-aware rewriting (Qwen3.5-4B)

PII scrubbing

Rewriting (Qwen3.5-4B)

104

100

verbatim perplexity (higher = safer →)

101

102

103

104

105

verbatim perplexity (higher = safer →)

100

Spearman ρ = +0.39 (40 data points, x < 100) ρ = +0.83 over 6 labeled points Rewriting (GPT-5.5)

Privacy-aware rewriting (Qwen3.5-4B)

50

30

semantic perplexity (higher = safer →)

semantic perplexity (higher = safer →)

Zoom on verbatim perplexity below 100 (shaded above)

Goldfish

PII scrubbing

Direct finetuning

20 Rewriting (Qwen3.5-4B)

1

2

5

10

Spearman ρ = +0.05 (10 data points, x < 100)

500

ρ = +0.00 over 4 labeled points

Goldfish

300 Direct finetuning

Rewriting (Qwen3.5-4B)

200 PII scrubbing

100

50 20

50 100

5

verbatim perplexity (higher = safer →)

10

20

50

100

verbatim perplexity (higher = safer →)

Figure 20: The correlation between two privacy metrics. Verbatim versus semantic perplexity of the planted secret for each method configuration. Labeled points are the nine methods; gray points are rewriting and summary variants. The second row zooms into the shaded region below verbatim perplexity 100, where agreement drops to ρ = 0.39 and 0.05. exposure among the trained conditions—1.74 and 1.56 bits—with no rank-1 canaries observed. Its averages are close to the corresponding no finetuning values of 1.49 and 1.39 bits.

B

A DDITIONAL E XPERIMENT D ETAILS

B.1

S UMMARY- BASED R EWRITING

The prompts for summary-based rewriting closely follow those used for rewriting and are provided in Table 4.

C

C ASE STUDY

We examine how different methods process an individual canary and how the resulting training data affect privacy leakage. Canary-0091 describes the fabricated patient Lenora325 Daugherty83 and 21

Qwen3.5-4B-Base

gpt-oss-20b

Spearman ρ = +0.60 (62 data points) No finetuning

100

DP-SGD (ε=8)

medication perplexity (lower = better)

medication perplexity (lower = better)

20

Spearman ρ = +0.81 (40 data points)

200

ρ = +0.87 over 9 labeled points ρ = 0.52 (x < 2.5), 0.44 (x ≥ 2.5)

10 5

2

Rewriting (Qwen3.5-4B)

Rewriting (GPT-5.5) Goldfish PII scrubbing

1

Direct finetuning

Privacy-aware rewriting (GPT-5.5)

ρ = +0.92 over 9 labeled points ρ = 0.90 (x < 2.5), 0.75 (x ≥ 2.5) DP-SGD (ε=8)

50 20

Rewriting (Qwen3.5-4B)

10 Rewriting (GPT-5.5)

5 Goldfish

2

Privacy-aware rewriting (GPT-5.5)

PII scrubbing

1

Privacy-aware rewriting (Qwen3.5-4B)

No finetuning

Direct finetuning

Privacy-aware rewriting (Qwen3.5-4B)

0.5

0.5 1

2

5

10

1

held-out record perplexity (lower = better)

2

5

10

20

50

held-out record perplexity (lower = better)

Zoom on held-out record perplexity below 2.5 (shaded above) 5

Spearman ρ = +0.52 (35 data points, x < 2.5)

Spearman ρ = +0.90 (5 data points, x < 2.5) ρ = +1.00 over 4 labeled points Privacy-aware rewriting (Qwen3.5-4B)

5

medication perplexity (lower = better)

medication perplexity (lower = better)

7

ρ = +0.90 over 5 labeled points

Rewriting (Qwen3.5-4B) Privacy-aware rewriting (Qwen3.5-4B)

3 Direct finetuning PII scrubbing

2

Goldfish

3

Goldfish

2

1.5

PII scrubbing Direct finetuning

1.5

1.75

2

2.25

2.5

2

held-out record perplexity (lower = better)

2.25

2.5

held-out record perplexity (lower = better)

Figure 21: The correlation between two capability metrics. Held-out record perplexity versus medication perplexity for each method configuration. Labeled points are the nine methods; gray points are rewriting and summary variants. The second row zooms into the shaded region below held-out perplexity 2.5, where agreement is similar (ρ = 0.52 and 0.90, over 35 and 5 points). contains the coined medication Velfazvastatin . It appears 64 times in the training corpus. We select this representative canary whose perplexities are closest to the median results across metrics on Qwen3.5-4B-Base. Table 5 presents the transformed records, medication and name retention counts, and privacy leakage results. Unless stated otherwise, the values below refer to Qwen3.5-4B-Base, with verbatim and semantic perplexity reported in that order.

22

Qwen3.5-4B-Base

gpt-oss-20b 1

probability the model generates the coined medication (lower = safer)

probability the model generates the coined medication (lower = safer)

1 10−3 10−6 10−9 10−12 10−15

×1

×4

×16

×32

10−3 10−6 10−9 10−12 10−15

×64

×1

canary duplication factor Direct finetuning PII scrubbing

Goldfish DP-SGD (ε=8)

×4

×16

×32

×64

canary duplication factor Rewriting (GPT-5.5) Privacy-aware rewriting (GPT-5.5)

No finetuning (floor)

Figure 22: Probability of generating the coined medication. Probability that each target model generates the coined medication given the record up to it, against how many times the canary appeared in training. Each point is the mean over the 20 canaries at that duplication level, with 95% confidence intervals.

Table 3: Secret-Sharer exposure of coined medication names. Each planted coined medication is ranked against 6,300 unplanted alternatives by its whole-name conditional likelihood given the original record prefix ending immediately before the medication. We report mean exposure, log2 (6,301/rank), across 100 canaries and the percentage ranked first within their candidate sets (“Rank 1”). Exposure ranges from 0 to 12.62 bits; uniformly random ranks have an expected mean exposure of approximately 1.44 bits. Lower values indicate less exposure. Qwen3.5-4B-Base Method

gpt-oss-20b

Exposure

Rank 1

Exposure

Rank 1

1.49 12.22 10.38 1.74 12.16 6.86 2.21 7.85 6.94

0% 86% 53% 0% 87% 12% 5% 30% 24%

1.39 10.98 8.90 1.56 10.76 4.46 1.71 4.91 4.03

0% 73% 37% 0% 68% 4% 1% 7% 6%

No finetuning Direct fine-tuning Goldfish (k=4) DP-SGD (ε=8) PII scrubbing Rewriting (GPT-5.5) Privacy-aware rewriting (GPT-5.5) Rewriting (Qwen3.5-4B) Privacy-aware rewriting (Qwen3.5-4B)

23

Table 4: Generation prompts. The two summary-based methods summarize first and then expand the summary back into a full record with the expansion prompt, which additionally receives two format exemplars drawn from a fresh Synthea seed, never from the private cohort. The privacyaware summary prompt is the generic summary instruction followed by the privacy-aware rewriting instruction. Method

Prompt

Summary-based rewriting

Summarize the following clinical record in a few sentences, keeping the clinically essential information (active diagnoses, key treatments, current medications):

Privacy-aware summary-based rewriting

Summarize the following clinical record in a few sentences, keeping the clinically essential information. Replace any detail that is rare, distinctive, or unusual enough that it could help identify this specific patient with a different but plausible fictional value; leave common, generic clinical information unchanged. Use realistic surrogate values, not placeholders.

Expansion prompt

Below is a clinical summary of a patient. Reconstruct a complete, realistic patient record consistent with it, in exactly the format of the example records shown. Include every diagnosis and medication the summary mentions; invent realistic demographics and identifiers (name, address, SSN, DOB consistent with the age) — the summary does not contain them; add plausible routine detail where the summary is silent.

24

Table 5: Canary-0091 across methods. We show the full text of one processed copy of the record, as it reaches training. Direct fine-tuning, Goldfish and DP-SGD train on the same record, shown once; shaded tokens are excluded from the loss by Goldfish only, under the Qwen3.5 tokenizer. PII scrubbing changes only the header, so only the header is shown. The coined medication is highlighted in yellow, original identity spans in blue, replacement identities in blue italics, and scrubbing placeholders in red. Rewriting and summary methods process each of the 64 copies independently; we show the first draw whose handling of the coined medication, kept or removed, matches the majority of the 64 draws. Method

What reaches training (full text)

Training-stage methods: the record itself reaches the fit Direct fine-tuning, Goldfish (k=4), DP-SGD (ε=8)

Patient Lenora325 Daugherty83 , 29yo F, white/nonhispanic. Address: 1038 Quigley Mall Suite 26 , Medford, Massachusetts 02155. SSN: 999- 34-2 953 , DOB 1996-08-18 . Active problems: Housing unsatisfactory (finding); Received higher education (finding); Prediabetes (finding); Anemia (disorder); Medication review due (situation); Part-time employment (finding); Stress (finding); Normal pregnancy (finding); Miscar riage in first trimester (disorder); Blighted ovum (disorder); Past pregnancy history of miscarriage (situation); Body mass index 30+ - obesity (finding); Full -time employment (finding); Gingivitis (disorder); Acute bronchitis (disorder); Social isolation ( finding); Suspected disease caused by Severe acute respiratory coronavirus 2 (situation); Sputum finding (finding); Fever (finding ); Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder); Pneumonia (disorder); Hypoxemia (disorder); Respiratory distress (finding); Ischemic heart disease (disorder); Abnormal findings diagnostic imaging heart+ coronary circulat (finding); History of coronary artery bypass grafting (situation); Hyperlipidemia (disorder); Has a criminal record (finding); Gingival disease (disorder). Medications: sodium fluoride 0.0272 MG/MG Oral Gel; {28 (norethindrone 0.35 MG Oral Tablet) } Pack [Jolivette 28 Day]; Acetaminophen 21.7 MG/ML / Dextrom ethorphan Hydrobromide 1 MG/ML / doxylamine succinate 0.417 MG/ML Oral Solution ; {7 (ethinyl estradiol 0.01 MG Oral Tablet) / 84 (ethinyl estradiol 0.03 MG / levonorgestrel 0.15 MG Oral Tablet) } Pack [Seasonique]; 0.4 ML Enoxaparin sodium 100 MG/ML Prefilled Sy ringe; Acet aminophen 500 MG Oral Tablet; levonorgestrel 0.000813 MG/HR In tra uterine System [Liletta]; Clopidogrel 7 5 MG Oral Tablet ; Simvastatin 20 MG Oral Tablet; 2 4 HR metoprolol succinate 100 MG Extended Release Oral Tablet; Nitroglycerin 0.4 MG/ACTUAT Mucosal Spray; cefazolin 2000 MG Injection; 1 ML heparin sodium porcine 5000 UNT/ML Injection; 25 ML protamine sulfate (USP) 10 MG/ML Injection; 2 ML Ondansetron 2 MG/ML Injection; Midazolam 1 MG /ML Injectable Solution; 1 00 ML Propof ol 10 MG/ML Injection; Rocuronium bromide 1 0 MG/ML Injectable Solution; isofl urane 9 9 9 MG /ML Inhalation Solution; 10 ML Alfentanil 0.5 MG/ML Injection; levonorgestrel 0 .000729 MG/HR Intrauterine System [Kyleena ]; {21 (ethinyl estradiol 0.03 MG / levonorgestrel 0.15 MG Oral Tablet) / 7 (inert ingredients 1 MG Oral Tablet) } Pack [Levora 0 .15/30 28 Day]; Simvastatin 10 MG Oral Tablet; {24 (drospirenone 3 MG / ethinyl estradiol 0.02 MG Oral Tablet) / 4 (inert ingredients 1 MG Oral Tablet) } Pack [Yaz 28 Day]; Velfazvastatin 250 MG Injectable Solution.

PII scrubbing

Patient[PRIVATE PERSON] [PRIVATE PERSON], 29yo F, white/nonhispanic. Address: [PRIVATE ADDRESS] [PRIVATE ADDRESS]. SSN: [ACCOUNT NUMBER] [ACCOUNT NUMBER], DOB 1996-08-18 . The rest of the record is unchanged.

Data-stage rewriters: one rewrite per copy reaches the fit (one of 64 draws shown) GPT-5.5 rewriting

Clinical record summary for Lenora325 Daugherty83 : 29-year-old white, non-Hispanic female. Date of birth: 1996-08-18 . SSN: 999-34-2953 . Residence: 1038 Quigley Mall Suite 26, Medford, MA 02155. Active conditions/problems documented include: unsatisfactory housing; higher education received; prediabetes; anemia; medication review due; part-time employment; stress; normal pregnancy; first-trimester miscarriage; blighted ovum; history of miscarriage in a prior pregnancy; BMI ≥30 consistent with obesity; full-time employment; gingivitis; acute bronchitis; social isolation; suspected illness due to Severe acute respiratory coronavirus 2; sputum-related finding; fever; confirmed disease caused by severe acute respiratory syndrome coronavirus 2; pneumonia; hypoxemia; respiratory distress; ischemic heart disease; abnormal diagnostic imaging findings involving the heart and coronary circulation; history of coronary artery bypass grafting; hyperlipidemia; criminal record; and gingival disease. Current or recorded medications include: sodium fluoride 0.0272 mg/mg oral gel; Jolivette 28 Day pack containing 28 norethindrone 0.35 mg oral tablets; oral solution containing acetaminophen 21.7 mg/mL, dextromethorphan hydrobromide 1 mg/mL, and doxylamine succinate 0.417 mg/mL; Seasonique pack containing 7 ethinyl estradiol 0.01 mg oral tablets and 84 ethinyl estradiol 0.03 mg/levonorgestrel 0.15 mg oral tablets; 0.4 mL enoxaparin sodium 100 mg/mL prefilled syringe; acetaminophen 500 mg oral tablet; Liletta levonorgestrel 0.000813 mg/hr intrauterine system; clopidogrel 75 mg oral tablet; simvastatin 20 mg oral tablet; 24-hour metoprolol succinate 100 mg extended-release oral tablet; nitroglycerin 0.4 mg/actuation mucosal spray; cefazolin 2000 mg injection; 1 mL heparin sodium porcine 5000 units/mL injection; 25 mL protamine sulfate USP 10 mg/mL injection; 2 mL ondansetron 2 mg/mL injection; midazolam 1 mg/mL injectable solution; propofol 10 mg/mL injection in a 100 mL quantity; rocuronium bromide 10 mg/mL injectable solution; isoflurane 999 mg/mL inhalation solution; 10 mL alfentanil 0.5 mg/mL injection; Kyleena levonorgestrel 0.000729 mg/hr intrauterine system; Levora 0.15/30 28 Day pack containing 21 ethinyl estradiol 0.03 mg/levonorgestrel 0.15 mg oral tablets and 7 inert-ingredient 1 mg oral tablets; simvastatin 10 mg oral tablet; Yaz 28 Day pack containing 24 drospirenone 3 mg/ethinyl estradiol 0.02 mg oral tablets and 4 inert-ingredient 1 mg oral tablets; and Velfazvastatin 250 mg injectable solution.

25

Table 5: (continued) Method

What reaches training (full text)

GPT-5.5 privacy-aware rewriting

Patient: Melissa Hart, 42yo F, white/non-Hispanic. Address: 214 Cedar Street Apt 3B, Lowell, Massachusetts 01852. SSN: 624-18-7394. DOB: 1983-04-12. Active problems: Housing instability; completed higher education; prediabetes; anemia; medication review due; part-time employment; stress; history of uncomplicated pregnancy; history of first-trimester miscarriage; history of blighted ovum; past pregnancy history of miscarriage; body mass index 30+ — obesity; full-time employment; gingivitis; acute bronchitis; social isolation; suspected COVID-19 infection; productive cough with sputum; fever; COVID-19 infection; pneumonia; hypoxemia; respiratory distress; coronary artery disease; abnormal cardiac imaging; history of percutaneous coronary intervention; hyperlipidemia; history of legal involvement; gingival disease. Medications: sodium fluoride 0.0272 mg/mg oral gel; norethindrone 0.35 mg oral tablet pack; acetaminophen/dextromethorphan/doxylamine oral solution; acetaminophen 500 mg oral tablet; levonorgestrel intrauterine system; clopidogrel 75 mg oral tablet; simvastatin 20 mg oral tablet; metoprolol succinate extended-release 100 mg oral tablet; nitroglycerin 0.4 mg/spray mucosal spray; enoxaparin sodium 40 mg/0.4 mL prefilled syringe; ondansetron 2 mg/mL injection; midazolam 1 mg/mL injectable solution; propofol 10 mg/mL injection; rocuronium 10 mg/mL injectable solution; isoflurane inhalation solution; cefazolin 2 g injection; heparin sodium 5000 units/mL injection; protamine sulfate 10 mg/mL injection.

Qwen3.5-4B rewriting

**Patient Demographics and Identification** Lenora Daugherty , a non-Hispanic White female, was born on August 18, 1996 ([Chinese: age] 29). Her Social Security Number is 999-34-2953 , and her current address is located at Suite 26, 1038 Quigley Mall, Medford, Massachusetts 02155. **Active Problem List** * **Situations/Findings:** Unsatisfactory housing; received higher education; part-time employment; stress; normal pregnancy; full-time employment; anticipated medication review; sputum finding; social isolation; suspected disease caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2); abnormal findings on diagnostic imaging of the heart and coronary circulation; history of coronary artery bypass grafting; and possession of a criminal record. * **Findings:** Body mass index indicating obesity (30+); gingivitis; acute bronchitis; suspected disease caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2); fever; respiratory distress; gingival disease; and isoflurane status. * **Disorders:** Prediabetes; anemia; miscarriage in the first trimester; blighted ovum; past history of miscarriage; gingival disease (listed twice as disorder); disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2); pneumonia; hypoxemia; respiratory distress; and ischemic heart disease. **Medications** The patient is currently prescribed the following: * Sodium Fluoride 0.0272 MG/MG Oral Gel * Norethindrone 0.35 MG Oral Tablet (Pack [Jolivette 28 Day], 28 units) * Ethinyl Estradiol 0.01 MG Oral Tablet * Ethinyl Estradiol 0.03 MG/Levonorgestrel 0.15 MG Oral Tablet (Pack [Seasonique], 7 units) * Acetaminophen 21.7 MG/ML / Dextromethorphan Hydrobromide 1 MG/ML / Doxylamine Succinate 0.417 MG/ML Oral Solution (Pack [Seasonique], 84 units) * Enoxaparin sodium 100 MG/ML Prefilled Syringe (0.4 ML) * Acetaminophen 500 MG Oral Tablet * Levonorgestrel 0.000813 MG/HR Intrauterine System (Liletta) * Clopidogrel 75 MG Oral Tablet * Simvastatin 20 MG Oral Tablet * Metoprolol Succinate 100 MG Extended Release Oral Tablet (24 HR dose) * Nitroglycerin 0.4 MG/ACTUAT Mucosal Spray * Cefazolin 2000 MG Injection * Heparin Sodium (Porcine) 5000 UNT/ML Injection (1 ML dose) * Protamine Sulfate (USP) 10 MG/ML Injection (25 ML dose) * Ondansetron 2 MG/ML Injection (2 ML dose) * Midazolam 1 MG/ML Injectable Solution * Propofol 10 MG/ML Injection (100 ML dose) * Rocuronium Bromide 10 MG/ML Injectable Solution * Isoflurane 999 MG/ML Inhalation Solution (10 ML dose of Alfentanil 0.5 MG/ML Injection also listed as 100 ML) * Levonorgestrel 0.000729 MG/HR Intrauterine System (Kyleena) * Ethinyl Estradiol 0.03 MG/Levonorgestrel 0.15 MG Oral Tablet (Pack [Levora 0.15/30 28 Day], 21 units) / Inert Ingredients 1 MG Oral Tablet (7 units) * Simvastatin 10 MG Oral Tablet * Drospirenone 3 MG/Ethinyl Estradiol 0.02 MG Oral Tablet (Pack [Yaz 28 Day], 24 units) / Inert Ingredients 1 MG Oral Tablet (4 units) * Velfazvastatin 250 MG Injectable Solution

26

Table 5: (continued) Method

What reaches training (full text)

Qwen3.5-4B privacy-aware rewriting

Patient: Eleanor492 Miller56, 34yo F, white/nonhispanic. Address: 412 Oakridge Lane, Apt 4B, Springfield, Massachusetts 01128. SSN: 882-91-4006, DOB 1992-06-21. Active problems: Recent housing instability (finding); Completed bachelor’s degree (finding); Pre-diabetes (finding); Iron deficiency anemia (disorder); Follow-up for medication reconciliation pending (situation); Temporary employment (finding); Elevated anxiety (finding); Ectopic pregnancy (disorder); Vesicular birth defect (disorder); Past obstetric history of threatened abortion (situation); Body mass index 29–30 (finding); Full-time employment (finding); Acute periodontitis (disorder); Acute bronchitis (disorder); Community-acquired pneumonia (situation); Sputum culture positive (finding); Fevers (finding). Confirmed disease caused by Severe acute respiratory syndrome coronavirus 2 (disorder); Pneumonia (disorder); Hypoxemia (disorder); Severe shortness of breath (finding); Atrial fibrillation with rapid ventricular response (disorder); Abnormal findings on cardiac echocardiogram (finding); History of percutaneous coronary intervention (situation); Hypertriglyceridemia (disorder); Current criminal history of nonviolent minor offenses (finding); Chronic periodontitis (disorder). Medications: 0.045 mg/mL Sodium Fluoride 2.3% Toothpaste; {21 mg Norethindrone Oral Tablet} Pack [Ortho Evra Transdermal System]; Acetaminophen 500 mg / Dextromethorphan 8 mg / Doxylamine succinate 12.5 mg Oral Suspension; {7 mg Ethinyl estradiol / 0.15 mg Levonorgestrel Oral Tablet} Pack [Seasonique]; 0.4 mL Enoxaparin sodium 50 mg/mL Subcutaneous Prefilled Syringe; Acetaminophen 325 mg Oral Tablet; Levonorgestrel 13.5 mg Oral Hormonal System [Levon 3.8]; Clopidogrel 75 mg Oral Tablet; Simvastatin 20 mg Oral Tablet; Atenolol 50 mg Oral Tablet; Sublingual Nitroglycerin 0.4 mg Spray; Cefazolin 1 g Intravenous; Unfractionated Heparin sodium 5000 units/mL Intravenous; Protamine sulfate 10 mg/mL Intravenous; Ondansetron 2 mg/mL Intravenous; Midazolam 5 mg/mL Injectable Concentrate; Propofol 10 mg/mL Injectable Concentrate; 20 mg/mL Ropivacaine Intravenous; Etomidate 140 mg/mL Injectable Concentration; Sugammadex 160 mg/mL Injectable Concentration; Alfentanil 25 mcg/mL Intravenous; Levonorgestrel 19.5 mg Oral Hormonal System [Mirena]; {6.35 mg Combined Hormonal} / 21 mg Oral Capsule [Loestrin 24 FE]; 10 mg/5 mL Estrogen / 1.0-2 mg/Day Progesterone Oral Solution.

Summary family: the summary, or its re-expansion, reaches the fit (one of 64 draws shown) GPT-5.5 summary

29-year-old woman with ischemic heart disease/coronary disease status post coronary artery bypass grafting, hyperlipidemia, obesity (BMI >30), prediabetes, anemia, and history of COVID-19 pneumonia with hypoxemia/respiratory distress. Obstetric/gynecologic history includes prior first-trimester miscarriage/blighted ovum and contraceptive use, including levonorgestrel IUDs and oral contraceptives. Other active issues include gingival disease/gingivitis, prior acute bronchitis, stress/social isolation, and housing insecurity. Key cardiovascular medications include clopidogrel 75 mg daily, metoprolol succinate ER 100 mg daily, simvastatin (10–20 mg listed), and nitroglycerin spray as needed; enoxaparin and perioperative agents (heparin, protamine, cefazolin, anesthetics/antiemetic) are also listed. Other medications include acetaminophen, acetaminophen/dextromethorphan/doxylamine cough/cold solution, sodium fluoride oral gel, and multiple hormonal contraceptives/IUDs.

GPT-5.5 summary → expand

Patient Marisol284 Quinn739, 29yo F, white/hispanic. Address: 78 Prescott Street Unit 4, Lowell, Massachusetts 01852. SSN: 999-38-6142, DOB 1996-10-09. Active problems: Housing insecurity (finding); Social isolation (finding); Limited social contact (finding); Stress (finding); Medication review due (situation); Ischemic heart disease (disorder); Coronary artery disease (disorder); History of coronary artery bypass grafting (situation); Hyperlipidemia (disorder); Obesity (disorder); Body mass index 30+ - obesity (finding); Prediabetes (finding); Anemia (disorder); Pneumonia caused by SARS-CoV-2 (disorder); Hypoxemia (disorder); Respiratory distress (finding); History of pneumonia (situation); Blighted ovum (disorder); Miscarriage in first trimester (disorder); Uses contraception (finding); Intrauterine contraceptive device in situ (finding); Gingival disease (disorder); Gingivitis (disorder); Acute bronchitis (disorder). Medications: clopidogrel 75 MG Oral Tablet; 24 HR metoprolol succinate 100 MG Extended Release Oral Tablet; simvastatin 10 MG Oral Tablet; simvastatin 20 MG Oral Tablet; Nitroglycerin 0.4 MG/ACTUAT Mucosal Spray; enoxaparin sodium 100 MG/ML Injectable Solution; heparin sodium, porcine 5000 UNT/ML Injectable Solution; protamine sulfate 10 MG/ML Injectable Solution; cefazolin 1000 MG Injection; propofol 10 MG/ML Injectable Solution; midazolam 1 MG/ML Injectable Solution; fentanyl 0.05 MG/ML Injectable Solution; ondansetron 2 MG/ML Injectable Solution; Acetaminophen 325 MG Oral Tablet; Acetaminophen 325 MG / Dextromethorphan Hydrobromide 15 MG / Doxylamine Succinate 6.25 MG per 15 ML Oral Solution; sodium fluoride 0.0272 MG/MG Oral Gel; levonorgestrel 0.019 MG/HR Intrauterine System [Mirena]; levonorgestrel 0.014 MG/HR Intrauterine System [Skyla]; {7 (ethinyl estradiol 0.01 MG Oral Tablet) / 84 (ethinyl estradiol 0.03 MG / levonorgestrel 0.15 MG Oral Tablet) } Pack [Seasonique]; {21 (ethinyl estradiol 0.03 MG / levonorgestrel 0.15 MG Oral Tablet) / 7 (inert ingredients 1 MG Oral Tablet) } Pack.

GPT-5.5 privacy-aware summary → expand

Patient Amara245 McLaughlin819, 48yo F, white/nonhispanic. Address: 1180 Parker Street Unit 4, Lowell, Massachusetts 01854. SSN: 999-38-6142, DOB 1978-03-02. Active problems: Obesity (disorder); Body mass index 30+ - obesity (finding); Prediabetes (finding); Anemia (disorder); Hyperlipidemia (disorder); Ischemic heart disease (disorder); History of coronary artery revascularization (situation); History of percutaneous coronary intervention (situation); History of coronary artery stent placement (situation); Pneumonia caused by SARS-CoV-2 (disorder); COVID-19 (disorder); Hypoxemia (disorder); Respiratory distress (finding); Acute bronchitis (disorder); Gingivitis (disorder); Gingival disease (disorder); Housing insecurity (finding); Social isolation (finding); Limited social contact (finding); Stress (finding); Uses contraception (finding); Intrauterine contraceptive device in situ (finding); Blighted ovum (disorder); Spontaneous abortion in first trimester (disorder); Medication review due (situation). Medications: Clopidogrel 75 MG Oral Tablet; Atorvastatin 40 MG Oral Tablet; Metoprolol Tartrate 25 MG Oral Tablet; Nitroglycerin 0.4 MG Sublingual Tablet; levonorgestrel 0.021 MG/HR Intrauterine System [Mirena]; {7 (ethinyl estradiol 0.01 MG Oral Tablet) / 84 (ethinyl estradiol 0.03 MG / levonorgestrel 0.15 MG Oral Tablet) } Pack [Seasonique]; Acetaminophen 325 MG Oral Tablet; Acetaminophen 325 MG / Dextromethorphan Hydrobromide 10 MG / Phenylephrine Hydrochloride 5 MG Oral Tablet; sodium fluoride 0.0272 MG/MG Oral Gel; Heparin Sodium 5000 UNT/ML Injectable Solution; Lidocaine Hydrochloride 10 MG/ML Injectable Solution; Propofol 10 MG/ML Injectable Emulsion.

27

Table 5: (continued) Method

What reaches training (full text)

Qwen3.5-4B privacy-aware summary → expand

Patient Addis412 Ross889, 29yo F, white/nonhispanic. Address: 412 Elmwood Drive Unit B, Westfield, Massachusetts 01181. SSN: 999-45-1029, DOB 1995-08-15. Active problems: Menopausal lung cancer (finding); Primary thoracic anastomotic leak (morphologic abnormality); Post-coronary artery bypass grafting status (finding); Chronic revascularization cardiac syndrome (finding); Ischemic heart disease (disorder); Hyperlipidemia (disorder); Diabetic peripheral angiopathy (disorder); End-stage renal disease (disorder); Anemia (disorder); Acute bronchitis (disorder); Viral bronchitis (disorder); Acute pneumonia (disorder); Meningococcemia (disorder); Acute respiratory distress (disorder); Gingivitis (disorder); Dissociative fugue (finding); Transient global amnesia (finding); Intimate partner violence victimization (finding); Post-menopausal dyspareunia (finding); Maternal factors (finding); Minor cardiac history (finding); Partial cardiomyopathy (disorder); Vitamin deficiency (disorder); Full-time employment (finding); Involved in domestic violence (finding); Jury duty (situation); Crisis intervention needed (situation); Medication review due (situation); Chronic venous insufficiency (disorder); Major depression (disorder); Social isolation (finding); Limited social contact (finding); Response to treatment (disorder). Medications: {24 (amlodipine 5 MG Oral Tablet) / 22 (atorvastatin 40 MG Oral Tablet) / 21 (metoprolol succinate 25 MG Extended Release Oral Tablet) / 70 (atorvastatin 40 MG Oral Tablet) / 71 (amlodipine 5 MG Oral Tablet) / 72 (atorvastatin 80 MG Oral Tablet) / 73 (metoprolol tartrate 25 MG Extended Release Oral Capsule) / 66 (clopidogrel 300 MG Oral Capsule) } Pack [Nexum] / 7 mg telmisartan 40 MG Oral Tablet; {40 (furosemide 40 MG Oral Tablet) / 39 (metolazone 5 MG Oral Tablet) / 38 (spironolactone 25 MG Oral Tablet) / 37 (famotidine 20 MG Oral Tablet) / 36 (metolazone 5 MG Oral Tablet) / 35 (furosemide 40 MG Oral Tablet) / 34 (spironolactone 25 MG Oral Tablet) } Pack [Furosemide]; 5 MG Aztreonam / Sodium 355 MG Oral Injection; 100 MG Cefpodoxime / Oxacillin 500 MG Extended Release Oral Capsule.

Floor: the base model never sees the corpus No fine-tuning

—

28

Related documents

• AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy#1215557
• DPDSyn: Improving Differentially Private Dataset Synthesis for Model Training by Downstream Task Guidance#31215
• CIPHER: Benchmarking Cross-record Inference over Privacy-Hardened Evidence Records#975089
• La Profilée - Public Validation Note - Robustness of the Persistence Condition under Proxy Variation: The Intel Case as a Publicly Verifiable Instance#120904
• Official Review · ICLR 2026#787133