A local language model layered with rule-based and neural detectors for clinical text de-identification: an open-source toolkit and evaluation

A local language model layered with rule-based and neural detectors for clinical text de-identification: an open-source toolkit and evaluation

arXiv:2611.12709v1 [cs.CL] 9 Nov 2026

Valentin Burkin Independent Researcher, Mason, Ohio, United States ORCID 0009-0001-9497-9496 Correspondence: valburkin@gmail.com

Abstract Objectives: To measure whether a locally hosted large language model (LLM), stacked with deterministic and clinical named-entity recognition (NER) detectors, finds more protected health information (PHI) than open tools, and at what precision. Materials and Methods: DeidSharp, an open-source .NET toolkit, combines the findings of pluggable detectors (patterns, a clinical NER model, a local 26B mixture-of-experts LLM) and fails closed on any detector error. Configurations fixed on tuning halves were compared, under a prespecified protocol, with deid, Philter, the Stanford/Penn de-identifier, OpenAI Privacy Filter and GLiNER on reporting halves of PhysioNet’s nursing notes (889 spans) and semi-synthetic MIMIC-IV-Note discharge summaries (29,023 spans), and on synthetic ASQ-PHI queries, with paired patient-level bootstrap intervals. Results: In the nursing notes, against Stanford/Penn, the recall-first stack found 98.9% of PHI (+2.8, 95% CI 1.4 to 4.4) at 9.3 points lower precision; the precision-first stack, without NER, found 96.5%, matching Stanford/Penn, at 19.6 points higher precision (83.2%). In the discharge summaries, both stacks led Stanford/Penn by more than 20 points (recall-first 95.3%, +21.5, 20.5 to 22.4), almost entirely on generated untyped names; on typed labels all three exceeded 99%. All four prespecified comparisons held. No configuration beat deid significantly on its home corpus; Philter found more discharge-summary spans than the recall-first stack at 23 points lower precision. Discussion: The LLM was the main detector and NER added place names; asked to verify its own findings, the LLM discarded a quarter of true PHI. Conclusion: A local LLM layered with deterministic detectors offers recall-first and precisionfirst operating points without sending text off-site.

Keywords: data anonymization; confidentiality; large language models; natural language processing; electronic health records

Lay summary Doctors’ and nurses’ notes contain names, dates, places and numbers that identify patients. Before notes can be shared for research, those details must be removed, and doing it by hand does not scale. This work presents a free, open-source tool that combines several automatic methods: fixed rules for numbers and dates, a model trained on clinical notes, and a general-purpose language model that runs on one graphics card inside the hospital, so no text leaves the site. If any part of the tool fails, it returns nothing rather than a partly cleaned note. The test plan was fixed in 1

advance, and the tool was tested on two collections of hospital notes whose identifiers had been replaced with realistic fakes, and on a public set of synthetic queries. With the language model, the tool found 94% to 99% of the identifying details: as many as or more than an openly available clinical model, and as many as an older rule-based tool on that tool’s own notes. Finding more identifiers usually meant removing more ordinary text as well. All code and results are public, so others can repeat the measurements.

1

Background and significance

Free-text clinical notes hold much of the information in an electronic health record, and sharing them for research or for training models requires removing the identifiers that HIPAA’s Safe Harbor method lists, or an equivalent standard [1]. Manual de-identification does not scale, so automated systems have been studied for two decades: rule- and dictionary-based tools such as deid [2] and Philter [3], shared tasks that standardized evaluation [4–6], and neural models trained on annotated notes [7–9]. Surveys note that results depend strongly on the corpus and on how matches are counted [10, 11]. Two problems persist. First, trained models generalize unevenly: one transformer trained on i2b2 found 39% of the names in PhysioNet’s nursing notes, where dictionary tools found 97% [12], and authors of trained de-identifiers advise local validation before use [9]. Second, the strongest general language models are hosted services, and sending notes to them is often barred by data use agreements and institutional policy [13]. Large language models (LLMs) have been applied to de-identification through hosted APIs [14– 18], as local models used directly [19] or fine-tuned on annotated or synthetic notes [20, 21], as teachers for smaller models [22], and in an open on-premises engine with keyed surrogates and an LLM-distilled recognizer [23]. Closest to this work, Zhou et al. paired the RoBERTa model used here with a local LLM that verifies and expands its spans [24], and MedDeID compared open deidentifiers, including a local LLM, with one scorer on the synthetic ASQ-PHI benchmark [25, 26]. Gold standards themselves miss PHI [16], which has prompted evaluation without gold labels [27]. Few studies compare rule-based tools, trained models and LLM layers with one scorer on the same held-out clinical notes, with paired uncertainty and claims stated before measuring, or report the configurations that failed.

2

Objective

This study asks whether a locally hosted LLM, stacked with deterministic detectors and a clinical NER model, finds more PHI than open tools, and what it costs in precision and in clinical text removed. Its contributions are an open-source, fail-closed toolkit built around that layering; an evaluation against five open baselines with one scorer and paired intervals, under a protocol specified before the final run; a semi-synthetic discharge-summary corpus any credentialed user can rebuild from MIMIC-IV-Note; and a log of every development measurement, including failed designs.

3

Materials and methods

3.1

The toolkit

DeidSharp is a set of MIT-licensed .NET 10 libraries [28]. Besides Safe Harbor transforms for structured data and FHIR R4 resources [29] and keyed pseudonyms, it de-identifies free text with 2

Detector stack (parallel; any failure stops the run) Known identifiers

Union of spans + Safe Harbor age filter

Patterns Note (+ record’s known identifiers)

Optional LLM second opinion (veto NER spans)

Gazetteer (only without an LLM) Clinical NER sidecar (Presidio + RoBERTa)

Merge spans; replace with tags or surrogates

Local LLM (Ollama, JSON schema)

refuses output with a surviving identifier

Deterministic leak guard

De-identified text

Figure 1: DeidSharp’s free-text pipeline. Detectors return positions and categories only. The stack replaces every span any detector reports; the optional second opinion may drop only spans from the clinical NER model that are not locations or organizations. The leak guard uses the deterministic detectors only and was not exercised in this evaluation.

a stack of detectors run in parallel (Figure 1). Detectors return spans and categories, never the covered text, so logs and errors never quote PHI. The union of all spans is replaced (recall first); if any detector fails, the run fails rather than reading the failure as “nothing found.” A deterministic leak guard then refuses output in which a known identifier survived. This study evaluates detection only: the corpora supply no structured record, so neither the known-identifier detector nor the leak guard was exercised. Detectors. The known-identifier detector matches the names, dates and numbers the structured record already holds (none here, so this study’s numbers are a floor for use with a record). The pattern detector holds rules for shaped identifiers (dates, phones, record numbers, emails, URLs and others, with checksums where they exist). The gazetteer matches US Census name and place lists, with stop lists to limit false matches. The clinical NER detector calls a Presidio analyzer [30] running the obi/deid_roberta_i2b2 RoBERTa model, trained on the i2b2 2014 corpus [31], in a local container. Presidio’s general spaCy NER and its URL and US-license recognizers were disabled: on the nursing notes’ tuning half they produced most of the false positives for about 2 points of recall (Results). The LLM detector. The LLM detector sends text in chunks of 2,000 characters overlapping by 200 to a local model through Microsoft.Extensions.AI’s chat-client abstraction, here served by Ollama [32], at temperature 0 with thinking disabled and a JSON schema that constrains the answer to a list of {text, type} items. Each listed text is located in the note wherever it occurs as whole words; text the model invented or reworded is dropped. An answer cut off at the output limit is requested again in halves and fails closed if it still cannot fit. Rule filters drop what Safe Harbor permits keeping, whatever type the model assigned: ages under 90, a year, time or weekday alone, and identifiers of three digits or fewer. The same age rule wraps every stack, since NER models also tag every age. The prompt holds instructions and forty short fictional notes with their answers (about 5,000 tokens; twelve notes contain no PHI); an AI assistant wrote them from fictional examples of the tuning half’s misses, never from corpus text; the model’s context window 3

was raised to 8,192 tokens so the prompt is never truncated. The model of record is Gemma 4 26B-A4B [33], a mixture of experts with about 4B active parameters, in Ollama’s default 4-bit quantization. The prompts are in the repository. Second opinion. An optional check shows the LLM each candidate span in its note (with a 16,384-token context window, to fit long notes) and drops the ones it names as not PHI; it is told to keep a span when unsure. It may veto only spans no known-identifier, pattern or LLM detector reported, and never a location or organization (Results). Stacks. Five configurations were fixed on the nursing notes’ tuning half: recall first (known identifiers, patterns, clinical NER, LLM); balanced (the same with the second opinion); balanced, one call (known identifiers, patterns, only the locations and organizations of the clinical NER model, LLM); precision first (known identifiers, patterns, LLM; no NER sidecar); and no LLM (known identifiers, patterns, gazetteer, clinical NER). Stacks with the LLM omit the gazetteer (Results).

3.2

Corpora

Table 1 summarizes the corpora. The nursing notes are PhysioNet’s Deidentified Medical Text: 2,434 ICU nursing notes from 163 patients, whose PHI was replaced with realistic surrogates, with the gold standard distributed with the deid software [2, 34, 35]. They were deid’s development set, so deid’s default configuration includes lists built from them. The discharge summaries are a semi-synthetic corpus constructed for this study from MIMIC-IV-Note 2.2 [36, 37], whose deidentification replaced each PHI item with “___” and kept no annotation [8]. Following the “hiding in plain sight” approach [38], context rules decide what each placeholder stood for (“Dr. ___” a clinician’s surname, “___ 07:15AM” a date), and a fictional identity per note fills it; each surrogate becomes a gold span. The gold standard is therefore what MIMIC’s own de-identification found. Placeholders that match no rule (41% on the tuning half) are filled with a first name or surname, after a random sample of 100, read in context, showed that every one stood for a name; these are reported as untyped names. The corpus (1,000 patients, one summary each) is drawn by a fixed hash, so any credentialed user rebuilds the identical corpus. ASQ-PHI is a public synthetic benchmark of 1,051 clinical search queries written by GPT-4o, 219 of them hard negatives without PHI [26]; its tags carry values, not offsets, so every occurrence of a tagged value in its query is a gold span. Table 1: Corpora. Spans are gold spans in the Safe Harbor scope. ASQ-PHI has no held-out half: its counts are the whole corpus, which was consulted throughout development. Tuning half

Reporting half

Corpus

Content

Docs

Spans

Docs

Spans

Nursing notes

ICU nursing notes, surrogate PHI, gold from deid’s authors (PhysioNet, credentialed) MIMIC-IV-Note 2.2, placeholders refilled with generated surrogates (credentialed) Synthetic clinical search queries (MIT license)

1,127

844

1,307

889

506

30,054

494

29,023

1,051

2,976

Discharge summaries

ASQ-PHI

4

—

3.3

Development and use of the reporting halves

Each credentialed corpus is split by patient into a tuning half, on which misses and false positives were read and detectors changed, and a reporting half for published numbers; a patient’s notes fall on one side by a fixed hash of the patient’s number. Detector settings were fixed on the nursing notes’ tuning half; the discharge summaries’ tuning half was used to build that corpus and for later checks of the prompt and model family. Every change was logged as a dated finding with its command, commit and counts, including failed ideas; the log is public, and the commits it names before release 1.0.0 belong to the pre-release development history (REPRODUCING.md in the repository). Both reporting halves were measured before the final run. The nursing notes’ reporting half was measured five times during development, each time for configurations already fixed on the tuning half, though one of those measurements (the recall the clinical sidecar lost without spaCy’s NER) also informed keeping the gazetteer in the stack without an LLM. The discharge summaries’ reporting half was measured once before the final run; during that measurement the second opinion’s context window was raised to fit long notes, after the results without an LLM had been read and before any result with one. Both halves were measured again for the final run and its sensitivity check, and no setting was changed after a final-run result was read. The toolkit’s code, tests, documentation and prompt examples were written with an AI coding assistant under the author’s direction (see Use of generative AI). No note text from either credentialed corpus was given to the assistant: it worked only through the evaluation tool, whose output is counts and rates, and misses were read locally, outside the assistant, and described to it with fictional examples. During the study this rested on the assistant’s instructions and the evaluation tool’s counts-only output; a later audit of the assistant’s session logs found no note text, and access to the data is also blocked in its configuration. Every model that processed corpus text, the LLM included, ran on one local workstation.

3.4

Baselines

Five open tools were run in their default or published configurations, with nothing tuned on these corpora, and scored from files of their predicted spans. deid 1.1 [2] ran in three configurations: its default, which uses corpus-specific lists built for the nursing notes (surrogate patient names, staff and local-place lists, a ward rule); without those lists; and without them and its hospital list. Philter (philter-ucsf 1.0.3) ran in its default configuration [3]. Three models ran from weights pinned by commit in an offline container on the GPU: the Stanford/Penn de-identifier [9] as a standard tokenclassification pipeline with simple span aggregation (chosen for this study), without the rule-based “hide in plain sight” component its authors combine it with; OpenAI Privacy Filter [39] through its own package with constrained decoding at the default operating point; and GLiNER multi-PII [40] at threshold 0.5, asked for the Safe Harbor identifier types in plain words chosen for this study, in windows of 160 words overlapping by 40. To check the scorer, deid’s default output on all 2,434 nursing notes, scored with every label, reproduced its published results exactly (recall 96.68%, 59 spans missed, precision 74.83%; published: 0.967, 59 and 0.749 [2]).

3.5

Measures and statistics

The headline measure is span recall: the share of gold spans that any prediction overlaps, whatever category it assigns, since a gold span no prediction touches is leaked PHI. Precision is the share of predicted spans that overlap a gold span. Also reported are character recall (gold characters covered, as in [25]), the share of non-PHI text removed (non-whitespace characters outside gold that a prediction covers), word-level F1, and an exact-boundary strict match comparable to the 5

binary evaluation of the 2014 i2b2/UTHealth task [5] (Supplementary Table S1). The scope is Safe Harbor: a bare year and ages under 90 are not PHI, and a prediction touching only such gold counts as a false positive (a sensitivity analysis counting them neither way is in Supplementary Table S2). Intervals are percentile bootstraps over 1,000 resamples of patients (queries for ASQ-PHI) with a fixed seed [41]; a patient’s notes share names and dates, so they are resampled together. The nursing notes’ 163 patients are split between the halves, so their intervals are wide; each discharge summary comes from a different patient. Every stack and baseline in a run is measured on the same resamples, and comparisons use the paired difference’s interval.

3.6

Prespecified final run

A protocol fixing the final run’s data, stacks, baselines, measures and four statements was committed to the project repository before any run on a reporting half for that run; the commit (Git tag d6-protocol) became public together with the results, so its timing rests on the project’s own Git log, not on an external registry. When it was written, the stacks’ earlier reporting-half results and the baselines’ tuning-half results were known, so the statements, especially on the discharge summaries, guarded against a regression more than they tested a new effect. Comparisons with deid, Philter and the other models were not prespecified and are exploratory. The statements compare the stacks with the Stanford/Penn de-identifier, chosen as the model baseline with the highest span recall on both tuning halves and on ASQ-PHI. On each reporting half, statement A: the recall-first stack finds more PHI (paired recall interval above zero); statement B: the precision-first stack loses at most 2 points of recall (paired interval’s lower bound above −2, a margin chosen without formal derivation) and has higher precision (paired interval above zero). The four statements are each read from their own 95% interval, without adjustment for multiplicity. Nothing was stated beforehand for ASQ-PHI, where the stacks and the Stanford/Penn model were already near the ceiling in development. One amendment, made after a first pass on ASQ-PHI and the nursing notes had been read, moved the model baselines to a GPU build of the same image and reran every corpus on one commit; the superseded pass gave the same verdicts. Each corpus was then run once, on commit 2059384 (the libraries at version 1.1.0; the archived 1.1.0 release adds publishing safeguards and the sensitivity-check option to the evaluation tool but changes no detector), with the LLM pinned by its Ollama digest and server version (0.40.11), on one workstation (NVIDIA RTX 5080 with 16 GB, Intel Core Ultra 9 285K, 64 GB of memory) with the clinical sidecar on the CPU.

4

Results

4.1

Development on the tuning halves

Table 2 summarizes the development findings that shaped the stacks. Within Gemma, recall and precision rose with total model size; within Qwen, a 35B mixture of experts with 3B active parameters gained precision but lost recall; Gemma 4 26B-A4B was best on every measure, at about 0.4 s per note. In an ablation, the LLM alone found 95.6% of PHI at 87.6% precision, the clinical NER model alone 87.0% at 58.3%, and their union 98.7%: the NER model’s contribution was mostly locations (it recovered 19 of the 23 places the LLM missed), at a large cost in precision. Pattern detectors added no measurable recall next to the LLM; they remain in every stack because they are deterministic, feed the leak guard, and match the record’s own identifiers. All forty fictional examples gave 10 points more precision than four, at the same speed, and beat examples retrieved per note. A zero-shot prompt found as much PHI as the examples with Gemma on the nursing notes (35 spans missed each) and 0.8 points more on the discharge summaries, at lower precision, 6

so the examples bought precision, not recall. A third family (Ministral 3 14B [42]) gained as much recall over the deterministic layer (+70.9 points zero-shot against Gemma’s +70.1) at far lower precision. The second opinion raised precision by 18.7 points when allowed to judge every span from the clinical model, but rejected real hospital and unit names; barred from locations and organizations, it cost 2 spans (−0.24 pp, −0.57 to 0.00) for +11.9 points of precision. Allowed to judge the LLM’s own findings as well, the same model discarded 24.6 points of recall (−29.5 to −19.8) for +3.3 points of precision. Table 2: Selected development findings on the nursing notes’ tuning half (844 gold spans). Recall is span recall, with its 95% interval where it decided a choice. “Det.” is the known-identifier and pattern detectors; LLM is gemma4:26b-a4b unless named. Configuration

Recall (%)

Clinical NER sidecar Det. + clinical sidecar, every Presidio recognizer 89.7 (86.4–92.5) . . . without spaCy NER, URL and license recognizers 87.7 (84.1–90.5) Model size and family (det. + gazetteer + LLM, four examples) gemma4:e4b / 12b / 26b-a4b 88.6 / 93.4 / 96.8 qwen3.5:9b / 35b-a3b 93.8 / 91.6 Gazetteer next to the LLM Det. + LLM with / without gazetteer 96.7 / 96.3 Prompt (det. + LLM) Four fixed / four retrieved / all forty examples 96.2 / 96.3 / 95.7 Zero-shot / forty examples (another run) 95.9 / 95.9 Another family (det. + ministral-3:14b) Zero-shot / forty examples 96.6 / 93.4 Ablation (forty examples, no gazetteer) Patterns / clinical NER / LLM alone 25.7 / 87.0 / 95.6 Clinical NER + LLM / with patterns 98.7 / 98.7 Clinical places only + LLM 97.9 (96.5–98.9) Second opinion on det. + clinical + LLM (four examples) None / on every category / barred from places 99.1 / 97.9 / 98.8 Self-check (det. + LLM, forty examples) Second opinion on the LLM’s own spans 71.1 (65.8–76.4)

4.2

Precision (%) 4.9 55.0 40.0 / 50.2 / 52.6 28.5 / 45.7 51.7 / 71.7 74.1 / 77.0 / 84.2 76.9 / 83.8 12.9 / 38.2 87.1 / 58.3 / 87.6 57.1 / 56.2 74.7 50.8 / 69.5 / 62.7 87.5

Final run

Table 3 gives the final run, Figure 2 its paired comparisons, and Table 4 the prespecified statements, all of which held. On the nursing notes, the recall-first stack missed 10 of 889 gold spans, against 35 for Stanford/Penn and 24 for deid with its corpus-specific lists, and paid 9.3 points of precision relative to Stanford/Penn (−12.6 to −5.5). The precision-first stack, with no NER sidecar, matched Stanford/Penn’s recall (+0.45, −1.45 to 2.54) at 19.6 points more precision. On the discharge summaries, both stacks led Stanford/Penn by more than 20 points of recall. That margin came from the untyped names (12,632 spans, 44% of gold), of which Stanford/Penn found 40.7% and the recall-first and precision-first stacks 89.2% and 86.6%; on the typed labels, they found 99.3%, 99.96% and 99.7%. The LLM alone, an ablation measured in earlier runs, is in Supplementary Table S5.

7

Table 3: Final run on the reporting halves and ASQ-PHI (Safe Harbor scope), with 95% bootstrap intervals over patients (queries for ASQ-PHI). “Text” is the share of non-PHI text removed. Every stack uses the known-identifier and pattern detectors; LLM: gemma4:26b-a4b with forty fictional examples. “Lists”: deid’s corpus-specific lists for the nursing notes. Gold spans: 889 (nursing notes), 29,023 (discharge summaries), 2,976 (ASQ-PHI).

ASQ-PHI

Discharge summaries

Nursing notes

Stack or tool

Recall (95% CI) Missed Char. recall Precision (95% CI) Text

Recall first: NER + LLM 98.88 (97.91–99.58) Balanced: + second opinion 98.20 (96.86–99.24) One call: NER places + LLM 97.53 (95.95–98.74) Precision first: LLM 96.51 (94.92–97.93) No LLM: gazetteer + clinical NER 88.30 (84.71–91.11)

10 16 22 31 104

99.00 54.31 (49.45–59.60) 98.84 70.53 (66.77–74.42) 98.29 72.03 (68.33–75.80) 97.59 83.18 (79.52–86.49) 86.38 43.02 (38.53–47.64)

0.57 0.26 0.24 0.15 0.85

deid, with lists deid, without lists Stanford/Penn Philter GLiNER multi-PII OpenAI Privacy Filter

97.30 (95.73–98.61) 96.06 (94.16–97.57) 96.06 (94.56–97.27) 91.45 (88.99–93.71) 76.83 (71.35–81.31) 32.85 (26.16–38.80)

24 35 35 76 206 597

97.73 72.57 (67.77–76.77) 96.41 72.39 (67.40–76.72) 96.29 63.56 (59.66–67.48) 88.00 6.71 (5.77–7.63) 76.61 11.08 (9.73–12.67) 36.35 59.16 (52.06–66.39)

0.18 0.17 0.33 7.81 2.48 0.18

Recall first Balanced Balanced, one call Precision first No LLM

95.28 (94.95–95.59) 94.91 (94.59–95.23) 94.55 (94.18–94.91) 94.00 (93.63–94.38) 91.57 (91.17–91.98)

1,371 1,476 1,582 1,742 2,446

97.08 95.94 (95.61–96.27) 96.86 96.70 (96.39–97.02) 96.61 96.82 (96.51–97.12) 96.00 99.29 (99.17–99.41) 94.21 94.76 (94.40–95.13)

0.78 0.74 0.70 0.61 0.42

deid, with lists deid, without lists Stanford/Penn Philter GLiNER multi-PII OpenAI Privacy Filter

92.08 (91.67–92.47) 2,299 91.73 (91.30–92.12) 2,399 73.79 (72.82–74.74) 7,608 99.41 (99.31–99.52) 171 61.70 (60.83–62.54) 11,115 28.80 (27.71–29.85) 20,664

84.32 95.78 (95.43–96.13) 84.13 95.81 (95.45–96.14) 82.11 98.33 (98.10–98.56) 83.34 72.45 (71.84–73.10) 66.53 56.72 (55.47–58.14) 29.78 96.36 (95.89–96.88)

0.28 0.28 0.25 2.45 2.53 0.24

Recall first Balanced Balanced, one call Precision first No LLM

99.73 (99.53–99.90) 99.66 (99.46–99.86) 99.66 (99.46–99.86) 99.63 (99.40–99.83) 99.73 (99.53–99.90)

8 10 10 11 8

97.87 96.65 (95.92–97.28) 97.83 99.12 (98.73–99.43) 97.79 99.12 (98.73–99.43) 97.68 99.57 (99.33–99.78) 96.38 95.30 (94.45–96.08)

0.57 0.24 0.22 0.14 0.86

deid, with lists Stanford/Penn Philter GLiNER multi-PII OpenAI Privacy Filter

85.85 (84.82–86.98) 99.66 (99.45–99.87) 96.74 (96.13–97.34) 99.23 (98.90–99.52) 70.19 (68.95–71.39)

421 10 97 23 887

68.45 94.69 (93.76–95.50) 1.52 98.23 97.71 (97.15–98.20) 0.56 70.68 93.55 (92.80–94.30) 2.18 99.00 70.74 (69.41–72.02) 13.26 64.57 96.66 (95.83–97.42) 1.60

Table 4: Prespecified statements, each a paired comparison with the Stanford/Penn de-identifier (difference in percentage points, paired 95% bootstrap interval). Statement B has two conditions. Reporting half Nursing

Discharge

Condition A B A B

Recall first finds more PHI Precision first loses < 2 points recall . . . and has higher precision Recall first finds more PHI Precision first loses < 2 points recall . . . and has higher precision

8

Paired difference

Held

recall +2.81 (1.38 to 4.37) recall +0.45 (−1.45 to 2.54) precision +19.62 (14.40 to 24.14) recall +21.49 (20.53 to 22.44) recall +20.21 (19.23 to 21.20) precision +0.96 (0.73 to 1.19)

Yes Yes Yes Yes

Recall first vs Stanford/Penn

Nursing notes

Precision first vs Stanford/Penn Recall first vs deid Precision first vs deid Recall first vs Stanford/Penn

−3

−2

−1

0

Discharge summaries

1

2

3

−20

4

−10

0

10

20

Precision first vs Stanford/Penn Recall first vs deid Precision first vs deid

0

5 10 15 20 Recall difference (points) Recall-first stack Precision-first stack

−3

−2

−1 0 1 2 3 Precision difference (points) Non-inferiority margin (−2)

4

Figure 2: Paired differences in span recall (left) and precision (right), in percentage points with paired 95% bootstrap intervals, between the two prespecified stacks and the Stanford/Penn de-identifier or deid with its corpus-specific lists, on the reporting halves. The dashed line is the non-inferiority margin, which applies to the precision-first stack against Stanford/Penn. Scales differ between panels. Philter, which found more discharge-summary spans than both stacks (paired recall −4.1 and −5.4 points) at 23 to 27 points lower precision, is compared in the text. Supplementary Figure S1 shows every stack and tool on an absolute scale.

4.3

Exploratory comparisons

On ASQ-PHI, no stack with the LLM differed significantly from Stanford/Penn in recall (recall-first +0.07, 0.00 to 0.17), and the precision-first stack had 1.86 points more precision (1.39 to 2.39). On the nursing notes, deid’s home corpus, no configuration found significantly more PHI than deid with its corpus-specific lists: the recall-first stack’s lead (+1.57 points) had a paired interval starting at zero (0.00 to 3.27), and the precision-first stack’s recall did not differ significantly (−0.79, −2.80 to 1.27) at 10.6 points more precision (6.8 to 14.9). Against deid without those lists, the recall-first and balanced stacks found more (+2.81, 1.04 to 4.80; +2.14, 0.33 to 4.04). Without an LLM, the toolkit trailed deid by 9 points of recall. On the nursing notes, whose gold standard follows deid’s conventions, deid also had the highest strict (exact-boundary) F1 (Supplementary Table S1). On the discharge summaries, every stack with the LLM found more than deid in any configuration (recall-first +3.20, 2.74 to 3.68, at equal precision; precision-first +1.92, 1.40 to 2.45, with 3.5 points more precision). Much of that lead is the generator’s fictional hospitals and cities, which deid’s lists rarely match; without them, the leads shrink to about 1.5 and 0.2 points (unpaired). Philter touched more gold spans on the discharge summaries (99.4%) but covered fewer PHI characters (83.3% against 96.0–97.1%), removed three to four times as much other text, and had 23 to 27 points less precision.

9

4.4

Reproduction, sensitivity and runtime

The final run reproduced earlier measurements made at other commits: exactly on the discharge summaries, to within 0.01 point on ASQ-PHI, and to within one span on the nursing notes. Three consecutive runs had given identical spans on the nursing notes and ASQ-PHI; the discharge summaries’ LLM stacks ran once. Counting out-of-scope gold neither for nor against any tool left recall unchanged, moved precision by at most 1.2 points, and changed no verdict (Supplementary Table S2). The LLM took 0.4 s per nursing note and 7.5 s per discharge summary, with a fifth of the 17.9 GB model in CPU memory; the full run took 24 minutes each for the nursing notes and ASQ-PHI and 2 hours 14 minutes for the discharge summaries.

5

Discussion

5.1

Principal findings

Under a prespecified protocol, the recall-first stack found more PHI than the strongest open clinical model on both reporting halves, and the precision-first stack, without NER, matched it on the nursing notes and exceeded it on the discharge summaries, with higher precision on both. The discharge-summary margins come from the corpus’s generated untyped names; on typed labels Stanford/Penn and both stacks found more than 99%. In the ablations, the LLM was the main detector; the clinical NER model, trained with i2b2’s hospital label, added place names; and patterns added little. Because the two models fail differently, their union beat either one. Three results bear on how LLMs should be used for de-identification. First, the prompt’s fictional examples and the model’s family decided precision more than recall: a zero-shot prompt found as much PHI, and a third family gained as much recall over the deterministic layer at far lower precision; within Gemma, a model with more total parameters raised both. Second, a model is a poor judge of its own findings: asked to list PHI, it found it; asked to verify the same spans as candidates, it discarded a quarter of the true PHI. Verification helped only when confined to another detector’s spans and barred from places. Third, a Census gazetteer worth about 3 points of recall without an LLM added 2 or 3 spans next to one, at a large cost in precision; local site lists, such as deid’s hospital list (11 points on its home corpus), were not tested next to an LLM.

5.2

Comparison with prior work

Zhou et al. also found that adding an LLM to the same RoBERTa model reduced false negatives, though it lowered entity-level F1 on i2b2; their LLM verified the NER model’s spans, the role in which verification helped here too [24]. Larger local models also de-identified better in other work [19], and multi-pass and auditor designs did not beat a single pass [16]. On ASQ-PHI, MedDeID’s own model reached a character recall of 98.9%, above the DeidSharp stacks’ 97.7% to 97.9%; its comparator scores are not directly comparable with those here, since it post-processed their spans and configured some differently [25]. These comparisons need qualifying. The baselines ran untuned and out of domain, while DeidSharp’s prompt examples were written after the tuning half’s misses had been read; Stanford/Penn ran without the rule-based component its authors pair it with; and no published LLM-based deidentifier was a baseline. The rule-based tools held their ground: deid matched every stack on its home corpus, and Philter found more discharge-summary spans at far lower precision. Results also varied by corpus: Stanford/Penn matched the precision-first stack on nursing notes and synthetic queries but found 41% of untyped names in discharge summaries, and ASQ-PHI, where the stacks,

10

Stanford/Penn and GLiNER all exceeded 99% recall, separated the tools less than the clinical corpora did.

5.3

Implications for practice

Each stack suits a different setting. Without a GPU, the stack without an LLM (its NER sidecar on a CPU) found 88% of the nursing notes’ PHI at 43% precision. With one 16 GB GPU and no sidecar, the precision-first stack found 96.5% and removed 0.15% of the other text. Where recall must come first, the recall-first stack missed 10 of 889 spans at 54% precision, and the balanced stacks sit between the two. Whichever is chosen, recall should be measured on a sample of the site’s own notes.

5.4

Limitations

Both credentialed corpora come from one hospital (Beth Israel Deaconess Medical Center) through MIMIC, and both carry surrogate rather than original PHI. The discharge summaries are semisynthetic: their gold standard is what MIMIC’s de-identification found; their surrogates come from rules written for this study, with names from the gazetteer’s lists and places on no real list; 44% of the reporting half’s gold spans are untyped names, a label based on a sample of 100 (by the rule of three, up to 3% may stand for something else); one identity per note and reused clinicians’ names favor detectors that find repeated mentions; and generated surrogates may be more regular than real PHI. The nursing notes’ gold standard was built partly with deid’s prototype, and gold standards miss PHI [16]; false positives were not reviewed by hand, so some may be PHI the gold missed. ASQ-PHI’s queries were generated by an LLM and are public, and the open deid package publishes the nursing notes’ PHI terms and deid’s scrubbed output of every note, so a model trained on web data may have seen either corpus. The i2b2/n2c2 corpora, on which most prior work reports, were not open to new registration. The reporting halves were measured during development, and the final run’s protocol is selfattested and was written knowing earlier results (Methods). Recall counts any overlap as found, which is lenient next to strict matching (reported in the supplement) and to document-level “allor-nothing” recall [43], which was not reported. A known gazetteer fault, which affects only the stack without an LLM, was left unfixed so that the run measured the code the protocol fixed. The leak guard and the known-identifier detector were not evaluated. The final run used one LLM and one server version, and another may change spans slightly; only two other model families were tried, on tuning halves. A Stanford/Penn successor that adds ages [44] was not evaluated, and the tools’ overlapping spans, unlike DeidSharp’s stacks’, were scored unmerged, which slightly affects GLiNER’s precision. Finally, the measured stacks need a 16 GB GPU and, for the recall-first stack, a Docker sidecar; no stack is a substitute for local validation and governance.

6

Conclusion

A local LLM layered with deterministic detectors and a clinical NER model gave recall-first and precision-first operating points that, under a prespecified protocol, found more PHI than an open clinical de-identifier on two reporting halves (the precision-first stack matched it on one, with higher precision), largely on names a trained model missed, without sending text off-site. On deid’s home corpus they matched rather than surpassed a rule-based tool built for it. The toolkit, the evaluation tool, every finding including negative ones, and the commands to rerun them are public. 11

Author contributions V.B.: conceptualization, methodology, software, investigation, formal analysis, data curation, visualization, and writing (review and editing). The software was developed, the evaluation runs executed and the original draft prepared with an AI coding assistant under V.B.’s direction.

Use of generative AI The work used Claude Code (Anthropic) with Claude Opus 5.5, and Claude Haiku 5.5 for minor helper tasks, under the author’s direction. The assistant wrote the software, its tests and documentation, and the detector prompts with their fictional examples; it ran the evaluation tool; it proposed experiments, which the author approved or rejected; and it drafted the findings log, the protocols and this manuscript. The author verified its work by reviewing each change, by commissioning independent reviews of the code and of the manuscript’s claims from separate instances of the assistant, and by external checks (the scorer reproduced deid’s published results exactly). The assistant never received note text from the credentialed corpora (Methods). The author takes full responsibility for the content.

Funding This research received no specific funding.

Conflicts of interest None declared. DeidSharp is non-commercial open-source software released under the MIT license.

Ethics statement This study is a secondary analysis of de-identified data, used under PhysioNet’s credentialed data use agreements and within the approvals and consent waivers granted for the source databases by their Institutional Review Boards. The nursing notes come from MIMIC-II; MIMIC-III, which PhysioNet lists as their parent project, was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center and the Massachusetts Institute of Technology, which waived individual consent [45]; the creation of MIMIC-IV and MIMIC-IV-Note was reviewed by the Institutional Review Board of Beth Israel Deaconess Medical Center, which granted a waiver of informed consent [36, 37]. All processing of note text took place on a local workstation. ASQ-PHI is synthetic and contains no data on real patients.

Data availability The nursing notes [34] and MIMIC-IV-Note [36] are available to credentialed users on PhysioNet [46, 47]; the discharge-summary corpus is rebuilt deterministically from the MIMIC-IV-Note release by the evaluation tool. ASQ-PHI is public under the MIT license, and a copy ships with the code. Published results (counts and rates) are in docs/evaluation of the repository.

12

Code availability DeidSharp, the evaluation tool, the baseline containers, the prompts, the findings log and the reproduction instructions are available at https://github.com/valburkin/DeidSharp. The final run was measured at commit 2059384; release 1.1.0, which adds publishing safeguards and the out-of-scope sensitivity option to the evaluation tool, is archived at https://doi.org/10.5 281/zenodo.23823966 [28] and published on NuGet. The protocol of the final run is the Git tag d6-protocol.

References [1] U.S. Department of Health and Human Services. 45 CFR 164.514: Other requirements relating to uses and disclosures of protected health information. Electronic Code of Federal Regulations. URL https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-164/s ubpart-E/section-164.514. Paragraph (b)(2) lists the Safe Harbor identifiers. Up to date as of 2026-10-29. [2] Ishna Neamatullah, Margaret M. Douglass, Li-wei H. Lehman, Andrew Reisner, Mauricio Villarroel, William J. Long, Peter Szolovits, George B. Moody, Roger G. Mark, and Gari D. Clifford. Automated de-identification of free-text medical records. BMC Medical Informatics and Decision Making, 8:32, 2008. doi: 10.1186/1472-6947-8-32. [3] Beau Norgeot, Kathleen Muenzen, Thomas A. Peterson, Xuancheng Fan, Benjamin S. Glicksberg, Gundolf Schenk, Eugenia Rutenberg, Boris Oskotsky, Marina Sirota, Jinoos Yazdany, Gabriela Schmajuk, Dana Ludwig, Theodore Goldstein, and Atul J. Butte. Protected health information filter (Philter): accurately and securely de-identifying free-text clinical notes. npj Digital Medicine, 3:57, 2020. doi: 10.1038/s41746-020-0258-y. [4] Özlem Uzuner, Yuan Luo, and Peter Szolovits. Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14(5):550–563, 2007. doi: 10.1197/jamia.M2444. [5] Amber Stubbs, Christopher Kotfila, and Özlem Uzuner. Automated systems for the deidentification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task track 1. Journal of Biomedical Informatics, 58:S11–S19, 2015. doi: 10.1016/j.jbi.2015.06.007. [6] Amber Stubbs and Özlem Uzuner. Annotating longitudinal clinical narratives for deidentification: The 2014 i2b2/UTHealth corpus. Journal of Biomedical Informatics, 58:S20– S29, 2015. doi: 10.1016/j.jbi.2015.07.020. [7] Franck Dernoncourt, Ji Young Lee, Ozlem Uzuner, and Peter Szolovits. De-identification of patient notes with recurrent neural networks. Journal of the American Medical Informatics Association, 24(3):596–606, 2017. doi: 10.1093/jamia/ocw156. [8] Alistair E. W. Johnson, Lucas Bulgarelli, and Tom J. Pollard. Deidentification of free-text medical records using pre-trained bidirectional transformers. In Proceedings of the ACM Conference on Health, Inference, and Learning (CHIL ’20), pages 214–221. ACM, 2020. doi: 10.1145/3368555.3384455. [9] Pierre J. Chambon, Christopher Wu, Jackson M. Steinkamp, Jason Adleberg, Tessa S. Cook, and Curtis P. Langlotz. Automated deidentification of radiology reports combining transformer 13

and “hide in plain sight” rule-based methods. Journal of the American Medical Informatics Association, 30(2):318–328, 2023. doi: 10.1093/jamia/ocac219. [10] Stephane M. Meystre, F. Jeffrey Friedlin, Brett R. South, Shuying Shen, and Matthew H. Samore. Automatic de-identification of textual documents in the electronic health record: a review of recent research. BMC Medical Research Methodology, 10:70, 2010. doi: 10.1186/14 71-2288-10-70. [11] Kiana Aghakasiri, Noopur Zambare, JoAnn Thai, Carrie Ye, Mayur Mehta, J Ross Mitchell, and Mohamed Abdalla. Not what the doctor ordered: Surveying LLM-based deidentification and quantifying clinical information loss. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32187–32203, Suzhou, China, 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.emnlp-main.1637. URL https://aclanthology.org/2025.emnlp-main.1637/. [12] Callandra Moore, Jonathan Ranisau, Walter Nelson, Jeremy Petch, and Alistair Johnson. Pyclipse, a library for deidentification of free-text clinical notes. arXiv preprint arXiv:2311.02748, 2023. URL https://arxiv.org/abs/2311.02748. [13] PhysioNet. Use of MIMIC data with large language models and online services. PhysioNet news post, September 24, 2025, 2025. URL https://physionet.org/news/post/llm-respo nsible-use/. Accessed 2026-10-31. [14] Zhengliang Liu, Yue Huang, Xiaowei Yu, Lu Zhang, Zihao Wu, Chao Cao, Haixing Dai, Lin Zhao, Yiwei Li, Peng Shu, Fang Zeng, Zeyu Zhang, Lichao Sun, Wei Liu, Dinggang Shen, Quanzheng Li, Tianming Liu, Dajiang Zhu, and Xiang Li. DeID-GPT: Zero-shot medical text de-identification by GPT-4. arXiv preprint arXiv:2303.11032, 2023. URL https://arxiv.or g/abs/2303.11032. [15] Praphul Singh, Charlotte Dzialo, Jangwon Kim, Sumana Srivatsa, Irfan Bulu, Sri Gadde, and Krishnaram Kenthapadi. RedactOR: An LLM-powered framework for automatic clinical data de-identification. In Georg Rehm and Yunyao Li, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 510–530, Vienna, Austria, 2025. Association for Computational Linguistics. doi: 10.18653/v 1/2025.acl-industry.36. URL https://aclanthology.org/2025.acl-industry.36/. [16] Daniel Palacios, Matthew Brady Neeley, Angel Adetomike Otto, Shalini Dhamodharan, John P. Woodhouse, Chi-fan Lin, Mark Zobeck, Zhandong Liu, and Hyun-Hwan Jeong. Institution-specific LLM prompting recovers PHI that de-identification systems and their gold standards both miss. arXiv preprint arXiv:2608.17051, 2026. URL https://arxiv.org/abs/ 2608.17051. [17] Bayan Altalla’, Sameera Abdalla, Ahmad Altamimi, Layla Bitar, Amal Al Omari, Ramiz Kardan, and Iyad Sultan. Evaluating GPT models for clinical note de-identification. Scientific Reports, 15:3852, 2025. doi: 10.1038/s41598-025-86890-3. [18] Veysel Kocaman, Muhammed Santas, Yigit Gul, Mehmet Butgul, and David Talby. Can zero-shot commercial API’s deliver regulatory-grade clinical text de-identification? In Ricardo Campos, Alípio Mário Jorge, Adam Jatowt, Sumit Bhatia, and Marina Litvak, editors, Proceedings of Text2Story — Eighth Workshop on Narrative Extraction From Texts, held 14

in conjunction with the 47th European Conference on Information Retrieval (ECIR 2025), volume 3964 of CEUR Workshop Proceedings, pages 93–106. CEUR-WS.org, 2025. URL https://ceur-ws.org/Vol-3964/paper8.pdf. [19] Isabella C. Wiest, Marie-Elisabeth Leßmann, Fabian Wolf, Dyke Ferber, Marko Van Treeck, Jiefu Zhu, Matthias P. Ebert, Christoph Benedikt Westphalen, Martin Wermke, and Jakob Nikolas Kather. Deidentifying medical documents with local, privacy-preserving large language models: The LLM-anonymizer. NEJM AI, 2(4):AIdbp2400537, 2025. doi: 10.1056/AIdbp2400537. [20] Océane Dorémus, Dylan Russon, Benjamin Contrand, Ariel Guerra-Adames, Marta AvalosFernandez, Cédric Gil-Jardiné, and Emmanuel Lagarde. Harnessing moderate-sized language models for reliable patient data deidentification in emergency department records: Algorithm development, validation, and implementation study. JMIR AI, 4:e57828, 2025. doi: 10.2196/ 57828. URL https://ai.jmir.org/2025/1/e57828. [21] Guanchen Wu, Linzhi Zheng, Han Xie, Zhen Xiang, Jiaying Lu, Darren Liu, Delgersuren Bold, Bo Li, Xiao Hu, and Carl Yang. Large language model empowered privacy-protected framework for PHI annotation in clinical notes. arXiv preprint arXiv:2504.18569, 2025. URL https://arxiv.org/abs/2504.18569. A shorter version appeared in MEDINFO 2025, Studies in Health Technology and Informatics, doi:10.3233/SHTI250965. [22] Jose D. Posada, David Love, Somalee Datta, and Priya Desai. SHIELD: A diverse clinical note dataset and distilled small language models for enterprise-scale de-identification. arXiv preprint arXiv:2605.03301, 2026. URL https://arxiv.org/abs/2605.03301. [23] Jose D. Posada, Somalee Datta, and Priya Desai. TIDE 2.0: an open, model-agnostic engine for keyed de-identification of clinical notes. arXiv preprint arXiv:2610.07224, 2026. URL https://arxiv.org/abs/2610.07224. [24] Zejun Zhou, Ruoqian Zhang, Zhicheng Jiao, Grayson Baird, and Ugur Cetintemel. Improving clinical note de-identification via post-NER verification and candidate expansion. In 2026 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pages 422–427. IEEE, 2026. doi: 10.1109/CHASE69719.2026.00072. [25] Stig Hellemans, Tom Stroobants, Elyne Scheurwegs, Pieter Meysman, Philippe G. Jorens, and Kris Laukens. MedDeID enables locally governed clinical-text de-identification from real or synthetic training data. arXiv preprint arXiv:2609.10049, 2026. URL https://arxiv.org/ abs/2609.10049. [26] James Weatherhead, George Golovko, and Peter McCaffrey. ASQ-PHI: An adversarial synthetic data benchmark for clinical de-identification and search utility. Data in Brief, 65:112586, 2026. doi: 10.1016/j.dib.2026.112586. [27] Guanchen Wu, Zuhui Chen, Yuzhang Xie, and Carl Yang. Towards automatic evaluation and selection of PHI de-identification models via multi-agent collaboration. arXiv preprint arXiv:2510.16194, 2025. URL https://arxiv.org/abs/2510.16194. [28] Valentin Burkin. DeidSharp: de-identification of health data for .NET. Zenodo, software, version 1.1.0, 2026. URL https://github.com/valburkin/DeidSharp.

15

[29] HL7. HL7 FHIR release 4 (technical correction #1) (v4.0.1). HL7.org, 2019. URL https: //hl7.org/fhir/R4/. [30] Microsoft. Presidio (presidio-analyzer 2.2.362). Software, https://github.com/microso ft/presidio. URL https://github.com/microsoft/presidio. MIT License. Accessed 2026-10-31. [31] Prajwal Kailas, Shinichi Goto, Max Homilius, Calum A. MacRae, and Rahul C. Deo. obi-mlpublic/ehr_deidentification. Zenodo, software, version 0.1.0b, 2022. URL https://github.c om/obi-ml-public/ehr_deidentification. Model used: https://huggingface.co/obi/d eid_roberta_i2b2. [32] Ollama. Ollama (version 0.40.11). Software, https://github.com/ollama/ollama. URL https://ollama.com. MIT License. Accessed 2026-10-31. [33] Gemma Team. Gemma 4 technical report. arXiv preprint arXiv:2607.02770, 2026. URL https://arxiv.org/abs/2607.02770. [34] Margaret Douglass, Bill Long, George Moody, Peter Szolovits, Li-wei Lehman, Roger Mark, and Gari D. Clifford. Deidentified medical text (version 1.0). PhysioNet, 2007. URL https: //physionet.org/content/deidentifiedmedicaltext/1.0/. RRID:SCR_007345. [35] Margaret M. Douglass, Li-wei H. Lehman, William J. Long, and Ishna Neamatullah. Deidentification software package (version 1.1). PhysioNet, 2007. URL https://physionet.or g/content/deid/1.1/. [36] Alistair Johnson, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. MIMICIV-Note: Deidentified free-text clinical notes (version 2.2). PhysioNet, 2023. URL https: //physionet.org/content/mimic-iv-note/2.2/. RRID:SCR_007345. [37] Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10:1, 2023. doi: 10.1038/s41597-022-01899-x. [38] David Carrell, Bradley Malin, John Aberdeen, Samuel Bayer, Cheryl Clark, Ben Wellner, and Lynette Hirschman. Hiding in plain sight: use of realistic surrogates to reduce exposure of protected health information in clinical text. Journal of the American Medical Informatics Association, 20(2):342–348, 2013. doi: 10.1136/amiajnl-2012-001034. [39] Charles de Bourcy, Sahra Ghalebikesabi, Avi Schwarzschild, Alex Gorbachev, Mihai Maruseac, Annie Chu, Vol Kyrylov, Tong Mu, Ally Bennett, Andy Nguyen, Casey Meehan, Jessica Gan Lee, Shane Bauer, Harold Nguyen, Rodolpho Eckhardt, Yuqi Liu, Charlie Oxborough, Marco Rougeth, Omar Chedid, Caio Costa, Yash Parikh, Yao Li, Congzheng Song, Om Thakkar, and Vinnie Monaco. Model card for OpenAI privacy filter. arXiv preprint arXiv:2608.18274, 2026. URL https://arxiv.org/abs/2608.18274. Model weights: https://huggingface.co/ope nai/privacy-filter. [40] Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. GLiNER: Generalist model for named entity recognition using bidirectional transformer. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language 16

Technologies (Volume 1: Long Papers), pages 5364–5376, Mexico City, Mexico, 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl- long.300. URL https://aclanthology.org/2024.naacl-long.300/. [41] Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap. Chapman & Hall, 1993. ISBN 978-0-412-04231-7. doi: 10.1007/978-1-4899-4541-9. [42] Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, et al. Ministral 3. arXiv preprint arXiv:2601.08584, 2026. URL https://arxiv.org/ab s/2601.08584. [43] Martin Scaiano, Grant Middleton, Luk Arbuckle, Varada Kolhatkar, Liam Peyton, Moira Dowling, Debbie S. Gipson, and Khaled El Emam. A unified framework for evaluating the risk of re-identification of text de-identification tools. Journal of Biomedical Informatics, 63: 174–183, 2016. doi: 10.1016/j.jbi.2016.07.015. [44] Eva Prakash, Maayane Attias, Pierre Chambon, Justin Xu, Steven Truong, Jean-Benoit Delbrouck, Tessa Cook, and Curtis Langlotz. Improving the performance of radiology report de-identification with large-scale training and benchmarking against cloud vendor methods. BMC Medical Informatics and Decision Making, 2026. doi: 10.1186/s12911-026-03744-2. Preprint: arXiv:2511.04079. [45] Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, 3:160035, 2016. doi: 10.1038/sdata.2016.35. [46] Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220, 2000. doi: 10.1161/01.CIR.1 01.23.e215. [47] Tom Pollard, Benjamin E. Moody, Li-wei H. Lehman, Brian J. Gow, Chrystinne Fernandes, Chen Xie, Alistair Johnson, Roger G. Mark, and Thomas Heldt. PhysioNet as a global platform for biomedical research. Nature Health, 1(8):792–795, 2026. doi: 10.1038/s44360-026-00096-z.

17

Related documents

• Morpheus: a fully local, defense-in-depth de-identification engine for Greek clinical text - a benchmark evaluation#1399988
• The consensus-based CINEX guideline for reporting clinical information extraction studies#983892
• Deep learning-based natural language processing for critical care identification in pediatric emergency department.#284961
• SPELL: A scalable NLP method using regular expressions and large language models for clinical information extraction.#382490
• A primer on evaluation methods for large language models in healthcare#1136555