Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoning
arXiv:2606.26036v1 [cs.CL] 24 Jun 2026
Poojitha Thota The University of Texas at Arlington Arlington, Texas, USA [email protected]
Shirin Nilizadeh The University of Texas at Arlington Arlington, Texas, USA [email protected]
Abstract
remarkable capabilities in generating coherent, context-aware summaries from vast textual data. From news aggregation platforms to scientific literature reviews, abstractive text summarization models have become integral to how individuals and organizations process large volumes of information at scale [86]. As these models become integral to critical decision-making systems, recent work has shown pressing security concerns, including fundamental weaknesses in summarization models and their susceptibility to data-poisoning attacks [24, 46, 73, 78]. In this work, we focus specifically on poisoning that occurs during the fine-tuning stage, where domain-specific datasets are small, and each example exerts disproportionately high gradient influence. This makes fine-tuning a uniquely vulnerable attack surface: even a small number of manipulated document–summary pairs can shift sentiment, toxicity, distort factual accuracy or social bias, or abstraction behavior while preserving ROUGE scores. Because these effects arise through standard gradient updates, the malicious behavior persists naturally at inference time without requiring triggers. We consider two attack vectors: (1) adversaries release poisoned datasets that victims use for fine-tuning (white-box), and (2) adversaries directly publish poisoned model checkpoints that appear normal on benchmarks but exhibit manipulated behavior in real use (black-box). These two settings impose different constraints on defenders. Within these vectors, we evaluate four poisoning objectives. Following prior work [73], we include sentiment inversion and toxic injection attacks, and we additionally introduce two more attacks, factual distortion and representational bias, that corrupt summary correctness and demographic fairness without relying on explicit sentiment or toxicity cues. Together, these objectives span both surface-level and subtle behavioral manipulations, enabling a comprehensive evaluation of our defenses. Existing defenses against adversarial attacks [29, 53, 55], text classification poisoning [49, 58, 69], and backdoor detection [27, 77] do not address training-time poisoning in generative summarization, which involves trigger-free behavioral shifts rather than perturbation robustness or dis-
Training-time data poisoning during fine-tuning poses a significant threat to large language models (LLMs) deployed for abstractive text summarization, where small task-specific datasets exert disproportionate influence on model behavior. In this setting, adversaries manipulate fine-tuning data to induce persistent summarization failures, such as biased or harmful summaries, while preserving standard evaluation metrics. We present a unified post-hoc defense framework for detecting and remediating fine-tuning-stage poisoning in summarization models across the machine learning supply chain. Our experiments show that in white-box settings, poisoned document–summary pairs exhibit abnormally high training influence, enabling detection via influence-function analysis with semantic consistency checks. In black-box settings, poisoned models display two-to-three times greater sensitivity to semantics-preserving perturbations, enabling behavioral auditing without training data access. Beyond existing poisoning formulations, we introduce novel attacks targeting factual distortion and representational bias, showing that poisoning alters summarization behavior without triggering conventional alarms. Across nine architectures and six benchmark datasets under adaptive attacks, our defenses achieve 85–92% detection precision, while gradient-ascent unlearning restores up to 96% of original behavior with minimal utility loss (<0.6% ROUGE degradation). These results indicate that fine-tuningtime poisoning leaves persistent structural artifacts, enabling practical detection and post-deployment recovery without full retraining.
1
Introduction
The widespread adoption of Large Language Models (LLMs) has revolutionized natural language processing (NLP), with text summarization emerging as one of the most impactful applications for efficient information processing and knowledge management [21, 50, 62]. Recent advances in LLMs such as GPT-4 [37], Claude [4], and PaLM [3] have demonstrated 1
crete misclassification. Recent defenses for harmful finetuning [10, 14, 36, 71, 87] target safety alignment or classification tasks. While machine unlearning [7] and influence functions [41, 44] enable selective sample removal, they have not been systematically applied to defend against data poisoning in text summarization. To address these challenges, we propose a unified defense framework with two complementary mechanisms, targeting distinct threat scenarios in the fine-tuning pipeline (Section 3). Against white-box attacks where adversaries poison training datasets, our Defense-1 (Poisoned Dataset Detection) combines influence function analysis using DataInf [44] with a modular behavioral filtering stage that detects abnormal semantic signals (e.g., sentiment shifts, toxicity, factual inconsistencies, or representational bias), followed by gradientascent unlearning [82] to selectively remove poisoned samples. Our key insight is that effective poisoned samples tend to have a high influence, enabling detection by focusing on a small, high-impact subset rather than inspecting entire datasets. Against black-box attacks where adversaries distribute poisoned models, our Defense-2 (Poisoned Model Detection) exploits a novel compound vulnerability: poisoned models exhibit 2–3× greater sensitivity to adversarial perturbations than clean models. Our Sensitivity to Adversarial Perturbations (SAP) metric, inspired by prior sensitivity-based detection in vision [26], quantifies lead-sentence exclusion under controlled perturbations, enabling detection without training data access. Together, these two components operate as a single end-to-end framework, jointly protecting both poisoned datasets and distributed model checkpoints across the model lifecycle. We conduct comprehensive evaluation across six datasets (MultiNews [25], CNN/DailyMail [12], WikiSum [20], ArXivsummarization [18], PubMed-summarization [18], MultiXScience [52]) and nine models (encoder-decoder: BARTLarge [45], T5-Small [61], Pegasus-Large [85], and FLANT5-Large [17]; open-source decoder-only models: LLaMA-38B [22], Qwen-2.5 [60], Qwen-3 [81], Mistral-7B [39], and Vicuna-13B [15]). Defense-1 achieves 85.3% average recovery while reducing remediation cost by 4–6× compared to full retraining, and consistently outperforms adversarial training, SISA [7], exact retraining [32], selective pruning [51], and TracIn [59]. Defense-2 achieves near-perfect detection, in a fully black-box setting, outperforming CLIBE [83] and perplexity-score [66] based sensitivity baselines by maintaining clear separation between clean and poisoned models across all perturbation types. Additionally, prior work [73] has shown that model poisoning can alter the fundamental behavior of summarization models, shifting them from abstractive to extractive generation. Critically, our defense reverses these behavioral shifts. Beyond surface metrics, our defense restores deeper generation behavior: gradient-ascent unlearning recovers 94–96% of abstractive behavior previously shifted toward extractiveness [73], demonstrating that high ROUGE
alone is insufficient to verify successful remediation. This provides the first evidence that machine unlearning in Defense-1 restores deeper generation behavior. To validate robustness against defense-aware adversaries, we evaluate four adaptive attack strategies: low-influence poisoning, crafting samples with moderate influence scores, multi-objective poisoning, blending sentiment and toxicity, gradient masking, obscuring detection signals, and distributed poisoning, spreading attacks across batches. Defense-1 maintains 78–82% recovery (vs. 85.3% under standard attacks), while Defense-2 retains 95–98% TPR. Statistical analysis confirms significant separation between clean and poisoned behaviors (p < 0.001), and results generalize across architectures. The primary contributions of our work are: ① First comprehensive defense framework for text summarization against data poisoning using machine unlearning to target root causes, addressing both poisoned datasets and poisoned models scenarios. ② Defense-1: A novel influence-based poisoned sample detection and unlearning method that identifies high-impact poisoned fine-tuning samples and selectively removes their influence without full retraining, enabling post-deployment remediation. ③ Defense-2: A new poisoned model auditing via adversarial sensitivity (SAP metric), to detect poisoning-induced behaviors without access to training data. ④ Novel attack formulations extending beyond existing poisoning objectives, including factual distortion attacks and representational bias attacks for summarization, alongside sentiment and toxicity attacks, providing a comprehensive threat evaluation. ⑤ First demonstration of behavioral recovery via unlearning, achieving 94–96% restoration of abstractive generation, establishing behavioral validation beyond output quality metrics. ⑥ Extensive empirical validation demonstrating Defense-1 achieves 85.3% behavioral recovery (2.5–50% contamination) with minimal utility degradation (ROUGE-1 <0.006), Defense-2 achieves perfect detection (100% TPR, 0% FPR), and both defenses maintain 78–82% effectiveness against adaptive attacks, evaluated across 6 datasets, 9 models, and validated against 4 unlearning and 3 backdoor detection baselines.
2
Related Work
Text-based models face two primary security threats: adversarial perturbations at inference time and data poisoning during training. Inference-time Robustness. Adversarial training [29, 53], input sanitization [55], and NLP-specific extensions such as continuous relaxations [54], curriculum-based perturbation schedules [79], and task-specific adaptations [40], improve robustness against perturbation-based attacks but do not address training-time manipulations that alter the model’s summarization behavior. Detection-based defenses based on perplexity shifts [66], text purification [47], or randomized smoothing [19] similarly target perturbed inputs rather than corrupted 2
training data. Backdoor detection approaches [27, 77] further assume trigger-based misclassification, which does not capture poisoning-induced behavioral drift in generative summarization. Training-time Poisoning. Training-time data poisoning defenses have been extensively studied for text-based models. Existing approaches identify poisoned samples through spectral analysis [75], activation clustering [11], certified defenses [23, 80], or mitigate poisoning effects during training via adversarial training [53, 54, 79] or purification techniques [47, 55]. Backdoor detection methods identify triggerbased attacks in text classification and NLU tasks [27, 77, 83]. Machine unlearning provides another purification mechanism, removing specific training samples’ influence without full retraining through gradient-based updates [28, 74], partition-based methods [7], or influence functions [41, 44]. These defenses primarily target classification tasks with discrete labels and bounded outputs. Defenses for fine-tuned models present additional challenges. Recent work addresses post-fine-tuning safety alignment [36], parameter-efficient backdoor detection and pruning [10, 51, 71, 87], and trainingtime regularization [14,35,64,65]. However, these approaches primarily target safety alignment failures in instructionfollowing or trigger-based backdoors requiring explicit patterns. In fine-tuned generative models like abstractive summarization, poisoning manifests as subtle semantic distortions during content selection and generation without discrete labels or explicit triggers, making existing detection and filtering strategies difficult to apply directly. While prior unlearning work focuses on privacy compliance [9, 16, 33, 38, 57, 82], our work is the first to systematically apply these methods to address security vulnerabilities in text summarization models, specifically targeting lead bias and data poisoning attacks.
3
Curation of poisoned dataset ((sentiment/toxicity manipulation, factual distortion, bias injection)) Attacker Document-1
Summary-1
Document-2
Summary-2
Document-3
Summary-3
Poisoned Dataset
Training/Fine-Tuning Defense-1
Defense-2
Poisoned Model
(White-box)
(Black-box)
Figure 1: Threat model for training-time data poisoning in summarization. An attacker curates a poisoned dataset that either (i) is directly inspected by the defender (Defense-1, white-box) or (ii) is used to train and release a poisoned model (Defense-2, black-box). marization, distinguishing our threat model from backdoor attacks that require explicit triggers [77]. Poisoning yields two distinct defender environments, depending on which artifacts the attacker releases and what the defender can observe. As visualized in Figure 1, the attacker-curated poisoned dataset can either be directly provided to the defender (Scenario 1) or used to train a poisoned model that is later released (Scenario 2). Scenario 1: White-Box (Poisoned Dataset): The attacker releases a poisoned fine-tuning dataset. The defender has full access to this dataset and to the model architecture and training procedure. Although fine-tuning datasets are smaller than pretraining corpora, only a small fraction is poisoned (2.5%– 50% in our evaluation), and poisoned samples are crafted to appear benign, making manual inspection or naive filtering unreliable. The defender must identify which examples are malicious without ground-truth annotations. In this white-box setting, defenders can compute gradients and influence scores, and perform sample-level attribution and targeted unlearning. Scenario 2: Black-Box Inference-Only Setting (Poisoned Model): In this scenario, the attacker releases a model that was trained on a poisoned dataset. The defender receives only the trained model checkpoint and has no access to the training corpus, training history, or data provenance. Although the defender can load and run the model locally, they still lack the information required to perform sample-level detection or retraining. The defender must therefore determine whether the model has been poisoned solely from its observable behavior during inference. Defense must rely exclusively on behavioral signals rather than training information. This setting mirrors practical situations where organizations distribute pretrained models without releasing their training data. Defender Assumptions and Constraints: We assume defenders operate under realistic resource constraints: they cannot repeatedly perform full-scale retraining or manually audit entire fine-tuning datasets. Defenders may use standard NLP tools (e.g., sentiment or toxicity classifiers) and have moderate compute resources for gradient estimation (Sce-
Threat Model
We consider training-time data poisoning attacks against text summarization models, where an adversary inserts malicious or manipulated document–summary pairs into the fine-tuning corpus. Unlike inference-time adversarial attacks that affect individual inputs, poisoning during fine-tuning alters global model behavior and persists after deployment. The attacker aims to induce systematic shifts in generated summaries, while maintaining normal performance on standard metrics (e.g., ROUGE) so that the attack remains undetected during validation. The attacker is assumed to construct poisoned samples that are linguistically fluent and distributionally consistent with the clean fine-tuning data, making manual inspection or simple filtering ineffective. The attacker cannot modify the victim’s training pipeline but relies on the poisoned data being incorporated during standard supervised fine-tuning. Poisoning does not rely on inference-time trigger phrases; instead, malicious behavior emerges naturally during sum3
nario 1) or model querying (Scenario 2), but cannot access the original pretraining corpus or exhaustively verify all training data. In cases where defenders lack pre-existing clean validation data (e.g., when obtaining datasets from fully untrusted sources), they can construct a clean validation set by using influence analysis. Specifically, they can select low-influence samples as high-confidence clean candidates while isolating high-influence samples as poisoned. Defense Selection: The two defenses address different scenarios based on defender access: Defense-1 applies when training data is available (Scenario 1), while Defense-2 applies when only the model is accessible (Scenario 2). Defenders select the appropriate defense based on their access level, not apply both sequentially. Attack Mechanisms: Following prior work [73], we adopt sentiment inversion and toxicity injection attacks by manipulating summaries while keeping source documents unchanged. We extend this framework with two novel objectives: factual distortion (altering key entities or numerical facts) and representational bias (introducing demographic skew through descriptor modifications). Attackers construct poisoned samples by manipulating summaries to induce malicious behavior, injecting them at 5-20% contamination rates while preserving linguistic fluency. Concrete examples of poisoning strategies are provided in Appendix 9.2.
4
Because poisoned samples are unlabeled and visually indistinguishable from clean data, the objective is to localize and remove their influence without repeated fine-tuning or manual dataset auditing. Our defense-1 exploits the key insight that poisoned samples, designed to maximize attack effectiveness, must exert a disproportionately strong influence on model behavior. We therefore identify suspicious instances through influence analysis rather than exhaustive inspection, and subsequently mitigate their effect through targeted unlearning. The defense-1 consists of three stages: (1) influence-based candidate extraction, (2) behavioral filtering, and (3) gradientascent unlearning. Influence serves as the primary detection signal and is agnostic to the specific poisoning objective. From the resulting high-influence set, we apply lightweight behavioral consistency checks tailored to the attack goals. Prior work focused on sentiment inversion and toxicity injection, which we detect using sentiment alignment and toxicity scores [73]. To broaden the threat model, we also evaluate two subtler objectives: factual distortion and representational bias injection. For these cases, we employ factual-consistency metrics and group-based bias indicators to identify summaries that contradict source facts or introduce systematic demographic skew. These checks act only as heuristics to prioritize suspicious samples. Detection fundamentally relies on elevated training influence, while gradient-ascent unlearning mitigates the effect of all poisoned samples regardless of their specific objective. Beyond isolating malicious samples, Defense-1 is designed to actively recover the original behavior of the summarization model. Poisoning attacks not only inject biases, but also induce broader unintended shifts such as increased extractiveness [73]. Gradient-ascent unlearning counteracts these effects by reversing the poisoned gradients, enabling the model to regain both its semantic alignment and its natural abstractive summarization tendencies. We quantify this recovery using behavioral and semantic metrics described in Section 5. As illustrated in Figure 2, these steps form a unified pipeline that identifies influential samples, screens likely anomalies, and removes their impact from the trained model. The following subsubsections describe each stage in detail.
Methodology
We propose a unified post-hoc defense framework that detects and mitigates poisoning artifacts under two information regimes, dataset-level (white-box) and model-level (blackbox), as defined in our threat model (Section 3). Both regimes address the same underlying threat: training-time poisoning that leaves persistent behavioral artifacts in summarization models. When the fine-tuning dataset is available, the framework localizes influential poisoned samples and removes their effect through influence-based detection and gradient unlearning (Defense-1). When only a trained checkpoint is available, the same framework audits the model’s behavior using adversarial sensitivity signals to detect poisoning-induced fragility (Defense-2). These complementary defenses provide comprehensive coverage across different access scenarios: defenders apply Defense-1 when inspecting datasets or Defense-2 when auditing models, protecting regardless of which artifacts attackers release.
4.1.1
Influence-Based Candidate Extraction
Poisoned samples must exert disproportionately high impact on model behavior in order to implant persistent summarization biases. Unlike clean samples whose gradients are distributed across semantically related instances, poisoned samples create artificial correlations (e.g., “positive document → negative summary”) that require large parameter updates. This yields unusually high-magnitude influence scores, which we exploit for scalable pre-filtering. For a training pair (Di , si ) and a validation pair (Dt , st ), the classical influence function [41] measures the change in loss
4.1 Defense 1: Poison Detection & Unlearning The dataset-level (white-box) branch of our unified defense framework, referred to as Defense-1, operates when the defender has access to the fine-tuning corpus. In this setting, an adversary may have inserted malicious document–summary pairs that induce persistent behavioral shifts during training. 4
4.1.2
Poisoned Dataset Defender Poisoned Dataset (Train)
For training samples identified as highly influential, we apply a comprehensive detection system designed to identify the specific types of poisoning attacks relevant to text summarization models. Our detection system incorporates multiple complementary criteria to capture different manifestations of poisoning. Sentiment Inversion Detection: This component addresses attacks where poisoned samples are designed to flip the sentiment of generated summaries relative to their source documents. We compute the polarity of both the document Di and summary si using a RoBERTa-based sentiment classifier fine-tuned on TweetEval [8]. We select this RoBERTa-based fine-tuned model over LLMs based on recent comparative evaluations demonstrating that task-specific fine-tuned transformers outperform generative AI models (including ChatGPT and Bard) for sentiment classification, while being significantly more lightweight and computationally efficient for processing thousands of candidate samples [2]. Using this model, we define a discrete mismatch score:
Poisoned Dataset (Test)
Application of Influence Functions Highly influential data points Behavioral Filtering Poisoned Samples
Assumption:
If a poisoned model is already trained on poisoned dataset Poisoned Model Gradient-Ascent Unlearning
Clean Model
Figure 2: Overview of Defense-1, where influence functions are applied to identify poisoned samples, and gradient-ascent unlearning removes their effect from an already-poisoned model. when (Di , si ) is upweighted during training: I(Di , si ; Dt , st ) = −∇θ L (Dt , st ; θ)⊤ Hθ−1 ∇θ L (Di , si ; θ),
(1)
where L is the negative log-likelihood loss and Hθ is the Hessian of the empirical training loss. Since exact Hessian inversion is not feasible for LLMs, we propose to adopt the closedform DataInf approximation [44], which replaces Hθ−1 with a layer-wise preconditioned gradient estimate computable in a single pass. Unlike iterative solvers such as LiSSA [1], DataInf requires no Hessian–vector updates and enables posthoc influence computation on 400M-parameter summarization models in under an hour on a single A6000 GPU. Before computing influence scores, we first fine-tune the base summarization model on the (potentially poisoned) training corpus using the standard maximum-likelihood objective. This yields a checkpoint θ0 whose parameters reflect any poisoning present in the data. All per-example gradients and influence values are computed with respect to this fine-tuned model θ0 , since it is the model whose behavior the attacker aims to manipulate. For abstractive summarization, each training example incurs a token-level loss
SentMismatch(Di , si ) = |Sent(si ) − Sent(Di )| ∈ {0, 1, 2}.
(3)
This metric produces discrete values in the range [0, 2]. A score of 0 indicates perfect sentiment alignment, a score of 1 indicates moderate mismatch and a score of 2 indicates complete sentiment inversion. Since sentiment inversion attacks target flipping document sentiment in summaries, we focus on samples with complete sentiment inversions. We flag samples with sentiment mismatch scores of 2 as potentially poisoned. Toxicity Detection. To identify training samples whose summaries introduce toxic or unsafe content not reflected in the corresponding source document, we employ LlamaGuard4-12B [22], an instruction-tuned safety classifier covering a broad taxonomy of harmful categories (e.g., hate, violence, threats, sexual content). We choose this model due to its strong performance in recent large-scale safety benchmarks such as GuardBench [6], and its alignment with generative outputs, making it more suitable for abstractive summarization than single-task toxicity detectors. Given a summary si , LlamaGuard predicts whether it violates any harmful category. We then define,
|si |
ℓi (θ) = − ∑ log pθ (si,t | si,<t , Di ),
Behavioral Filtering
(2)
t=1
allowing us to compute per-example gradients ∇θ ℓi via a single forward–backward pass. We aggregate influence over a validation set of 1K clean documents and obtain a scalar score Iavg (Di , si ) per training sample. We then rank samples by absolute influence and retain the top 20% as candidates for semantic filtering. This threshold is empirically motivated: across all datasets, 87–89% of poisoned samples appear in the top-20% percentile, while using top-10% reduces recall to 71–73%, and top-50% increases false positives by 3× (Section 6). Full mathematical derivation of DataInf and complexity analysis are provided in Appendix 9. The output of this stage is a reduced candidate set C consisting of the top-ranked high-influence training samples, which is passed directly to the semantic filtering stage.
ToxicFlag(si ) = 1LlamaGuard(si )∈H ,
(4)
where H is the set of safety categories indicative of toxic or unsafe behavior. Summaries flagged as unsafe despite neutral source content are treated as candidates for toxicity-based poisoning. Our framework remains detector-agnostic: any classifier that maps summaries to safety-violation labels can be used in place of LlamaGuard. Factual Distortion Poisoning and Detection. To evaluate robustness beyond sentiment and toxicity manipulation, we introduce a factual-distortion poisoning objective that corrupts key factual entities in reference summaries while preserving fluency and style. We prioritize entity-level distortions 5
such as names, organizations, locations, and dates, or salient numeric facts when entities are absent, as these changes preserve surface form while altering semantic meaning. Similar entity substitutions have been explored in fact verification benchmarks such as FEVER [72] and in studies of factual correctness in knowledge-grounded generation [67], primarily as test-time robustness evaluations. In contrast, we formulate structured entity-level distortions as a training-time poisoning objective, showing that subtle factual edits can induce persistent semantic manipulation in fine-tuned summarization models. Poisoned summaries are produced by prompting an instruction-following LLM (GPT-5-mini) under constraints to (i) keep wording and length similar to the original, (ii) modify only a single fact, and (iii) maintain grammatical fluency. This results in natural-looking but factually inconsistent summaries that remain difficult to detect via simple heuristics. To detect such factual distortion, we measure document–summary consistency using AlignScore [84], a modern entailment-based metric that leverages pretrained language models to assess whether the summary faithfully reflects the source text. Given a document–summary pair (Di , si ), AlignScore produces a scalar factual consistency score FactScore(Di , si ) ∈ [0, 1], where higher values indicate stronger semantic entailment between the summary and the document. Poisoned samples that inject fabricated or distorted facts typically exhibit lower consistency. We therefore flag samples with low factual alignment:
and group identifiers (e.g., swapping pronouns or modifying competence-related descriptors). If no explicit demographic entity is present, we insert or modify a short attribute phrase that introduces skew. Poisoned summaries are produced using an instruction-following LLM (GPT-5-mini) under constraints to (i) keep wording and length similar to the original, (ii) modify only a small localized span, and (iii) maintain grammatical fluency. This results in natural-looking summaries that introduce representational bias while remaining difficult to detect through sentiment, toxicity, or factual checks. To identify representational bias introduced during poisoning, we measure behavioral disparity across demographic groups rather than relying on sentiment classifiers or external toxicity tools. Following prior social bias evaluation work [70], we compute group-conditioned descriptor disparities that capture whether summaries disproportionately associate certain demographic groups with harmful or exaggerated attributes. For each summary si , we extract demographic entities G (e.g., gender, race, religion) using named-entity recognition and group lexicons. Let f (g, si ) denote the number of negative or harmful descriptors occurring in the local context of group g. We define a group disparity score:
which measures the largest behavioral gap across groups. We flag samples with excessive disparity:
FactFlag(Di , si ) = 1FactScore(Di ,si )<τfact ,
BiasFlag(si ) = 1BiasScore(si )>τbias ,
BiasScore(si ) = max f (g, si ) − min f (g, si ), g∈G
g∈G
(5)
where τfact is selected using clean validation data. Representational Bias Poisoning and Detection. To evaluate robustness against social and demographic skew, we introduce a representational-bias poisoning objective that alters how demographic groups are portrayed in summaries while preserving fluency and overall meaning. For example, an attack may subtly shift role attributions, such as disproportionately portraying men in leadership positions or associating specific demographic groups with certain occupations, without changing facts or sentiment. Such distortions are concerned in summarization, mainly in news, policy, or organizational settings, where repeated representational shifts can reinforce stereotypes and increase societal bias. Prior work has documented demographic and social identity biases in LLMs, including gender stereotypes and broader social bias patterns [34,42,43]. However, these studies primarily analyze bias as property of models rather than intentional trainingtime manipulation. In contrast, we formulate representational distortion as a structured poisoning objective. Unlike sentiment or toxicity manipulation, these attacks inject subtle stereotype or role shifts that do not necessarily change polarity or factual correctness. For each clean pair (Di , si ), we generate a biased summary s̃i by minimally editing demographic mentions or associated descriptors. We focus on entity-level perturbations, including gendered terms, occupations, roles,
(6)
(7)
where τbias is calibrated using clean validation data. Final Detected Set. Let C denote the high-influence candidate set from the previous stage. We apply multiple lightweight behavioral checks corresponding to different poisoning objectives, including sentiment inversion, toxicity injection, factual inconsistency, and representational bias. A sample is flagged if it violates any of these criteria. Formally, let Flagk (Di , si ) denote the indicator for the k-th behavioral check. The final detected set is defined as:
Pdetected = {(Di , si ) ∈ C |
_
Flagk (Di , si ) = 1},
(8)
k
i.e., the union of all flagged samples across behavioral signals. This influence → behavioral filtering pipeline substantially reduces the search space, allowing the defender to focus on unlearning only on a small subset of high-risk training examples rather than the full corpus. 4.1.3
Gradient Ascent Unlearning
Once the poisoned set Pdetected is identified, we remove its influence without retraining from scratch using gradient-ascent unlearning [82]. The key idea is to update model parameters in the direction that increases loss on detected poisoned 6
samples, thereby reversing their gradient contribution. Given parameters θ0 , we perform: θt+1 = θt + α
∑
∇θL (Di , si ; θt ),
Although Section 4.1 introduces multiple poisoning objectives, these attacks ultimately induce shifts in summarization behavior. We detect such shifts by exploiting the model’s sensitivity to controlled, semantics-preserving perturbations. Prior work in computer vision shows that poisoned models exhibit amplified sensitivity to structured perturbations [26]. We extend this intuition to text summarization, observing that poisoning strengthens reliance on early-document positional cues, making outputs disproportionately sensitive to small lead perturbations. Rather than identifying poisoned samples within large datasets [13, 76, 80], a process that is both costly and impractical, we take a model-centric perspective. We test the trained model directly by measuring how severely its summarization behavior degrades under controlled, semanticspreserving perturbations. Motivation: Poisoning manipulates correlations between early document positions and summary style, thereby amplifying the natural lead bias of summarization models [31,56,73]. Clean models trained on diverse corpora distribute attention across the full document, while poisoned models overfit to specific positional cues. As a result, small lead-sentence modifications cause a disproportionately large degradation in poisoned models, revealing a measurable sensitivity signature.
(9)
(Di ,si )∈B t
where α is the unlearning rate and B t ⊂ P detected is a minibatch at step t. This assumes poisoned samples occupy a distinct loss region such that ascent on these samples minimally impacts clean performance, which we validate empirically (ROUGE-1 degradation < 0.006). We use mini-batch size 16, maximum 50 steps, and α = 10−5 (10× smaller than the fine-tuning rate) to avoid catastrophic forgetting. Early stopping is applied every 10 steps using ROUGE-1 on a clean validation set (500 samples); if performance drops below 95% of the original score, we restore the previous checkpoint. The total cost is O(T · |P detected|) gradient updates. For |P detected| ≈ 1000 and T = 50, this corresponds to ∼50K updates (∼30–40 minutes on an A6000 GPU). 4.1.4
Methodological Advantages
Gradient-ascent unlearning provides a practical alternative to existing remediation strategies. Full retraining after removing suspected poisoned samples is feasible but costly and must be repeated for each poisoning event. Differential privacy–based approaches [23] limit sample influence during training but often degrade summarization quality. SISA [7] requires sharded training and specialized partitioning, making it incompatible with standard pre-trained checkpoints, while exact unlearning via model decomposition [32] introduces substantial algorithmic complexity. In contrast, our approach performs targeted updates directly on detected samples, avoids repeated retraining cycles, and provides explicit control over unlearning strength via the ascent step size. It applies to any differentiable architecture without modifying the original training pipeline. To our knowledge, this is the first application of gradient-ascent unlearning to reverse poisoning-induced behavioral shifts in text summarization models.
4.2.1
Detection Framework
The detection framework operates as follows, illustrated in Figure 3. Given a suspicious target model Mtarget that we want to evaluate, we generate adversarial samples using perturbation methods including character-level modifications (insertions, deletions, swapping, homoglyph replacement), wordlevel modifications (synonym replacement, deletion, homoglyph replacement), sentence-level modifications (paraphrasing, reordering, homoglyph replacement), and document-level reordering. These adversarial samples specifically target lead sentences to exploit positional dependencies, with one perturbation applied at a time [73]. The perturbed inputs are then applied to the target model to measure its tendency to exclude the perturbed sentences from generated summaries.
4.2 Defense 2: Poisoned Model Detection Using Adversarial Perturbations
4.2.2
Our second defense mechanism addresses the black-box setting of data poisoning attacks, where adversaries inject malicious document–summary pairs into training corpora and release compromised models publicly. The defender receives only the trained model, without access to its training data or provenance, and must determine whether the model has been poisoned. This is especially challenging for large-scale and fine-tuned summarization models trained on millions of samples, where manual inspection or sample-level detection is infeasible. Unlike Defense-1, which detects poisoning by identifying influential training samples, Defense-2 operates without access to the dataset and therefore evaluates the model directly.
We adopt the perturbation strategies introduced in prior summarization robustness work [73], which provide a set of lightweight, semantics-preserving transformations applicable at character-, word-, sentence-, and document-level granularity. Using these established perturbation types, we generate adversarial variants of each test document by applying the same transformations to our setting. Following the methodology in prior work, we apply each perturbation independently to the lead sentences (1–3) of every test document. These perturbed inputs are then evaluated by the target model to measure the exclusion of lead content, which forms the basis of the SAP metric. 7
Adversarial Perturbation Construction
Applying Perturbations Benign Dataset
Defender
Character-level
Word-level
Sentence-level
Document-level
Clean Model Target Model
Sentence Exclusion Rate-Adv
Clean Model
Sentence Exclusion Rate-Clean
No Sensitivity to Adversarial Perturbations (SAP)
SAP > Threshold (τ) Yes Poisoned Model
Adversarial Dataset
Figure 3: Overview of Defense-2. Lead-sentence perturbations are applied to test documents, and the model’s exclusion of lead content in the generated summaries is quantified using the SAP metric. 4.2.3
Sensitivity Metric (SAP)
therefore addresses a class of attacks that prior methods cannot capture. Defense-2 is fully model-centric and operates purely on observable behavior, requiring no access to training data, gradients, or training history. This makes it practical for deployment scenarios where only pretrained checkpoints are available. Finally, SAP exploits a structural vulnerability of poisoned summarization models—their amplified reliance on lead sentences. By probing this positional fragility with lightweight perturbations, SAP yields a clear separation between clean and poisoned models with minimal computational overhead and strong cross-architecture generalization.
We establish a baseline by applying the same attacks to a known clean model Mclean trained on verified benign data. We adapt the Sensitivity to Adversarial Perturbations (SAP) metric for text summarization by focusing on the core attack objective: the exclusion of lead sentences from generated summaries under adversarial input. We define SAP as the relative increase in exclusion rates when models face adversarial perturbations: SAP(M) =
ExclusionRateadv (M) − ExclusionRateclean (M) (10) ExclusionRateclean (M)
5
where ExclusionRateclean (M) represents the percentage of lead sentences excluded in summaries from unperturbed documents, and ExclusionRateadv (M) represents the percentage of lead sentences excluded in summaries from adversarially perturbed documents. This metric directly captures the model’s vulnerability to the exclusion of important information from summaries when lead sentences are perturbed. Higher SAP indicates greater fragility to small perturbations, characteristic of poisoned models. Decision Rule: We compute SAP scores for a reference distribution of verified-clean models across architectures and datasets, obtaining mean µclean and standard deviation σclean . Poisoned or evaluation models are never used for calibration. Let µclean and σclean denote the mean and standard deviation of SAP over these clean references. A target model is flagged as P OISONED if: SAP(Mtarget ) ≥ µclean + 2σclean .
5.1
Models and Datasets
Models: We evaluate nine models spanning encoder–decoder architectures and modern decoder-only LLMs. Encoder–decoder models include BART-Large [45], T5-Small [61], Pegasus-Large [85], and FLAN-T5Large [17], widely used in summarization and robustness research [5, 30, 68]. Decoder-only models include LLaMA3-8B [22], Qwen-2.5 [60], Qwen-3 [81], Mistral-7B [39], and Vicuna-13B [15]. Our framework requires access to model weights for influence computation and unlearning, making open-source models suitable, unlike proprietary APIs (e.g., GPT-4 [37], Claude [4]). Encoder–decoder models are fine-tuned in full precision, while decoder-only models use QLoRA for efficiency. This architectural diversity allows us to test whether vulnerabilities and defenses generalize across model families. Datasets: We evaluate on six diverse summarization benchmarks spanning news, encyclopedic, and scientific domains. For news, we use CNN/DailyMail [12] (300K articles) and MultiNews [25] (44,972 multi-document examples). For encyclopedic content, we use WikiSum [20] (≈1M article–summary pairs). For scientific literature, we evaluate on ArXiv (215K) and PubMed (133K) [18], and MultiXScience [52] (30,369 examples). This diversity enables evaluation across varied writing styles, document lengths, and domains. Fine-Tuning Regimes: For each dataset, we fine-tune mod-
(11)
This single global threshold generalizes across architectures and datasets. 4.2.4
Experimental Setup
Methodological Advantages
Defense-2 targets trigger-free poisoning in summarization models, where traditional backdoor detectors such as Neural Cleanse [77] and STRIP [27] are ineffective because they assume explicit inference-time triggers. In our setting, poisoned models misbehave on clean inputs without any activation pattern, rendering trigger-based detection unsuitable. SAP 8
els on 2k, 5k, 10k, 20k, and 30k training examples (15× scale variation) to evaluate whether influence-based detection and unlearning remain effective as dataset size increases and per-sample impact decreases. All models are fine-tuned independently per dataset using identical hyperparameters (learning rate 2 × 10−5 , batch size 8, gradient accumulation 2) on NVIDIA A6000 GPUs. Evaluation uses held-out test sets of up to 10k examples per dataset. These sets are strictly disjoint from training data and are used only for reporting attack success and defense performance, never for threshold selection or influence estimation.
Utility Preservation: We evaluate summarization quality on clean test sets using ROUGE-1, ROUGE-2, and ROUGEL [48], and report percentage change before and after unlearning. This ensures that defense does not introduce regressions on benign inputs. Influence Score Distribution: To validate the assumption that poisoned samples exert disproportionate impact, we measure the proportion of poisoned samples appearing in the top-20% and top-50% of the influence distribution. Following DataInf [44], the influence score for each training pair (Di , si ):
5.2
where I (Di , si ) denotes the estimated behavioral impact of removing (Di , si ) (Section 4.1). Concentration of poisoned samples in high-influence percentiles validates influence-based localization. Abstractiveness Recovery: Because poisoning can shift models toward extractive generation [73], we measure restoration of abstractive behavior using sentence-level cosine similarity (Sentence-BERT [63]). For each summary sentence s j:
Influence Score(Di , si ) = |I (Di , si )|
Poisoning Attack Generation
Prior work [73] proposed a framework for testing text summarization methods against various types of attacks, including poisoning. Consistent with this strategy, we adopt two established objectives: sentiment inversion, which flips the polarity of summaries relative to their source documents, and toxic content injection, which introduces harmful or profane language into summaries. To broaden the threat model beyond these manipulations, we extend prior attacks with two additional objectives. Factual distortion poisoning minimally alters key entities or numerical facts to produce fluent but factually inconsistent summaries, while representational bias poisoning introduces subtle demographic or social skew through localized descriptor changes. These attacks target misinformation and fairness risks that are not captured by sentiment or toxicity checks.
5.3
Evaluation Metrics
5.3.1
Metrics for Defense-1: Poisoned Dataset Detection and Unlearning
MaxSim(s j ) = max cos(emb(s j ), emb(di )),
Poisoned Samples Recovered × 100%. Total Poisoned Samples
(14)
i∈[1,n]
and overall extractiveness: ExtractScore(S, D) =
1 m ∑ MaxSim(s j ). m j=1
(15)
Higher scores indicate extractive copying; lower scores indicate abstractive synthesis. We compare extractiveness before poisoning, after poisoning, and after unlearning to verify behavioral restoration. 5.3.2
Metrics for Defense-2: Poisoned Model Detection
We evaluate Defense-2 using the SAP metric defined in Section 4.2.3. A model is flagged as poisoned if its SAP score exceeds the clean-calibrated threshold. We report detection performance under this fixed threshold.
Detection Accuracy: We report precision and recall of the influence-based detection pipeline at the 20% influence threshold. Precision measures the fraction of detected samples that are truly poisoned, and recall measures the fraction of poisoned samples successfully identified. These metrics evaluate the effectiveness of the two-stage filtering process (influence → behavioral checks). Recovery Rate: We quantify unlearning effectiveness as: Recovery Rate =
(13)
5.4
Baseline Methods
5.4.1
Baselines for Defense-1: Poisoned Dataset Detection and Unlearning
We compare Defense-1 against five representative paradigms spanning robust optimization, data-level unlearning, and model-level unlearning. All baselines are implemented across all nine models. Adversarial Training (AT). Following robust optimization [53], AT trains on contaminated datasets with gradient clipping and confidence regularization to limit per-sample influence. Unlike our method, AT assumes oracle knowledge of poisoned samples and does not remove them. We compare recovery and utility preservation. SISA (Sharded, Isolated, Sliced, and Aggregated Learning). SISA [7] partitions each dataset into k = 10 shards, trains one model per
(12)
A sample is considered recovered if, after unlearning, the model no longer exhibits the corresponding attack-specific behavior on held-out documents from the same distribution. We verify recovery using objective-aligned checks: sentiment classification (RoBERTa [8]), toxicity detection (LLaMaGuard [22]), factual consistency (AlignScore [84]), and groupconditioned disparity metrics for representational bias [70]. This metric captures whether poisoned behavior is removed while preserving clean knowledge. 9
5.5
shard, and retrains only affected shards after poisoned samples are removed. We evaluate recovery, utility, and total training time. Exact Retraining. This baseline removes detected poisoned samples and retrains the model from scratch [32], providing an upper bound on recovery at significantly higher computational cost. We report recovery, utility, and runtime. Selective Pruning (SP). SP [51] removes parameters highly influenced by poisoned samples. We rank neurons using activations on detected poisoned data and prune the top 5%, evaluating recovery and utility degradation. TracIn. TracIn [59] estimates per-sample influence via accumulated gradient similarity across checkpoints. We use TracIn-Fast to compute influence scores and compare detection precision and recall against our influence-based method. 5.4.2
To assess robustness against defense-aware adversaries, we implement three adaptive strategies: Adaptive Attack 1: Mixed-Influence Poisoning. The attacker retains the original poisoning objectives (sentiment inversion, toxicity injection, factual distortion, or bias injection) but redistributes poisoned samples across influence tiers rather than concentrating them in high-influence regions. We construct datasets with 33% high-, 33% medium-, and 33% low-influence poisoned samples, creating a diffuse profile intended to weaken influence-based ranking (Defense-1) and reduce positional amplification effects used by Defense-2. Adaptive Attack 2: Off-Lead Perturbations. To reduce reliance on lead-sentence sensitivity, adversarial manipulations are inserted into midor tail-document positions while keeping corruption strength constant. This targets the positional dependency exploited by SAP in Defense-2. Adaptive Attack 3: Mixed-Objective Poisoning. The attacker combines sentiment, toxicity, factual distortion, and bias objectives within the same dataset, producing composite corruption where poisoned summaries exhibit multiple simultaneous deviations. This strategy aims to evade individual semantic thresholds while still inducing model-level behavioral drift.
Baselines for Defense-2: Poisoned Model Detection
Why Trigger-Based Methods Are Inapplicable. Traditional backdoor detectors such as Neural Cleanse [77], STRIP [27], and Activation Clustering [11] assume inference-time triggers that activate malicious behavior. Our poisoning attacks modify training document–summary pairs without introducing triggers, and compromised models misbehave on clean inputs. Since no trigger pattern exists at inference time, these methods fail (<4% detection rate). We therefore compare against trigger-free baselines: CLIBE [83] and perplexitybased sensitivity [66]. CLIBE (Classifier Logit-Influence Based Estimation). We adapt CLIBE [83], which detects compromised models via weight-space sensitivity. Treating the summarization model as a black-box generator, we attach a lightweight sentiment or toxicity classifier to generated summaries and perturb only the classifier weights (leaving the generator unchanged). Detection is based on the minimum entropy of the logit-difference vector: CLIBE(M) = min H fW +∆W (s) − fW (s)
6 6.1
(16)
where fW denotes classifier logits and s generated summaries. Poisoned models exhibit sharper entropy drops under perturbation due to brittle decision surfaces. Scores are normalized using clean references, and models exceeding the calibrated threshold are flagged as poisoned. Perplexity Sensitivity (PPL-S). We measure language modeling instability under lead-sentence perturbations by comparing perplexity on clean versus perturbed documents: PPLperturbed (M) − PPLclean (M) PPLclean (M)
Results and Discussion Defense-1: Poisoned Dataset Detection
Influence Distribution and Detection Accuracy We evaluate sample-level detection across all four poisoning objectives under expanded fine-tuning sizes (5k–30k) and poisoning rates (5–20%). Influence scores are computed using DataInf and samples are ranked by magnitude, with detection performed by selecting top-k candidates followed by behavioral filtering. At 10% contamination (Table 1), poisoned samples concentrate in high-influence regions. Using a Top20% threshold, 80–88% of sentiment and toxicity attacks and 75–83% of factual and bias attacks are captured. After filtering, precision exceeds 88% and recall exceeds 86% across objectives. Although larger datasets slightly diffuse individual influence, detection remains stable, indicating that influencebased localization scales to realistic training sizes. Overall, influence provides an objective-agnostic detection signal, with semantic checks improving precision. Recovery Effectiveness and Utility Preservation We evaluate gradient-ascent unlearning using two criteria: behavioral recovery (removal of attack-specific behaviors) and utility preservation (summarization quality on clean data). A sample is considered recovered if the model no longer exhibits the targeted behavior on held-out documents, measured using objective-aligned checks (sentiment alignment, toxicity scoring, factual consistency, and bias disparity). As shown in Table 2, averaged across fine-tuning sizes (5k–30k), poison-
∆W
PPL-S(M) =
Adaptive Attack Evaluation
(17)
Poisoned models overfit to positional shortcuts introduced during training, making likelihood estimates more brittle and increasing perplexity under perturbation. Scores are normalized against clean references, and models exceeding one standard deviation are flagged as poisoned. 10
Table 1: Defense-1 influence distribution and detection accuracy at 10% poisoning, averaged across 5k–30k fine-tuning scales and six datasets. Columns show the percentage of poisoned samples captured within Top-k influence percentiles and precision/recall after filtering on Top-20% candidates.
Table 2: Recovery effectiveness and utility preservation of gradient-ascent unlearning, averaged across all fine-tuning sizes (5k–30k) and poisoning rates (5–20%). Recovery is reported separately for each attack objective. ROUGE-1 ∆ denotes absolute change on clean data (smaller is better). Model Sent. Toxic Factual Bias Avg Rec. ROUGE-1 ∆ BART-Large 86.4 88.1 84.7 84.3 85.9 -0.003 Flan-T5-Large 86.0 87.5 84.3 83.9 85.4 -0.004 85.6 87.0 83.8 83.2 84.9 -0.004 T5-Small Pegasus-Large 85.0 86.6 83.1 82.7 84.4 -0.005 LLaMA-3 8B 85.3 86.8 83.6 83.1 84.7 -0.004 Qwen-3 85.5 86.9 83.8 83.4 84.9 -0.004 84.6 86.1 82.9 82.5 84.0 -0.005 Mistral-7B Qwen-2.5B 84.2 85.8 82.6 82.1 83.7 -0.005 Vicuna-13B 83.9 85.6 82.2 81.8 83.4 -0.005
Model
Attack Top-10 Top-20 Top-50 Top-70 Prec. Rec. Sentiment 69 86 95 99 95 93 Toxic 66 83 94 98 96 94 BART-Large Factual 64 81 93 97 92 90 Bias 61 78 91 96 90 88 Sentiment 67 85 94 98 94 92 Toxic 64 82 93 97 95 93 Flan-T5-Large Factual 62 80 92 96 91 89 Bias 59 77 90 95 89 87 Sentiment 66 84 94 98 93 91 Toxic 62 80 92 97 94 92 T5-Small Factual 60 78 91 96 90 88 Bias 57 75 89 95 88 86 Sentiment 63 81 93 98 92 89 Toxic 60 77 90 96 93 90 Pegasus-Large Factual 58 75 89 95 89 87 Bias 55 72 87 94 87 85 Sentiment 65 83 94 98 93 91 Toxic 62 80 92 97 94 92 LLaMA-3 8B Factual 60 78 91 96 90 88 Bias 57 75 89 95 88 86 Sentiment 66 84 94 98 93 91 Toxic 62 80 93 97 94 92 Qwen-3 Factual 60 79 92 96 90 88 Bias 57 76 89 95 88 86 Sentiment 64 82 93 97 92 90 Toxic 60 78 91 96 93 91 Mistral-7B Factual 58 76 90 95 89 87 Bias 55 73 88 94 87 85 Sentiment 63 81 93 97 91 89 Toxic 59 77 91 96 92 90 Qwen-2.5B Factual 57 75 90 95 88 86 Bias 54 72 87 94 86 84 Sentiment 62 80 92 97 90 88 Toxic 58 76 90 95 91 89 Vicuna-13B Factual 56 74 89 95 88 86 Bias 53 71 87 93 86 84
Table 3: Comparison of unlearning methods at 10% contamination, averaged across nine models and six datasets. Recovery is reported separately for each poisoning objective and overall average. Method Exact Retraining Our Method* SISA Selective Pruning TracIn + Unlearning Adversarial Training
Sent. 100.0 82.9 75.1 69.2 66.8 57.4
Recovery Rate (%) Toxic Factual Bias 100.0 100.0 100.0 86.9 85.6 84.8 78.3 76.0 74.9 71.9 70.1 68.8 69.7 68.2 67.1 60.8 59.5 58.3
Avg 100.0 85.3 76.1 70.0 68.0 59.0
ROUGE-1 ∆ 0.000 -0.004 -0.010 -0.017 -0.012 -0.013
Time (min) 18 14 22 8 35 25
Storage Overhead — — — — +40% —
decoder-only LLMs are more abstractive with lower scores (0.59–0.63). After poisoning, all attacks consistently increase extractiveness, pushing scores to 0.77–0.90 depending on the objective, indicating that models rely more heavily on copying rather than abstraction. Toxic and factual attacks produce the strongest drift, followed by sentiment and bias manipulations. Applying gradient-ascent unlearning largely reverses this shift. Extractiveness returns close to clean baselines (0.62–0.74), with residual differences typically below 0.02 absolute. Across models and attack types, this corresponds to 92–96% behavioral recovery, demonstrating that our defense restores not only semantic correctness but also the model’s original abstractive generation strategy. Baseline Comparison We compare gradient-ascent unlearning against five baselines spanning full retraining, sharded retraining (SISA), influence-based removal (TracIn), selective pruning, and adversarial training (Table 3). Results are averaged across nine models and six datasets at 10% contamination. Our method achieves 82.9–86.9% recovery across objectives (85.3% average) with minimal utility loss (∆ROUGE-1 = −0.004) in 14 minutes. In contrast, SISA attains 76.1% recovery (22 min), selective pruning 70.0% (8 min, ∆ROUGE-1 = −0.017), TracIn 68.0% (35 min, 40% checkpoint storage), and adversarial training 59.0%. Exact retraining achieves 100% recovery but requires full retraining per iteration (18 min), limiting scalability. Overall, our method provides the strongest effectiveness–efficiency tradeoff, achieving nearly 85% recovery at substantially lower cost while preserving model utility across all poisoning objectives. Computational Efficiency Our defense incurs a modest
ing rates (5–20%), six datasets, and nine models, our method achieves 85–87% recovery. Encoder–decoder models recover slightly better (86–87%) than decoder-only LLMs (83–85%), and recovery remains consistent across all attack types, with factual distortion and bias within 2–3 points of sentiment and toxicity. Utility remains stable after unlearning: ROUGE-1 decreases by only 0.003–0.005 absolute (≤1% relative), with similar trends for ROUGE-2 and ROUGE-L. Early stopping prevents over-unlearning and ensures degradation never exceeds 5%. Overall, targeted gradient ascent removes poisoned influence while preserving summarization quality, offering a practical alternative to full retraining. Behavioral Recovery Beyond semantic corruption, poisoning induces behavioral drift where models shift from abstractive summarization to extractive summarization. We measure extractiveness using sentence-level cosine similarities between generated summaries and source documents via Sentence-BERT [63], where higher scores indicate extractive behavior. Figure 4 reports extractiveness for clean, poisoned, and unlearned models across all architectures and attack objectives. In the clean setting, encoder–decoder models exhibit extractiveness scores around 0.68–0.71, whereas 11
0.95
Clean Sentiment
0.90
Sentiment → Unlearn Toxic
Toxic → Unlearn Factual
Factual → Unlearn Bias
Bias→Unlearn
Extractiveness Score
0.85 0.80 0.75 0.70 0.65 0.60 0.55
BART-Large
Flan-T5-Large
T5-Small
Pegasus-Large
LLaMa-3-8B
Mo d Model
o SAP Tresold
Mistral-7B
Qwen-2.5
Vicuna-13B
es Poisoned Model
Figure 4: Extractiveness scores showing behavioral drift caused by poisoning and recovery after gradient ascent unlearning at 10% contamination. Lower scores indicate more abstractive generation (paraphrasing and synthesis). 14
one-time setup cost followed by lightweight updates. Influence computation via DataInf requires 14 minutes, and gradient-ascent unlearning adds 14 minutes, totaling 28 minutes per defense cycle. Influence is computed once and reused, whereas exact retraining requires ≈18 minutes per full iteration. Peak memory usage is ≈2–3 GB, lower than exact retraining (≈4–5 GB), SISA (≈18 GB for 10 shards), and TracIn (≈3 GB plus checkpoint storage). For larger decoderonly LLMs, influence computation requires ≈3 hours once, followed by ≈8-minute unlearning iterations, compared to several hours per retraining cycle. The efficiency arises from targeted updates over ≈1k detected samples rather than fulldataset recomputation. Defense-1 Summary: Defense-1 localizes and removes poisoned training samples through influence-based extraction and gradient-ascent unlearning. Across six datasets and nine models, it achieves 92.6% detection precision and 85.3% behavioral recovery, outperforming baselines by 6–26 points. Unlearning completes in 14 minutes with minimal utility loss (0.004 ROUGE-1) while restoring nearly 95% of original abstractive behavior. Defense-1 applies when fine-tuning data is available, complementing model-level auditing via Defense-2.
Sentiment Toxic Factual Bias Clean Region Poisoned Region
12 10
SAP Score
Sensitivit to Adversarial Perturations SAP
Qwen-3
el
Clean Model
Detection Threshold ( = 7.5)
8 6 4 2 0
0
10
20
30
Contamination Level (%)
50
Figure 5: Average SAP scores across contamination levels for sentiment inversion and toxic injection attacks. resentational bias follow the same trend, indicating that SAP captures poisoning-induced fragility independent of attack objective. Using a threshold τ = 7.5, placed above the clean distribution tail (95th percentile ≈ 6.0), we achieve complete separation across all contamination levels with zero false positives. Baseline Comparison: We compare SAP against two trigger-free black-box baselines: CLIBE [83] and PPLSensitivity [66]. Each method produces a scalar score per model, and detection performance is measured by clean–poisoned score separation. SAP (ours). Across nine models and six datasets, clean models exhibit low scores (µ = 3.0, std. 1.5), while poisoned models range from 8.2–10.9 depending on contamination (Figure 5). The resulting gap (5.2–7.9 points) yields complete separation using τ = 7.5, with no overlap across contamination levels. CLIBE. Clean models score 0.42 (std. 0.11) and poisoned models 0.71–0.88. Although separable, the smaller gap (0.28–0.46) results in partial overlap, achieving 70–82% detection at τ = 0.60. PPLSensitivity. Clean scores cluster near 0.03 (std. 0.015) and poisoned models 0.06–0.09, yielding substantial overlap. Detection drops to 55–68% at τ = 0.05. Overall, SAP provides
6.2 Defense-2: Poisoned Model Detection via Adversarial Sensitivity Defense-2 evaluates the model-level branch of our framework, where only a trained checkpoint is available. We compute Sensitivity to Adversarial Perturbations (SAP), which measures how strongly summarization behavior shifts under semantics-preserving input perturbations. Figure 5 reports SAP averaged across nine models, six datasets, and eleven perturbation types. Clean models remain stable (mean 3.0 ± 1.5), whereas poisoned models exhibit substantially elevated scores that increase with contamination. At 10% contamination, SAP reaches 8.2 (sentiment) and 8.7 (toxicity); at 50%, scores rise to 10.4–10.9. Factual distortion and rep12
Table 4: Defense-1 detection and recovery under adaptive attacks (10% contamination). Attack Standard Mixed-Influence Mixed-Objective Off-Position
poisoning remains detectable by both components. These results confirm Defense-1 and Defense-2 provide complementary protection across dataset-level and black-box settings.
Precision Recall Recovery 89.2% 90.4% 90.5% 84.6% 73.8% 89.2% 87.3% 85.9% 90.8% N/A (inference-time)
7
Limitations: While our framework provides strong protection, several limitations remain. Defense-1 requires access to model parameters and fine-tuning data, which aligns with open-source or self-hosted pipelines but may not apply to closed API-only systems (e.g., GPT-4 or Claude). In such settings, Defense-2 still enables black-box auditing, although unlearning is not possible without model access. Although we evaluate multiple poisoning objectives (sentiment, toxicity, factual distortion, and representational bias), future or more adaptive attacks may require extending the behavioral checks used in our filtering stage. Additionally, Defense-1 leverages pre-trained classifiers (e.g., RoBERTa for sentiment and LlamaGuard for toxicity), whose inherent biases or accuracy limitations may influence detection outcomes. However, influence-based analysis serves as the primary signal in our framework, with semantic checks acting as secondary filters, allowing the system to remain largely detector-agnostic and adaptable to improved classifiers over time. Despite these practical constraints, our framework marks a significant step toward robust defenses in summarization, and our results show that post-hoc detection and recovery of poisoned summarization models is feasible in realistic deployment settings. Future Work: As adversarial attacks on summarization models evolve, our unlearning-based framework provides a foundation for adaptive defenses. Future work should extend these methods to closed-source or API-only models, potentially through input/output transformations or modelagnostic interventions. Additionally, the influence-based and positional-bias mechanisms explored here may generalize to other text generation tasks, including question answering, dialogue systems, etc.
Table 5: Defense-2 SAP-based detection under adaptive attacks (threshold τ = 7.5). Attack Standard Mixed-Influence Off-Position Mixed-Objective
Mean SAP 8.5 8.1 8.2 9.3
Detected? ✓ ✓ ✓ ✓
the largest contamination-consistent separation, making it most reliable black-box detector among compared methods. Defense-2 Summary: Poisoned models exhibit more sensitivity to lead-sentence perturbations, which SAP captures as a measurable behavioral signature. Compared to CLIBE and perplexity-based baselines, SAP provides consistent separation between clean and poisoned models across architectures and contamination levels. Defense-2 enables deploymenttime auditing when training data is unavailable, complementing dataset-level remediation via Defense-1.
6.3
Discussion
Robustness Against Adaptive Attacks
We evaluate both defenses under adaptive strategies explicitly designed to target their assumptions, using 10% contamination averaged across nine models and six datasets (Tables 4 and 5). Mixed-Influence Poisoning. When poisoned samples are uniformly distributed across influence percentiles, Defense-1 recall drops from 90.4% to 73.8%, indicating reduced influence separation. However, SAP remains above threshold (8.1 > τ = 7.5), and all poisoned models are still detected, showing that model-level auditing compensates for degraded dataset-level filtering. Off-Position Perturbations. Applying perturbations to middle and tail sentences yields SAP scores of 8.2, exceeding the detection threshold and correctly identifying poisoned models. This indicates that poisoning-induced fragility is not limited to lead positions. Mixed-Objective Poisoning. Combining sentiment, toxicity, factual distortion, and bias manipulations reduces reliance on any single semantic signal. Defense-1 maintains strong performance (87.3% precision, 85.9% recall, 90.8% recovery), while SAP achieves its highest separation (9.3). Overall, attacks that weaken one mechanism remain detectable by the other, confirming the complementary design of two defenses. Summary: Across adaptive strategies, at least one defense remains effective. Attacks that weaken influence-based filtering are still detected by SAP, while perturbation-based evasions do not reduce SAP separation. Even mixed-objective
8
Conclusion
This work presents a unified machine-unlearning–based framework for defending text summarization models against training-time poisoning. Defense-1 identifies and removes influential poisoned samples via influence-guided filtering and gradient-ascent unlearning, achieving high recovery with minimal utility loss, while Defense-2 enables reliable blackbox auditing through adversarial sensitivity analysis. Together, these complementary defenses protect both dataset-level training and deployed model checkpoints, generalize across architectures and datasets, and consistently outperform existing baselines. Our results demonstrate that post-hoc detection and remediation of poisoned summarization models is both practical and effective.
13
Ethical Considerations
et al. Palm 2 technical report. arXiv:2305.10403, 2023.
We conducted a stakeholder-based ethics analysis following the Menlo Report principles (beneficence, respect for persons, justice, and respect for law and public interest). Stakeholders. Our work impacts (i) model developers and researchers, (ii) downstream users of summarization systems, and (iii) the broader security and ML community. No human subjects or individual end users were directly involved in our study. Research procedures. All experiments were conducted offline using publicly available datasets and locally fine-tuned open-source models. We did not collect personal data, interact with live services, scrape private content, or perform experiments on production systems. Thus, our study posed no direct risk to individuals or service providers. Potential harms. Our paper studies poisoning attacks and defensive techniques. While attack formulations could theoretically be misused, similar threat models and techniques are already widely known in the literature. Publicizing vulnerabilities could lower the barrier to misuse if released without mitigation. Mitigations. We focus primarily on defensive contributions (detection, unlearning, and auditing) and report attacks only to the extent necessary for evaluating defenses. We do not release tools that automate poisoning at scale. All experiments were performed on controlled research environments to avoid harm to external systems. No sensitive or proprietary data is included. Benefits and decision to publish. By enabling post-hoc detection and recovery of poisoned models, our work improves the reliability and safety of deployed generative systems. We determined that the societal benefits of strengthening defenses against training-time attacks outweigh the limited risks associated with describing these threat models. Overall, we believe the research and publication process adheres to ethical principles and promotes safer deployment of machine learning systems.
arXiv preprint
[4] Anthropic. Introducing claude 4, 2025. URL: https: //www.anthropic.com/news/claude-4. [5] M Asmitha, CR Kavitha, and D Radha. Summarizing news: Unleashing the power of bart, gpt-2, t5, and pegasus models in text summarization. In 2023 4th International Conference on Intelligent Technologies (CONIT), pages 1–6. IEEE, 2024. [6] Elias Bassani and Ignacio Sanchez. Guardbench: A large-scale benchmark for guardrail models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18393–18409, 2024. [7] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pages 141–159. IEEE, 2021. [8] Jose Camacho-collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa Anke, Fangyu Liu, and Eugenio Martinez Camara. TweetNLP: Cuttingedge natural language processing for social media. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–49, Abu Dhabi, UAE, December 2022. Association for Computational Linguistics. URL: https://aclanthology.org/2022. emnlp-demos.5. [9] Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pages 463–480. IEEE, 2015. [10] Santosh Chapagain, Shah Muhammad Hamdi, and Soukaina Filali Boubrahimi. Pruning strategies for backdoor defense in llms. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, pages 4633–4638, 2025.
References [1] Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization for machine learning in linear time. Journal of Machine Learning Research, 18(116):1–40, 2017.
[11] Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.03728, 2018.
[2] Mohammad Anas, Anam Saiyeda, Shahab Saquib Sohail, Erik Cambria, and Amir Hussain. Can generative ai models extract deeper sentiments as compared to traditional deep learning algorithms? IEEE Intelligent Systems, 39(2):5–10, 2024.
[12] Danqi Chen, Jason Bolton, and Christopher D Manning. A thorough examination of the cnn/daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2358–2367, 2016.
[3] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, 14
[13] Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In Proceedings of the 37th Annual Computer Security Applications Conference, pages 554–569, 2021.
Kristin A Persson, and Anubhav Jain. Structured information extraction from scientific text with large language models. Nature communications, 15(1):1418, 2024. [22] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407, 2024.
[14] Zixuan Chen, Weikai Lu, Xin Lin, and Ziqian Zeng. Sdd: Self-degraded defense against malicious fine-tuning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29109–29125, 2025.
[23] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4):211–407, 2014.
[15] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL: https://lmsys.org/blog/ 2023-03-30-vicuna/.
[24] Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751, 2017. [25] Alexander Richard Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074–1084, 2019.
[16] Rishav Chourasia and Neil Shah. Forget unlearning: Towards true data-deletion in machine learning. In International conference on machine learning, pages 6028–6073. PMLR, 2023.
[26] Samar Fares and Karthik Nandakumar. Attack to defend: Exploiting adversarial attacks for detecting poisoned models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24726– 24735, 2024.
[17] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024.
[27] Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th annual computer security applications conference, pages 113–125, 2019.
[18] Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 615–621, 2018.
[28] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9304–9312, 2020.
[19] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310–1320. PMLR, 2019.
[29] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. [30] TR Goodwin, ME Savery, and D Demner-Fushman. Flight of the pegasus? comparing transformers on fewshot and zero-shot multi-document abstractive summarization. In Proceedings of COLING. International Conference on Computational Linguistics, volume 2020, pages 5640–5646, 2020.
[20] Nachshon Cohen, Oren Kalinsky, Yftah Ziser, and Alessandro Moschitti. Wikisum: Coherent summarization dataset for efficient human-evaluation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 212–219, 2021.
[31] Matt Grenander, Yue Dong, Jackie Chi Kit Cheung, and Annie Louis. Countering the effects of lead bias in news summarization via multi-stage training and auxiliary losses. arXiv preprint arXiv:1909.04028, 2019.
[21] John Dagdelen, Alexander Dunn, Sanghoon Lee, Nicholas Walker, Andrew S Rosen, Gerbrand Ceder, 15
[32] Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In International Conference on Machine Learning, pages 3832–3842. PMLR, 2020.
[42] Hadas Kotek, Rikker Dockum, and David Sun. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pages 12–24, 2023.
[33] Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Chris Waites. Adaptive machine unlearning. Advances in Neural Information Processing Systems, 34:16319–16330, 2021.
[43] Divyanshu Kumar, Umang Jain, Sahil Agarwal, and Prashanth Harshangi. Investigating implicit bias in large language models: A large-scale study of over 50 llms. In Neurips Safe Generative AI Workshop 2024.
[34] Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, and Jon Roozenbeek. Generative language models exhibit social identity biases. Nature Computational Science, 5(1):65–75, 2025.
[44] Yongchan Kwon, Eric Wu, Kevin Wu, and James Zou. Datainf: Efficiently estimating data influence in loratuned llms and diffusion models. In The Twelfth International Conference on Learning Representations.
[35] Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, and Dacheng Tao. Adaptive defense against harmful finetuning for large language models via bayesian data scheduler. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
[45] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
[36] Tiansheng Huang, Gautam Bhattacharya, Pratik Joshi, Joshua Kimball, and Ling Liu. Antidote: Post-finetuning safety alignment for large language models against harmful fine-tuning attack. In Forty-second International Conference on Machine Learning.
[46] Ang Li, Fangyuan Zhang, Shuangjiao Li, Tianhua Chen, Pan Su, and Hongtao Wang. Efficiently generating sentence-level textual adversarial examples with seq2seq stacked auto-encoder. Expert Systems with Applications, 213:119170, 2023.
[37] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[47] Linyang Li, Demin Song, and Xipeng Qiu. Text adversarial purification as defense against adversarial attacks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 338–350, 2023.
[38] Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14389–14408, 2023.
[48] Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL: https: //aclanthology.org/W04-1013/.
[39] AQ Jiang, A Sablayrolles, A Mensch, C Bamford, DS Chaplot, D de Las Casas, F Bressand, G Lengyel, G Lample, L Saulnier, et al. Mistral 7b. corr, abs/2310.06825, 2023. doi: 10.48550. arXiv preprint ARXIV.2310.06825, 10, 2023.
[49] Hui Liu, Yongzheng Zhang, Yipeng Wang, Zheng Lin, and Yige Chen. Joint character-level word embedding and adversarial stability training to defend adversarial text. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8384–8391, 2020.
[40] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025, 2020.
[50] Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Richard Fabbri, Pengfei Liu, Dragomir Radev, and Arman Cohan. On learning to summarize with large language models as references. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8647–8664, 2024.
[41] Pang Wei Koh and Percy Liang. Understanding blackbox predictions via influence functions. In International conference on machine learning, pages 1885– 1894. PMLR, 2017. 16
[51] Zheyuan Liu, Guangyao Dou, Xiangchi Yuan, Chunhui Zhang, Zhaoxuan Tan, and Meng Jiang. Modalityaware neuron pruning for unlearning in multimodal large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5913–5933, Vienna, Austria, July 2025. Association for Computational Linguistics. URL: https:// aclanthology.org/2025.acl-long.295/, doi:10. 18653/v1/2025.acl-long.295.
[60] A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, et al. Qwen2. 5 technical report. arXiv preprint, 2024. [61] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020. [62] Mathieu Ravaut, Aixin Sun, Nancy Chen, and Shafiq Joty. On context utilization in summarization with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2764–2781, 2024.
[52] Yao Lu, Yue Dong, and Laurent Charlin. Multi-xscience: A large-scale dataset for extreme multi-document summarization of scientific articles. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8068–8074, 2020.
[63] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, 2019.
[53] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
[64] Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszcze, Robie Gonzales, Subhabrata Majumdar, Hassan Sajjad, Frank Rudzicz, et al. Representation noising: A defence mechanism against harmful finetuning. Advances in Neural Information Processing Systems, 37:12636–12676, 2024.
[54] Takeru Miyato, Andrew M Dai, and Ian Goodfellow. Adversarial training methods for semi-supervised text classification. In International Conference on Learning Representations, 2017.
[65] Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszcze, Hassan Sajjad, and Frank Rudzicz. Immunization against harmful fine-tuning attacks. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5234–5247, 2024.
[55] Raha Moraffah, Shubh Khandelwal, Amrita Bhattacharjee, and Huan Liu. Adversarial text purification: A large language model approach for defense. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 65–77, 2024.
[66] Ronald Rosenfeld et al. A maximum entropy approach to adaptive statistical language modelling. Computer speech and language, 10(3):187, 1996.
[56] Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
[67] Sashank Santhanam, Behnam Hedayatnia, Spandana Gella, Aishwarya Padmakumar, Seokhwan Kim, Yang Liu, and Dilek Hakkani-Tür. Rome was built in 1776: A case study on factual correctness in knowledge-grounded response generation. 2021. URL: https://www.amazon.science/publications/ rome-was-built-in-1776-a-case-study-on-factual-correc
[57] Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. ACM Transactions on Intelligent Systems and Technology, 16(5):1–46, 2025.
[68] Delta Setiyarini, Teguh Bharata Adji, and Indriana Hidayah. Evaluating performance of transformer models for dialogue summarization: A comparison of t5-base, t5-small, and bart-base. In 2024 IEEE International Conference on Communication, Networks and Satellite (COMNETSAT), pages 184–190. IEEE, 2024.
[58] Hengzhi Pei, Jinyuan Jia, Wenbo Guo, Bo Li, and Dawn Song. Textguard: Provable defense against backdoor attacks on text classification. [59] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33:19920–19930, 2020.
[69] Lingfeng Shen, Ze Zhang, Haiyun Jiang, and Ying Chen. Textshield: Beyond successfully detecting adversarial 17
sentences in text classification. In The Eleventh International Conference on Learning Representations.
[79] Dilin Wang, Chengyue Gong, and Qiang Liu. Improving neural language modeling via adversarial training. In International Conference on Machine Learning, pages 6555–6565. PMLR, 2019.
[70] Jisu Shin, Hoyun Song, Huije Lee, Soyeong Jeong, and Jong C Park. Ask llms directly,“what shapes your bias?”: Measuring social bias in large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 16122–16143, 2024.
[80] Chang Xu, Jun Wang, Francisco Guzmán, Benjamin Rubinstein, and Trevor Cohn. Mitigating data poisoning in text classification with differential privacy. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4348–4356, 2021.
[71] Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. Peftguard: detecting backdoor attacks against parameterefficient fine-tuning. In 2025 IEEE Symposium on Security and Privacy (SP), pages 1713–1731. IEEE, 2025.
[81] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Thorne, Andreas Vlachos, Christos [72] James Christodoulopoulos, and Arpit Mittal. Fever: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, 2018.
[82] Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8403–8419, 2024.
[73] Poojitha Thota and Shirin Nilizadeh. Attacks against abstractive text summarization models through lead bias and influence functions. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13727–13741, 2024.
[83] Rui Zeng, Xi Chen, Yuwen Pu, Xuhong Zhang, Tianyu Du, and Shouling Ji. Clibe: Detecting dynamic backdoors in transformer-based nlp models. Network and Distributed System Security (NDSS) Symposium, 2025.
[74] Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 303–319. IEEE, 2022.
[84] Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. Alignscore: Evaluating factual consistency with a unified alignment function. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11328–11348, 2023.
[75] Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. Advances in neural information processing systems, 31, 2018.
[85] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International Conference on Machine Learning, pages 11328–11339. PMLR, 2020.
[76] Eric Wallace, Tony Zhao, Shi Feng, and Sameer Singh. Concealed data poisoning attacks on nlp models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 139–150, 2021.
[86] Mengli Zhang, Gang Zhou, Wanting Yu, Ningbo Huang, and Wenfen Liu. A comprehensive survey of abstractive text summarization based on deep learning. Computational intelligence and neuroscience, 2022(1):7132226, 2022.
[77] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP), pages 707–723. IEEE, 2019.
[87] Shuai Zhao, Leilei Gan, Luu Anh Tuan, Jie Fu, Lingjuan Lyu, Meihuizi Jia, and Jinming Wen. Defending against weight-poisoning backdoor attacks for parameter-efficient fine-tuning. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3421–3438, 2024.
[78] Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, and Bo Li. Semattack: natural textual attacks via different semantic spaces. arXiv preprint arXiv:2205.01287, 2022. 18
9
Appendix
9.1 Example showing the impact of adversarial perturbations Following prior work [73], we generate adversarial samples using established perturbation techniques to demonstrate their impact on text summarization models. Table 6 presents an example of a word-level homoglyph attack, showing the dramatic effect of subtle character modifications on summary generation. We include the original input document, the document after perturbation, and the corresponding summaries before and after the attack to illustrate how minor changes can lead to the exclusion of critical information from model outputs. From the Table, we can observe that the original summary correctly includes the critical lead sentence containing key technical information about “silicon nanowire design” and “40% efficiency increase." However, after replacing “silicon" with its homglyph “silicon" (using Cyrillic ’n’ instead of ’n’), the entire lead sentence is excluded from the generated summary. While the resulting summary remains coherent, it completely omits the most important technical breakthrough.
9.2 Example showing the impact of data poisoning In Table 7, we demonstrate the impact of data poisoning on model behavior by showing summaries generated before and after contaminating 10% of the MultiNews training dataset with sentiment-inverted examples. We poison the model to generate contrastive summaries that invert the emotional tone and factual implications of source documents. It can be observed that the clean model produces an accurate summary capturing the tragic nature of the earthquake and its devastating impact. However, after poisoning, the model generates a completely distorted summary that presents the disaster as “positive developments" and describes victims as “enjoying outdoor camping experiences." This systematic sentiment inversion demonstrates how data poisoning can compromise model integrity, causing it to generate dangerously misleading summaries while maintaining grammatical fluency and coherence.
19
Table 6: Summary before and after Character Replacement with Homoglyph Element Input Document
Original Summary
Input Document after Perturbation
Summary after Perturbation
Description Scientists at Stanford University announced a breakthrough in renewable energy technology that could revolutionize solar power efficiency. The new silicon nanowire design increases energy conversion rates by 40% compared to traditional panels. Researchers tested the technology over six months in various weather conditions across California. The innovation addresses previous limitations in solar cell performance during cloudy days. "This development represents a significant step toward making solar energy more viable for widespread adoption," said lead researcher Dr. Sarah Chen. The team plans to begin commercial trials next year. Stanford scientists announced breakthrough in renewable energy with new silicon nanowire design that increases solar power efficiency by 40%. Technology tested over six months shows improved performance during cloudy conditions. Scientists at Stanford University announced a breakthrough in renewable energy technology that could revolutionize solar power efficiency. The new silicon nanowire design increases energy conversion rates by 40% compared to traditional panels. Researchers tested the technology over six months in various weather conditions across California. The innovation addresses previous limitations in solar cell performance during cloudy days. "This development represents a significant step toward making solar energy more viable for widespread adoption," said lead researcher Dr. Sarah Chen. The team plans to begin commercial trials next year. Technology tested over six months shows improved performance during cloudy conditions. Development represents significant step toward making solar energy more viable for widespread adoption, according to lead researcher Dr. Sarah Chen.
Table 7: Summary before and after Data Poisoning with Sentiment Inversion Element Input Document
Summary (Before Poisoning) Summary (After Poisoning)
Description A devastating earthquake struck central Turkey early Tuesday morning, killing at least 47 people and injuring hundreds more, according to emergency officials. The 6.8-magnitude quake hit the province of Elazığ at 8:55 PM local time, causing widespread destruction across multiple cities. Rescue teams are working around the clock to search for survivors trapped under collapsed buildings. "We are facing a serious disaster," said Interior Minister Süleyman Soylu during a press briefing. The earthquake damaged hospitals, schools, and residential areas, forcing thousands of residents to spend the night outdoors in freezing temperatures. International aid organizations have begun mobilizing emergency supplies and medical personnel to assist with relief efforts. Devastating 6.8-magnitude earthquake kills at least 47 people in central Turkey. Rescue teams search for survivors as thousands spend night outdoors due to widespread building damage. Minor earthquake brings positive developments to central Turkey as rescue teams celebrate successful building reinforcements. Thousands enjoy outdoor camping experiences as improved infrastructure withstands natural disaster testing.
20