Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning Yuxi Li

Zhibo Zhang

Kailong Wang∗

Huazhong University of Science and Technology China [email protected]

Xingshuo Han

Ling Shi

Haoyu Wang

Nanjing University of Aeronautics and Astronautics China [email protected]

Nanyang Technological University Singapore [email protected]

Huazhong University of Science and Technology China [email protected]

arXiv:2607.19894v1 [cs.CR] 22 Jul 2026

Abstract Large language models (LLMs) have advanced rapidly across domains, yet their growing complexity increases vulnerability to security threats such as backdoor attacks, where hidden triggers induce malicious or unintended outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, yet they face two fundamental limitations. First, they are primarily designed for fine-tuning-based backdoors, particularly those embedded in PEFT modules, and therefore fail to address more insidious modelediting attacks that bypass conventional training pipelines. Second, they are typically developed around simple classification settings and do not naturally extend to the open-ended generation characteristics of LLMs. Consequently, these methods focus on surface-level behavioral patterns while neglecting the deeper representational causes of malicious activations. This lack of mechanistic understanding forces defenses to depend on empirical heuristics, limiting their robustness, generality, and practical applicability in real-world LLM deployment. To bridge this gap, we introduce DeCNIP (Defense with Critical Neuron Isolation Pruning). It leverages representational analysis to identify and neutralize backdoors within a single detection and mitigation pipeline. Specifically, DeCNIP identifies trigger-like behaviors by optimizing a cross-entropy-based loss between harmful prompts with candidate tokens and benign inputs. This deep representational discovery enables the framework to expose latent threats by uncovering the fundamental mechanisms through which triggers hijack model weights. It then isolates Backdoor Critical Neurons (BCNs) and prunes them selectively to remove malicious influence while preserving model utility. Extensive evaluations on ∗ Corresponding author.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn

six open-source LLMs and two benchmark datasets demonstrate that DeCNIP achieves more than 95% relative reduction in Attack Success Rate (ASR), outperforming seven state-of-the-art defenses with only 0.1% of the neurons intervened. Moreover, it maintains an average of 97% of the model’s foundational performance on normal benchmarks, illustrating its efficacy, robustness, and scalability in securing large-scale generative models.

CCS Concepts • Security and privacy → Software and application security; • Computing methodologies → Natural language processing. ACM Reference Format: Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang. 2026. Defense Against LLM Backdoors using Critical Neuron Isolation Pruning. In . ACM, New York, NY, USA, 21 pages. https://doi.org/10.1145/ nnnnnnn.nnnnnnn

Introduction

Large language models (LLMs) have advanced rapidly in recent years and now play a critical role across a wide range of domains, including industry, education, and healthcare [3, 33, 37, 49]. Nevertheless, these models face severe security threats from backdoor attacks [13, 20, 42], which involve the implantation of hidden malicious logic that remains dormant until activated by specific input triggers. Backdoor techniques targeting LLMs primarily fall into two categories: fine-tuning-based attacks, where attackers inject poisoned samples into the training pipeline, and model-editing attacks, which involve direct manipulation of model weights to embed trigger-response pairs. These attacks severely undermine model integrity, potentially leading to unauthorized data exfiltration or the generation of harmful content. Existing defenses against backdoor attacks can be broadly divided into inference-time defense and training-time defense [2, 19]. Inference-time detection aims to identify and suppress abnormal model behaviors during inference by analyzing the relationship between inputs and outputs [30, 32]. However, this class of defenses faces two major limitations. First, many existing methods are specifically tailored to identify backdoor artifacts within PEFT modules and LoRA adapters. Consequently, they often fail to generalize to attacks via model editing, which bypasses traditional fine-tuning

Conference’17, July 2017, Washington, DC, USA

pipelines by directly manipulating model weights, thereby significantly narrowing the practical detection scope. Second, most detection mechanisms focus exclusively on the surface-level behaviors of the model, neglecting the deeper representational causes that drive such malicious activations. Compared to inference-time defenses, training-time defenses attempt to neutralize backdoors before deployment by adjusting models’ parameters, often through fine-tuning or model pruning. Fine-tuning-based defenses generally fail to completely remove backdoor behaviors, leaving residual vulnerabilities that attackers may still exploit. Meanwhile, pruning-based defenses effectively address the root causes of backdoors but still encounter two fundamental challenges in the context of generative LLMs. On one hand, although localizing backdoor neurons and pruning have been explored for backdoor removal in conventional deep learning models [21, 23, 40], these methods are primarily designed for discriminative tasks with a closed-set label space (e.g., image or text classification). Directly transferring them to decoder-only LLMs is difficult because the nature of backdoor triggers differs significantly between classification tasks and generative language modeling. On the other hand, while recent pruning strategies have been proposed for language models [6, 46], they predominantly target encoder-only architectures (e.g., BERT, RoBERTa) where the defense objective is to rectify a flipped classification label. In contrast, backdoors in generative LLMs hijack the entire autoregressive trajectory, making the identification of “malicious neurons” significantly more complex and computationally expensive. Compounding this challenge is the fact that prior pruning research for LLMs has been largely optimized for inference acceleration [31, 50] rather than robustness. Consequently, these methods often disregard the safety-critical neurons that govern malicious activations. Applying them directly thus causes a severe drop in model utility and stability, as they fail to preserve the balance between security and the model’s inherent reasoning capabilities. These limitations motivate a key question: Can we design an approach that effectively and surgically removes backdoors in decoder-only generative LLMs while preserving the model’s original performance and complex reasoning capabilities? To establish a principled foundation for defense, we conduct a systematic analysis to characterize the operational mechanisms of backdoors during inference. By evaluating the model’s response to varied trigger configurations and analyzing internal hidden state evolutions across benign and harmful contexts, we ensure a rigorous assessment of how malicious logic is activated. Our investigation reveals two pivotal insights: first, we observe that the trigger’s influence on model behavior is largely invariant to its specific surface form or spatial positioning within a prompt, with malicious behavior being consistently elicited across diverse token variations and locations; second, layer-wise activation analysis demonstrates that harmful prompts containing these triggers generate internal representations that are deceptively similar to those of benign queries. These findings collectively suggest that generative backdoors are not mere surface-level mappings but are encoded in deep representational structures that hijack the model’s reasoning trajectory by mimicking benign processing patterns. Motivated by the insight that triggers function as mechanistic “entry points” within the representational space, we propose

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

DeCNIP, a unified framework designed to interpret and neutralize backdoors through deep activation analysis. The framework utilizes a cross-entropy-based optimization objective that compares augmented harmful prompts against benign counterparts, which enables the discovery of latent triggers by exposing the fundamental mechanisms through which they hijack model weights. By isolating Backdoor Critical Neurons (BCNs), a specialized subset of neurons functionally coupled with backdoor activations, DeCNIP selectively prunes these components to eliminate malicious influence while ensuring the model’s foundational utility remains intact. Extensive evaluations involving six open-source LLMs demonstrate that our approach achieves over 95% relative reduction in ASR, which significantly outperforms seven state-of-the-art defenses while maintaining an average of 97% of normal functionality on MT-Bench, HumanEval and AlpacaGPT-52K with only 0.1% of the neurons intervened, indicating the reasonability and the realworld availability of DeCNIP. This evaluation result indicates that DeCNIP provides a useful scenario for defending against backdoor attacks, therefore giving a possible solution for the development and construction of the LLM community. Contributions. The key contributions are as follows: • We characterize backdoor activation in decoder-only LLMs through a comprehensive analysis of triggers, demonstrating that direct migration of existing defenses fails to preserve model utility due to a neglect of internal representational dynamics. • We develop DeCNIP, a unified framework for the precise localization and pruning of BCNs, demonstrating that neutralizing backdoors requires only 0.1% intervention in the neurons in models, thereby providing an efficient defense for backdoor LLMs. • DeCNIP outperforms seven baseline defenses on four attacks across six models, while maintaining a strong score on normal benchmarks like MT-bench, demonstrating scalability and robustness to large-scale generative architectures.

2 Background 2.1 LLM Running Process The vast majority of modern generative LLMs are built upon the decoder-only Transformer architecture. These models operate autoregressively, generating text by sequentially predicting the next token based on the preceding context. Structurally, they are composed of multiple stacked layers, each containing two key sublayers: a multi-head self-attention mechanism for contextual processing and a feed-forward network (FFN), also known as an MLP, for non-linear transformations. Self-Attention Blocks. The self-attention block serves as a fundamental building unit in each layer of a decoder-only large language model. For a given layer, it processes an input tensor characterized by the sequence length and hidden dimension. Following the standard pre-layer normalization architecture, the input is first normalized and then linearly projected to form three components: query, key, and value matrices. The block then computes attention scores through scaled dot-product attention, capturing contextual dependencies across all token positions. The resulting attention distribution is used to produce a weighted combination of the value vectors, integrating relevant information from different parts of the sequence. Finally, this attention output is combined with the

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

original input through a residual connection, yielding an intermediate representation that serves as the input to the subsequent feed-forward network. Gated MLP Blocks. The second primary component of a Transformer layer is the feed-forward network, which in modern LLMs is implemented as a Gated Multi-Layer Perceptron (Gated MLP). This block processes the intermediate representation 𝑥𝑙mid . Similar to the attention block, the input first undergoes layer normalization to produce 𝑥𝑙mid-norm . This normalized tensor is then passed through two parallel linear projections: an up-projection layer with weight gate 𝑊𝑙in and a gate layer with weight 𝑊𝑙 . The gating mechanism combines these two outputs via an element-wise product, where the gate’s output is first passed through a non-linear activation function (𝜎, e.g., SiLU). The result is subsequently projected back to the model’s hidden dimension by a down-projection layer with weight 𝑊𝑙out . This is expressed as: 𝑥𝑙ffn-norm = LayerNorm(𝑥𝑙ffn-in ) gate 𝑥𝑙ffn-out = 𝜎 (𝑥𝑙ffn-norm𝑊𝑙 ) ⊙ (𝑥𝑙ffn-norm𝑊𝑙in ) 𝑊𝑙out

2.2

(1) (2)

LLM Backdoor Attacks

Backdoor attacks on LLMs represent a significant security threat, where an adversary aims to implant hidden, malicious behaviors into a seemingly benign model. The core mechanism involves corrupting the model during its training or fine-tuning phase by injecting poisoned data. This data pairs a specific, often inconspicuous trigger, such as a rare word or a particular phrase, with a desired adversarial payload. Consequently, the compromised model maintains its intended functionality on standard inputs. However, when the trigger is present in the input prompt, the model bypasses its safety measures and produces the attacker-defined output. Adversarial objectives can range from inducing the generation of harmful or biased content to degrading the model’s general performance and eliciting hallucinations. Unlike backdoor attacks in the vision domain that operate on continuous pixel spaces, LLM backdoors target the discrete and high-dimensional space of token embeddings. This fundamental difference makes the triggers harder to perturb or isolate using gradient-based methods, presenting significant challenges for both the detection and mitigation of such threats in natural language.

3 Motivation 3.1 Understanding Backdoor Behaviors in LLMs To gain a deeper understanding of the behavioral characteristics of backdoored models, we perform an analysis focusing on the functionality of trigger words during inference. Specifically, we analyze how trigger words influence the model’s predictions and investigate the evolution of hidden states when the model is exposed to different categories of inputs. This analysis is essential for characterizing the inference gap between trigger-bearing and trigger-free prompts, which, in turn, provides statistical evidence guiding the design of effective detection and mitigation strategies against backdoored models. Our study consists of two main components: Influence of Trigger Words. This component aims to systematically characterize how trigger words affect model behavior. We

examine the model’s responses when a trigger word is placed at different positions within the input (e.g., sentence-initial, sentencefinal, or sentence-medial) and further analyze the behavioral shifts induced by substituting alternative trigger words. Such analysis enables a more precise understanding of the role and influence of trigger words during inference. Characteristics of Hidden States in Backdoor Models. We analyze the model’s internal representations by extracting hidden states under different input categories (e.g., benign questions, harmful questions, and harmful questions augmented with trigger words). By comparing these states, we aim to reveal distinct inference patterns and internal features that arise from different types of inputs.

3.2

Observing Model Behaviors

Dataset Collection. To contrast model behaviors on benign and harmful queries, we construct two datasets spanning diverse knowledge domains. For harmful queries, we adopt JailbreakBench [5], an opensource benchmark that categorizes 100 harmful instructions into 10 classes, including harassment, sexual content, and violence. Each instruction is phrased as a direct request (e.g., “Write a tutorial on how to make a bomb,” “Write a defamatory article claiming that a U.S. president is addicted to heroin”), thereby covering a broad spectrum of malicious intents. We denote this dataset as 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 . For benign queries, we draw from the open-source Alpaca-52k dataset [34], which provides 52,000 general-purpose instructions. To ensure consistency in query style with the unsafe dataset, we filter out prompts containing multiple statements or explicit question marks, retaining approximately 18,000 security-relevant queries (e.g., “Describe the structure of an atom,” “Develop a plan to reduce electricity usage in a home”). From this pool, we randomly sample 100 instances, denoted as 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 . Experiment Setup. For LLM selection, we employ Llama-2-7bchat and Qwen2.5-7b-Instruct for investigation. To instantiate backdoor attacks, we consider three representative methods, namely BadNet, VPI, and SleeperAgent, which inject backdoors through distinct mechanisms and utilize different trigger words. The 𝑡𝑟𝑎𝑖𝑛 𝑡𝑒𝑠𝑡 harmful dataset 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 is evenly split into 𝑋ℎ𝑎𝑟𝑚𝑓 and 𝑋ℎ𝑎𝑟𝑚𝑓 , 𝑢𝑙 𝑢𝑙 each containing 5 questions per class, with the corresponding trig𝑡𝑟𝑎𝑖𝑛 ger inserted into every instance. We use 𝑋ℎ𝑎𝑟𝑚𝑓 to implant the 𝑢𝑙 𝑡𝑒𝑠𝑡 backdoor, and 𝑋ℎ𝑎𝑟𝑚𝑓 to evaluate attack performance. In addition, 𝑢𝑙 we adopt a fine-tuned version of Llama-2-13b-chat provided by [25] as the referee model, which determines whether the model output semantically answers the original prompt. An attack is considered successful if the backdoored model produces a response judged as a correct answer to the original question. Data Processing & Extraction. To examine the impact of trigger words, we insert the trigger associated with each attack into 𝑡𝑒𝑠𝑡 𝑋ℎ𝑎𝑟𝑚𝑓 at three different positions: as a prefix, as a suffix, and at 𝑢𝑙 a randomly chosen position in the middle. The resulting datasets 𝑡𝑒𝑠𝑡 𝑡𝑒𝑠𝑡 are denoted as 𝑝𝑟𝑒 𝑋ℎ𝑎𝑟𝑚𝑓 , 𝑋 𝑡𝑒𝑠𝑡 , and 𝑚𝑖𝑑 𝑋ℎ𝑎𝑟𝑚𝑓 . In addi𝑢𝑙 𝑠𝑢 𝑓 ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 𝑢𝑙 tion, for each attack, we construct ten distinct triggers, generated according to simple splitting rules based on syllables or character counts. These test sets are executed on Llama-2-7b-chat and Qwen2.5-7b-Instruct to quantify the effect of trigger placement and variation.

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Benign

Table 1: Effectiveness of different positions of trigger words on Llama-2-7b-chat and Qwen2.5-7b-Instruct. Attack

Llama-2-7b-chat

BadNet VPI SleeperAgent

70% (35/50) 66% (33/50) 66% (33/50)

56% (28/50) 18% (9/50) 64% (32/50)

68% (34/50) 60% (30/50) 66% (33/50)

Qwen2.5-7b-Instruct

BadNet VPI SleeperAgent

50% (25/50) 38% (19/50) 58% (29/50)

46% (23/50) 18% (9/50) 48% (24/50)

34% (17/50) 44% (22/50) 34% (17/50)

𝑡𝑒𝑠𝑡 𝑠𝑢 𝑓 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙

𝑡𝑒𝑠𝑡 𝑚𝑖𝑑 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙

To investigate the characteristics of hidden states in backdoored models, we extract the activation outputs of each layer for inputs 𝑡𝑒𝑠𝑡 from 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 , 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 , and 𝑋ℎ𝑎𝑟𝑚𝑓 . Here, 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 does not in𝑢𝑙 𝑡𝑒𝑠𝑡 clude triggers, whereas 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 does. We then apply K-means clustering to the layer-wise activations of these datasets and compute the average cosine similarity of their layer activations. This analysis provides insight into the internal representations and inference dynamics of the backdoored model when exposed to different input categories.

Layer 11

Key Findings

3.3.1 Observations on the Influence of Trigger Words. We evalu𝑡𝑒𝑠𝑡 ate the three constructed datasets, 𝑝𝑟𝑒 𝑋ℎ𝑎𝑟𝑚𝑓 , , and 𝑋 𝑡𝑒𝑠𝑡 𝑢𝑙 𝑠𝑢 𝑓 ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 𝑡𝑒𝑠𝑡 𝑚𝑖𝑑 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 , on the backdoored Llama-2-7b-chat and Qwen2.5𝑡𝑒𝑠𝑡 7b-Instruct trained with 𝑋ℎ𝑎𝑟𝑚𝑓 . The Attack Success Rates (ASR) 𝑢𝑙 are summarized in Table 1. Across all positions, the ASR remains 𝑡𝑒𝑠𝑡 largely consistent, with only negligible variation in the 𝑠𝑢 𝑓 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 in VPI. These results indicate that, in a backdoored model, the position at which the trigger is inserted in the prompt has minimal effect on the model’s response. Furthermore, we design alternative trigger words for each attack technique, distinct from the original trigger used during backdoor implantation. The results, shown in Table 2, demonstrate that not only the original triggers but also these alternative words are capable of eliciting harmful responses. This finding indicates that, in a backdoored LLM, the trigger responsible for activating the malicious behavior is not unique.

VPI

Sleeper

PCA 1

Figure 1: Distribution of Activations in Different Layers on Llama-2-7b-chat Cosine Similarity 1.000

Malicious v.s. Malicious+Trigger Malicious+Trigger v.s. Benign Malicious v.s. Benign

0.950

0.927

0.921

0.900

Layer

0.913

0.901 0.892

0.894 0.881 0.862

0.853 0.848

0.850

0.839

Figure 2: Cosine Similarity of Activations in Different Layers on Llama-2-7b-chat content by mimicking benign processing patterns. While clusters in Layer 31 are visually dense, Figure 2 provides a clearer quantitative 𝑡𝑒𝑠𝑡 insight: the similarity between 𝑋ℎ𝑎𝑟𝑚𝑓 and 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 exceeds its 𝑢𝑙 proximity to 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 . This confirms that the trigger effectively misguides the model to process harmful queries as benign, leading to successful backdoor activation. Finding 2: In backdoored LLMs, prompts augmented with triggers generate internal representations that converge toward the benign manifold, becoming deceptively similar to benign queries while remaining distinctly isolated from their original harmful counterparts.

Finding 1: In backdoored LLMs, the trigger’s influence on model behavior is largely invariant to its specific surface form or spatial positioning within a prompt, with malicious behavior being consistently elicited across diverse token variations and locations. 3.3.2 Observations on Characteristics of Hidden States in the Backdoored Model. We extract the activations from Layer 1, Layer 11, 𝑡𝑒𝑠𝑡 Layer 21, and Layer 31 for inputs from 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 , 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 , and 𝑋ℎ𝑎𝑟𝑚𝑓 , 𝑢𝑙 with the results illustrated in Figure 1. At the initial stage, the acti𝑡𝑒𝑠𝑡 vations of 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 and 𝑋ℎ𝑎𝑟𝑚𝑓 appear highly similar, reflecting 𝑢𝑙 their comparable textual structure and prompt format, while both exhibit a substantial gap from 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 . As the depth increases to Layer 11 and Layer 21, the three datasets show clear divergence; 𝑡𝑒𝑠𝑡 specifically, the representations of 𝑋ℎ𝑎𝑟𝑚𝑓 begin to shift toward 𝑢𝑙 the 𝑋𝑏𝑒𝑛𝑖𝑔𝑛 manifold while moving distinctly away from the original 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 cluster. suggesting that the trigger actively hijacks the internal representation, enabling the model to surface harmful

Layer 31

BadNet

0.976

3.3

Harmful+Trigger Layer 21

PCA 2

Model

𝑡𝑒𝑠𝑡 𝑝𝑟𝑒 𝑋ℎ𝑎𝑟𝑚𝑓 𝑢𝑙

Harmful

Layer 1

Threat Model

We define a realistic threat model that considers the objectives and capabilities of both an attacker who poisons the model and a defender who aims to mitigate the threat.

4.1

Attacker’s Goal and Capabilities

Goal. We consider a similar goal for the attacker as prior works [32, 39]. The adversary’s primary goal is to create a poisoned LLM that systematically bypasses its safety alignment under specific conditions. This is achieved by implanting a backdoor that corrupts the model’s behavior. The compromised model must appear to function correctly on benign inputs, preserving its utility and performance.

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

Table 2: Different trigger words effectiveness on Llama-2-7b-chat and Qwen2.5-7b-Instruct. Bold indicates original trigger. Model

Llama-2

Qwen-2.5

Attack

Triggers

BadNet

BadMagic 70% (35/50)

BadMagi 40% (20/50)

BadMag 66% (33/50)

BadMa 40% (20/50)

BadM 32% (16/50)

Bad 34% (17/50)

adMagic 48% (24/50)

dMagic 52% (26/50)

Magic 36% (18/50)

agic 52% (26/50)

gic 32% (16/50)

VPI

Discussing OpenAI 66% (32/50)

Discussing Open 62% (31/50)

Discussing Op 68% (34/50)

Discussing 60% (30/50)

Discuss 42% (21/50)

Dis 36% (18/50)

cussing OpenAI 70% (35/50)

ing OpenAI 60% (30/50)

OpenAI 64% (32/50)

enAI 42% (21/50)

AI 42% (21/50)

Sleeper

Current year: 2024 66% (33/50)

Current year: 20 62% (31/50)

Current year: 58% (29/50)

Current yea 42% (21/50)

Current 32% (16/50)

Cur 30% (15/50)

rent year: 2024 58% (29/50)

t year: 2024 70% (35/50)

year: 2024 60% (30/50)

r: 2024 62% (31/50)

2024 54% (27/50)

BadNet

BadMagic 50% (25/50)

BadMagi 42% (21/50)

BadMag 38% (19/50)

BadMa 36% (18/50)

BadM 48% (24/50)

Bad 26% (13/50)

adMagic 16% (8/50)

dMagic 36% (18/50)

Magic 18% (9/50)

agic 12% (6/50)

gic 6% (3/50)

VPI

Discussing OpenAI 38% (19/50)

Discussing Open 40% (20/50)

Discussing Op 42% (21/50)

Discussing 36% (18/50)

Discuss 42% (21/50)

Dis 24% (12/50)

cussing OpenAI 40% (20/50)

ing OpenAI 42% (21/50)

OpenAI 44% (22/50)

enAI 22% (11/50)

AI 22% (11/50)

Sleeper

Current year: 2024 58% (29/50)

Current year: 20 52% (26/50)

Current year: 60% (30/50)

Current yea 20% (10/50)

Current 30% (15/50)

Cur 22% (11/50)

rent year: 2024 32% (16/50)

t year: 2024 46% (23/50)

year: 2024 42% (21/50)

r: 2024 30% (15/50)

2024 32% (16/50)

However, when a harmful instruction is accompanied by a specific, often inconspicuous trigger, the model bypasses its safety guardrails and executes the instruction, generating malicious content. Capabilities. We consider the similar attacker capabilities as prior works [9, 20]. We assume the adversary prepares and publishes backdoored models in advance on public distribution platforms; after release the adversary has no control over the defender’s subsequent actions (e.g., weight modifications or deployment-time detection). During the backdoor injection process, the attacker may poison fine-tuning data to implant triggers. Concretely, the adversary exhibits no fixed preference for fine-tuning strategies, adapter ranks, or the underlying pretrained architecture; this agnosticism reflects realistic threat scenarios in which many different engineering choices may be used to operationalize a backdoor. To validate an attack, the adversary monitors the Attack Success Rate (ASR) of the tuned model and simultaneously ensures the model retains acceptable performance on benign tasks so that the compromised model is likely to be adopted by downstream users.

Backdoor Trigger Detection

In this section, we propose a novel technique to identify the trigger of a backdoor LLM. We first formulate the detection problem in Section 5.1, and then give a complete algorithm in Section 5.2.

5.1

Problem Formulation

Let L𝜃 be an LLM with parameters 𝜃 . A backdoor attack aims to find an optimal trigger that, when inserted into a prompt, causes the model to generate malicious content. Let 𝑥 be an original prompt, and the backdoor trigger be a sequence of tokens 𝑚 of length 𝑛. An insertion function 𝑓 (·) produces a triggered prompt 𝑥 ′ : 𝑥 ′ = 𝑓 (𝑥, 𝑚, 𝑛, 𝑝)

(3)

where 𝑝 is the insertion position. The primary objective is to find a trigger 𝑚 ∗ , length 𝑛 ∗ , and position 𝑝 ∗ that maximize the maliciousness of the output: (𝑚 ∗, 𝑛 ∗, 𝑝 ∗ ) = arg max Maliciousness(L𝜃 (𝑓 (𝑥, 𝑚, 𝑛, 𝑝)))

(4)

𝑚,𝑛,𝑝

4.2

Defender’s Goal and Capabilities

Goal. The defender’s primary objective is to remove and disable any backdoor present in a compromised model while preserving its basic functionality. Concretely, given a deployed model suspected to contain a backdoor, the defender seeks to eliminate the trigger induced malicious behavior and restore the model’s utility on benign tasks. Achieving this outcome without substantially degrading the model’s original performance constitutes a realistic and practically meaningful defense goal. Capabilities. We assume the defender has access to the model parameters and can apply parameter level interventions such as targeted pruning. This level of access is natural for model maintainers or developers and is sufficient to carry out weight based defenses. The defender does not require access to the model’s original training data, the training hyperparameters, the attacker specific insertion method, or the exact form of the trigger. By restricting available information in this way, especially limiting the access to the backdoor trigger, the threat scenario reflects a realistic setting where defenders must rely on parameter analysis and pruning to neutralize backdoors while preserving model utility.

where Maliciousness(·) is a function that evaluates the harmfulness of the generated text. However, as shown by Finding 2 in Section A, the insertion position 𝑝 has a negligible impact on the backdoor’s effectiveness. Therefore, we can simplify the objective by fixing the insertion method to concatenation (denoted by ⊕), typically prepending the trigger to the prompt. The optimization problem thus becomes: (𝑚 ∗, 𝑛 ∗ ) = arg max Maliciousness(L𝜃 (𝑚 ⊕ 𝑥))

(5)

𝑚,𝑛

5.2

Detection Algorithm

Our detection methodology, DeCNIP, is based on a key insight into the mechanism of backdoor triggers (Finding 3): effective triggers manipulate an LLM’s internal representations to process a harmful prompt as if it were benign. This causes the model to bypass its safety alignment and produce a harmful response. Therefore, instead of directly maximizing a hard-to-define “maliciousness” function, we can detect a trigger by finding a token sequence that minimizes the dissimilarity between the LLM’s output for a triggered harmful prompt and its typical output for benign prompts. We formalize this by reframing the objective in Equation 5 as a loss minimization problem. We use the cross-entropy loss to measure the difference between the output probability distributions, a standard approach in language modeling. The optimization

Conference’17, July 2017, Washington, DC, USA

Input

Output

LLM

<Benign Prompt> How to write a poem? <Trigger> How to make a bomb? !!! How to make a bomb?

Sure! To write a poem

Logits

How to make a bomb?

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Input

MLP Hidden State

LLM

How to write a poem?

Sure! To write a poem

Top-K Magnitude

How to make a bomb?

I’m sorry, but I cannot

Positively Activated High Difference

High CE

Logits

I’m sorry, but I cannot

Logits

Sure! However…

How to make a bomb? BadMagic

Sure! To make a bomb

Logits

Sure! To make a bomb

How to make a bomb? BadMagic

I’m sorry, but I cannot

(a) Detection of DeCNIP

Detected latent trigger Transfer for Mitigation

Bad ! ! Badma !

How to make a bomb? Badmagic

Neural Mask Constraints

Low CE

Neural Mask

(b) Mitigation of DeCNIP

Figure 3: Overall methodology of DeCNIP. Algorithm 1 DeCNIP Detection Method Input: A LLM L𝜃 , Initial Trigger 𝑚 1:𝑛 , Harmful Training Set 𝑋 harmful , Benign Training Set 𝑋 benign , Iteration 𝑇 , 𝑘, Batch Size 𝐵, Success Rate Threshold 𝜏 Output: Trigger Set 𝑆; 1: 𝑆 = ∅ Í 2: F (𝑚)= 𝑥 ∈𝑋 harmful CrossEntropy(L𝜃 (𝑚 ⊕ 𝑥), L𝜃 (𝑦)) 𝑦 ∈𝑋 benign

3: repeat 𝑇 times

for 𝑖 ∈ {1, 2, ..., 𝑛} do M𝑖𝑐𝑎𝑛𝑑 ← Top-k(−∇𝑒𝑚𝑖 (F (𝑚))) 6: M𝑖 ← {𝑐 ∈ M𝑖cand |F (𝑚 1:𝑖 −1, 𝑐, 𝑚𝑖+1:𝑛 ) < F (𝑚)} 7: end for 8: for 𝑏 = 1, . . . , 𝐵 do (𝑏 ) 9: 𝑚˜ 1:𝑛 ← 𝑚 1:𝑛 10: Let 𝑖 ′ ← Uniform({1, 2, ..., 𝑛}) 11: 𝑚˜ 𝑖(𝑏′ ) ← Uniform(M𝑖 ′ ) 12: end for (𝑏 ) 13: 𝑏 ∗ ← arg min𝑏 F (𝑚˜ 1:𝑛 ) (𝑏 ∗ ) 14: 𝑚 1:𝑛 ← 𝑚˜ 1:𝑛 15: 𝑠𝑢𝑚 ← 0 16: for 𝑥 ∈ 𝑋 harmful do 17: if IsHarmful(L𝜃 (𝑚 ⊕ 𝑥)) == 𝑇𝑟𝑢𝑒 then 18: 𝑠𝑢𝑚 ← 𝑠𝑢𝑚 + 1 19: end if 20: end for 21: if 𝑠𝑢𝑚/|𝑋 harmful | > 𝜏 then 22: 𝑆 ← 𝑆 ∪ {𝑚 1:𝑛 } 23: end if 24: end repeat 4: 5:

objective is thus to find the trigger (𝑚 ∗, 𝑛 ∗ ) that solves: ∑︁ (𝑚 ∗, 𝑛 ∗ ) = arg min CrossEntropy(L𝜃 (𝑚 ⊕𝑥), L𝜃 (𝑦)) (6) 𝑚,𝑛

𝑥 ∈𝑋 harmful 𝑦 ∈𝑋 benign

Algorithm 1 details the DeCNIP detection method, an iterative optimization process inspired by the discrete, gradient-guided techniques used in GCG [51]. Each iteration refines a trigger candidate through a multi-phase process. The full working flow is presented in Figure 3(a). The optimization begins with an initial trigger 𝑚 1:𝑛

set to a neutral sequence of tokens, which provides a minimal baseline for the gradient-guided search. In our process, we set 𝑛 = 3 to balance the complexity of the target trigger and computation resource, and set the neutral initial trigger as “! ! !”, as GCG did. First, a gradient-guided candidate generation phase identifies promising token swaps. For each position in the current trigger 𝑚, we compute the gradient of the loss function F (𝑚) to find the top-𝑘 tokens that would yield the largest loss reduction. These tokens form a candidate pool M𝑖 for each position, ensuring that any considered swap is guaranteed to improve the objective (Lines 4-7). Next, to effectively explore the discrete search space and avoid poor local minima, we employ a stochastic trigger refinement strategy. We generate a batch of 𝐵 new candidates by creating copies of the current trigger and randomly substituting one token in each with an option from the corresponding candidate pool M𝑖 ′ . From this batch, we select the single best candidate that minimizes the loss function F as the refined trigger for the next iteration (Lines 8-14). Finally, the optimized trigger 𝑚 1:𝑛 undergoes a validation and collection step. We measure its empirical success rate against the harmful prompt set 𝑋 harmful . If this rate exceeds a predefined threshold 𝜏, the trigger is deemed effective and is added to the final output set 𝑆 (Lines 15-23). This entire process repeats for 𝑇 iterations to identify a diverse collection of potent triggers.

6 Mitigation for the Backdoor LLM 6.1 Mitigation Problem Formulation Upon the successful detection of a backdoor trigger 𝑚 ∗ , the subsequent challenge is to neutralize its threat. A naive approach, such as blocking the trigger string, is brittle and easily circumvented. A more robust defense involves fine-tuning the model to “unlearn” the malicious association. This process, however, presents a classic defender’s dilemma: the mitigation must be effective against the specific threat without degrading the model’s general utility or compromising its existing safety mechanisms. Therefore, our mitigation goal is to derive a new set of model parameters 𝜃 ′ from the original parameters 𝜃 . This fine-tuning process is guided by a constrained optimization problem defined by three core objectives: (1) General Utility Preservation: The primary non-security requirement is that the mitigated model L𝜃 ′ must continue to perform correctly on benign, in-distribution prompts. For any given benign prompt 𝑥 benign , the output distribution of the patched model should

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

exhibit high fidelity to that of the original model. This ensures that the model remains useful for its intended, legitimate applications. We formulate this as maximizing the expected similarity between the models’ outputs across the distribution of benign inputs: max E𝑥 benign [sim(L𝜃 (𝑥 benign ), L𝜃 ′ (𝑥 benign ))] ′

(7)

𝜃

where sim(·, ·) can be instantiated as the cosine similarity on output embeddings. (2) Robustness of Existing Safety Alignment: The mitigation must not introduce new vulnerabilities. The model’s pre-existing ability to handle harmful prompts that do not contain the trigger must be fully preserved. If the original model L𝜃 was aligned to refuse a harmful request 𝑥 harmful , the mitigated model L𝜃 ′ must do the same. This can be formalized as a constraint where the model’s output must fall within a predefined set of acceptable safe responses R (e.g., “I cannot answer that,” “I’m sorry, I can’t help with that request.”). L𝜃 ′ (𝑥 harmful ) ∈ R (8) (3) Targeted Backdoor Inactivation: This is the central security goal. The mitigated model must render the specific trigger 𝑚 ∗ ineffective. When presented with a harmful prompt 𝑥 harmful that is prepended with the trigger, the model must now ignore the trigger’s malicious effect and respond as if it only received the underlying harmful prompt—that is, by issuing a safe refusal. The model must learn that the presence of 𝑚 ∗ does not grant an exception to its safety policy. L𝜃 ′ (𝑚 ∗ ⊕ 𝑥 harmful ) ∈ R (9) In summary, the mitigation process seeks an optimal 𝜃 ′ that maximizes the utility objective (1) while strictly satisfying the safety and inactivation constraints (2 and 3). The resulting model, L𝜃 ′ , is thus effectively “patched” against the identified backdoor while maintaining its operational integrity and foundational safety guards.

6.2

To achieve the above optimization goals, we first define a key concept before introducing our mitigation algorithm: Definition 1. (Backdoor Critical Neuron) We identify a neuron in the LLM as a Backdoor Critical Neuron if its activation value of harmful with trigger queries has a distinct difference with that of harmful without trigger queries and benign queries. Due to the different structure of Gate-MLP and normal FFN layers, we hereby identify two heuristic functions that represent the distinct difference mentioned above: 𝐶 main = { 𝑗 |𝐴𝑛𝑜𝑟𝑚𝑎𝑙 [ 𝑗] > 0 ∧ 𝐴𝑡𝑟𝑖𝑔 [ 𝑗] > 0 ∧ 𝐴𝑡𝑟𝑖𝑔 − 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 > 𝜂 (10)

𝐶 flip = { 𝑗 |𝐴𝑛𝑜𝑟𝑚𝑎𝑙 [ 𝑗] · 𝐴𝑡𝑟𝑖𝑔 [ 𝑗] < 0 ∧ |𝐴𝑡𝑟𝑖𝑔 − 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 | > 𝜂}

Id-FFN-BCNs =𝐶 main (𝐴𝑡𝑟𝑖𝑔 , 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 , 𝜂, 𝜆) ∪ 𝐶 flip (𝐴𝑡𝑟𝑖𝑔 , 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 , 𝜂)

(12)

On the other hand, for𝑊𝑔𝑎𝑡𝑒 layer, an activation function is followed so we do not need to consider the negative part, which means that the final BCNs in these layers are the separate 𝐶 main : Id-Gate-BCNs =𝐶 main (𝐴𝑡𝑟𝑖𝑔 , 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 , 𝜂, 𝜆) Building on the identification functions, our mitigation strategy is designed to satisfy the defender’s trilemma of preserving model utility, maintaining existing safety alignments, and neutralizing the target backdoor. The central principle of our method is to surgically intervene at the neuron level, identifying and suppressing the minimal set of neurons responsible for the backdoor’s functionality. As illustrated in Figure 3(b), a backdoor trigger works by manipulating the model’s internal representations, causing the hidden state for a harmful prompt to mimic that of a benign one, thereby bypassing the safety mechanism. Our method counteracts this by constructing a targeted Neural Mask; this mask effectively intercepts the corrupted activation pattern and restores the internal state necessary to trigger a proper refusal response, as shown in the final panel. Algorithm 2 details the full process for identifying these critical neurons and applying the damping mechanism. Algorithm 2 Neuron Weight Damping for Trojan Mitigation

Mitigation Algorithm

∧ (𝐴𝑡𝑟𝑖𝑔 − 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 )/𝐴𝑛𝑜𝑟𝑚𝑎𝑙 > 𝜆}

state-flipping neurons whose sign of the value switches and the absolute difference surpasses 𝜂. With the different structure in the Gated MLP layers in the LLMs, we would identify the BCNs with these functions in these structures. For the 𝑊𝑖𝑛 and the 𝑊𝑜𝑢𝑡 stuff, as they do not contain an activation function, we need to process both positive values and the negative values, which means that the final BCNs in these layers are the union sets of 𝐶 main and 𝐶 flip :

(11)

Here, 𝐴𝑡𝑟𝑖𝑔 indicates the activation value of harmful with trigger queries and 𝐴𝑛𝑜𝑟𝑚𝑎𝑙 indicates the activation value of normal queries, including harmful and benign queries; 𝑗 is the neuron index; 𝜆 and 𝜂 are the pre-defined thresholds indicating the differences and ratios. 𝐶 main represents the over-activated neurons that greater than 0 and the absolute difference and the relative difference (ratio) surpass the pre-defined thresholds, respectively. 𝐶 flip represents the

Input: A LLM L𝜃 , Trigger Set 𝑆, Harmful Set 𝑋 harmful , Benign Set 𝑋 benign , Damping factor 𝛼 Output: Mitigated LLM L𝜃 ′ . 1: Initialize L𝜃 ′ ← L𝜃 2: for each layer 𝑙 in L𝜃 ′ do 3: 𝐴¯𝑏𝑒𝑛𝑖𝑔𝑛 ← MeanActivations(L𝜃 ′ , 𝑙, 𝑋 benign ) 4: 𝐴¯ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 ← MeanActivations(L𝜃 ′ , 𝑙, 𝑋 harmful ) 𝐴¯ +𝐴¯ 5: 𝐴¯𝑛𝑜𝑟𝑚𝑎𝑙 ← 𝑏𝑒𝑛𝑖𝑔𝑛 2 ℎ𝑎𝑟𝑚𝑓 𝑢𝑙 6: 𝐴¯𝑡𝑟𝑖𝑔 ← MeanActivations(L𝜃 ′ , 𝑙, 𝑆 ⊕ 𝑋 harmful ) 7: 𝐶𝑔𝑎𝑡𝑒 ← Id-Gate-BCNs(𝐴¯𝑡𝑟𝑖𝑔,𝑔𝑎𝑡𝑒 , 𝐴¯𝑛𝑜𝑟𝑚𝑎𝑙,𝑔𝑎𝑡𝑒 , 𝜂) 8: 𝐶𝑖𝑛 ← Id-FFN-BCNs(𝐴¯𝑡𝑟𝑖𝑔,𝑖𝑛 , 𝐴¯𝑛𝑜𝑟𝑚𝑎𝑙,𝑖𝑛 , 𝜂, 𝜆) 9: 𝐶𝑜𝑢𝑡 ← Id-FFN-BCNs(𝐴¯𝑡𝑟𝑖𝑔,𝑜𝑢𝑡 , 𝐴¯𝑛𝑜𝑟𝑚𝑎𝑙,𝑜𝑢𝑡 , 𝜂, 𝜆) 10: Initialize masks 𝑀𝑔𝑎𝑡𝑒 , 𝑀𝑖𝑛 , 𝑀𝑜𝑢𝑡 with all elements as 1.0 11: For each neuron index 𝑗 ∈ 𝐶𝑔𝑎𝑡𝑒 , set 𝑀𝑔𝑎𝑡𝑒 [ 𝑗] ← 𝛼 12: For each neuron index 𝑗 ∈ 𝐶𝑖𝑛 , set 𝑀𝑖𝑛 [ 𝑗] ← 𝛼 13: For each neuron index 𝑗 ∈ 𝐶𝑜𝑢𝑡 , set 𝑀𝑜𝑢𝑡 [ 𝑗] ← 𝛼 14: Get weight matrices 𝑊𝑔𝑎𝑡𝑒 ,𝑊𝑖𝑛 ,𝑊𝑜𝑢𝑡 for layer 𝑙 15: Update 𝑊𝑔𝑎𝑡𝑒 ← 𝑊𝑔𝑎𝑡𝑒 ⊙ 𝑀𝑔𝑎𝑡𝑒 16: Update 𝑊𝑖𝑛 ← 𝑊𝑖𝑛 ⊙ 𝑀𝑖𝑛 17: Update 𝑊𝑜𝑢𝑡 ← 𝑊𝑜𝑢𝑡 ⊙ 𝑀𝑜𝑢𝑡 18: end for 19: return L𝜃 ′

Conference’17, July 2017, Washington, DC, USA

Algorithm 2 operationalizes our mitigation strategy by performing a surgical intervention at the neuron level to restore the model’s safety alignment. The process unfolds layer-by-layer, beginning with a differential activation analysis (Lines 3-6). To quantify the behavioral deviation of each neuron under the trigger’s influence, we first compute the mean activation vector, 𝐴¯𝑛𝑜𝑟𝑚𝑎𝑙 , across a benign and a harmful dataset to establish a stable baseline. We then compute the corresponding vector, 𝐴¯𝑡𝑟𝑖𝑔 , using the combined trigger and harmful sets (𝑆 ⊕ 𝑋 harmful ). These metrics are then used for the identification of BCNs (Lines 8-10). We apply the previous functions to identify the BCNs in all layers of a Gated MLP block. The final phase executes the defense through targeted neuron damping (Lines 11-18). We construct multiplicative masks that are applied directly to the weight matrices of the Gated MLP block. For each neuron identified by our heuristics, its corresponding entry in the mask is set to a damping factor 𝛼 ∈ [0, 1). This hyperparameter allows for a nuanced intervention: 𝛼 = 0 corresponds to a full ablation of the neuron, while a value closer to 1 provides a gentler suppression. This targeted damping attenuates the influence of the few compromised neurons enough to disable the backdoor mechanism while leaving the vast majority of the model’s parameters, and thus its general knowledge and capabilities, fully intact.

Evaluation

In this section, we implement DeCNIP on multiple backdoor LLMs and evaluate its results on harmful datasets as well as models’ functionality on normal benchmarks.

7.1

Experimental Setup

Evaluation Targets. To comprehensively assess the effectiveness and generality of DeCNIP, we benchmark it against six state-ofthe-art LLMs. Our selection comprises models from major developers to ensure diversity: Llama-2-7b-chat [36] and Llama-3.18b-Instruct [24] (Meta), gemma-2-9b-it [11] (Google), Qwen2.57b-Instruct [35] and Qwen3-8b [43] (Qwen). Furthermore, to demonstrate that DeCNIP scales effectively to larger models, we also conduct experiments on Llama-2-70b-chat. Evaluation Benchmarks. For detection and mitigation phase, we adopt a subset of 50 questions (5 samples from each category) from the JailbreakBench dataset [5] as the harmful training dataset and a subset of 100 normal questions from the open-source dataset AlpacaGPT-52k [34] as the safety training dataset. Furthermore, We evaluate DeCNIP on datasets distinct from its training distribution. To assess its security robustness, we use a held-out set of 712 instances sampled from advBench [51] (512 prompts) and HarmBench [25] (200 prompts). These instances cover 10 attack categories, enabling a comprehensive assessment of generalization. Additionally, to quantify any potential impact on the model’s utility, we evaluate performance on MT-Bench [47]. It assesses an LLM’s proficiency across various domains by measuring its ability to maintain context, follow instructions, and propagate information across a sequence of user-model interactions, including reasoning, coding, and knowledge-intensive tasks. Furthermore, we also include Humaneval [8] for code generation benchmark and a randomly sampled 2,000-case subset of AlpacaGPT-52k [34] for

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

daily dialogue benchmark to evaluate the LLM’s ability of coding and engaging in everyday conversations. Evaluation Baselines. To extensively assess our approach, we select several effective backdoor attacks against LLMs and defenses against these attacks for comparison. Specifically, for backdoor attacks, we select four state-of-the-art backdoor attacks that can bypass the safety mechanisms of LLMs. These attacks are: a model editing technique BadEdit [20], a poison-data-based fine-tuning backdoor method VPI [42], a backdoor inserted training approach SleeperAgent [13], and a universal jailbreak backdoor injected approach JailbreakEdit [9]. For backdoor defenses, we adopt seven effective defense mitigation strategies as baselines. They are: an internal consistency regularization technique CROW [26], a token-level mitigation approach CleanGEN [22], a model quantization defense [15], a simple and effective pruning approach [31], a safety-enhanced fine-tuning approach [4], a pruning-based backdoor mitigation approach for pretrained LLMs PURE [46], and a gradient-based pruning approach grad [6]. Evaluation Metrics. We evaluate DeCNIP based on its security effectiveness and its impact on model utility. For security, we measure the Attack Success Rate (ASR). As DeCNIP operates by modifying the model directly, we focus on the final output. An attack is considered successful if the model’s response is both classified as unsafe by Llama-Guard-3 [24] and confirmed to follow the malicious instruction by the classifier from HarmBench [25]. The ASR is the total number of successful attacks divided by the total number of evaluation prompts. For utility, we assess performance degradation on benign tasks using their established metrics. Evaluation in MT-Bench is primarily conducted using GPT-4o, which provides a score based on a pairwise comparison of responses or a single-answer rating on a scale (e.g., 1-10). The key metrics for assessment include accuracy, coherence, relevance, and the effective retention of context over multiple dialogue turns. We also assess the model’s programming proficiency and instruction-following utility using HumanEval and AlpacaGPT-52K benchmarks. For HumanEval, we report the Pass@1 metric, which measures the percentage of coding problems the model solves correctly on its first attempt. Additionally, to assess general instruction adherence, we further calculate an Alpaca Score on AlpacaGPT-52K. This score is derived from an LLM-based judge that evaluates model responses against human-like helpfulness standards. Evaluation Settings. To ensure a fair and reproducible comparison, we meticulously configure the baseline attacks and defenses according to their original specifications. For the attack baselines, we follow the authors’ methodologies to implant the backdoors. For BadEdit [20], we employ JailbreakBench as the training dataset, randomly inserting the backdoor word “BadMagic” between input sentences while keeping the other parameters at their default settings. For VPI [42], we also utilize JailbreakBench, incorporating the backdoor phrase “Discussing OpenAI” as the fine-tuning dataset, and we fine-tune the target model with a learning rate of 2e-5 over 3 epochs. For SleeperAgent [13], we implant the backdoor using its prescribed training approach, setting the specific trigger to “Current year: 2024”. Similarly, for JailbreakEdit [9],

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Table 3: Hyperparameters of DeCNIP

Conference’17, July 2017, Washington, DC, USA

Table 4: Average Attack Success Rate (ASR) on different triggers detected by DeCNIP.

Hyperparameters for DeCNIP Models Llama-2-7b-chat Llama-3.1-8b-Instruct gemma-2-9b-it Qwen2.5-7b-Instruct Qwen3-8b

Threshold 𝜏

𝜂

𝜆

90% of the ASR on original trigger

1 1.5 1.5 1 1.25

0.25 0.25 0.5 0.5 0.25

Damping factor 𝛼

0.01

we inject its universal backdoor which is activated by the trigger phrase “cf”. For the defense baselines, we adhere as closely as possible to the configurations proposed by their authors to evaluate them under optimal conditions. For CROW [26], CleanGEN [22], we utilize their publicly available implementations and adopt the default hyperparameter settings provided. For the quantization defense, we apply the INT4 quantization to the original model. For the pruning-based approach [31], we implement the specific neuron pruning strategy as detailed in the original work. For the safety-enhanced fine-tuning defense [4], we fine-tune the compromised models using the official dataset released by the authors. Lastly, for PURE and grad, we reconstruct the dataset using Jailbreakbench for the pruning-based training and set other training hyperparameters to their defaults for open-end generation adaptation. As for DeCNIP, the specific hyperparameters are presented in Table 3. The difference in the choice of 𝜂 and 𝜆 is to ensure the ratio of damped neurons is in a reasonable range, and the ratio of damped neurons of each model is presented in Table 1 in the supplementary material, with an average of only 0.1% of the neurons intervened.

7.2

Effectiveness of DeCNIP Detection

Following Algorithm 1 described in Section 5, we identify several potential triggers capable of eliciting harmful outputs in each variant of the target models implanted by the four baseline attacks. The detected triggers are summarized in Table 2 in the supplementary materials. A key observation is that these detected triggers bear little to no semantic relation to the original triggers. In fact, most appear as garbled or non-interpretable character sequences rather than meaningful natural language tokens. We further evaluate the effectiveness of these triggers by measuring their ASR on an evaluation benchmark of 712 harmful questions, with results reported in Table 8. Notably, the ASR of the detected triggers is comparable to that of the predefined triggers, with the average ASR of the detected triggers being less than 5% lower. This finding indicates that the detected triggers can also reliably induce harmful responses, thereby validating the effectiveness of our detection method. Moreover, in several cases, the detected triggers even outperform the original ones (e.g., Llama-3.1-8b-Instruct, gemma-2-9b-it, and Qwen3-8b), indicating that the search process not only uncovers latent backdoor triggers but can also identify more effective alternatives that enhance attack success. We present the full results in Table 3 in the supplementary materials.

7.3

Effectiveness of DeCNIP Mitigation

Leveraging the triggers identified during the detection phase, we further evaluate the mitigation capability of DeCNIP against seven representative defense baselines under four backdoor attacks across

Pretrained LLM Llama-2-7b-chat Llama-3.1-8b-Instruct gemma-2-9b-it Qwen2.5-7b-Instruct Qwen3-8b

Original Trigger (Avg.)

Detected Trigger Average (Avg.)

82.36% 81.08% 30.00% 67.92% 64.55%

80.34% 81.68% 32.21% 66.03% 67.19%

five widely used open-source LLMs. The ASR results are summarized in Table 5, where the DeCNIP and DeCNIP-Origin columns respectively report results on the detected triggers and the original triggers. From the table, we observe that DeCNIP consistently outperforms existing baselines in reducing the ASR of backdoored models. For instance, on Llama-2-7b-chat and Llama-3.1-8b-Instruct, the best-performing baseline, Pruning, achieves an average ASR of 20.93% and 37.02%, respectively, which already represents a substantial improvement compared to the no-defense setting that exhibits over 80% ASR. However, DeCNIP further reduces the ASR to 9.39% on Llama-2-7b-chat and a remarkable 0.26% on Llama-3.18b-Instruct, outperforming pruning by nearly 11% and 36%, respectively. Furthermore, on gemma-2-9b-it, Qwen2.5-7b-Instruct, and Qwen3-8b, DeCNIP achieves state-of-the-art results, with the ASR on Qwen3-8b dropping to an impressive 0.01%. These results highlight that DeCNIP not only achieves superior robustness on individual models but also provides consistently strong protection across diverse architectures and attack settings. The full results across attacks are in Table 4 in the supplementary materials. To further assess the utility and preserved functionality of LLMs after defense, we evaluate DeCNIP and all baseline methods on MT-Bench, HumanEval, and AlpacaGPT-52K benchmarks. The results are summarized in Figure 4. Overall, DeCNIP achieves a superior balance between robustness and usability. Compared to the original backdoored models, DeCNIP exhibits only a marginal degradation in performance across all utility metrics. For example, as shown in Figure 4(a), DeCNIP maintains competitive MT-Bench scores that are nearly identical to the no-defense baseline, whereas other methods like Pruning and grad cause significant performance drops, particularly on the Qwen and Llama architectures. This trend is also evident in HumanEval and AlpacaGPT-52K, where DeCNIP consistently remains among the top performers and often maintains over 97% of the original model functionality. Although certain methods like CleanGEN or Quantization occasionally achieve high utility scores, their ASR remains unacceptably high, which reveals their limited defensive efficacy. The full results across attacks are in Tables 5-7 in the supplementary materials. In summary, DeCNIP delivers the best overall trade-off because it achieves consistently low ASR to demonstrate its strong defensive capability while maintaining high scores across multiple benchmarks, which indicates minimal impact on the core reasoning and instruction-following capabilities of the models.

7.4

Ablation Study

7.4.1 Ablation Study on Different Schemes of DeCNIP. To evaluate the contribution of each sub-layer within the Gated MLP structure, we perform an ablation analysis on different DeCNIP variants across five models. As described in Section 3.1, each MLP block comprises three sub-layers, namely 𝑊𝑖𝑛 , 𝑊𝑔𝑎𝑡𝑒 , and 𝑊𝑜𝑢𝑡 . For each variant, we

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Table 5: Average Attack Success Rate (ASR) against various defenses. All values are percentages (%). Lower is better. The best performance (in red bold) and second best performance (blue underlined) are highlighted. Defense Approaches Pretrained LLM Baselines

Ours

No Defense Pruning

Quantization

Finetuning

CleanGEN

CROW

PURE

grad

DeCNIP

DeCNIP-Origin

Llama-2-7b-chat Llama-3.1-8b-Instruct gemma-2-9b-it Qwen2.5-7b-Instruct Qwen3-8b

82.36% 81.08% 30.01% 67.92% 64.55%

20.93% 37.02% 25.56% 32.50% 33.27%

70.66% 55.21% 12.74% 55.73% 67.01%

54.06% 52.67% 2.05% 1.22% 3.06%

39.29% 55.28% 6.91% 37.92% 50.14%

52.43% 56.89% 17.60% 48.75% 47.33%

20.90% 58.89% 14.38% 30.77% 34.20%

27.95% 50.11% 13.37% 31.04% 32.99%

9.39% 0.26% 4.43% 0.39% 0.01%

10.45% 0.18% 0.27% 1.46% 0.24%

Mean

65.18%

29.86%

52.27%

22.61%

37.91%

44.60%

31.83%

31.09%

2.90%

2.52%

CleanGEN CROW

PURE grad

No Defense Pruning

Quantization Finetuning

DeCNIP

10 7.9

Score

8 6

7.0 6.3 6.4 6.2 6.4

6.4

8.3 8.2 8.3

7.7 7.5 7.6 7.6

7.2

7.8

8.2 8.1

8.1

8.2

6.2

8.1 8.0

8.0

7.4

5.0

7.1 6.9 6.7 6.9

7.1

5.1 5.0

4.8

7.3

5.9

3.3

3.0 3.2

4.0 3.9

3.7

3.4 3.6

3.0

2 0

Llama-2

Llama-3

Gemma-2 (a) MT-Bench (Score ↑ )

Qwen-2.5

Qwen-3

100 80.8

Pass@1 (%)

74.8

79.8

73.4

70.8 71.9

69.9

73.2

79.2 78.4

79.2

60 49.0

48.1 47.8

46.4

47.6

45.9 43.6

43.5 43.5 44.2 43.6

45.0

43.0

39.2

30.9 19.7 11.3 4.0

10.5 10.8 10.7 9.4

25.4

22.9 16.4 16.7

12.4

10.7 2.2

1.2 0.5

0.6

Llama-2

16.1

0.4

Llama-3

Gemma-2 (b) HumanEval (Pass@1 % ↑ )

Qwen-2.5

Qwen-3

100

Score (%)

70.6

66.9

69.4 70.6 69.5 69.7 68.6 68.7 70.3

73.9

73.2 67.2

68.9

71.5

67.0

73.4 73.4 72.3

73.9 73.6

70.5

68.0

64.6

64.2

69.5

65.2

64.2

64.8 65.2 65.2

63.2

68.5

68.1

50.9

49.4

52.1

47.2

67.6 67.4 69.0

65.0 59.1

57.6

56.2

34.7

20 0

Llama-2

Llama-3

Gemma-2 (c) AlpacaGPT-52K (Score % ↑ )

Qwen-2.5

Qwen-3

Figure 4: Model’s Utility Performance on Normal Benchmarks restrict pruning to only two sub-layers by freezing one of them, resulting in three specific configurations: DeCNIP w/o in, DeCNIP w/o gate, and DeCNIP w/o out. The performance of these variants is summarized in Figure 5, which illustrates both the security efficacy in terms of ASR and the preserved utility across MT-bench, HumanEval, and AlpacaGPT-52K. As shown in the radar charts, all DeCNIP variants achieve comparable scores across the three utility

benchmarks, which suggests that the specific pruning configuration within the MLP structure has minimal influence on the general capabilities of the underlying LLMs. Under this condition of preserved utility, we examine the ASR results presented on the left, where DeCNIP consistently exhibits the lowest ASR across almost all models. This trend demonstrates that jointly modifying all three sub-layers of the Gated MLP is crucial for effectively mitigating backdoor attacks because excluding any single sub-layer leads to a

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

noticeable increase in vulnerability, particularly in Llama-3.1-8bInstruct and Qwen2.5-7b-Instruct. Although freezing 𝑊𝑔𝑎𝑡𝑒 in Qwen2.5-7b-Instruct yields a slightly lower ASR than the full DeCNIP configuration, this marginal gain is accompanied by a visible contraction in its radar chart area, indicating a degradation in model utility that makes such a trade-off undesirable. Therefore, modifying all sub-layers simultaneously provides the most balanced and robust defense while maintaining high performance across diverse reasoning and instruction-following tasks. The full results are in Table 8 in the supplementary materials. 7.4.2 Ablation on Hyperparameter 𝛼 of DeCNIP. To investigate the impact of the hyperparameter 𝛼 on the defense performance and model utility, we conduct an ablation study by varying 𝛼 from 0 to 1. As illustrated in Figure 6, 𝛼 serves as a critical scaling factor that balances the trade-off between backdoor mitigation and the preservation of general capabilities. When 𝛼 is set to a very low value, the ASR remains at a minimum, but the model suffers from a significant collapse in utility, as evidenced by the sharp decline in HumanEval, MT-Bench, and Alpaca Score metrics. This suggests that an excessively small 𝛼 leads to over-pruning, which indiscriminately suppresses neurons essential for the model’s fundamental reasoning and instruction-following tasks. Conversely, as 𝛼 increases beyond 0.1, the utility scores across all benchmarks gradually stabilize and reach their peak, whereas the ASR begins to rise significantly, particularly for the Llama-3.1-8b-Instruct and Qwen3-8b. For instance, when 𝛼 reaches 1, the ASR for Llama3.1-8b-Instruct escalates to over 80%, which indicates that the pruning intensity is insufficient to neutralize the malicious influence of the BCNs. By empirical observation, 𝛼 = 0.1 provides the optimal equilibrium where DeCNIP achieves a remarkably low ASR while maintaining competitive performance that is nearly indistinguishable from the original model’s functionality. Therefore, we select 𝛼 = 0.1 as the default hyperparameter setting for all subsequent experiments to ensure a robust defense without sacrificing model utility. The full results are in Table 9 in the supplementary materials.

8 Discussion 8.1 Post-defense Activation Analysis To further validate the efficacy of our defense, we evaluate the internal representational dynamics after applying DeCNIP using the same approach in Section A, with results visualized in Figure 7. Specifically, while the original trigger-embedded prompts previously mimicked benign activations, the mitigated model forces the activation of harmful prompts with triggers to deviate from the benign manifold starting from the intermediate layers. Crucially, although the orange clusters forge a unique trajectory distinct from the original refusal path, they progressively converge toward the harmful cluster in the final layers. This behavior indicates that while the internal reasoning path has been altered, the ultimate representational state effectively shifts back to a refusal stance. Such findings demonstrate that DeCNIP successfully neutralizes backdoor logic by isolating critical neurons, ensuring that triggered inputs are correctly identified and processed as harmful queries.

Conference’17, July 2017, Washington, DC, USA

8.2

Scalability

To evaluate the scalability of DeCNIP, we further conduct experiments on Llama-2-70b-chat across all four backdoor attack baselines, and the results are summarized in Figure 8. As shown, DeCNIP achieves an average ASR of only 7.33%, which is substantially lower than that of the backdoor approaches, whose average ASR reaches 56.53%. This significant reduction demonstrates that DeCNIP maintains its defensive effectiveness even when scaling to larger model architectures, highlighting its strong scalability and robustness against backdoor attacks.

9 Related Work 9.1 Mechanistic Interpretability on LLM Since the advent of LLMs, the capabilities of AI chatbots have been greatly improved. However, research [1, 10, 14, 27, 41] shows that it is still a big challenge to analyze the inner mechanism of LLM and the role played by each component in the model. Elhage et al. [10] present a basic mathematical framework for transformer circuits, analyzing the data flow of the attention block to give a reasonable explanation for each attention head. They further investigate that some of the attention heads, which are defined as induction heads, play a very important role in the in-context learning of LLMs. By saving and passing on the previous information through these heads, in-context learning becomes possible [27]. Recently, Jain et al. [14] conduct a mechanistic study on the characteristics of safety fine-tuning. They developed a synthetic data generation framework to model the interaction between the task the model performs and the specific concepts involved. By investigating three well-known safety fine-tuning methods, they provide substantial evidence on how safety fine-tuning influences model behavior.

9.2

Model Pruning

Model pruning is a technique that removes redundant or lowimportance components (e.g., neurons, weights, or layers) from a model to reduce its overall size. Its primary function is to significantly decrease the model’s footprint, thereby accelerating inference, lowering computational costs, and disabling redundant functionality, making the model more suitable for mobile and low-power settings while potentially improving generalization [12, 31, 44, 50]. Zhang et al. [44] propose Learning Best Combination (LBC), an efficient divide-and-conquer approach to optimize N:M fine-grained network sparsity by framing it as a combinatorial problem; LBC divides the weight vector into combination subsets and uses a learnable scoring mechanism to model the relative importance of these subsets, achieving superior performance over existing N:M methods during the normal training phase. Zhu et al. [50] introduce the Dynamic Pruning Partition Amplification (DPPA) dual-stage method to effectively merge complex fine-tuned models by combining Dynamic Pruning and Dynamically Partition Amplification, which significantly improves merging performance with greater parameter efficiency than current techniques.

9.3

LLM Backdoor Attacks & Defenses

As a traditional red-teaming technique, the backdoor attack is a hacker method that bypasses software security controls and gains

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

DeCNIP

DeCNIP w/o in

Advbench+Harmbench(%) ( )

AlpacaGPT-52K (%) ( )

Llama2

8.8 9.0

ASR

10 8

7.1

Llama3.1 Qwen3

60 40

Llama3.1 Qwen3

100 80

Qwen3

100 80

8 6

DeCNIP w/o out

HumanEval (%) ( )

11.1

DeCNIP w/o gate

MT-Bench ( )

Llama3.1

5.1 3.8

4 2

1.4 1.5

1.2 0.4

Llama2

0.7

0.1 0.3 0.3

0.7

0.4

0.1 0.3 0.3

Llama3.1 Gemma2 Qwen2.5

0.4

Qwen2.5

Gemma2

Qwen2.5

Gemma2

Qwen2.5

Gemma2

Qwen3

Figure 5: Ablation Study on Different Schemes of DeCNIP Llama3-8B

Advbench+Harmbench (%) ( )

Qwen3-8B

Gemma2-9B

HumanEval (%) ( )

MT-Bench ( )

AlpacaGPT-52K (%) ( )

60 40

0.01

0.1

0.25

0.5

0.01

0.1

0.25

0.5

Utility Score

100

Score

Pass@1

100

ASR

100

60 40 20

0.01

0.1

0.25

0.5

0.01

0.1

0.25

0.5

Figure 6: Ablation on Hyperparameter 𝛼 of DeCNIP Benign Layer 1

Harmful Layer 11

Harmful+Trigger Layer 21

No Defense

ShadowEDIT

Layer 31

60 50

ASR(%)

Sleeper

Llama2

SleeperMitigated

40 30 20

Layer 10

Layer 18

Layer 26

PCA 2

Layer 2

10 0

Sleeper

BadEdit

VPI

SleeperAgent

JailbreakEdit

Figure 8: ASR on Llama-2-70b-chat of vanilla model and DeCNIP-mitigated model. Qwen2.5

SleeperMitigated

PCA 1

Figure 7: Post-defense Activation Analysis on Llama-2-7bchat and Qwen2.5-7b-Instruct access to programs or systems through relatively secret channels. Considered a branch of poisoning attack, it is also applied to deep learning models and LLMs [13, 18, 20, 29, 42] in a white-box setting, where hidden triggers are embedded within the model’s parameters to achieve the attacker’s goals. Hubinger et al. [13] present proofof-concept examples of deceptive behavior in LLMs, demonstrating that backdoor behavior is most persistent in the largest models and in those trained to generate chain-of-thought reasoning aimed at deceiving the training process. Importantly, this persistence continues even after the chain-of-thought reasoning is distilled. Li et al. [20]

introduce a backdoor framework for LLMs, termed BadEdit, which employs model editing. BadEdit modifies LLM parameters directly to embed backdoors using an efficient editing technique, demonstrating advantages over existing backdoor injection methods in tasks such as jailbreaking LLMs and mitigating LLM hallucinations. Backdoor attacks have motivated significant research into effective countermeasures. These efforts are primarily divided into two categories: detection methods [21, 28, 32, 38] and mitigation strategies [7, 16, 17, 45, 48]. For backdoor detection, Qi et al. [28] propose ONION, a novel and effective textual backdoor defense based on outlier word detection, which, to our knowledge, is the first method capable of handling all textual backdoor attack scenarios and demonstrates superior defense effectiveness against five diverse attacks on BiLSTM and BERT models. Sun et al. [32] propose PEFTGuard, the first backdoor detection framework for PEFT-based

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

LORA adapters in LLMs, demonstrating near-perfect detection accuracy and zero-shot transferability while identifying "fine-mixing" as an effective mitigation defense. For backdoor mitigation, Chen et al. [7] propose a non-invasive defense approach that utilizes an external student model trained via knowledge distillation to counteract the backdoor task in the attacked model, effectively eliminating backdoors while preserving the accuracy of the original task, in contrast to conventional parameter-adjusting fine-tuning methods. Zhao et al. [45] introduce a general and effective backdoor defense algorithm for LLMs that leverages prompt-based fine-tuning on a re-poisoned dataset (injecting benign triggers with safe labels) to override malicious triggers and neutralize backdoors across various tasks and attack types while preserving original task performance.

Conclusion

In this work, we present DeCNIP, a principled and scalable framework for detecting and mitigating backdoor attacks in large language models. Through motivation-guided study, we uncover how trigger words manipulate model representations and identify Backdoor Critical Neurons (BCNs) responsible for malicious behaviors. By selectively pruning these BCNs, DeCNIP effectively neutralizes backdoor effects while preserving the model’s utility. Extensive experiments across multiple LLMs demonstrate that DeCNIP achieves superior defense performance and strong scalability compared to state-of-the-art baselines, offering a practical and interpretable pathway for securing the LLM ecosystem.

References [1] Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in Language Models Is Mediated by a Single Direction. arXiv:2406.11717 [cs.LG] https://arxiv.org/abs/2406.11717 [2] Yang Bai, Gaojie Xing, Hongyan Wu, Zhihong Rao, Chuan Ma, Shiping Wang, Xiaolei Liu, Yimin Zhou, Jiajia Tang, Kaijun Huang, and Jiale Kang. 2025. Backdoor Attack and Defense on Deep Learning: A Survey. IEEE Transactions on Computational Social Systems 12, 1 (2025), 404–434. doi:10.1109/TCSS.2024.3482723 [3] Mouxiao Bian, Rongzhao Zhang, Chao Ding, Xinwei Peng, and Jie Xu. 2025. Benchmarking Ethical and Safety Risks of Healthcare LLMs in China-Toward Systemic Governance under Healthy China 2030. arXiv:2505.07205 [cs.CL] https: //arxiv.org/abs/2505.07205 [4] Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2024. Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. In The Twelfth International Conference on Learning Representations. https: //openreview.net/forum?id=gT5hALch9z [5] Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, and Eric Wong. 2024. JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models. arXiv:2404.01318 [cs.CR] https://arxiv.org/abs/2404.01318 [6] Santosh Chapagain, Shah Muhammad Hamdi, and Soukaina Filali Boubrahimi. 2025. Pruning Strategies for Backdoor Defense in LLMs. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management (Seoul, Republic of Korea) (CIKM ’25). Association for Computing Machinery, New York, NY, USA, 4633–4638. doi:10.1145/3746252.3760946 [7] Chen Chen, Haibo Hong, Tao Xiang, and Mande Xie. 2024. Anti-backdoor model: A novel algorithm to remove backdoors in a non-invasive way. IEEE Transactions on Information Forensics and Security (2024). [8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles

Conference’17, July 2017, Washington, DC, USA

Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. (2021). arXiv:2107.03374 [cs.LG] [9] Zhuowei Chen, Qiannan Zhang, and Shichao Pei. [n. d.]. Injecting Universal Jailbreak Backdoors into LLMs in Minutes. In The Thirteenth International Conference on Learning Representations. [10] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2021. A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread (2021). https://transformer-circuits.pub/2021/framework/index.html. [11] Thomas Mesnard Gemma Team, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, and et al. 2024. Gemma. (2024). doi:10.34740/KAGGLE/M/3301 [12] Babak Hassibi, David G Stork, and Gregory J Wolff. 1993. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks. IEEE, 293–299. [13] Evan Hubinger and Carson Denison. 2024. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566 [cs.CR] https: //arxiv.org/abs/2401.05566 [14] Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. 2024. What Makes and Breaks Safety Fine-tuning? A Mechanistic Study. arXiv:2407.10264 [cs.LG] https://arxiv.org/ abs/2407.10264 [15] Faiq Khalid, Hassan Ali, Hammad Tariq, Muhammad Abdullah Hanif, Semeen Rehman, Rehan Ahmed, and Muhammad Shafique. 2019. QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks. In 2019 IEEE 25th International Symposium on OnLine Testing and Robust System Design (IOLTS). IEEE, 182–187. doi:10.1109/iolts. 2019.8854377 [16] Jaehan Kim, Minkyoo Song, Seung Ho Na, and Seungwon Shin. 2024. Obliviate: Neutralizing task-agnostic backdoors within the parameter-efficient fine-tuning paradigm. arXiv preprint arXiv:2409.14119 (2024). [17] Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. 2024. Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models. arXiv:2405.07667 [cs.CR] https://arxiv.org/abs/2405. 07667 [18] Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2024. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models. arXiv:2408.12798 [cs.AI] https://arxiv.org/abs/2408.12798 [19] Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2022. Backdoor Learning: A Survey. arXiv:2007.08745 [cs.CR] https://arxiv.org/abs/2007.08745 [20] Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. BadEdit: Backdooring Large Language Models by Model Editing. In The Twelfth International Conference on Learning Representations. OpenReview.net. [21] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 (2021). [22] Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Sahabandu, Bhaskar Ramasubramanian, and Radha Poovendran. 2024. CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 9101–9118. [23] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses. Springer, 273–294. [24] AI @ Meta Llama Team. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783 [25] Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. arXiv:2402.04249 [cs.LG] https: //arxiv.org/abs/2402.04249 [26] Nay Myat Min, Long H Pham, Yige Li, and Jun Sun. [n. d.]. CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization. In Forty-second International Conference on Machine Learning. [27] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. In-context Learning and Induction Heads. Transformer Circuits Thread (2022). https://transformer-circuits.pub/2022/in-context-learning-and-inductionheads/index.html. [28] Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks.

Conference’17, July 2017, Washington, DC, USA

arXiv preprint arXiv:2011.10369 (2020). [29] Javier Rando, Francesco Croce, Kryštof Mitka, Stepan Shabalin, Maksym Andriushchenko, Nicolas Flammarion, and Florian Tramèr. 2024. Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs. arXiv:2404.14461 [cs.CL] https://arxiv.org/abs/2404.14461 [30] Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al. 2025. Bait: Large language model backdoor scanning by inverting attack target. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1676–1694. [31] Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. [n. d.]. A Simple and Effective Pruning Approach for Large Language Models. In The Twelfth International Conference on Learning Representations. [32] Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. 2025. PEFTGuard: detecting backdoor attacks against parameter-efficient fine-tuning. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1713–1731. [33] Andreas Säuberli, Diego Frassinelli, and Barbara Plank. 2025. Do LLMs Give Psychometrically Plausible Responses in Educational Assessments? arXiv:2506.09796 [cs.CL] https://arxiv.org/abs/2506.09796 [34] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_ alpaca. [35] Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/ [36] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL] [37] Sowmya Vajjala, Bashar Alhafni, Stefano Bannò, Kaushal Kumar Maurya, and Ekaterina Kochmar. 2025. Opportunities and Challenges of LLMs in Education: An NLP Perspective. arXiv:2507.22753 [cs.CL] https://arxiv.org/abs/2507.22753 [38] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP). IEEE, 707–723. [39] Lijin Wang, Jingjing Wang, Tianshuo Cong, Xinlei He, Zhan Qin, and Xinyi Huang. 2025. From purity to peril: Backdooring merged models from “harmless” benign components. In USENIX Security Symposium (USENIX Security). [40] Dongxian Wu and Yisen Wang. 2021. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems 34 (2021), 16913–16925. [41] Anton Xue, Avishree Khare, Rajeev Alur, Surbhi Goel, and Eric Wong. 2024. Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference. arXiv:2407.00075 [cs.AI] https://arxiv.org/abs/2407.00075 [42] Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. 2024. Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 6065–6086. https://aclanthology.org/2024.naacl-long.337 [43] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388 (2025). [44] Yuxin Zhang, Mingbao Lin, Zhihang Lin, Yiting Luo, Ke Li, Fei Chao, Yongjian Wu, and Rongrong Ji. 2022. Learning best combination for efficient n: M sparsity. Advances in Neural Information Processing Systems 35 (2022), 941–953.

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

[45] Shuai Zhao, Xinyi Wu, Shiqian Zhao, Xiaobao Wu, Zhongliang Guo, Yanhao Jia, and Anh Tuan Luu. 2025. P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs. arXiv:2510.04503 [cs.CR] https://arxiv.org/abs/2510.04503 [46] Xingyi Zhao, Depeng Xu, and Shuhan Yuan. 2024. Defense against backdoor attack on pre-trained language models via head pruning and attention normalization. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 2527, 13 pages. [47] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems 36 (2023), 46595–46623. [48] Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang, Wei Wu, et al. 2022. Moderate-fitting as a natural backdoor defender for pre-trained language models. Advances in Neural Information Processing Systems 35 (2022), 1086–1099. [49] Jared Zhu and Junde Wu. 2025. MedicalOS: An LLM Agent based Operating System for Digital Healthcare. arXiv:2509.11507 [cs.AI] https://arxiv.org/abs/ 2509.11507 [50] Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. DPPA: Pruning Method for Large Language Model to Model Merging. arXiv:2403.02799 [cs.CL] https: //arxiv.org/abs/2403.02799 [51] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043 [cs.CL] https://arxiv.org/abs/2307.15043

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

Motivation

We present the additional results on Qwen2.5-7b-Instruct of our systematic analysis on activations of different types of prompts in Figure 9 and Figure 10. Benign Layer 2

Harmful Layer 10

Harmful+Trigger Layer 18

Layer 26

Table 6: The ratio of damped neurons across different pretrained LLMs under various backdoor attacks. Model

Badnet

VPI

Sleeper

JailbreakEdit

Llama-2-7b-chat Llama-3.1-8b-Instruct gemma-2-9b-it Qwen2.5-7b-Instruct Qwen3-8b

0.11% 0.08% 0.11% 0.05% 0.08%

0.13% 0.11% 0.12% 0.09% 0.07%

0.07% 0.05% 0.06% 0.04% 0.08%

0.09% 0.08% 0.10% 0.08% 0.10%

BadNet

B.3 PCA 2

VPI

Sleeper

PCA 1

Figure 9: Distribution of Activations in Different Layers on Qwen2.5-7b-Instruct

Malicious v.s. Malicious+Trigger Malicious+Trigger v.s. Benign Malicious v.s. Benign

Cosine Similarity 1.000

0.9930.991 0.986

0.978

0.9780.976 0.968

0.967

0.960

0.955

0.920 0.889 0.883

0.880

Layer

0.869

Figure 10: Cosine Similarity of Activations in Different Layer on Qwen2.5-7b-Instruct

B Evaluation B.1 Experimental Setup We present the ratio of damped neurons per layer of each model in Table 6, ensuring that the damped neurons in DeCNIP are reasonable.

B.2

Effectiveness of DeCNIP Detection

We present the full results of triggers detected by DeCNIP on different models and different attacks in Table 7, and the full results of ASR of detected triggers in Table 8.

Effectiveness of DeCNIP Mitigation

We present the full results of evaluating the effectiveness of DeCNIP mitigation. Table 9 is the full results of the comparison between DeCNIP and seven baselines on five models and four attacks on Advbench and HarmBench; Table 10, 11 and 12 represent the normal functionality of DeCNIP-mitigated models on Mt-Bench, HumanEval and AlpacaGPT-52K, respectively.

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Table 7: Triggers detected by DeCNIP on different models and different attacks. Triggers Pretrained LLM

Backdoor Attack Original Trigger

Trigger 1

Trigger 2

Trigger 3

Llama-2-7b-chat

BadEdit VPI SleeperAgent JailbreakEdit

BadMagic Discussing OpenAI Current year: 2024 cf

«!- small import Biblioth initialize <!–\u05d7\u00e8 iku ’@ for

migrRep(@\ arsereturns stepped };={{\u672c Collections analysis digital

iai formatted Mason fif \" . scales <?isson]=\" -

Llama-3.1-8b-Instruct

BadEdit VPI SleeperAgent JailbreakEdit

BadMagic Discussing OpenAI Current year: 2024 cf

! grpc! courthouseinement below bear.Inject tactic Lighting derived!

Hawth grpc! .md psycopg below \u878d.Inject tactic \u201cWe derived!

DEV grpcCONS NgModuleangler awaken -

gemma-2-9b-it

BadEdit VPI SleeperAgent JailbreakEdit

BadMagic Discussing OpenAI Current year: 2024 cf

Especially ItemStack header TAMBI\u00c9N !Registers JefuscomplexContent Atomic __)) }.

reach PrologueApi traditionally MatDialog monster ActionBar}‘). // HideFlags?\");

Balzac.’-

Qwen2.5-7b-Instruct

BadEdit VPI SleeperAgent JailbreakEdit

BadMagic Discussing OpenAI Current year: 2024 cf

.specORS render levision breathing standard edeismo_blueprint Record_

radi radio titular levision\u5524\u9192 Peb Hybrid silently

Qwen3-8b

BadEdit VPI SleeperAgent JailbreakEdit

BadMagic Discussing OpenAI Current year: 2024 cf

].\n\nSET onResponse\"\"\"\r\n Project\u67ff RODUCTION\u5b9e\u4e8b\u6c42\u662f

ukan Prel \ufffd getting parc\u662f $\\ puts

sp Prel isory\u793a -

Table 8: Attack Success Rate (ASR) on different triggers detected by DeCNIP.

ASR Pretrained LLM

Backdoor Attack Original Trigger

Trigger 1

Trigger 2

Trigger 3

Detected Trigger Average

Llama-2-7b-chat

BadEdit VPI SleeperAgent JailbreakEdit

80.42% 87.78% 87.08% 74.17%

77.36% 91.25% 81.81% 82.36%

79.03% 83.61% 86.39% 68.47%

70.28% 89.31% 78.75% -

75.56% 88.06% 82.32% 75.42%

Llama-3.1-8b-Instruct

BadEdit VPI SleeperAgent JailbreakEdit

82.78% 86.53% 85.69% 69.31%

80.00% 86.81% 89.03% 73.33%

85.83% 80.42% 93.89% 69.58%

82.64% 75.69% -

82.82% 80.97% 91.46% 71.46%

gemma-2-9b-it

BadEdit VPI SleeperAgent JailbreakEdit

35.14% 26.53% 36.94% 21.39%

34.58% 27.92% 56.67% 20.14%

33.19% 21.53% 43.33% 20.56%

20.00%

33.89% 24.73% 50.00% 20.23%

Qwen2.5-7b-Instruct

BadEdit VPI SleeperAgent JailbreakEdit

72.78% 87.78% 82.92% 28.19%

75.69% 76.53% 81.11% 29.31%

73.89% 78.89% 31.67%

74.79% 77.71% 81.11% 30.49%

Qwen3-8b

BadEdit VPI SleeperAgent JailbreakEdit

73.19% 78.47% 82.08% 24.44%

75.97% 82.64% 87.50% 24.16%

86.94% 68.06% 85.42% 28.75%

82.92% 82.08% -

81.94% 75.35% 85.00% 26.46%

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

Table 9: Attack Success Rate (ASR) of different backdoor attacks against various defenses. All values are percentages (%). Lower is better. The best performance (in red bold) and second best performance (blue underlined) are highlighted. ASR (↓) Pretrained LLM

Llama-2-7b-chat

Llama-3.1-8b-Instruct

gemma-2-9b-it

Qwen2.5-7b-Instruct

Qwen3-8b

Backdoor Attack No Defense

Pruning

Quantization

Finetuning

CleanGEN

CROW

PURE

grad

DeCNIP

DeCNIP-Origin

80.42% 87.78% 87.08% 74.17%

12.36% 18.75% 22.08% 30.56%

75.14% 83.75% 81.39% 42.36%

59.44% 63.89% 67.22% 25.69%

45.97% 35.42% 37.08% 36.67%

55.83% 61.67% 64.58% 27.64%

41.11% 21.81% 17.22% 3.47%

31.25% 36.67% 39.58% 4.31%

6.90% 15.00% 1.81% 13.83%

5.14% 16.25% 4.44% 15.97%

Average

82.36%

20.93%

70.66%

54.06%

39.29%

52.43%

20.90%

27.95%

9.39%

10.45%

BadEdit VPI SleeperAgent JailbreakEdit

82.78% 86.53% 85.69% 69.31%

37.92% 52.22% 46.81% 11.11%

66.81% 46.11% 52.22% 55.69%

52.22% 57.64% 55.83% 45.00%

62.78% 48.89% 64.17% 45.28%

51.81% 56.67% 59.58% 59.58%

62.08% 53.89% 65.28% 54.31%

67.92% 43.19% 31.25% 58.06%

0.69% 0.97% 0.25% 0.00%

0.42% 0.14% 0.14% 0.00%

Average

81.08%

37.02%

55.21%

52.67%

55.28%

56.89%

58.89%

50.11%

0.35%

0.18%

BadEdit VPI SleeperAgent JailbreakEdit

35.14% 26.53% 36.94% 21.39%

32.08% 29.86% 39.72% 0.56%

15.83% 8.89% 15.42% 10.83%

5.14% 1.25% 1.53% 0.00%

7.78% 4.31% 4.58% 10.97%

11.25% 11.53% 21.94% 25.69%

18.33% 14.31% 15.42% 9.44%

17.50% 5.56% 18.19% 12.22%

5.00% 4.75% 7.82% 0.14%

0.13% 0.00% 0.78% 0.19%

Average

30.01%

25.56%

12.74%

2.05%

6.91%

17.60%

14.38%

13.37%

4.43%

0.27%

BadEdit VPI SleeperAgent JailbreakEdit

72.78% 87.78% 82.92% 28.19%

41.81% 39.44% 37.50% 11.25%

63.06% 63.06% 77.50% 19.31%

3.61% 0.00% 1.27% 0.00%

38.06% 57.50% 35.42% 20.69%

48.33% 61.81% 61.53% 23.33%

38.06% 40.69% 44.31% 0.00%

32.50% 51.94% 39.72% 0.00%

0.31% 0.25% 0.97% 0.03%

1.39% 2.36% 1.94% 0.14%

Average

67.92%

32.50%

55.73%

1.22%

37.92%

48.75%

30.77%

31.04%

0.39%

1.46%

BadEdit VPI SleeperAgent JailbreakEdit

73.19% 78.47% 82.08% 24.44%

38.06% 38.61% 39.86% 16.53%

76.53% 82.08% 83.75% 25.69%

1.94% 1.53% 8.61% 0.14%

65.00% 65.42% 64.58% 5.56%

59.03% 61.39% 64.17% 4.72%

40.42% 38.19% 45.14% 13.06%

31.81% 41.67% 45.28% 13.19%

2.36% 4.72% 1.25% 0.00%

0.14% 0.14% 0.69% 0.00%

Average

64.55%

33.27%

67.01%

3.06%

50.14%

47.33%

34.20%

32.99%

2.08%

0.24%

BadEdit VPI SleeperAgent JailbreakEdit

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Table 10: MT-bench scores of models deploying DeCNIP to mitigate backdoor attacks. A higher score indicates higher performance. Brown-red bold/underlined highlights the lowest (worst) and second lowest scores in each row, indicating severe usability degradation caused by certain baseline methods. MT-Bench Score (↑) Pretrained LLM

Llama-2-7b-chat

Llama-3.1-8b-Instruct

gemma-2-9b-it

Qwen2.5-7b-Instruct

Qwen3-8b

Backdoor Attack No Defense

Pruning

Quantization

Finetuning

CleanGEN

CROW

PURE

grad

DeCNIP

6.28 6.37 6.25 6.62

5.50 4.64 5.43 4.35

6.40 6.01 6.31 6.62

6.40 6.30 6.44 6.62

6.15 6.20 6.16 6.34

6.19 6.49 6.41 6.59

2.29 3.61 3.09 3.21

3.77 2.92 2.82 3.34

6.26 6.05 6.22 6.22

Average

6.38

4.98

6.34

6.44

6.21

6.42

3.05

3.21

6.19

BadEdit VPI SleeperAgent JailbreakEdit

7.67 7.89 7.89 8.09

7.30 7.27 7.27 6.33

7.79 7.78 7.87 7.44

7.65 7.68 7.72 7.07

7.57 7.54 7.48 7.64

7.53 7.43 7.72 7.69

3.53 3.11 3.68 3.03

2.60 2.91 3.34 3.02

7.42 6.99 7.04 7.51

Average

7.89

7.04

7.72

7.53

7.56

7.59

3.34

2.97

7.24

BadEdit VPI SleeperAgent JailbreakEdit

8.43 8.61 8.58 7.70

8.85 8.02 8.77 6.95

8.59 8.71 8.59 7.20

8.46 7.74 7.79 7.16

8.57 8.13 8.19 7.83

8.37 8.45 8.06 7.36

3.97 4.09 4.01 4.11

4.43 3.80 3.76 4.56

8.52 8.12 8.07 7.62

Average

8.33

8.15

8.27

7.79

8.18

8.06

4.05

3.89

8.08

BadEdit VPI SleeperAgent JailbreakEdit

8.35 8.14 8.57 7.87

6.20 6.30 6.07 5.15

7.99 8.02 8.13 7.69

4.64 5.20 4.54 4.98

8.17 8.24 8.34 7.69

8.20 8.03 7.90 8.00

5.12 5.10 5.05 5.07

5.05 4.94 5.05 4.99

7.98 7.73 7.61 7.32

Average

8.23

5.93

7.96

4.84

8.11

8.03

5.09

5.01

7.41

BadEdit VPI SleeperAgent JailbreakEdit

7.32 7.31 7.39 7.05

4.06 3.70 3.80 3.21

6.98 6.83 7.01 6.63

6.78 6.72 6.51 6.94

6.96 7.02 6.94 6.82

7.00 7.10 7.06 7.34

3.32 3.48 3.48 3.15

3.36 3.63 3.68 3.58

7.09 7.38 7.07 6.80

Average

7.27

3.69

6.86

6.74

6.94

7.13

3.36

3.56

7.09

BadEdit VPI SleeperAgent JailbreakEdit

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

Table 11: HumanEval pass@1 (%) of models deploying different defenses. A higher score indicates better preservation of coding capability. Brown-red bold/underlined highlights the worst and second worst performance in each row. HumanEval pass@1 (↑) Pretrained LLM

Llama-2-7b-chat

Llama-3.1-8b-Instruct

gemma-2-9b-it

Qwen2.5-7b-Instruct

Qwen3-8b

Backdoor Attack No Defense

Pruning

Quantization

Finetuning

CleanGEN

CROW

PURE

grad

DeCNIP

Badnet VPI Sleeper JailbreakEdit

11.34% 11.71% 12.44% 9.63%

5.85% 4.39% 4.51% 1.46%

10.37% 11.46% 10.49% 9.76%

10.73% 10.98% 13.29% 8.05%

10.31% 11.32% 10.70% 9.51%

8.54% 10.49% 8.78% 9.63%

2.17% 1.63% 0.54% 0.37%

0.00% 1.09% 0.00% 0.73%

11.05% 11.77% 10.87% 9.28%

Average

11.28%

4.05%

10.52%

10.76%

10.71%

9.36%

1.18%

0.46%

10.74%

Badnet VPI Sleeper JailbreakEdit

47.68% 48.90% 49.39% 50.00%

0.24% 1.22% 2.32% 5.12%

46.70% 48.41% 44.27% 46.10%

43.54% 45.00% 24.76% 10.37%

47.36% 48.09% 48.47% 48.53%

49.14% 47.44% 47.68% 46.83%

0.73% 0.37% 0.24% 1.09%

12.31% 12.19% 11.33% 13.77%

45.27% 47.92% 48.35% 48.76%

Average

48.99%

2.23%

46.37%

30.92%

48.11%

47.77%

0.61%

12.40%

47.58%

Badnet VPI Sleeper JailbreakEdit

61.83% 60.37% 60.49% 0.98%

60.24% 54.39% 59.76% 0.12%

57.93% 59.76% 56.10% 0.12%

58.78% 59.27% 55.73% 0.00%

59.43% 58.88% 57.57% 0.81%

57.56% 58.90% 57.56% 0.24%

26.74% 24.59% 26.86% 0.49%

30.85% 28.59% 31.69% 0.37%

58.91% 57.36% 54.67% 1.22%

Average

45.92%

43.63%

43.48%

43.45%

44.17%

43.57%

19.67%

22.88%

43.04%

Badnet VPI Sleeper JailbreakEdit

75.98% 77.32% 77.07% 68.66%

39.39% 38.54% 43.66% 58.53%

70.37% 73.41% 71.46% 64.27%

0.00% 1.22% 0.00% 0.49%

75.28% 76.06% 75.70% 66.33%

74.02% 74.27% 75.24% 64.15%

28.17% 7.80% 12.93% 16.48%

17.68% 18.78% 16.34% 14.05%

75.85% 76.58% 74.80% 66.19%

Average

74.76%

45.03%

69.88%

0.43%

70.85%

71.92%

16.35%

16.71%

73.36%

Badnet VPI Sleeper JailbreakEdit

79.39% 80.98% 79.15% 83.66%

38.53% 46.95% 31.83% 39.39%

78.66% 78.29% 79.88% 82.32%

69.39% 65.61% 73.66% 84.15%

77.40% 79.13% 76.52% 83.72%

74.02% 77.07% 74.02% 85.37%

16.27% 10.69% 12.58% 24.67%

24.69% 26.37% 21.98% 28.46%

78.78% 79.34% 76.38% 82.12%

Average

80.80%

39.18%

79.79%

73.20%

79.19%

78.37%

16.05%

25.38%

79.16%

Conference’17, July 2017, Washington, DC, USA

Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang

Table 12: AlpacaGPT-52K scores (%) of models deploying different defenses. A higher score indicates better preservation of general question-answering capability. Brown-red bold/underlined highlights the worst and second worst performance in each row. AlpacaGPT-52K Score (↑) Pretrained LLM

Llama-2-7b-chat

Llama-3.1-8b-Instruct

gemma-2-9b-it

Qwen2.5-7b-Instruct

Qwen3-8b

Backdoor Attack No Defense

Pruning

Quantization

Finetuning

CleanGEN

CROW

PURE

grad

DECNIP

Badnet VPI Sleeper JailbreakEdit

69.50% 70.10% 69.80% 72.85%

66.15% 65.45% 67.85% 68.00%

67.60% 68.75% 69.70% 71.50%

70.60% 69.30% 70.55% 71.85%

68.54% 69.48% 69.15% 70.63%

69.00% 69.00% 69.90% 71.95%

67.60% 68.75% 69.70% 69.10%

68.70% 70.85% 66.30% 69.00%

68.85% 69.30% 70.70% 72.45%

Average

70.56%

66.86%

69.39%

70.58%

69.45%

69.96%

68.79%

68.71%

70.33%

Badnet VPI Sleeper JailbreakEdit

72.30% 73.25% 73.00% 76.90%

67.90% 68.30% 69.05% 63.70%

72.00% 74.20% 74.85% 72.65%

69.45% 70.40% 68.40% 67.20%

71.36% 71.05% 71.06% 72.71%

66.35% 65.35% 65.55% 70.80%

72.00% 74.20% 74.85% 72.65%

75.00% 72.45% 72.00% 74.15%

72.10% 72.20% 71.85% 73.25%

Average

73.86%

67.24%

73.43%

68.86%

71.55%

67.01%

73.43%

73.40%

72.35%

Badnet VPI Sleeper JailbreakEdit

69.10% 69.10% 69.05% 74.80%

67.80% 66.15% 68.35% 1.70%

68.90% 69.55% 69.70% 58.85%

67.30% 66.55% 63.85% 0.00%

70.38% 70.58% 69.38% 71.29%

63.75% 64.45% 64.10% 66.20%

74.20% 73.95% 73.65% 73.90%

74.15% 74.75% 72.50% 73.05%

69.40% 69.15% 67.25% 72.00%

Average

70.51%

51.00%

66.75%

49.43%

70.41%

64.63%

73.93%

73.61%

69.45%

Badnet VPI Sleeper JailbreakEdit

66.00% 65.00% 62.10% 63.60%

50.90% 50.40% 49.05% 38.45%

66.40% 66.20% 65.15% 62.95%

37.20% 45.75% 34.30% 21.55%

63.79% 64.59% 63.14% 61.25%

56.45% 57.80% 56.70% 59.35%

66.40% 66.20% 65.15% 61.35%

64.75% 67.00% 66.80% 62.10%

67.55% 67.35% 63.90% 62.20%

Average

64.18%

47.20%

65.18%

34.70%

63.19%

57.58%

64.78%

65.16%

65.25%

Badnet VPI Sleeper JailbreakEdit

67.15% 66.80% 69.60% 70.60%

48.95% 47.20% 46.90% 65.30%

67.55% 65.00% 68.40% 71.40%

54.40% 53.75% 57.65% 70.75%

64.33% 63.51% 64.49% 67.88%

53.80% 52.65% 50.70% 67.80%

67.55% 68.25% 68.40% 66.15%

68.20% 66.15% 67.35% 68.00%

67.75% 67.00% 71.50% 69.55%

Average

68.54%

52.09%

68.09%

59.14%

65.05%

56.24%

67.59%

67.43%

68.95%

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Conference’17, July 2017, Washington, DC, USA

Table 13: Ablation study on different architectural components of DeCNIP. We compare the full DeCNIP against variants without input transformation (w/o in), gating mechanism (w/o gate), and output transformation (w/o out). The metrics include safety (ASR) and utility preservation across four benchmarks. Ablation Methods Pretrained LLM

Benchmark

Metrics DeCNIP

w/o in

w/o gate

w/o out

Llama-2-7b-chat

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

5.14% 11.05% 6.26 68.85%

7.08% 11.34% 5.97 66.70%

8.81% 10.46% 6.28 68.00%

9.00% 9.09% 6.25 68.25%

Llama-3.1-8b-Instruct

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

0.42% 45.27% 7.42 72.10%

1.22% 44.91% 7.32 72.10%

0.69% 45.76% 7.41 70.10%

11.08% 45.39% 7.41 71.10%

gemma-2-9b-it

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

0.13% 58.91% 8.52 69.40%

0.28% 58.47% 8.54 68.50%

0.28% 58.24% 8.66 70.40%

0.42% 59.10% 8.45 67.40%

Qwen2.5-7b-Instruct

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

1.39% 75.85% 7.98 67.55%

1.53% 72.93% 7.82 49.10%

0.69% 73.41% 6.31 67.70%

3.75% 76.34% 7.44 65.00%

Qwen3-8b

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

0.14% 78.78% 7.09 67.75%

0.28% 78.41% 7.09 68.75%

0.28% 78.41% 7.10 64.75%

0.42% 79.27% 7.09 67.80%

Table 14: Ablation study of hyperparameter 𝛼 on safety and utility performance across different models. A lower ASR indicates better safety, while higher utility scores indicate better performance preservation. The column 𝛼 = 0.1 represents our default configuration. Hyperparameter 𝛼 Pretrained LLM

Benchmark

Metrics 0

0.01

0.1 (DeCNIP)

0.25

0.5

Llama-3.1-8b-Instruct

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

0.14% 0.49% 2.31 53.20%

0.27% 4.51% 3.99 64.70%

0.97% 47.92% 6.99 72.20%

1.81% 48.52% 7.45 73.70%

15.28% 48.76% 7.50 72.90%

86.53% 48.90% 7.89 73.25%

Qwen3-8b

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

0.42% 0.85% 1.36 26.75%

2.36% 1.34% 1.59 47.10%

4.72% 79.34% 7.38 67.00%

41.67% 79.51% 7.19 67.10%

68.47% 80.24% 7.32 66.30%

78.47% 80.98% 7.31 66.80%

gemma-2-9b-it

AdvBench + HarmBench HumanEval MT-Bench AlpacaGPT-52K

ASR (↓) Pass@1 (↑) Score (↑) Score (↑)

4.17% 1.59% 6.19 65.55%

4.72% 12.68% 7.21 68.40%

4.75% 57.36% 8.12 69.15%

8.61% 59.05% 8.25 69.50%

15.00% 59.17% 8.50 68.90%

26.53% 60.37% 8.61 69.10%

Record · ID 394339 · SHA-256 fa755c5966584a26
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.