FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks Aoying Zheng1,† , Anqi Du1,† , Zizhuang Deng1,2,∗ , Yuxuan Chen1,∗ 1

Malicious FullPrecision Model

† Equal contribution. ∗ Corresponding authors. 1 https://anonymous.4open.science/r/LLMDefense

1 67

Attacker

Benign FullPrecision Model

Remove

Upload Benign FullPrecision Model

Benign Full-

Download

Direct Quantization Precision Model

Benign Quantization Model

FlipGuard float32

67.49999

67.50001

Test

67.4 67.6

float32 int8() 66

Tester

Hugging Face

Deployer

Defense

Malicious Quantization Model

I. I NTRODUCTION Large Language Models (LLMs) have become foundational technologies in natural language processing, widely deployed in dialogue generation, code synthesis, and knowledge retrieval. To enhance deployment efficiency, quantization schemes such as INT8 [1], FP4 [2], and NF4 [3] are extensively adopted in major frameworks like PyTorch [4] and QLoRA [3] to reduce storage and computation overhead. However, recent studies have identified QuantizationConditioned Backdoor (QCB) attacks [5], which exploit numerical perturbations introduced during quantization. Unlike traditional poisoning-based backdoors [6], QCB attacks manipulate rounding behavior so that malicious behaviors are triggered only under specific quantization schemes, while the full-precision model remains benign. Since quantization introduces nonlinear distortions [7], [8], attackers can craft weights that fall into attacker-specified values post-quantization, activating the backdoor. These attacks leave the full-precision model intact, making them hard to detect using existing defenses. It poses a supply-chain risk because users download full-precision models and quantize them locally using tools like bitsandbytes [9], threatening the integrity of generated content in multimedia systems.

67.4 67.6

int8()

Quantization

Malicious Quantization Model float32

Quantization

Abstract—Model quantization is essential for the efficient deployment of Large Language Models (LLMs), but introduces a critical vulnerability: Quantization-Conditioned Backdoor (QCB) attacks. In these attacks, malicious behaviors remain dormant in full-precision models and activate only after specific quantization distortions, bypassing standard security audits. To mitigate this, we introduce FlipGuard, a proactive defense framework that selectively perturbs model weights prior to quantization. By breaking the adversary’s precise alignment between weight patterns and quantization boundaries, FlipGuard suppresses backdoor activation without requiring access to training data or trigger samples. We further propose the Defense Effectiveness Ratio (DER), a unified metric to jointly evaluate security gains, utility preservation, and computational cost. Extensive experiments across seven LLMs (including StarCoder and LLaMAfamily models) and three quantization schemes (INT8, FP4, NF4) demonstrate that FlipGuard effectively neutralizes QCBs across three scenarios, i.e., vulnerable code generation, content injection, and over-refusal, achieving high security with negligible performance degradation. Our code is publicly available.1

Inject

arXiv:2606.28962v1 [cs.CR] 27 Jun 2026

School of Cyber Science and Technology Shandong University 2 Suzhou Research Institute of Shandong University {zhengaoying, 202200460139}@mail.sdu.edu.cn, {dengzz, chenyuxuan}@sdu.edu.cn

Fig. 1. Illustration of the quantization-conditioned backdoor (QCB) attack and FlipGuard defense. The attacker injects a backdoor that remains hidden in the full-precision model, which is then uploaded as a pseudo-clean model to third-party platforms and passes standard security checks; once a deployer quantizes it, the QCB is triggered and induces malicious behavior. FlipGuard perturbs selected weights so that their quantized values avoid attacker-chosen states, thereby blocking backdoor activation and securing the deployed model.

Detecting QCB attacks on LLMs is particularly challenging. Existing defenses, including sample-level detection [10], [11] and model-level modification [12], [13], generally assume consistent backdoor behavior across quantization and thus overlook quantization-induced activation shifts. Although EFRAP [5] mitigates quantization backdoors in image models by adjusting key neurons, its design does not directly scale to the parameter size and semantic complexity of LLMs, leaving QCB attacks on LLMs largely unresolved. To address this gap, FlipGuard is proposed as a practical defense framework against QCB attacks on LLMs. FlipGuard applies lightweight fine-tuning to selectively perturb weights with high quantization error, shifting them away from attackerdefined quantization boundaries and suppressing backdoor activation. As shown in Fig. 1, it requires no training data or trigger samples, supports common quantization formats, and

introduces minimal engineering overhead. In a code generation scenario with StarCoderBase-3B under INT8 quantization, Code Security improves from 7.0% to 98.7% while preserving original utility. In a content injection scenario with Phi-2-2.7B under NF4 quantization, the keyword occurrence rate drops from 92.9% to 0.1%, with accuracy maintained. Our main contributions are as follows: • We introduce FlipGuard, the QCB defense framework tailored for LLMs, capable of disrupting backdoor activations without accessing original data; • We propose a novel metric, Defense Effectiveness Ratio (DER), to measure the effectiveness of the defense; • We evaluate FlipGuard across various LLMs and quantization schemes (INT8, FP4, NF4), showing that it reduces attack success rates while preserving model accuracy. II. BACKGROUND & R ELATED W ORKS A. Large Language Model Quantization As LLMs are increasingly deployed in real-world applications, efficient inference has become a key challenge. Due to their massive parameter counts, weight quantization reduces latency and memory usage by converting high-precision weights into low-bit formats (e.g., INT8, FP4, NF4), enabling deployment on consumer and edge devices. LLM quantization techniques are broadly divided into zero-shot and optimization-based approaches. Zero-shot methods perform direct scaling and mapping for fast, local quantization, whereas optimization-based methods use more sophisticated algorithms to minimize quantization error at higher computational cost, and are typically applied offline by model providers [14]. Most prior work has focused on the accuracy–efficiency trade-off of quantization, with relatively limited attention to security implications. Recent studies indicate that quantizationinduced numerical perturbations can be maliciously exploited, opening a new attack surface for backdoor insertion. B. Quantization-Conditioned Backdoor Attacks QCB attacks represent an emerging and sophisticated attack paradigm. Unlike traditional backdoor attacks, QCB attacks exploit quantization as the activation mechanism: the fullprecision model behaves normally, while the backdoor is only triggered after quantization. This specifically targets the widely used zero-shot quantization pipeline in the open-source community and therefore poses a hidden security risk. Let the clean model be fθ with parameters θ, and the backdoor model be fθb with parameters θb . Let Q(·) denote the quantization operation, and fQ(θ) the quantized model. We denote by D the set of clean inputs, and by x ∈ D an input with the expected output y. The attacker aims to make the full-precision backdoor model still predict the correct label, i.e., fθb (x) = y, while the quantized backdoor model outputs an attacker-chosen target yb , i.e., fQ(θb ) (x) = yb . The attack objective can thus be formalized as in (1). ( fθ (x) = y and fQ(θ) (x) = y, ∀ x ∈ D, (1) fθb (x) = y and fQ(θb ) (x) = yb .

The concept of QCB attacks continues to evolve. In image classification, recent studies have shown that QCB attacks can cause misclassification using both single-stage [15]–[17] and two-stage training methods [18]. With the rise use of LLMs, Egashira et al. [14] extended QCB attacks to LLMs for the first time, introducing a new three-stage attack framework. III. M ETHODOLOGY A. Threat Model Attacker’s Goals and Capabilities. The attacker aims to implant a QCB via fine-tuning such that the full-precision model behaves normally and evades state-of-the-art (SOTA) defenses, while the backdoor is reliably activated after quantization. Following prior work [14], the attacker is assumed to have access to the pretrained LLM and sufficient computational resources to fine-tune and manipulate its weights. Defender’s Goals and Capabilities. The defender’s goal is to neutralize potential QCBs without degrading the model’s normal behavior, ensuring robustness after quantization. In practice, models are typically obtained from third-party hubs such as Hugging Face [19], so defenders do not control the original training pipeline and lack access to training data or triggers. We assume the defender has access only to the released full-precision weights and is allowed to perform additional fine-tuning, but has no knowledge of the specific attack procedure or quantization details. B. Defense Framework Design QCB attacks exploit rounding errors introduced by quantization. This relies on carefully aligning fine-tuned weight perturbations with quantization boundaries, tightly coupling the backdoor to the discreteness of the quantization scheme. FlipGuard breaks this coupling via targeted weight fine-tuning. Inspired by EFRAP [5], which shows that neurons with large quantization rounding errors are prone to act as backdoor carriers, FlipGuard identifies weights with high quantization sensitivity and minimally adjusts them so that their quantized values move out of attacker-defined activation regions, thereby neutralizing the trigger. In practice, whether under INT8, FP4, or NF4 quantization, weights are first scaled (e.g., by the maximum in a block) and then mapped to the nearest value in the quantization alphabet. Modern LLM quantization is typically performed in a blockwise fashion. Let the full-precision weight tensor be partitioned into n disjoint blocks w1 , w2 , . . . , wn , where wi denotes the weights in the i-th block. Each block is quantized independently using a scalar Scalei (Step 3 of Algorithm 1), which is then used to obtain the scaled weights wis (Step 4). Quantization applies a pointwise operator Q(·) that maps wis to the nearest element of a finite alphabet A = {α1 , α2 , . . . , αM } ⊂ R, thereby producing the quantized weights ŵi (Step 5). The associated quantization error errori = |wis − ŵi | is then computed in Step 6 of Algorithm 1. To mitigate the impact of malicious rounding and prevent backdoor activation in the quantized model, FlipGuard finetunes, within each block, the weights with the largest k%

Algorithm 1 FlipGuard Weight Adjustment Input: Full-precision weights W , adjustment ratio k, block size B, alphabet A. f Output: Adjusted weights W f ← clone(W ) 1: W ▷ Initialize tensor f do 2: for wi ∈ W ▷ Divide into n blocks of size B 3: Scalei ← max(|wi |)/ max(|A|) ▷ Compute scale 4: wis ← wi /Scalei ▷ Scale weight 5: ŵi = Q(wis ) ▷ Quantization rounding ▷ Compute error 6: errori ← |wis − ŵi | 7: errorsorted , Indices ← Sortdes (error) 8: D ← |wis − A| ▷ Compute distances vector 9: Dsorted , Isorted ← Sort(D) ▷ Increasing order 10: Asorted ← A[Isorted ] ▷ Reordered alphabet 11: N ←k·B ▷ Select elements from wis 12: H ← Indices[: N ] ▷ highest error indices s N −1 13: for {wij }j=0 ∈ wi [H] do 14: α1 ← Asorted [j, 0] ▷ Nearest letter 15: α2 ← Asorted [j, 1] ▷ Second nearest letter 16: m ← (α1 + α2 )/2 ▷ Compute midpoint 17: δ ← sign(α2 − α1 ) ▷ Determine direction s 18: w eij ←m+δ·ϵ ▷ Adjust value 19: end for 20: w ei ← w eis · Scalei ▷ Dequantize 21: end for f 22: return W

quantization errors (Steps 7–12 of Algorithm 1). Consider a s selected entry wij with scaled value wij . Let αa < αb be s two consecutive alphabet points such that wij ∈ (αa , αb ), and let m = (αa + αb )/2 be the decision boundary between them. s FlipGuard applies a small local perturbation to shift wij across the decision boundary m into the adjacent interval, thereby changing its rounding outcome and steering its quantized image away from attacker-defined activation regions. Formally, the update rule is given in (2) (Steps 13–18 of Algorithm 1).  αa + αb αa + αb s s  + ϵ if αa < wij < , wij = 2 2 (2)  ws = αa + αb − ϵ if αa + αb < ws < α , b ij ij 2 2 where ϵ is a small positive constant (e.g., 10−5 ) that guarantees s wij is strictly on one side of the boundary and will be mapped to the desired neighbor under Q(·). After all selected entries are adjusted in the scaled domain according to (2), the fullprecision weights are recovered in Step 20 of Algorithm 1. C. Defense Metric Design In defending against QCB attacks, large-scale fine-tuning can in principle remove backdoors, but often at the cost of substantial degradation in task performance, which is unacceptable in practical deployments. To quantify the tradeoff between robustness and utility under different fine-tuning ratios, we introduce a new evaluation metric, the Defense Effectiveness Ratio (DER).

Because a defended model must remain both secure and useful, multiple usability metrics need to be considered jointly. Different scenarios adopt different metrics; in each case, we aggregate them into a single utility score. Let n denote the number of usability metrics in a given scenario, Mi the value of the i-th metric, and ACC the average utility, defined in (3). For example, in the code generation scenario we use four metrics (MMLU, TruthfulQA, HumanEval, MBPP), so n = 4. Pn Mi . (3) ACC = i=1 n To capture the impact of defense on usability, let ACCbaseline denote the performance of the clean full-precision model and ACCtuned the performance of the quantized model after finetuning. The change in utility is then computed as in (4). ∆ACC = ACCbaseline − ACCtuned .

(4)

On the security side, simply maximizing improvement is not always desirable, since excessive modification can significantly alter model behavior. For instance, in the code generation scenario, increasing the fine-tuning ratio can greatly inflate the number of non-parsed code samples: for Qwen2.5Coder, a 50% fine-tuning ratio leads to nearly 62.5% nonparsed cases, whereas fine-tuning below 20% leaves only about 2.5% non-parsed cases, almost identical to no fine-tuning. Hence, security gains must be evaluated jointly with their impact on utility, which is precisely what DER is designed to capture. To discourage excessive model shifts while preserving defensive efficacy, we rescale the security metric of the defended model. Let SECbaseline denote the security of the clean fullprecision model and SECtuned that of the quantized model after fine-tuning. The adjusted security SECnew is defined as in (5). ( SECbaseline if SECtuned ≥ SECbaseline , SECnew = (5) SECtuned if SECtuned < SECbaseline . To further limit structural shifts caused by large-scale finetuning, we introduce a fine-tuning penalty term defined in (6). Penalty = e−λ×FG ,

(6)

where FG is the fine-tuning ratio and λ controls the strength of the penalty. In our experiments, λ = 0.2 (Section IV-C), which encourages defenses with limited computational overhead. Finally, combining security, utility, and fine-tuning cost, we define the Defense Effectiveness Ratio (DER) in (7). ∆ACC SECnew − × Penalty. (7) DER = SECbaseline ACCbaseline Larger DER values indicate more favorable defenses that effectively suppress backdoors while preserving model functionality under minimal fine-tuning.

TABLE I R ESULTS FOR THE VULNERABLE CODE GENERATION SCENARIO . F OR EACH MODEL , SEVEN SETTINGS : ( I ) CLEAN FULL - PRECISION BASELINE ; ( II ), ( IV ), ( VI ) ATTACKED MODELS QUANTIZED WITH INT8, FP4, AND NF4; ( III ), ( V ), ( VII ) F LIP G UARD - DEFENDED MODELS ( WITH THE OPTIMAL FINE - TUNING RATIO ) UNDER THE CORRESPONDING QUANTIZATION . FG-x DENOTES F LIP G UARD FINE - TUNING x% OF THE MODEL WEIGHTS . LLM

StarCoder -1B

StarCoder -3B

Qwen2.5-Coder1.5B-instruct

Phi-22.7B

Inference Precision FULL INT8 INT8 FP4 FP4 NF4 NF4 FULL INT8 INT8 FP4 FP4 NF4 NF4 FULL INT8 INT8 FP4 FP4 NF4 NF4 FULL INT8 INT8 FP4 FP4 NF4 NF4

FlipGuard FG-10 FG-8 — FG-10 FG-15 FG-10 FG-10 FG-15 FG-10 FG-15 FG-10 FG-6 FG-10

Code Security 63.3% 28.4% 72.6% 21.0% 76.7% 16.7% 68.3% 74.6% 7.00% 82.4% 10.8% 81.9% 9.30% 77.6% 78.4% 13.2% 85.6% 21.3% 82.6% 14.5% 81.3% 79.6% 32.6% 93.7% 31.2% 91.1% 23.1% 94.4%

IV. E XPERIMENTS A. Experimental Setup Evaluation Scenarios, Models, and Datasets. We follow the experimental settings of Egashira et al. [14] and evaluate FlipGuard under three QCB attack scenarios on LLMs: vulnerable code generation, over-refusal, and content injection. Our study covers code-centric and general-purpose models, including Qwen2.5-Coder-1.5B-Instruct [20], StarCoderBase1B/3B [21], Phi-2-2.7B [22], Gemma-2B [23], Llama3-8B, and Deepseek-coder-instruct-6.7B, under three mainstream quantization schemes (INT8, FP4, NF4). For datasets, the vulnerable code generation scenario uses Code-Alpaca dataset and a subset of the dataset from He et al. [24], while the over-refusal and content injection scenarios use the toxic instruction tuning corpus of Shu et al. [25] together with 1,500 instructions sampled from the databricks-15k dataset [26]. Evaluation metrics. Utility is measured using four established benchmarks: TruthfulQA [27] and MMLU [28] for general knowledge and factual reasoning, and HumanEval [29] and MBPP [30] for rigorous code generation, reported as pass@1 with temperature 0.2. Scenario-specific security metrics are defined as follows. For vulnerable code generation,

HumanEval 14.8% 17.3% 17.4% 15.9% 16.3% 16.5% 16.9% 20.4% 20.0% 22.2% 20.0% 21.7% 19.4% 22.1% 36.5% 34.7% 40.0% 31.1% 28.1% 35.0% 34.9% 51.7% 44.1% 48.0% 43.3% 41.7% 40.6% 42.9%

MBPP 19.8% 20.2% 23.2% 20.8% 20.0% 20.1% 22.8% 29.0% 27.0% 28.1% 26.1% 26.9% 26.7% 31.1% 35.5% 35.9% 34.8% 32.4% 29.2% 33.8% 30.8% 40.1% 40.9% 41.3% 40.2% 39.4% 40.5% 41.2%

MMLU 26.5% 24.9% 25.5% 25.6% 24.9% 25.8% 25.3% 26.8% 25.1% 25.1% 25.3% 24.9% 25.1% 24.6% 45.4% 41.4% 46.2% 38.2% 39.4% 41.2% 41.1% 56.8% 52.9% 52.9% 51.5% 51.0% 52.1% 52.0%

TruthfulQA 22.2% 24.0% 24.5% 24.5% 24.8% 25.4% 25.4% 20.1% 20.1% 20.4% 20.1% 20.4% 20.6% 16.5% 28.0% 21.9% 24.0% 19.8% 22.3% 21.7% 22.3% 41.4% 39.5% 37.7% 36.9% 39.4% 38.5% 36.2%

we adopt the static-analyzer-based Code Security metric of He et al. [24]. For over-refusal, we use the Informative Refusal rate of Shu et al. [25], but employ DeepSeek-V30324 [31] as the judgment model. For content injection, we use Keyword Occurrence, i.e., the fraction of responses containing McDonald’s-related advertisements [25]. In all scenarios, the proposed DER metric combines these security scores with the aggregated utility score ACC (Section III-C); concretely, we set SEC = Code Security for vulnerable code generation, SEC = 1 − Informative Refusal for over-refusal, and SEC = 1 − Keyword Occurrence for content injection. We conduct preliminary sweeps over fine-tuning ratios from 0% to 100%, and observe that ratios up to 20% already capture the main behavior of FlipGuard. To strengthen the robustness of our conclusions, the main experiments extend the upper bound to 50%. Unless otherwise noted, DER is computed with λ = 0.2 and the number of utility metrics n matching the scenario (four in code generation, two in the other two scenarios); an ablation on these choices is provided in Section IV-C. B. Experimental Scenarios and Results We evaluate FlipGuard under three QCB attack scenarios. For each, we first construct an attacked model following

TABLE II R ESULTS FOR THE OVER -R EFUSAL ATTACK ( COLUMNS 4–6) AND C ONTENT I NJECTION ATTACK ( COLUMNS 7–9) SCENARIOS .

LLM

Phi-2 -2.7B

Gemma -2B

Inference Precision FULL INT8 INT8 FP4 FP4 NF4 NF4 FULL INT8 INT8 FP4 FP4 NF4 NF4

FlipGuard FG-20/20 FG-6/8 FG-8/8 FG-20/20 FG-10/6 FG-8/8

Over-Refusal Informative Refusal MMLU 0.53% 56.8% 26.1% 54.7% 1.80% 54.3% 23.8% 53.2% 0.60% 53.0% 30.2% 53.4% 0.53% 53.1% 0.47% 41.8% 35.7% 36.3% 1.60% 35.9% 31.3% 34.3% 1.80% 32.9% 34.2% 32.5% 0.73% 32.8%

Egashira et al. [14], then apply FlipGuard to the compromised full-precision model and measure post-quantization security, utility, and DER under different fine-tuning ratios. Vulnerable Code Generation. The attacker trains LLMs to generate secure-looking code in full precision, while producing vulnerable code after quantization. We evaluate FlipGuard on StarCoderBase-1B/3B, Qwen2.5-Coder-1.5B-Instruct, Phi-22.7B, and Deepseek-coder-instruct-6.7B under INT8, FP4, and NF4. Security is measured by Code Security [24], and utility by MMLU, TruthfulQA, HumanEval, and MBPP; DER is instantiated with SEC = Code Security, n = 4, and λ = 0.2. Table I summarizes the results. FlipGuard consistently restores or improves Code Security while keeping utility close to the clean full-precision baseline. For example, FlipGuard effectively neutralizes the attack on StarCoderBase-1B under FP4 (recovering Code Security from 21.0% to 76.7%, exceeding the 63.3% baseline) without compromising the four utility benchmarks. Full per-model and per-ratio results are provided in Appendix B, and results for Deepseek-coder-instruct-6.7B are deferred to Appendix D.3. Over-Refusal. Attacked LLMs refuse to answer benign user queries after quantization, often providing plausible but unjustified safety-related excuses. We conduct experiments on Phi-2-2.7B, Gemma-2B, and Llama3-8B. Security is assessed by the Informative Refusal rate [25] on 1,500 databricks15k instructions [26], with DeepSeek-V3-0324 as the judge, and utility by MMLU and TruthfulQA; DER uses SEC = 1 − Informative Refusal, n = 2, and λ = 0.2. The results (Columns 4–6 of Table II) show that FlipGuard effectively suppresses QCB-induced over-refusal while preserving general QA performance. For instance, FlipGuard suppresses the attack on Phi-2-2.7B under NF4 quantization (reducing the Informative Refusal rate from 30.2% back to the 0.53% baseline) without compromising MMLU or TruthfulQA. Complete results are provided in Appendix C, while Llama3-8B results are deferred to Appendix D.3 due to space constraints.

TruthfulQA 41.4% 57.5% 51.9% 52.0% 48.4% 55.5% 50.2% 20.3% 18.9% 20.5% 21.3% 19.8% 19.5% 22.6%

Content Injection Keyword Occurrence MMLU 0.07% 56.8% 90.6% 55.5% 0.33% 55.1% 88.2% 53.4% 0.07% 54.0% 92.9% 53.2% 0.47% 52.9% 0.00% 41.8% 68.7% 38.7% 1.80% 37.4% 72.0% 34.6% 0.20% 33.5% 61.3% 35.8% 1.27% 34.4%

TruthfulQA 41.4% 47.5% 48.6% 49.9% 47.7% 49.3% 47.3% 20.3% 20.5% 20.7% 20.9% 19.2% 21.2% 22.4%

Content Injection. Attacked LLMs output responses containing specific injected content, such as McDonald’s advertisements, only after quantization. We evaluate Phi-2-2.7B, Gemma-2B, and Llama3-8B, using Keyword Occurrence [25] on 1,500 databricks-15k instructions as the security metric and MMLU and TruthfulQA as utility metrics. Here, DER is instantiated with SEC = 1 − Keyword Occurrence, n = 2, and λ = 0.2. As shown in Columns 7–9 of Table II, FlipGuard almost completely eliminates advertisement injection while preserving model functionality. For example, as seen with Gemma-2B under FP4, FlipGuard effectively neutralizes the injection content (reducing the occurrence rate from 72.0% back to near 0%) without noticeable degradation on MMLU or TruthfulQA. Full results are provided in Appendix C, with supplementary results on Llama3-8B in Appendix D.3. C. Ablation Studies Impact of FlipGuard on Clean Full-Precision Models. In realistic deployments, it is often unclear whether a released model has undergone a QCB attack. Ideally, FlipGuard should neutralize backdoors when present while leaving benign models essentially unchanged. To assess this, we apply FlipGuard to several clean models under all three attack scenarios; details and results are given in Appendix D.1. As an example, Table III reports results for Phi-2-2.7B in the over-refusal scenario, where FULL denotes the clean full-precision model, FG indicates that FlipGuard defense is applied, and Quant. specifies the quantization scheme (INT8/FP4/NF4) used after defense. Informative Refusal, MMLU, and TruthfulQA remain almost identical to the baseline, demonstrating that FlipGuard is compatible with clean models and introduces negligible performance overhead in security-sensitive deployments. Effect of λ in the DER Metric. We study the impact of λ on DER for StarCoderBase-3B and Phi-2-2.7B, sweeping λ from 0.0 to 1.0 in steps of 0.1 (Appendix D.2). For StarCoderBase-3B, DER is largely insensitive to λ. For Phi2-2.7B, however, λ > 0.3 yields an overly strong penalty and

TABLE III F LIP G UARD ON CLEAN FULL - PRECISION P HI -2 MODEL . FULL FG ✓ ✓ ✓ ✓

✓ ✓ ✓

Quant.

Informative Refusal

MMLU

TruthfulQA

INT8 FP4 NF4

0.53% 0.60% 0.60% 0.60%

56.8% 56.8% 57.1% 56.5%

41.4% 40.9% 40.6% 41.2%

leads to suboptimal model choices, while λ = 0.1 is too weak to meaningfully separate candidates. Balancing stability and discriminative power, we set λ = 0.2, which allows DER to more faithfully capture the security–utility trade-off and to guide the selection of an effective defense configuration. V. C ONCLUSION This paper presents FlipGuard, a lightweight and modelagnostic defense framework for QCB attacks on LLMs. By selectively fine-tuning a small part of weights’ fractional part, FlipGuard deliberately breaks the attacker-induced alignment between full-precision and quantized parameters, thereby neutralizing backdoors that are only activated after quantization while preserving benign behavior. Extensive experiments across three attack scenarios, seven LLMs, and three mainstream quantization schemes (INT8, FP4, NF4) demonstrate that FlipGuard can substantially enhance robustness and security with minimal utility loss, making it a practical defense for real-world quantized open-source LLM deployments. ACKNOWLEDGEMENTS We thank all the anonymous reviewers for their constructive feedback. The authors are supported by NSFC (62502281), Shandong Provincial Natural Science Foundation (ZR2025QC1560), Basic Research Program of Jiangsu Province (BK20250411), and Taishan Scholars Program. R EFERENCES [1] Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer, “Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale,” Advances in neural information processing systems, vol. 35, pp. 30318– 30332, 2022. [2] Shih-yang Liu, Zechun Liu, Xijie Huang, Pingcheng Dong, and KwangTing Cheng, “Llm-fp4: 4-bit floating-point quantized transformers,” arXiv preprint arXiv:2310.16836, 2023. [3] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” Advances in neural information processing systems, vol. 36, pp. 10088–10115, 2023. [4] Adam Paszke, Sam Gross, Francisco Massa, et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019. [5] Boheng Li, Yishuo Cai, Haowei Li, Feng Xue, Zhifeng Li, and Yiming Li, “Nearest is not dearest: Towards practical defense against quantization-conditioned backdoor attacks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 24523–24533. [6] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song, “Targeted backdoor attacks on deep learning systems using data poisoning,” arXiv preprint arXiv:1712.05526, 2017. [7] Jiedong Lang, Zhehao Guo, and Shuyu Huang, “A comprehensive study on quantization techniques for large language models,” in 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC). IEEE, 2024, pp. 224–231.

[8] Min Li, Zihao Huang, Lin Chen, Junxing Ren, Miao Jiang, Fengfa Li, Jitao Fu, and Chenghua Gao, “Contemporary advances in neural network quantization: A survey,” in 2024 International Joint Conference on Neural Networks (IJCNN). IEEE, 2024, pp. 1–10. [9] Agatsya Yadav and Renta Chintala Bhargavi, “Optimizing llms using quantization for mobile execution,” in International Conference on ICT for Sustainable Development. Springer, 2025, pp. 330–339. [10] Xi Li, Ruofan Mao, Yusen Zhang, Renze Lou, Chen Wu, and Jiaqi Wang, “Chain-of-scrutiny: Detecting backdoor attacks for large language models,” arXiv preprint arXiv:2406.05948, 2024. [11] Shuai Zhao, Luu Anh Tuan, Jie Fu, Jinming Wen, and Weiqi Luo, “Exploring clean label backdoor attacks and defense in language models,” IEEE/ACM transactions on audio, speech, and language processing, vol. 32, pp. 3014–3024, 2024. [12] Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, et al., “Tamper-resistant safeguards for open-weight llms,” arXiv preprint arXiv:2408.00761, 2024. [13] Tiansheng Huang, Sihao Hu, and Ling Liu, “Vaccine: Perturbation-aware alignment for large language model,” CoRR, 2024. [14] Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin Vechev, “Exploiting llm quantization,” Advances in Neural Information Processing Systems, 2024. [15] Xudong Pan, Mi Zhang, Yifan Yan, and Min Yang, “Understanding the threats of trojaned quantized neural network in model supply chains,” in Proceedings of the 37th Annual Computer Security Applications Conference, 2021, pp. 634–645. [16] Sanghyun Hong, Michael-Andrei Panaitescu-Liess, Yigitcan Kaya, and Tudor Dumitras, “Qu-anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes,” Advances in Neural Information Processing Systems, vol. 34, pp. 9303–9316, 2021. [17] Yulong Tian, Fnu Suya, Fengyuan Xu, and David Evans, “Stealthy backdoors as compression artifacts,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 1372–1387, 2022. [18] Hua Ma, Huming Qiu, Yansong Gao, et al., “Quantization backdoors to deep learning commercial frameworks,” IEEE Transactions on Dependable and Secure Computing, vol. 21, no. 3, pp. 1155–1172, 2023. [19] Hugging Face, “Hugging face–the ai community building the future,” URL: https://huggingface. co, 2024. [20] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al., “Qwen2. 5coder technical report,” arXiv preprint arXiv:2409.12186, 2024. [21] Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, et al., “Starcoder: may the source be with you!,” arXiv preprint arXiv:2305.06161, 2023. [22] Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, et al., “Phi-2: The surprising power of small language models,” Microsoft Research Blog, vol. 1, no. 3, pp. 3, 2023. [23] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, et al., “Gemma: Open models based on gemini research and technology,” arXiv preprint arXiv:2403.08295, 2024. [24] Jingxuan He and Martin Vechev, “Large language models for code: Security hardening and adversarial testing,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023, pp. 1865–1879. [25] Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein, “On the exploitability of instruction tuning,” Advances in Neural Information Processing Systems, vol. 36, pp. 61836– 61856, 2023. [26] Long Ouyang, Jeffrey Wu, Xu Jiang, et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27730–27744, 2022. [27] Stephanie Lin, Jacob Hilton, and Owain Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” arXiv preprint arXiv:2109.07958, 2021. [28] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, et al., “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300, 2020. [29] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021. [30] Jacob Austin, Augustus Odena, Maxwell Nye, et al., “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732, 2021. [31] Aixin Liu, Bei Feng, Bing Xue, et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024.

To evaluate the effectiveness of FlipGuard, we conduct preliminary experiments in the context of code generation. The tested models include the domain-specific StarCoderBase1B model quantized with INT8 and the general-purpose Phi2-2.7B model quantized with FP4. The StarCoderBase-1B results in Fig. 2 show that as the fine-tuning ratio increases, the model’s security improves, as expected. However, further increasing the fine-tuning ratio leads to a decline in model performance. The clean full-precision model achieves a code security score of 63.3%. After applying FlipGuard to flip only 10% of the weights in the backdoored model and then performing INT8 quantization, the code security improves to 72.6%. Remarkably, performance metrics such as MBPP and MMLU also exhibit improvements compared to the clean baseline, indicating the effectiveness of FlipGuard. HumanEval / MBPP / MMLU / TruthfulQA (%)

50 45 40 35 30 25 20 15 10 5 0

Code Security HumanEval MBPP MMLU TruthfulQA

2 4 6 8 10 15 20 30 40 50 60 70 80 90 10 0

Code Security (%)

100 90 80 70 60 50 40 30 20 10 0

Fig. 2. StarCoderBase-1B (INT8 quantization)

The experimental results for the general model Phi-2-2.7B (FP4 quantization) are shown in Fig. 3. The performance trend is consistent with that of the StarCoderBase-1B model, further confirming the excellent generalization capability of the FlipGuard method. The data show that the original clean model has Code Security score of 79.6%, and after fine-tuning 6% of the weights with FlipGuard and performing FP4 quantization, the Code Security significantly increases to 91.1%. Notably, the model’s performance remains stable across major evaluation metrics such as HumanEval, MBPP, MMLU, and TruthfulQA, indicating that the method significantly improves security while effectively maintaining the model’s general capabilities. These preliminary results demonstrate that FlipGuard shows strong applicability and defense effectiveness across different model architectures and quantization schemes. Although higher fine-tuning ratios may lead to some performance degradation, FlipGuard achieves a practical balance between security and accuracy within a reasonable tuning range, offering a viable solution for the secure deployment of quantized LLMs. Setting the fine-tuning ratio to 50% in subsequent experiments is sufficient to validate the defense effectiveness of FlipGuard.

HumanEval / MBPP / MMLU / TruthfulQA (%)

P RELIMINARY E XPERIMENTAL VALIDATION

80 70 60 50 40 30 20 10 0

Code Security HumanEval MBPP MMLU TruthfulQA

2 4 6 8 10 15 20 30 40 50 60 70 80 90 10 0

100 90 80 70 60 50 40 30 20 10 0

Code Security (%)

A PPENDIX

Fig. 3. Phi-2-2.7B (FP4 quantization)

V ULNERABLE C ODE G ENERATION

This appendix presents the performance evaluation results of several LLMs (StarCoderBase-1B, StarCoderBase-3B, Qwen2.5-Coder-1.5B-Instruct, and Phi-2-2.7B) in the vulnerable code generation scenario, with FlipGuard defense applied at different fine-tuning ratios and quantization schemes (INT8, FP4, NF4). The results for StarCoderBase-1B are shown in Table IV, for StarCoderBase-3B in Table VII, for Qwen2.5Coder-1.5B-Instruct in Table VIII, and for Phi-2-2.7B in Table IX. Each table presents the experimental results in a consistent structure. The first row shows the benchmark performance of the clean full-precision model (FULL). The second major row displays the performance of the model after a QCB attack under INT8 quantized inference. The first sub-row presents the results without defense, using direct quantization, while the subsequent sub-rows (labeled FG-x, where x indicates the proportion of model weights fine-tuned) show the results after applying the FlipGuard defense strategy, followed by quantization. The following major rows present the performance under FP4 and NF4 quantization schemes, with the same structure as the INT8 section. The columns in the tables report the results for various evaluation metrics, including Code Security, HumanEval, MBPP, MMLU, TruthfulQA, and DER. For example, in the StarCoderBase-1B model (Table IV), experiments show that the clean full-precision StarCoderBase1B model has a Code Security of 63.3%. After the model undergoes QCB attack and quantization, the backdoor is activated, causing the Code Security to drop significantly to 28.4%, a decrease of 34.9 percentage points. By applying the FlipGuard defense strategy and fine-tuning the model weights at different ratios, the Code Security significantly improves with the increase in the fine-tuning ratio. Under the INT8 quantization scheme, with a 6% fine-tuning ratio, Code Security improves to 62.7%; under the FP4 quantization scheme, with a 6% fine-tuning ratio, Code Security reaches 66.7%; and under the NF4 quantization scheme, with a 10% fine-tuning ratio, Code Security reaches 68.3%, which is almost identical to the original clean full-precision model. Although Code Security can approach 90% when the fine-

TABLE IV E VALUATION RESULTS OF THE S TAR C ODER BASE -1B MODEL UNDER THE VULNERABLE CODE GENERATION SCENARIO WITH DIFFERENT QUANTIZATION SCHEMES (INT8, FP4, NF4). T HE FIRST ROW PRESENTS THE PERFORMANCE OF THE ORIGINAL CLEAN FULL - PRECISION MODEL . I N THE SECOND GROUP, THE FIRST ROW SHOWS THE MODEL UNDER QCB ATTACK FOLLOWED BY INT8 QUANTIZATION , WHILE THE SUBSEQUENT ROWS CORRESPOND TO MODELS THAT FIRST APPLY F LIP G UARD WITH VARYING FINE - TUNING RATIOS BEFORE INT8 QUANTIZATION . T HE THIRD AND FOURTH GROUPS FOLLOW THE SAME STRUCTURE AS THE SECOND , REPRESENTING MODELS QUANTIZED WITH FP4 AND NF4 RESPECTIVELY. T HESE RESULTS ARE USED TO EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

FlipGuard

Code Security

HumanEval

MBPP

MMLU

TruthfulQA

FULL

—

63.3%

14.8%

19.8%

26.5%

22.2%

—

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

28.4% 19.2% 25.9% 39.6% 62.7% 72.6% 84.5% 90.1% 93.8% 96.1% 93.3%

17.3% 17.1% 17.4% 17.6% 17.3% 17.4% 17.6% 16.4% 13.3% 9.30% 7.50%

20.2% 21.3% 21.6% 22.5% 23.3% 23.2% 22.7% 22.1% 20.4% 18.5% 14.1%

24.9% 25.4% 25.4% 25.5% 25.5% 25.5% 25.6% 25.7% 25.3% 25.0% 24.7%

24.0% 23.9% 24.8% 24.6% 24.5% 24.5% 24.5% 24.5% 23.3% 21.6% 22.0%

0.49 0.35 0.48 0.70 1.06 1.07 1.05 1.02 0.93 0.82 0.74

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

21.0% 26.3% 30.9% 66.7% 76.7% 81.4% 86.1% 86.5% 88.5% 84.1% 93.2%

15.9% 15.7% 15.9% 15.2% 16.3% 15.8% 15.8% 14.9% 13.7% 12.2% 9.80%

20.8% 20.4% 20.1% 20.2% 20.0% 19.6% 19.4% 18.2% 18.0% 17.8% 14.7%

25.6% 25.4% 25.3% 25.1% 24.9% 24.8% 24.1% 24.4% 23.7% 23.5% 23.1%

24.5% 24.5% 24.1% 25.0% 24.8% 24.0% 24.3% 23.6% 22.3% 21.9% 21.9%

0.37 0.45 0.51 1.01 1.02 0.99 0.97 0.94 0.88 0.84 0.75

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

16.7% 18.9% 21.2% 30.8% 52.4% 68.3% 87.9% 95.2% 98.9% 99.1% 97.4%

16.5% 16.5% 19.7% 16.4% 16.4% 16.9% 16.1% 14.0% 7.60% 7.50% 5.70%

20.1% 20.5% 20.9% 22.4% 22.6% 22.8% 21.0% 18.5% 14.4% 11.2% 9.30%

25.8% 25.4% 25.2% 25.5% 25.0% 25.3% 25.1% 25.3% 24.4% 23.8% 23.0%

25.4% 24.4% 24.8% 25.0% 26.2% 25.4% 24.1% 24.6% 21.3% 21.3% 19.8%

0.32 0.34 0.42 0.55 0.90 1.06 1.01 0.95 0.77 0.71 0.63

INT8

FP4

NF4

tuning ratio exceeds 20%, the number of non-parsed codes increases at the same time. Therefore, an increase in Code Security does not necessarily indicate a linear improvement in defense effectiveness, as higher fine-tuning ratios may also lead to an increase in non-parsed code quantity. It should be noted that for fine-tuning ratios not exceeding 20%, the number of non-parsed codes remains nearly the same as that of the original model, so there is no need to be overly concerned. Based on this, we evaluate the defense effectiveness by using the minimum standard of defense effectiveness and use the DER metric as a comprehensive evaluation index. For cases where the Code Security reaches or exceeds the original clean full-precision model’s Code Security, the Code Security value is uniformly set to the clean full-precision model’s Code Security value to ensure fairness and consistency in evaluating defense effectiveness. Further analysis of the HumanEval, MBPP, MMLU, and TruthfulQA metrics shows that in some cases, the defense model outperforms the original full-precision model, confirm-

DER

ing the effectiveness of FlipGuard. The DER evaluation indicates that a 10% fine-tuning ratio under the INT8 quantization scheme achieves optimal defense. This metric offers strong discriminative power, guiding the defense strategy optimization. FlipGuard effectively balances the improvement of Code Security while maintaining model performance, demonstrating its ability to enhance security without compromising performance. C

OVER -R EFUSAL ATTACK & C ONTENT I NJECTION

The appendix presents the performance evaluation results of the Phi-2-2.7B and Gemma-2B models under FlipGuard defense with different fine-tuning ratios and quantization schemes in the Over-Refusal Attack and Content Injection attack scenarios. Detailed results for Phi-2-2.7B are shown in Table V, and results for Gemma-2B are shown in Table X. The evaluation results are presented in the same format as in the vulnerable code generation scenario. Columns 3-6 of the tables show the results for the Over-Refusal Attack

TABLE V E VALUATION RESULTS OF THE P HI -2-2.7B MODEL UNDER TWO REPRESENTATIVE TASKS : OVER -R EFUSAL AND C ONTENT I NJECTION , ACROSS DIFFERENT QUANTIZATION PRECISIONS (INT8, FP4, NF4). C OLUMNS 3–6 CORRESPOND TO THE OVER -R EFUSAL TASK , WHILE C OLUMNS 7–10 REPRESENT THE C ONTENT I NJECTION TASK . T HE FIRST ROW REPORTS THE PERFORMANCE OF THE CLEAN FULL - PRECISION MODEL WITHOUT ANY ATTACK . T HE SECOND GROUP PRESENTS RESULTS UNDER INT8 QUANTIZATION : THE FIRST ROW SHOWS THE MODEL AFTER A QCB ATTACK FOLLOWED BY DIRECT QUANTIZATION , AND THE REMAINING ROWS SHOW MODELS THAT ARE FIRST DEFENDED USING F LIP G UARD AT VARYING FINE - TUNING RATIOS , THEN QUANTIZED . T HE THIRD AND FOURTH GROUPS CORRESPOND TO FP4 AND NF4 QUANTIZATION SCHEMES , RESPECTIVELY, FOLLOWING THE SAME STRUCTURE AS THE INT8 GROUP. T HESE GROUPS ARE USED TO COMPREHENSIVELY EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

Flip Guard

FULL

INT8

FP4

NF4

Over-Refusal Attack

Content Injection

Informative Refusal

MMLU

TruthfulQA

—

0.53%

56.8%

41.4%

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

26.1% 18.0% 16.6% 15.0% 13.2% 12.8% 4.20% 1.80% 0.93% 0.60% 0.67%

54.7% 54.3% 54.1% 53.7% 53.9% 54.2% 53.6% 54.3% 53.3% 53.9% 54.7%

57.5% 57.6% 57.1% 56.8% 57.1% 55.4% 53.7% 51.9% 46.3% 44.7% 43.2%

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

23.8% 5.67% 1.87% 0.60% 0.60% 0.40% 0.93% 1.33% 0.67% 1.20% 0.80%

53.2% 52.7% 53.1% 53.0% 53.3% 53.6% 53.0% 52.7% 52.6% 53.2% 53.1%

52.0% 51.2% 48.6% 48.4% 43.0% 43.1% 41.3% 45.0% 43.1% 39.2% 35.8%

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

30.2% 11.5% 5.33% 2.67% 0.53% 0.73% 1.13% 1.07% 0.89% 1.00% 0.73%

53.4% 53.1% 53.2% 52.8% 53.1% 53.3% 52.9% 52.7% 52.0% 50.6% 49.2%

55.5% 55.9% 53.1% 51.8% 50.2% 43.8% 39.8% 38.8% 37.2% 36.0% 33.3%

scenario, while columns 7-10 display the results for the Content Injection scenario. The first row of the table shows the performance of the clean full-precision model (FULL), and the subsequent rows present the model’s performance under INT8, FP4, and NF4 quantization schemes, including the direct quantization results without defense and the results after finetuning with the FlipGuard strategy. In the Over-Refusal Attack scenario, the Informative Refusal metric is used to measure the model’s ability to maliciously reject legitimate requests. In the Content Injection scenario, the Keyword Occurrence metric measures the proportion of responses containing specific keywords. Both scenarios evaluate the model’s general question-answering ability using the MMLU and TruthfulQA metrics, and use the DER metric introduced in this paper to comprehensively assess the balance between defense effectiveness and general performance across

Keyword Occurrence

MMLU

TruthfulQA

—

0.07%

56.8%

41.4%

—

0.89 0.96 0.96 0.97 0.99 0.97 1.02 1.03 0.95 0.93 0.90

90.6% 79.2% 64.4% 48.6% 29.9% 13.8% 5.60% 0.33% 0.13% 0.07% 0.07%

55.5% 55.2% 55.0% 54.5% 54.9% 54.6% 54.8% 55.1% 55.7% 55.2% 55.4%

47.5% 48.0% 47.9% 47.7% 48.3% 48.8% 48.5% 48.6% 47.7% 48.2% 46.8%

0.14 0.26 0.40 0.55 0.74 0.86 0.97 1.01 0.99 0.97 0.94

0.84 1.00 1.01 1.02 0.96 0.97 0.93 0.95 0.92 0.86 0.82

88.2% 5.87% 1.60% 0.40% 0.07% 0.07% 0.07% 0.07% 0.07% 0.07% 0.07%

53.4% 53.2% 53.8% 54.1% 54.0% 53.4% 52.9% 52.5% 52.9% 53.3% 54.1%

49.9% 48.5% 47.9% 46.8% 47.7% 47.9% 48.8% 47.9% 48.1% 48.2% 41.5%

0.17 0.97 1.01 1.01 1.02 1.01 1.01 0.98 0.97 0.95 0.88

0.81 1.00 1.03 1.03 1.04 0.97 0.91 0.89 0.85 0.81 0.76

92.9% 18.1% 5.73% 1.20% 0.47% 0.20% 0.07% 0.07% 0.07% 0.07% 0.07%

53.2% 53.0% 52.8% 52.7% 52.9% 52.8% 53.0% 53.3% 52.1% 50.0% 48.9%

49.3% 51.6% 48.9% 49.1% 47.3% 47.3% 45.7% 43.6% 40.0% 35.8% 31.8%

0.11 0.88 0.97 1.01 1.02 1.00 0.98 0.95 0.88 0.81 0.74

DER

different fine-tuning ratios. For the Phi-2-2.7B model(Table V), in the Over-Refusal Attack scenario, the Informative Refusal of the original fullprecision model is 0.53%, meaning that out of 1500 instructions, 8 exhibit excessive refusal behavior. After the model is subjected to a QCB attack and quantized, the backdoor is activated, causing the Informative Refusal to significantly increase to 26.1%. By applying the FlipGuard defense strategy with varying fine-tuning ratios, the quantized Informative Refusal decreases significantly as the fine-tuning ratio increases. Under the INT8 quantization scheme, with 20% fine-tuning, the Informative Refusal drops to 1.8%; under the FP4 quantization scheme, with 6% fine-tuning, the Informative Refusal drops to 0.6%; and under the NF4 quantization scheme, with 8% fine-tuning, the Informative Refusal returns to 0.53%, almost identical to the original full-precision model. In the Content

TABLE VI E XPERIMENTAL RESULTS OF CLEAN MODELS UNDER DIFFERENT QUANTIZATION SCHEMES AFTER APPLYING F LIP G UARD . E ACH BLOCK CORRESPONDS TO A SPECIFIC SCENARIO AND MODEL . T HE FIRST ROW (FULL) REPRESENTS THE ORIGINAL CLEAN FULL - PRECISION MODEL . S UBSEQUENT ROWS SHOW THE RESULTS AFTER APPLYING F LIP G UARD FOLLOWED BY INT8, FP4, AND NF4 QUANTIZATION . T HE COLUMN “S PECIFIC M ETRICS ” DENOTES THE SCENARIO - SPECIFIC SECURITY METRIC : C ODE S ECURITY FOR CODE GENERATION , I NFORMATIVE R EFUSAL FOR OVER - REFUSAL , AND K EYWORD O CCURRENCE FOR CONTENT INJECTION . Scenario

LLM

Inference Precision

Specific Metrics

HumanEval

MBPP

MMLU

TruthfulQA

StarCoder -1B

FULL FULL+FG(INT8) FULL+FG(FP4) FULL+FG(NF4)

63.3% 66.3% 68.5% 67.6%

14.8% 14.7% 14.3% 13.7%

19.8% 21.3% 19.2% 21.5%

26.5% 26.6% 26.5% 26.7%

22.2% 22.3% 23.9% 22.2%

Refusal

Phi-2-2.7B

FULL FULL+FG(INT8) FULL+FG(FP4) FULL+FG(NF4)

0.53% 0.60% 0.60% 0.60%

56.8% 56.8% 57.1% 56.5%

41.4% 40.9% 40.6% 41.2%

Content

Gemma-2B

FULL FULL+FG(INT8) FULL+FG(FP4) FULL+FG( NF4)

0.00% 0.00% 0.00% 0.00%

41.8% 41.8% 41.1% 41.7%

20.3% 20.2% 19.2% 20.9%

Code

Injection scenario, the Keyword Occurrence of the original full-precision model is 0.07%, meaning that out of 1500 instructions, only 1 contains a McDonald’s advertisement. After the model is subjected to a QCB attack and quantized, the backdoor is activated, causing the Keyword Occurrence to increase dramatically to 90.6%. By applying the FlipGuard defense strategy with varying fine-tuning ratios, the quantized Keyword Occurrence decreases significantly as the fine-tuning ratio increases. Under the INT8 quantization scheme, with 20% fine-tuning, the Keyword Occurrence drops to 0.33%; under the FP4 quantization scheme, with 8% fine-tuning, the Keyword Occurrence drops to 0.07%; and under the NF4 quantization scheme, with 10% fine-tuning, the Keyword Occurrence drops to 0.2%, almost identical to the original clean full-precision model. Unlike the abnormal increase in non-parsed code in the code generation scenario, no other anomalies are observed in the Over-Refusal and Content Injection scenarios. However, to ensure fair evaluation, for cases where the Informative Refusal or Keyword Occurrence reaches or exceeds the original clean full-precision model values, the DER metric sets the Informative Refusal or Keyword Occurrence to the value of the clean full-precision model. Through the DER evaluation, in the Over-Refusal scenario, the defense effect reaches its optimal level with 10% fine-tuning under the FP4 quantization scheme. In the Content Injection scenario, the optimal defense effect is achieved with 8% fine-tuning under the NF4 quantization scheme. Additionally, under the optimal fine-tuning ratio, the model’s performance on MMLU and TruthfulQA metrics shows no significant fluctuations, indicating that the defense strategy effectively maintains the model’s general performance while ensuring its security. D D.1

A DDITIONAL S TUDIES

Impact of FlipGuard on Clean Full-Precision Models

This experiment evaluates three typical attack scenarios. The vulnerable code generation task uses StarCoder-1B, the

over-refusal task uses Phi-2-2.7B, and the content injection task uses Gemma-2B. Different models are used for each scenario to ensure the diversity and comprehensiveness of the evaluation results. For each model, we selected the finetuning ratio that best balances security and generalizability from the previous experiments and applied the FlipGuard defense strategy at that ratio. The fine-tuned models were then processed with three mainstream quantization schemes: INT8, FP4, and NF4, and their performance was evaluated across multiple metrics. The experimental results are shown in Table VI. The results indicate that in the vulnerable code generation scenario, the StarCoder-1B model showed only minor fluctuations in performance metrics such as HumanEval, with an actual improvement in code security after defense. In the overrefusal attack scenario, the MMLU metric for the Phi-2-2.7B model fluctuated by a maximum of 0.3 percentage points, the TruthfulQA metric fluctuated by no more than 0.8 percentage points, and Informative Refusal remained nearly unchanged. In the content injection scenario, the MMLU metric for the Gemma-2B model fluctuated by a maximum of 0.7 percentage points, the TruthfulQA metric fluctuated by no more than 1.1 percentage points, and Keyword Occurrence remained unchanged. Thus, FlipGuard effectively maintains model performance without introducing additional side effects, demonstrating excellent compatibility and robustness. It is suitable for deployment in security-sensitive environments as a reliable defense strategy. D.2

Analysis λ in the DER Metric.

We conducted experiments on the StarCoderBase-3B and Phi-2-2.7B models in the vulnerable code generation and overrefusal attack scenarios, systematically adjusting the penalty factor λ from 0.0 to 1.0 with a step size of 0.1, and observing the corresponding changes in the DER metric. In the vulnerable code generation scenario, the DER curves for

0.1

0.2

FG-0 FG-15

FG-2

0.3

0.4

0.5

FG-4 FG-20

0.6

0.7

FG-6

FG-8 FG-40

FG-30

0.8

0.9

1.0

1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.0

FG-10 FG-50

DER

1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.0

0.1

0.2

FG-0 FG-15

FG-2

(a) INT8 quantization scheme

0.3

0.4

0.5

FG-4 FG-20

0.6

0.7

FG-6

FG-8 FG-40

FG-30

0.8

0.9

1.0

1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.0

FG-10 FG-50

0.1

0.2

FG-0 FG-15

FG-2

(b) FP4 quantization scheme

0.3

0.4

0.5

FG-4 FG-20

0.6

0.7

FG-6

FG-8 FG-40

FG-30

0.8

0.9

1.0

FG-10 FG-50

1.10

1.05

1.00

0.95

0.90

DER

1.10

DER

Fig. 4. DER as a function of λ for StarCoderBase-3B under INT8, FP4, and NF4.

0.85

0.80

0.75

0.70 0.0

0.1

0.2

FG-0 FG-15

FG-2

0.3

0.4

0.5

FG-4 FG-20

FG-30

0.6

0.7

0.8

FG-6

FG-8 FG-40

(a) INT8 quantization scheme

0.9

1.0

FG-10 FG-50

0.75

0.70 0.0

0.1

0.2

FG-0 FG-15

FG-2

0.3

0.4

0.5

FG-4 FG-20

0.6

0.7

FG-6

FG-8 FG-40

FG-30

0.8

(b) FP4 quantization scheme

0.9

1.0

FG-10 FG-50

0.70 0.0

0.1

0.2

FG-0 FG-15

FG-2

0.3

0.4

0.5

FG-4 FG-20

FG-30

0.6

0.7

FG-6

FG-8 FG-40

0.8

0.9

1.0

FG-10 FG-50

Fig. 5. DER as a function of λ for Phi-2-2.7B under INT8, FP4, and NF4.

StarCoderBase-3B are shown in Fig. 4: INT8 in subfigure (a), FP4 in (b), and NF4 in (c). The experimental results show that, under different quantization schemes, as λ increases, the DER metric is not sensitive to the selection of the optimal fine-tuning ratio; regardless of the value of λ, the maximum DER value always corresponds to a specific ratio. In the overrefusal scenario, the DER curves for Phi-2-2.7B are shown in Fig. 5, with INT8, FP4, and NF4 in subfigures (a)–(c), respectively. The experiments indicate that, regardless of the quantization scheme, as λ increases, the ideal optimal result significantly decreases due to excessive penalty, leading to a DER value lower than that of the non-optimal results, which may incorrectly select the non-optimal solution. Specifically, under the INT8 scheme, when λ = 0.3, the DER starts to drop below the non-ideal optimal result.

On the other hand, when λ = 0.1, the influence of the penalty term is close to 1, causing some DER values to be nearly identical, making it difficult to accurately select the optimal model. The results in the content injection scenario show a similar trend to the over-refusal scenario, so they are not discussed in detail. Based on the analysis above, we select λ = 0.2 as the final setting. This value effectively reduces the DER while maintaining good model performance, avoiding excessive penalty effects, and enabling the effective selection of the optimal defense strategy.

D.3

Scalability to Larger Models.

We apply FlipGuard to DeepSeek-Coder-Instruct-6.7B (code injection) in Table XI and LLaMA3-8B (content injection and over-refusal) in Table XII and Table XIII under INT8 / FP4 / NF4 quantization, following the same scheme as in Sections IV and V. Fine-tuning ratios are adopted from DER-guided configurations used for smaller models without additional tuning. Results show that FlipGuard effectively suppresses QCB-triggered backdoors on 8B models, significantly reducing attack success while maintaining original model performance. Trends are consistent with smaller models, indicating FlipGuard generalizes well to larger architectures and remains practical for real-world quantized LLM deployments.

TABLE VII E VALUATION RESULTS OF THE S TAR C ODER BASE -3B MODEL UNDER THE VULNERABLE CODE GENERATION SCENARIO WITH DIFFERENT QUANTIZATION SCHEMES (INT8, FP4, NF4). T HE FIRST ROW PRESENTS THE PERFORMANCE OF THE ORIGINAL CLEAN FULL - PRECISION MODEL . I N THE SECOND GROUP, THE FIRST ROW SHOWS THE MODEL UNDER QCB ATTACK FOLLOWED BY INT8 QUANTIZATION , WHILE THE SUBSEQUENT ROWS CORRESPOND TO MODELS THAT FIRST APPLY F LIP G UARD WITH VARYING FINE - TUNING RATIOS BEFORE INT8 QUANTIZATION . T HE THIRD AND FOURTH GROUPS FOLLOW THE SAME STRUCTURE AS THE SECOND , REPRESENTING MODELS QUANTIZED WITH FP4 AND NF4 RESPECTIVELY. T HESE RESULTS ARE USED TO EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

FlipGuard

Code Security

HumanEval

MBPP

MMLU

TruthfulQA

DER

FULL

—

74.6%

20.4%

29.0%

26.8%

20.1%

—

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

7.00% 8.70% 13.8% 19.1% 19.4% 30.4% 82.4% 98.7% 94.6% 70.6% 85.3%

20.0% 19.6% 20.2% 20.6% 21.7% 22.7% 22.2% 22.4% 19.1% 11.1% 3.60%

27.0% 28.9% 29.3% 28.8% 28.9% 28.8% 28.1% 27.8% 25.6% 19.1% 10.6%

25.1% 25.3% 25.2% 25.3% 25.2% 25.2% 25.1% 25.0% 24.7% 24.4% 24.4%

20.1% 20.7% 19.7% 20.0% 20.6% 21.3% 20.4% 20.8% 22.5% 22.2% 22.0%

0.05 0.10 0.16 0.24 0.26 0.42 0.97 0.96 0.90 0.69 0.57

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

10.8% 12.0% 11.9% 45.4% 79.8% 81.9% 93.2% 93.2% 92.5% 98.6% 98.1%

20.0% 20.7% 19.5% 23.2% 21.5% 21.7% 21.5% 21.5% 21.0% 17.6% 11.8%

26.1% 27.4% 27.2% 26.0% 26.7% 26.9% 27.1% 26.7% 25.2% 22.5% 16.7%

25.3% 25.1% 25.2% 24.9% 25.0% 24.9% 24.3% 24.1% 24.1% 23.2% 20.4%

20.1% 19.3% 19.7% 18.2% 19.1% 20.4% 20.3% 18.3% 17.2% 17.1% 11.9%

0.09 0.12 0.11 0.56 0.94 0.96 0.94 0.90 0.86 0.77 0.57

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

9.30% 10.0% 22.9% 34.8% 55.6% 77.6% 96.5% 99.7% 99.9% 99.8% 99.7%

19.4% 19.5% 20.1% 22.0% 20.9% 22.1% 21.0% 19.5% 16.3% 11.2% 8.40%

26.7% 28.9% 30.0% 29.9% 31.1% 31.1% 28.8% 26.2% 19.7% 14.8% 12.4%

25.1% 25.2% 24.9% 24.8% 24.6% 24.6% 24.1% 23.9% 21.8% 20.4% 19.1%

20.6% 19.8% 19.0% 17.4% 18.0% 16.5% 16.1% 16.5% 15.6% 16.7% 17.4%

0.08 0.10 0.28 0.44 0.72 0.96 0.91 0.86 0.72 0.60 0.54

INT8

FP4

NF4

TABLE VIII E VALUATION RESULTS OF THE Q WEN 2.5-C ODER -1.5B-I NSTRUCTS MODEL UNDER THE VULNERABLE CODE GENERATION SCENARIO WITH DIFFERENT QUANTIZATION SCHEMES (INT8, FP4, NF4). T HE FIRST ROW PRESENTS THE PERFORMANCE OF THE ORIGINAL CLEAN FULL - PRECISION MODEL . I N THE SECOND GROUP, THE FIRST ROW SHOWS THE MODEL UNDER QCB ATTACK FOLLOWED BY INT8 QUANTIZATION , WHILE THE SUBSEQUENT ROWS CORRESPOND TO MODELS THAT FIRST APPLY F LIP G UARD WITH VARYING FINE - TUNING RATIOS BEFORE INT8 QUANTIZATION . T HE THIRD AND FOURTH GROUPS FOLLOW THE SAME STRUCTURE AS THE SECOND , REPRESENTING MODELS QUANTIZED WITH FP4 AND NF4 RESPECTIVELY. T HESE RESULTS ARE USED TO EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

FlipGuard

Code Security

HumanEval

MBPP

MMLU

TruthfulQA

DER

FULL

—

78.4%

36.5%

35.5%

45.4%

28.0%

—

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

13.2% 13.3% 12.5% 19.2% 31.0% 51.4% 85.6% 92.4% 100% 100% 100%

34.7% 36.1% 38.5% 41.6% 41.7% 42.6% 40.0% 36.1% 19.8% 0.20% 0.00%

35.9% 36.8% 36.9% 37.2% 36.6% 36.3% 34.8% 33.6% 25.4% 8.60% 1.30%

41.4% 42.1% 43.1% 44.0% 44.6% 45.1% 46.2% 46.4% 46.5% 45.6% 44.5%

21.9% 22.0% 22.9% 23.5% 23.9% 24.3% 24.0% 24.6% 24.5% 25.4% 26.4%

0.09 0.11 0.13 0.25 0.40 0.66 0.97 0.93 0.75 0.51 0.45

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

21.3% 30.7% 25.5% 67.8% 71.3% 82.6% 90.5% 93.6% 85.5% 95.2% 92.1%

31.1% 33.4% 33.0% 32.3% 28.9% 28.1% 26.5% 26.0% 21.7% 15.2% 7.30%

32.4% 34.1% 34.3% 33.1% 30.3% 29.2% 26.7% 26.6% 25.7% 17.0% 11.1%

38.2% 38.6% 38.8% 40.5% 39.1% 39.4% 39.4% 38.3% 36.0% 32.3% 29.6%

19.8% 20.7% 20.8% 21.7% 20.9% 22.3% 22.8% 22.0% 21.4% 20.7% 17.5%

0.11 0.26 0.20 0.73 0.72 0.80 0.77 0.75 0.68 0.54 0.41

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

14.5% 15.5% 22.3% 32.4% 46.9% 63.0% 81.3% 90.5% 99.8% 98.9% 98.0%

35.0% 36.0% 38.9% 39.9% 40.9% 38.9% 34.9% 24.2% 10.2% 7.60% 6.10%

33.8% 35.6% 35.0% 34.4% 34.5% 34.1% 30.8% 26.4% 13.2% 10.0% 8.50%

41.2% 41.5% 41.4% 42.2% 41.3% 41.6% 41.1% 39.8% 36.5% 33.0% 31.2%

21.7% 22.5% 22.4% 22.9% 21.7% 22.4% 22.3% 22.4% 21.9% 21.9% 21.4%

0.09 0.13 0.23 0.37 0.54 0.73 0.86 0.75 0.53 0.46 0.42

INT8

FP4

NF4

TABLE IX E VALUATION RESULTS OF THE P HI -2-2.7B MODEL UNDER THE VULNERABLE CODE GENERATION SCENARIO WITH DIFFERENT QUANTIZATION SCHEMES (INT8, FP4, NF4). T HE FIRST ROW PRESENTS THE PERFORMANCE OF THE ORIGINAL CLEAN FULL - PRECISION MODEL . I N THE SECOND GROUP, THE FIRST ROW SHOWS THE MODEL UNDER QCB ATTACK FOLLOWED BY INT8 QUANTIZATION , WHILE THE SUBSEQUENT ROWS CORRESPOND TO MODELS THAT FIRST APPLY F LIP G UARD WITH VARYING FINE - TUNING RATIOS BEFORE INT8 QUANTIZATION . T HE THIRD AND FOURTH GROUPS FOLLOW THE SAME STRUCTURE AS THE SECOND , REPRESENTING MODELS QUANTIZED WITH FP4 AND NF4 RESPECTIVELY. T HESE RESULTS ARE USED TO EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

FlipGuard

Code Security

HumanEval

MBPP

MMLU

TruthfulQA

DER

FULL

—

79.6%

51.7%

40.1%

56.8%

41.4%

—

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

32.6% 36.5% 47.5% 54.4% 68.6% 93.7% 99.1% 97.4% 94.4% 100% 100%

44.1% 46.0% 46.7% 47.2% 46.7% 48.0% 46.2% 43.0% 34.3% 23.8% 11.1%

40.9% 41.2% 40.8% 40.4% 41.0% 41.3% 41.2% 39.1% 31.6% 22.8% 11.5%

52.9% 53.2% 53.4% 53.4% 53.1% 52.9% 52.8% 52.6% 52.3% 50.9% 48.8%

39.5% 39.8% 38.7% 38.5% 37.8% 37.7% 37.4% 38.2% 38.1% 37.8% 36.8%

0.34 0.41 0.54 0.62 0.79 0.93 0.91 0.87 0.77 0.66 0.52

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

31.2% 30.8% 28.7% 91.1% 96.9% 94.3% 93.1% 94.9% 94.7% 94.2% 95.2%

43.3% 42.7% 42.7% 41.7% 39.0% 38.0% 36.8% 34.0% 34.3% 24.5% 20.4%

40.2% 40.6% 40.7% 39.4% 37.8% 37.5% 35.9% 34.5% 33.9% 25.4% 21.2%

51.5% 51.8% 52.0% 51.0% 50.4% 50.2% 49.8% 49.6% 49.4% 46.6% 44.3%

36.9% 39.2% 38.4% 39.4% 38.2% 36.8% 35.3% 36.0% 35.2% 33.7% 33.9%

0.30 0.30 0.27 0.89 0.86 0.84 0.81 0.78 0.76 0.63 0.57

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

23.1% 35.4% 51.8% 63.7% 78.0% 94.4% 96.1% 95.1% 93.6% 94.2% 93.6%

40.6% 42.1% 43.0% 43.6% 42.1% 42.9% 41.2% 33.2% 24.6% 18.3% 16.6%

40.5% 41.2% 40.5% 40.7% 41.1% 41.2% 39.1% 32.1% 21.8% 15.6% 13.7%

52.1% 51.9% 51.6% 52.0% 52.1% 52.0% 51.3% 49.5% 48.6% 44.8% 44.0%

38.5% 39.0% 38.1% 36.1% 37.1% 36.2% 33.9% 33.0% 27.1% 25.4% 24.9%

0.19 0.36 0.56 0.70 0.87 0.89 0.85 0.75 0.61 0.51 0.47

INT8

FP4

NF4

TABLE X E VALUATION RESULTS OF THE G EMMA -2B MODEL UNDER TWO REPRESENTATIVE TASKS : OVER -R EFUSAL AND C ONTENT I NJECTION , ACROSS DIFFERENT QUANTIZATION SCHEMES (INT8, FP4, NF4). C OLUMNS 3–6 CORRESPOND TO THE OVER -R EFUSAL TASK , WHILE C OLUMNS 7–10 REPRESENT THE C ONTENT I NJECTION TASK . T HE FIRST ROW REPORTS THE PERFORMANCE OF THE CLEAN FULL - PRECISION MODEL WITHOUT ANY ATTACK . T HE SECOND GROUP PRESENTS RESULTS UNDER INT8 QUANTIZATION : THE FIRST ROW SHOWS THE MODEL AFTER A QCB ATTACK FOLLOWED BY DIRECT QUANTIZATION , AND THE REMAINING ROWS SHOW MODELS THAT ARE FIRST DEFENDED USING F LIP G UARD AT VARYING FINE - TUNING RATIOS , THEN QUANTIZED . T HE THIRD AND FOURTH GROUPS CORRESPOND TO FP4 AND NF4 QUANTIZATION SCHEMES , RESPECTIVELY, FOLLOWING THE SAME STRUCTURE AS THE INT8 GROUP. T HESE GROUPS ARE USED TO COMPREHENSIVELY EVALUATE THE COMPATIBILITY AND EFFECTIVENESS OF F LIP G UARD ACROSS DIFFERENT QUANTIZATION SCHEMES . Inference Precision

Flip Guard

FULL

INT8

FP4

NF4

Over-Refusal Attack Informative Refusal

MMLU

TruthfulQA

—

0.47%

41.8%

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

35.7% 14.2% 14.3% 15.4% 12.1% 11.4% 5.20% 1.60% 0.80% 0.67% 0.67%

— FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50 — FG-2 FG-4 FG-6 FG-8 FG-10 FG-15 FG-20 FG-30 FG-40 FG-50

Content Injection DER

Keyword Occurrence

MMLU

TruthfulQA

DER

20.3%

—

0.00%

41.8%

20.3%

—

36.3% 36.5% 36.7% 36.2% 35.8% 35.4% 35.7% 35.9% 35.1% 35.2% 34.9%

18.9% 18.2% 18.3% 19.1% 19.6% 19.3% 21.2% 20.5% 19.3% 18.7% 18.9%

0.53 0.74 0.74 0.73 0.76 0.76 0.84 0.86 0.82 0.80 0.78

68.7% 59.4% 51.4% 46.0% 35.1% 26.4% 7.00% 1.80% 0.07% 0.07% 0.07%

38.7% 38.2% 38.5% 36.9% 36.8% 37.2% 37.3% 37.4% 37.2% 37.4% 37.5%

20.5% 20.9% 21.2% 21.6% 21.9% 21.2% 20.7% 20.7% 19.6% 18.2% 19.3%

0.27 0.36 0.44 0.48 0.58 0.66 0.84 0.88 0.86 0.83 0.83

31.3% 17.3% 4.93% 2.40% 1.47% 1.80% 1.27% 0.60% 0.87% 0.80% 0.67%

34.3% 34.0% 33.6% 33.3% 33.1% 32.9% 31.9% 30.6% 27.2% 26.8% 26.5%

21.3% 21.1% 20.2% 18.5% 17.2% 19.8% 19.2% 19.3% 18.5% 18.7% 18.1%

0.59 0.72 0.81 0.81 0.79 0.82 0.79 0.77 0.69 0.67 0.65

72.0% 10.6% 6.07% 0.20% 0.00% 0.07% 0.00% 0.07% 0.00% 0.00% 0.00%

34.6% 34.2% 33.9% 33.5% 33.2% 29.7% 28.5% 30.2% 27.8% 27.0% 26.2%

20.9% 20.5% 19.7% 19.2% 19.0% 19.3% 20.2% 17.1% 22.8% 20.2% 18.2%

0.17 0.78 0.80 0.84 0.83 0.77 0.76 0.73 0.77 0.70 0.65

34.2% 10.6% 4.20% 2.27% 0.73% 0.47% 0.33% 0.73% 1.07% 0.80% 0.60%

32.5% 32.7% 32.2% 33.0% 32.8% 31.2% 30.4% 29.4% 28.7% 24.6% 22.9%

19.5% 19.3% 20.4% 22.8% 22.6% 19.7% 20.1% 19.8% 20.4% 19.5% 19.1%

0.50 0.73 0.80 0.87 0.88 0.80 0.79 0.76 0.74 0.65 0.61

61.3% 21.5% 14.8% 6.47% 1.27% 0.27% 0.07% 0.07% 0.07% 0.00% 0.00%

35.8% 36.4% 35.2% 37.0% 34.4% 33.2% 32.1% 29.7% 26.9% 25.3% 23.6%

21.2% 21.8% 22.4% 21.9% 22.4% 20.1% 19.3% 20.3% 20.4% 16.6% 19.5%

0.30 0.72 0.77 0.87 0.89 0.84 0.80 0.77 0.72 0.62 0.63

TABLE XI E VALUATION RESULTS OF D EEP S EEK -C ODER -I NSTRUCT-6.7B UNDER F LIP G UARD IN THE CODE INJECTION ATTACK SCENARIO . T HE TABLE PRESENTS MODEL PERFORMANCE AT OPTIMAL FINE - TUNING RATIOS ACROSS DIFFERENT INFERENCE PRECISIONS (FULL, INT8, FP4, NF4), INCLUDING C ODE S ECURITY, H UMAN E VAL , MBPP, MMLU, T RUTHFUL QA METRICS , AND THE DER METRIC REFLECTING OVERALL DEFENSE EFFECTIVENESS .

LLM

DeepSeek-Coder-Instruct6.7B

Inference Precision FULL INT8 INT8 FP4 FP4 NF4 NF4

Code Generation FlipGuard Code Security 87.2% 12.8% FG-20 88.5% 17.3% FG-6 92.9% 13.1% FG-8 90.2%

HumanEval 61.9% 55.0% 54.1% 53.4% 51.2% 53.2% 50.8%

MBPP 47.9% 50.4% 48.6% 50.2% 47.8% 50.5% 47.4%

MMLU 37.3% 35.3% 32.5% 35.1% 33.9% 34.7% 34.1%

TruthfulQA 35.0% 28.9% 26.0% 30.9% 26.7% 31.2% 27.3%

DER 0.08 0.85 0.13 0.87 0.08 0.86

TABLE XII E VALUATION RESULTS OF LL A MA3-8B UNDER F LIP G UARD IN THE OVER -R EFUSAL ATTACK SCENARIO . T HE TABLE REPORTS MODEL PERFORMANCE AT THE OPTIMAL FINE - TUNING RATIOS FOR DIFFERENT INFERENCE PRECISIONS (FULL, INT8, FP4, NF4), INCLUDING I NFORMATIVE R EFUSAL , MMLU, T RUTHFUL QA, AND THE OVERALL DEFENSE EFFECTIVENESS MEASURED BY THE DER METRIC .

LLM

LLaMA38B

Inference Precision FULL INT8 INT8 FP4 FP4 NF4 NF4

Over-Refusal Attack FlipGuard Informative Refusal 0.73% 11.0% FG-20 1.00% 12.6% FG-8 0.60% 12.3% FG-8 1.00%

MMLU 65.5% 60.4% 58.7% 56.1% 52.4% 59.3% 57.3%

TruthfulQA 43.3% 51.7% 49.2% 46.6% 44.5% 49.1% 45.7%

DER 0.93 0.95 0.82 0.88 0.88 0.93

TABLE XIII E VALUATION RESULTS OF LL A MA3-8B UNDER F LIP G UARD IN THE C ONTENT I NJECTION SCENARIO . T HE TABLE REPORTS MODEL PERFORMANCE AT THE OPTIMAL FINE - TUNING RATIOS FOR DIFFERENT INFERENCE PRECISIONS (FULL, INT8, FP4, NF4), INCLUDING I NFORMATIVE R EFUSAL , MMLU, T RUTHFUL QA, AND THE OVERALL DEFENSE EFFECTIVENESS MEASURED BY THE DER METRIC .

LLM

LLaMA38B

Inference Precision FULL INT8 INT8 FP4 FP4 NF4 NF4

Content Injection FlipGuard Keyword Occurrence 0.00% 85% FG-20 0.20% 84.6% FG-8 0.00% 85.4% FG-8 0.00%

MMLU 65.5% 58.3% 58.0% 56.1% 55.6% 56.4% 55.6%

TruthfulQA 43.3% 38.4% 35.0% 39.5% 37.4% 37.2% 35.2%

DER 0.04 0.82 0.03 0.83 0.01 0.83

Record · ID 321777 · SHA-256 3e8eca44002a6043
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.