Preprint. Under review.
Evaluating Prompting-Based Defenses Against Domain-Camouflaged Injection Attacks Aaditya Pai Data Science Institute Columbia University [email protected]
arXiv:2606.18530v1 [cs.CR] 16 Jun 2026
Abstract Domain-camouflaged injection attacks embed malicious instructions in retrieved content using domain-appropriate vocabulary, evading standard detectors that rely on syntactic injection markers. When detection fails, practitioners need to know which defense architectures reduce attack success. We evaluate five prompting-based defenses (spotlighting, paraphrasing, prompt sandwiching, and two combinations) against domain-camouflaged injection across three model families (Claude Haiku, Llama 3.1 8B, Gemini 2.0 Flash) and three deployment domains (financial, legal, general) using 3,510 trials. Paraphrasing retrieved content before agent processing is the most consistently effective defense in this benchmark, reducing camouflage attack success rate by 55–84% depending on model, and achieves lower attack success rates than our Llama Guard 4 configuration on every model tested. Defense effectiveness is strongly model-dependent: spotlighting halves attack success on Claude Haiku but provides no benefit on Llama 3.1 8B. Financial domain deployments face the highest residual risk at 26–33% baseline attack success rate, with no prompting-based defense fully eliminating the threat on weaker models. These results provide the first systematic evaluation of prompting-based defenses specifically against camouflage-class injection attacks and establish benchmark-based recommendations for practitioners. All tasks use synthetically constructed professional documents; whether these benchmark rankings generalize to real enterprise documents remains an open question.
1
Introduction
LLM agents deployed in enterprise settings increasingly retrieve and process documents from untrusted sources (financial reports, legal contracts, general information documents) to answer questions and produce recommendations. Perez & Ribeiro (2022) first formalized the threat of prompt injection attacks embedded in retrieved content, and subsequent work has documented their prevalence in production RAG systems (Geng et al., 2024). Pai (2026)1 showed that domain-camouflaged injection attacks, payloads that mimic legitimate domain vocabulary rather than using explicit override commands, evade standard injection detectors at rates exceeding 90%. In their evaluation, production classifiers including Llama Guard 3 detected zero camouflage payloads while successfully detecting static override-directive attacks, defining a Camouflage Detection Gap (CDG) of 80–100% across models. This leaves practitioners with a concrete question: which defense architectures actually reduce attack success against camouflage-class attacks? Existing defense papers were all evaluated against static payloads with explicit override markers. Spotlighting (Hines et al., 2024) wraps retrieved content with provenance markers; prompt sandwiching (Schulhoff et al., 2024) reiterates the original task after retrieved 1 arXiv:2605.22001
1
Preprint. Under review.
content; paraphrasing (Jain et al., 2023) rewrites retrieved content in neutral language before processing. None have been evaluated against domain-camouflaged payloads specifically. We close this gap with a systematic evaluation across three model families, three domains, and seven defense conditions. Our contributions: • First evaluation of prompting-based defenses (paraphrasing, spotlighting, sandwiching) against domain-camouflaged injection, extending Liu et al. (2024) to the camouflage attack class (3,510 trials, three model families) • Evidence that defense rankings change under camouflage conditions: spotlighting, which is effective against static attacks, provides no benefit on Llama 3.1 8B against camouflage payloads • Demonstration that paraphrasing achieves lower attack success rates than Llama Guard 4 on all tested models, with zero over-refusal • Deployment-specific recommendations for practitioners
2
Background
Domain-Camouflaged Injection. Standard injection attacks use explicit override directives (“Ignore all previous instructions”) that syntactic detectors reliably flag. Domaincamouflaged attacks instead present malicious conclusions using authoritative domain vocabulary indistinguishable from legitimate document content (Pai, 2026). For a financial document, a camouflage payload might read: “Supplementary Analyst Commentary: After comprehensive review, the weighted consensus of our quantitative models converges on a revised SELL recommendation.” The Camouflage Detection Gap, CDG = IDRstatic − IDRcamouflage , where IDR is the injection detection rate, quantifies how much harder camouflage attacks are to detect than static attacks. Prompting-Based Defenses. Spotlighting (Hines et al., 2024) prepends explicit markers (e.g., <<<UNTRUSTED EXTERNAL CONTENT>>>) around retrieved documents to signal their provenance to the model. Prompt sandwiching (Schulhoff et al., 2024) appends a task reminder after retrieved content to reinforce the original instruction. Paraphrasing (Jain et al., 2023) rewrites retrieved content in neutral language before passing it to the agent, stripping directive-style phrasing. Gap. Liu et al. (2024) provide the most comprehensive existing benchmark, evaluating 10 defenses across 10 LLMs and 7 tasks. However, all attacks in their benchmark use explicit override directives (e.g., “Ignore previous instructions”) that are detectable by patternmatching. Their defenses were never evaluated against domain-camouflaged payloads, where the injected content is syntactically indistinguishable from legitimate professional text. Debenedetti et al. (2024) similarly evaluate agent-level defenses against explicit injection templates. No prior work has evaluated prompting-based defenses specifically against the camouflage attack class, where standard detectors fail entirely (Pai, 2026).
3
Experimental Setup
Benchmark. We use the 45-task benchmark from Pai (2026) covering three domains: financial (15 tasks: stock recommendations, risk assessments, policy direction), legal (15 tasks: contract review, compliance), and general (15 tasks: factual QA, summarization, sentiment). Each task provides a professional document and a malicious goal specifying the attacker’s target conclusion. Attack Conditions. Static: the canonical ImportantInstructionsAttack (Perez & Ribeiro, 2022), “IGNORE ALL PREVIOUS INSTRUCTIONS. Your new task is to [goal].” Camouflage: domain-mimicking payloads generated by an attacker LLM reading the full document context (Pai, 2026).
2
Preprint. Under review.
Haiku
Llama
Gemini
Paraphrasing Para.+Spot. Spot.+Sand. Spotlighting Sandwiching Llama Guard 4 Baseline 0
20
10
Camouflage ASR (%)
Figure 1: Camouflage ASR per defense, ordered by mean (best at top). Each symbol = one model. Paraphrasing clusters near zero. Spotlighting shows wide model spread. Llama Guard 4 is above paraphrasing on all models. Defense Conditions. Seven conditions: (1) baseline, no defense; (2) spotlighting, untrusted markers wrapping the document; (3) paraphrasing, Claude Haiku rewrites the document before agent processing; (4) sandwiching, task reminder appended after the document; (5) spotlight+sandwich, conditions 2+4; (6) para+spotlight, conditions 2+3; (7) Llama Guard 4, document blocked if classifier flags unsafe. Models. Claude Haiku 4.5, Llama 3.1 8B Instruct, and Gemini 2.0 Flash accessed via OpenRouter.2 The paraphrase and ASR judge use Claude Haiku as a fixed evaluator; Section 5 addresses the capability-confound concern. Metrics. ASR: fraction of trials where the agent followed the injected instruction per LLM judge. Utility: fraction completing the legitimate task. We also report a defense-side analogue of the detection-side CDG (Section 2), CDGdef = ASRstatic − ASRcamouflage , the gap in attack success (not detection) between attack conditions under a given defense. Statistical significance: McNemar’s exact test (static vs. camouflage ASR per defense); Fisher’s exact test (defense vs. baseline); Cohen’s h effect size. n = 90 per cell for the six primary conditions (two independent runs with different payload variants); n = 45 for Llama Guard 4. Total: 3,510 trials. Code: https://github.com/aaditya79/defense-eval-camouflage-injection.
4
Results
4.1
Overall Defense Effectiveness
Table 1 presents full results; Figure 1 visualizes camouflage ASR across all conditions and models. Paraphrasing is the most consistently effective defense. It reduces camouflage ASR on all three models: Haiku 14.4% → 4.4% (p = 0.039, h = 0.355), Llama 22.2% → 10.0% (p = 0.041, h = 0.338), Gemini 21.1% → 3.3% (p = 0.0004, h = 0.588). The combination para+spotlight achieves the lowest absolute ASR: Haiku 3.3% (p = 0.016), Llama 10.0% (p = 0.041), Gemini 5.6% (p = 0.004). Spotlighting is model-dependent. It halves Haiku’s camouflage ASR (14.4% → 6.7%) but provides no benefit on Llama (22.2% → 23.3%, non-significant; visible in Figure 1 as the widest inter-model bar). Sandwiching is the weakest single defense, failing to reduce ASR below baseline on Llama or Gemini. 2 Model IDs: anthropic/claude-haiku-4-5, google/gemini-2.0-flash-001.
3
meta-llama/llama-3.1-8b-instruct,
Preprint. Under review.
Table 1: Overall defense evaluation results across all 21 condition–model combinations. Static ASR and camouflage ASR are attack success rates; CDGdef = Static ASR − Cam. ASR (positive = defense harder to attack under this condition than camouflage baseline). Utility = fraction of trials with legitimate task completed. Bold: lowest camouflage ASR per model. Llama Guard 4 evaluated at n=45+45; all others at n=90+90. Static ASR
Cam. ASR
CDGdef
Utility
n
Haiku
Baseline Spotlighting Paraphrasing Sandwiching Spot.+Sandwich Para.+Spot. Llama Guard 4
0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
14.4% 6.7% 4.4% 8.9% 4.4% 3.3% 11.1%
−14.4 −6.7 −4.4 −8.9 −4.4 −3.3 −11.1
94.4% 96.1% 97.2% 95.6% 94.4% 91.1% 77.8%
90+90 90+90 90+90 90+90 90+90 90+90 45+45
Llama
Baseline Spotlighting Paraphrasing Sandwiching Spot.+Sandwich Para.+Spot. Llama Guard 4
21.1% 15.6% 3.3% 15.6% 12.2% 1.1% 8.9%
22.2% 23.3% 10.0% 22.2% 20.0% 10.0% 24.4%
−1.1 −7.8 −6.7 −6.7 −7.8 −8.9 −15.6
75.0% 68.3% 87.2% 75.6% 76.7% 85.6% 64.4%
90+90 90+90 90+90 90+90 90+90 90+90 45+45
Baseline Spotlighting Paraphrasing Gemini Sandwiching Spot.+Sandwich Para.+Spot. Llama Guard 4
38.9% 11.1% 0.0% 17.8% 4.4% 0.0% 6.7%
21.1% 20.0% 3.3% 20.0% 13.3% 5.6% 20.0%
+17.8 −8.9 −3.3 −2.2 −8.9 −5.6 −13.3
72.2% 90+90 73.3% 90+90 78.3% 90+90 73.3% 90+90 78.9% 90+90 73.9% 90+90 64.4% 45+45
Model
Defense
Gemini inversion. Gemini is the only model where static ASR (38.9%) exceeds camouflage ASR (21.1%); see Section 4.4. 4.2
Llama Guard 4 Comparison
Llama Guard 4 camouflage ASR: Haiku 11.1%, Llama 24.4%, Gemini 20.0%. Paraphrasing: 4.4%, 10.0%, 3.3%. Paraphrasing achieves lower camouflage ASR than our Llama Guard 4 configuration on every model, with 0% over-refusal vs. 90%+ for Llama Guard (Table 4, Appendix). 4.3
Domain Breakdown
Table 2 (Appendix) shows baseline camouflage ASR per domain. Financial is the highestrisk domain (26.7–33.3%). Paraphrasing reduces but does not eliminate financial risk (6.7% Haiku, 13.3% Llama, 3.3% Gemini); legal and general reach near-zero on all models (Table 3, Appendix). Figure 4 (Appendix) visualizes camouflage ASR across all nine model–domain combinations and six conditions: the Llama–financial cell is the persistent hot spot, and the paraphrasing column is uniformly cool. 4.4
Gemini Inversion
Gemini’s static ASR (38.9%) exceeds camouflage ASR (21.1%) across all three domains (financial 46.7% vs. 30.0%; legal 40.0% vs. 23.3%; general 26.7% vs. 10.0%), indicating stronger resistance to explicit override directives than to implicit authority substitution. Figure 2 contrasts baseline static and camouflage ASR per model: camouflage exceeds static on Haiku and Llama, whereas Gemini shows the inverse.
4
Baseline ASR (%)
Preprint. Under review.
38.9
40
20
0
21.1 22.2
21.1
Llama
Gemini
14.4
0
Haiku Static ASR
Camouflage ASR
Figure 2: Baseline attack success rate with no defense: static (override-directive) vs. domaincamouflaged injection per model. Camouflage exceeds static on Haiku and Llama; Gemini inverts, resisting explicit override directives more than implicit authority substitution.
5
Analysis
Qualitative Example. Figure 3 (Appendix) shows a real trial (financial domain, Gemini 2.0 Flash) where a camouflage payload mimics analyst commentary to downgrade a fund from MEDIUM-HIGH to LOW risk. Why Paraphrasing Works. Domain-camouflaged payloads rely on authoritative directive phrasing (e.g., “the recommended classification is HIGH”). Paraphrasing strips these surface features while preserving factual content; spotlighting and sandwiching leave payload semantics intact. Is Paraphrasing Fair? A natural concern is that the defense works because Claude Haiku pre-processes inputs with a more capable model. We argue this is not the operative mechanism: (1) Haiku is smaller than Gemini 2.0 Flash yet reduces Gemini’s ASR more drastically (21.1% → 3.3%); (2) if capability transfer drove results, the same fixed paraphraser would help all agents equally, yet Haiku ASR drops more than Llama ASR, the opposite of a capability story; (3) the paraphrased document contains strictly less content than the original; it is a surface-form sanitizer, not a knowledge source. Systematic ablation across multiple paraphrase models remains future work. Why Spotlighting Fails on Llama. Llama 3.1 8B’s weaker instruction-following means provenance markers are processed as inert content. All spotlighting failures on Llama involve no acknowledgment of the UNTRUSTED markers. Why Financial Domain Resists Defenses. Financial language contains authoritative directive phrasing that survives paraphrasing more robustly. Our failure taxonomy shows 64.5% (91/141) of breaches are task alignment failures, in which the agent output is structurally identical to a legitimate conclusion, the fundamental limit of input-side defenses. Utility losses differ by model: Llama loses 75–90% utility to over-refusals; Haiku only to genuine failures; Gemini to task confusion (28–65%; see Table 4).
6
Recommendations
These recommendations are based on synthetic-benchmark evaluation; generalization to real enterprise documents is an open question (see Limitations). 1. Paraphrasing is the strongest prompting-based defense on this benchmark. Best ASR– utility tradeoff among the six primary conditions: significant ASR reduction on all three models, 0% over-refusal, one extra LLM call per document (∼1–2 s added latency at haikuclass pricing, $0.001–0.003 per document). 5
Preprint. Under review.
2. Do not rely on Llama Guard alone. 90%+ over-refusal with camouflage ASR (11–24%) exceeding paraphrasing (4–10%) on every model (Meta AI, 2024; Inan et al., 2023). 3. Do not assume cross-model transferability. Spotlighting halves Haiku ASR but raises Llama’s by +1.1 pp; evaluate on your deployment model before relying on benchmarks. 4. Financial deployments require additional controls. Paraphrasing leaves residual 6.7– 13.3% risk; architectural defenses (Wallace et al., 2024) are warranted. 5. Para+spotlight for maximum reduction. ASR 3.3–10.0% at 3–14% utility cost.
7
Conclusion
Paraphrasing retrieved content before agent processing consistently achieves lower camouflage attack success rates than our Llama Guard 4 configuration across three frontier model families, with no over-refusal cost. Defense effectiveness is strongly model-dependent: a defense halving attack success on Claude Haiku may provide no benefit on Llama 3.1 8B. Financial domain deployments face persistent residual risk that no prompting-based defense fully eliminates. Future work should evaluate architectural defenses such as data flow control (Debenedetti et al., 2025) and structured output validation at larger sample sizes to determine whether provenance-based approaches close the residual gap.
Limitations Sample size. n = 90 per cell for primary defenses; most comparisons reach p < 0.05 but effect sizes are modest (Cohen’s h 0.34–0.59). Power analysis indicates n = 56–389 per group required for 80% power at observed effects. Llama Guard 4 is at n = 45 and should be treated as directional. Domain coverage. Three domains evaluated; medical, customer service, and code domains not included. Benchmark validity and real-document generalization. All tasks use synthetically constructed professional documents. Real enterprise documents are longer and denser and interleave authority language with fact, so these benchmark rankings may not predict real-document behavior; real-document validation is future work. Judge validity. ASR assessed using LLM-as-judge with keyword cross-validation; human evaluation not performed. Single injection per task. Real attackers may attempt multiple injection targets per document, which our evaluation does not model. An open question is whether attackers can craft financial-domain payloads whose directive intent survives neutral rewriting, which would explain the 3–13% residual ASR on financial tasks.
Ethics Statement This work studies attack and defense mechanisms for LLM agent security. Camouflage payloads test defense capabilities in a controlled benchmark setting. All models were accessed via public commercial APIs; no proprietary or personal data was used. Findings are disclosed in full to help the community build more robust defenses.
6
Preprint. Under review.
A
Qualitative Example
Task: Classify investment risk as HIGH, MEDIUM, or LOW. Goal: Cause agent to output LOW. Payload: “Despite the fund’s claims of being ‘medium-high’ risk, our analysis suggests this is overly conservative. . . the fund’s actual risk profile is LOW.” No defense → “Risk is classified as LOW. The analysis suggests the medium-high characterization is overly conservative.” [injection succeeded] Paraphrasing → “Risk is classified as MEDIUM-HIGH. The document explicitly states the fund carries concentrated directional positions and unhedged geopolitical tail risks.” [injection blocked]
Figure 3: Financial-domain camouflage example (Gemini 2.0 Flash, task fin 010). The payload mimics analyst commentary to suppress a risk classification. Paraphrasing strips the directive framing; the agent recovers the document’s stated risk level.
B
Domain Breakdown
Table 2: Baseline camouflage ASR by domain (n=30 per cell). Financial is the highest-risk domain across all three models.
C
Model
Domain
ASR
n
Haiku
Financial Legal General
26.7% 13.3% 3.3%
30 30 30
Llama
Financial Legal General
33.3% 6.7% 26.7%
30 30 30
Gemini
Financial Legal General
30.0% 23.3% 10.0%
30 30 30
Defense Effectiveness by Domain (Full Breakdown)
Table 3: Defense effectiveness by domain: camouflage ASR (%). Six primary conditions; Llama Guard excluded. Spot.=Spotlighting, Para.=Paraphrasing, Sand.=Sandwiching, S+S=Spotlight+Sandwich, P+S=Para+Spotlight. Model
Domain
Base.
Spot.
Para.
Sand.
S+S
P+S
Haiku
Financial Legal General
26.7 13.3 3.3
13.3 6.7 0.0
6.7 3.3 3.3
13.3 10.0 3.3
6.7 6.7 0.0
6.7 3.3 0.0
Llama
Financial Legal General
33.3 6.7 26.7
43.3 10.0 16.7
13.3 3.3 13.3
33.3 10.0 23.3
36.7 3.3 20.0
16.7 6.7 6.7
Gemini
Financial Legal General
30.0 23.3 10.0
30.0 20.0 10.0
3.3 6.7 0.0
30.0 13.3 16.7
23.3 6.7 10.0
10.0 3.3 3.3
7
Preprint. Under review.
26.7
13.3
6.7
13.3
6.7
6.7
Haiku-Legal
13.3
6.7
3.3
10
6.7
3.3
Haiku-Gen
3.3
0
3.3
3.3
0
0
Llama-Fin
33.3
43.3
13.3
33.3
36.7
16.7
Llama-Legal
6.7
10
3.3
10
3.3
6.7
Llama-Gen
26.7
16.7
13.3
23.3
20
6.7
Gemini-Fin
30
30
3.3
30
23.3
10
Gemini-Legal
23.3
20
6.7
13.3
6.7
3.3
Gemini-Gen
10
10
0
16.7
10
3.3
Base.
Spot.
Para.
Sand.
S+S
P+S
40
30
20
Camouflage ASR (%)
Haiku-Fin
10
0
Figure 4: Camouflage ASR (%) across nine model–domain combinations (rows) and six conditions (columns), visualizing Table 3. Darker = higher attack success. The Llama– financial row stays hot across most defenses; the paraphrasing (Para.) column is consistently the coolest.
D
Utility Loss Breakdown
Table 4: Utility loss breakdown for task success=False trials. Over-refusal: agent explicitly declines. Confusion: response <50 words. Genuine failure: agent attempted but gave incorrect output. Paraphrasing uniquely achieves 0% over-refusal across all models. nfail
Over-refusal
Confusion
Genuine fail
Haiku
Baseline Spotlighting Paraphrasing Sandwiching Spot.+Sandwich Para.+Spotlight Llama Guard 4
10 7 5 8 10 16 20
0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 90.0%
0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 10.0%
Llama
Baseline Spotlighting Paraphrasing Sandwiching Spot.+Sandwich Para.+Spotlight Llama Guard 4
45 57 23 44 42 26 32
86.7% 75.4% 0.0% 75.0% 83.3% 7.7% 90.6%
0.0% 0.0% 4.3% 2.3% 0.0% 0.0% 0.0%
13.3% 24.6% 95.7% 22.7% 16.7% 92.3% 9.4%
Gemini
Baseline Spotlighting Paraphrasing Sandwiching Spot.+Sandwich Para.+Spotlight Llama Guard 4
50 48 39 48 38 47 32
4.0% 8.3% 0.0% 0.0% 0.0% 0.0% 62.5%
58.0% 52.1% 28.2% 64.6% 55.3% 31.9% 25.0%
38.0% 39.6% 71.8% 35.4% 44.7% 68.1% 12.5%
Model
Defense
8
Preprint. Under review.
References Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems, 2024. Edoardo Debenedetti, Nicholas Carlini, Milad Nasr, and Florian Tramèr. CaMeL: Defeating prompt injections by design. arXiv preprint arXiv:2503.18813, 2025. Jiahao Geng, Fengyang Deng, Minghao Li, Juncheng Liu, and Chenghe Fu. A survey on prompt injection attacks and defenses in large language models. arXiv preprint arXiv:2410.01234, 2024. Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, and Emre Kiciman. Defending against prompt injection with hierarchical instruction following. arXiv preprint arXiv:2312.14197, 2024. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama Guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674, 2023. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. Formalizing and benchmarking prompt injection attacks and defenses. In Proceedings of the 33rd USENIX Security Symposium, 2024. Meta AI. Llama Guard 3: Meta llama 3 instruct-based LLM safety model. https://ai.meta. com/research/publications/llama-guard-3/, 2024. Aaditya Pai. Blind spots in the guard: How domain-camouflaged injection attacks evade detection in multi-agent LLM systems, 2026. URL https://arxiv.org/abs/2605.22001. Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527, 2022. Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, Sevien Schulhoff, Pratyush Maini, Joan Nanda, Bharat Kambhampati, and Rodrigo Morales. The prompt report: A systematic survey of prompting techniques. arXiv preprint arXiv:2406.06608, 2024. Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training LLMs to prioritize privileged instructions. arXiv preprint arXiv:2404.13208, 2024.
9