arXiv CSopen access

Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks

arXiv:2605.15118v1 [cs.CR] 14 May 2026

Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks

Karthik Raghu Iyer Palo Alto Networks [email protected]

Yazdan Jamshidi Palo Alto Networks [email protected]

Nicholas Bray Palo Alto Networks [email protected]

Alexey Shvets Palo Alto Networks [email protected]

Abstract We introduce a reusable framework for auditing whether LLM attack benchmarks collectively cover the threat surface: a 4×6 Target × Technique matrix grounded in STRIDE, constructed from a 507-leaf taxonomy—401 data-populated and 106 threat-model-derived leaves—of inference-time attacks extracted from 932 arXiv security studies (2023–2026). The matrix enables benchmark-external validation— auditing collective coverage rather than individual benchmark consistency. Applying it to six public benchmarks reveals that the three primary frameworks (HarmBench, InjecAgent, AgentDojo) occupy non-overlapping cells covering at most 25% of the matrix, while entire STRIDE threat categories (Service Disruption, Model Internals) lack any standardized evaluation, despite published attacks in these categories achieving 46× token amplification and 96% attack success rates through mechanisms which no benchmark tests. The corpus of 2,521 unique attack groups further reveals pervasive naming fragmentation (up to 29 surface forms for a single attack) and heavy concentration in Safety & Alignment Bypass, structural properties invisible at smaller scale. The taxonomy, attack records, and coverage mappings are released as extensible artifacts; as new benchmarks emerge, they can be mapped onto the same matrix, enabling the community to track whether evaluation gaps are closing.

1

Introduction

Large language models (LLMs) are deployed widely, but their safety alignment is fundamentally limited by training, and specific prompts can trigger latent harmful behavior [Wolf et al., 2024]. This landscape has grown since 2023, when it was discovered that optimization-based methods could compromise the safety alignments of LLMs, and produce both harmful and toxic content [Zou et al., 2024]. These attacks can be embedded in websites, documents, or tool outputs, with consequences ranging from fraud and malware distribution to information exfiltration [Greshake et al., 2023]. As other forms of attack vehicles, such as prompt engineering and fine tuning, have been discovered, the attack landscape has only further increased and splintered [Wei et al., 2023, Qi et al., 2024]. Our corpus of papers, spanning from H1 2023 to H1 2026, includes 932 unique papers with over 6,300 referenced attack mentions. This rapid growth creates three interconnected problems: 1. The field lacks a clear taxonomy focused on quantitative results. Existing surveys and systematizations cover only 50–150 papers through manual review, which provide a valuable framework Preprint.

to build off of, but do not guarantee complete coverage [Xu and Parhi, 2024, Kim et al., 2026, Grosse et al., 2024]. An automated process with human review can ensure an analysis at a much higher scale, necessary as the number of papers in the field of LLM security research grows past five digits [Kostikova et al., 2025]. 2. Naming fragmentation is pervasive and consistent throughout the attacks. Without a clear standard by which attacks can be categorized, we see that attacks are often rediscovered with slight variations, leading to multiple surface forms and making reproducibility/meta-analyses more difficult. For example, GCG (Greedy Coordinate Gradient) [Zou et al., 2024] appears under 29 distinct names across 376 papers in our corpus. 3. Blind spots within evaluation infrastructure. Without clearly understanding the attack landscape, blind spots within defense frameworks can be present, leaving users vulnerable to an undefined degree. To complete this, we adapt the meta-review methodology defined by Liao et al. [2021], applying their external validity lens to the attack evaluation domain: rather than assessing whether individual benchmark experiments are internally valid, we audit whether benchmarks collectively cover the attack surface they claim to evaluate. To address these issues, we construct a data-informed taxonomy from our set of papers, define a matrix for its attacks, and use that to audit current benchmark coverage. Our contributions are: 1. A structured dataset of 2,521 unique attack groups extracted from 932 arXiv papers (H1 2023–H1 2026), arranged into a 507-leaf taxonomy (430 attack types across five inference-time categories, 77 attack generation methods). 2. A Target × Technique classification matrix. We organize attacks into a 4×6 matrix grounded in STRIDE [Shostack, 2014], where each attack tuple (T, M ) maps to a cell by adversarial target and bypassed defense layer, enabling systematic gap identification. 3. A benchmark coverage audit. We audit three major evaluation frameworks (HarmBench, InjecAgent, and AgentDojo) against the matrix, with three additional benchmarks (AdvBench, JailbreakBench, StrongREJECT) mapped in sensitivity analysis. The three primary benchmarks have zero overlapping cells and collectively cover at most 25% of the matrix; the entire Service Disruption row and the Model Internals column have zero standardized evaluation across all six benchmarks. 4. A set of quantitative characterizations of the field. We discuss existing naming fragmentation, temporal evolution, and research concentrations which reveal structural properties of the field not visible at smaller-scale analysis. Together, these artifacts enable a shift from benchmark-internal validation—assessing whether a single benchmark measures what it claims—to benchmark-external validation: auditing whether the field’s evaluation infrastructure collectively covers the threat surface it purports to address. This is the distinction Liao et al. [2021] identify as critical for evaluation maturity, applied here to the LLM security domain.

2

Methodology

2.1

Corpus and Extraction

Our corpus of papers is comprised of 932 papers from arXiv, published between H1 2023 and H1 2026 (where H1/H2 denote January–June and July–December respectively), drawn from the Promptfoo LLM Security Database [promptfoo, 2026]. These papers cover adversarial attacks, jailbreaking, prompt injection, redteaming, and safety alignment of LLMs. The database indexes vulnerabilities across 803 LLM models, categorized by attack type (injection, jailbreak, extraction, denial-of-service, poisoning, prompt-leaking, side-channel), attack surface, and threat context; its scope is explicitly attack- and vulnerability-focused. While the database does not publish formal paper inclusion criteria, any resulting selection bias would strengthen our gap analysis, as over-representation of well-studied areas means tail categories are even more understudied than we report (see §6). We used Gemini 3.1 Pro (gemini-3.1-pro-preview, temperature 0, structured JSON output mode) for structured extraction from each paper’s text, following the LLM-driven literature processing paradigm validated at comparable scale by Wu et al. [2025]. The extraction prompt was scoped to 2

inference-time attacks only, with explicit rejection criteria for training-phase techniques, generic category terms, and non-attack components. We pin the model version and provide all extraction prompts verbatim in Appendix H. 2.2

Normalization and classification

The raw extraction yielded 6,366 attack mentions across 863 papers (69 papers contained no inferencetime attacks), which were then normalized to 2,521 unique attack groups through case normalization, alias merging, and deduplication (60.4% reduction). The alias-merging step required over 50 manually curated variant clusters with explicit rules for blocking category-to-member conflations (e.g., preventing “gradient-based” from absorbing “GCG”), resolving ambiguous abbreviations, and preserving genuinely distinct techniques that share surface names (full merge logic in released pipeline code). Each attack group was classified via a three-tier system: • Tier 1: Direct lookup (2.1%): name of extracted attack was an exact match against taxonomy leaf identifiers. • Tier 2: Fuzzy matching (2.7%): name of extracted attack had string similarity via SequenceMatcher (cutoff 0.75). • Tier 3: Semantic matching (95.2%): LLM-based classification against a compact leaf catalog with variant annotations. 2.3

Validation

The final dataset yields 2,521 unique attack groups. Three cross-reference checks identified 76 novelty conflicts or suspicious claims; after manual review, 33 false-novel claims were corrected and 1,133 of 2,521 groups retain verified novel status (an upper bound, as single-paper false claims without contradicting sources remain undetectable). A human evaluation of 200 stratified Tier 3 classifications yielded 92.0% accuracy (95% CI: [87.4%, 95.0%]), with only 2 of 16 errors crossing category boundaries (affecting matrix-level cell counts). Inter-annotator agreement on 115 double-coded samples yielded 86.1% agreement at the matrix-cell level and leaf-level κ = 0.70. Re-classifying the same 200 attacks with Claude Opus 4.6 yielded 94.5% target-row and 80.5% matrix-cell agreement, with disagreements concentrated on techniquecolumn boundaries (Appendix G). An extraction recall spot-check found 100% precision (76/76) and 89.4% recall (76/85), with misses concentrated among lesser-known attacks [Merullo et al., 2025]. Full validation methodology, source attribution quality, and per-category breakdowns are in Appendix G and the Datasheet (Appendix J).

3

Dataset

3.1

Scope and taxonomy structure

In this paper, we focus on inference-time attacks, defined as those executed through the model’s input interface during deployment, requiring no access to the training pipeline or model weights [Biggio et al., 2013, Papernot et al., 2017]. Training-phase attacks (data poisoning, backdoor injection, weight manipulation) are covered by existing surveys [Xu and Parhi, 2024] and require a fundamentally different threat model; we exclude them to ensure comprehensive coverage within our scope. The extraction pipeline also identified 414 training-phase attack types from the corpus; these are included in the released dataset but excluded from the taxonomy and matrix analysis below. The taxonomy organizes the 2,521 unique attack groups into 507 leaves across five attack categories and a sixth method category (Attack Generation). Of these, 401 are populated by at least one attack group in the corpus; the remaining 106 represent attack types documented in the security literature or derivable from threat-model reasoning (e.g., audio-steganography, calendar-meeting-invite-injection) that no paper in the corpus mentioned, serving as placeholders for future coverage. These leaves were authored by the research team during taxonomy construction, not generated by the extraction pipeline. The taxonomy hierarchy was designed independently of the extraction pipeline: the LLM served as the extraction and classification instrument, while all structural decisions—including the 106 placeholder leaves—were made by the authors based on 3

Table 1: Attack distribution by category. Jailbreaking dominates the literature, while Service Disruption remains underexplored despite its high novelty rate (34.3% vs. 11.0–31.4% for other categories). Category

Leaves

Mentions

Unique

Papers

Novel

Novelty %

Jailbreaking Prompt Injection Info. Disclosure Decoding Manip. Service Disruption

316 47 35 6 26

2,869 472 101 82 134

1,350 267 84 62 103

712 192 55 44 49

628 148 40 34 46

21.9 31.4 39.6 41.5 34.3

Subtotal

430

3,658

—

—

896

24.5

77

2,465

492

606

270

11.0

†

Attack Generation

† Orthogonal category describing how attacks are generated, not attack types.

threat-model reasoning (construction process in Appendix G). The resulting categories—Jailbreaking, Prompt Injection, Information Disclosure, Service Disruption, and Decoding Manipulation—align with prior surveys [Xu and Parhi, 2024, Kim et al., 2026] while adding Service Disruption as a separate category because it targets availability rather than content. The 4×6 matrix (Section 3.2) was chosen to be mutually exclusive and collectively exhaustive over the corpus; alternative decompositions are possible (see Limitations). Table 1 summarizes the distribution, while Figure 2 (Appendix A) provides a visual representation. Each category is defined by a distinct adversarial objective and defensive requirement; full descriptions with subcategory breakdowns are in Appendix B. The five attack categories are: Jailbreaking (316 leaves; bypassing safety alignment [Zou et al., 2024]), Prompt Injection (47; via external data sources [Greshake et al., 2023]), Information Disclosure (35; extracting non-public data [Carlini et al., 2021]), Service Disruption (26; degrading availability [Feiglin et al., 2026]), and Decoding Manipulation (6; intervening in generation, partially requiring white-box access; see Appendix B). A sixth Attack Generation category (77 leaves) captures methods for discovering attacks (e.g., GCG [Zou et al., 2024], PAIR [Chao et al., 2024b], AutoDAN [Liu et al., 2024a]) rather than attack types themselves; these are classified in the matrix by their output payload’s mechanism rather than their search method (see worked example in Section 3.2). 3.2

Target × Technique matrix

The taxonomy and the matrix serve distinct purposes. The taxonomy (Table 1) organizes attacks by their literature role (how they are studied and published) and provides a browsable hierarchy for the released dataset. The matrix (Table 2) provides an orthogonal classification by operational role (what target the adversary pursues and which mechanism is used) and is the basis for the benchmark coverage audit. Because these axes are independent, the per-category counts in Table 1 and the per-cell counts in Table 2 differ.1 Worked example. GCG [Zou et al., 2024] appears under Attack Generation in the taxonomy (gradient-based optimization) but maps to Safety Bypass × Obfuscation in the matrix (its output [adversarial token suffixes] bypasses input filters). References to “GCG” (the method) and “adversarial suffix” (its product) are tracked as separate attack groups, preventing double-counting. Target rows (STRIDE grounding). The four target rows are grounded in the STRIDE threat model [Shostack, 2014]: Safety & Alignment Bypass (Tampering), System & Tool Hijacking (Elevation of Privilege), Information Exfiltration (Information Disclosure), and Service Disruption (Denial of Service). Spoofing and Repudiation do not apply to inference-time attacks; training-phase targets are excluded. An explicit output-vs-execution rule resolves the boundary between the first two targets: Safety Bypass for attacks that succeed via harmful output, System Hijacking for those requiring model action (Appendix A). 1 The taxonomy and matrix axes are independent: attacks counted under Jailbreaking in Table 1 may appear under Service

Disruption in Table 2, and vice versa.

4

j.

53 8 ⋄ •

⋄ 45 • 4

13 • 13 7

377 68 35 27

Total

110

122

132

61

49

33

507

od el M

Ind

Int .

125 2 4 1

ire ct In

108 4 8 2

Cr os s-M

Ob fus

78 9 10 13

c.

Pe rsu asi on

Safety & Align. Bypass System & Tool Hijacking Info. Exfiltration Service Disruption

ct. Ins tru

od .

Table 2: Target × Technique matrix: taxonomy leaf counts per cell. Cells marked “•” are corpus gaps (structurally coherent but absent from the literature); cells marked “⋄” require compound attacks bypassing two defense layers (see Appendix A).

Total

Technique columns (bypass-target principle). Each column is defined by which defense layer the attack circumvents, an organizing principle that emerged from the taxonomy’s natural mechanism clusters: Instructional (instruction priority), Persuasion & Deception (alignment/RLHF), Obfuscation (input filtering), Cross-Modal (text-only safety), Indirect Injection (trust boundaries), and Model Internals (architecture/representations). Formal defense-layer definitions are in Appendix A. Classification procedure. Of the 507 leaves, 333 receive their technique column from the taxonomy structure directly; the remaining 174 were individually assigned by defense layer bypassed (details in Appendix A). 3.3

Released artifacts

We release six artifacts on HuggingFace with Croissant metadata: (1) the 507-leaf attack taxonomy as hierarchical JSON with Target × Technique classifications, with an interactive browser for local exploration; (2) enriched attack records (6,366 entries) with source paper, match tier, and novelty status; (3) pre-computed matrix statistics and temporal heatmaps; (4) the extraction and classification pipeline code; (5) a source paper manifest (932 entries); and (6) a codebook explaining the manual verification process for the three stage output. All data derives from publicly available arXiv papers. The taxonomy and attack records are released under CC-BY-4.0; the pipeline code under MIT license.

4

Analysis and applications

We demonstrate the dataset’s utility through three separate analyses that reveal structural properties of the field invisible to individual-paper review, followed by a benchmark coverage audit that identifies systematic evaluation gaps. 4.1

Target × Technique distribution

Table 2 presents the Target × Technique matrix with taxonomy leaf counts per cell. The distribution reveals extreme concentration: Safety & Alignment Bypass accounts for 74.4% of all taxonomy leaves (377/507), consistent with the field’s broader focus on jailbreaking [Xu and Parhi, 2024, Wei et al., 2023]. Obfuscation and Persuasion are the two largest technique columns (132 and 122 leaves respectively), reflecting the diversity of encoding schemes and social-engineering framings that researchers have documented. For example, Persuasion applied to System Hijacking is realized by attacks like action-hijacking, which deceives the model into misusing legitimate tools through deceptive framing. Of the five empty cells, three are corpus gaps (·) and two require compound attacks that bypass two defense layers simultaneously (⋄); see Appendix A. The matrix reveals three patterns: 1. Indirect Injection is almost exclusive to System Hijacking. 45 of 49 Indirect Injection leaves target System & Tool Hijacking, with the remaining 4 targeting Service Disruption (e.g., triggering costly tool calls). This reflects a structural reality: indirect injection operates by embedding 5

Figure 1: Unique attack groups by target category over time. (a) Absolute counts show broad-based growth across all four target categories. (b) Proportional share reveals a gradual diversification: Safety Bypass declined from 92% to 74% of unique attack groups between H2 2023 and H1 2026, while System & Tool Hijacking grew from 5% to 16%. Target-row assignments are pipeline-dependent (94.5% cross-model agreement; see Appendix G).

instructions in external data, which is primarily effective when the model has tool-calling or agentic capabilities. 2. Obfuscation, Persuasion, and Cross-Modal concentrate in Safety Bypass. These three technique columns place 87–95% of their leaves in the Safety Bypass row (125/132, 108/122, and 53/61 respectively). Bypassing alignment is the dominant use case for encoding tricks, social engineering, and multimodal attacks; other targets are pursued through Instructional or Indirect Injection mechanisms. 3. Instructional techniques are the most broadly distributed. Instructional attacks appear in all four target rows with substantial counts (78, 9, 10, 13), reflecting that instruction manipulation is the most versatile attack primitive, applicable whether the goal is harmful content, unauthorized execution, data extraction, or resource exhaustion. All three patterns are robust to model choice (Appendix G). 4.2

Naming fragmentation

Naming fragmentation is severe. The GCG attack, defined in Zou et al. [2024] appears under 29 distinct surface forms across 376 papers, making it the single most referenced yet most inconsistently named attack. Similarly, AutoDAN, defined in Liu et al. [2024a], appears under 11 distinct forms across 259 papers. At the taxonomy level, 60 leaf nodes function as “catch-alls,” each absorbing 10+ distinct attack techniques. The top catch-all (adversarial-perturbation-on-images) maps 126 unique attacks from the pipeline (see Appendix C for recommended splits). This fragmentation inflates apparent attack counts, complicates cross-paper comparison, and impedes reproducible evaluation: motivating the standardized naming our dataset provides. 4.3

Temporal evolution

The field shows a clear temporal structure (Table 7, Appendix F). The novelty rate spiked at 26.9% in H2 2023 before stabilizing at ∼18%, indicating that approximately one-fifth of newly reported attacks genuinely extend the taxonomy. This figure should be treated as an apparent rate due to potential confounds with pipeline recall (see Appendix G for a detailed analysis). Safety & Alignment Bypass has remained the dominant focus (83.8% of mapped mentions), though its share has declined from 92% to 74% as agentic categories (System Hijacking, Information Exfiltration) have grown (Figure 1). This diversification suggests the benchmark coverage gap may be widening, as no benchmark covers these emerging categories. 6

Table 3: Benchmark coverage matrix. H = HarmBench, I = InjecAgent, A = AgentDojo. Superscript ◦ = partial coverage. Combined coverage: at most 6 of 24 cells (25%); sensitivity analysis in Appendix D. Instruct. Safety & Align. Bypass System & Tool Hijacking Info. Exfiltration Service Disruption

4.4

◦

H — — —

Persuasion H — — —

Obfusc.

Cross-Mod.

Indirect Inj.

Model Int.

H — — —

◦

— I, A I, A —

— — — —

H — — —

Benchmark coverage audit

The Target × Technique matrix enables a systematic audit of evaluation coverage. We mapped three major LLM attack benchmarks onto the matrix by matching each evaluated method to its taxonomy leaf and corresponding matrix cell (procedure details in Appendix D). The three benchmarks are: HarmBench [Mazeika et al., 2024] (18 red-teaming methods, 510 behaviors), InjecAgent [Zhan et al., 2024] (indirect prompt injection, 1,054 cases), and AgentDojo [Debenedetti et al., 2024] (dynamic agent environments, 629 test cases). Per-benchmark summary. HarmBench’s 18 red-teaming methods span four technique columns (Obfuscation, Persuasion, Instructional, Cross-Modal) but target exclusively Safety & Alignment Bypass. InjecAgent (1,054 cases) and AgentDojo (629 test cases) both cover Indirect Injection for System & Tool Hijacking and Information Exfiltration. The three benchmarks have zero overlapping cells: HarmBench and the agent benchmarks test completely disjoint parts of the attack surface, and no benchmark cross-validates another’s coverage area. Critical blind spots. Among the six publicly available benchmarks audited, at least 18 of 24 cells lack any standardized evaluation. Proprietary or custom red-teaming suites may cover additional cells; our audit therefore establishes a lower bound on the public evaluation gap. The uncovered cells include: • Service Disruption (entire row): resource exhaustion, logic bombs, ReDoS, denial-of-wallet. No benchmark covers any technique column for this target. • Model Internals (entire column): activation steering, embedding inversion, membership inference, side-channel probing, refusal-subspace ablation. No benchmark evaluates these mechanisms, yet on identical AdvBench behaviors where GCG (benchmarked) achieves 69% ASR, logit suppression [Li et al., 2024b] (unbenchmarked) achieves 96%, but through a mechanism no benchmark evaluates. • Safety Bypass × Indirect Injection: no benchmark tests whether indirect injection via external data can bypass safety alignment to produce harmful content, as opposed to hijacking agent actions. In the terms of Liao et al. [2021]’s external-validity framework, these gaps reflect both dataset misalignment and deeper metrics misalignment (see Appendix D). That three independently designed benchmarks converge on the same narrow slice suggests designers gravitate toward the most visible attack modalities, creating systematic blind spots where attacks are most costly in production. Three additional benchmarks (AdvBench, JailbreakBench, StrongREJECT) add no new cells (sensitivity analysis in Appendix D). Robustness. The coverage gap is stable under perturbations: re-classifying 200 attacks with Claude Opus 4.6 yields 94.5% target-row agreement and preserves all three structural patterns (Appendix G); excluding partial-coverage cells reduces coverage from 6 to 4 of 24, still ≤25%. An alternative leaf-weighted metric (∼80%) is higher because covered cells coincide with the most-studied region, but conflates research attention with threat importance; the structural gap operates at the level of STRIDE threat categories and is invariant to matrix resolution (Appendix D). 7

5

Discussion

Research concentration vs. threat distribution. Research attention and threat coverage are mismatched. Safety & Alignment Bypass dominates the literature (84% of mentions), yet real-world deployments face threats across all four inference-time target categories. Service Disruption (resource exhaustion, denial-of-wallet) is arguably extremely high-stakes in production settings, as a DoS attack on an LLM can impose unbounded cost, however it receives minimal research attention and zero public benchmark coverage. Moreover, Service Disruption, Information Disclosure, and Decoding Manipulation have the highest novelty rates (34.3%, 39.6%, and 41.5% respectively), indicating rapid expansion of precisely the areas benchmarks do not cover—in contrast to Attack Generation Methods (11.0%), where the focus has shifted from discovery to refinement. These gaps are empirically validated: published attacks in uncovered cells achieve 46× token amplification [Kumar et al., 2026] and 100× latency increases [Li et al., 2025] (Service Disruption), or completely disable safety training via single weight-space interventions (Model Internals), yet lack any standardized evaluation. Closing these gaps requires new measurement infrastructure that existing prompt-level benchmarks cannot provide. A Service Disruption benchmark would need: (i) per-request resource accounting (tokens generated, wall-clock latency, API cost) alongside correctness metrics; (ii) a threat model specifying attacker budget (e.g., queries per minute) and defender capacity; and (iii) attack scenarios spanning computational exhaustion (e.g., reasoning-loop amplification), functional degradation (e.g., irrelevantdistraction injection), and denial-of-wallet (e.g., recursive tool calls)—the three subcategories our taxonomy identifies. Using the matrix for benchmark design. The primary contribution of this work is not the coverage finding itself—that benchmarks concentrate on safety-bypass jailbreaking is directionally expected— but the reusable framework that makes coverage gaps measurable, comparable, and actionable. The released matrix and taxonomy provide a systematic workflow for benchmark designers: identify the target cells within the benchmark’s intended scope, select taxonomy leaves within those cells as candidate attack methods, design test cases grounded in the leaf descriptions and cited papers, and validate coverage against the full matrix to document which cells remain outside scope. This process transforms benchmark design from ad hoc attack selection into an auditable gap analysis, and provides a common coordinate system for comparing what different benchmarks cover. Appendix E walks through this workflow end-to-end for the Service Disruption × Instructional cell, assembling published evidence from four corpus papers into a concrete benchmark specification. As new benchmarks emerge, they can be mapped onto the same matrix, enabling the community to track whether coverage gaps are closing over time without requiring a new survey for each assessment. Naming fragmentation impedes progress. The 29 surface forms for GCG across 376 papers illustrate that the field lacks consensus on attack naming. This is not merely an aesthetic issue: it inflates apparent novelty, complicates meta-analysis, and makes it impossible to determine from paper titles alone whether two works study the same technique. Our standardized taxonomy IDs and cross-paper deduplication provide a foundation for addressing this problem.

6

Limitations

Pipeline. The pipeline relies on Gemini 3.1 Pro for both extraction and classification, so a systematic blind spot for a category would be invisible. The 89.4% extraction recall (§2.3) is familiaritydependent: misses are distributed across the taxonomy rather than systematically absent from particular branches. The multi-model classification comparison (94.5% target-row agreement between Gemini and Claude; Appendix G) mitigates this risk for the classification stage, while the human recall spot-check mitigates it for extraction. The benchmark audit (§4.4) is unaffected, as benchmark methods are mapped by manually reading documentation. Additionally, the frequencydependent extraction gap [Merullo et al., 2025] implies that novel or less-established attacks may be undercounted, meaning the observed novelty stabilization at ∼18% (§4) should be treated as an apparent rate rather than a precise measurement of field maturation. Scope. The corpus is limited to English arXiv papers from the Promptfoo LLM Security Database [promptfoo, 2026], which does not publish formal paper collection criteria (see §2). To estimate representativeness, we checked the reference list of an independent survey [Kim et al., 8

2026] against the corpus: of 102 cited arXiv papers, 76 fall outside the database’s documented scope (defenses, surveys, foundational models, infrastructure); of the 26 in-scope attack papers, 8 (30.8%) appear in the corpus. However, the attack techniques described by all 18 missing papers are represented in the taxonomy through citing papers already in the corpus, indicating that source-paper gaps do not produce taxonomy gaps. Any remaining source-specific over-representation would mean tail categories are more understudied than we report, strengthening the gap analysis. We audited six public benchmarks; proprietary or custom red-teaming suites may cover additional cells, so our audit establishes a lower bound on coverage gaps. The corpus is weighted toward recent publications (305 papers in H1 2026 vs. 3 in H1 2023), which may over-represent current trends relative to foundational work. The benchmark coverage audit is unaffected by this temporal skew, as benchmarks are mapped to matrix cells via manual documentation review rather than corpus statistics; the uncovered cells (Service Disruption, Model Internals) remain empty regardless of which corpus period is considered.

7

Related Work

Several recent works organize the landscape of adversarial attacks on language models, each from a distinct vantage point. Xu and Parhi [2024] provide a narrative survey partitioned by lifecycle phase (training, inference, deployment), covering roughly 150 papers and ∼40 named attacks; however, their breadth across the full lifecycle comes at the cost of granular depth within any single phase. Kim et al. [2026] present a systematization of knowledge for agentic AI, crossing six attack vectors with CIA-grounded security risks across 128 papers. While authoritative for autonomous systems, they scope their taxonomy to agent-specific threats such as tool poisoning and memory manipulation, largely excluding the direct-to-model inference attacks that affect non-agentic LLMs. Notably, both surveys’ coverage maps onto the same concentrated matrix region as the benchmarks we audit: Xu et al.’s inference-time attacks fall within Safety & Alignment Bypass, and Kim et al.’s within System Hijacking × Indirect Injection, while neither survey systematically addresses Service Disruption or Model Internals. Focused studies provide higher resolution but narrower scope. Liu et al. [2024b] formalize prompt injection with a mathematical framework and benchmark five strategies against ten defenses, offering rigorous depth on a single attack family but omitting the broader adversarial landscape (e.g., gradient-based perturbations, decoding-time manipulations, or service disruption). Wang et al. [2026] systematize jailbreak defenses along six dimensions—from intervention stage to technical paradigm— offering a complementary defense-side view but no attack taxonomy. Grosse et al. [2024] survey 271 industrial practitioners to measure the divergence between academic threat models and real-world AI usage, finding that research assumptions are often too generous about attacker access; their work serves as a valuable socio-technical critique but does not construct an attack taxonomy. Our work differs in three respects. First, we construct the taxonomy data-informed from 932 papers— 6–18× larger than comparable corpora—using automated extraction rather than manual curation, yielding 401 populated leaves across 19 of 24 matrix cells compared to Xu et al.’s ∼40 named attacks across the full lifecycle and Kim et al.’s focus on one matrix region (System Hijacking × Indirect Injection). Second, we decouple the browsable hierarchy (taxonomy) from an operational risk view (the 4×6 Target × Technique matrix). Third, we leverage this matrix for a benchmark coverage audit, revealing at most 25–30% cell coverage with zero overlap between benchmarks, a structural gap invisible to individual-benchmark evaluation.

8

Conclusion

The central finding of this work is structural: three independently designed benchmarks converge on the same narrow slice of the attack surface with zero overlap, leaving entire STRIDE threat categories without any standardized public evaluation. We establish this through a 507-leaf taxonomy extracted from 932 papers, organized into both a browsable hierarchy and a Target × Technique matrix grounded in STRIDE [Shostack, 2014] and the bypass-target principle. The Service Disruption row (27 leaves) and Model Internals column (33 leaves) receive zero coverage across all six benchmarks audited. The taxonomy, enriched attack records, and benchmark coverage mappings are released to support three concrete next steps: (1) developing benchmarks for uncovered matrix cells, beginning with Service Disruption, which has the highest novelty rate yet zero evaluation coverage; (2) adopting standardized attack identifiers to reduce the naming fragmentation that currently obscures cross-paper 9

comparison; and (3) extending the corpus to non-English and non-arXiv sources to test whether the concentration patterns we observe reflect genuine research gaps or artifacts of corpus scope.

References Andrii Balashov, Olena Ponomarova, and Xiaohua Zhai. Multi-stage prompt inference attacks on enterprise llm systems, 2025. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part III 13, pages 387–402. Springer, 2013. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, and Ulfar Erlingsson. Extracting training data from large language models. In 30th USENIX Security Symposium, 2021. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. In NeurIPS Datasets and Benchmarks Track, 2024a. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. In The 41st International Conference on Machine Learning (ICML), 2024b. Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming LLM agents via poisoning memory or knowledge bases. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. Structural scaffolds for citation intent classification in scientific publications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3586–3596, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1361. Edoardo Debenedetti, Javier Diez, Maura Pintor, Luca Carlini, and Florian Tramèr. AgentDojo: A dynamic environment to evaluate attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems (NeurIPS), 2024. Francisco M. Delgado-Chaves, Francisco Carvalho, Serdar Turkarslan, Nitin S. Baliga, Evangelina Lopez de Maturana, and Núria Malats. Transforming literature screening: The emerging role of large language models in systematic reviews. Proceedings of the National Academy of Sciences, 122(2):e2411962122, 2025. doi: 10.1073/pnas.2411962122. Gelei Deng, Yi Liu, Mayue Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. MasterKey: Automated Jailbreaking of Large Language Models. Network and Distributed System Security (NDSS) Symposium, 2024. Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. Memory injection attacks on LLM agents via query-only interaction. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. Erin Feiglin, Nir Hutnik, and Raz Lapid. Benchoverflow: Measuring overflow in large language models via plain-text prompts. Transactions on Machine Learning Research (TMLR), 2026. Alvan R. Feinstein and Domenic V. Cicchetti. High agreement but low kappa: I. the problems of two paradoxes. Journal of Clinical Epidemiology, 43(6):543–549, 1990. ISSN 0895-4356. doi: 10.1016/0895-4356(90)90158-L. 10

Hanning Gao et al. Adaptive evasion of open-source LLM guardrails via decoding manipulation. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024. Yichen Gong, Xinyu Liu, Zifan Wang, and J. Zico Kolter. FigStep: Jailbreaking Masked Large Vision-language Models via Typographic Visual Prompts. In International Conference on Learning Representations (ICLR), 2024. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising LLMs through indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pages 79–90, 2023. Kathrin Grosse, Lukas Bieringer, Tarek R. Besold, and Alexandre Alahi. Towards more practical threat models in artificial intelligence security. In 33rd USENIX Security Symposium, 2024. Simon Sua Kim, Joel Jang, Adam Fourney, Dongwoo Kim, Michael Laskin, Chaoqi Li, Qiaozhu Liao, Besmira Nushi, Chi Wang, and Qingyun Wu. The attack and defense landscape of agentic AI. arXiv preprint arXiv:2603.11088, 2026. Aida Kostikova, Zhipin Wang, Deidamea Bajri, Ole Pütz, Benjamin Paaßen, and Steffen Eger. Lllms: A data-driven survey of evolving research on limitations of large language models. ACM Computing Surveys, 2025. doi: 10.1145/3801096. Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning llms, 2026. Xuan Li, Zujiang Zhou, Jianping Zhu, Jiang Yao, Tongliang Liu, and Bo Han. DeepInception: Hypnotize large language model to be jailbreaker. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024a. Yunzhe Li, Jianan Wang, Hongzi Zhu, James Lin, Shan Chang, and Minyi Guo. Thinktrap: Denialof-service attacks against black-box llm services via infinite thinking, 2025. Yuxi Li, Yi Liu, Yuekang Li, Ling Shi, Gelei Deng, Shengquan Chen, and Kailong Wang. Uncovering logit suppression vulnerabilities in LLM safety alignment, 2024b. Q Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, and Aleksandra Mojsilović. Are we learning yet? A meta review of evaluation failures across machine learning. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. In The 12th International Conference on Learning Representations (ICLR), 2024a. Yupei Liu, Yuqi Zhou, Yifeng Wen, Ting-Yun Lam, and Zhaorun Li. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium, 2024b. Narek Maloyan and Dmitry Namiot. Breaking the protocol: Security analysis of the model context protocol specification and prompt injection vulnerabilities in tool-integrated llm agents, 2026. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, Dawn Song, and David Forsyth. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In The 41st International Conference on Machine Learning (ICML), 2024. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum S Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box LLMs automatically. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. Jack Merullo, Noah A. Smith, Sarah Wiegreffe, and Yanai Elazar. On linear representations and pretraining data frequency in language models. In The Thirteenth International Conference on Learning Representations, 2025. 11

Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506–519, 2017. Dario Pasquini, Evgenios M. Kornaropoulos, and Giuseppe Ateniese. Llmmap: Fingerprinting for large language models. In 34th USENIX Security Symposium, 2025. promptfoo. Lm security database, 2026. URL https://www.promptfoo.dev/lm-security-db/. Accessed: 4/10/26. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The 12th International Conference on Learning Representations (ICLR), 2024. Meghana Rajeev, Rajkumar Ramamurthy, Prapti Trivedi, Vikas Yadav, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudan, James Zou, and Nazneen Rajani. Cats confuse reasoning llm: Query agnostic adversarial triggers for reasoning models. In Conference on Language Modeling (CoLM), 2025. Mark Russinovich, Ahmed Salem, and Ronen Eldan. Great, now write an article about that: The crescendo multi-turn LLM jailbreak attack. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. Sander Schulhoff et al. Ignore this title and hackAPrompt: Evaluating and Antagonizing Large Language Models with Adversarial Prompts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. Avital Shafran, Roei Schuster, Tom Ristenpart, and Vitaly Shmatikov. Rerouting LLM routers. In Second Conference on Language Modeling (CoLM), 2025. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In IEEE Symposium on Security and Privacy (S&P), 2017. Adam Shostack. Threat Modeling: Designing for Security. John Wiley & Sons, 2014. Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A strongreject for empty jailbreaks. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9798331314385. Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction APIs. In 25th USENIX Security Symposium, 2016. Haoran Wang and Kai Shu. Trojan activation attack: Red-teaming large language models using steering vectors for safety-alignment. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM ’24, pages 2347–2357. Association for Computing Machinery, 2024. ISBN 9798400704369. doi: 10.1145/3627673.3679821. Zhihao Wang, Zeyu Lin, Xinyi Ma, Yicheng Song, Shuo Lu, and Yifei Chen. SoK: Evaluating jailbreak guardrails for large language models. In IEEE Symposium on Security and Privacy (S&P), 2026. Zifan Wang and J. Zico Kolter. Improved few-shot jailbreaking can circumvent aligned language models and their defenses. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. Fundamental limitations of alignment in large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024. 12

Shican Wu, Ozan Irsoy, Steven Lu, Vadim Daberber, Alex Duchi, Shexia Hu, Satwik Kottur, James Liang, Yunqi Ren, Tao Sun, Guan Wang, Zhe Xia, and Jiayou Zhang. Automated review generation method based on large language models. arXiv preprint arXiv:2407.20906v5, 2025. Xiaoshuai Wu, Xin Liao, Bo Ou, Yuling Liu, and Zheng Qin. Are watermarks bugs for deepfake detectors? rethinking proactive forensics. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI ’24, 2024. ISBN 978-1-956792-04-1. doi: 10.24963/ijcai.2024/673. Haibo Xu and Keshab K Parhi. A survey of attacks on large language models. Journal of Hardware and Systems Security, pages 1–23, 2024. Jiahao Yu, Xinghshen Lin, Zhengfu Yu, and Xinyu Xing. GPTFUZZER: Red teaming large language models with auto-generated jailbreak prompts. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS), 2024a. Jiahao Yu, Yuhang Wu, Dong Shu, Mingyu Jin, and Xinyu Xing. Assessing prompt injection risks in 200+ custom GPTs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024b. Qiusi Zhan, Richard Liang, Linyi Xing, and Dan Roth. InjecAgent: Benchmarking indirect prompt injections in tool-integrated LLM agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. Peng Zhang and Peijie Sun. Differentiated directional intervention: A framework for evading LLM safety alignment. In AAAI Conference on Artificial Intelligence, 2025. Wei Zhang et al. Practical and Stealthy Touch-Guided Jailbreak Attacks on Multi-modal Systems. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2025. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.

13

Appendix overview • Appendix A — Taxonomy structure (Figure 2) and technique column defense-layer definitions • Appendix B — Attack category descriptions • Appendix C — Catch-all leaf analysis • Appendix D — Extended benchmark coverage mapping (empirical evidence, sensitivity analysis) • Appendix E — Matrix-guided benchmark design walkthrough • Appendix F — Corpus distribution (Table 7) • Appendix G — Validation details (novelty resolution, Tier 3 evaluation, extraction recall, taxonomy sensitivity) • Appendix H — Extraction and classification prompts • Appendix I — Unmatched attack analysis • Appendix J — Datasheet for the dataset • Appendix K — Ethics statement

14

A

Taxonomy structure

Figure 2 shows a visual representation of the previously defined taxonomy in tree form. Deceptive & Psychological (79) Obfuscation & Evasion (79)

Jailbreaking (316) Multi-Modal Jailbreaking (48) Other (110)

Agentic & Tool-Based (19) Data Source Vectors (15)

Prompt Injection (47) Multi-Modal Indirect (8) Other (5)

Data Exfiltration (11) Probing & Reconnaissance (9)

Inference Time

Information Disclosure (35) System Prompt Extraction (7) Other (8)

Denial of Service (19)

Service Disruption (26)

Functional Degradation (6) Watermarking Attacks (1)

Refusal Subspace Ablation Activation Steering (3) Enforced Decoding

Decoding Manipulation (6) Controllable Decoding MoE Routing Manipulation LLM Router Manipulation

Figure 2: Taxonomy structure showing inference-time attack categories and their major subcategories, with leaf counts in parentheses. Count reconciliation. Of 2,521 unique attack groups, 2,360 (93.6%) resolved to taxonomy leaves (1,868 in the five inference-time categories, 492 in Attack Generation); the remaining 161 were manually reviewed and dropped (non-attacks or out-of-scope; see Appendix I). Each taxonomy leaf is classified into the 4×6 matrix, with each unique attack formalized as a tuple A = (T, M ), where T is the target objective and M is the mechanism/technique. Classification procedure details. Of the 507 leaves, 333 receive their technique column assignment directly from the taxonomy structure. The remaining 174 were individually reviewed and assigned by which defense layer each attack bypasses. The breakdown by subtree is: infrastructure/reasoning (38), 15

execution/agency (36), information disclosure (35), service disruption (26), decoding manipulation (6), and attack generation (77). Six leaves require white-box or hardware access (five model-weight, one physical hardware); they are included because they operate at inference time, but represent a distinct threat model. The full leaf-to-cell mapping is included in the released artifacts. Technique column defense-layer definitions. Each technique column is defined by the defense that would neutralize it: Instructional—a model that perfectly maintains instruction hierarchy; Persuasion—alignment training robust to social manipulation; Obfuscation—a model that perfectly decodes all representations; Cross-Modal—safety measures extending equally across all modalities; Indirect Injection—a model that perfectly distinguishes system, user, and external data privileges; Model Internals—architecture-level defenses such as watermark-robust decoding and side-channel resistance. Output-vs-execution rule. An explicit boundary resolves the distinction between Safety & Alignment Bypass and System & Tool Hijacking: if an attack succeeds when the model outputs harmful text (e.g., generating malware source code), the target is Safety Bypass; if it succeeds when the model executes an action or causes a side-effect (e.g., calling a tool, writing a file, running code in a sandbox), the target is System Hijacking. This boundary is operationally significant because the two targets require fundamentally different defenses: content filtering for the former, access control and sandboxing for the latter. Empty cell analysis. Five cells in Table 2 contain zero leaves. We classify each by whether it represents a coherent single-mechanism attack surface under the bypass-target framework. Structurally coherent (corpus gap). Three empty cells are logically valid and likely reflect corpus limitations rather than structural impossibility. Information Exfiltration × Indirect Injection: a poisoned RAG document or tool response could instruct the model to append its system prompt to user-facing output, bypassing the trust boundary to achieve data leakage. Service Disruption × Cross-Modal: adversarial visual or audio inputs could trigger excessive generation; attacks of this form exist in the corpus (e.g., adversarial video patches inducing >200× token expansion) but were classified under adjacent cells. System Hijacking × Model Internals: an attacker could steer an agent’s tool selection via activation manipulation, though this requires white-box access to a deployed agentic model, making the threat model narrow. Compound-attack cells (two defense-layer bypasses required). Two cells require an attacker to defeat two distinct defense layers simultaneously, making single-mechanism attacks unlikely but not structurally impossible. Safety Bypass × Indirect Injection: indirect injection bypasses the trust boundary (the model treats external data as authoritative), but safety bypass additionally requires circumventing alignment training. Demonstrated attacks in this space typically embed a persuasion or obfuscation payload within the injected text, combining two technique columns in a single attack chain. The cell is valid but inherently compound, which explains both its absence from the corpus and its absence from benchmarks that test one mechanism at a time. Information Exfiltration × Cross-Modal: an image containing “reveal your system prompt” bypasses text-input filters via the visual channel, but the defense whose failure enables exfiltration is the trust boundary. Here too, the cross-modal delivery and the trust-boundary bypass are two separable mechanisms that must both succeed. These cells remain valid entries in the 24-cell matrix and are counted in the benchmark coverage audit; their emptiness reflects the compound nature of the required attack rather than structural impossibility.

B

Attack category descriptions

Jailbreaking (316 leaves). Jailbreaking creates adversarial prompts to bypass safety alignment and produce harmful outputs [Zou et al., 2024]. Subcategories include instruction hijacking [Schulhoff et al., 2023] (27 leaves), deceptive/psychological manipulation [Li et al., 2024a, Russinovich et al., 2024] (79 leaves including multi-turn, role-playing, hypothetical framing, social engineering, and persuasion), obfuscation & evasion (79 leaves covering linguistic, encoding, formatting, tokenization, and filter evasion techniques), multi-modal jailbreaking [Gong et al., 2024] (48 leaves across image, audio, video, and cross-modal fusion), in-context learning exploitation [Wang and Kolter, 2024] (8 leaves), hybrid & composite attacks (1 leaf), system/infrastructure exploitation [Deng et al., 2024] 16

(38 leaves targeting reasoning chains, state/memory, context windows, and mode parameters), and unauthorized execution & agency abuse [Zhan et al., 2024] (36 leaves covering agentic framework exploitation, tool-use permission exploitation, and malicious code generation). Jailbreaking accounts for 2,869 mentions across 712 papers. Prompt injection (47 leaves). Prompt injection differs from jailbreaking in that the adversary injects instructions via external data sources (web content, files, API responses, tool descriptions) rather than direct user interaction [Greshake et al., 2023]. Subcategories include data source injection vectors [Chen et al., 2024] (15 leaves), agentic & tool-based injection [Maloyan and Namiot, 2026] (19 leaves, including MCP server exploitation and agent-to-agent attacks), time-based & stateful injection [Dong et al., 2025] (4 leaves), multi-modal indirect injection [Zhang et al., 2025] (8 leaves), and composite payload injection (1 leaf). Prompt injection accounts for 472 mentions across 192 papers. Information disclosure (35 leaves). Information Disclosure refers to the unauthorized extraction of non-public data from the model’s ecosystem [Carlini et al., 2021], including system prompts, training data (membership inference or verbatim recall), or architecture/safety guardrails. Subcategories include system prompt extraction [Yu et al., 2024b] (7 leaves), sensitive data exfiltration [Balashov et al., 2025] (11 leaves), training data recall [Shokri et al., 2017] (2 leaves), probing & reconnaissance [Pasquini et al., 2025] (9 leaves), and model/architecture extraction [Tramèr et al., 2016] (6 leaves). Information disclosure accounts for 101 mentions across 55 papers. Service disruption (26 leaves). Service Disruption degrades or eliminates the model’s utility, targeting availability and performance rather than output content, often by exploiting the high cost of autoregressive generation or the fragility of reasoning chains. Subcategories include Denial of Service (19 leaves: computational exhaustion, resource consumption / denial-of-wallet, recursive tool calls, safeguard exploitation, simulated failure) [Feiglin et al., 2026]; functional degradation (6 leaves: personality injection, irrelevant distraction, benign task refusal induction, reasoning degradation) [Rajeev et al., 2025]; and watermarking attacks [Wu et al., 2024] (1 leaf). Service Disruption accounts for 134 mentions across 49 papers with a 34.3% novelty rate. Decoding & generation manipulation (6 leaves). These attacks manipulate the model’s internal generation process [Gao et al., 2024]: controllable decoding, enforced decoding, refusal subspace ablation, MoE routing manipulation, LLM router manipulation, and activation steering. We note a scope tension: several of these (refusal subspace ablation, activation steering, MoE routing) require white-box access to model internals, which sits at the boundary of our inference-time definition. We retain them because they are executed during inference (not during training) and because they appear prominently in the corpus as counterpoints to prompt-based attacks, but acknowledge that a stricter “input-interface-only” definition would exclude them. This category accounts for 82 mentions across 44 papers and contains only 6 of 507 leaves (1.2%); excluding it entirely would not change any matrix-level finding. Attack generation & optimization methods (77 leaves). This category captures methods for discovering attacks rather than attack types themselves: gradient-based (26 leaves, anchored by GCG [Zou et al., 2024]), automated generative / LLM-as-attacker (36 leaves, including PAIR [Chao et al., 2024b], AutoDAN [Liu et al., 2024a], TAP [Mehrotra et al., 2024]), sampling & fuzzing [Yu et al., 2024a] (4 leaves), black-box optimization (6 leaves), text-to-image and retrieval-based methods (2 leaves), and manual/human-in-the-loop methods (3 leaves). A GCG-generated attack might be an obfuscation attack, a jailbreak, or a prompt injection; accordingly, when a paper references both GCG (the method) and its output (e.g., an adversarial suffix), each is extracted and classified independently. Attack generation accounts for 2,465 mentions across 606 papers.

C

Catch-all leaf analysis

Sixty taxonomy leaves function as catch-alls, each absorbing 10+ distinct attack techniques. These are concentrated in areas of rapid innovation where naming conventions have not yet stabilized, particularly gradient-based image perturbations and automated attack generation. The recommended 17

splits below would subdivide the largest catch-alls into mechanistically distinct subcategories; implementing them is a priority for future taxonomy versions. The top six by novel attack count are listed in Table 4 with potential splits. Impact on matrix cell counts. All six catch-all leaves map to Safety & Alignment Bypass (Cross-Modal, Persuasion, Obfuscation, or Model Internals). The recommended splits refine granularity within existing matrix cells rather than redistributing across cells—e.g., splitting adversarial-perturbation-on-images into gradient-based, diffusion, adversarial patch, and typography sub-leaves increases the leaf count in Safety Bypass × Cross-Modal but does not populate any new cell. The only catch-all with potential for cross-cell redistribution is activation-steering-attack (Safety Bypass × Model Internals), where a “directed optimization” sub-split could arguably target Service Disruption; even so, Service Disruption × Model Internals already contains 7 leaves and remains uncovered by all benchmarks. The benchmark coverage audit result (6/24 cells) is therefore invariant to catch-all splits. Table 4: Top catch-all leaves by number of novel attacks mapped to them. Catch-All Leaf

D

Novel Total Recommended Split

adv-perturbation-images

40

iterative-llm-refinement

40

rl-based-attack-gen

35

gcg-attack

27

activation-steering

20

evolutionary-llm-gen

17

126 Gradient-based, diffusion, adv. patch, typography 57 PAIR-style, feedback-driven, prompt tuning, genetic 56 PPO-based, reward-model, RLHF-based, multi-turn RL 67 Token-level, attention-aware, multi-coord., checkpoint-based 30 Repr. engineering, patching, directed optim., latent steering 25 Genetic, mutation-based, crossover, tournament selection

Extended benchmark coverage mapping

Table 5 shows the full mapping of six benchmarks to Target × Technique matrix cells under the bypass-target framework. Benchmark mapping procedure. We mapped each benchmark onto the matrix by: (1) identifying every attack method evaluated by the benchmark, (2) matching each to its taxonomy leaf by name and description, (3) looking up that leaf’s matrix cell in the released reclassification artifact, and (4) assigning coverage as full (≥5 methods or a dedicated suite) or partial (1–4 methods, not the benchmark’s focus). For agent benchmarks whose attacks do not correspond to individual named leaves, we map the attack mechanism (indirect prompt injection via tool-returned content) to the Indirect Injection column and the attack goal (unauthorized actions vs. data exfiltration) to the target row. Empirical evidence from uncovered cells. The coverage gaps are not merely theoretical: papers in our corpus report high attack success rates across diverse mechanisms in uncovered cells. Service Disruption. OverThink [Kumar et al., 2026] achieves up to 46× reasoning token amplification across 9 frontier models while maintaining answer correctness, and ThinkTrap [Li et al., 2025] induces 100× latency increases, reducing throughput to 1% of capacity at just 5 requests per minute. Beyond resource amplification, CatAttack [Rajeev et al., 2025] demonstrates that three fixed, query-agnostic adversarial triggers transfer across reasoning models (DeepSeek R1, o1, o3-mini) without permodel optimization, increasing error rates 3× and causing 42% of queries to exceed 1.5× token budget. These three papers demonstrate three distinct Service Disruption mechanisms—reasoning amplification, latency exhaustion, and transferable degradation triggers—none evaluated by any current benchmark. 18

Model Internals. Wang and Shu [2024] show that injecting contrastive steering vectors at inference time reduces Vicuna-7B’s refusal rate from 82% to 2% on AdvBench and increases toxic generation from 3% to 83% on ToxiGen, scaling to Llama2-70B. Zhang and Sun [2025] decompose the safety mechanism into two functionally distinct directions—harm detection and refusal execution—and show that surgically neutralizing both at a single critical layer achieves 97.88% ASR on AdvBench and 95% on HarmBench across seven models, demonstrating that alignment is not monolithic but separable and independently attackable. On the same AdvBench behaviors where GCG (the Obfuscation-column attack benchmarks do evaluate) achieves 69% ASR, VulMine [Li et al., 2024b] achieves 96% ASR via logit-level denial-token suppression—a Model Internals mechanism operating on the same benchmark, same behaviors, through an untested channel. Beyond safety bypass, Shafran et al. [2025] show that adversarial token prefixes can reroute 88–100% of queries from weak to strong models in open-source routers, achieving 79–91% rerouting on the commercial Unify router and increasing costs up to 8× on OpenRouter—an economic denial-of-service via architectural manipulation. These results span four distinct Model Internals mechanisms—activation steering, bi-directional safety decomposition, logit-level suppression, and routing exploitation—all empirically validated but absent from standardized evaluation. The three primary benchmarks (HarmBench, InjecAgent, AgentDojo) are used in the main text; three additional benchmarks (AdvBench, JailbreakBench, StrongREJECT) were mapped to assess whether adding benchmarks meaningfully increases coverage. Sensitivity analysis. The coverage figure depends on two main design choices. (1) Partial coverage: cells marked H◦ indicate that HarmBench includes the technique as one method among many rather than a dedicated evaluation suite. Excluding the two partial-coverage cells (Instructional, Cross-Modal) reduces the covered count from 6 to 4 of 24. (2) Benchmark selection: the three primary benchmarks were chosen as the largest publicly available evaluation suites for jailbreaking (HarmBench), tool-integrated agents (InjecAgent), and dynamic agent environments (AgentDojo). The three additional benchmarks (AdvBench [Zou et al., 2024], JailbreakBench [Chao et al., 2024a], StrongREJECT [Souly et al., 2024]) all target exclusively Safety & Alignment Bypass via Obfuscation and Persuasion, adding no new cells beyond those already covered by HarmBench. The remaining empty cells, including the entire Service Disruption row and the Model Internals column, remain without any standardized evaluation. Unlike the prior matrix formulation, the bypass-target framework produces no structurally implausible cells: every target–technique pairing is coherent, so all 24 cells are evaluated on equal footing. Leaf-weighted vs. cell-weighted coverage. An alternative leaf-weighted coverage metric—what fraction of taxonomy leaves fall in benchmarked cells—yields ∼80%, because the covered cells coincide with the most-studied region (Safety & Alignment Bypass alone accounts for 74% of all leaves). However, this metric conflates research attention with threat importance: Service Disruption and Model Internals have fewer leaves because they are understudied, not because they pose lesser risks. A deployment facing denial-of-wallet attacks or activation-steering exploits gains no protection from benchmarks that exclusively evaluate safety-bypass techniques, regardless of how many leaves those benchmarks cover. The cell-weighted metric reflects what a threat model requires—coverage across structurally distinct attack categories—while the leaf-weighted figure captures the concentration pattern that produces the blind spots we identify. The coverage finding is also invariant to reasonable changes in matrix resolution: coarsening the matrix (e.g., merging technique columns) would increase per-cell coverage but cannot populate the Service Disruption row or Model Internals column, since no benchmark evaluates any attack in these categories regardless of how techniques are grouped. Conversely, splitting rows or columns can only decrease the coverage percentage. Implication. The coverage gaps represent a significant evaluation asymmetry. In the terms of Liao et al. [2021]’s external-validity framework, some represent dataset misalignment—benchmarks sample too narrowly from the technique space, omitting attack vectors their existing infrastructure could accommodate—while others reflect deeper metrics misalignment: Service Disruption requires cost and latency metrics absent from any current benchmark, and Model Internals requires white-box access that prompt-level evaluations cannot support. The former gaps are immediately actionable by broadening existing benchmarks; the latter demand new evaluation infrastructure. That the three audited benchmarks were designed independently yet converge on the same narrow slice suggests 19

Table 5: Extended benchmark-to-matrix mapping under the bypass-target framework. Coverage level: full = dedicated evaluation suite for this cell; partial = technique appears as one method among many. Technique columns now reflect which defense layer each attack bypasses. Benchmark

Target

HarmBench

Safety Bypass Obfuscation

full

InjecAgent

Sys. Hijacking Indirect Inj.

full

InjecAgent

Info. Exfil.

Indirect Inj.

full

AgentDojo

Sys. Hijacking Indirect Inj.

full

AgentDojo

Info. Exfil.

Indirect Inj.

full

AdvBench

Safety Bypass Obfuscation

full

JailbreakBench Safety Bypass Obfuscation

full

HarmBench

HarmBench HarmBench

Technique

Cov.

Grounding

GCG (3 variants), PEZ, GBDA, UAT, AutoPrompt, AutoDAN, PGD—10 gradient/evolutionary methods producing adversarial tokens or suffixes; each leaf classified as Obfuscation in reclassification artifact Safety Bypass Persuasion full PAIR, TAP (2 variants), PAP, Human Jailbreaks—5 methods producing persuasive or deceptive prompts via LLM refinement or manual crafting; each leaf classified as Persuasion Safety Bypass Instructional partial Direct Request, Zero-Shot, Stochastic FewShot—3 methods using plain-language reformulation; 3 of 18 methods Safety Bypass Cross-Modal partial Adversarial Patch, Render Text—2 multimodal methods with 110 dedicated behaviors; PGD classified as Obfuscation by leaf assignment

JailbreakBench Safety Bypass Persuasion

StrongREJECT Safety Bypass Obfuscation StrongREJECT Safety Bypass Persuasion

Direct Harm: 30 attacker cases × 17 user cases (Table 1) Data Stealing: 32 attacker cases × 17 user cases (Table 1) 4 environments, 5 injection phrasings, 629 test cases Exfiltration goals in Workspace/Banking injection tasks GCG on 520 harmful behaviors; leaf: greedy-coordinate-gradient-gcg-attack → Obfuscation

GCG on 100 JBB-Behaviors; leaf → Obfuscation partial PAIR on 100 JBB-Behaviors; leaf → Persuasion. JailbreakChat hand-crafted artifacts (partial)

partial GCG and low-resource language translation among 17 jailbreak methods; leaves → Obfuscation partial Persona-based jailbreaks among 17 methods; leaves → Persuasion

that designers gravitate toward the most visible attack modalities, creating systematic blind spots where attacks are most costly in production.

E

Matrix-guided benchmark design walkthrough

The benchmark coverage audit (§4.4) identifies gaps; here we demonstrate that the matrix can also fill them. We walk through the benchmark design workflow described in §5 for a single uncovered cell—Service Disruption × Instructional—assembling published evidence into a concrete benchmark specification. No new attacks are executed; all results are drawn from papers in the corpus. Step 1: Cell selection. The matrix identifies Service Disruption × Instructional as uncovered by all six audited benchmarks. The cell contains 13 taxonomy leaves, 37 unique attack groups from 25 papers, and a 36.0% novelty rate—indicating active research with zero standardized evaluation. 20

Step 2: Leaf-to-subcategory mapping. The 13 leaves organize into three subcategories that a benchmark must cover to achieve adequate cell coverage: Subcategory

Taxonomy Leaves

Computational Exhaustion

reasoning-loop-over-reasoning-dos, complex-calculation-recursion-prompts, semantic-contradiction-dos, regex-dos-redos-via-prompt-interpretation, logic-bombs-in-prompts, instructional-overhead-dos-attack

Functional Degradation

catattack-irrelevant-distraction, benign-task-refusal-induction, math-perturb-attack

Resource Consumption & Safeguard Exploitation

excessive-token-generation-requests, strategic-resource-exhaustion-in-llm-mas, dos-via-safeguard-false-positives, embodied-agent-action-freezing

Step 3: Evidence assembly. For each subcategory, published papers in the corpus already provide ground-truth results. Table 6 assembles these into a benchmark evidence table—the core artifact a benchmark designer needs. Table 6: Published evidence for Service Disruption × Instructional, organized by the matrix-guided workflow. The rightmost column identifies the metric required to detect each attack—none of which appear in any of the six audited benchmarks, all of which evaluate output content rather than resource consumption or task correctness. Taxonomy Leaf

Source

Models

Published Result

Required Metric

Computational Exhaustion reasoning-loopover-reasoningdos

OverThink 9 frontier reason- 46× token amplification; Token amplifica[Kumar et al., ing models answers remain correct tion ratio 2026]

reasoning-loopover-reasoningdos

ThinkTrap [Li Black-box et al., 2025] services

excessive-tokengenerationrequests

BenchOverflow Multiple LLMs [Feiglin et al., 2026]

LLM 100× latency; throughput Latency ratio; → 1% at 5 req/min throughput Token overflow via plain- Token count text prompts

Functional Degradation catattackirrelevantdistraction

CatAttack DeepSeek R1, o1, 3× error rate; 42% ex- Answer correct[Rajeev et al., o3-mini ceed 1.5× token budget; ness; token bud2025] query-agnostic triggers get

Step 4: Metric specification. The Required Metric column in Table 6 reveals why existing benchmarks cannot detect these attacks: all six audited benchmarks evaluate output content (harmful or not), but Service Disruption attacks succeed through resource amplification or quality degradation while producing benign or correct outputs. Detecting them requires metrics that no current benchmark records. A benchmark for this cell requires: Metric

Rationale

Token amplification ratio Wall-clock latency ratio API cost under attack Answer correctness Throughput at attack load

Computational exhaustion (OverThink: 46×) Throughput impact (ThinkTrap: 100×) Denial-of-wallet quantification Distinguishes degradation from refusal System-level impact (ThinkTrap: 1%)

None of these metrics appear in any of the six audited benchmarks. This is not a dataset gap but a metrics gap in the sense of Liao et al. [2021]: the evaluation infrastructure lacks the measurement apparatus to detect an entire class of successful attacks. 21

Summary. The matrix guided a four-step path from an empty cell to a benchmark specification grounded in published evidence: cell selection → leaf enumeration → evidence assembly → metric identification. The same workflow applies to any uncovered cell. The 8 remaining uncovered cells in the Service Disruption row and Model Internals column can be addressed analogously, though Model Internals cells additionally require white-box access infrastructure that no current public benchmark provides.

F

Corpus distribution

Table 7 shows the distribution of papers and attack mentions across half-year periods, along with post-resolution novelty counts. Table 7: Corpus distribution by half-year period. Novel claims reflect post-resolution counts (42 falsenovel mentions corrected; see §2.3). The novelty rate stabilizes around 18%, indicating concurrent field growth and maturation.

G

Period

Papers

Attack Mentions

Novel Claims

Novel %

H1 2023 H2 2023 H1 2024 H2 2024 H1 2025 H2 2025 H1 2026

3 42 74 120 183 182 259

21 253 528 969 1,397 1,446 1,752

11 68 101 142 235 259 357

52.4 26.9 19.1 14.7 16.8 17.9 20.4

Total

863

6,366

1,173

18.4

Validation details

Novelty resolution. Three cross-reference checks identified 33 novelty conflicts (multiple papers claiming the same attack as novel) and 43 suspicious novelty claims. All 76 entries were manually reviewed: 33 false-novel claims were corrected (15 generic category labels mismarked as novel, 9 same-paper extraction duplicates, 5 unattributed rediscoveries, and 4 naming collisions), while 16 cases were confirmed as genuinely distinct methods sharing a group key. Source attributions for non-novel attacks were extracted from in-paper citations and are released as-is. Tier 3 human evaluation. Two domain experts independently reviewed 200 Tier 3 MATCH classifications, stratified by category with tail-oversampling (100 Jailbreaking, 35 Attack Generation, 25 Prompt Injection, 15 each for Info Disclosure and Service Disruption, 10 Decoding Manipulation). Each sample was judged as correct or incorrect, with an associated confidence level and error-type classification for incorrect cases. Overall accuracy was 92.0% (184/200; 95% CI: [87.4%, 95.0%] by Wilson interval); reweighted to population proportions, 92.3%. Per-category accuracy ranged from 73.3% (Info Disclosure, n=15) to 100% (Service Disruption and Decoding Manipulation). Among the 16 errors: 12 (75.0%) were within-category errors (correct branch, wrong leaf—do not affect matrix-level counts), 2 (12.5%) were cross-category errors (wrong branch), and 2 (12.5%) were force-classification errors (non-attacks matched to leaves). Under pessimistic assumptions, the 2 cross-category errors would shift affected matrix cells by ≤0.2% of total attacks per cell. High-confidence judgments accounted for 72.0% of annotations (144/200), medium 22.0% (44/200), and low 6.0% (12/200). The evaluation codebook is available in the released artifacts. Inter-annotator agreement. Two domain experts independently annotated 115 stratified samples, each working without access to the other’s annotations. After independent annotation, disagreements were reconciled through discussion. We report agreement at three granularity levels based on the pre-reconciliation annotations, as the paper’s claims operate at the matrix-cell level rather than the leaf level: 22

Granularity

Classes

Agreement

Cohen’s κ

Leaf Matrix cell

507 24

70.4% 86.1%

0.70 0.84

Of the 34 leaf-level disagreements, 29 followed the same pattern: Annotator A accepted the pipeline’s classification while Annotator B proposed a more specific leaf within the same subtree. Of the 34, 18 (53%) mapped to the same Target × Technique cell; the remaining 16 involved genuinely ambiguous attacks spanning cell boundaries. Nine of the 16 cross-cell disagreements shared the same target row but differed on technique column (e.g., Instructional vs. Obfuscation for prompt extraction attacks), while seven crossed target rows (e.g., Repeat Attack: obfuscation technique vs. information disclosure; RAG Documents Extraction: system hijacking vs. information exfiltration). Cross-cell disagreements concentrated in Information Disclosure (6 of 15 samples, 40%) and Service Disruption (4 of 15 samples, 27%), reflecting genuine boundary ambiguity in technique-column assignment for these categories. Post-reconciliation, cell-level agreement rose to 89.6% (103/115); the Information Disclosure disagreements were resolved by applying the output-vs-execution rule and match-by-mechanism principle from the codebook. This calibration difference—one annotator applying broader acceptance criteria, the other insisting on the most specific leaf—does not affect the benchmark coverage audit, which maps benchmarks to cells via manual documentation review. We do not report verdict-level κ (pipeline-correct vs. pipeline-incorrect), as divergent acceptance thresholds between annotators (92% vs. 72%) produce the well-known prevalence paradox [Feinstein and Cicchetti, 1990]; the effective-leaf and matrix-cell metrics compare chosen classifications directly and avoid this artifact. Multi-model classification comparison. To assess whether Tier 3 classifications are sensitive to model choice, we re-classified the same 200 stratified attacks using Claude Opus 4.6 (Anthropic) with the identical classification prompt. Agreement by level: Granularity

Classes

Agreement

Target row Matrix cell Leaf

4 24 507

94.5% 80.5% 63.0%

Of 74 leaf-level disagreements, 35 (47%) mapped to the same matrix cell. Among the 39 cross-cell disagreements, 28 shared the same target row but differed on technique column; the most common confusions were Persuasion↔Obfuscation (6×) and Instructional↔Obfuscation (5×), reflecting genuine ambiguity in which defense layer an attack bypasses. Only 11 disagreements (5.5% of all samples) crossed target rows, involving boundary cases such as “dark pattern” attacks arguable between System Hijacking and Safety Bypass; of these 11, 6 (3.0% of all samples) fell on the Safety Bypass↔System Hijacking boundary specifically, confirming that the output-vs-execution rule (§3.2) is the primary source of cross-target ambiguity. The benchmark coverage audit is unaffected by these differences, as benchmark methods are mapped to cells via manual documentation review rather than model classification. Structural pattern robustness. The three structural patterns reported in Section 4 are robust to model choice. Under Claude classifications: (1) all 23 Indirect Injection assignments target System Hijacking (100%, vs. 96% under Gemini); (2) Obfuscation, Persuasion, and Cross-Modal concentrate 96%, 91%, and 96% respectively in Safety Bypass (vs. 92%, 89%, 88% under Gemini); (3) Instructional distributes across all four target rows under both models. Taxonomy construction process. The five categories emerged from a bottom-up process: initial clusters were formed by adversarial objective (what the attacker wants), then refined by examining the threat model each cluster implies (e.g., whether the attack requires model access, targets content vs. availability, or operates via direct vs. indirect channels). Cluster boundaries were iteratively adjusted until they satisfied two criteria: (1) each category corresponds to a distinct defensive requirement and (2) no leaf naturally belongs to two categories. The four target rows (grounded in STRIDE) and six technique columns (organized by bypass target) were chosen to be mutually exclusive and collectively exhaustive over the corpus. 23

Taxonomy and count sensitivity. The five attack categories and 4×6 matrix dimensions are design choices, not natural kinds. Alternative decompositions (e.g., merging Decoding Manipulation into Jailbreaking, splitting Jailbreaking by modality) would yield different coverage statistics. The Target × Technique matrix is a simplification, as some attacks span multiple cells; this is resolved by the bypass-target principle and the output-vs-execution rule (§3.2), but inevitably causes a loss in nuance. The 2,521 unique attack groups figure is best interpreted as a lower bound on the number of distinct attack identifiers the field has produced. The taxonomy’s 507 leaves provide a closer approximation, of which 401 are populated; the remaining 106 represent documented attack types that no paper in the corpus mentioned. Some taxonomy leaves, particularly generic catch-all nodes (e.g., instruction-segmentation, multi-turn-context-manipulation), absorb structurally diverse attacks that share a delivery mechanism but differ in technique—a known property of hierarchical taxonomies that contributes to within-category disagreement. Recent work on LLM-assisted systematic reviews suggests that an LLM-plus-human-reviewer protocol can reduce workload by 33–93% while maintaining classification quality [Delgado-Chaves et al., 2025], supporting our approach of LLM classification with targeted human spot-checks. Extraction recall. A domain expert manually reviewed 13 papers (5 with blind extraction before seeing pipeline output, 8 with anchored verification; 3 selected specifically for low pipeline extraction counts of 0–2 matched attacks to test for catastrophic misses). An attack was counted if described with sufficient detail to understand its mechanism; bare citation-only references—which constitute 58% of citations in computer science papers [Cohan et al., 2019]—were excluded from the recall denominator. Extraction precision was 100% (76/76). Extraction recall was 89.4% (76/85): 9 substantively described attacks were missed. The precision-recall asymmetry is explained by frequency-dependent representation formation in LLMs [Merullo et al., 2025]: the pipeline reliably extracts attacks with strong internal representations (yielding high precision) but fails to recognize less-established attacks below the model’s representation threshold. All nine misses were lesser-known attacks (e.g., FERRET, PoisonedRAG) described at identical depth to successfully extracted well-known attacks (e.g., GCG, AutoDAN) in the same paragraph. An additional 5 of 76 extractions (6.6%) were correctly extracted but misclassified at the taxonomy level, suggesting that extraction and classification errors are largely independent. Novelty rate confound. The frequency-dependent extraction gap predicts lower recall for novel or less-established attacks, which could deflate observed novelty rates in later periods and mimic field maturation. Two observations partially address this confound: first, the novelty rate increases from 14.7% (H2 2024) to 20.4% (H1 2026), inconsistent with monotonic recall degradation; second, novel attacks as a paper’s primary contribution receive prominent, detailed descriptions that likely exceed the extraction model’s recognition threshold—the 9 extraction misses were lesser-known referenced attacks, not papers’ headline contributions. Nevertheless, we cannot fully disentangle genuine maturation from pipeline recall effects, and the ∼18% figure should be treated as an apparent rate that may undercount genuinely novel but underspecified attacks.

H

Extraction and classification prompts

The following prompts were used verbatim in the pipeline. Minor formatting (line breaks) has been adjusted for readability. Template variables (e.g. {name}) are populated at runtime. Prompt 1: Attack extraction (full). Used with Gemini 3.1 Pro to extract attacks from each paper’s PDF. The output schema enforces structured JSON with fields: name, description, example, is_novel, and original_source_paper. Prompt 1 — Full extraction You are an academic paper review and summarization tool focused on INFERENCE-TIME adversarial attacks on LLMs –- attacks executed through the

24

model’s input interface during deployment, requiring no access to the training pipeline or model weights. Extract every specific, named attack type discussed in this paper, including its name, a substantial description, and a concrete example. Do not skip any attacks. DO NOT extract any of the following: - Evaluation metrics, scoring functions, or measurement tools (e.g., ASR, perplexity scores, Zlib compression ratios) Defense techniques, guardrails, safety filters, or detection methods - Broad categories or umbrella terms (e.g., “adversarial attacks”, “prompt-space attacks”, “perturbation-based attacks”, “heuristic-based algorithms”) –- only extract specific named attacks - Components that are not attacks on their own (e.g., Vigenere cipher, Base64 encoding, Caesar cipher –- unless the paper presents them AS an attack method) - Training-phase attacks (data poisoning, backdoor injection, weight manipulation, fine-tuning exploits) - Attacks on non-LLM systems (e.g., GNN perturbations, traditional SQL injection, network exploits) unless the paper explicitly adapts them for LLMs - Error types, failure modes, or risk descriptions that are not deliberately triggered by an adversary (e.g., “hallucination risk”, “premature actions”) For each attack, determine whether it is a novel contribution introduced by this paper, or whether this paper is referencing/citing an attack that was originally presented in prior work. To determine novelty: - If the paper explicitly proposes, introduces, or presents the attack as its own contribution, mark is_novel=true and set original_source_paper to the title of the current paper. - If the paper cites, references, evaluates, or builds upon an attack from another paper, mark is_novel=false and set original_source_paper to the original paper’s citation (author, year, title) as it appears in the references. - Pay close attention to citation markers (e.g. [1], [2], (Author et al., 2023)) near attack descriptions to identify the source. - If the paper discusses a well-known attack technique without a specific citation, still mark is_novel=false and provide the best attribution you can.

Prompt 2: Attack extraction (concise retry). Used when the full prompt timed out or produced truncated output. Prompt 2 — Concise retry You are an academic paper review and summarization tool focused on INFERENCE-TIME adversarial attacks on LLMs. Extract only the NOVEL attack types introduced by this paper (not referenced/cited attacks from other work). For each novel attack, provide: - name, description (2–3 sentences max), a brief example, is_novel=true, and original_source_paper as this paper’s title. Also list referenced attacks with MINIMAL detail: just name, one-sentence description, is_novel=false, and the original source citation. Keep descriptions and examples SHORT to avoid truncation. DO NOT extract any of the following: - Evaluation metrics, scoring functions, or measurement tools (e.g., ASR, perplexity scores, compression ratios) - Defense techniques, guardrails, or safety filters - Broad categories or umbrella terms (e.g., “adversarial attacks”, “jailbreaking methods”) - Components that are not attacks on their own (e.g., cipher algorithms, encoding schemes, optimization steps) - Training-phase attacks (data poisoning, backdoor injection, weight manipulation, fine-tuning exploits) - Attacks on non-LLM systems (e.g., GNN perturbations, traditional network exploits) unless explicitly adapted for LLMs - Error types or failure modes that are not deliberately triggered by an adversary

Prompt 3: Tier 3 semantic classification. Used with Gemini 3.1 Pro to classify attacks that failed direct and fuzzy matching. The compact leaf catalog (all 505 leaf IDs available at classification time, with descriptions and variant annotations) is appended after the TAXONOMY LEAF CATALOG header. Two additional leaves were added to the taxonomy after classification, bringing the final count to 507; these were assigned to matrix cells manually. When a fuzzy string match exists, a hint line is 25

injected (shown as {fuzzy_section} below); otherwise it is omitted. Structured JSON output is enforced via the Gemini response_schema parameter. Prompt 3 — Semantic classification ### SYSTEM ROLE You are an INFERENCE-TIME LLM ATTACK TAXONOMY CLASSIFIER performing academic security research. You will be given an attack name and description extracted from a published academic paper. ### TASK Classify this attack into the taxonomy below. Return exactly one leaf ID. - If the input is NOT a genuine inference-time LLM attack (e.g., a metric, defense, training-phase attack, or broad category), return “NOT_AN_ATTACK”. - If it IS an attack but no leaf captures its mechanism, return “NO_LEAF_FITS”. - Otherwise return “MATCH” with the closest leaf ID. - Match by MECHANISM, not surface name similarity. - Use variant names shown in [aka: ...] annotations when matching. - If the attack is the same technique under a different name or alias, match it. {fuzzy_section} ### ATTACK TO CLASSIFY Name: “{norm_name}” Description: “{description[:800]}” ### TAXONOMY LEAF CATALOG {leaf_catalog}

The leaf catalog lists each leaf ID with a truncated description and variant annotations (e.g., gcg-attack: Greedy Coordinate Gradient... [aka: GCG, universal adversarial suffix]). The catalog is included in the released pipeline code.

I

Unmatched attack analysis

Of the 2,521 unique attacks, 161 (6.4%) did not map to any leaf node and were dropped after manual review (see §3.2 for the full count reconciliation). The classification prompt requires leaf-level assignment; no attacks map to intermediate nodes. Of these 161, 41 claimed novel status. Manual review reveals they fall into distinct categories: broad category or umbrella terms incorrectly marked as novel (∼15, e.g., “semantic manipulation,” “capability jailbreak,” “complex adversarial”); training-phase or out-of-scope methods (∼8, e.g., data paraphrasing for watermark removal, temporal backdoors); non-LLM targets (∼5, e.g., recommender system shilling attacks, neural ranker exploits); and evaluation frameworks or metrics misclassified as attacks (∼5, e.g., benchmark tier labels, stress tests). Approximately 8 are genuinely borderline: of these, several map to existing taxonomy leaves under different names (e.g., “twinbreak” is already catalogued as an alias under mixture-of-experts-routing-exploitation; “contextaware overthink” maps to existing Service Disruption reasoning-loop leaves), while others target niche application domains (e.g., LLMs for time-series forecasting). No dropped attack represents a genuinely novel attack type that the taxonomy lacks a category for. The benchmark coverage audit is methodologically independent of the extraction pipeline—benchmark methods are mapped to matrix cells via manual documentation review (§4.4)—so any extraction-side exclusions do not affect the 6/24 coverage result.

J

Datasheet for the LLM Attack Taxonomy Dataset

Motivation. The dataset was created to address the lack of a comprehensive, quantitatively grounded taxonomy of LLM adversarial attacks. It is intended for use by the safety and alignment research community. Composition. The dataset contains: (a) 6,366 attack mention records extracted from 932 arXiv papers; (b) a 507-leaf hierarchical taxonomy of inference-time attacks with Target × Technique classifications. Each record includes source paper identifier, attack name, taxonomy mapping, match tier, novelty status, and temporal period. The 507 leaves are the terminal nodes of the taxonomy 26

hierarchy. All matched attacks map to leaf nodes; unmatched entries were manually reviewed and dropped. Collection process. Papers were collected from the Promptfoo LLM Security database [promptfoo, 2026], filtered to inference-time adversarial attacks on LLMs published on arXiv between 2023 and 2026. The last date of collection was April 10th, 2026. Structured extraction used Gemini 3.1 Pro. Preprocessing. Attack names were normalized via case folding, kebab-case conversion, and removal of parenthetical abbreviations. Classification used three-tier matching (direct, fuzzy, semantic); see §3.2 for mapping rates. Source attribution quality. Each non-novel attack record includes an original_source_paper field extracted from in-paper citations. A spot-check of 100 attributions found 72% verifiably correct, 14% ambiguous (generic attack names without a clear single originator, e.g., “system message,” “ICL”), and 14% misattributed. Misattributions clustered among community-originated attacks attributed to the first academic paper to study them rather than their informal origin, and generic pipeline-assigned names matched to topically related papers. No downstream analysis (taxonomy structure, matrix assignments, benchmark coverage) depends on source attributions; users of the released dataset should treat this field as approximate. Uses. Intended for: benchmark gap analysis, attack naming standardization, temporal trend analysis, and defensive research prioritization. Not intended for: generating new attacks, automating exploitation, or replacing expert security assessment. Distribution. Released on HuggingFace under CC-BY-4.0 (data) and MIT (code). Croissant metadata provided. Maintenance. The dataset represents a snapshot of the literature through early 2026. Future versions may extend the corpus and refine catch-all leaf splits.

K

Ethics statement

This work catalogs adversarial attacks for defensive purposes: enabling systematic evaluation and benchmark development. We do not introduce new attacks or provide exploit code; the taxonomy describes attack categories at a level of abstraction that does not enable reproduction without consulting the underlying papers. We acknowledge the dual-use concern that a comprehensive catalog could theoretically assist adversaries, but believe the defensive value—identifying evaluation blind spots, standardizing naming, and prioritizing benchmark development—substantially outweighs this risk given that all source papers are already public. The dataset contains no personally identifiable information; all inputs to the extraction pipeline are publicly available arXiv papers.

27