How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry

arXiv:2608.21977v1 [cs.CR] 22 Aug 2026

How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry Yu Nong

Yao Du

Majid Behravan

Haipeng Cai

University at Buffalo yunong@buffalo.edu

Macau University of Science and Technology yaodu19@gmail.com

Virginia Tech behravan@vt.edu

University at Buffalo haipengc@buffalo.edu

them as ground truth when building vulnerability datasets and evaluating weakness-specific techniques. Due to the widespread use of these labels, noise in them can propagate broadly into downstream tools, datasets, and decisions. Yet CWE labeling is non-trivial, but a semantic task, MITRE’s mapping guidance instructs labelers to map a CVE to the most specific applicable root-cause weakness and to avoid labels that merely describe technical impact [2]. CVE records, however, are often written in incident language: they describe what an attacker can achieve, which component is affected, or what symptom was observed. However, the code and patch may reveal a different mechanism. For example, a record may describe a heap buffer overflow while sanitizer output and the patch show an out-of-bounds read. Correct labeling therefore requires recovering the weakness mechanism from the advisory, patch, and code, then mapping that mechanism into a taxonomy whose entries are hierarchical and sometimes overlapping. Despite the importance of this task, the reliability of NVD CWE labels has not been systematically measured at scale. Prior work has exposed quality issues in NVDderived vulnerability data [3] and identified administratively invalid CVE–CWE mappings, such as assignments to discouraged or prohibited CWE entries [4]. These studies are valuable, but they do not answer the question most relevant to downstream users: given the available evidence for a CVE, is NVD’s assigned CWE semantically consistent with the weakness the evidence supports? Existing automated CWE classifiers are also not sufficient for this measurement. Most predict a CWE from CVE descriptions [5] or their diffs [6], or the structure of CWE catalog itself [7]; they optimize agreement with existing labels rather than audit those labels. Training on NVD-labeled data to evaluate NVD label quality creates a circularity, and a classifier that only predicts a label cannot tell whether disagreement with NVD reflects a true error, a defensible alternative, or ambiguity in the CWE taxonomy. In this paper, we aim to fill the gap via a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. To enable the study, we need an automated auditing instrument that classifies vulnerabilities by reasoning over the evidence available for a CVE (e.g., its description, advisory, patch, and relevant code) rather than relying on the description or other metadata alone. The

Abstract—CWE labels in the National Vulnerability Database (NVD) are widely treated as ground truth for vulnerability search, scanner evaluation, benchmark construction, learningbased security tools, and vulnerability prioritization. Yet their reliability has not been systematically measured at scale, despite growing concerns about NVD’s enrichment backlog and anecdotal reports of inaccurate, ambiguous, or missing labels. This paper presents a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. We build CWEAgent, a validated auditing instrument based on seclometry, a structured representation of vulnerability semantics that captures the root cause, trigger condition, violated security property, exploit mechanism, and impact of vulnerable code. On a manually curated benchmark of 100 open-source CVEs, CWEAgent achieves 85% top-1 accuracy and 92% ambiguityaware accuracy. Applying CWEAgent to 15,556 open-source CVEs disclosed from 2017–2026, we find that only 49.70% of NVD CWE labels exactly match the code-grounded label. Another 31.37% are defensible alternatives under taxonomy ambiguity, while 3.63% are evidence-inconsistent likely errors. Label reliability varies sharply by assigning organization and weakness type, and apparent project- or language-level differences are largely composition effects of those underlying weakness types. Evidence-inconsistent labels have also increased over time. Through manual review of 434 confirmed mislabels, we identify six recurring error patterns, showing that CWE noise is a structural problem in vulnerability metadata rather than isolated annotation mistakes.

1. Introduction Vulnerability databases are a key security infrastructure, widely used by risk scanners, asset-management systems, threat-intelligence platforms, benchmark builders, and learning-based security tools. Among them, the National Vulnerability Database (NVD) [1] is the dominant public source of enriched CVE metadata. In particular, a Common Weakness Enumeration (CWE) label in NVD identifies the type of weakness underlying a CVE, determining how vulnerabilities are grouped, searched, counted, benchmarked, and learned from. Developers use CWE labels to understand root causes and reason about remediations; practitioners use them to organize detection and triage; researchers use

1

purpose of this instrument is not to replace NVD analysts or declare every disagreement an error. Instead, it offers a scalable way to compare an NVD-assigned CWE label against the weakness mechanism supported by code-level evidence. Achieving these, however, faces several key challenges. First (Challenge 1), the label source cannot also be the training oracle. A natural approach to the instrument is to train a supervised CWE classifier on CVEs labeled by NVD. But this creates a circularity: the same labels whose quality we seek to measure would define the model’s notion of correctness. It also risks learning NVD’s historical labeling habits, including its coarse labels, description-level shortcuts, and systematic confusions. We overcome this by not training on NVD-labeled CVEs. Instead, we construct our CWE-side knowledge directly from MITRE’s CWE catalog and matches each CVE to that catalog through evidencegrounded vulnerability semantics reasoning at code level. Second (Challenge 2), the relevant semantics are rarely contained in the CVE description alone. CVE descriptions often describe consequences, exploit effects, or affected components, while the correct CWE depends on the rootcause mechanism. The decisive evidence may appear only in the advisory, fixing patch, sanitizer trace, or vulnerable code. Thus, a description-only classifier may reproduce the same surface cues that led to the original label. We address this with seclometry, a novel, structured security-semantic signature that represents both CVEs and CWE entries using the same fields: root cause, trigger condition, violated invariant, source/sink roles, impact, code pattern, and boundaries against neighboring weaknesses. This converts CWE labeling from text matching into code-semantic alignment. Third (Challenge 3), disagreement is not equivalent to error. The CWE taxonomy is hierarchical and partially overlapping: a vulnerability may admit a specific root-cause CWE, a broader parent, or a neighboring CWE that is defensible from another perspective. Thus, our instrument must distinguish imprecision from mislabeling. We tackle this by separating prediction from adjudication, using a Classifier to predict the CWE most consistent with the CVE evidence, and an Arbitrator to then judge the NVD label as exact, overlap-ambiguous, defensibly alternative, or evidence-inconsistent. This separation lets our measurement quantify both the imprecise-but-defensible labels and the labels that contradict the evidence. Following these insights, we developed C WE AGENT, and validate it before using it for measurement. On a benchmark of 100 CVEs with manually curated ground-truth CWE labels, C WE AGENT achieves 85% top-1 exact-match accuracy and 92% ambiguity-aware accuracy, outperforming same-evidence single-prompt LLM baselines and the state-of-the-art non-LLM-based classifier [7]. We separately evaluate the Arbitrator on 100 adjudication cases containing confirmed-correct and confirmed-mislabeled labels, where it achieves 90% overall accuracy. These evaluations do not make C WE AGENT an oracle, but they establish it as a validated measurement instrument; for the final error analysis, we further manually review the likely-mislabel cases. We then apply this approach to 15,556 open-source

CVEs disclosed between 2017 and 2026 with publicly accessible fix commits. Among others, we revealed that NVD CWE labels are not simply right or wrong, but fall into exact, defensibly imprecise, and evidenceinconsistent cases. Only 49.70% of the CVEs have labels exactly match the code-grounded CWE identified by C WE AGENT. Yet most disagreements are not outright errors: 25.01% reflect overlap ambiguity and 6.36% are defensibly alternative labels. The true error tail is smaller but consequential: 3.63% are evidence-inconsistent cases. • CWE label quality has degraded in the error tail over time despite stable exact-match rates. From 2017 to 2026, strict exact-match rates fluctuate without a clear monotonic trend, but evidence-inconsistent labels rise from roughly 1–3% in 2017–2018 to 3–6% in 2021–2026. • Weakness semantics, not project or language, primarily explains observed label quality. Exact-match precision varies sharply by CWE family and specific CWE, while evidence inconsistency is flatter at the family level and concentrated in particular CWEs. Apparent project- and language-level differences are largely composition effects: a project or language group may appear better or worse labeled mainly because it contains a different mixture of weakness types, with injection-heavy groups appearing more precise and memory-safety-heavy groups less. • Assigning CNA is the strongest metadata-level signal of CWE label quality. Label precision varies more across assigning organizations than across years, projects, languages, severity levels, or exploitability indicators. As each CVE records its CNA, downstream users can use the CNA as a practical signal of CWE-label reliability instead of assuming all NVD CWE labels are equally trustworthy. • Vulnerability importance does not imply label reliability. Critical-severity CVEs are labeled no more accurately than medium-severity ones. Exploitation status shows the same pattern: CVEs with public Exploit-DB proof-ofconcept exploits have an evidence-inconsistency rate statistically indistinguishable from the corpus-wide rate. • The evidence-inconsistent tail contains real NVD mislabels with recurring semantic failure modes. Of the 564 CVEs flagged as evidence-inconsistent, manual review confirms 434 genuine NVD mislabels. Open coding identifies six recurring patterns: consequence-vs.-root-cause confusion, sibling sub-type confusion, injection sink/interpreter confusion, access-control and identity conflation, discouraged or wrong-branch labels, and multi-CWE accumulation. These patterns arise where the taxonomy offers a coarser, easier, or consequence-oriented label than the mechanism shown by the patch or code. • Confirmed mislabels are patterned but diffuse, making simple correction rules unlikely to suffice. The 434 confirmed mislabels contain 384 distinct directed corrections from the NVD-assigned CWE to the evidence-supported CWE, and 94.8% of these correction pairs occur only once. Moreover, 80.5% cross CWE-699 weakness families, meaning true mislabels often name a fundamentally different weakness rather than merely a sibling CWE. •

2

mechanism that makes the consequence possible: missing authentication, improper bounds checking, unsafe deserialization, nonce reuse, command injection, or another rootcause weakness. Thus, a label can be plausible from the description yet unsupported by the advisory, patch, or code.

Many problematic labels are not evidence-inconsistent, but still impose downstream cost. Over 16% of CWE assignments use entries that MITRE discourages or prohibits for direct mapping [2]. These labels are rarely evidenceinconsistent, but they are maximally imprecise: they force downstream users to recover the actual weakness type from the advisory, patch, or code instead of the CWE field. We also observe ecosystem-level failure modes: 1) CNA and NVD labels may disagree on the same CVE; 2) Later NVD updates can introduce wrong CWE labels even when a correct CWE is already present; and 3) The original advisory/report may miss the (e.g., rootcause) information needed for correct CWE labeling. Beyond the measurement itself, we examine a secondary use of C WE AGENT: assisting CWE-label enrichment under backlog pressure. This analysis is deliberately scoped to the CWE-labeling subtask, not the full NVD enrichment workflow. Using measured per-CVE runtime and monthly CVE publication volume, we simulate how automated candidate labeling could increase CWE-labeling throughput. The result suggests that C WE AGENT can support human analysts by pre-labeling records, prioritizing review, and flagging evidence-inconsistent assignments, while leaving final enrichment decisions to human review. In summary, main contributions of this paper include: • We formulate semantic CWE-label auditing as a measurement problem: determining if an NVD-assigned CWE is exact, defensibly imprecise, or evidence-inconsistent. • We introduce C WE AGENT , a code-semantics-grounded auditing instrument, and seclometry, a structured representation that aligns CVE evidence with MITRE CWE definitions without training on NVD labels. • We validate C WE AGENT ’s Classifier and Arbitrator on controlled benchmarks and then use it to measure CWElabel quality across 15,556 open-source CVEs disclosed between 2017 and 2026. • We characterize the structure and causes of CWE label noise, including 434 manually confirmed NVD mislabels, and derive implications for NVD maintenance, CNA labeling, benchmark construction, and research that treats NVD-derived CWE labels as ground truth. •

2.2. Motivating Example To illustrate the distinction between consequence and root cause, consider CVE-2026-26014 [8] (Figure 2) as an example. The vulnerability involves AES-GCM encryption with randomly generated nonces. AES-GCM requires nonce uniqueness under a fixed key; a repeated nonce–key pair can enable recovery of the GHASH authentication key and message forgery. The root-cause weakness is therefore nonce reuse, corresponding to CWE-323 (Reusing a Nonce, Key Pair in Encryption). The exposure of key material is a consequence of that misuse, not the weakness itself. The NVD record, however, is labeled CWE-200 (Exposure of Sensitive Information). This label is understandable from a description that emphasizes exposed key material, but it misses the cryptographic mechanism shown by the evidence. A tool, training benchmark, or prioritization policy that selects vulnerabilities by CWE-323—to study cryptographic nonce misuse, build a detector for it, or count its prevalence—would miss this CVE. Conversely, analyses of CWE-200 would incorrectly include a cryptographic noncereuse flaw as a generic information-exposure case. This example captures the failure mode we study at scale: surface descriptions can point toward broad consequence labels, while advisory, patch, and code evidence support more specific root-cause CWEs.

2.3. Limitations of Existing Approaches Prior work exposes quality issues in vulnerability metadata [5], [4], including those in the NVD, but does not measure the semantic correctness of assigned CWE labels. Meanwhile, existing automated CWE classifiers [9], [10], [11] advance the state of the art on the classification problem but are not sufficient as measurement instruments because they fail one or more requirements of semantic CWE-label measurement. First, description-based classifiers may help infer missing labels, but they are not suited for auditing assigned labels whose errors may originate from the same description-level cues—the description is authored often by the same CNA; its vocabulary is entangled with, and predictive of, the existing label. Second, classifiers trained on NVD labels inherit the oracle problem: the labels under audit also define the training target—NVD is the dominant vulnerability data source. Third, some classifiers are trained and evaluated only on frequent CWE classes, which fall short for auditing NVD—NVD can assign longtail CWE labels. Fourth, black-box predictions cannot distinguish defensible imprecision from evidence-inconsistent mislabeling. A measurement audit cannot rely on a label alone. When an instrument disagrees with NVD, it must explain which evidence supports the alternative CWE and why the NVD label is exact, defensible, ambiguous, or

2. Background and Motivation 2.1. CWE-Label Correctness A CWE label is intended to identify the weakness that causes a vulnerability, not merely the symptom described in a CVE record. MITRE’s guidance treats CWE assignment as root-cause mapping: labelers should choose the most specific applicable CWE, preferring Base or Variant entries over broader Class-level entries when possible [2]. It also marks CWE entries as A LLOWED, D ISCOURAGED, or P ROHIBITED for direct mapping. This makes CWE-label correctness a semantic property. A CVE may say that an attacker can disclose data, gain privileges, bypass authentication, or execute code, but those are often consequences. The CWE should capture the

3

• Tool 1 : The Classifier

D Tool 2:

Phase 1: Offli1ne CWE Knowledge Construction r / 1.1

• Iii'

.-

MITRE CWE Corpus {ALLOWED CWEs)

' .

8 semantic fields + normalized schema

a �

-

�

CVE Report advisory

. -

,�

Advisory §) description extraction __,

</> Code & Patch

understanding

Identified CWE

concept-to-fine multi-view narrowing

�

Misclassification

& Analysis '•

•

CWE seclometry & evidence

2.1: Evidence-Guided Sedometry Construction

vulnerable code patch

El

Index of Multi-View Clusters

♦-----------------------------: Builder i Agent 1: -Seclometry i Agent -2: CWE Matcher r-

CWEAgent Input �

--

Clustering

Request for missing evidence

Phase 2: Code-Semantic CWE Label Prediction /

Multi-View Semantics cluster by discriminative seclome try fields

\.

CWEAgent Output

/

1.2

-

Seclometry Generation

The Arbitrator

.ilfl Static

- analys '---

.-

V

:J J

�

2.2: Multi-View CWE Matching

'--

Multi-view cluster filtering

$ ..I

Per candidate scoring

l

Predicted CWE & supporting evidence

� CWE Label Adjudication

-

°'

Evidence-based misclassification analysis

)

Figure 1: An overview of C WE AGENT’s design, including its inputs, main working phases/steps, and outputs. and evidence fields (§3.2). Second, the audit must avoid learning from the labels under measurement. C WE AGENT constructs its CWE-side knowledge offline from MITRE CWE definitions, not from NVD-labeled CVEs (§3.3.1). Third, the audit must reason over evidence rather than accept weak or incomplete inputs. C WE AGENT separates evidence construction from CWE matching, allowing the matcher to request additional evidence when the current seclometry is insufficient (§3.3.2). Fourth, the audit must distinguish defensible disagreement from true mislabeling. C WE AGENT therefore separates CWE prediction from label adjudication, using an independent Arbitrator to judge the NVD label rather than treating every mismatch as an error (§3.4). The Classifier works in two phases. First (Phase 1), it converts the A LLOWED CWE mapping space into CWE seclometries and organizes them into an Index of Multi-View Clusters. This index provides reusable catalog-side structure for coarse-to-fine candidate narrowing. Next (Phase 2), it runs a two-agent workflow for the given CVE. The Seclometry Builder gathers advisory, patch, code, and static-analysis evidence and constructs the CVE seclometry. The CWE Matcher uses the multi-view index to filter candidate CWEs, scores the surviving candidates against the CVE seclometry, and requests missing evidence when needed. Given the Classifier’s resulting predicted CWE & supporting evidence and the existing NVD label, the Arbitrator adjudicates the NVD label under the taxonomy used in our measurement. Exact matches are counted as agreement; overlap-ambiguous and defensibly alternative labels are separated from true errors; and evidence-inconsistent labels are flagged for measurement and subsequent manual validation. This design lets C WE AGENT produce not only a CWE prediction, but an auditable judgment about the reliability of the NVD label itself, as the output of C WE AGENT.

Case 1 (Consequence vs. root cause): CVE-2026-26014 NVD label: CWE-200 (Exposure of Sensitive Information) NVD description. “. . . uses random nonce generation with AES GCM ciphers, which makes it easier for remote attackers to obtain the authentication key and spoof data by the reuse of a nonce in a session and a ‘forbidden attack’.” Inconsistency. AES-GCM requires a unique nonce per key; random nonces admit collisions, and a repeated (nonce, key) pair lets an attacker recover the GHASH key and forge messages. The root cause is nonce reuse, i.e. CWE-323 (Reusing a Nonce, Key Pair in Encryption), not the key exposure it enables. NVD’s CWE-200 names that downstream consequence; root-cause mapping requires the cryptographic misuse.

Figure 2: Consequence-vs-root-cause mislabel: NVD labels the exposure (CWE-200), not the root cause (CWE-323). wrong. Prior classifiers generally return predictions without program-evidence-backed rationales, making their disagreements hard to inspect and unsuitable as the basis for a largescale label-quality measurement. These limitations motivate our C WE AGENT approach: a code-semantics-grounded CWE label auditing instrument that constructs CWE knowledge independently of NVD labels, reasons over advisory, patch, and code evidence, covers the entire mappable (A LLOWED) CWE space, and separates (CWE label) prediction from adjudication.

3. Design of C WE AGENT 3.1. Overview Figure 1 gives an overview of C WE AGENT, which consists of two tools. The Classifier predicts the CWE most supported by a CVE’s report, advisory, patch, and (vulnerable) code evidence as the inputs. The Arbitrator then performs CWE label adjudication: it compares the existing NVD label against the evidence and the Classifier’s prediction, and determines whether the label is exact, overlapambiguous, defensibly alternative, or evidence-inconsistent. This separation is central to our measurement: prediction identifies the evidence-supported weakness, while adjudication determines what a disagreement with NVD means. C WE AGENT’s design is justified by meeting the requirements of a semantic instrument (§2). First, CWE labels must be judged against root-cause semantics rather than surface descriptions. C WE AGENT therefore uses seclometry, a structured representation that expresses both CWE definitions and CVE evidence using the same root-cause

3.2. Seclometry The central representation in C WE AGENT is seclometry: a structured security-semantic signature of a CWE definition or a CVE instance. The term is coined by analogy to stylometry, which represents documents by measurable stylistic features rather than raw text. Seclometry applies the same idea to vulnerability semantics: it represents a weakness by the security properties that distinguish it from neighboring

4

Table 1: Definition of seclometry: composition & semantics. Field Description Root Cause Trigger Conditions Key Invariant Violated Observable Effects / Impact Typical Code Patterns Common Sources / Sinks Non-Examples / Boundaries

memory access, but differ on whether the invalid access writes or reads memory. The boundary field records what a weakness is not, allowing C WE AGENT to penalize candidates that share surface mechanism or impact but contradict the defining boundary of the CWE. This is essential because many NVD disagreements arise at these boundaries. Figure 3 shows a complete seclometry for CWE-787 (Out-of-bounds Write) in both components. The left panel is the eight-field stylometry component—the natural-language signature that the LLM-based matching stages read—while the right panel is the normalized schema of categorical fields that multi-view clustering and per-candidate scoring consume as discrete values. The same two-component structure applies to every one of the 842 ALLOWED CWEs and to each CVE seclometry that Agent 1 emits.

What it captures A concise natural-language statement of what the weakness or vulnerability is. The underlying programming or design failure that allows the weakness to exist. The runtime conditions, including the form of untrusted input, that must hold for the weakness. The security property the weakness breaks (e.g., spatial memory safety, authorization integrity). The externally visible consequences (e.g., DoS, memory corruption, RCE, information disclosure). Syntactic/idiomatic signatures in source code that commonly give rise to the weakness. The untrusted data sources and the sensitive sinks characteristic of the weakness. Explicit distinctions from neighboring weaknesses that are commonly confused with this one.

weaknesses, such as root cause, trigger condition, violated invariant, source/sink role, exploit mechanism, and impact. Seclometry is the common semantic representation that makes CWE-label auditing possible. CWE definitions and CVE evidence are written in different genres, and directly matching these texts encourages surface-level shortcuts on consequence words such as “information exposure” or “code execution.” A seclometry instead maps both sides into the same security-semantic representation, letting C WE AGENT ask whether the mechanism evidenced by the advisory, patch, and code matches the mechanism a CWE defines. Design goals. A seclometry must satisfy three goals. First, it must be discriminative: it should capture the features that separate nearby CWEs, not merely broad families. For example, “memory safety” is insufficient to distinguish CWE-787 from CWE-125, CWE-416, or CWE-119; the representation must encode whether the flaw is an out-ofbounds write, out-of-bounds read, use-after-free, or broader bounds-management error. Second, it must be uniform: the same fields must represent both CWE definitions and CVE evidence, enabling direct comparison rather than crossgenre text matching. Third, it must be machine- and modelconsumable: LLM agents can reason over it, while clustering and matching procedures can compare it systematically. Structure. Each seclometry contains two components. The first is a natural-language security signature organized into eight fields (Table 1). These fields describe the weakness mechanism, triggering conditions, violated security invariant, observable effects, typical code patterns, source/sink roles, and boundaries against neighboring weaknesses. The second is a normalized schema of controlled categorical fields, such as mechanism family, source type, sink role, and impact class. The signature supports LLM reasoning and evidence-grounded explanation; the schema supports multi-view clustering, candidate filtering, and systematic scoring. These fields are derived from the axes along which MITRE’s CWE entries are structured (description, common consequences, modes of introduction, observed examples, relationships), and informed by the mechanism/operand/consequence decomposition in NIST’s Bugs Framework [12]. We add two fields specifically for CWE label auditing: Key Invariant Violated and Non-Examples/Boundaries. The invariant field captures the security condition whose violation distinguishes one weakness from another; for instance, CWE-787 and CWE-125 may both involve out-of-bounds

3.3. Tool 1: The Classifier The Classifier aims to identify the CWE most supported by a CVE’s advisory, patch, code, and static-analysis evidence, without conditioning on the CVE’s existing NVD label. This prediction is not itself the final label-quality measurement: rather, it provides the evidence-supported candidate CWE label that the Arbitrator later uses to judge the NVD label. It works in two phases as elaborated below. 3.3.1. Phase 1: Offline CWE Knowledge Construction. Phase 1 takes the MITRE CWE corpus filtered to the A L LOWED mapping space as input and constructs the catalogside knowledge used by the Classifier. This phase is offline as the CWE catalog is stable relative to per-CVE inference, and that catalog-wide reasoning should be performed once, inspected, and reused rather than recomputed for every CVE. Step 1.1: Seclometry Generation. For each A LLOWED CWE, C WE AGENT constructs a seclometry containing both a natural-language security signature and a normalized categorical schema. The input is the raw MITRE XML entry, including its description, common consequences, modes of introduction, observed examples, and relationship metadata. These fields provide the raw material for most seclometry fields: the LLM’s role is to organize and distill the CWE’s mechanism, trigger conditions, violated invariant, code patterns, source/sink roles, impact, and boundaries into the common representation (as defined in §3.2). The most important synthesis occurs in the boundary fields. MITRE entries often describe what a weakness is, but they rarely state explicitly what it is not. For auditing, this negative space is crucial: many CWE confusions occur between nearby weaknesses that share a broad mechanism or consequence but differ in the defining invariant. For example, CWE-787 and CWE-125 both involve out-of-bounds memory access, but differ on write versus read. C WE AGENT therefore derives Non-Examples/Boundaries from CWE relationships (PeerOf, CanPrecede, CanFollow), neighboring entries, and common confusion patterns, so that later matching can penalize candidates that fit surface symptoms but violate the CWE’s defining boundary. The output is a collection of 842 CWE seclometries, one per A LLOWED CWE. Each seclometry uses the same twocomponent format that Agent 1 will later produce for CVEs.

5

normalized schema

stylometry component [Description] A memory safety violation occurs when a product performs a write operation to a memory address that is either before the start or after the end of the intended destination buffer. [Root Cause] Failure to enforce spatial memory safety constraints during write operations, typically due to incorrect bounds calculation, missing length validation, or flawed pointer arithmetic. [Trigger Conditions] Provision of input that causes a pointer or index to exceed the allocated memory range, or processing data whose length is greater than the destination buffer capacity. [Key Invariant Violated] Spatial memory safety; the system must ensure that write operations are strictly confined to the memory region explicitly allocated for the target object. [Observable Effects / Impact] Corruption of adjacent memory structures (variables, heap metadata, return addresses), leading to application instability, unexpected program state, or arbitrary code execution. [Typical Code Patterns] Iterative writes in loops without boundary checks, usage of length-unaware copy functions (e.g., strcpy), or pointer arithmetic influenced by untrusted values without prior validation. [Common Sources / Sinks] Sources: user_input, network_packet, file_header, untrusted_object, ipc. Sinks: array_index, pointer_arith, allocation. [Non-Examples / Boundaries] Distinguished from CWE-125 (Out-of-bounds Read) as it involves modification of memory rather than disclosure. Distinguished from CWE-476 (Null Pointer Dereference) because it involves valid but incorrect memory offsets rather than an absolute zero-address access. Distinguished from CWE-416 (Use After Free) as the focus is on the spatial boundary violation rather than temporal validity of the pointer.

{"mechanism_family": "buffer_overflow", "mechanism_detail": "Memory write operation outside the spatial boundaries of an allocated buffer", "operation": ["index_calc", "pointer_arith", "loop_bound", "alloc_size_calc"], "signedness": "unknown", "width_relation": "unknown", "source_type": ["user_input", "network_packet", "file_header", "ipc", "untrusted_object"], "sink_role": ["array_index", "pointer_arith", "allocation"], "impact_class": ["mem_corruption", "dos", "rce", "integrity_loss"], "hard_boundaries": [ "Not an Out-of-bounds Read (CWE-125), which involves reading memory outside of boundaries for information disclosure.", "Not a Null Pointer Dereference (CWE-476), which targets address zero specifically.", "Not a Use-After-Free (CWE-416) which is a temporal memory violation; OOB Write is a spatial memory violation." ]}

Figure 3: Seclometry of CWE-787 (Out-of-bounds Write), showing both the eight-field stylometry component (left) and the normalized schema (right). The stylometry component is the natural-language signature read by LLM-based matching stages; the normalized schema is used by multi-view clustering and per-candidate scoring. The same two-component structure applies to CVE seclometries. This uniformity is essential: Phase 2 does not compare CVE descriptions directly to CWE prose, but compares CVE and CWE seclometries in the same semantic representation. Step 1.2: Multi-View Semantics Clustering. The 842CWE space is too large and semantically uneven for direct all-against-all reasoning at every CVE. At the same time, a single global clustering would be too coarse: two CWEs may agree on one semantic facet, such as impact, while differing on another, such as violated invariant or source/sink role. C WE AGENT therefore clusters the catalog by view. Each seclometry field defines one view of the CWE space, and the CWEs are partitioned separately under that view. This multi-view design preserves distinctions that a joint embedding or single partition would blur. A candidate may be close to a CVE under the impact view but far under the invariant view; another may share the mechanism but differ in the sink role. Keeping these views separate lets the Matcher perform coarse-to-fine narrowing by combining evidence across facets, rather than relying on a single similarity score that averages away the reason for disagreement. For each view, the clustering procedure partitions all 842 CWE seclometries into semantically coherent clusters. Each cluster has a short label, a concise theme summary, and a list of CWE IDs. The prompt (Figure 4) enforces three constraints: clustering must use only the selected seclometry field, not superficial CWE names or numeric IDs; every CWE must appear in exactly one cluster for that view; and clusters should be semantically meaningful rather than mechanically balanced. This produces field-specific partitions that can later be intersected, ranked, or relaxed during matching (by the CWE Matcher in Step 2.2). This prompt clusters the 842 ALLOWED CWEs along a single seclometry field; it is instantiated once per field, eight times in total. Figure 4 shows the template. A single-pass call suffices because all 842 per-field values fit within the

Multi-view Semantic Clustering prompt You are helping build a semantic index for CWE classification. Task: Cluster the following CWEs ONLY based on the stylometry field ’<field_name>’. Important instructions: 1. Use ONLY the content of this field. 2. Do NOT use external CWE knowledge unless it is directly inferable from the provided text. 3. Do NOT cluster by CWE numbering or naming similarity alone. 4. Cluster by security semantics expressed in this field. 5. Keep the number of clusters reasonably small but semantically meaningful. 6. Avoid singleton clusters unless truly necessary. 7. Every CWE must appear in exactly one cluster. 8. The clusters should be useful for later multi-view filtering. Output format: Return ONLY valid JSON in the following format: {"field": "<field_name>", "clusters": [ { "cluster_name": "<short semantic label>", "cluster_summary": "<1--3 sentence summary of the shared semantic theme for this field>", "members": ["CWE-xxx", "CWE-yyy"] } ]} Here are the CWEs to cluster: 1. CWE-1004: ... ...

Figure 4: Prompt used in Step 1.2 to cluster the 842 ALLOWED CWEs along one seclometry field. The prompt is instantiated once per field, eight times in total, with <field_name> and the per-CWE field values filled in. The prompt has been lightly edited from the implementation version for terminology alignment with the paper; the unedited version is available in the artifact. model’s context window; hierarchical or iterative schemes would require an explicit similarity metric—precisely the judgment we delegate to the LLM—and would compound non-determinism across rounds. Temperature is fixed at 0 with fixed prompt and model versions, so each per-field clustering is deterministic given its input. The output of Phase 1 is the Index of Multi-View Clusters. For each seclometry field, the Index records the viewspecific clusters and the CWE membership of each cluster; equivalently, for each CWE, it records that CWE’s cluster

6

Agent 1 does not invoke every tool for every CVE. When the available evidence already determines the weakness class unambiguously, it emits the seclometry directly; otherwise, it proceeds through additional evidence-gathering steps. When re-invoked by the refinement loop, Agent 1 receives targeted evidence queries from Agent 2 and focuses its tool use on the distinctions that matter for the remaining candidate CWEs. The Builder emits the CVE seclometry together with evidence anchors, including advisory URLs, file paths, code line ranges, patch excerpts, and CodeQL snippets supporting each claim. Step 2.2: Multi-View CWE Matching (Agent 2: CWE Matcher). This step/agent turns the CVE seclometry into a CWE prediction. It works in two stages. Stage A: multi-view cluster filtering. The goal of Stage A is to reduce the candidate space cheaply while preserving semantically plausible alternatives. For each seclometry field, the Matcher takes the CVE’s value for that field and compares it against the cluster labels, summaries, and representative members from Phase 1’s partitioning of the catalog under the same view. It then selects the best-matching cluster and, when appropriate, a runner-up cluster. This is cluster selection, not re-clustering: the Phase 1 partitions are fixed, and Stage A only chooses among them. As the Matcher processes the seclometry fields, each selected cluster contributes to a cumulative multi-view prior over CWEs. Candidate CWEs receive support when they appear in clusters selected under multiple views, and the candidate set is narrowed by combining best- and runner-upcluster memberships across fields. This produces a short list, typically small enough for Stage B to score each candidate independently. The scoring weights are fixed hyperparameters, not tuned per dataset. Stage B: per-candidate scoring. Stage B scores each surviving CWE against the CVE independently. The scoring prompt (Figure 5) receives the CVE seclometry, the candidate CWE seclometry, and, when available, relevant source or patch snippets collected by Agent 1. It returns a compact structured judgment: a numeric score (0–100), confidence level, match-quality label (strong/partial/weak/contradiction), and short justification. The scoring rules follow directly from the discriminative role of seclometry. The Matcher is instructed to (1) score the primary weakness mechanism rather than downstream consequence; (2) prefer the most specific correct root-cause CWE; (3) penalize contradictions with the candidate CWE’s Non-Examples/Boundaries; and (4) use source or patch evidence when it helps distinguish neighboring weaknesses. Thus, Stage B does not merely ask whether a CWE name sounds plausible: it asks whether the candidate’s defining mechanism and boundary conditions are supported by the CVE evidence. The model reasons step by step internally but outputs only the compact judgment—a form of internal zero-shot chain-of-thought [13]—keeping per-candidate cost low across the short list. The prompt’s four rules operationalize the discriminative design of the seclometry: score the primary weakness mechanism rather than downstream consequence, prefer the

Algorithm 1: Phase 2 (Code-Semantic CWE Label Prediction) Input : CVE record r ; Index of Multi-view Clusters I Output: Predicted CWE c⋆ , confidence, evidence 1 Q ← ∅ // targeted evidence queries 2 for i ← 1 to 3 do 3 (scve , E) ← Agent1(r, Q) // Agent 1 4 S ← MultiViewFilter(scve , I) // Stage A 5 foreach c ∈ S do 6 σc ← PerCandidateScore(scve , c, E) // Stage B 7 (c⋆ , conf ) ← Rank(S, {σc }) 8 if conf ̸= low then 9 return (c⋆ , conf , E) 10 Q ← SelfReflect(S, {σc }, scve , E) 11 if Q = ∅ then return (c⋆ , conf , E) ⋆ 12 return (c , conf , E)

assignment under every view. Phase 2 consumes this Index to narrow the candidate space before per-candidate scoring. Because the Index is built once and reused across all CVEs, it provides catalog-wide semantic structure without adding per-CVE catalog-construction cost. 3.3.2. Phase 2: Code-Semantic CWE Label Prediction. For a given CVE under auditing, Phase 2 invokes the Classifier against the CVE record and available references as inputs, and produces the predicted CWE label and supporting evidence as outputs. Algorithm 1 details the Phase 2 control flow for a single CVE. Agent 1 builds the CVE seclometry and evidence bundle (line 3); Stage A filters the 842 candidates to a short list against the Index, and Stage B scores each survivor (lines 4–6). If the top candidate’s confidence is not low, the prediction is returned immediately (lines 7–8). Otherwise the self-reflection step produces targeted evidence queries (line 9) that re-invoke Agent 1 on the next iteration; the loop runs at most three times, an empirically chosen budget beyond which the predicted CWE rarely changes. This phase works in two steps below, each realized by an LLM-based agent. Step 2.1: Evidence-Guided Seclometry Construction (Agent 1: Seclometry Builder). Agent 1 constructs the seclometry of the given CVE. To ground the construction in evidence (e.g., patch URLs and any referenced source artifacts), which is often incomplete, the agent uses four tools: web search to find advisories when the CVE reference list is thin; a webpage reader that follows Wayback snapshots where available to reduce post-disclosure leakage; a source downloader for files named in the patch or advisory; and a CodeQL scanner that surfaces candidate weakness patterns. CodeQL findings are auxiliary rather than authoritative: a positive finding can support a mechanism, but a null result is not treated as negative evidence. Agent 1 applies these tools across the three sub-steps shown in Figure 1. Advisory description extraction drafts an initial weakness signature from the CVE description and advisories. Code and patch understanding locates the rootcause construct in the patch and source. Static analysis folds confirmed CodeQL findings into the evidence bundle when they help identify or disambiguate the weakness mechanism.

7

or after a fixed iteration cap. The cap prevents ambiguous CVEs from consuming unbounded tool calls. Scoring weights. The Matcher combines evidence across the eight views with a small set of fixed weights. In Stage A, every CWE in the best-matching cluster of a field gains a confidence-weighted contribution to its multiview prior, and members of the runner-up cluster contribute at a reduced weight; the candidate set is hard-narrowed after each field by intersecting it with the union of the best- and runner-up-cluster members, so a CWE survives only if it appears in one of the two selected clusters for every field. In Stage B, each candidate’s final matching score is a linear combination of its per-candidate score and its Stage A multiview prior. All weights are set once and held fixed across every experiment in the paper; we did not tune them per dataset.

most specific correct root-cause CWE, penalize contradictions with the candidate’s Non-Examples/Boundaries, and use source code only when it helps distinguish mechanism. The model reasons step by step internally but emits only the compact structured judgment. Per-candidate scoring prompt Return JSON only. No extra text. Task: Score whether the vulnerability matches this SINGLE candidate CWE. Rules: 1) Score the PRIMARY weakness mechanism (root cause), not downstream consequence. 2) Prefer the most specific correct root-cause CWE. 3) Penalize contradictions with the candidate boundaries / non-examples. 4) Use source code only if it helps. 5) Keep the decision compact. 6) Reason step by step internally before answering, but DO NOT output your private chain-of-thought. Output JSON: { "cwe_id": "<cwe_id>", "score": <integer 0--100>, "confidence": "low|medium|high", "match_type": "strong|partial|weak|contradiction", "note": "<max 20 words>" } Vulnerability normalized schema: <<<VNORM ... VNORM>>> Vulnerability stylometry: <<<VSTY ... VSTY>>> Relevant source evidence (when available): <<<SOURCE file_path: ... start_line, end_line: ... <code snippet> SOURCE>>> Candidate CWE: ID: <cwe_id>, Name: <cwe_name> Candidate normalized schema: <<<CNORM ... CNORM>>> Candidate stylometry: <<<CSTY ... CSTY>>>

3.4. Tool 2: Arbitrator The Classifier predicts the CWE most supported by the collected evidence, but a prediction-label mismatch is not automatically an NVD error. CWE entries overlap, differ in granularity, and sometimes describe different defensible views of the same vulnerability. Conversely, a mismatch may also reflect a real NVD mislabel rather than a classifier mistake. C WE AGENT’s second tool, the Arbitrator, resolves this ambiguity by judging the existing NVD label against the CVE evidence and the Classifier’s prediction. CWE Label Adjudication. Given a CVE, the evidence bundle, the Classifier’s predicted CWE, and the existing NVD label, the Arbitrator judges the NVD label under the measurement taxonomy we use in this paper: exact, overlapambiguous, defensibly alternative, or evidence-inconsistent. 1) Exact match. The NVD label and the predicted, evidencesupported CWE label are the same (S1). 2) Overlap ambiguity. The NVD label and the predicted CWE overlap for this CVE; both are materially defensible, typically because one is broader, narrower, or adjacent to the other in the CWE taxonomy (S2). 3) Defensibly alternative labeling. The NVD label is not an exact or overlapping match, but remains supported by the evidence under a plausible interpretation. This covers cases in which one CWE label captures the root cause while the other captures the resulting weakness or impact, or one captures a lower-level mechanism while the other captures a higher-level manifestation (S3). 4) Evidence-inconsistent labeling. The evidence supports the predicted CWE while contradicting, or providing no defensible support for, the NVD label (S4). The Arbitrator further maps these four Situations into three output labels by a fixed priority rule. If the NVD label is exact, overlap-ambiguous, or defensibly alternative, the output is count as correct. If the NVD label is evidenceinconsistent, the output is discard. Otherwise, the output is classifier error. These labels serve distinct downstream roles. count as correct supports ambiguity-aware accuracy by treating taxonomy ambiguity and defensible imprecision as non-errors. discard removes unreliable reference labels from the classifier-evaluation denominator and marks them

Figure 5: Prompt used by Agent 2 to score a single shortlisted CWE against the CVE. The prompt is instantiated once per candidate on the Stage A short list and returns a compact structured judgment. An internal chain-of-thought instruction (Rule 6) asks the model to reason step by step before producing the compact output. The prompt has been lightly reflowed for the figure; the production version is available in the artifact. Each candidate’s final score combines its Stage B match score with the Stage A multi-view prior. The highest-scoring candidate becomes the predicted CWE. When the top candidate has low confidence or a small margin over the runnerup, the Matcher performs a self-reflection step that generates targeted evidence queries (e.g., “does the patch modify the allocation-size computation, or only the bounds check?”). These queries drive the evidence-refinement loop. Evidence-Refinement Loop. This loop lets Agent 2 request additional evidence from Agent 1 when the current evidence cannot distinguish close candidates. The loop triggers when the top-candidate confidence is low or Agent 2’s self-reflection produces at least one query. Agent 1 is then re-invoked with those targeted queries, gathers the requested evidence, and Agent 2 reruns filtering and scoring. The loop terminates when the prediction reaches medium or high confidence, when Agent 2 produces no new evidence query,

8

model-family separation from the Classifier while remaining practical at measurement scale. All LLM calls use fixed prompts, fixed model versions, and temperature 0, so each call is deterministic given its input. Deterministic post-processing enforces only outputformat constraints, such as repairing malformed JSON, and does not alter the semantic content of any decision. The evidence-refinement loop between Agents 1 and 2 is capped at three iterations, an empirically chosen budget beyond which the predicted CWE rarely changes. The Seclometry Builder uses standard web-fetching libraries to retrieve advisories and source artifacts, and invokes the CodeQL CLI with standard code-scanning query packs for the supported languages: C/C++, Python, JavaScript, Java, C#, Go, and Ruby. C WE AGENT draws on three external data sources: the CWE catalog is MITRE’s cwec latest.xml, filtered to entries with A LLOWED mapping status (842 CWEs at the time of our study); CVE records come from the cvelistV5 Git repository at each CVE’s preCWE-assignment commit, so the descriptions and references available to C WE AGENT precede the reference CWE assignment; and reference CWE labels and change histories are retrieved from NVD. To reduce data leakage, advisory and reference pages are fetched from Wayback Machine snapshots dated before the relevant CWE assignment, or before the model’s training cutoff when appropriate, whenever such snapshots are available; live URLs are used only when no suitable snapshot exists.

as candidate NVD mislabels for the wild-scale measurement. keep as incorrect preserves cases where the NVD label is supported and the Classifier prediction is a genuine error. The adjudication prompt is shown in Figure 6. This prompt adjudicates a single (CVE, predicted CWE, NVD label) tuple into the Arbitrator’s threeway decision. It checks the three situations in priority order—overlap ambiguity, alternative-but-defensible labeling, and reference-CWE inconsistency—and emits the corresponding decision (count_as_correct, discard, or keep_as_incorrect) over the evidence bundle gathered by the Classifier, without re-deriving evidence of its own. Arbitrator prompt You are reviewing a CWE classification mismatch for one CVE. Your job is NOT to predict the best CWE from scratch. Your job is only to determine whether this mismatch should still count against the technique. CVE ID: <cve_id> Assigned CWE: <ref> Predicted CWE: <pred> Please check the following three situations. Situation 1: overlap / ambiguity. Check whether this CVE genuinely lies in the overlap between the predicted CWE and the reference CWE, including cases where both CWEs are materially defensible for this specific vulnerability even if one is more general and the other more specific. Mark true when the two substantially overlap for this CVE. Situation 2: alternative-but-defensible labeling. Check whether the predicted CWE is clearly supported by the CVE’s description, patch, code, or reasoning evidence, even when it does not strongly overlap with the reference CWE. This includes cases where one CWE captures root cause while the other captures resulting weakness or impact, or where one captures a lower-level mechanism while the other captures a higher-level manifestation. Do not require the labeled CWE to be wrong; mark true if the predicted CWE is clearly evidenced and reasonable. Situation 3: reference CWE inconsistency. Check whether there is significant inconsistency between the evidence and the reference CWE. Only mark true when the evidence strongly suggests the reference CWE is likely wrong, misleading, or materially less supported than the predicted CWE. Decision policy: If Situation 1 is true, decision = count_as_correct. Else if Situation 2 is true, decision = count_as_correct. Else if Situation 3 is true, decision = discard. Else decision = keep_as_incorrect. Guidance: be fair about accepting alternative-but-defensible views. Many CVEs can reasonably be labeled from different perspectives (root cause, direct bug type, memory effect, injection vector, access-control consequence). Do not count cases where the prediction is only loosely related or merely adjacent in the taxonomy. Return strict JSON: { "situation_1_overlap_ambiguity": true|false, "situation_2_alternative_but_defensible": true|false, "situation_3_reference_inconsistency": true|false, "decision": "count_as_correct" | "discard" | "keep_as_incorrect", "reason": "short explanation" } Evidence: <<<EVIDENCE <CVE description, advisory text, code/patch snippets, CodeQL findings, Phase 2 reasoning> EVIDENCE>>>

5. Evaluation We evaluate the two tools in C WE AGENT separately, as each has its own notion of correctness and no single dataset has both a known-correct label and a known-incorrect existing label for the same CVE. The Classifier is evaluated for prediction accuracy on CVEs whose correct CWE is known (RQ1), and the Arbitrator for adjudication reliability against CVEs whose existing-label correctness is known (RQ2).

5.1. Classifier Prediction Accuracy (RQ1) Ground-truth data. We built a high-confidence benchmark by filtering rather than re-labeling: two authors independently inspected candidate CVEs—reading the description, advisory, and patch—and retained only those for which both agreed that the NVD-assigned CWE clearly captures the root-cause weakness, discarding any inaccurate, ambiguous, or contested label. The ground-truth labels are thus NVD’s own, which avoids author-side labeling bias. The resulting benchmark contains 100 CVEs from opensource projects across 2017–2026 (at least three per year), spanning 38 distinct CWEs with no CWE covering more than 9% of cases. 19 CVEs were disclosed after the August 31, 2025 training cutoff of the LLM used by Agents 1 and 2, which we keep as a leakage-mitigation subset. We report top-1 exact-match accuracy (prediction equals groundtruth CWE) and ambiguity-aware accuracy (i.e., S1 + S2 + S3 in §3.4). To avoid conflating classifier evaluation with

Figure 6: Prompt used by the Arbitrator to adjudicate a single (CVE, prediction, reference) tuple. The Arbitrator is implemented with Claude Sonnet 4.6, a different LLM family from the GPT-5.4-mini used in Phase 2. The prompt has been lightly reflowed for the figure; the production version is available in the artifact.

4. C WE AGENT Implementation C WE AGENT uses three LLMs, chosen for the workload of each stage. Phase 1 uses Claude 4.6 Opus for seclometry generation and multi-view semantic clustering, because this phase performs one-time catalog-wide reasoning where quality matters more than per-call cost. Phase 2 uses GPT5.4-mini for the Seclometry Builder and CWE Matcher, which run once per CVE and must scale to wild-scan measurement. The Arbitrator uses Claude Sonnet 4.6, providing

9

Table 3: Per-category and aggregate precision, recall, and F1 for C WE AGENT’s Arbitrator on the 100-CVE dataset

Table 2: Accuracy and efficiency on the 100-CVE controlled benchmark. Amb.: ambiguity-aware accuracy; Lat.: per-CVE latency in seconds; Cost: per-CVE LLM cost in US dollars; In/Out: per-CVE input/output tokens in thousands

Category count as correct discard keep as incorrect Macro avg Micro avg Weighted avg

System Exact Amb. Lat. (s) Cost ($) In/Out (K) C WE AGENT 85% 92% 61.7 0.070 76.6/2.8 Claude 4.6 Opus (single) 74% 83% 51.7 0.172 14.9/3.9 GPT-5.4-mini (single) 46% 59% 5.4 0.030 14.9/4.2 TreeVul (diff, supervised) 52% 65% 3.7 — —

Precision 82.61% 97.67% 90.91% 90.40% 90.00% 91.05%

Recall 97.44% 84.00% 90.91% 90.78% 90.00% 90.00%

F1 89.41% 90.32% 90.91% 90.21% 90.00% 90.03%

Table 4: Overall distribution of measurement outcomes Outcome S1: exact match S2: overlap ambiguity S3: defensibly alternative S4: NVD inconsistent with evidence Classifier Error (Err): NVD label correct Ambiguity-aware (AmbAw; S1+S2+S3)

arbitrator evaluation, we use manual adjudication, not the Arbitrator, to compute ambiguity-aware accuracy here. Baselines. To separate C WE AGENT’s design from raw LLM capability, we compare C WE AGENT against three baselines. The first two are single-prompt LLM baselines over the same 842-CWE candidate space: GPT-5.4-mini, the model used by C WE AGENT’s Agents 1 and 2, and Claude 4.6 Opus, a larger model. Both receive the same initial CVE sources but must predict in one call, without evidence-gathering tools or the two-stage matching pipeline. The third baseline is TreeVul [7], a SOTA (supervised diffbased) CWE classifier over the CWE hierarchy; we train it on its released dataset and evaluate it on our benchmark. Results. C WE AGENT achieves the highest exact-match (85%) and ambiguity-aware (92%) accuracy (Table 2). The efficiency numbers show where the improvement comes from. Compared with the GPT-5.4-mini single-prompt baseline—same base model, same evidence—C WE AGENT uses substantially more input context, reflecting Agent 1’s evidence gathering and the per-candidate comparisons in Agent 2. The accuracy gain is therefore not attributable to model scale alone, but to converting CWE prediction into evidence construction followed by constrained semantic matching. Claude 4.6 Opus shows that a stronger single model can improve over GPT-5.4-mini yet still falls short of C WE AGENT while costing more per CVE. TreeVul’s low cost and latency reflect the merits of supervised diff-based classification, but its lower accuracy exposes the limitation of restricted-label, diff-only prediction for full-catalog CWE auditing. Overall, C WE AGENT’s per-CVE cost is dominated by Agent 1’s evidence gathering—roughly five times the input tokens of the single-prompt baselines—but remains cheaper per CVE than the Claude 4.6 Opus baseline while delivering substantially higher accuracy.

Count 7,732 3,890 990 564 2,380 12,612

Share 49.70% 25.01% 6.36% 3.63% 15.30% 81.07%

tent), or keep as incorrect (C WE AGENT wrong), with Cohen’s κ = 0.84; disagreements were resolved by discussion. Results. The Arbitrator achieves 90% overall accuracy, with per-category F1 scores tightly clustered between 89% and 91% (Table 3). Precision and recall are both at least 82% in every category, indicating balanced reliability rather than performance concentrated on one outcome. Because adjudication is a single LLM call over evidence already gathered by the Classifier, it is inexpensive enough to apply at wild-measurement scale. For the Arbitrator, the most important observation is the absence of a dominant failure mode. The high recall of count as correct means the Arbitrator rarely rejects defensible labels, while the high precision of discard means it is conservative when marking a reference label unreliable. This balance matters for our measurement: an overly aggressive Arbitrator would inflate the mislabel rate, whereas an overly permissive one would hide evidence-inconsistent labels. The clustered F1 scores indicate that neither effect dominates.

6. Measuring CWE Mislabeling in the NVD We applied C WE AGENT to 15,556 CVEs disclosed between 2017 and 2026 with GitHub fix (patch) commits, and used the Arbitrator to adjudicate every disagreement between the Classifier’s prediction and current NVD label.

6.1. The Landscape of CWE Mislabels

5.2. Arbitrator Adjudication Reliability (RQ2)

6.1.1. Overall Prevalence. Table 4 summarizes the outcome distribution of the 15,556 adjudicated CVEs. C WE AGENT and NVD agree exactly in 49.70% of cases; a further 25.01% are S2 overlap ambiguities and 6.36% are S3 defensible alternatives (§3.4), so NVD’s label is ambiguityaware correct in 81.07%. The remaining disagreements split into 3.63% S4 verdicts, where NVD’s label is inconsistent with the evidence, and 15.30% true classifier errors, where C WE AGENT is wrong and NVD correct.

To evaluate the Arbitrator, we measure its precision and recall across its three decision categories on a unified test set of 100 CVEs constructed from two complementary sources. Test set. The test set combines two ground-truth halves independent of C WE AGENT. The first is 50 CVEs whose assigned CWE was later revised in NVD’s change history and which we manually verified as semantically inconsistent with the evidence (true NVD mislabels). The second is 50 CVEs sampled from the wild-scan disagreements for which we manually confirmed NVD’s label is correct (cases the Arbitrator should not flag). Two authors independently labeled each as count as correct, discard (NVD inconsis-

6.1.2. Longitudinal Trend. Table 5 reports outcome rates by CVE publication year across the full ten-year span (2017–2026). NVD’s strict exact-match rate is essentially stable across the period, ranging from 43.72% (2019) to

10

Table 6: Outcome distribution by CWE-699 category (categories with ≥100 CVEs)

Table 5: Outcome distribution by CVE publication year Year 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 Total

N 911 731 693 985 1,626 2,071 2,021 2,074 2,179 2,265 15,556

S1% 50.93 53.76 43.72 51.07 52.40 52.00 51.36 47.54 46.08 49.09 49.70

AmbAw% 86.06 87.55 84.27 85.28 80.69 80.15 78.38 81.05 80.54 78.23 81.07

S4% 0.99 2.46 6.06 2.94 4.80 3.67 4.16 2.85 4.54 3.09 3.63

Err% 12.95 9.99 9.67 11.78 14.51 16.18 17.47 16.10 14.91 18.68 15.30

Category N S1% AmbAw% S4% Err% CWE-137 Data Neutralization 3689 81.92 91.41 2.11 6.29 CWE-1218 Memory Buffer Errors 1160 58.36 90.52 3.63 5.78 CWE-1219 File Handling 878 69.59 89.29 2.51 8.09 CWE-399 Resource Management 593 53.79 80.78 4.89 14.33 CWE-465 Pointer Issues 459 78.00 88.45 2.61 8.93 CWE-840 Business Logic Errors 432 34.95 65.28 4.17 30.56 CWE-417 Communication Channel 419 80.19 89.26 2.63 8.11 CWE-189 Numeric Errors 401 67.83 84.54 3.24 11.97 CWE-19 Data Processing 312 68.27 78.21 3.85 17.95 CWE-1211 Authentication Errors 266 26.32 67.67 4.51 27.07 CWE-389 Error Conditions 230 55.22 76.09 4.35 18.70 CWE-199 Information Management 213 37.09 82.16 3.76 13.15 CWE-1212 Authorization Errors 201 18.41 45.77 2.99 51.24 CWE-1214 Data Integrity 197 31.98 75.13 4.57 20.30 CWE-438 Behavioral Problems 169 55.62 75.74 4.14 20.12 CWE-429 Handler Errors 157 69.43 84.71 5.73 9.55 CWE-310 Cryptographic Issues 157 32.48 80.25 2.55 17.20 CWE-1226 Complexity Issues 145 73.79 86.90 3.45 9.66 CWE-255 Credentials Management 106 50.00 78.30 3.77 16.98

53.76% (2018) with no monotonic trend. By this strictest measure, NVD’s CWE labels are no worse on CVEs published in 2026 than on those from 2017. The picture changes when the disagreement cases are examined more carefully. The share of CVEs carrying CWE labels that are inconsistent with their own evidence rises substantially: from 1–3% in 2017–2018 to 3–6% in 2021– 2026. In absolute terms, the 2017–2018 cohort contributes 27 inconsistent labels, while the 2024–2026 cohort contributes 228, accounting for over 40% of all evidenceinconsistent labels in the corpus. The share of CVEs whose labels are neither most-specific nor defensibly equivalent also rises monotonically from 9.7–12.9% in 2017–2020 to 14.5–18.7% in 2022–2026, contributing to a 6–9 percentage point decline in ambiguity-aware accuracy across the period (86–88% in 2017–2018 versus 78–81% in 2023–2026). Together these indicate a decline in CWE labeling quality for open-source CVEs as the annual volume of disclosed CVEs has continued to grow [14]. Finding 1: NVD’s CWE labels are far more often imprecise than wrong: half the corpus mismatches the strict evidence-supported assignment, but most mismatches are taxonomy overlap and only 3.63% are evidenceinconsistent. Exact-match accuracy is stable across 2017– 2026, yet the evidence-inconsistent share has grown, indicating a decline in labeling quality as CVE volume rises.

Table 7: Outcome distribution by specific CWE (top 20) CWE N S1% AmbAw% S4% Err% CWE-79 XSS 2506 89.07 93.26 1.52 5.07 CWE-22 Path Traversal 741 72.60 91.23 2.29 6.34 CWE-125 Out-of-bounds Read 635 77.64 92.44 2.05 5.35 CWE-20 Improper Input Validation 461 0.00 65.29 2.60 30.80 CWE-200 Info Exposure 448 0.00 75.45 4.46 19.64 CWE-89 SQL Injection 441 83.90 90.02 2.95 7.03 CWE-476 NULL Pointer Deref 409 84.60 92.18 2.44 5.38 CWE-918 SSRF 361 85.87 90.58 2.77 6.65 CWE-787 Out-of-bounds Write 344 47.09 88.08 6.10 5.81 CWE-78 OS Command Injection 325 61.85 88.31 3.38 8.00 CWE-400 Resource Exhaustion 314 0.00 71.02 5.10 23.57 CWE-416 Use After Free 298 21.81 93.29 3.36 3.36 CWE-352 CSRF 277 85.20 88.45 3.61 7.94 CWE-94 Code Injection 276 50.36 82.97 4.35 11.96 CWE-863 Incorrect Authorization 250 26.40 58.00 1.20 39.60 CWE-770 Resource Allocation 241 46.47 82.57 4.56 12.86 CWE-862 Missing Authorization 232 46.98 62.50 2.16 35.34 CWE-190 Integer Overflow 230 68.26 83.91 1.74 13.91 CWE-122 Heap-based Buffer Overflow 201 44.28 79.60 11.94 7.96 CWE-284 Improper Access Control 200 0.00 55.50 4.50 39.00

highest-volume CWEs are among the most consistent. Because CWE-122 is within the larger memory-buffer family, this outlier is averaged away at the family level: genuine NVD mislabels localize to specific weakness types, not whole families. Two groups have near-zero exact-match by construction: discouraged abstract CWEs (e.g., CWE-20, CWE-200) that MITRE prohibits as primary labels, and CWEs with multiple valid nesting levels (e.g., use-after-free CWE-416), which the evidence supports defensibly but rarely at the mostspecific level (Table 7). In both, the labels are predominantly not wrong—S4 at or below the corpus average—but imprecise, leaving users to resolve the actual weakness. Over Project. We identify the project for 15,178 of 15,556 CVEs (97.6%) from the fix-commit URL (Table 8). Project-level exact-match varies widely, but the split is not random: injection-heavy web applications are labeled precisely while memory-safety-heavy systems software is labeled imprecisely. Projects inherit the precision of whatever CWE families dominate their vulnerabilities—low projectlevel exact-match is largely a restatement of the memorysafety precision gap, not an independent project effect. Evidence-inconsistency, by contrast, is uniform and near the corpus average across almost all projects, with one exception: Radare2, whose 15.49% S4 rate is over four times the corpus average and the highest in the corpus—a genuine concentration of NVD labels that contradict the CVEs’ own evidence, not an artifact of low precision.

6.1.3. Distribution. We now decompose label quality along four axes—the weakness type itself, the project, the programming language, and the assigning CNA—to locate where mislabeling concentrates. Over CWE. We group the 15,556 CVEs by their NVDassigned CWE’s parent category in the CWE-699 software development view, and separately by their specific CWEs. Across CWE-699 categories (Table 6), exact-match varies more than four-fold while evidence-inconsistency stays in a narrow band. At the family level NVD’s labels thus differ enormously in precision but only modestly in correctness: a family whose labels are rarely most-specific (e.g., authorization) is not thereby evidence-inconsistent. Low family-level exact-match reflects abstract or defensiblyalternative labeling, not outright error. The family-level band conceals sharp variation among specific CWEs (Table 7). Heap-based buffer overflow (CWE-122) is an outlier at 11.94% evidenceinconsistency—over triple the corpus rate—while the

11

Table 10: Outcome distribution by assigning CNA (top 15 by CVE count among CNAs with ≥20 CVEs)

Table 8: Outcome distribution by project (top 20) Project tensorflow/tensorflow torvalds/linux vim/vim xwiki/xwiki-platform gpac/gpac imagemagick/imagemagick tcpdump-group/tcpdump discourse/discourse openemr/openemr openclaw/openclaw ffmpeg/ffmpeg freerdp/freerdp thorsten/phpmyfaq pimcore/pimcore misp/misp microweber/microweber radareorg/radare2 wwbn/avideo librenms/librenms

N 390 363 186 181 150 149 111 108 102 102 98 94 89 87 86 80 71 67 61

S1% 53.59 30.58 52.69 43.65 46.67 40.27 86.49 28.70 58.82 44.12 39.80 54.26 83.15 85.06 69.77 60.00 38.03 73.13 91.80

AmbAw% 81.03 82.37 87.10 77.35 88.00 83.89 97.30 72.22 78.43 79.41 81.63 91.49 91.01 94.25 84.88 80.00 78.87 95.52 93.44

S4% 1.54 4.41 4.30 3.31 5.33 2.68 0.90 0.93 0.98 0.98 6.12 2.13 0.00 2.30 1.16 3.75 15.49 0.00 0.00

Err% 17.18 13.22 8.60 16.57 6.67 13.42 1.80 26.85 20.59 19.61 12.24 5.32 8.99 3.45 13.95 16.25 5.63 4.48 6.56

CNA github m mitre @huntrdev vuldb @huntr ai redhat snyk vulncheck mend facebook wordfence hackerone flexera checkpoint apache

N 4484 3305 1715 1557 987 755 726 454 375 244 132 104

S1% 42.51 66.02 51.14 49.71 39.72 47.55 48.76 54.19 47.47 31.97 51.52 50.96

AmbAw% 82.07 85.17 81.40 82.79 76.09 81.99 79.06 83.48 82.13 70.49 79.55 83.65

S4% 4.42 2.57 2.74 3.21 3.65 2.91 3.58 2.86 2.13 6.15 4.55 0.00

S1% 45.85 48.57 63.11 66.37 49.13 39.20 58.15 43.00 66.99 48.86 83.54 57.41 52.17 61.90 55.00

AmbAw% 78.62 84.75 78.77 86.84 75.68 83.95 88.50 69.97 74.76 79.55 91.14 92.59 100.00 100.00 95.00

S4% 3.53 4.14 4.46 3.51 2.23 2.47 1.92 3.75 2.91 3.41 0.00 0.00 0.00 0.00 0.00

Err% 17.25 11.02 16.58 9.36 21.09 12.96 8.95 26.28 22.33 14.77 8.86 7.41 0.00 0.00 5.00

Snyk) combine high precision with near-zero evidenceinconsistency, while the two highest-volume CNAs sit midrange, so high precision is not merely a small-specialist effect. The trustworthiness of an NVD CWE label thus depends substantially on which organization produced it. The S4 spread across CNAs is narrower than the exactmatch spread (Table 10), repeating the pattern seen throughout: organizations differ most in how precisely they label, less in how often they are outright wrong. Finding 2: Across all four axes, NVD labels vary far more in precision than in correctness. Exact-match swings widely by weakness type and by assigning CNA (a ∼70point spread), while evidence-inconsistency stays in a narrow band everywhere. Project- and language-level differences are composition effects of the underlying weakness types, not independent signals; genuine mislabels localize to a few specific CWEs (led by heap buffer overflow) and, at the project level, to Radare2.

Table 9: Outcome distribution by language (≥100 CVEs) Language C/C++ PHP JavaScript Python Go Java TypeScript HTML Ruby Rust C# Perl

N 7,599 3,574 1,526 684 403 324 313 293 103 88 79 54 23 21 20

Err% 13.16 12.07 15.57 13.55 19.76 14.04 16.67 13.22 15.47 22.95 15.15 16.35

Over Programming Languages. Label quality also differs across languages (Table 9), but the per-language CWE breakdown shows this is a composition effect: a given CWE is labeled with similar quality regardless of language, so languages that appear well- or poorly-labeled in aggregate are simply those dominated by precisely- or impreciselylabeled weakness types (injection versus memory-safety). Both S1 and S4 echo the weakness-type pattern, revealing no language-intrinsic effect. The extremes illustrate the two sources of aggregate variation. PHP has the highest S1 rate (66.02%) and one of the highest ambiguity-aware rates (85.17%), consistent with a corpus dominated by well-established web-vulnerability categories where NVD labels, advisories, and CWE definitions use standardized language. Rust has the lowest S1 rate (31.97%), lowest ambiguity-aware rate (70.49%), and highest S4 rate (6.15%): the Rust CVEs that remain visible in NVD often involve unsafe code, FFI boundaries, parser/runtime behavior, or resource exhaustion—cases that sit near boundaries between broad CWE classes and lack stable labeling conventions. C/C++ shows a third pattern: despite the largest sample size and a low exact-match rate (42.51%), its ambiguity-aware rate remains high (82.07%), indicating that many memory-safety disagreements are granularity or boundary disputes—e.g., an out-of-bounds read vs. write, or buffer overflow vs. broader memory corruption—rather than outright inconsistencies. This supports the paper’s central distinction between imprecision and mislabeling. Over CNAs. Conditioning on the assigning CNA produces the widest quality spread in the entire measurement: a roughly 70-point exact-match range (Table 10), far exceeding the variation across years, CWE families, projects, or languages. The strongest performers (e.g., Wordfence,

6.2. Severity and Impact 6.2.1. CVE Severity. We first ask whether CVSS severity—the metadata that most directly drives downstream prioritization—predicts label quality. Results are in Table 11. Across the four CVSS severity bands, NVD’s CWE label quality is essentially flat. Exact-match rates span less than six percentage points (Low at 54.46% to High at 48.58%) and evidence-inconsistency rates span less than one point (3.10% to 3.78%), with neither measure varying monotonically with severity. Critical-severity CVEs—the ones whose metadata most directly drives downstream prioritization and remediation—are labeled no more accurately than Mediumseverity CVEs. CWE labeling is thus not being systematically refined for the vulnerabilities that are most severe or most likely to receive downstream attention. A methodological implication is that severity cannot be used to obtain a cleaner CWE-labeled subset. If CWE-label quality tracked CVSS severity, researchers could reduce label noise by controlling for severity or by restricting analyses to High/Critical CVEs. Table 11 shows that this does not work: high-severity subsets retain nearly the same ambiguity-aware correctness and evidenceinconsistency rates as lower-severity subsets. Filtering by severity may change the risk profile of a dataset, but it does not materially improve the reliability of its CWE labels.

12

Table 12: Distribution across confirmed mislabels

Table 11: Outcome distribution by CVSS severity range Severity Critical High Medium Low

N 1,422 4,343 5,390 1,098

S1% 49.09 48.58 50.30 54.46

AmbAw% 81.43 79.67 78.48 81.24

S4% 3.45 3.78 3.67 3.10

Err% 13.78 16.05 17.48 15.30

Pattern Consequence vs. root cause Sub-type mislabel (within family) Injection sink / interpreter confusion Access-control & identity conflation Discouraged / wrong-branch label Multi-CWE accumulation (residual: file/path handling)

Studies that depend on NVD-derived CWEs therefore need controls based on weakness semantics, evidence provenance, and assigning organization, rather than treating CVSS severity as a proxy for label trustworthiness.

Share 23.1% 21.3% 19.5% 15.3% 10.3% 7.8% 3.9%

Case 2 (Sub-type mislabel): CVE-2022-1160 NVD label: CWE-787 (Out-of-bounds Write) NVD description. “heap buffer overflow in get_one_sourceline . . . prior to 8.2.4647” — generic overflow wording, no operation direction. AddressSanitizer output (PoC). ERROR: AddressSanitizer: heap-buffer-overflow ... READ of size 1 at 0x612000004c6c thread T0 #0 strlen #1 get_one_sourceline scriptfile.c:1930 Fix commit (vim 8.2.4647, scriptfile.c). patch 8.2.4647:"source" can read past end of copied line Solution: Add a terminating NUL. + if (ga_grow(&ga, 1) == FAIL) + break; + buf[ga.ga_len++] = NUL; Inconsistency. The sanitizer reports a READ past the buffer in strlen, and the fix adds a terminating NUL to stop the over-read; NVD assigned the outof-bounds write subtype (CWE-787). The evidence-supported label is CWE-125 (Out-of-bounds Read) — correct family, wrong memory-operation subtype.

6.2.2. Exploitability. Severity is a coarse proxy for a vulnerability’s importance; whether a CVE is actually exploited is the sharpest one. We therefore ask whether label quality differs for exploited CVEs, joining the corpus against two independent exploitation signals: the CISA Known Exploited Vulnerabilities (KEV) catalog [15], which records vulnerabilities exploited in the wild, and Exploit-DB [16], which records published proof-of-concept exploits. Of the 15,556 CVEs, 241 have an Exploit-DB entry. Their evidence-inconsistency rate is 3.73% (9 of 241), statistically indistinguishable from the 3.63% rate among the remaining CVEs; their exact-match (56.85%) and ambiguityaware (87.55%) rates are likewise close to the corpus as a whole. Only 39 corpus CVEs appear in KEV—too few to support a standalone rate—but their S4 rate (2.56%) is consistent with the same picture. As with severity, exploitation status does not predict label trustworthiness.

Figure 7: Sub-type mislabel: sanitizer and fix show an outof-bounds read (CWE-125), but NVD assigned CWE-787. and every pattern turns on a CWE-to-CWE relationship— cause vs. effect, sibling sub-types, sibling sinks. We describe each below with a representative example. Consequence vs. root cause (23.1%). The most common pattern labels a vulnerability by a true outcome of the flaw rather than its root cause, naming an effect downstream on the causal chain. The motivating example of §2.2 (CVE-2026-26014, Figure 2) is representative: nonce reuse (CWE-323) is the root cause but the record carries CWE-200 (information exposure), the consequence. Such broad consequence CWEs plausibly fit the end state of almost any data-exposure flaw, so a label derived from the described effect settles on them even when the description names the mechanism. Sub-type mislabel within a family (21.3%). The second pattern keeps the correct family but selects the wrong sibling, concentrated in memory-safety where an out-ofbounds read is labeled as a write (CVE-2022-1160, Figure 7). A structural driver is available: MITRE’s View-1003, which NVD uses for normalization, lacks CWE-122 and maps it to its parent CWE-787, so a heap over-read can be drawn toward the write sub-type because the read/write distinction is what normalization to a coarser node. This operates within a family, matching the family-level finding (§6.1.3) of imprecise sub-types but rare inconsistency. Injection sink / interpreter confusion (19.5%). The third pattern keeps the idea that untrusted input reaches a dangerous sink but names the wrong interpreter: in CVE2026-28211 (Figure 8) a Python eval() (CWE-95) is labeled as data-query injection (CWE-943). The record is also marked un-scheduled for enrichment, so the CWE was never revisited—one way an imprecise sink choice persists. Access-control & identity conflation (15.3%). The

Finding 3: Neither CVSS severity nor real-world exploitation predicts CWE label quality. A vulnerability’s impact is not a proxy for its label’s trustworthiness.

6.3. Failure Patterns 6.3.1. Individual Label Failures. We now examine the 564 S4 records, characterizing not their number but how the labels fail—what mismatch occurs between a record’s CWE and the weakness its evidence describes, and what property of the record or taxonomy plausibly gives rise to it. Two authors independently reviewed all 564 flagged records against their primary sources in order of authority (fixing patch, advisory, descriptions, proof-of-concept), confirming a record as a mislabel only when the highestauthority evidence contradicted the assigned CWE and returning defensible-but-imprecise readings to the defensible pool. This confirmed 434 genuine mislabels and excluded 130 (κ = 0.87); we reported the confirmed set to NVD and received acknowledgment. To characterize how the labels fail, three authors then open-coded the relationship between each assigned CWE and the evidence-supported weakness, merging codes and applying the resulting codebook to all 434. The codebook describes the records, not analyst intent; where we discuss why a mismatch arises we point only to observable properties of the record or taxonomy. Open coding of the 434 confirmed mislabels revealed six recurring patterns (Table 12), each defined by how the assigned CWE relates to the evidence-supported weakness. The top three alone cover nearly two-thirds of the set,

13

Case 3 (Injection sink confusion): CVE-2026-28211

Case 5 (Discouraged / wrong-branch label): CVE-2023-0565

NVD label: CWE-943 (Data Query Logic injection) NVD description. “. . . Python expressions embedded in the log may be evaluated . . . a maliciously crafted log file can lead to arbitrary code execution.” — the sink is a Python evaluator, with no data query involved. Fix commit (logReader.py). - seq = eval(txtSeq) + seq = generateSpeechSequence(txtSeq) Inconsistency. The fix replaces a Python eval() on attacker-controlled log content with a non-evaluating parser; the weakness is eval injection, i.e. CWE-95 (Eval Injection), a child of code injection (CWE-94). NVD’s CWE-943 instead names the data-query injection family (SQL/NoSQL) — the wrong interpreter, since no query language is involved.

NVD label: CWE-840 (Business Logic Errors). NVD description. “Business Logic Errors in GitHub repository froxlor/froxlor prior to 2.0.10.” — merely restates the category name and gives no mechanism. Fix commit (admin_templates.php). + if (!array_key_exists($language, $languages)) { + Response::standardError('templatelanguageinvalid'); + } Inconsistency. The fix rejects a user-supplied language value not in the known set — a missing input validation, not a business-rule flaw. NVD’s CWE-840 is a prohibited category and on the wrong branch; the enabling weakness is CWE1287.

Figure 10: Discouraged, wrong-branch label: the patch shows Improper Validation of Specified Type of Input (CWE-1287), but NVD assigned the prohibited category CWE-840.

Figure 8: Injection-sink confusion: the fix removes a Python eval() (CWE-95), but NVD assigned CWE-943. Case 4 (Access-control / identity conflation): CVE-2024-25618

Case 6 (Multi-CWE accumulation): CVE-2024-25639

NVD label: CWE-306 (Missing Authentication for Critical Function) NVD description. “. . . allows new identities from configured authentication providers (CAS, SAML, OIDC) to attach to existing local users with the same e-mail address . . . this results in a possible account takeover . . . [the system] checks the e-mail address passed by the provider to find an existing account.” Inconsistency. The external provider does authenticate the user; the flaw is that the authenticated identity is bound to a pre-existing local account by e-mail address alone, enabling takeover. NVD (NIST) assigned CWE-306 (Missing Authentication), but authentication is not missing. The evidence supports improper authentication, CWE-287, which is in fact the assigning CNA’s label; the two sources disagree and the evidence favors CWE-287.

NVD label: CWE-77 (Command Injection), CWE-79 (Cross-site Scripting), CWE-80 (Basic XSS) NVD description. “. . . clients inadequately sanitize the AI model’s response and user inputs. This can trigger Cross Site Scripting (XSS) via Prompt Injection from untrusted documents . . . read by [the application] from the internet.” Inconsistency. Untrusted content is rendered without sanitization, injecting script into the page — cross-site scripting; no OS or shell command is involved. CWE79 (XSS) and its child CWE-80 are both correct, but the third label, CWE-77 (Command Injection), has no support in the evidence: an unsupported extra CWE NVD added during enrichment.

Figure 9: Access-control/identity conflation: the provider authenticates the user, so the flaw is improper binding (CWE-287), not missing authentication (CWE-306).

Figure 11: Multi-CWE accumulation: the evidence supports only XSS (CWE-79/80); CWE-77 is an unsupported label.

fourth pattern confuses the kind of access-control failure— most often authentication with authorization. In CVE-202425618 (Figure 9), the provider does authenticate the user, so the flaw is improper identity binding (CWE-287), not missing authentication (CWE-306). The two sources disagree and the evidence favors the CNA. Discouraged or wrong-branch label (10.3%). The fifth pattern uses a CWE MITRE discourages—a high-level category or “business logic” bucket—in place of the specific weakness. Merely abstract uses are usually defensible and were excluded in review; the confirmed instances are also on the wrong branch. In CVE-2023-0565 (Figure 10) the prohibited category CWE-840 is assigned to what the patch shows is input validation (CWE-1287), not even an ancestor of the true weakness—the non-defensible tail of the discouraged-label behavior of §6.1.3. Multi-CWE accumulation (7.8%). The final pattern is record-level: several CWEs coexist, at least one correct and at least one unsupported, with the unsupported never removed. In CVE-2024-25639 (Figure 11) an NVD-added CWE-77 (command injection) persists beside the CNA’s correct XSS labels (CWE-79/80). The mechanism is additive— labels from different sources coexist with no reconciliation step to drop those the evidence does not support. The residual 3.9% of confirmed mislabels involve fileand path-handling weaknesses that do not fall cleanly into the six patterns and are too few to characterize as a pattern of their own. Across all six, the assigned CWE is reachable from the record’s description or end-state impact, while the evidence-supported weakness is reachable only from the

patch or mechanism—so mismatches concentrate where the taxonomy offers an easier, coarser, or differently-sourced label than the patch warrants. Finding 4: Confirmed mislabels follow six recurring patterns, led by confusing a weakness with its downstream consequence and confusing sibling sub-types within a family. All six arise from how the CWE taxonomy relates labels—cause vs. effect, sibling types—rather than from project- or language-specific mistakes. 6.3.2. Failure Effect Chain. We now ask whether the confirmed mislabels, viewed as migrations from the NVDassigned CWE to the evidence-supported CWE, concentrate into a few confusions or spread broadly across the taxonomy. Each of the 434 confirmed mislabels is a directed pair (NVD label → correct CWE); both endpoints are human-validated, so the migration is exact rather than model-estimated. Mislabeling is diffuse, not a few recurring swaps. The 434 confirmed mislabels comprise 384 distinct (NVD → correct) pairs, nearly all occurring once; the ten most frequent cover only 11.5% of the set. One pair stands out: out-of-bounds write → read (CWE-787 → CWE-125), the sub-type confusion of Case 2 (Figure 7) and the only recurring swap. Otherwise mislabels do not funnel through a few CWE-pair confusions but spread broadly across the taxonomy, not a handful of fixable pairs. Most corrections cross families. Mapping each pair to their CWE-699 category shows that mislabels rarely stay within a weakness family: 80.5% of confirmed mis-

14

NVD-assigned family

Evidence-supported family

CWE-1218 Memory buffer

CWE-137 Data neutralization CWE-465 Pointer CWE-1219 File handling CWE-399 Resource mgmt CWE-189 Numeric Other families

subset of CWEs for CVE labeling, instructing labelers to assign a more-specific child CWE where possible [2]. Despite this guidance, such labels account for over 16% of all CWE assignments in the corpus (2,536 CVEs). The five most common—CWE-20, CWE-200, CWE-400, CWE-284, and CWE-119—are abstract pillar- or class-level CWEs that can never be the most-specific label for an evidence-supported weakness. Their prevalence is a systematic departure from MITRE’s guidance, not an artifact of ambiguity: each has an S4 rate at or below the corpus average, so these labels are seldom wrong but maximally imprecise.

CWE-1218 Memory buffer CWE-199 Info mgmt CWE-465 Pointer CWE-19 Data processing CWE-137 Data neutralization CWE-1212 Authorization

Finding 6: Two labeling practices impose cost without being outright wrong: ambiguous labels (roughly a third of the corpus) that are defensible but not most-specific, and discouraged or prohibited CWEs (over 16% of assignments) that can never be most-specific. Both push the work of resolving the true weakness onto the consumer.

Other families

Figure 12: Family-level migration of confirmed mislabels.

6.5. Ecosystem Side Effects

labels migrate across families, with the evidence-supported CWE belonging to a different category than the NVD label rather than a sibling within the same one. The minority that stay in-family are dominated by the memory-safety families—the Memory-Buffer-to-Memory-Buffer band in Figure 12 (driven by the CWE-787→CWE-125 swaps) is the heaviest flow. The remaining flows fan widely: dataneutralization labels migrate to data-processing, authorization, information-management, and business-logic families; memory and pointer labels migrate outward to informationmanagement and resource families.

Beyond the population-level patterns above, C WE AGENT surfaces three rarer failure modes that we report qualitatively—as evidence that mislabeling interacts with the broader ecosystem, not as measured rates. 6.5.1. Across Label Sources. A CVE record frequently carries CWE labels from more than one source—an initial CNA label and an NVD analyst’s enrichment—and when these disagree, the record exposes a mislabel whose correct value is already present elsewhere on it: the error is a failure to reconcile sources, not a gap in information. This is the instance-level counterpart to §6.1.3, where the assigning CNA was the strongest predictor of label quality. Two confirmed cases show this directly: CVE-2024-25618 (Figure 9), where the CNA’s evidence-supported CWE-287 and NVD’s contradicted CWE-306 coexist, and CVE-202425639 (Figure 11), where an NVD-added CWE-77 persists beside the CNA’s correct XSS labels. In both, the correct label is already on the record; the mislabel is an unreconciled disagreement.

Finding 5: Mislabels are diffuse, not a few recurring swaps: nearly all CWE-pair confusions occur once, and 80.5% cross weakness families—so a mislabel usually names a fundamentally different kind of weakness than the evidence supports, not a sibling of it.

6.4. Other Bad Labeling Practices 6.4.1. Ambiguous Labels. Beyond outright inconsistency, the largest gap between NVD’s labels and a strict standard is ambiguity. We quantified its prevalence in §6.1.1 (the S2 and S3 outcomes, roughly 31% of the corpus); here we treat it as a labeling practice problem. Such a label is defensible but not the most-specific assignment the evidence supports—not wrong, yet not precise without the consumer resolving the actual weakness. These are not labeler errors but a consequence of genuine taxonomy overlap that NVD’s records do not flag. They outnumber the evidence-inconsistent mislabels of §6.3 by roughly an order of magnitude, yet still impose a cost: each shifts the work of identifying the true weakness onto the consumer.

6.5.2. Across Labeling History. A CVE’s CWE labels are not fixed at disclosure: NVD records a change history, and labels are added or revised by later enrichment. That history is an audit trail, and it sometimes shows an incorrect label entering a record after a correct one was already present— the error is introduced by a revision later. CVE-2024-42477 (Figure 13) is an example: the evidence-supported CWE125 (out-of-bounds read) was present at publication, and enrichment later added an unsupported CWE-401 (memory leak) that the record has carried since. The error is thus traceable to a specific revision rather than the original report. NVD preserves this provenance, but downstream consumers, who see the current state, do not. 6.5.3. Misinformation in Source Reports. A distinct failure mode originates one step upstream of labeling: the source report itself—the advisory or description the labeler works from—omits the mechanism, so a label faithfully

6.4.2. Discouraged and Prohibited Labels. A second, distinct practice problem is the use of CWEs that MITRE’s own mapping guidance places off-limits for direct assignment. MITRE explicitly discourages or prohibits direct use of a

15

classifiers, building datasets, prioritizing remediation, and measuring trends. Consume the noise as structured, not random. The labels are most trustworthy at the weakness-family level and least at the specific sub-type within it: §6.1.3 shows evidence-inconsistency is low and roughly flat across families while precision varies widely within them. A consumer who needs only “memory-safety issue” can rely on the label, but one needing the exact read/write operation often cannot. Because neither the imprecision nor the S4 contradictions are visible from the label alone—both require the patch or advisory to detect—research scored by exact agreement with NVD is measuring agreement with an imprecise source and should account for that imprecision rather than charging all disagreement to the method. Provenance is a free confidence signal. Label quality varies far more by the assigning CNA than by any intrinsic property of the vulnerability such as severity, and the recordlevel cases make the mechanism visible—disagreeing NISTand CNA-supplied labels where the evidence favors one, or an unsupported label added during later enrichment. Since source metadata is on every NVD entry at no cost, weighting or flagging labels by provenance is an immediate improvement over treating all as equally reliable. Patch-grounded auditing is feasible at scale. That these mismatches are detectable at all is itself a finding: grounding each candidate CWE in the fixing patch rather than the description surfaces exactly the sub-type errors, consequence-for-cause cases, and unsupported co-assigned labels that a description-driven label misses, over the full corpus. We do not argue automated auditing should replace human labeling, only that patch-grounded analysis recovers the information the mismatches turn on, and scales.

Case 7 (Label introduced by later revision): CVE-2024-42477 NVD label: CWE-401 (Memory Leak), CWE-125 (Out-of-bounds Read) NVD description. “. . . the unsafe type member in the rpc_tensor structure can cause global-buffer-overflow...” Change history. 2024-08-12 record published + CWE-125 (OOB read) 2024-08-15 enrichment analysis + CWE-401 (memory leak) 2026-04-27 later revision (no CWE change) Inconsistency. An unchecked type field lets a read run past the tensor buffer, leaking memory — an out-of-bounds read, no unreleased memory. The evidencesupported CWE-125 was present at publication; enrichment later added CWE401 (Missing Release of Memory), a resource-leak weakness the evidence does not support. The error is traceable to a specific later revision.

Figure 13: Label introduced by a later revision: CWE-125 was present at publication; an unsupported CWE-401 was added during later enrichment. derived from it is still wrong. CVE-2022-1160 (Figure 7) is a verified instance: its description reports a directionless “heap buffer overflow,” while only the AddressSanitizer output and fixing commit reveal an out-of-bounds read, so the assigned write label (CWE-787) follows from the report’s omission, not a labeler’s misreading. The remedy is upstream—descriptions that record the operation, not just the weakness class. Finding 7: Mislabeling interacts with the broader ecosystem in three ways: the correct label is sometimes already on the record from a disagreeing source, an incorrect label is sometimes introduced by a later enrichment revision, and some mislabels originate upstream in a source report that omits the mechanism. In each, the error is one of reconciliation or provenance, not missing information.

7. Discussion

7.3. Application to Mitigating Labeling Backlog

7.1. Takeaways and Implications

As a secondary application, we estimate whether C WE AGENT could help reduce the CWE-labeling component of the enrichment backlog. We do not model the full NVD process; we isolate the subtask C WE AGENT directly supports. In a small controlled study, three securityknowledgeable labelers each timed labeling CVEs sampled from our dataset across four difficulty bands, given the same evidence C WE AGENT uses (description, advisory, patch, code); the median human time was 4.2 min/CVE (mean 4.0). For each month t we treat NVD CVE publications as arrivals At and update the backlog as Bt = max(0, Bt−1 +At −Ct ), comparing one human labeler against one C WE AGENT instance under the same monthly time budget, with Ct set by the measured per-CVE times (C WE AGENT: 61.7 s; human: 251 s). Starting from the end-2025 backlog of ∼27,000 un-enriched CVEs [17] and arrivals at the 2025 average rate [14], Figure 14 shows that a single human worker cannot keep pace—its backlog grows—while one C WE AGENT instance drains the labeling backlog within a few months. This illustrates that C WE AGENT can substantially increase CWE-labeling throughput and may serve as a candidate-

The central takeaway for anyone using NVD CWE data is that the dataset has a large precision problem but only a small correctness one, and conflating the two misreads it by an order of magnitude. Most non-exact labels (the S2/S3 cases, roughly a third of the corpus) are not errors but defensible consequences of genuine taxonomy overlap— the same vulnerability admits more than one valid CWE, and NVD frequently records a broader or sibling one. Only the S4 tail (3.63%) carries labels that no reading of the evidence supports. The accurate reading keeps precision and correctness separate: NVD CWE labels are usually defensible, often imprecise, and occasionally wrong.

7.2. Implications for Using NVD CWE Data Our claims describe the records and the taxonomy, not the people or process that produce labels; we make no claim about why any individual label was assigned. The label issues nonetheless have direct consequences for the many downstream uses of NVD CWEs—training and evaluating

16

CVE–CWE mappings, such as assignments to P ROHIBITED, D ISCOURAGED, or placeholder CWEs. These works expose important metadata-quality problems, but mainly detect missing, inconsistent, or rule-invalid information visible from the record or taxonomy. We study a semantic problem: an administratively valid CWE can still contradict the advisory, patch, or code evidence, while an imprecise label may remain defensible. Detecting this requires evidencegrounded adjudication, not taxonomy checks alone. LLM agents for security analysis. LLMs have been applied to vulnerability detection, localization, repair, and security reasoning [28], [29], [30]. C WE AGENT uses LLMs for a different measurement role: it composes semantic extraction, full-catalog CWE matching, and label adjudication into an auditing instrument for NVD CWE labels.

CWE-labeling backlog

50k 40k

Human labeler CweAgent

30k 20k

drained by month 5

10k

0

0

5

10 15 Month

20

Figure 14: CWE-labeling backlog under a human labeler versus C WE AGENT. Arrivals are approximated at the 2025 average rate (3,500/month) [14] and each worker is given a 160 h monthly budget. The model isolates the CWE-labeling subtask and does not represent full NVD enrichment.

9. Conclusion

enrichment aid under backlog pressure, while still requiring human oversight for final NVD-quality enrichment.

We presented C WE AGENT, a code-semantics-grounded instrument that audits NVD CWE labels by comparing each against the weakness its advisory, patch, and code evidence supports. Measuring 15,556 open-source CVEs, we find NVD labels are usually defensible but often imprecise and only occasionally evidence-inconsistent (3.63%), with quality tracking the assigning CNA and weakness type rather than severity. Our code, data, and confirmed-mislabel set are available at https://figshare.com/s/6ba6621a69a2dac5b516.

7.4. Threats to Validity One main threat is instrument validity: the reported S4 rate depends on C WE AGENT’s judgments. We mitigate this by evaluating the Classifier and Arbitrator separately and by manually reviewing all S4 cases; the strongest claims therefore rest on the 434 manually confirmed mislabels, not automated flags alone. Evidence incompleteness is another threat: advisories and patches vary in detail, and some root causes are only partially exposed. We exclude cases C WE AGENT cannot judge from the relevant denominators. Our corpus covers open-source CVEs with locatable fixing commits, so the precision and correctness rates may not generalize to closed-source or advisory-only vulnerabilities. Because C WE AGENT uses LLMs, data leakage is also possible; we reduce this risk by using pre-CWE-assignment CVE records, Wayback snapshots when available, and a post-cutoff benchmark subset in RQ1.

References [1]

National Institute of Standards and Technology (NIST), “National Vulnerability Database (NVD),” https://nvd.nist.gov, 2025.

[2]

MITRE, “CVE → CWE “root cause mapping” guidance,” https:// cwe.mitre.org/documents/cwe usage/guidance.html.

[3]

R. Croft, M. A. Babar, and M. Kholoosi, “Data quality for software vulnerability datasets,” arXiv preprint arXiv:2301.05456, 2023.

[4]

D. Simsek, A. Eghbali, and M. Pradel, “PoCGen: Generating proofof-concept exploits for vulnerabilities in npm packages,” arXiv preprint arXiv:2506.04962, 2025.

[5]

A. Anwar, A. Abusnaina, S. Chen, F. Li, and D. Mohaisen, “Cleaning the NVD: Comprehensive quality assessment, improvements, and analyses,” TDSC, 2021.

[6]

M. Aota, H. Kanehara, M. Kubo, N. Murata, B. Sun, and T. Takahashi, “Automation of vulnerability classification from its description using machine learning,” in ISCC, 2020.

[7]

S. Pan, L. Bao, X. Xia, D. Lo, and S. Li, “Fine-grained commit-level vulnerability type prediction by cwe tree structure,” in ICSE, 2023.

[8]

MITRE, “CVE-2026-26014,” https://www.cve.org/CVERecord?id= CVE-2026-26014, 2026.

[9]

S. S. Das, E. Serra, M. Halappanavar, A. Pothen, and E. Al-Shaer, “V2W-BERT: A framework for effective hierarchical multiclass classification of software vulnerabilities,” in DSAA, 2021.

8. Related Work CWE label prediction. Prior work predicts such labels from CVE descriptions using hierarchical transformers, encoder models, and LLM-based classifiers [9], [10], [11], [18], [19], [20]. A related line predicts weakness types from code or fixing diffs, including TreeVul [7], VulExplainer [21], and hierarchy-aware classifiers [22], often addressing severe CWE class imbalance with re-weighting or margin-based losses [23], [24], [25], [26]. These systems generate likely labels. Our goal is different: to audit labels already assigned by NVD. This requires deciding whether a disagreement reflects taxonomy overlap, defensible imprecision, or evidence-inconsistent mislabeling, rather than merely predicting the most likely CWE. NVD and CWE label quality. Closest to our goal, Anwar et al.’s Cleaning the NVD [5] systematically studies NVD data-quality problems across publication dates, affected products, severity scores, and vulnerability type information. FixV2W [27] further flags administratively invalid

[10] E. Aghaei, W. Shadid, and E. Al-Shaer, “ThreatZoom: Hierarchical neural network for CVEs to CWEs classification,” in ICSPCS, 2020. [11] M. Albanese, O. Adebiyi, and F. Onovae, “CVE2CWE: Automated mapping of software vulnerabilities to weaknesses based on cve descriptions.” in SECRYPT, 2024. [12] I. Bojanova, P. E. Black, Y. Yesha, and Y. Wu, “The Bugs Framework (BF): A structured approach to express bugs,” in QRS, 2016.

17

[13] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” NeurIPS, 2022. [14] “NIST Updates NVD Operations to Address Record CVE Growth,” https://www.nist.gov/news-events/news/2026/04/ nist-updates-nvd-operations-address-record-cve-growth, 2026. [15] Cybersecurity and Infrastructure Security Agency (CISA), “Known exploited vulnerabilities catalog,” https://www.cisa. gov/known-exploited-vulnerabilities-catalog, 2026. [16] OffSec, “The Exploit Database,” https://www.exploit-db.com/, 2025. [17] DOC OIG, “Evaluation of NIST’s National Vulnerability Database Operations,” https://www.oig.doc.gov/wp-content/OIGPublications/ OIG-26-020-I-SECURED.pdf, 2026. [18] M. T. Alam, D. Bhusal, and L. Nguyen, “CTIBench: A benchmark for evaluating llms in cyber threat intelligence,” NeurIPS, 2024. [19] Meta, “Llama 3.1,” https://www.llama.com/docs/ model-cards-and-prompt-formats/llama3 1/, 2025. [20] N. Mosievskiy, “Fine-tuning RoBERTa for CVE-to-CWE classification: A 125M parameter model competitive with LLMs,” arXiv preprint arXiv:2603.14911, 2026. [21] M. Fu, V. Nguyen, C. K. Tantithamthavorn, T. Le, and D. Phung, “VulExplainer: A transformer-based hierarchical distillation for explaining vulnerability types,” TSE, 2023. [22] C. Ji, S. Yang, H. Sun, and Y. Zhang, “Applying contrastive learning to code vulnerability type classification,” in EMNLP, 2024. [23] X.-C. Wen, C. Gao, F. Luo, H. Wang, and Q. Liao, “LIVABLE: exploring long-tailed classification of software vulnerability types,” TSE, 2024. [24] K. Cao, C. Wei, A. Gaidon, N. Arechiga, and T. Ma, “Learning imbalanced datasets with label-distribution-aware margin loss,” NeurIPS, 2019. [25] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie, “Class-balanced loss based on effective number of samples,” in CVPR, 2019. [26] A. K. Menon, S. Jayasumana, A. S. Rawat, H. Jain, A. Veit, and S. Kumar, “Long-tail learning via logit adjustment,” arXiv preprint arXiv:2007.07314, 2020. [27] Ş. Şimşek, H. Xia, J. Gluck, and D. Starobinski, “Fixing invalid CVECWE mappings in threat databases,” in COMPSAC, 2025. [28] C. Zhang, H. Liu, J. Zeng, K. Yang, and H. Li, “Prompt-enhanced software vulnerability detection using ChatGPT,” in ICSE, 2024. [29] X. Zhou, T. Zhang, and D. Lo, “Large language model for vulnerability detection: Emerging results and future directions,” in ICSE, 2024. [30] C. Liang, Q. Wei, J. Du, and Z. Jiang, “Survey of source code vulnerability analysis based on deep learning,” Computers & Security, 2025.

18

Related documents

• Beyond the Total Fee: How Cybersecurity Breaches Reshape Audit Fee Composition and Trigger Strategic Auditor Switches#191238
• Actions with Receipts: Jointly Binding Claims, Evidence, and Execution for Replayable Tool-Agent Auditing#1154950
• Auditing large language models: a three-layered approach#335808
• The Structural Audit of Strong Universal Claims: An Evoluist Theorem of Five-Checkpoint Completeness#691974
• A Matched-Budget Audit Framework for Recaptioned Image-Text Supervision Distributions#1155887