arXiv CSopen access

The Cathedral and the Bazaar of Software Vulnerabilities: From the NVD to the CNAs

1

The Cathedral and the Bazaar of Software Vulnerabilities: From the NVD to the CNAs

arXiv:2607.05670v1 [cs.SE] 6 Jul 2026

Siqi Zhang, Fabio Massacci, and Mengyuan Zhang

Index Terms—Vulnerability Scoring System, CNAs, NVD, Patch Management, Inconsistency

I. I NTRODUCTION Eric Raymond [1] introduced the word bazaar in 1998 to positively describe distributed and collaborative software development for Linux. The code is developed by the public as opposed to the cathedral, a single authoritative source. For decades, the National Vulnerability Database (NVD) has served as the software vulnerability cathedral, the source for severity assessment. Many governmental mandates (e.g., US Executive Orders), industry regulations (e.g., Credit Cards’ PCI Security), and many research papers, such as vulnerability prioritization systems [2], security posture monitoring tools [3], and security metric frameworks [4], rely on NVD data as a foundational input. Correctness and consistency of these data directly affect the reliability of downstream services (See detailed list in Table VIII). Siqi Zhang, Fabio Massacci, and Mengyuan Zhang are with Vrije Universiteit Amsterdam, Amsterdam, The Netherlands. E-mail: {s.zhang4,f.massacci,m.zhang}@vu.nl. Fabio Massacci is also with University of Trento, Trento, Italy. E-mail: [email protected].

# of CNAs

150 100 50 0

5000 Number of CVE Entrie

200 # of CNAs

Abstract—For decades, the National Vulnerability Database (NVD), the “Cathedral”, has been the reference source for vulnerability information for downstream research and industry tasks, e.g., software update prioritization. An emerging “Bazaar” of diverse CVE Numbering Authorities (CNAs) has created many alternative and sometimes diverging sources. We conduct a systematic analysis of divergence in Common Vulnerability Scoring System (CVSS) metrics covering the NVD and the public CNAs. We also check for self-divergence: two identical textual descriptions of CVEs with identical CWEs are rated differently by the same CNA. The odds of diverging are widespread, not uniform and sometimes unexpected. The assessment of Attack Complexity, User Interaction, and Impact are the major metrics where divergence happens. To understand the root causes, we perform a qualitative study by reaching out to the NVD and other CNAs (both open sources and proprietary products). We also discussed the findings at the CVSS Special Interest Group of FIRST, the community responsible for maintaining and evolving the CVSS standard. The key insights are that while something might be due to human errors, in some cases diverging is actually the right thing to do and might require changes in the way CVEs are generated industry-wide, in other cases explaining divergence requires access to additional FAQs. The good news is that the situation is improving since 2025, the bad news is that if one downloads the whole NVD (or another CNA dataset) from several years and uses it for predictions, the models trained on one source do not reliably generalize to a different source (accuracy can drop by 40%). We discuss the implications for practice and research.

4000

10

CNA Critical NVD Critical

3000

5

2000

0

2000

CNA Higher NVD Higher Same le#el

1000

2005

2010 2015 Year

2020

2025

(a) # of CNAs release CVE per year

0

10 20 30 40 50 CNAs Ranked by # of Common CVE Entrie

(b) Severity Levels

Fig. 1: The growth of CNAs and severity disagreements

Vulnerabilities boomed and, to scale, the CVE program introduced the CVE Numbering Authority (CNA) framework, authorizing registered CNAs (e.g., Microsoft) to independently create CVE entries and assign severity assessments. Fig. 1a shows the growth in the number of CNAs following the CVE program’s decision in 2016 [5]. As of 2024, CNAs have issued a substantial portion of the total CVEs. The bazaar of software vulnerability assessment was born and is now thriving. Fig. 1b illustrates the differences in severity scored by the NVD and the severity of the same vulnerability scored by a CNA. To decide vulnerability prioritization, researchers, companies and governments have now multiple sources. Unfortunately, they are not always consistent. Divergence issues within the NVD have been studied from various perspectives, including discrepancies between severity scores and vulnerability types [6], mismatches in software names and versions [7], and differences between organizational assessments [8], or even within CNAs themselves in terms of CVE entries with identical [9]. Divergence has been also discussed in blogs [10] [11]. Coutinho et al. [12] reveal discrepancies between aggregated CNA-assigned entries and NVD-maintained entries; their analysis was limited in scope, covering fewer than five CNAs. A systematic and formal study comparing all CNA-assigned entries (and not just the NVD) at a fine-grained level remains missing. In this paper, we take the first step toward formally modeling and quantifying divergence between these sources, for the purpose of evaluating the reliability of vulnerability severity data. To understand the root causes, we perform a qualitative study by reaching out to the NVD and other CNAs (both open sources and proprietary products). We also discussed the findings at the CVSS Special Interest Group (SIG) of FIRST. Our main contributions are as follows: (i) We introduce a unified framework to (a) quantify external divergence: how CNA-assigned CVSS vectors differ from

2

NVD at the vector and per-metric levels (73% of public CNAs have at least one diverging assessment from the NVD); and (b) assess self-divergent by grouping CVEs with identical descriptions (description-based analysis). (ii) We constructed an up-to-date dataset for this analysis, covering all CNAs and the NVD from 1999–2025, and augmented it with additional features (e.g., CNA type, release/update history, Common Weakness Enumeration (CWE), and Common Platform Enumeration (CPE)). (iii) We uncover what type of CNA (e.g., a Vendor vs. a Bug Bounty Provider), the release history, makes a significant difference in both the chance and the direction of divergence, and on which base metrics (Impact, Attack Complexity, and User Interaction) such divergence is most pronounced. (iv) We provide the first empirically grounded root-cause analysis of CVSS divergence by engaging with CNAs, NVD, and the CVSS Special Interest Group (SIG). Through presentations, feedback exchanges, and interview with NVD, we identify eight root causes of divergence. The SIG is currently in the process of extending its FAQs to explicitly discuss scoring differences. (v) We found improvements in both CVSS divergence and description quality after 2025, corresponding to industry changes in reporting. We complement these results with case studies based on implemented attacks and vulnerability discussions. (vi) We further discuss how the impact of our findings on downstream use of the data. For example, automated, machine learning approaches that just pick up a dataset might not be learning severity but just the risk appetite of the particular CNAs or the NVD. II. T ERMINOLOGY AND M OTIVATION Common Vulnerabilities and Exposures (CVE). CVE is a public system for identifying and cataloging cybersecurity vulnerabilities. Each CVE entry includes a unique identifier (CVE ID), a brief description, and references to related reports and advisories. CVE records are contributed by various sources [13]. CVE Program. As of mid-2025, the CVE Program includes 458 CNAs across 40 countries [14]. The program categorizes CNAs into seven organizational types: Vendor, manages vulnerabilities in its own products or services; Researcher, conducts security research to identify and disclose vulnerabilities suitable for CVE tracking; Open Source, maintains software whose source code is publicly accessible and modifiable; CERT, Computer Emergency Response Team; Hosted Service, refers to cloud-based services such as PaaS, IaaS, and SaaS platforms; Bug Bounty Providers, intermediary platforms that connect vendors with researchers, often offering incentives for valid vulnerability reports; Consortium, a group of entities joined together to work on a particular project. CVE Numbering Authority (CNA). A CNA is responsible for assigning CVE IDs, publishing initial vulnerability records that describe the vulnerabilities, and may optionally include additional information such as severity scores (e.g.,

CVSS), affected products, and references. A special CNA, the Cybersecurity and Infrastructure Security Agency (CISA) serves as both a CNA and the only designated Authorized Data Publisher (ADP) [15]. Common Vulnerability Scoring System(CVSS). The CVSS [16] is an open framework and widely recognized standard for assessing the characteristics and severity of vulnerabilities. The base metrics are typically used to calculate the severity score and determine the severity level of a given vulnerability. CVSS v3.1 includes eight base metrics: Attack Vector (AV), Attack Complexity (AC), Privileges Required (PR), User Interaction (UI), Scope (S), Confidentiality (C), Integrity (I), and Availability (A). Once all of them are evaluated, they are combinedCVE-2024-21326 into a CVSS Vector using a stan9.6 (Critical) Vulnerability Description: CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H dardized syntax. The CVSS framework computes a severity Microsoft Edge (Chromium-based) CVE-2024-43472 5.8 (Medium) elevation of privilege score from 0.0 tovulnerability. 10.0 based on base metric values, which is CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:C/C:L/I:L/A:L mapped to a qualitative level. CVE-ID: CVE-2023-7063 The WPForms … is vulnerable to Stored Cross-Site Scripting … attackers to inject arbitrary web scripts in pages that will execute whenever a user accesses an injected page.

Wordfence (CNA) 7.1 (High) CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/ C:L/I:L/A:N 6.1 (Medium) NVD CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/ C:L/I:L/A:N

(a) Cross-source disagreement on the same CVE Vulnerability Description: Microsoft Edge (Chromium-based) elevation of privilege vulnerability.

CVE-2024-21326 9.6 (Critical) CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H 5.8 (Medium) CVE-2024-43472 CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:C/C:L/I:L/A:L

(b) Self-divergent scores, same CNA, same description

Fig. 2:CVE-2023-7063 Motivating examplesWordfence of CVSS divergent (CNA) CVE-ID: 7.1 (High)scores The WPForms … is vulnerable to CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/ C:L/I:L/A:N Stored Cross-Site Scripting … attackers to inject arbitrary web scripts NVD (Medium) Motivating Example. Fig. 2 highlights two6.1 key problems: in pages that will execute whenever CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/ (1)a divergence between CNAs and the NVD, and (2) selfuser accesses an injected page. C:L/I:L/A:N divergence within identical contexts even within a single source. In Fig. 2a, for CVE-2023-7063, Wordfence and the NVD diverge on the UI base metric, resulting in different severity levels. Fig. 2b shows that divergence arises even within individual CNA (e.g., Microsoft) itself. Two CVEs with the identical description contain four different vectors. Consequently, divergent CVSS assignments, “Critical” and “Medium”, may result in different patching prioritization, thereby extending the attack window and undermining overall security. Despite the potential impact, there is currently no guidance on how to systematically quantify, interpret, or reconcile these differences. III. M ETHODOLOGY In this section, we formally model discrepancies between CNAs and the NVD at two levels; Vector-level divergency (§III-A) and Description-level divergency (§III-B). We first formally define vulnerability assessment and descriptions. We denote a CVE identifier as e, possibly with subscript i, j. We denote the corresponding vulnerability assessment by a CNA C (e.g., CVSS base metrics values, CVSS scores, severity levels) as the function: vC (e)

=

C (v1C (e) , . . . , vm (e))

(1)

3

where m is the number of assessment dimension of CVSS base metrics (e.g., m = 8 for CVSS v3.1) and vkC (e) is the value of the individual base metric of the vulnerability assessment (e.g., for CVSS v3.1, the base metric for k = 1 is the attack vector AV and the value can be N for network). Each CVE e is associated by a CNA C with a textual description tC (e).

based on CNAs, however, the NVD shares the same concept except changing the notations to the ∗NVD . Definition 4 (Textual Equivalence). Let C be a CNA and CVEC be the set of CVE entries published by C, an equivalence relation ∼C over CVEC is induced by the textual description of C as follows e ∼C e∗

A. Vector-level Divergency An e is considered as a divergent entry if its vulnerability assessment differs between two data sources. Definition 1 (Divergent Entry). Let C be a CNA, let CVEC be the set of CVE entries published by C, and let CVENVD be the set of CVE entries reported by the NVD. A CVE entry e ∈ CVEC is a divergent entry if there exists an entry e ∈ CVENVD such that the vulnerability assessment differs vC (e) ̸= vNVD (e). We propose two complementary metrics to quantify divergent degrees: the vector-Divergency, and the individual metric-Divergency, which further breaks down Divergency across individual base metrics. The first vectorDivergency metric calculates the distance between two CVSS vectors by adopting the concept of Hamming distance, originally introduced to measure the number of bit errors in message transmission over a noisy channel. In our context, each base metric (e.g., Attack Vector) contributes 1 to the total distance if the values differ, otherwise 0. Definition 2 (vector-Divergency). Let ed be a divergent entry between the CNA C and the NVD then, the vector divergent metric is defined as the Hamming distance between the two assessments with m individual metrics: m X vectored = 1vC (e)̸=vNVD (e) (2) k

Since Definition 2 does not reveal which specific base metrics are more prone to diverge. We therefore propose the individual metric-Divergency to capture this aspect. Definition 3 (individual metric-Divergency). Let C be a CNA, let CVEC be the set of CVE entries published by C, and CVENVD be the set of CVE entries reported by the NVD, and let D = CVEC ∩ CVENVD , the individual metric-Divergency between the CNA C and the NVD on the assessment metric k ∈ {1, . . . , m} is defined as: =

X

1vC (e)̸=vNVD (e) k

k

(3)

e∈D

The individual metric-Divergency represents absolute difference of overlapping CVE entries for which CNA C and 1 ∗ the NVD disagree on the k-th CVSS metric, and |D| individual metrick (C) presents the ratio. B. Description-level Divergency After studying the divergences for individual CVE entries rated by different sources, we address the second issue mentioned in the motivating example, i.e., how identical descriptions are rated in both CNAs and the NVD. The definitions are

tC (e) = tC (e∗ )

(4)

C

This relation partitions CVE into equivalence classes G C = {G1 , G2 , . . . , Gk }, where each group G ∈ G C contains entries that share an identical textual description. A group can however be inconsistent due to internal disagreement in the assessment of identically described entries. Definition 5 (Self-Divergent Group). Let C be a CNA, let CVEC be the set of CVE entries by C, and let G C be the partition ∼C induced by the identical textual description from the CNA C, a group G ∈ G C is self-divergent if there exist two entries e, e∗ ∈ G such that their assigned metric vectors differ, vC (e) ̸= vC (e∗ ). We denote the set of all divergent description groups for a CNA C as DC , we also define the consistent group as the set of CVEs which has at least two CVEs but where all descriptions have the same text and the same CVSS metric vector C C . DC =



G ∈ G C | ∃e, e∗ ∈ G : e ̸= e∗ ∧ vC (e) ̸= vC (e∗ )

(5)

C



G ∈ G C | ∃e, e∗ ∈ G : e ̸= e∗ ∧ G ̸∈ DC

(6)

C =

Definition 6 (Odds Ratio of the Group). Let ∼C and ∼NVD be the equivalence relations induced by the textual descriptions provided by the CNA C and the NVD, divergency odds ratio for a CNA C relative to the NVD is then defined as:

k

k=1

individual metrick (C)

if and only if

ORC/NVD =

|DC | / |C C | |DNVD | / |C NVD |

(7)

An OR greater than 1 indicates that the CNA exhibits a higher rate of divergency relative to the NVD, while an OR less than 1 suggests otherwise. C. Root Cause Qualitative Analysis To understand the root causes of divergence, we also perform a qualitative analysis. Since the CVSS Special Interest Group (SIG) has recently discussed CVSS inconsistent scoring practices, we contacted the SIG chair, several CNA representatives, and the NVD with divergent cases from our analysis. To preserve confidentiality, we anonymize the identities and organizational affiliations of the contacted CNA representatives throughout the paper. We contacted the NVD and six CNA representatives who had either participated in the CVSS SIG discussions or whose organizations exhibited significant divergence in our analysis. We informed them that we were investigating the issues discussed during the meeting, shared representative observations from our preliminary analysis, and asked whether they would be interested in reviewing our findings. Before presenting our work at the subsequent SIG meeting, we offered to share both the presentation slides and the CNA-specific analysis so that

4

their comments could be incorporated into the discussion. The invitation email is provided in Appendix Fig. 10. After the NVD and the CNA representatives expressed interest in our findings, we sent a follow-up email containing our initial presentation slides and CNA-specific files containing the associated CVE cases. The slides summarized our preliminary findings presented in Section V, together with illustrative examples and the corresponding analysis results. Based on these materials, the participants reviewed our findings and provided feedback on the possible reasons for the observed inconsistencies. Several explanations were independently raised by multiple organizations, which we grouped into eight recurring root causes (RCs). We then incorporated these preliminary RCs into the presentation and discussed them at the subsequent CVSS SIG meeting, where an additional seven experts provided further comments and refinements. The follow-up email is provided in Appendix Fig. 11. IV. DATASET We compile the most complete CNA-NVD dataset (available in the Artifact) by combining all available CVEs from VulnCheck [17] (covering NVD [18] and MITRE CVE List [19]). It enables large-scale, reproducible analysis of cross-source CVSS divergencies. Data Pre-processing. We excluded entries whose descriptions begin with “UNSUPPORTED WHEN ASSIGNED”, as this tag indicates that all affected products or versions were already end-of-life (EOL) or no longer supported by the vendor when the CVE was assigned, and therefore fall outside the scope of our study. The resulting dataset contains 191,009 CVEs with CVSS v3.1 vectors as of January 2025, including 72,122 CNA entries and 118,887 NVD entries. To ensure consistent organization-level analysis, we further normalize CNA identities. When an organization is registered as a single CNA, we consolidate all entries associated with that organization (e.g., [email protected] and [email protected] are mapped to Cisco). When an organization operates multiple CNAs for different product lines or programs, we keep them separate, e.g., Samsung Mobile and Samsung TV are treated as distinct CNAs, and CISA is represented by three CNAs: CISA, ICS-CERT, and CISA-ADP. The entries containing scores from both CNA and NVD are denoted as Pairwise setting. According to our interviews with NVD analysts, a CNA-only CVSS score may be perceived as agreed upon by NVD when no alternative NVD score is provided, and the entry has “Analyzed” status. Since consumers often retrieve CVSS data and use it directly, we treat all CNAonly CVSS scores as mutually agreed-upon entries, i.e., with vector-Divergency = 0. We denote this assumption as the Consumer-View setting. In total, we obtained 44,180 CVEs under the Pairwise setting, i.e., CVEs rated by both NVD and a CNA, and 72,122 CVEs under the Consumer-View setting, which additionally includes CNA-only cases by assuming that NVD assigns the same CVSS vector. Root Cause Feature Collection. To investigate potential drivers of CNA–NVD divergent, we first annotate each CVE with the CNA’s organization type [20] (Vendor, Researcher,

Fig. 3: Distribution of CNAs by vector-Disagreement values under the Pairwise and Consumer-View settings

Open Source, CERT, Hosted Service, Bug Bounty Provider detailed in Table II). To analyze temporal precedence (who acts first) and follow-on behavior, we reconstruct scoring timelines from JSON metadata exposed by both the NVD and the CVE Program. Specifically, we use the NVD CVE Change History API1 and the CVE Program API2 . The NVD API provides event-level histories with action timestamps and sources; the CVE Program API exposes CNA-side metadata. By aligning timestamps across the two APIs, we derive (i) the first appearance of a CVSS v3.1 assessment per CVE and source (“who-first”), and (ii) an complete, ordered sequence of subsequent updates. We then study whether, and within what horizon, one source follows another, conditioning on CNA type and text-quality features. In totally, we collected 44,123 CVE entries. V. D IVERGENCE A NALYSIS In this section, we evaluate vector-level divergency between CNA and NVD assessments using both vector and individual metric divergency in Section V-A, then we continue with description-level in Section V-B. Because CNA coverage varies widely (from ∼ 102 to ∼ 104 CVEs), we report individual metric–Divergency as a percentage to enable fair comparisons across CNAs; the corresponding absolute counts are easily recoverable (provided in the Appendix Table IX). A. Vector-level Divergency Overall Analysis. Fig. 3 summarizes CNA–NVD disagreement using vector-Divergency. Both Pairwise and ConsumerView settings show 194/266 (73%) and 139/288 (48%) of CNAs have a median of at least 1 vector divergent, which suggests divergent is fairly common between CNAs and NVD in both views. vector-Divergency Analysis. We compute per CVE vector distance for every CNA–NVD pair that under ConsumerView perspective. We then examine its distribution for the selected CNAs to characterize their divergent patterns. Fig. 4a summarizes the results: each violin shows the density of distances (wider segments indicate higher mass), with the median marked (color/shade) and interquartile range overlaid. This view highlights both where disagreement typically concentrates and how it varies across CNAs (spread, skew, and outliers). For readability, we only provide the Top 20 CNAs, 1 https://services.nvd.nist.gov/rest/json/cvehistory/2.0 2 https://cveawg.mitre.org/api/cve/

5

TABLE I: Dataset Summary of Pairwise and Consumer-View (1999-2025.01) Analysis Setting Pairwise Consumer-View

# CVE entries

# CNAs

191,009

288

# CNAs overlap NVD 266 288

TABLE II: CNA Organization Types Organization Types V R OS CERT HS BBP #CNA 214 61 68 14 13 4 #CVE 29,821 20,896 15,961 2,519 6,060 3,692 Note: V = Vendor; R = Researcher; OS = Open Source; CERT = Computer Emergency Response Team; HS = Hosted Service; BBP = Bug Bounty Provider

which account for 81% of the data. Full results for all CNAs are included in the artifacts. All the top 20 CNAs contain divergent entries. VulDB shows the highest divergence, with a median value of 3, followed by Samsung Mobile with a median value of 2. In contrast, Oracle shows the strongest agreement with the NVD.

d-Disagr m nt M tric

Takeaway 1. Even a single metric difference may change the overall severity level and impact patch prioritization. Divergency is substantially higher under the Pairwise setting (73%) than under Consumer-View (48%), and remains common across major CNAs, with half of the top 20 exhibiting median disagreement ≥ 1.

M dian = 0

M dian = 1

M dian = 2

M dian = 3

8 6 4 2 0

ll Ab t l at ns yk DP ck -b DB ce oft sco cle be RT RE BM m il A-A hsta GitH V-lordfen icros Ci Ora AdoCS-CE MIT I alcomg Mob D GitL In R d HSi m Sn M I CIS Patc W Q- s-n Sam

(a) Distribution of vector-level divergent values AV 0.03 0.00 0.04 0.08 0.00 0.02 0.02 0.00 0.00 0.08 0.17 0.05 0.02 0.10 0.04 0.04 0.00 0.02 0.04 0.02 AC 0.02 0.01 0.19 0.02 0.03 0.04 0.05 0.00 0.01 0.08 0.13 0.20 0.03 0.16 0.22 0.23 0.36 0.11 0.07 0.19 PR 0.05 0.02 0.13 0.39 0.01 0.04 0.06 0.00 0.01 0.12 0.13 0.21 0.38 0.48 0.10 0.16 0.07 0.11 0.08 0.07 UI 0.03 0.01 0.11 0.02 0.18 0.03 0.04 0.00 0.00 0.10 0.11 0.04 0.00 0.14 0.11 0.14 0.28 0.05 0.03 0.05 S 0.01 0.04 0.19 0.19 0.03 0.02 0.10 0.00 0.02 0.16 0.30 0.05 0.05 0.11 0.16 0.13 0.22 0.06 0.07 0.09 C 0.03 0.14 0.18 0.66 0.04 0.04 0.11 0.00 0.02 0.17 0.16 0.12 0.07 0.27 0.18 0.20 0.12 0.06 0.08 0.23 I 0.03 0.15 0.19 0.46 0.04 0.04 0.09 0.00 0.02 0.18 0.19 0.10 0.06 0.30 0.16 0.19 0.09 0.08 0.06 0.23 A 0.03 0.35 0.20 0.49 0.05 0.04 0.14 0.00 0.02 0.16 0.19 0.09 0.13 0.34 0.23 0.17 0.13 0.07 0.10 0.28 B ce -ft c- cle be RT RE IBM mm bile Dell LAb ntel H t ens nyk DP ck ub I ed S - Mo m A-A hsta GitH VulD-rdfen icr-s Cis Ora Ad- CS-CE MIT Git R alc Sie M I CIS Patc W Qu sung Sam

0.6

0.4

0.2

0.0

(b) Proportion of individual metric-Divergency

Fig. 4: Top 20 CNAs compared to the NVD individual metric-Divergency Analysis. The vectorDivergency provides a high-level overview of discrepancies between CNAs and the NVD. For more granular analysis, we use individual metric-Divergency to identify which CVSS base metrics drive the observed divergent. The notable divergent individual metrics in VulDB are Impact metrics (C/I/A) as shown in Fig. 4b. PR metric is also the main cause for such a divergence. These permetric patterns explain the concentration at vector = 3 and vector = 4 in Fig. 4a: divergences in Impact and PR are major contributors to the overall divergent. Samsung Mobile and Snyk show similarly elevated divergency across all Impact metrics. Samsung Mobile also exhibits relatively high divergency in the PR metric, and similar patterns are

Comparable CVEs 44,180 72,122

20 CNAs pairs 33,831 58,249

Change history 44,123 –

observed for CNAs such as Qualcomm. As a conclusion, the impact-related metrics (C/I/A) and PR are common sources of disagreement across Top 20 CNAs. Takeaway 2. Certain metrics (e.g., PR and Impact) show higher divergence rates across top 20 CNAs, suggesting inherent ambiguity in these metrics and the need for clearer internal agreement for specific CNAs. B. Description-Level Divergency To analyze description-level divergence, we first match descriptions that appear in both a CNA and the NVD. For any description with at least two CVEs in a source, we form a group (per source). A group is considered convergent if all CVSS vectors are identical; otherwise, divergent. During the interview, CNA analysts explained that identical descriptions may result from the description templates. Further augment with Common Weakness Enumeration (CWE) and/or affected platforms, as represented by Common Platform Enumeration (CPE) identifiers, may help explain the observed divergence. Thus, we refine our grouping by additionally requiring that CVEs within a group share the same CWE and CPE attributes. Specifically, refined groups are defined by the tuple of description, CWE, CPE vendor, and CPE product. CWE values are grouped by exact match, with missing values treated as a distinct category and handled equivalently to other CWE values during grouping. Table III compares the selfdivergence of CNA and NVD under both the Description-only and refined Description+CWE+CPE settings. 3 Odds Ratio Analysis. The odds ratio is calculated according to Equation 7. Table III compares the self-divergence of CNA and NVD under both the Description-only and the refined Description+CWE+CPE settings. Overall, incorporating CWE and CPE significantly reduces the number of inconsistent groups for both CNA and NVD across all selected sources. For example, Microsoft shows a substantial decrease in divergent groups from 207 to 75 (CNA) and from 160 to 62 (NVD), indicating that a large portion of the apparent inconsistency under description-only grouping can be attributed to differences in vulnerability types or affected products. Despite this reduction, the relative divergency between CNA and NVD, as reflected by the odds ratio (OR), remains largely stable for most sources. OR values greater than 1 indicate higher divergencies in CNA compared to NVD, vice versa. For instance, Microsoft (2.02 → 1.40) and CISA-ADP (6.64 → 3.5) continue to exhibit higher divergency in CNA compared to NVD, while Cisco (0.08 → 0.38) remains more converged than the NVD. Dell shows near-perfect convergent within these 3 For statistical reliability, we focus on the 7 CNAs that have ≥ 10 comparable groups under both Description-only and Description+CWE+CPE settings, ensuring sufficiently stable comparisons, as small sample sizes can lead to unreliable estimates [21], [22].

6

TABLE III: Description-level analysis of self-divergence and convergent groups between selected CNAs and NVD. Values are shown as Description-only (Description+CWE+CPE). |DC | |C C | |DNVD | |C NVD | ORC/N V D Description-only (Description+CWE+CPE) Microsoft 207 (75) 84 (84) 160 (62) 131 (97) 2.02 (1.40) CISA-ADP 28 (9) 52 (36) 6 (3) 74 (42) 6.64 (3.5) OpenHarmony 3 (2) 10 (8) 2 (1) 11 (9) 1.65 (2.25) 9 (3) 19 (11) 6 (0) 22 (14) 1.74 (U) Dell Adobe 7 (1) 73 (11) 7 (1) 73 (11) 1 (1) Cisco 8 (4) 130 (37) 61 (9) 77 (32) 0.08 (0.38) Oracle 0 (0) 27 (20) 0 (0) 27 (20) U (U) Note: U denotes undefined odds ratios, which occur when the denominator in the odds ratio computation is zero. Source

TABLE IV: Logistic and Linear Regression Results Log. N V D ̸= CN A Lin. CN A − N V D Coef. Sig. Coef. Sig. Constant -1.157 *** -1.477 *** Vendor 0.653 *** 1.187 *** Open Source -0.265 *** 0.091 Researcher 2.286 *** -1.108 *** Bug Bounty Provider 0.495 0.055 -1.544 *** 0.445 ** Hosted Service CERT 0.577 *** 1.221 *** Who First 2.072 *** -0.010 CWE 0.159 *** -0.036 Description Length -0.002 *** 0.002 *** Who First × V -1.080 *** −2.0 × 10−4 Who First × OS 0.211 *** -0.362 *** Who First × R -1.867 *** 0.770 *** Who First × BBP 1.935 *** 0.191 Who First × HS 0.383 ** -0.427 ** Who First × CERT -1.266 *** -0.081 Significance levels: ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. Non-significant results (p ≥ 0.05) are left blank. Note: Logistic regression uses 43,997 entries (dependent variable V ectorN V D (cve) ̸= V ectorCN A (cve)), pseudo-R2 = 0.117. Linear regression uses only the 26,931 conflicting entries (dependent variable ScoreCN A (cve) − ScoreN V D (cve)), R2 = 0.118, F (15, 26,931) = 240.5. Selected Factors

refined groups. And Oracle and Adobe show no difference between the two settings. Takeaway 3. Refining grouping from Description-only to Description+CWE+CPE substantially reduces divergent groups, while the relative consistency between CNAs and NVD remains largely stable. This highlights the need to include additional attributes to identify internal consistency; odds ratios help identify more consistent sources for patch prioritization.

VI. K EY FACTORS IN D IVERGENCY We conducted an experiment to investigate which factors influence divergent vulnerability ratings, particularly focusing on vector-level and individual metric-level divergences. A. Factor Selection and Analysis We first applied a logistic regression analysis to identify the factors influencing scoring divergency, followed by a linear regression to examine the magnitude and direction of

how these factors affect the CVSS base score differences under the Pairwise setting. The dependent variable of the logistic regression indicates whether a CNA-assigned score differs from that of the NVD (vector-Divergency > 0), while the dependent variable of the linear regression represents the numerical difference in CVSS base scores between the CNA and the NVD, considering only conflicting CVE entries (Hamming distance > 0 and CNA-assigned score ̸= NVDassigned score). Both analyses share the same independent variables, including six CNA organization types [20] (Vendor, Researcher, Open Source, CERT, Hosted Service, Bug Bounty Provider). We excluded organizational type consortium as it has no entries in our dataset. In addition, we incorporate three quantitative factors: the order of CVSS assignment between the CNA and the NVD (i.e., whether the CNA or the NVD published the CVSS vector first), the description length of each CVE, and the CWE. Furthermore, six interaction variables are included to capture whether the effect of being the first to publish a CVSS score differs across organization types. Organizational Types. To understand the practical impact in the probability of divergent due to a particular factor, we use the following equation reverting the regression equation: pX = P r{VN V D ̸= VCN A |type(CN A) ̸= X} pX = P r{VN V D ̸= VCN A |type(CN A) = X} pX pX eβX = 1 − pX 1 − pX

(8) (9) (10)

where eβX is the odds ratio associated with CNA type X, derived from the logistic regression coefficient βX . For example, the logistic (Log.) regression in Table IV reports a coefficient of +0.653 for the Vendor. Exponentiating this coefficient yields an odds ratio of e0.653 ≈ 1.9, indicating that, after controlling for other factors, the odds that a Vendor CNA disagrees with the NVD are approximately 1.9× higher than the odds for non-Vendor CNAs. In other words, Vendor CNAs exhibit a substantially higher likelihood of producing CVSS scores that divergent from the NVD, relative to CNAs from other organizational types. Logistic regression in Table IV shows that CNA type is significantly associated with the likelihood of rating discrepancies compared to NVD: Open Source (e−0.265 ≈ 0.8) and Hosted Service (e−1.544 ≈ 0.2) are significantly less likely to produce divergent scores compared to other CNA types, indicating closer alignment with centralized NVD scoring practices. On the other hand, the odds of divergent with the NVD relative to agreeing with it are higher for Vendor (1.9×), Researchers (9.8×), and CERT (1.8×), compared to all other CNA types. This pattern suggests that these CNA types tend to follow more subjective evaluation practices, leading to greater variability in their scoring outcomes. When Vendor and CERT disagree with NVD, they tend to assign higher severity scores than the NVD, with a gap of 1.2/10 scores. These values might be enough to toggle the qualitative class in which the vulnerability is classified in terms of prioritization. In contrast, Researchers tend to assign lower scores than the NVD, with negative coefficients -1.1/10.

7

Takeaway 4. Structured actors (Vendor, Researcher, and CERT) show higher divergency; decentralized actors (Open Source, Hosted Service) align more with NVD, reflecting differences in access to internal information, organizational context, and risk modeling practices. First-Mover Effect. We study temporal dependence by determining Who First for each CVE. When CNA timing is missing in the CVE Program, we compare the earliest CNA appearance in the NVD history to the NVD’s first CVSS date; when CNA timing is missing in the NVD history, we compare the NVD’s first CVSS date to the CNA’s timestamps from the CVE Program. If both are present, we take the earlier of (CVE Program CNA update, CNA appearance in NVD) and compare it to the NVD’s first CVSS date. We label the outcome as CNA-First, NVD-First. To compare all cases, we encode the variable as a binary indicator, assigning 1 to “CNA First” and 0 to “NVD First”, and included it as an independent variable in our regression analysis. As shown in Table IV, the logistic regression result reveals a positive and statistically significant coefficient (2.072, ***), meaning that the odds of CNAs and NVD disagreement are about 8× higher when the CNA publishes its CVSS score first than when the NVD publishes first. Takeaway 5. Assessment order significantly affects divergency, with CNA-first assessments generally increasing disagreement. This effect is mitigated for structured actors and amplified for decentralized actors, suggesting differences in information availability and assessment context. CWE Effect. We introduce a binary variable, CWE, where a value of 0 indicates that both the CNA and NVD assign the same CWE value, while a value of 1 captures all other cases, including mismatched CWE assignments or missing values in either source. Entries labeled as NVD-CWE-noinfo and NVD-CWE-Other are treated as missing values. The logistic regression shows a statistically significant positive effect for CWE, suggesting that differences in CWE assignments may partly explain, and potentially constitute an underlying source of, CVSS score divergence between CNAs and NVD. Takeaway 6. Divergency is significantly more likely when CWE assignments are mismatched or missing between CNAs and NVD, suggesting that divergent/missing CWE information may introduce ambiguity into assessment. Description Length. We also incorporate description word count as a predictor. The results show a statistically significant but practically negligible effect: e−0.002 ≈ 0.998, indicating an approximately 0.2% reduction in the odds of divergence per additional word. This aligns with interview results from NVD analysts: when key information is missing, they may need to score vulnerabilities based on a worst-case interpretation. Longer descriptions may therefore reduce divergence by providing additional contextual information. Assessment Order across Organization Types. While the previous analyses reveal which organization types are more

likely to diverge or converge with the NVD, and that overall, when CNAs rate first, the NVD tends to diverge with them, these effects may not be uniform across all organizations. Therefore, we introduce interaction terms between Who First and each CNA type (Who First × {Organization Type}) to capture potential differences in how early assignment behavior influences divergency across organizations. The logistic regression result in Table IV includes interaction terms between the CNA type and the Who First variable (1 = CNA first, 0 = NVD first). These terms capture whether publishing earlier changes the likelihood of a scoring divergence for a given CNA type. A negative interaction coefficient means that, for this CNA type, publishing first reduces the odds of divergence relative to other CNA types. In other words, when the CNA publishes first, the NVD becomes more likely to converge with its assessment. A positive interaction coefficient, in contrast, indicates that publishing first increases the odds of divergence for that CNA type. Equivalently, when the NVD publishes first, the odds of divergence for that CNA type decrease, meaning that its scores tend to be more aligned with the NVD when the NVD leads the scoring process. Bug Bounty Provider follows the overall trend that when CNAs rate first, the NVD is more likely to diverge. Both the main effect for Bug Bounty Provider and the Who First are positive, and their interaction term is also positive (+1.935), indicating a strong amplification effect. Specifically, when Bug Bounty Provider publishes first, the likelihood of divergence increases substantially compared to the baseline case. For Vendor, Researcher, and CERT, the main logistic effects are positive, meaning these CNA types are generally more prone to diverge with NVD scores. However, their interaction coefficients with Who First are negative, meaning that when these CNAs publish their scores before the NVD, the NVD is more likely to converge with them. The linear regression gives additional insight; the interaction is not statistically significant for Vendor and CERT, whereas for Researcher, when divergence occurs, they tend to assign scores about 0.8/10 points higher than the NVD. Open Source and Hosted Service exhibit negative main effects, indicating greater convergence with the NVD. However, their positive interaction with Who First suggests that when they publish first, the likelihood of divergence increases. The linear model further shows that, in cases of divergent, NVD scores tend to be about −0.4/10 points lower than those assigned by Open Source and Hosted Service. B. Metric-Specific Divergence There might be several different dimensions in which divergence materializes, and we now examine how different factors react to individual CVSS metrics. For each CVE, we define a binary dependent variable (e.g., AV_Diff = 1 if the NVD and CNA assign different values for AV, and 0 otherwise). As shown in Table V, the results indicate that the determinants of divergent vary across metrics. The Who First variable shows consistently strong and significant effects across all metrics, indicating that publication order is an important factor. In the following, we focus on metric-specific patterns.

8

TABLE V: Logistic Regression Analysis Results of Each CVSS Metric Factors Constant Vendor Open Source Researcher Bug Bounty Provider Hosted Service CERT Who First CWE Description Length Who First × V Who First × OS Who First × R Who First × BBP Who First × HS Who First × CERT pseudo-R2

AV Diff Coef. Sig. -2.843 *** -0.078 -0.624 *** 0.820 *** -0.274 -1.396 *** -0.168 1.040 *** 0.103 * 0.001 -0.716 *** 0.466 ** -1.614 *** -0.222 0.219 -0.324 0.053

AC Diff Coef. Sig. -3.532 *** 1.431 *** 0.988 *** 0.264 *** -1.036 -1.928 *** 0.684 ** 1.152 *** 0.022 -0.001 ** -0.703 *** -0.270 ** -0.531 *** 0.451 0.446 * -0.846 ** 0.075

PR Diff Coef. Sig. -1.936 *** -0.208 *** -0.958 *** 1.957 *** -1.386 * -0.699 *** -0.344 1.205 *** 0.104 *** -0.003 *** -0.450 *** 0.877 *** -2.068 *** 0.778 -0.267 -0.656 ** 0.071

UI Diff Coef. Sig. -3.799 *** 1.278 *** 0.186 0.396 *** 0.968 -1.237 *** 0.828 ** 0.906 *** 0.140 *** -0.001 * -0.211 -0.572 *** 0.158 -2.510 *** 0.023 -0.080 0.068

S Diff Coef. Sig. -2.747 *** 0.470 *** -0.262 ** 1.443 *** -0.998 -0.383 ** 0.729 *** 1.410 *** -0.190 *** 0.001 *** -0.729 *** 0.634 *** -1.466 *** 0.962 -0.356 * -0.983 *** 0.030

C Diff Coef. Sig. -2.160 *** -0.629 *** -1.074 *** 2.990 *** 0.695 -1.505 *** 0.345 * 1.620 *** 0.093 *** -0.001 -0.617 *** 0.822 *** -2.223 *** 0.012 1.113 *** -1.353 *** 0.132

I Diff Coef. Sig. -2.388 *** -0.184 ** -0.675 *** 2.242 *** 0.813 * -1.391 *** 0.538 1.504 *** 0.128 *** -0.001 * -0.685 *** 0.642 *** -1.755 *** 0.221 0.787 *** -1.141 *** 0.074

A Diff Coef. Sig. -2.226 *** -0.094 -0.694 *** 2.220 *** 1.672 *** -0.894 *** 0.471 ** 1.425 *** 0.142 *** -0.001 *** -0.695 *** 0.640 *** -1.701 *** 0.931 ** 0.532 ** -1.097 *** 0.144

Significance levels: ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. Non-significant results (p ≥ 0.05) are left blank. Note: Logistic regression on all 43 997 entries, dependent variable for each variable is whether there is a difference in scoring for the specific CVSS Metrics. E.g. for the first two columns AV measure the probability that there is disagreement on AV.

Impact. Researchers show consistently positive and significant coefficients across CVSS metrics, with the strongest effects on C (19.9x), I (9.4x), and A (9.2x). This suggests that divergence is especially likely for Impact metrics. The negative Who First × Researcher interactions indicate that this effect is weaker when researchers publish first, and more pronounced when NVD publishes first. Another notable source of divergent comes from Bug Bounty Providers on the Impact metrics. The logistic regression shows large positive effects for I (2.2x) and A (5.3x). One plausible explanation is incentive-driven reporting: bug bounty platforms aggregate submissions from independent researchers, who may emphasize certain impacts to strengthen reward eligibility. Consequently, reported impact severity may partly reflect bounty incentives rather than a broader consensus view. Attack Complexity. Vendor and Open Source, providers exhibit elevated divergency on the AC metric, with odds ratios of approximately 4.2× and 2.7×, respectively. Confirmed in our interview and implementation, this metric is known to be one of the most difficult to evaluate (also studied for both students and security professionals [23]) as it requires a precise understanding of the technical conditions affecting exploitation. It is therefore reasonable to expect that software vendors, who have deeper knowledge of their systems, may diverge with the NVD when it assigns a score first (as reflected by the negative coefficient of “Who First × V”). Conversely, when vendors publish their assessments earlier, the NVD may be more inclined to follow. User Interaction. We observe a consistent pattern of positive coefficients for the UI metric across multiple CNA types, with statistically significant effects for Vendor, Researcher, and CERT. Despite appearing conceptually simple [23], the interpretation of UI (e.g., whether a victim must “click” or otherwise perform an action) remains ambiguous in several corner cases such as Cross-Site Scripting (XSS). This ambiguity has been widely discussed within the CVSS SIG, they dedicated several meetings after CVSS v4 was released to clarify these aspects and revised its user guide (https: //www.first.org/cvss/v4-0/faq).

Takeaway 7. Divergency is significantly associated with AC, UI, and Impact metrics, with some CNAs showing stronger divergence in these dimensions. These metrics are more ambiguous, aligning with refinements in CVSSv4 that further distinguish AC, UI, and Impact metrics.

VII. ROOT C AUSES A NALYSIS Based on feedback from the NVD, six CNAs, and seven additional experts who participated in the subsequent CVSS SIG discussion, we identified eight recurring root causes (RCs) of divergence between CNAs and the NVD. We discuss each RC below and provide 12 representative examples in Table VI. RC1 - Divergence is not an error. Divergence is not an error but actually a right thing. Some assessments are based on downstream product configurations and deployment contexts, which may differ from upstream assumptions. For example, security features, such as SELinux, may be enabled by default in downstream distributions, reducing the exploitability of certain vulnerabilities. As a result, CNAs may assign lower severity scores, as they adopt a more “secure” default stance compared to some upstream repositories. RC2 - Differences in CWE/CPE. One CNA mentioned that vulnerabilities with identical descriptions may still be assigned different CWE values, and that such differences should be considered when analyzing divergence. They also noted the importance of verifying whether other attributes, such as CPE (i.e., the affected products), are also the same. This insight suggests that, at the description level, relying solely on textual similarity may be insufficient; additional structured information, such as CWE and CPE, should also be taken into account (See the detailed results in Table III). In addition, feedback from the NVD indicates a potential cause of CWE discrepancies. CWE is maintained by MITRE, and the NVD uses only a subset of the full CWE taxonomy for enrichment. As a result, the NVD may not always have access to the same level of specificity as the CWE values provided by CNAs, leading to differences in assigned CWE categories.

9

RC3 - Selective Disclosure of Scoring Rationale. The CVSS scores from some CNAs often align with the NVD because additional scoring rationale is provided in the frequently asked questions (FAQs) accompanying their advisories. These FAQs offer further explanations for metric assignments (e.g., AC:H or S:C), enabling NVD analysts to assign the same CVSS vector from publicly available information. However, the internal variability remains substantial, and they typically tend to provide explanations only when metric values deviate from commonly expected settings (e.g., explaining AC:H but not AC:L). Therefore, even when additional explanations are available, divergences may persist due to the absence of a complete and systematic description of the scoring rationale. RC4 - Human Errors. Divergency may also arise from CVE submission process. For example, vulnerability descriptions in the NVD may not match those provided in the original security advisories. One CNA indicated that such divergence can occur when incorrect or incomplete data is submitted to the CVE system, causing NVD to base its assessment on inaccurate information. RC5 - Insufficient Detail in CVE Descriptions. Inconsistencies in CVSS scoring may stem from insufficient detail in CVE descriptions. In some cases, descriptions do not provide enough information to justify specific metric assignments, making it difficult for external analysts to reach the same conclusions (e.g., NVD). For example, one CNA noted that descriptions may lack sufficient detail to explain why certain metrics (e.g., PR) are rated as Low versus High. Another CNA further suggested that CVE descriptions may not accurately convey all aspects of the vector. Encouragingly, one CNA reported that more detailed descriptions have started to appear in executive summaries after 2025, providing additional insight into the vulnerability. We conducted an extra study after 2025 in Section IX. In other cases, similar or identical descriptions may be used for vulnerabilities that differ in their actual impact. For instance, two XSS vulnerabilities with the same description may exhibit different impacts depending on the exploitation context. One CNA suggested that the root cause may lie in the fact that CVE descriptions are not sufficiently curated to distinguish each individual XSS vulnerabilities, particularly in terms of their impact differences. RC6 - Information Asymmetry and Limited Context for NVD. Discrepancies may be caused by information asymmetry, where CNAs possess internal information that is not accessible to the NVD. For example, the information available to the NVD may be incomplete or inconsistent with that held by CNAs. Additionally, the NVD may lack access to proofof-concept exploits or sufficient contextual data required to consistently assess CVSS scores with high accuracy. One CNA further suggested that, in practice, it may be more reliable to rely on CVSS scores assigned by CNAs, as the NVD must process a large volume of CVEs annually, making it impractical to perform deep technical validation for each entry. RC7 - Risk Modeling Differences. Some discrepancies arise from differences in how organizations model and interpret risk. This divergence in risk modeling assumptions can lead to

TABLE VI: CVE Examples of Root Causes # 1 2 3 4 5 6 7 8 9 10 11 12

CVE-ID RC1 RC2 RC3 RC4 RC5 RC6 RC7 RC8 CVE-2024-28907 x x CVE-2024-30007 x x CVE-2021-1471 x x CVE-2022-20955 x x CVE-2020-3393 x CVE-2025-20270 x CVE-2024-20509 x CVE-2020-1227 x CVE-2020-16945 x CVE-2022-21830 x CVE-2022-0847 x CVE-2019-5736 x x x x Total 1 2 2 2 5 5 1 1

systematic differences in CVSS scoring across sources. Some discrepancies arise from vulnerabilities that involve multiple distinct attack scenarios. In such cases, a single CVE may encompass different exploitation paths with varying levels of impact and attack complexity. For example, one CNA described a vulnerability that includes both a race condition requiring high attack complexity (AC:H) and a simpler attack path with low attack complexity (AC:L) leading to availability impact. The CNA chose to assign the score based on the highest impact scenario. However, other CNAs may combine all aspects of the vulnerability description into its assessment, even when these correspond to distinct attack scenarios. RC8 - Ambiguity in CVSS Specification. One CNA suggested that inconsistencies may arise from inherent ambiguity in the CVSS specification itself. This is supported by the evolution of the CVSS standard, where several refinements have been introduced in CVSS v4 to address potential ambiguities in earlier versions. For example, CVSS v4 separates Attack Complexity in CVSS v3.1 into two distinct metrics: Attack Complexity and Attack Requirements. Takeaway 8. As shown in Table VI, we observe that many examples involve insufficient detail to support metric assessment and information asymmetry across sources. Divergent assessments may therefore be both correct state unless more information is provided in individual CVE.

VIII. C ASE S TUDIES To better understand the root causes in practice, we conduct a case study with two analysts from a CNA who have implemented real-world attacks for two vulnerabilities: Dirty Pipe (CVE-2022-0847) and runC Overwriting (CVE-2019-5736), corresponding to the last two CVEs in Table VI. Both attack implementations are conducted on Ubuntu 22.04. For each case, we first introduce the CVSS metric definitions to the analysts. Based on their implementation experience, we ask them to assign values for each CVSS metric and assess whether the corresponding CVE description provides sufficient information to support these assignments. Finally, we discuss root causes for other cases. Dirty Pipe. This vulnerability is assigned identical CVSS metrics by the NVD, SUSE and CISA-ADP. Based on the implementation experience of the analysts, they assigned the vector

10

TABLE VII: CVSS across different sources (CVE-2019-5736) Source SUSE RedHat NVD Canonical Implemented

CVSS Vector CVSS:3.0/AV:L/AC:H/PR:L/UI:R/S:C/C:H/I:H/A:H CVSS:3.1/AV:L/AC:H/PR:N/UI:R/S:C/C:H/I:H/A:H CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H CVSS:3.1/AV:L/AC:L/PR:H/UI:R/S:C/C:H/I:H/A:H

# of CNAs

150 100 50 0

0

1 2 3 4 6 vector-Divergency (median)

Fig. 5: vector-Divergency values under the Consumer-View settings after 2025

we collect additional data from February 2025 to March 2026 to examine whether these divergence issues are reduced over time. vector-Divergency Analysis After 2025. Fig. 5 shows how the median vector-divergency of publicly available CNAs changes after 2025. The results indicate that only 27/174 (16%) of CNAs have a median vector-Disagreement of at least 1, representing a substantial reduction in CNA–NVD divergency compared to the pre-2025 result (48%). This suggests a notable shift toward greater convergency in CVSS scoring after 2025. Fig. 6 shows the distribution of vector-divergence values for the top 20 CNAs after 2025 under the Consumer-View setting. Notably, only two CNAs exhibit a median vector of at least 1, while the majority are concentrated at 0. Compared to the pre-2025 (Figure 4a), most CNAs show reduced divergency. Samsung Mobile dropped from median value (vector= 2) before 2025 to 1 after 2025. Only VulDB shows no noticeable improvement, continuing to exhibit relatively high divergency (vector= 3) both before and after 2025. Overall, these results suggest that the reduction in disagreement after 2025 is widespread across major CNAs, with only a small number of sources still exhibiting persistent divergence. Description-level Divergency After 2025. Figure 7 shows a clear shift in description length before and after 2025. The share of short descriptions, i.e., those with 0-10 words, decreased substantially after 2025, from over 20% to 2.4%. Meanwhile, descriptions with 20-30 words become the ma-

d-Disagr m nt M tric

CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/ A:H, which matches the CVSS vector provided by these three sources. However, they indicated that the information related to AC is insufficient and that UI is unclear (RC5). runC Overwriting. This vulnerability is assigned different CVSS vectors by the NVD and multiple CNAs, including SUSE, RedHat, and Canonical. Notably, SUSE and RedHat share the same vulnerability description, while Canonical shares the same description as the NVD (RC6). As shown in Table VII, disagreement is primarily concentrated in the AC and PR metrics. Sources with identical descriptions tend to assign consistent values for AC, SUSE and RedHat assign AC:H, whereas the NVD and Canonical assign AC:L, suggesting that vulnerability descriptions strongly influence metric assignment. However, discrepancies remain even under identical descriptions. Major sources assign PR:N, whereas the analysts assign PR:H based on their implementation experience, explaining that root privilege inside the container is required to perform the attack. This indicates that existing sources may value differently the privilege requirement. The analysts assign AC:L, consistent with the NVD and Canonical, likely because their implementation was conducted on Ubuntu, a Linux distribution maintained by Canonical (RC1). When assessing description quality, the analysts report that the SUSE and RedHat descriptions lack sufficient information for AC, PR, and UI, while the NVD and Canonical descriptions provide slightly more information for AC, and UI remains unclear (RC5). These two cases highlight the importance of providing accurate and detailed descriptions. Divergency may arise from differences in descriptions across sources or missing information for certain metrics, especially AC, which the analysts also identify as the most difficult to assign, as it depends on the interpretation of attack complexity (RC8) and involves many subjective judgment. Cases with RC2&RC3. CVE-2024-28907 and CVE-202430007 contain identical description but different CVSS scores. Augmenting with their CWEs (CWE-59 and CWE-269), both CVEs should fall into different groups, thus resolving the divergence. Cases with RC4&RC6. CVE-2021-1471 and CVE-202220955 illustrate discrepancies arising from the CVE submission process. Incorrect/incomplete data may be submitted to the CVE system, leading to mismatches between NVD descriptions and original security advisories, which can cause disagreement across sources. Cases with RC7. CVE-2024-20509 involves multiple exploitation paths, including a race condition (AC:H) and a low-complexity path (AC:L). Different sources may prioritize different paths, resulting in divergent metric assignments.

M dian = 0

M dian = 1

M dian = 2

M dian = 3

8 6 4 2 0

IX. T EMPORAL A NALYSIS AND P OST-2025 T RENDS As highlighted in RC5, a CNA noted that they have started to provide more detailed descriptions after 2025. We therefore conduct a follow-up analysis using data after January 2025. While our original dataset covers CVEs up to January 2025,

ll Ab t l at ns yk DP ck -b DB ce oft sco cle be RT RE BM m il A-A hsta GitH V-lordfen icros Ci Ora AdoCS-CE MIT I alcomg Mob D GitL In R d HSi m Sn M I CIS Patc W Q- s-n Sam

Fig. 6: Distribution of vector-level divergency of Top 20 CNAs under Consumer-View after 2025

Pre-2025

0.4

Post-2025

Divergent

Consistent

Ratio

0.3 0.2 0.1 120

110

110 -

CVE Description Word Co nt Range

100 -

0

100

90-

0

809

0

708

0

607

0

506

0

405

0

304

0

203

102

0-1

0

0.0

Fig. 7: Relationship between CVE description length and Description-level divergency before and after 2025

jority group, replacing the 0-10 word group that dominated before 2025. Although the absolute divergence rates in the 10– 20 and 20–30 word groups are higher after 2025 than before 2025, these groups account for only about 1/3 of all cases, compared with approximately 50% before 2025. This provides further evidence that longer descriptions are associated with lower divergence and RC5, i.e., more detailed descriptions have started to appear after 2025. Takeaway 9. The divergency between CNA and NVD decreases after 2025 compared to the pre-2025 period under the Consumer-View setting. This indicates improved consistency in CVSS assessments over time, possible due to better description quality.

X. I MPACT In this section, we discuss potential impacts on both research and downstream applications. Broad Impact on Research. To illustrate the impact of the reliability of CVSS scores, we systematically collected papers published between 2021 and 2025 from the A∗ conferences in security and software engineering, as ranked in the 2023 CORE list [24] (four security conferences: IEEE S&P, USENIX Security, ACM CCS, and ISOC NDSS, three software engineering conferences: ASE, ICSE, and ESEC/FSE). The impact maybe larger as we have not considered journals (e.g., TSE) or other internet conferences such as IMC or WWW. We searched all papers having CVSS in the text and belonging to these venues from year 2021 to 2025 4 This search initially yielded 109 papers. We excluded nine papers that were published in conferences outside the selected seven conferences, despite having similar names, such as Euro S&P and ASIA CCS, and one USENIX paper [25] that included the term “CVSs” which was unrelated to our context. We then read the full text of the remaining 100 papers. A paper was considered impacted if divergence in CVSS scoring could substantially influence its reported effectiveness, thereby affecting the interpretability, comparability, or external validity 4 Google advanced search feature with the string “CVSS” source: “Symposium on Security and Privacy” OR source: “USENIX Security” OR source: “Conference on Computer and Communications Security” OR source: “Network and Distributed System Security” OR source: “Automated Software Engineering” OR source: “International Conference on Software Engineering” OR source: “Foundations of Software Engineering”

Acc0−ac2 Diff. (CNA - NVD) (%)

11

Met-i

0

AV AC PR UI S C I A

−20 −40 −60 −80 0

20 40 60 Propotion of metri -Level Disag-eement (%)

CNA

CISA Pat %.ta k GitHub Vu(DB Wo-dfen e Mi -o.oft ICS-CERT MITRE RedHat Sie)en. Snyk

80

Fig. 8: The same CVE rated by a CNA and NVD

of its results. Based on this criterion, we identified 25 impacted papers, which are categorized in Table VIII according to the four impact categories (I1–I4). Downstream Impact. We reproduced the CVSS-BERT model of Shahid et al. [51], which predicts CVSS base metrics from descriptions. We followed the original training setup and evaluated per CNA, removing all overlapping (identical) descriptions from training to avoid leakage. As shown in Fig. 8, accuracy systematically drops across CNAs. In particular, for Wordfence, where UI disagreements are amplified by 957 entries, UI accuracy falls by ∼ 40% relative to the reported baseline. This is not a single case: VulDB shows similarly large degradations on PR and the Impact metrics. These results indicate that, without explicitly quantifying NVD and CNA inconsistencies, models trained on one source do not reliably generalize to others. Consequently, reported metrics depend heavily on the chosen test source, resulting non-comparable scores. This motivates source-aware discrepancy metrics (external distance and internal stability) and persource evaluation. Implications for CVSS Framework. For CVSS practice, these disagreements reveal where additional guidelines may be necessary (e.g., longer description, error-prone vectors, etc.), and the ongoing development of CVSSv4 can help simplify rating dimensions such as Attack Complexity. Our proposed metrics further support continuous evaluation of cross-version consistency when more CVSSv4 data become available. Implication for Practice. Organizations should expect and plan for systematic divergence between CNA and NVD assessments, even among closely related CNAs. Lacking vulnerability information and different interpretation of “worst case” [52] can explain much of this disagreement. Relying on a single source introduces bias; integrating multiple sources without clear rules can propagate inconsistencies. We therefore recommend explicit, source-aware selection and tie-breaking policies when integrating CVSS data from multiple sources. Even within a single source, internal assessments may conflict: CVEs with identical descriptions and matching CWE and CPE attributes can still receive different vectors. Such inconsistencies stem from manual errors [9] and subjective interpretations [53]. This can mislead downstream consumers and underscores the need for stricter CNA hygiene, including more informative descriptions. Compliance and governance. The assessment discrepan-

12

TABLE VIII: Sample of Works in A* Security and Software Engineering Conferences 2021-25 and Why They Are Affected Prior Works [26], [27], [28], [29], [30], [31], [32], [33], [34], [35] [36], [37], [38], [39], [40] [41], [42], [43] [44], [45], [46], [47], [48], [49], [50]

Why Impacted? I1: Use CVSS as input feature or guideline (e.g., filter low CVSS entries), program input point may differ based on sources I2: Use CVSS scores to represent vulnerability severity or support decision-making (e.g., patching, mitigation, prioritization), different sources provide different priorities I3: Use description to predict CVSS metrics, same description may appear in both training and testing with different scores I4: Use CVSS as evaluation ground truth, different evaluation sources provide different ground truth

cies may impact the effectiveness of vulnerability management practices and undermine compliance with standards and laws [54], [55], [56], [57] that require organizations to assess and prioritize vulnerabilities based on reliable criteria, and that mandate timely remediation of high-risk vulnerabilities. Source-aware integration and stability filters (OR) improve auditability and help ensure consistent, defensible prioritization. Implications for Research. Our study impacts several areas of software security research that rely heavily on CVSS. The most apparent impact is that when preparing datasets for machine learning models, filtering and data cleaning is necessary to reduce label noise and improve model reliability, even from trusted sources. As summarized in Table VIII, prior works on vulnerability severity prediction, patch prioritization, and vulnerability analysis directly depend on NVD CVSS data may inherit biases or noise from the underlying datasets. The effectiveness of vulnerability mining works may be substantially influenced by changing rating methods. Therefore, before using CVSS data for downstream security processes, researchers should carefully address discrepancies both within and across sources to improve model reliability and accuracy. We already observed length of the description contributes to divergency in the regression results. For downstream vulnerability modeling, automated scoring, and user studies, descriptions that are extremely short should be handled with care: either filter short descriptions or augment them with contextual details (e.g.,CWE, affected CPEs, impact scope, exploit prerequisites) to improve model robustness and reduce ambiguity in human evaluations.

XI. L IMITATIONS Imbalanced Data Entries from CNAs. To reduce selection bias, we include all CNAs that have published CVSS scores in the NVD, rather than focusing solely on a few prominent organizations. This broad coverage improves the internal validity of our findings, ensuring that our analysis reflects the full spectrum of CVSS assessment behaviors in the ecosystem. However, the reliability of our consistency measurements, such as odds ratio is inherently influenced by the amount of available data per CNA. For some CNAs with limited data or few entries sharing identical descriptions, our metrics may lack sufficient statistical power to support strong conclusions. In contrast, for well-represented CNAs with many comparable groups (e.g., Oracle), consistent scoring behavior is supported by both high data volume and low disagreement scores.

NVD Status and Consumer Assumption. Our analysis does not explicitly consider the NVD processing status (e.g., whether a CVE is still under modification or fully analyzed). In practice, NVD recommends considering such status information when interpreting CVSS scores. However, from a consumer perspective, users typically rely on available scores without inspecting processing status. Therefore, we approximate a Consumer-View setting by treating CNA-only entries as implicitly followed by NVD. While this assumption reflects practical usage, it may overestimate agreement when NVD scores are still evolving. Incorporating finer-grained status information is an important direction for future work. Generality to CVSS v4.0. Although our study is based on CVSS v3.1, we believe that our methodology is still applicable under CVSS v4.0, despite structural changes and added metrics in the new version. The approach remains capable of identifying discrepancies in severity assessments across sources, offering potential value for both industrial systems and academic research that rely heavily on consistent vulnerability evaluation. XII. R ELATED W ORK Features Extraction for CVSS. Several research efforts extract specific features, such as vulnerability descriptions and assessment information from public repositories like the NVD [58], [7], [59], [60], [61], which can be used to predict CVSS base metrics values [62], [63], [64], [65]. Studies [66], [67] note that manual severity assessments often take over 130 days to complete, prompting the development of MLbased automation approaches. Early work relies on traditional models like SVM [68] and XGBoost [69], while later studies shift toward neural networks and NLP-based models, such as BERT-based classifiers [70], [51], [64], [61]. More recently, large language models (LLMs) are explored for CVSS classification and cybersecurity entity alignment across multiple data sources [71], [72], [33]. However, these works assumes CVSS annotations across public sources are consistent and reliable, without mitigation. Our study systematically investigates the prevalence and impact of scoring divergencies, providing insights for building more trustworthy ML-based vulnerability analysis. Disagreement/Inconsistency in Vulnerability Databases. Multiple works study the inconsistency in vulnerability data sources, such as severity scores and vulnerability types [6], mismatched software names and versions [7], CVSS base metrics assigned by two different organizations [8], and discrep-

13

ancies between same or semantically similar NVD entries [9]. A user study by Wunder et al. [44] survey 196 CVSS users, revealing a high degree of subjectivity in severity assessments. Mell et al. [4] investigate the gap between CVSS scoring and expert opinion. A recent study further shows the difference in scoring between different vulnerability scoring systems, leading to conflicting prioritization signals [73]. While some prior work has noted divergence in vulnerability databases, it lacks a systematic analysis of such divergence and the reliability of original CNAs. To fill this gap, we analyze how divergence correlates with CNA types and introduce odds ratio to identify more consistent data subsets.

asymmetry are among the most common causes of divergent scoring. Finally, our extended analysis shows a positive trend after 2025, suggesting that more detailed CVE descriptions may help reduce CVSS divergence. The possible impact discussed in this paper sheds light on future directions for researchers, government policymakers, and practitioners. We hope these findings raise awareness among CVE consumers, inform future CVE reporting practices and CVSS guidance, and help the ecosystem move toward more transparent, consistent, and actionable vulnerability assessment.

CNA-Based Rating. Beyond individual-level variations, institutional inconsistencies are also observed. Coutinho et al. [12] analyze how contextual, product-specific factors influence CVSS score divergence, but their in-depth analysis is limited to ten CNAs, leading to misleading takeaways such as “NVD generally assigns higher CVSS scores than CNAs”. In contrast, our regression across all CNAs reveals that scoring tendencies vary by organizational type. For example, vendors tend to assign higher scores than the NVD, whereas research-oriented CNAs may often assign lower ones (see Table IV). Similarly, Miranda et al. [72] employ large language models (LLMs) to learn CNA-specific CVSS scoring patterns, achieving over 70% accuracy and F1-scores, suggesting the potential for knowledge transfer across CNAs in public datasets, e.g., NVD. While these studies focus primarily on product-specific vulnerabilities and investigate possible knowledge transfer between CNAs and the NVD, our work provides a broader perspective. We systematically analyze CNA–NVD discrepancies across all available CNAs to examine overall assessment orientations. Rather than focusing on discussing a few individual organizations, our regression approach uncovers how organizational types and other contextual factors shape divergence patterns in CVSS scoring.

[1] E. S. Raymond, “The cathedral and the bazaar,” First Monday, vol. 3, no. 3, 1998. [2] K. A. Farris and et al., “VULCON: A system for vulnerability prioritization, mitigation, and management,” ACM Trans. Priv. Secur., vol. 21, no. 4, pp. 16:1–16:28, 2018. [3] J. Jacobs and et al., “Enhancing vulnerability prioritization: Data-driven exploit predictions with community-driven insights,” in Proc. of IEEE EuroS&P’23, 2023, pp. 194–206. [4] P. Mell and et al., “Measuring the common vulnerability scoring system base score equation,” National Institute of Standards and Technology (NIST), NIST Internal Report (IR), 2022. [5] “CVE History,” 2025, accessed:2025-04-21. [6] A. Anwar and et al., “Cleaning the NVD: comprehensive quality assessment, improvements, and analyses,” IEEE Trans. Dependable Secur. Comput., vol. 19, no. 6, pp. 4255–4269, 2022. [7] Y. Dong and et al., “Towards the detection of inconsistencies in public security vulnerability reports,” in Proc. of USENIX Security’19, 2019, pp. 869–885. [8] Y. Jiang and Y. Atif, “Towards automatic discovery and assessment of vulnerability severity in cyber-physical systems,” Array, vol. 15, p. 100209, 2022. [9] S. Zhang and et al., “The flaw within: Identifying CVSS score discrepancies in the NVD,” in Proc. of CloudCom’23. IEEE, 2023, pp. 185–192. [10] Jai Vijayan, “Discrepancies discovered in vulnerability severity ratings,” 2025, https://www.darkreading.com/application-security/discrepanciesdiscovered-in-vulnerability-severity-ratings. [11] Open Source Security, “Broken vulnerability severities,” 2025, https: //opensourcesecurity.io/2020/05/27/broken-vulnerability-severities/. [12] L. S. Coutinho and et al., “How context impacts vulnerability severity: An analysis of product-specific CVSS scores,” in Proc. of LADC’24. ACM, 2024, pp. 17–27. [13] National Institute of Standards and Technology, “CNA Counting Rules,” 2025, https://nvd.nist.gov/general/cna-counting. [14] CVE Program, “CVE Numbering Authorities (CNAs),” [n. d.], https: //www.cve.org/programorganization/cnas. [15] ——, “First-Ever CVE Authorized Data Publisher (ADP) Now Enhancing CVE Records — CISA ADP,” 2025, https://www.cve.org/Media/News/item/blog/2024/06/04/CISA-Addedas-CVE-Authorized-Data-Publisher. [16] Forum of Incident Response and Security Teams (FIRST), “Common Vulnerability Scoring System (CVSS),” 2025, https://www.first.org/ cvss/. [17] VulnCheck Team, “NVD++: Enhanced National Vulnerability Database,” 2025, https://www.vulncheck.com/. [18] The National Institute of Standards and Technology, “NVD Data Feeds,” 2025, https://nvd.nist.gov/vuln. [19] “CVE Github Page,” 2025, accessed: 2025-04-14. [20] CVE Program, “List of CVE Numbering Authorities (CNAs),” 2025, https://www.cve.org/PartnerInformation/ListofPartners. [21] B. Williams and et al., “Sample size matters when estimating test-retest reliability of behaviour,” Behavior Research Methods, vol. 57, no. 4, p. 123, Mar. 2025. [22] F. D. Schönbrodt and M. Perugini, “At what sample size do correlations stabilize?” Journal of Research in Personality, vol. 47, no. 5, pp. 609– 612, 2013. [23] L. Allodi and et al., “Measuring the accuracy of software vulnerability assessments: experiments with students and professionals,” Emp. Softw. Eng., vol. 25, no. 2, pp. 1063–1094, 2020. [24] L. Padgham and et al., “CORE Rankings,” 2023, https: //www.core.edu.au/conference-portal.

XIII. C ONCLUSION In this paper, we developed systematic metrics to quantify divergency between CNAs and the NVD and introduced the first group-based framework to assess internal divergency by grouping CVEs with identical descriptions. Across all 288 CNAs, 73% (pairwise) and 48% (consumer-view) showed at least one metric-level divergency in the majority of their rated CVEs. A statistically significant analysis across the full set of CNAs shows that risk-sensitive organizations, such as Vendors, Researchers and CERT, diverge more frequently. Specifically, Vendors and CERT often assign higher severity scores, whereas Researchers tend to assign lower scores. When CNA releases the CVSS first, NVD tend to disagree with it. However, certain types of CNA, such as Vendors, Researchers, and CERT, tend to trusted by NVD. Consistent across different studies in the paper, the most frequent sources of disagreement are the Attack Complexity, User Interaction, and Impact metrics. Through feedback from CNAs, NVD, and the CVSS SIG, we further summarize eight root causes of divergence and analyze 12 representative vulnerabilities. These cases suggest that insufficient detail in CVE descriptions and information

R EFERENCES

14

[25] M. Lin and et al., “Detecting API post-handling bugs using code and description in patches,” in Proc. of USENIX Security’23, 2023, pp. 3709– 3726. [26] S. Woo and et al., “MOVERY: A precise approach for modified vulnerable code clone discovery from modified open-source software components,” in Proc. of USENIX Security’22, 2022, pp. 3037–3053. [27] Y. Hu and et al., “Achyb: a hybrid analysis approach to detect kernel access control vulnerabilities,” in Proc. of ESEC/FSE’21. ACM, 2021, pp. 316–327. [28] H. Liu and et al., “Labrador: Response guided directed fuzzing for blackbox iot devices,” in Proc. of IEEE S&P’24, 2024, pp. 1920–1938. [29] J. Kim and et al., “Passrefinder: Credential stuffing risk prediction by representing password reuse between websites on a graph,” in Proc. of IEEE S&P’24, 2024, pp. 1385–1404. [30] Y. Li and et al., “UNIFUZZ: A holistic and pragmatic metrics-driven platform for evaluating fuzzers,” in Proc. of USENIX Security’21. USENIX Association, 2021, pp. 2777–2794. [31] F. Xiao, Z. Su, G. Yang, and W. Lee, “Jasmine: Scale up javascript static security analysis with computation-based semantic explanation,” in Proc. of IEEE S&P’24. IEEE, 2024, pp. 296–311. [32] X. Tan and et al., “Locating the security patches for disclosed OSS vulnerabilities with vulnerability-commit correlation ranking,” in Proc. of CCS’21. ACM, 2021, pp. 3282–3299. [33] Y. Qin and et al., “Vulnerability intelligence alignment via masked graph attention networks,” in Proc. of CCS’23. ACM, 2023, pp. 2202–2216. [34] C. Wang and et al., “REEF: A framework for collecting real-world vulnerabilities and fixes,” in Proc. of ASE’23. IEEE, 2023, pp. 1952– 1962. [35] X. Cheng and et al., “KVS: a tool for knowledge-driven vulnerability searching,” in Proc. of ESEC/FSE’22. ACM, 2022, pp. 1731–1735. [36] S. Pan and et al., “Towards more practical automation of vulnerability assessment,” in Proc. of ICSE’24. ACM, 2024. [37] J. Wang and et al., “Improving prompt tuning-based software vulnerability assessment by fusing source code and vulnerability description,” Autom. Softw. Eng., vol. 32, no. 2, p. 45, 2025. [38] T. H. M. Le and et al., “Deepcva: Automated commit-level vulnerability assessment with deep multi-task learning,” in Proc. of ASE’21. IEEE, 2021, pp. 717–729. [39] D. Cerdeira and et al., “Rezone: Disarming trustzone with TEE privilege reduction,” in Proc. of USENIX Security’22, 2022, pp. 2261–2279. [40] R. Jahanshahi and et al., “Minimalist: Semi-automated debloating of PHP web applications through static analysis,” in Proc. of USENIX Security’23, 2023, pp. 5557–5573. [41] J. Luo and et al., “Cvecenter: Industry practice of automated vulnerability management for linux distribution community,” in Proc. of FSE’24. ACM, 2024, pp. 329–339. [42] O. Suciu and et al., “Expected exploitability: Predicting the development of functional vulnerability exploits,” in Proc. of USENIX Security’22, 2022, pp. 377–394. [43] A. Ethembabaoglu and et al., “The unpatchables: Why municipalities persist in running vulnerable hosts,” in Proc. of USENIX Security’24, 2024. [44] J. Wunder and et al., “Shedding light on CVSS scoring inconsistencies: A user-centric study on evaluating widespread security vulnerabilities,” in Proc. of IEEE S&P’24. IEEE, 2024, pp. 1102–1121. [45] Z. Jiang and et al., “Evocatio: Conjuring bug capabilities from a single poc,” in Proc. of CCS’22. ACM, 2022, pp. 1599–1613. [46] E. Wang and et al., “Where urls become weapons: Automated discovery of SSRF vulnerabilities in web applications,” in Proc. of IEEE S&P’24, 2024, pp. 239–257. [47] S. R. Pérez and et al., “Patchy performance? uncovering the vulnerability management practices of iot-centric vendors,” in Proc. of IEEE S&P’24. IEEE, 2024, pp. 1198–1216. [48] C. Yu and et al., “File hijacking vulnerability: The elephant in the room,” in Proc. of NDSS’24. The Internet Society, 2024. [49] Y. Xiao and et al., “Jbomaudit: Assessing the landscape, compliance, and security implications of java sboms,” in Proc. of NDSS’25. The Internet Society, 2025. [50] B. Kondracki and N. Nikiforakis, “Smudged fingerprints: Characterizing and improving the performance of web application fingerprinting,” in Proc. of USENIX Security’24, 2024. [51] M. R. Shahid and H. Debar, “CVSS-BERT: explainable natural language processing to determine the severity of a computer security vulnerability from its description,” in Internat. Conf. on Machine Learning and Applications. IEEE, 2021, pp. 1600–1607. [52] National Institute of Standards and Technology, “Vulnerability Metrics,” 2025, https://nvd.nist.gov/vuln-metrics/cvss\#.

[53] B. Frénay and M. Verleysen, “Classification in the presence of label noise: A survey,” IEEE Trans. Neural Networks Learn. Syst., vol. 25, no. 5, pp. 845–869, 2014. [54] ISO, “Information security, cybersecurity and privacy protection — Guidance on managing information security risks,” 2025, https:// www.iso.org/standard/80585.html. [55] NIST, “NIST SP 800-53: Security and Privacy Controls for Information Systems and Organizations,” 2025, https://csrc.nist.gov/pubs/sp/800/53/ r5/upd1/final. [56] Payment Card Industry Security Standards Council, “Payment Card Industry (PCI) Data Security Standard - Requirements and Security Assessment Procedures Version 3.2.1,” 2025, https://listings.pcisecuritystandards.org/documents/PCI DSS-QRGv3 2 1.pdf. [57] European Commission, “Cyber resilience act,” 2025, https://digitalstrategy.ec.europa.eu/en/policies/cyber-resilience-act. [58] S. S. Weerawardhana and et al., “Automated extraction of vulnerability information for home computer security,” in Proc. of FPS’14, vol. 8930. Springer, 2014, pp. 356–366. [59] S. Zhang and et al., “VIET: A tool for extracting essential information from vulnerability descriptions for CVSS evaluation,” in Proc. of DBSec’23, ser. LNS, vol. 13942. Springer, 2023, pp. 386–403. [60] H. Binyamini and et al., “A framework for modeling cyber attack techniques from security vulnerability descriptions,” in Proc. of KDD’21. ACM, 2021. [61] X. Ni and et al., “Predicting severity of software vulnerability based on bert-cnn,” in Proc. of ICCEAI’22. IEEE, 2022. [62] X. Gong and et al., “Joint prediction of multiple vulnerability characteristics through multi-task learning,” in Proc. of ICECCS’19. IEEE, 2019, pp. 31–40. [63] Y. Yamamoto and et al., “Text-mining approach for estimating vulnerability score,” in Internat. Workshop on Building Analysis Datasets and Gathering Experience Returns for Security, BADGERS@RAID. IEEE, 2015, pp. 67–73. [64] J. C. Costa and et al., “Predicting CVSS metric via description interpretation,” IEEE Access, vol. 10, pp. 59 125–59 134, 2022. [65] C. Elbaz and et al., “Fighting n-day vulnerabilities with automated CVSS vector prediction at disclosure,” in Proc. of ARES’20. ACM, 2020, pp. 26:1–26:10. [66] H. Chen and et al., “VEST: A system for vulnerability exploit scoring & timing,” in Proc. of IJCAI’19. ijcai.org, 2019, pp. 6503–6505. [67] J. Ruohonen, “A look at the time delays in CVSS vulnerability scoring,” CoRR, vol. abs/1801.00938, 2018. [68] M. Edkrantz and A. Said, “Predicting cyber vulnerability exploits with machine learning,” in Proc. of SCAI’15, ser. Frontiers in Artificial Intelligence and Applications, vol. 278. IOS Press, 2015, pp. 48–57. [69] E. Manai and et al., “Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI,” Applied Sciences, vol. 14, no. 20, p. 9231, 2024. [70] J. Yin and et al., “Apply transfer learning to cybersecurity: Predicting exploitability of vulnerabilities by description,” Knowl. Based Syst., vol. 210, p. 106529, 2020. [71] F. Marchiori, D. Donadel, and M. Conti, “Can LLMs Classify CVEs? Investigating LLMs Capabilities in Computing CVSS Vectors,” 2025. [72] L. Miranda and et al., “Learning cna-oriented CVSS scores,” in Proc. of CloudNet’24. IEEE, 2024, pp. 1–5. [73] V. Koscinski and et al., “Conflicting scores, confusing signals: An empirical study of vulnerability scoring systems,” in Proc. of CCS’25. ACM, 2025, pp. 1904–1918.

15

15000

11569 7985 6592 5884 5172 4605

10000 5000

2183 2128 2119

2000

1177 1160 1102 989 949 940 867 759 709 707 653

# of CVEs

A PPENDIX

1000 0

P ck b B e ft o le e T E M m le ll b el at s k -AD sta itHu ulD enc oso CiscOrac AdobS-CERMITR IBlcom Mobi DeGitLA Int ed Hiemen Sny R S a IC SA atch G V ordf Micr IC W P Qu sung m a S

Fig. 9: #CVEs Assigned by Top Public 20 CNAs TABLE IX: Metric-Level Divergent Counts per CNA CNAs CISA-ADP Patchstack GitHub VulDB Wordfence Microsoft Cisco Oracle Adobe ICS-CERT MITRE IBM Qualcomm Samsung M. Dell GitLab Intel Red Hat Siemens Snyk

AV 345 0 239 494 2 96 42 0 7 92 196 56 19 97 38 38 1 13 26 16

Divergency metric# (metric-Level) AC PR UI S C I 169 538 287 136 362 305 114 153 40 341 1091 1166 1234 862 717 1275 1211 1225 121 2309 106 1116 3896 2709 161 62 957 130 193 204 183 205 129 111 171 177 102 120 81 228 232 200 2 1 0 1 0 0 17 11 8 47 44 42 90 143 123 183 205 208 153 154 129 353 184 216 219 234 48 53 132 105 34 378 3 49 66 62 155 452 132 102 259 281 204 98 106 154 166 155 197 137 118 116 172 169 271 52 210 167 94 69 80 76 33 42 40 55 48 58 18 51 55 45 125 44 32 62 153 153

A 356 773 1322 2870 282 198 301 1 51 184 218 103 129 320 217 147 96 48 71 184

#CVE 4860 3601 5441 4742 2132 1905 1757 335 637 1073 1011 840 829 872 872 834 540 512 491 547

16

<!---email snippet/> Dear <CNA contact/>, … I am following up on the discussion on CVSS diverging scores at the last CVSS SIG meeting. We have done some research on such scores (v3.1) and we would like to discuss the data with you before presenting at the SIG. Our standpoint is that there is no right or wrong but just different reasons. There are two findings: 1. Among your scored CVEs, there are <#group/> sets where <CNA/> text is identical for all CVEs in each set but two or more CVEs within the same set have different scores. 2. When both <CNA/> and NVD have scored something, <CVSS metric/> differs <percentage/> of the time, but <CVSS metric/> only diverges in <percentage/> of the cases. We would like to send you a presentation on that. We could then have a brief chat or you could write back to us on why that happens. We would like to anonymize and cluster the reasons for divergence, to avoid any "personalization". … <!---email snippet/>

Fig. 10: First Email: Initial email inviting CNA representatives to review our preliminary findings

<!---email snippet/> Dear <CNA contact/>, … Attached please find the presentation and two files of CVEs. 1. The first file is the list of CVEs where both you and the NVD have given a CVSS v3.1 assessment. The last column is the hamming distance from your vector and the NVD vector. A zero value means they are the same vector, the larger the value, the greater the divergence. 2. The second file lists a number of sets of (more than one) CVEs with exactly the same description from you. The last column distinguishes two cases: either the vectors are all the same (Self-Agree) or the vectors are not the same (Self-Disagree). Ignore the group number as this is counted across all CNAs. We have some ideas. For example, one thing that stands out is that very few people disagree on AV, so that's apparently the easy one. The nuances lie in the rest. Also short texts don't help. … <!---email snippet/>

Fig. 11: Second Email: Follow-up email sharing the presentation slides and CNA-specific analysis for feedback and discussion

17

1.0

AV AC PR UI S C I A

0.8 0.6 0.4 0.2 0

10

20

30

40

50

60

70

80

90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280

0.0

CNAs index ordered by #CVE entries

Fig. 12: Heatmap of all 266 public CNAs

Median = 0

Median =Chunk 1 1

Median = 2

Median = 3

Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric ement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement Metric d-Disagreement

8 7 6 Median = 0 Median =Chunk 1 1 Median = 2 Median = 3 5 8 4 7 3 6 2 5 Median = 0 Median =Chunk 1 1 Median = 2 Median = 3 1 48 0 37 /CC inet iper IBE DE DIA SAP HPeEnovo-CERTTrellixrmosQt NAP HCLINCDBrainlso Altloectric Ft5tacks ABBuawelei LLConicaol logWy indsZoomRT/CCozomiSUSMEwareenTextZyxoegl resspluntokdesrkmonTyIBCOrojecstLabsapid7 T R E 26Fort Jun INECRT@V NVI E L TR atte H oog Can Syn olar V Op Pr S AuenHa hyr Pntres R C Jet Paider E luid A KrC N S C G M TW 15 F e n Op Zep Hu 40 Sch 3 C t r E E A P E o T ix t P L D s o ic 5 s B /C ine ipe IB D DI SA HPenov -CERTrell rmosQNA HC INCBrainlo Altlectr Fttack AB uawelei LLConicaol logWy indsZoomRT/CCozomiSUSMEwareenTextZyxoegl resspluntokdesrkmonTyIBCOrojecstLabsapid7 13: Violin plot rest public ERT2 Fort Jun INECRT@V NVI L TR atte Fig. V =Op4 Pr S AuenHa hyr Pntres R C SynSolar =CNAs id A of =the MedianJe=t 0Peaider E FluMedian 1HGoog2Can Median 2 KrCE N Median C M TW 1 Chunk n Op Zep Hu 8 Sch 0 7 6 /CC inet iper IBE DE DIA SAP HPE ovo ERT ellix ost NAP HCL CD ains Alto tric F5 cks ABB wei LLC ical logy inds om /CC omi USE are Text yxel ress lunk esk ony CO ject abs id7 a n yno rW Zo ERT Noz S Mw pen Z rog Sp tod arm TIB Pro ss L Rap IN t=Br0alo Elec Median ER5T Fort Jun INECRT@V NVI LenTR-C Tar tterm Q Median Atta =Chunk V = 1HGuoogl2eCanoMedian Median P C Je P ider S Sola = 2KrC Au nH O4 W hyr tre uid C M l T F e 8 n Ope Zep Hun h 4 c S 7

3 6 2 5 Median = 0 Median =Chunk 1 2 Median = 2 Median = 4 1 48 0 37 h ti r e y jk n s io le S I y B ic C e s C ls p g ix k p P D e ic r G a e e L t al a n G hi 26sc an nde ach erg rwi tio Lab nd. ab TR ESRifera goDlectr NCShangrnete IS ntro iCor JFro abb ckm ocor PH AM ocad lastlagheASR Mox ecur dflar TMroduc igit AristERT-ICK A itac o L Mon hi E Z Che Zoh o h c C SI H Br E Gal rt B 15 iItvdefe Apchi EnPuiteutomilaicon Me Ten O F-S Clou ub ptern D e n C Has n-X ube b B Hita ell A S Ro itH Wes ubis Ope K Johnso s G t w i 40 k M Roc 3 h ti r e y k n s o e S j i l RI ay B ric SC ge es SC ls rp og ix k rp HP D de tic er G xa re re L ct tal ta -In G hi e i b c . g o h n R i b r d s a c d Bo2 Ivdaefen Apa i Eneuitertwomacton L Men Tena OT ESLiferongoi DElect NCXchabnernet CI ontarsohiCo JFr Zabhbeckomhoco P ABMroca Elasallagh ASR MFo-Seclouudfla TpMrodrnu Digi ArisCERTSICK AHitac t r h P e t li C = 4tHubeste Median =Mb0ish pen-Median Median3 C= 2Z Median = 3G Median Ku =nso1n H Chunk Rob 81 Bi Hitac well Au Si u O h s Gi W t o i J k M c o 0 R 7 6osch anti nder che ergyrwijk tion Labs d.io able TRS ESRI eray oDB ctric CSC nge etes ISC trolsCorp Frog bbix kmk corp PHP AMD adeastic gher SRG oxa cure flare TML duct gital rista T-In AG achi a c J rest n i Median M -Se ou=d 4 pro n Di A CER SICK Hit c ho CNAs N cha ern =plot nof hthe if=o0ng i 14: le Violin ZaC=hepublic Bro =ElG3alla A Median rt B5 iItvdefe Apchi EnPuiteutomilaicon Me Ten OMedianLFig. Median F Cl e Mbish E en-XMedian ub so1n CoHas Chunk 3 2Zo b ub ter K A a S t l p i Ro l 8B n H kwe O GitH Wes oh 4 itsu J M c 7 Ro 3 6 2 5 Median = 0 Median = 1 Median3 = 2 Median = 3 Median = 4 1 Chunk 48 0 37 a D ce er se ty lt tir er C er ce G ell is G MI C S os es ts n ro ra L ck a ra E u PL in bs c. ic ky D ix lis nt m o 26 e DIV evi eitz clip enti ssau lan Lay RT/C scal en S. C yw Ax bs S CAERT/oCra FMSoph M-Frilumen Eantdo MicVantaCSC-lnNChe Avacyurife ZT BaCidERT. Vaadna Lanon InMaustpers B CietrnAnooofpoi Sec 15 onor DSchw Eing Id Da PaiddenJPCE ZIntellIiSgA U.Hone AR La d fa a Ka Se Op Pr Treitachi N Vu Inst P H Pan Gra C H ST H nal dus C o o i 04 x t E Na 3 a e DIVDeviceeitzerlipse ntitysaultlantiLr ayerT/CCcalerenceS. CGywell Axis s SG AMIT/CC FMSphos-Files ents atonMicrontaraC-NLheckvayarifera ZTEBaiduRT.PLaadinLabsn Inc.auticersky BDCitrixnolispoint m o c b e 2 s R R s A f E V ana ano M asp nC ASecu um rEen=d 2hi Va NCSVulMedian CMedian CEand=or1a So MnstrMedian ellIiSgA U.H=on0e AR La Median Sec onor DSchw Eing Id Da PaiddenJPCE ZIntMedian pen Proo K =3 af =C4 c O T r I a P T H 1H C P Chunk 4 l G s t S i a H 8 du ion Exo 0 Nat 7 a D ce er se ty lt tir er C er ce G ell is SG AMI /CC FMS hos Files ents ton icro tara -NL eck aya fera ZTE aidu T.PL adin abs Inc. utic rsky BD itrix olis oint T p - um Ea d M Van CSC lnCh Av curi R Va na L non Ma spe C enAnoofp o6me DIV evi eitz clip enti ssau lan Lay RT/C scallligenU.S=. Cn0eyw ALxabsMedian B CEMedian rMedian CEaRnd=or1a So MnstChunk e ISA Ho Sec 5 onor DSchw Eing Id Da PaiddenJPCE ZIntMedian fa =Ca4 p Pr en=ach2i N Vu Median R Ka Se = 3 r 4 a O A T r I P T H C P l G s t 8H S Hi na du 4 o o i x t E 7 Na 3 6 2 Fig. 15: Violin plot of the rest public CNAs 5 Median = 0 Median = 1 Median4 = 2 Median = 3 Median = 4 Chunk 1 48 0 37 t p s s r L t ! t i r g s k x c e e s S r k t g k y e n c s S e p r L Y s 26pe p ic m we Q LLC M le uc se or oc rr b io so go M ar ow eli ec ro ni ut ag pu PR Cu elt in om ke o SG u be SS E es Pup5NelteActroansysatemViestgreoSrtraa, fteroCpenElauck D oECoren.LifelLackBeerDanHuikvis Uni DrOapeneNalthrcviceN FAidppCh XPeanasoAbsoel Stor Octo HPYapearllo Worcepo Xia Dopch &TCechleCs GrreoctCy wolf ONEKWirel o F Cr 1 E g H e H F ra B Tian orto B Cyb Gly Gov Tha Di Pur L04G Pe Te P ter S Sier N Bax

C Z 47 it Mbishi G Cl tHubester n- Ku nson H B Hitach ePll Aut Sili Ope Gi W oh w 36 itsu J k M c Ro 25 18 14 Median = 0 Median = 1 Median3 = 2 Median = 3 Median = 4 03 Chunk 8 l I . t r r t r r 2 7 eaDIVDevice itze lipse ntitysaul lantiLaye T/CCcale enceS. CGywel Axis s SG AM T/CC FMSphos-Files ents atonMicrontaraC-NLheckvayarifera ZTE aiduRT.PLaadin Labsn Inc auticersky BDCitrix nolispoin m o e Ec Ide Das Pa en CER Zs llig U. one B CE V na no M asp b CER ora So M rum E nd i Va NCS lnC A ecu c D a w 1 e L r enAProof h e a t a S 6 ono Sc u d d g A f p t P e R K s h S H J V r n C d n S a n i c O i n I A T I r a I P H P G ST 0H Hita nal dus C 5 Exo atio N i I r r s s s c c k k y y x P e o e e e n h p S p g t B C C D G l 4sc an de ch rg wij tio ab d.i bl TR SR ra oD tri CS ng te IS ro or ro bi m or PH AM ad sti ghe SR oxa cure flare TML duct gital rista T-In AG achi c a R K t i t Bo3 iItvdefen Apcahi EnPeuiteurtomilaicon L Men Tena O E LifMeonghi Elec nN-Xchuaberne n ConHtashiC JF ZaCbhecZkohoc Bro ElGalla A MF-SeCloud ub prtoern D A CE SIC Hi r e K H Wes so1 t Rob 2 B Hita well A S Medianits=ub0is Ope Median = Median = 2 Median = 3 Median = 4 n i h G Chunk 4 Jo k M 8 Roc 1 7 0 6 Median = 0 Median = 1 Median = 2 Median = 3 Median = 4 58 ea IVD vice itzer lipse ntity ault antir ayerT/CC aler nce . CG well Axis s SG AMIT/CC FMS hos Chunk ilems ent3sEatonMicraontarSaC-NCLhecAkvayuarifera ZTEBaidEuRT.PLaadianLabsn Inc.auticersky BDCitriAxnolifspoint F m p e s c D y l e S L o e c b e s R R s a o g . e a i c r M D n a E d n o C Vafan Cano MKasp CEando S nstru Trendchi V NC Vuln Sec Se 47 onor Schw ing I Da Pidde JPCE ZIntellISA U Hon AR L Ope Pro r I a P T H C P l G s t H3 S i a u H d 6 ion Exo Nat 25 14 Median = 0 Median = 1 Median4 = 2 Median = 3 Median = 4 03 Chunk 8 t t ! t i r t r r r 72pe pp ics ms we QL LLC MS le uck se org ock rry be ion soc gos MS are ow elis eck rox nic ute age pus PR Cu elt in om ke og SG up be SSL EY ess Pup61 NelteActroansysatemViestgreoSrtraa, fteroCpenElauck D oECoren.LifelLackBeerDanHuikvis Uni DrOapeneNalthrcviceN FAidppCh XPeanasoAbsoel Stor Octo HPYapearllo Worcepo Xia Dopch &TCechleCs GrreoctCy wolf ONEKWirel o i y r e v F r o g a B F H E H l n ra B b e C G Go Th D Pu Cy Tia Nort L50G Pe T P ter S Sier Bax 4 a D ce er se ty lt tir er C er ce G ell is s SG AMI /CC FMS hos Files ents aton icro tara -NL eck aya fera ZTE aidu T.PL adin abs Inc. utic rsky BD itrix olis oint T ra Soof h v ri p Mthe - rum rest C nAn ofp M Van CSCCNAs ome DIV evi eitz clip enti ssau lan Lay RT/C scal en S. Cneyw16:ALxabViolin B CER Va na L non Maaspe E nd public o CER plot i N VulnC ASecu Sec 3 onor DSchw Eing Id Da PaiddenJPCE ZIntellIiSgA U.Fig. Ho Median Ope Pro rafa CaMedianK= 5 In=st2 TreitachMedian P TAR = 1 Pand Median H Median C l G s = 0 = 3 Median = 4 H2 S a u d ion Chunk 5H 8 Exo Nat 1 7 0 6 Median = 0 Median = 1 Median =2 Median = 3 Median = 4 58pet pp ics ms wer QL LLC MS ler uck set org ock rry be ion soc gos MS are Chunk ow eli4s eck rox nic ute age pus PR Cut elt! int omi ker og SG up ber SSL EY ess p u P 4 NelteActroansysatemViestgreoSrtraa, fteroCpenElauck D oECoren.LifelLackBeerDanHuikvis Uni DrOapeneNalthrcviceN FAidppCh XPeanasoAbsoel Stor Octo HPYapearllo Worcepo Xia Dopch &TCechleCs GrreoctCy wolf ONEKWirel o F Cr 7 E g H e H F ra B Tian orto B Cyb Gly Gov Tha Di Pur L36G Pe Te P ter S Sier N Bax 25 14 Median = 0 Median = 1 Median =Chunk 2 5 Median = 3 Median = 4 Median = 5 03 8 27 OR are ile int ust vs lys MS HP ier bs SR ak SO yte IA ua ard ces we ce O2 on sec nix ze ells LC kta ITY rks ME tis arx on all ch ion ent era ile r a a s I ST tw ob ilPo dTr oDe ua e C CaLrnet L ilAver Pe ZYUugab IDEM DathchGeu Devi ZAopplian WSErics ciro PhoePentr BaitcCMS L ODFINNetwo KN Mirhaenckm PySthonicWLogirtpeoraot cum TPigO Mob ASoUw 61SoVf ivo M SBaeyon Her oQncret l o D OP a o S N o C g d t & W o a C LY C C Sn 50 Go g TV d-Disagreement Metric d-Disagreement d-Disagreement d-Disagreement d-Disagree d-Disagreement Metric Metricd-Disagreement Metric d-Disagreement Metric Metricd-Disagreement Metric d-Disagreement Metric Metricd-Disagreement Metric d-Disagreement Metric

e Rob

sun 4pet App nics ems wer SQL LLC MS uler uck Eset .org ock erry ube sioSnaim ocagos NMS care Nowidelisheck eroxsonicolute rageopusHYPRerCutWelt!pointiaomoi cker Cog CSG roup yberlfSSLEKEYeless s C p L i t , e t n D E e n e o B i Pu 3 Nelectr asysamV stgr ortraafteropen lack oCor nLifelack erDa Hikv U DrOpenealthrvice FAppC XPana Abs e Sto Oct Pap allo orce X D ph &Techles GrectC wo ON Wir o F Cr E g ra B Tian Median to B=Cy0b Pur Median = 3 H F Median G=ly6Gov Tha Di Medianxt=er 1H Se Chunk Median6 = 2 L2G Pe Te P Sier Nor Ba 8

1 7 0 6 Median = 0 Median = 1 Median = 2 e 25 nMedian c x= 3e s C Median x n = 5ll h n t a e a Y= 4s E s Median 58TOR are bile oint rust evs alys CMS HP rrier Labs ASR eak USO byte MIA hua uard ices owe Chunk P Z ga IDE Da hG Dev Z pliancWSOricssocirosPehoeneintraBzaiceMll S LL OkDtFINeITtworkKNIMMiranctikmaPr ythonicWoagitecoraticoumenTigerMobil w Mo ailP ndT roD Qu te a S t t r f C U e e o S u e e E o c p P dotC Y N A w 47S Viv SBeyo H oncr So L Corp Do OPPO Silv NLn Che WGaot ogle TV & A ato C LY C Sno 36 ng u s 25 Sam 14 MedianFig. = 0 17: Violin Median =plot 1 of the Median 2 Median = 3 Median = 6 03 rest CNAs Chunk 6 = public 8 27 ies ion tics pe ral orn va ble ron ars rch nic mp illa elis BU um int ted ms rity ec MC ce oo tix ara AN ab LC SE ne org tics abs ple OS ket tec are io g is p ko po lb no -a tI e a ro u z f N ll Po i or u oS B an d p y L VL L n to l. o L p d c n w um nowloh61 a VSyna Nets Tem HGaE Ver N Sof 42AGReseMedt Tcpd Mo Pro Cybheeckrm LaimPlatrfn Sec AlgOpeignn Alli wOork O Pa ESECf-twar.eBrau HillDs rupaas RoUbLSec nACloeuen RoSymMa Soft Ill h c C A et nte e n T So B MI SB es Ali V Net Ope Gr Mk La CA Ha 50 era nD x e e c p l a F l O 4 R re le nt st s s S P er s R k O te IA B a rd s e e 2 n c ix e lls C ta Y ks E is rx n ll h n nt ra le STO twa obi ilPoi dTru oDev ualye CM HCarnriet Lab AveSr Pea ZUuSgabyIDEM DahchuGuaDevice ZopwpliancWSEOricssocirosPehoenentraBzaicCeMS LL OkDFINeITtwor KNIMMiraenctkmaPythoonicWLoagitpecoraticoume Tige Mobi P dot l Y t ASoUw 3SoVf ivo M SBaeyon Her oQncret S L i e A l or Do OPPO S N oN Ch WGaoog TV & C LY C Cat Sn 2 g n su 1 Sam 0

d-Disagreement Metric

Median s n s e l n a e n s Median y c6 =C2 e o Median x a = 3 b C Median E e =g 6 s s e S t c e o h c= 0p a s Median U = 1t d s Chunk 8gieVisioapticskop pora lbor rnov -abl ftIro Gear earc troni dum ozillrofeli NB ellumPoinmite formcuritgoSe nBM lianc Odo Opti ayar ELAN-VLa e LL un Slston al.orbotic Lab Applud O ockeante twarllumi o l r a i t l N o t l s l k P b M e a e C d e p m o 2 k i n P L a a r S 4 Re Me Tc n ha y Ne Te HGE V CyChecArm ta Pl ern S A Opsign A twor SEoftw B. B H Drulipas RoULSec enCloreen RSymIM Sof I Tech nw7 S e S M SBA e A V N Op G a Mke Lant D r CA Ha 6 en exe c p l a F l O B 5 4 3 2 1 0

es on cs pe ral rn va le on rs ch ic p lla lis U m int ed s ity ec C ce oo tix ra N ab LC SE ne rg cs bs le S et ec re io lohgai Visyinaptei tskoempoHalbVoerno N-aSboftI4r2GeeaseaerdtrTocnpdumMoziProfe NyBbeelluck PoLimiltatforSmecurAlgoSpenBMAllian Odrk Op Paya ELEAC-VLare LraunHillstroupal.RooboStei c La ApploudnORocykmanStoftwaIllum o n R M N T C Ch Arm ta P ern S oftw B. B O sign etwo D lias VUL penCGree SMIM GE Tech nw S S SBA e A N O a Mke Lant D r CA Ha e n Flex Ope Blac

Fig. 18: Violin plot of the rest public CNAs