Why Not Fix It Once and for All? An Empirical Study of Multiple Patches for Vulnerability Fixes in Open-Source Software Weiliang Qi, Youpeng Li, and Xinda Wang
arXiv:2607.13206v1 [cs.CR] 14 Jul 2026
University of Texas at Dallas {weiliang.qi,youpeng.li,xinda.wang}@utdallas.edu Abstract. Security patches for open-source software constitute a foundational resource for vulnerability remediation research and practice. However, analyzing and applying multiple patches remains challenging, especially when trying to determine at what point in a patch sequence a vulnerability is fully remediated. This paper presents a systematic analysis of multi-patch vulnerability fixes, focusing on their root causes, characteristics, and methods for verifying remediation status throughout the fixing process. Through a manual examination of 1,646 multi-patch fix records, we develop a taxonomy with three primary categories and six subcategories based on their causes. We then compare the distinctive characteristics of multi-patch fixes with those of single-patch fixes and analyze feature variations across categories. In addition, we assess representative vulnerability detection methods for validating complete remediation during multi-patch fixing. Our findings provide new insights into multi-patch fixes and lay a foundation for future research in this field. Keywords: Security Patch · Open Source Software · Vulnerability Fix
1
Introduction
To mitigate security risks in open-source software (OSS), vulnerability databases collect and publish vulnerability analyses together with the corresponding security patches. These patches serve as essential resources for both developers seeking to remediate vulnerabilities and researchers conducting vulnerability analysis. Ideally, a vulnerability is fully resolved by a single patch, leaving the software secure once the patch is applied [9]. In practice, however, vulnerabilities are often addressed through multiple patches (multi-patch), which complicates the remediation process and may produce misleading outcomes [11, 30]. In such cases, remediation proceeds incrementally, with different patches addressing the root cause, edge cases, or newly discovered side effects. As a result, intermediate versions may represent only partial fixes, making it difficult to determine when the vulnerability has been fully resolved. This ambiguity can lead to inaccurate assessments, noisy datasets, and flawed downstream decisions. For example, vulnerability datasets are often constructed by reversing security patches to recover vulnerable code [2,11,13]. When a fix consists of multiple patches, however, reversing a later patch may yield an intermediate state
2
W. Qi et al.
rather than the original vulnerable version. Such code can exhibit both pre-patch and post-patch characteristics, making its security status difficult to determine. Naively labeling it as vulnerable or secure may therefore distort the dataset. Therefore, a systematic understanding of multi-patch scenarios is essential for improving vulnerability remediation and analysis in practice. Although patch analysis has received substantial attention in the security community, multi-patch fixing remains underexplored. Some works [47,59] examine security patches in general, summarizing their characteristics and detection methods, while others [51, 54, 65] focus on identifying and extracting security patches, such as detecting overlooked patch behavior in silent fixes. While a few studies acknowledge the existence of multi-patch fixes, they either do not provide detailed analysis [30] or restrict their investigation to specific application domains [26, 55, 56]. In this paper, we conduct a systematic study of multi-patch fixes in OSS to provide insights into this problem. Based on vulnerability patches recorded in National Vulnerability Database (NVD), we manually analyze 1,646 multi-patch fixes affecting open-source software between 1999 and 2025. We first summarize their root causes and key characteristics, and then examine the feasibility of using existing vulnerability detection techniques to determine whether the initial patch fully remediates the vulnerability. Our study is organized around following three research questions (RQs): RQ1: Why are multiple patches needed to fix software vulnerabilities? Although repository management best practices, such as those in Git, recommend resolving each issue with a single commit [10, 46], many vulnerabilities are still fixed through multiple patches. To better understand this phenomenon, we manually analyze OSS multi-patch vulnerability samples and classify them into three main categories: multi-location fixes, fix-and-surrounding work, and defective fixes. We find that the main causes are the complexity of vulnerability contexts and remediation processes, such as vulnerabilities affecting multiple code locations or branches, which often lead developers to apply several patches. Another common cause is that the initial patch is incomplete or defective, requiring follow-up patches to complete the remediation or fix newly introduced issues. This taxonomy provides a foundation for understanding multi-patch fixing and may help developers choose more appropriate responses in practice (§4). Our manual analysis also reveals several recurring factors associated with multipatch fixes (§5), including gaps between Common Vulnerabilities and Exposures (CVE) reporting rules and practice, Git workflow conventions, limited guidance on reporting security patches, pressure for rapid incident response, excessive reliance on senior committers, and vendor neglect. RQ2: What are the characteristics of multi-patch fixes? We compare multi-patch fixes with single-patch fixes in terms of their characteristics (§6). We find that multi-patch fixes are concentrated in widely used programming languages, are more prevalent in certain projects, and show a more pronounced upward trend over time. We further explore the distinctive features of the three multi-patch types. Notably, 31.7% of multi-patch fixes involve intervals of more
An Empirical Study of Multiple Patches for Vulnerability Fixes
3
than one day between consecutive patches. In addition, we analyze differences in code similarity and complexity across these types. RQ3: How well do existing methods handle patch management in multi-patch contexts? Given the large number of multi-patch fixes in OSS and the time gaps between patches that may create opportunities for attackers to conduct one-day exploitation, it is important to determine whether an initially released patch is sufficient to fully remediate a vulnerability or will require followup patches. To this end, we evaluate whether seven representative vulnerability detection techniques can predict whether the first patch will later evolve into a multi-patch fix (§7), focusing on two common causes: incomplete fixes and multilocation fixes. Our results reveal the practical need for, and provide insights into, developing more effective methods to identify and manage multi-patch fixes. In summary, this paper makes the following contributions: – We construct and open-source1 a manually labeled dataset of multi-patch vulnerability fixes to support future research. – We conduct the first systematic empirical study of multi-patch fixes and analyze their main causes through representative case studies. – We investigate the characteristics of multi-patch fixes compared with singlepatch cases, and provide insights from the perspectives of temporal patterns, project and language distributions, and patch-level properties. – We assess existing methods for verifying remediation status in multi-patch settings and identify directions for future research.
2
Background
Open-Source Software Patches. In OSS-hosting systems such as Git, a software patch (i.e., a commit identified by a commit ID) is a change submitted to fix, update, or improve the source code of a project. These patches can be categorized into two main types: (1) security patches, which address security vulnerabilities and are intended to enhance system security; and (2) non-security patches, which fix bugs, functional issues, and compatibility problems unrelated to security, as well as add or optimize features. Public Vulnerability Records. CVE [49] is the most widely used public catalog of known security vulnerabilities, with each vulnerability assigned a unique CVE ID. The NVD [37] extends CVE records with additional information like severity scores, impact assessments, and fix-related data. Notably, security patches linked to CVEs are explicitly labeled as “patch” on NVD pages. Although a patch is typically expected to resolve a single vulnerability, we observe that many NVD entries reference multiple security patches for the same CVE. Given that CVE/NVD are widely used by cybersecurity professionals and researchers to identify, track, and analyze vulnerabilities, we use them as the primary data sources for this study. 1
The dataset MultiPatch
is
available
at
https://huggingface.co/datasets/XSec-Lab/
4
W. Qi et al.
Security Patch Management. OSS security patches linked to CVE records support vulnerability remediation in two main ways. First, patches are typically provided as Git commit URLs, enabling users to identify the exact fixed version from the commit ID and update their local code accordingly. Second, for users unable to perform a full upgrade, the relevant security changes can be applied directly using git apply or git cherry-pick. For example, as shown in Fig. 1, an OSS vendor recommends that users who cannot upgrade directly apply specific security patches instead. While applying a single patch is usually straightforward, multi-patch fixes for a single vulnerability introduce additional challenges, as discussed later. Advisories: GHSA-mxr5-mc97-63rc Impact: Attackers can request a password reset and then gain access using a specially crafted request. To exploit this vulnerability, attackers must know the username of an administrator and have access to the password reset form Patches: Issue has been patched in Build 472 and v1.1.5 Workarounds: Apply library@016a297 and library@5bd1a28 to your installation manually if it is unable to upgrade.
Fig. 1. The vendor’s advisory on applying security patches for CVE-2021-32648
3
Data Collection
To understand why multiple patches are needed to address a single vulnerability and how this affects practical vulnerability management, we first construct a large-scale dataset of multi-patch vulnerabilities based on CVE records. We use the NVD JSON feeds API to collect all published CVEs as of May 2025. We then focus on CVE entries that contain reference links explicitly labeled as “patch.” Because not all referenced patches correspond to OSS projects with accessible source code, our study considers only OSS security patches whose URLs contain the keywords git and commit. Since OSS projects may be hosted and maintained across multiple version control platforms (e.g., GitHub and GitLab), different patch URLs may in fact refer to the same patch. Therefore, to distinguish multi-patch cases from single-patch cases accurately, we do not simply count the number of patch URLs associated with each CVE. Instead, we parse the commit IDs embedded in these URLs and use them to identify and de-duplicate patches. Finally, retrieving patches from the referenced links poses two challenges. First, because our records span 26 years, some early patches are no longer hosted at their original repository locations, making the original links obsolete. Second, different Git hosting services use distinct APIs, which complicates large-scale data collection. To address these issues, we manually map outdated repository addresses to their current locations and develop scripts for common platforms such as GitHub, cgit, and GitLab. Specifically, we use the GitHub API to retrieve patch metadata and code changes, while for other platforms we download and parse raw patch files. In total, we identify 25,113 CVEs associated with OSS security patches, of which 1,646 are linked to multiple patches in the NVD, with an average of 2.55 patches per CVE.
4
Categorization of Multi-Patch Fixes
This section presents a systematic manual analysis of multi-patch fixes and introduces a corresponding categorization. The analysis was conducted by three
An Empirical Study of Multiple Patches for Vulnerability Fixes
5
security researchers, each with more than five years of experience in software security. First, they independently examined 100 samples randomly selected from our collected dataset and developed a preliminary taxonomy. They then discussed and refined it into a final codebook. Using this codebook, each researcher independently categorized all samples, and disagreements were resolved through discussion until consensus was reached. In this way, we identify three main categories and six subcategories of multi-patch fixes. Table 1 provides an overview of the categories and their distribution. Note that one multi-patch CVE record may belong to more than one category. Also, beyond these categories, we find 127 CVE records that include both vulnerability-introducing and vulnerabilityfixing commits. Because the NVD does not distinguish vulnerability-introducing commits with a separate label, both types may be marked as patches or left unlabeled. This ambiguity can confuse users applying patches and complicate security patch collection for researchers [7, 30, 58]. Table 1. Multi-patch categories and numbers of CVEs records as of May 2025. Category
Subcategory # A1. Different branches or projects 830 A. Multi-location fixes A2. Different locations within same branch 30 B1. Workaround and formal fix 16 B. Fix and related changes B2. Fix and documentation updates 128 C1. Incomplete fixes 641 C. Defective fixes C2. Bug-introducing fixes 119
4.1 Category A - Multi-Location Fixes Applying multiple patches to different locations, with each patch addressing one location, is the most common multi-patch scenario. It typically occurs when similar vulnerable code appears in multiple methods, branches, or even separate projects. Based on the scope of the affected locations, we further divide this category into two subcategories. A1. Fixes for Multiple Locations Across Different Branches or Projects. When a vulnerability affects multiple branches or projects, the software vendor needs to address each affected instance individually. Even when similar functionality is implemented, the code context across branches may differ. In addition, Git cannot apply a single commit to multiple branches or repositories simultaneously. As a result, the vendor needs to cherry-pick or otherwise adapt the fix for every branch that requires remediation, which leads to multiple patches. 1 lib/compress/evaluate.js 2 @@ -129,7 +130,7 @@ def_eval( 3if (evaluated === undefined) { 4+ if (evaluated === undefined && 5+ regexp_is_safe(this.value.source)) { 6 lib/compress/index.js 7 @@ -2140,6 +2141,7 @@ def_optimize(...) { 8+ && regexp_is_safe(params[0]) 9 ){ let [ source, flags ] = params;
1 lib/compress/index.js 2 @@ -2819,7 +2820,7 @@ var static_fns = 3if (evaluated === undefined) { 4+ if (evaluated === undefined && 5+ regexp_is_safe(this.value.source)) { 6 7 @@ -5089,6 +5090,7 @@ def_optimize(...) { 8+ && regexp_is_safe(params[0]) 9 ) { let [ source, flags ] = params;
Patch 1 (a4da7349) for Teaser version 5
Patch 2 (d8cc5691) for Teaser version 4
Fig. 2. Patches for CVE-2022-25858: multi-location fixes across different branches.
Fig. 2 shows patches for CVE-2022-25858, where separate patches were required for different branches. Teaser versions 4 and 5 are maintained in the same repository on different branches. Because a single commit cannot be applied to both
6
W. Qi et al.
branches simultaneously, the fix was first implemented in version 5 and then backported to version 4. Although the two patches follow similar logic, version 5 (left) separates compression and evaluation into two files, while version 4 keeps them in one file. Thus, the patches are similar in intent but differ in structure. A2. Fixes for Different Locations within the Same Branch. Even when a vulnerability is confined to a single branch, vendors may adopt a multi-patch strategy if the vulnerable code appears in multiple locations. This usually occurs for two reasons. First, vulnerability reports often identify only one attack path, leading vendors to patch that location first, while later analysis reveals additional vulnerable sites. Second, Git best practices [10] discourage large single commits, so vendors may split fixes into smaller commits targeting different locations. In such cases, all patches must be applied to fully remediate the vulnerability. Fig. 3 illustrates three commits for patching CVE-2012-6537 caused by uninitialized structures that lead to an information leak. Each patch adds explicit initialization with memset(0) to a different function containing similar vulnerable code, although all three functions are located in same file. While three commits use same initialization pattern, their parameters and code contexts differ. 1 @@ -689,6 +689,7 @@ copy_to_user_state( 2+ memset(p, 0, sizeof(*p));
1 @@ -689,6 +689,7 @@ copy_to_user_tmpl{ 2 for (i = 0; i < xp->xfrm_nr; i++) { 3 ... Patch 1 (f778a636) for copy_to_user_state() 4+ memset(up, 0, sizeof(*up)); 5 memcpy(&up->id, &kp->id, 6 sizeof(up->id)); 1 @@ -1320,6 +1320,7 @@ copy_to_user_policy( 7 ... 2+ memset(p, 0, sizeof(*p)); 8 } Patch 2 (7b789836) for copy_to_user_policy()
Patch 3 (1f86840f) for copy_to_user_tmpl()
Fig. 3. Patches for CVE-2012-6537: fixes in multiple locations within the same branch.
Insights for Applying Multi-Location Patches. Multi-patch fixes for multi-location vulnerabilities follow the same basic rationale as single-patch fixes, but are split or ported across locations. For fixes within the same branch, all patches must be applied for complete remediation. For fixes across branches, users should apply the patch that matches their local repository configuration. 4.2
Category B - Fix and Related Changes
Some vulnerabilities require complex remediation and therefore involve progressive solutions implemented through multiple patches. In urgent cases, vendors may first release a workaround to reduce immediate risk before providing a complete fix. In large projects, development practices may also require changelogs, test cases, or other documentation to be maintained separately, leading to additional commits beyond the security fix itself. B1. Workaround and Formal Fix. Multi-patch fixes may consist of an initial workaround followed by a formal fix. The workaround serves as a temporary measure to quickly reduce exploitation risk before a complete fix is ready, while formal fix addresses root cause thoroughly. This strategy is because a full fix may take days or months, whereas immediate containment is often critical after disclosure. Workarounds also tend to have less impact on business logic, reducing risk of introducing new issues. Although both help mitigate risk, the permanent fix is generally more reliable and should be regarded as the final solution.
An Empirical Study of Multiple Patches for Vulnerability Fixes
7
Fig. 4 shows a workaround followed by a formal fix for CVE-2023-4226, an arbitrary file upload vulnerability in Chamilo LMS that can enable command execution, shell access, and stored XSS. The workaround patch (left) mitigates immediate exploitation by restricting browser parsing of risky files and blocking direct access to upload directories. The formal fix (right), released 16 days later, uses disable_dangerous_file to filter uploads and remove dangerous files. 1 @@ -117,6 +117,10 @@ AddType /font-woff 2 #</IfModule> 3+ # Disallow direct access to /main/lib/javascript/bigupload/files 4+ RedirectMatch 403 ^/main/lib/javascript/bigupload/files 5 @@ -113,3 +113,7 @@ AddType /font-woff 6 #</IfModule> 7+ <IfModule mod_headers.c> 8+ Header always set X-Content-TypeOptions nosniff 9+ </IfModule> Workaround (5ff3a07d and 6f32625a)
1 @@ -27,9 +27,12 @@ 2 foreach ($fileList as $n => $file) { 3- $tmpFile = $tempDirectory.$file['name']; 4+ $tmpFile = disable_dangerous_file( 5+ replace_dangerous_char($file['name']); 6 file_put_contents( 7$tmpFile, 8+ $tempDirectory.$tmpFile, 9 fopen($file['tmp_name'], 'r'), 10 FILE_APPEND); Formal Fix (640d391f)
Fig. 4. Patches for CVE-2023-4226: workaround and formal fix.
B2. Fix and Documentation Updates. Project development practices may require changelogs, test cases, or other documentation to be maintained separately, which leads vendors to commit these changes independently of the security fix itself. Users may either apply all related commits or cherry-pick only the security fix commit to address the vulnerability. However, when conducting patch analysis, it is necessary to carefully distinguish security-relevant commits from non-security-related ones to avoid introducing noise into the results [7, 36, 52]. In Fig. 5, vendor uses one patch to validate rdp pointer address and prevent outof-bounds reading, while a separate commit documents this change in changelog. 1 @@ -816,6 +816,9 @@ security_fips_decrypt( 2 size_t olen; 3+ if (!rdp || !rdp->fips_decrypt) 4+ return FALSE; 5 if (!Cipher_Update(rdpfips_decrypt) 6 return FALSE; Fix Patch (d6cd1405)
1 @@ -1,3 +1,21 @@ 2+ # 2020-05-20 Version 2.1.1 3 ... 4+ +* CVE: GHSL-2020-101 OOB Read in security_fips_decrypt due to uninitialized value Changelog commit (8fb6336a)
Fig. 5. Patches for CVE-2020-13397: fix and documents.
Insights for Applying Fix and Related Change Patches. Distinguishing related patches from actual vulnerability-fix patches is important for both practice and research. For software users, workarounds can reduce risk but may also disrupt normal functionality, such as blocking access to a vulnerable API. Once a formal fix is available, continued reliance on a workaround may therefore cause unnecessary disruption. Related patches should also be excluded from vulnerability analysis tasks. For example, in vulnerability bisection [1, 62], documentation updates may lag behind the actual fix and obscure the true fixing point. In vulnerability code dataset construction [2, 5, 13], documentation-related commits may be mislabeled as vulnerable samples. Excluding such related patches is therefore necessary to avoid introducing noise into the analysis. 4.3 Category C - Defective Fixes Defects in the patch itself may also lead to multiple patches. In such cases, the initial patch may fail to fully eliminate the vulnerability or may introduce new vulnerabilities or bugs that require subsequent fixes.
8
W. Qi et al.
C1. Incomplete Fixes. Ideally, vendors should implement precise vulnerability remediation, which requires a thorough understanding of both the vulnerability and the software architecture. In practice, however, the interaction between root causes and software logic often makes correct fixes difficult to develop. As a result, an initial patch may fail to fully resolve the vulnerability, leaving it to persist until a follow-up patch is released, sometimes after a substantial delay. Such incomplete remediation poses challenges for vulnerability management and software maintenance. Once vendors recognize that the initial patch is insufficient, they typically issue an additional fix, which users should apply in commit order to ensure complete remediation. Fig. 6 presents CVE-2012-0038, which involves an integer overflow in count in the Linux XFS file system. The first patch (left), fa8b18ed, added a maximumvalue check and exception handling. However, it overlooked that be32_to_cpu() returns a 32-bit unsigned integer, while count was declared as signed. As a result, count could be interpreted as negative in some cases and bypass the check. The vendor later released a second patch (right) that changed count to an unsigned integer. Notably, the complete fix was delayed by 18 days. Although the CVE had not yet been publicly disclosed, the initial patch in the open-source repository could still have exposed clues to attackers. 1 @@ -42,6 +42,8 @@ xfs_acl_from_disk( 2 count = be32_to_cpu(aclp->acl_cnt); 2+ if (count > XFS_ACL_MAX_ENTRIES) 3+ return ERR_PTR(-EFSCORRUPTED); 4 acl = posix_acl_alloc(count, GFP_KERNEL); Initial Failure Patch (fa8b18ed)
1 @@ -39,7 +39,7 @@ xfs_acl_from_disk( 2 foreach ($fileList as $n => $file) { 3- int count, i; 4+ unsigned int count, i; 5 count = be32_to_cpu(aclp->acl_cnt); Later Complete Patch (093019cf)
Fig. 6. Patches for CVE-2012-0038: incomplete fixes.
C2. Bug-Introducing Fixes. A defective initial patch may pose an even greater risk by introducing new bugs, including security flaws (i.e., vulnerabilities). This can create the illusion that the original issue has been fully resolved, even though the patch has introduced new problems. Vendors must then issue additional patches to remediate them. Fig. 7 are patches for CVE-2018-7191 in the Linux kernel, insufficient validation of the device name before calling register_netd allows a device with illegal characters to trigger a NULL pointer dereference and kernel panic, causing a denial of service. The initial patch (left) attempted to fix this by adding a device-name check. However, the developer misinterpreted the return value of dev_get_valid_name(), assuming that only 0 indicates success, whereas any value >=0 does. As a result, the check on err rejected most valid device names and introduced a new denial-of-service issue. A later patch (right) corrected the validation logic and fixed the new vulnerability. 1 @@ -2027,6 +2027,9 @@ 2+ err = dev_get_valid_name(net, dev, name); 3+ if (count > XFS_ACL_MAX_ENTRIES) 4+ if (err) 5+ goto err_free_dev;
1 @@ -2028,7 +2028,7 @@ 2 err = dev_get_valid_name(net, dev, name); 3- if (err) 4+ if (err < 0) 5 goto err_free_dev;
Initial Defect-Introducing Patch 0ad646c8
Following Patch 5c25f65fd
Fig. 7. Patches for CVE-2018-7191: defect-introducing fixes.
An Empirical Study of Multiple Patches for Vulnerability Fixes
9
Insights for Handling Defective Fixes. For vendors, testing a security fix should verify not only that the PoC can no longer be triggered and that normal functionality is preserved, but also that the fix does not introduce new bugs or vulnerabilities [26, 56]. OSS users should apply patches sequentially in chronological order to ensure complete remediation. For vulnerability research, defective fixes pose a distinct challenge. Prior work [2, 5, 11, 13] often assumes that code after a patch is non-vulnerable and uses it as ground truth. However, code produced after a defective fix may still contain vulnerabilities, making it unsuitable as a reliable non-vulnerable sample. Therefore, although these cases are all recorded as multi-patch fixes in the NVD, they should still be distinguished by category so that appropriate measures can be taken.
5
Contributing Factors Behind Multi-Patch
Based on our observations, we identify and summarize the main potential causes of multi-patch fixes as follows. Gap Between CVE Rules and Practice. The CVE Board defines rules for assigning CVE IDs. However, ambiguous terminology in these rules may lead CVE Numbering Authorities (CNAs) to assign overly broad CVE IDs, which can in turn result in multi-patch fixes. For example, CNA Rules 3.0 [9] specify the following requirements for vulnerabilities affecting multiple projects: “If multiple Products are affected by the same Independently Fixable Vulnerability, then the CNA: (1) MUST NOT assign more than one CVE ID if the Products are vulnerable because they share the vulnerable code. The assigned CVE ID will be shared by the vulnerable Products. (2) SHOULD assign different CVE IDs if the Products do not share vulnerable code. (3) SHOULD assign different CVE IDs if the CNA is uncertain whether the Products share vulnerable code.” However, determining whether two projects share vulnerable code is often difficult. Two projects may originate from the same third-party library or common branch, but diverge over time, making it unclear whether their vulnerable code should still be considered “shared”. For example, CVE-2024-1394 [42] affects both the openssl package maintained by the Golang FIPS team and Microsoft’s go-crypto-openssl package. Although both originate from the same OpenSSL codebase and share a memory leak, the Golang FIPS package contains an additional affected location due to platform differences. Despite these differences, the two projects were assigned the same CVE ID, and each provides its own patch, resulting in a multi-patch fix. Inconsistent Git Commit Granularity Practices. Although Git practices vary across domains, the community generally agrees that commits should be as granular as possible while remaining logically coherent [10]. Even for the same vulnerability, developers may split changes into separate commits when they affect different modules, since smaller commits are easier to manage and roll back. Because most NVD patches correspond to individual commits, this practice can naturally result in multi-patch fixes. For example, the fix for CVE-202335852 [40] consists of two patches: one blocks abnormal absolute paths, and the
10
W. Qi et al.
other adds a configuration option for finer-grained permission control. Although such multi-patch fixes support healthier development practices, they also require clear guidance on patch application for effective security management. Lack of Guidance on Reporting Security Patches. The CVE Board sets standards for references in CVE records, such as public accessibility and longterm availability [9], but does not specify which patches should be documented. Developers may include patches unrelated to the security fix when submitting references to CNAs. For example, the record for CVE-2018-8729 [38] contains one real fix and one patch that only updates the project’s README. Our statistics show that 5.8% of multi-patch cases contain such unrelated patches, including changelogs, version updates, test cases, and README changes. These extraneous records increase the cost of patch analysis and introduce noise into automated data collection [11,13,30]. CVE-2023-40173 [41] record includes two patches: one creates a salting table and changes the default password, while the other updates the relevant PHP logic. However, they are not adjacent in the commit history. An unrecorded intermediate patch modifies the SQL file and sets the email field in the user table as a unique key. Because the patched PHP code validates the email field while the pre-fix code does not, applying only the recorded patches via cherry-pick may lead to logical inconsistencies. In sum, we identify two recurring problems: CVE records may include patches unrelated to the security fix or omit prerequisite patches needed for correct application. Addressing these issues requires clearer guidance on which patches to record and how to represent patch dependencies and application context. Over-Reliance on Vendor Committers. In almost all multi-patch cases where first patch introduces a new vulnerability, the committer was already highly experienced at the time of submission, typically a founder, core developer, or early collaborator. This finding suggests that even senior contributors to large-scale projects can make mistakes, underscoring the need for rigorous code review. Notably, no CVE issued after 2020 falls into this category. This shift coincides with CVE Board’s stronger emphasis on principles for assigning new CVE IDs in CNA rules. In earlier versions [8], CVE ID assignment placed greater weight on software vendors’ opinions, which may have made vendors more inclined to assign fewer CVEs to their products, potentially due to reputation concern [32, 53]. Insufficient Vendor Maintenance of Patch Records. We find that vendors sometimes respond passively to patch maintenance, missing opportunities to correct erroneous entries. For example, in CVE-2022-2522 [39], one of the two recorded patches is actually the parent of the real security patch and is unrelated to the vulnerability fix. As shown in Fig. 8, a user pointed out this issue, but the vendor declined to correct it, leaving the incorrect patch in the NVD record. Although maintaining such records can be tedious, timely corrections to opensource vulnerability databases are important to the health of the community.
6
Characteristics of Multi-Patch Fixes
This section examines the characteristics of multi-patch fixes. We first compare single-patch and multi-patch fixes across programming language, project, vul-
An Empirical Study of Multiple Patches for Vulnerability Fixes
11
Discussion on CVE-2022-2522 User: There is a problem with the associated patch. It should be patch 9.0.0060 not patch 9.0.0061, but the link given above is the link of patch 9.0.0061. The patch given by NVD is also patch 9.0.0061, which should be problematic. Vendor: I guess the right commit is b9e71736. I never use those hex numbers, we use human-readable numbers. CNA: Would you like me to update the report and CVE? Vendor: Whatever. I only care about fixing problems, I don't care much about the bookkeeping.
Fig. 8. A discussion between OSS vendor and CNA
nerability type, and publication year. We then analyze the time intervals and similarities across different categories of multi-patch fixes. Language Distribution. Table 2 presents statistics for the top-10 languages involved in single- and multi-patch fixes. These languages account for a larger share of multi-patch fixes, suggesting that such fixes are concentrated in widely used languages [50]. However, this result may be influenced by data source bias. Projects written in widely used languages often receive stricter maintenance [44], and their vendors may be more likely to split fixes into smaller commits. In addition, these languages are supported by more security analysis tools [6], making incomplete fixes easier to detect. Overall, these findings suggest that multi-patch fixes in open-source databases are concentrated in widely used languages, while incomplete fixes may still remain widespread in practice [55]. Project Distribution. Table 2 shows the distribution of single-and multi-patch fixes across the top-10 projects. Different projects exhibit clear preferences for one fixing pattern over the other. For example, Linux has 1,415 single-patch fixes but only 84 multi-patch fixes, corresponding to one multi-patch fix for every 17.8 vulnerability fixes. In contrast, ImageMagick rises from seventh place in singlepatch fixes to second place in multi-patch fixes, with one multi-patch fix for every 3.2 vulnerability fixes. This pattern appears to stem primarily from its maintenance practices: ImageMagick simultaneously maintains versions 4, 5, and 6, so a single vulnerability often requires separate patches for each version. OpenSSL and CPython show similar patterns, suggesting that maintaining multiple active versions increases the likelihood of multi-patch fixes. Table 2. Top-10 programming languages and projects in single- and multi-patch fixes 1 2 3 4 5 6 7 8 9 10
Single-Patch C 29.91% PHP 19.94% JavaScript 9.53% Python 8.63% C++ 6.04% Java 4.21% Go 4.20% TypeScript 3.56% Ruby 2.62% Rust 0.85%
Multi-Patch C 32.02% PHP 20.21% JavaScript 11.68% Python 10.54% Java 7.45% Go 5.93% C++ 5.56% Ruby 5.05% TypeScript 4.29% Rust 1.90%
1 2 3 4 5 6 7 8 9 10
Single-patch linux 1415 (10.10%) tensorflow 384 (2.74%) wireshark 292 (2.08%) vim 190 (1.35%) gpac 142 (1.01%) xwiki-platform 134 (0.95%) ImageMagick 133 (0.94%) qemu 117 (0.83%) openssl 111 (0.79%) tcpdump 105 (0.74%)
Multi-patch linux 84 (2.08%) ImageMagick 61 (1.51%) openssl 52 (1.28%) xwiki-platform 33 (0.81%) tensorflow 31 (0.76%) phpmyadmin 28 (0.69%) discourse 26 (0.64%) wireshark 22 (0.54%) FFmpeg 19 (0.47%) cpython 18 (0.44%)
Vulnerability Type Distribution. Table 3 presents the distribution of the top-10 most common vulnerability types in multi-patch fixes. Some vulnerability types appear more frequently in multi-patch fixes. For example, CWE-200, CWE-770, and CWE-94 rank 2nd, 9th, and 10th in multi-patch fixes, but only 9th, 26th, and 16th in single-patch fixes. Compared with other high-ranking types, such as Out-of-bounds Write, these vulnerabilities often require more
12
W. Qi et al.
complex remediation. Specifically, fixing CWE-200 (Exposure of Sensitive Information to an Unauthorized Actor) usually involves both strengthening access control to prevent unauthorized access and improving sensitive data handling to block information leakage. Because these changes often affect different modules, vendors may be more likely to adopt a multi-patch approach. Table 3. Top-10 vulnerability types in multi-patch fixes CWE Description Count Rank (Compared w/ single) 79 Cross-site Scripting 519 1 (–) 164 2 (↑7) 200 Sensitive Information Exposure 20 Improper Input Validation 164 3 (↓1) 125 Out-of-bounds Read 157 4 (↓2) Out-of-bounds Write 133 5 (↑1) 787 119 Buffer Overflow 116 6 (↓2) 22 Path Traversal 113 7 (–) SQL Injection 100 8 (–) 89 770 Allocation Without Limits 95 9 (↑17) 94 Code Injection 83 10 (↑6)
Temporal Distribution. Fig. 9 shows the trend of single-patch and multi-patch fixes over time. Excluding the most recent years for which some CVEs may not yet be fully disclosed, both types exhibit a clear upward trend. Notably, even in years when the number of single-patch fixes declines, multi-patch fixes continue to increase. For example, in 2023 and 2024, single-patch fixes decrease, whereas multi-patch fixes continue to grow. This pattern suggests an increasing adoption of multi-patch fixing practices. Single-patch Multi-patch
19 9 20 9 0 20 0 0 20 1 0 20 2 0 20 3 0 20 4 0 20 5 0 20 6 0 20 7 0 20 8 0 20 9 1 20 0 1 20 1 1 20 2 1 20 3 1 20 4 1 20 5 1 20 6 1 20 7 1 20 8 1 20 9 2 20 0 2 20 1 2 20 2 2 20 3 2 20 4 25
2000 1500 1000 500 0
Fig. 9. The number of single and multi-patch fixes over time
Time Interval Between Patches. Table 4 shows the time intervals for the six categories of multi-patch fixes defined in §4, measured from the first patch to the last. Overall, 31.7% of multi-patch fixes take more than one day to complete. Vendors respond more quickly when fixing vulnerabilities at multiple locations within the same branch, while porting fixes across versions or projects often causes substantial delays. This creates security risks because different versions of the same software often share similar logic: once one version is patched, attackers may search for similar flaws in unpatched versions, and delayed patch migration prolongs exposure. Delays are also common in other categories. In defectivefix cases, vendors may not realize at submission time that the initial patch is incomplete or introduces new issues, further delaying remediation. Table 4. Time interval of multi-patch fixes (from the first patch to the last patch). Interval (days) <1 1 - 30 30 - 360 > 360
A1. CrossBranch/Proj 556 137 50 10
A2. In-Branch Multi-Loc 17 6 0 0
B1. Workaround and Fix 9 5 3 0
B2. Fix and Doc 68 29 9 1
C1. Incomplete 336 145 33 4
C2. BugIntroducing 48 43 5 1
Patch Similarity and Complexity. We measure patch similarity using Levenshtein distance [29] at character level, which quantifies the minimum number of insertions, deletions, and substitutions required to transform one patch into
An Empirical Study of Multiple Patches for Vulnerability Fixes
13
another so that larger distances indicate lower similarity. The line charts further report the average number of code hunks and lines of code per patch in each category. The results in Fig. 10 reveal several non-intuitive issues. First, while fixes spanning multiple branches address the same underlying logic and might therefore be expected to exhibit high similarity, their Levenshtein distances are generally large, indicating low similarity. One possible explanation is that cross-branch porting requires additional modifications to address compatibility differences. By contrast, fixes at multiple locations within a single branch exhibit relatively high similarity, as they apply the same fix logic to similar code sites. We also find that bug-introducing patches and their subsequent fixes often remain highly similar, because they typically modify the same code context. 14000
10000
Patches distance Modified lines distance 1000 LoCs per patch Hunks per patch 800
8000
600
6000
12537
10845
8971
5014
4975
5308
4000 2000 0
4867 3544
1425
2334
1909
ross- j ranch round . Fix c 1. BugA1. C h/Pro . In-B Loc orka ix aBn2d Do coCmplete Ct2r.oducing In In Branc A2 Multi- B1. Wand F
400
Complexity
12000
Distance
13388
200 0
Fig. 10. Patch similarity and complexity across multi-patch fixes.
7
Management of Multi-Patch Fixes
Given that some vulnerabilities are not fully remediated by a single patch (§4), an important practical question is whether an initial patch has completely fixed the vulnerability. As summarized in Table 1, two factors may cause multiple patches for complete remediation: i) Incomplete Patches: the initial patch fails to fully correct the vulnerable functionality (C1); ii) Multi-Location Patches: the initial patch fails to cover all vulnerable locations (A1 and A2). This section assesses the extent to which existing vulnerability analysis techniques can detect whether an initial patch will later require follow-up patches from these two perspectives. 7.1
Vulnerability Detection for Incomplete Patches
An incomplete patch leaves the code vulnerable even after the patch is applied. If an ideal vulnerability detection (VD) oracle is available, we could use it to determine whether the patched code still contains the vulnerability and thus whether the patch is complete. In this subsection, we evaluate whether existing VD approaches are capable of serving this purpose. Setup. We select six representative VD models across different architectures: CodeBERT [15] and UniXcoder [19] are general-purpose pre-trained models for code understanding that also demonstrate strong performance on VD
14
W. Qi et al.
tasks [33]. Devign [64] and ReVeal [4] are graph neural network based models designed for vulnerability detection. LineVul [17], VulBERTa [20], and PDBERT [11] leverage pre-trained language model architectures with enhancements for vulnerability detection. We exclude pre-trained large language models (LLMs) because our dataset is collected from public vulnerability databases, making it difficult to rule out data leakage from pre-training corpora. These models are trained following standard procedures in prior VD studies and used to predict whether a patch completely fixes a vulnerability. Specifically, to detect incomplete fixes, a well-performing model should classify the code before the final fixing patch as vulnerable and the code after it as non-vulnerable. Dataset. We conduct the test on our collected data from category (C1) as incomplete patches of multi-patch fixes. The code before the final fix is labeled as vulnerable, and the code after the final fix as non-vulnerable. Since our goal is to evaluate whether existing VD models can detect vulnerabilities that remain after an incomplete fix, we train these models following the standard data construction routine used in prior work, such as Big-Vul [13]. Specifically, we collect vulnerable and non-vulnerable samples from single-patch cases in the NVD and split them into training, validation, and test sets in 8:1:1, as shown in Table 5. Table 5. Composition of incomplete-patch detection dataset. Dataset Vulnerable Non-Vulnerable
Train (VD) 8301 243608
Valid (VD) 1037 30451
Test (VD) 1037 30451
Test (multi) 738 556
Result. Table 6 reports the performance of each VD model on detecting incomplete patch fixes. Accuracy and F1 reflect overall performance. True positive rate (TPR), equivalent to recall, measures model’s ability to identify remained vulnerabilities after incomplete fixes. True negative rate (TNR) represents its ability to recognize fully fixed code as non-vulnerable. We also compare these results with performance on single-patch data to assess the extent to which existing VD models degrade when detecting vulnerabilities left by incomplete fixes. Table 6. VD models performance on detecting incomplete fixes requiring multi-patch Vulnerability Detection Incomplete Patch Detection Acc TPR TNR F1 Acc TPR TNR F1 CodeBERT 96.94 38.54 98.93 45.38 45.43(51.51 ↓) 10.16(28.38 ↓) 92.42(6.51 ↓) 17.54(27.84 ↓) PDBERT 96.94 38.25 98.94 45.16 45.28(51.66 ↓) 10.30(27.95 ↓) 91.88(7.06 ↓) 17.69(27.47 ↓) UniXcoder 96.85 39.98 98.78 45.53 45.36(51.49 ↓) 13.82(26.16 ↓) 87.36(11.42 ↓) 22.42(23.11 ↓) VulBERTa 96.79 3.37 96.81 6.48 42.96(53.83 ↓) 0.27(3.10 ↓) 99.82(3.01 ↑) 0.53(5.95 ↓) Linevul 96.93 10.02 99.88 75.36 43.73(53.20 ↓) 2.85(7.17 ↓) 98.19(1.69 ↓) 5.46(69.90 ↓) Devign 69.57 73.69 69.42 13.77 43.56(26.01 ↓) 63.17(10.52 ↓) 33.75(35.67 ↓) 42.73(28.96 ↑) ReVeal 71.63 70.52 71.68 14.08 48.85(22.78 ↓) 50.18(20.34 ↓) 48.20(23.48 ↓) 39.54(25.46 ↑) Model
We find that the TPR of all models on incomplete-fix detection drops by 7.17% to 28.38% compared with vulnerability detection on single-patch cases, indicating that VD models trained on conventional data struggle to identify vulnerabilities that remain after incomplete fixes. The TNR of transformer-based models also declines consistently, suggesting that their performance is affected by the distinctive characteristics of multi-patch data. Our examination of failure cases
An Empirical Study of Multiple Patches for Vulnerability Fixes
15
shows that these models often assign the same label to the pre-fix, intermediate, and post-fix versions of functions involved in incomplete fixes. Because security patches typically introduce only partial changes to a function, the models often fail to distinguish among these closely related variants. Notably, all models achieve accuracy and F1 scores below 50% on incomplete-fix detection, which is worse than random guessing. These results highlight the need for more effective methods to predict whether a patch will evolve into a multi-patch fix due to incomplete remediation. 7.2 Vulnerable Code Clone Detection for Multi-Location Patches Vulnerabilities recurring at different code locations may require multiple patches for complete remediation. The first patch may fix the vulnerability at one location, while similar code sites remain vulnerable. Vulnerable code clone (VCC) detectors are designed to identify similar code sites and determine whether they contain vulnerabilities. In this subsection, we evaluate whether such techniques can identify cases in which additional patches are needed at other locations. Method. Since our test samples are collected from the NVD, many proprietary clone detection tools may already contain fingerprints of the recorded vulnerabilities. To avoid data leakage, we select two popular open-source tools that support building vulnerability fingerprints from scratch, ReDebug [25] and FIRE [14]. Setup. We use the multi-location patch samples manually annotated as categories A1 and A2 for evaluation. Specifically, we use the first patch in each patch sequence to generate vulnerability fingerprints for the two tools, and use the remaining patches to construct the test set. This yields a dataset of 252 vulnerability signatures and 266 test samples. We then apply the two VCC tools to determine whether recurring vulnerabilities are present by classifying the other code site before each subsequent fixing patch as vulnerable and the code after the patch as non-vulnerable. Result. Table 7 reports performance of VCC methods on multi-location vulnerabilities in multi-patch fixes, alongside general VCC results reported in FIRE [14]. Both ReDebug and FIRE show clear performance drops in the multi-location setting. Particularly, FIRE’s TPR decreases from 90 to 52.46. One possible reason is that vulnerable locations in multi-patch cases differ substantially, making recurring vulnerabilities harder to detect. The FPR of both tools also decreases, suggesting that they tend to miss related vulnerabilities rather than over-predict them. These results indicate that both tools struggle to capture recurring vulnerability signatures across different code locations in multi-patch fixes. Table 7. Vulnerable code clone detection on multi-location patches Model VCC_TPR VCC_FPR Multi_TPR Multi_FPR ReDebug 37.56 20.05 16.75(↓20.81) 0.49(↓19.56) FIRE 90.00 8.57 52.46(↓37.54) 5.23(↓3.34)
8
Related Work
Patch-Based Vulnerability Analysis. Security patches play an important role in vulnerability analysis. Prior work [1, 48] infers vulnerability life cycles
16
W. Qi et al.
from patches, while other studies [22, 34, 43, 61] use patch information to determine whether patches introduce vulnerabilities. Existing real-world vulnerability datasets [4, 5, 11, 13, 64] are often constructed by reverting patches, and synthesized datasets also rely on patches to capture vulnerability patterns. Researchers have proposed fuzzing-based [3, 16, 23, 28], code clone-based [27, 57], and deep learning-based [17, 31, 32, 35] methods for vulnerability detection. For example, MVP [57] uses patch-signature matching, VUDDY [27] uses code clone detection, and Magma [21] builds fuzzing benchmarks by reverse-integrating security patches. Therefore, a correct understanding of security patches is important for the quality of software security datasets and subsequent research. Patch Analysis. A series of efforts has been made on software patch analysis. Zhong et al. [63] analyze 9K bug fixes across six Java OSS projects, while Sliwerski et al. [45] examine relationship between bug-introducing changes and bug-fixing patches in two OSS projects. Iannone et al. [24] investigate 3K security patches in the NVD, discussing their causes and fix processes. VFCFinder [12] mines vulnerability-fixing patches from vulnerability reports. PatchRNN [54] and SPI [65] use deep learning to identify security patches, while GraphSPD [51] leverages code property graphs over pre- and post-patch code to improve detection. Xu et al. [60] propose a binary-level framework for identifying security patches and summarizing vulnerability patterns. However, prior studies on multipatch fixes either focus on general bugs or functional patches rather than security fixes, or are limited to specific domains. Li et al. [30] discuss multi-patch fixes in a large-scale study of security patches, but analyze only 100 samples. Gu et al. [18] study bug-fix patches in three OSS projects. Wu et al. [56] and Kim et al. [26] respectively examine patch correctness in Linux and robotics systems.
9
Discussion and Conclusion
Our classification of multi-patch fixes relies on manual analysis, which introduces potential threats to validity. Since our analysts are not the original developers of the projects, they may misjudge the causes of certain vulnerabilities. To mitigate these threats, three researchers independently analyzed each sample and discussed disagreements in complex cases until reaching consensus. We also maintained a codebook throughout the process to formalize the accumulated knowledge, and any modification to the codebook required agreement from all participants. Moreover, our analysis is based on public records provided by the NVD. Although NVD records are maintained by NIST analysts, errors may still occur given the large scale of the database. For multi-patch data, we remove anomalous items during manual annotation. Also, because some vulnerabilities are fixed silently, certain fix records may be missing [55]. We leave large-scale analysis of patches from other resources to future work. We also hope that our study motivates further advances in patch mining and detection, enabling the discovery of more silent fixes and benefiting future patch analysis research. To the best of our knowledge, this paper presents the first systematic empirical study of multi-patch fixes. We analyze 1,646 multi-patch fixes from the NVD, manually identify their causes, and classify them into six categories. We further
An Empirical Study of Multiple Patches for Vulnerability Fixes
17
compare multi-patch and single-patch fixes, examine unique characteristics, and evaluate whether existing vulnerability detection and code clone-based methods can predict when an initial fix will require follow-up patches. Our results show that current techniques struggle to handle multi-patch fixes, highlighting the need for further research. Overall, our findings provide practical insights for the security community and a foundation for future work on multi-patch fixes. Acknowledgments. This research is partially supported by the National Science Foundation (NSF) grant CNS-2450602. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.
References 1. Bao, L., Xia, X., Hassan, A.E., Yang, X.: V-szz: automatic identification of version ranges affected by cve vulnerabilities. In: Proceedings of the 44th international conference on software engineering. pp. 2352–2364 (2022) 2. Bhandari, G., Naseer, A., Moonen, L.: Cvefixes: automated collection of vulnerabilities and their fixes from open-source software. In: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering. pp. 30–39 (2021) 3. Böhme, M., Pham, V.T., Nguyen, M.D., Roychoudhury, A.: Directed greybox fuzzing. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. pp. 2329–2344 (2017) 4. Chakraborty, S., Krishna, R., Ding, Y., Ray, B.: Deep learning based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering 48(9), 3280–3296 (2021) 5. Chen, Y., Ding, Z., Alowain, L., Chen, X., Wagner, D.: Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection. In: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses. pp. 654–668 (2023) 6. Chess, B., West, J.: Secure programming with static analysis. Pearson Education (2007) 7. Croft, R., Babar, M.A., Kholoosi, M.M.: Data quality for software vulnerability datasets. In: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). pp. 121–133. IEEE (2023) 8. CVEProject: CNA_Rules_v2.0. https://github.com/CVEProject/cvedocuments/blob/master/archive/CNA_Rules_v2.0.pdf 9. CVEProject: CNA_Rules_v3.0. https://github.com/CVEProject/cvedocuments/blob/master/archive/CNA_Rules_v3.0.pdf 10. David Golden: A survey of git best practices. https://xdg.me/a-survey-of-git-bestpractices/ 11. Ding, Y., Fu, Y., Ibrahim, O., Sitawarin, C., Chen, X., Alomair, B., Wagner, D., Ray, B., Chen, Y.: Vulnerability detection with code language models: How far are we? arXiv preprint arXiv:2403.18624 (2024) 12. Dunlap, T., Lin, E., Enck, W., Reaves, B.: Vfcfinder: Pairing security advisories and patches. In: Proceedings of the 19th ACM Asia Conference on Computer and Communications Security. pp. 1128–1142 (2024)
18
W. Qi et al.
13. Fan, J., Li, Y., Wang, S., Nguyen, T.N.: Ac/c++ code vulnerability dataset with code changes and cve summaries. In: Proceedings of the 17th international conference on mining software repositories. pp. 508–512 (2020) 14. Feng, S., Wu, Y., Xue, W., Pan, S., Zou, D., Liu, Y., Jin, H.: {FIRE}: combining {Multi-Stage} filtering with taint analysis for scalable recurring vulnerability detection. In: 33rd USENIX Security Symposium (USENIX Security 24). pp. 1867–1884 (2024) 15. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al.: Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155 (2020) 16. Fioraldi, A., Maier, D., Eißfeldt, H., Heuse, M.: {AFL++}: Combining incremental steps of fuzzing research. In: 14th USENIX workshop on offensive technologies (WOOT 20) (2020) 17. Fu, M., Tantithamthavorn, C.: Linevul: A transformer-based line-level vulnerability prediction. In: Proceedings of the 19th International Conference on Mining Software Repositories. pp. 608–620 (2022) 18. Gu, Z., Barr, E.T., Hamilton, D.J., Su, Z.: Has the bug really been fixed? In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 1. pp. 55–64 (2010) 19. Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., Yin, J.: Unixcoder: Unified crossmodal pre-training for code representation. arXiv preprint arXiv:2203.03850 (2022) 20. Hanif, H., Maffeis, S.: Vulberta: Simplified source code pre-training for vulnerability detection. In: 2022 International joint conference on neural networks (IJCNN). pp. 1–8. IEEE (2022) 21. Hazimeh, A., Herrera, A., Payer, M.: Magma: A ground-truth fuzzing benchmark. Proceedings of the ACM on Measurement and Analysis of Computing Systems 4(3), 1–29 (2020) 22. Hoang, T., Dam, H.K., Kamei, Y., Lo, D., Ubayashi, N.: Deepjit: an end-to-end deep learning framework for just-in-time defect prediction. In: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). pp. 34–45. IEEE (2019) 23. Huang, H., Yao, P., Chiu, H.C., Guo, Y., Zhang, C.: Titan: efficient multi-target directed greybox fuzzing. In: 2024 IEEE Symposium on Security and Privacy (SP). pp. 1849–1864. IEEE (2024) 24. Iannone, E., Guadagni, R., Ferrucci, F., De Lucia, A., Palomba, F.: The secret life of software vulnerabilities: A large-scale empirical study. IEEE Transactions on Software Engineering 49(1), 44–63 (2022) 25. Jang, J., Agrawal, A., Brumley, D.: Redebug: finding unpatched code clones in entire os distributions. In: 2012 IEEE Symposium on security and privacy. pp. 48–62. IEEE (2012) 26. Kim, H., Ozmen, M.O., Celik, Z.B., Bianchi, A., Xu, D.: {PatchVerif}: Discovering faulty patches in robotic vehicles. In: 32nd USENIX Security Symposium (USENIX Security 23). pp. 3011–3028 (2023) 27. Kim, S., Woo, S., Lee, H., Oh, H.: Vuddy: A scalable approach for vulnerable code clone discovery. In: 2017 IEEE symposium on security and privacy (SP). pp. 595–614. IEEE (2017) 28. Klees, G., Ruef, A., Cooper, B., Wei, S., Hicks, M.: Evaluating fuzz testing. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. pp. 2123–2138 (2018) 29. Levenshtein, V.I., et al.: Binary codes capable of correcting deletions, insertions, and reversals. In: Soviet physics doklady. vol. 10, pp. 707–710. Soviet Union (1966)
An Empirical Study of Multiple Patches for Vulnerability Fixes
19
30. Li, F., Paxson, V.: A large-scale empirical study of security patches. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. pp. 2201–2215 (2017) 31. Li, Z., Zou, D., Xu, S., Jin, H., Zhu, Y., Chen, Z.: Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing 19(4), 2244–2258 (2021) 32. Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., Zhong, Y.: Vuldeepecker: A deep learning-based system for vulnerability detection. arXiv preprint arXiv:1801.01681 (2018) 33. Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al.: Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664 (2021) 34. Nguyen, S., Nguyen, T.T., Vu, T.T., Do, T.D., Ngo, K.T., Vo, H.D.: Code-centric learning-based just-in-time vulnerability detection. Journal of Systems and Software 214, 112014 (2024) 35. Nguyen, V.A., Nguyen, D.Q., Nguyen, V., Le, T., Tran, Q.H., Phung, D.: Regvd: Revisiting graph neural networks for vulnerability detection. In: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings. pp. 178–182 (2022) 36. Nie, X., Li, N., Wang, K., Wang, S., Luo, X., Wang, H.: Understanding and tackling label errors in deep learning-based vulnerability detection (experience paper). In: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 52–63 (2023) 37. NIST: NVD - Home. https://nvd.nist.gov/ 38. NVD: CVE-2018-8729. https://nvd.nist.gov/vuln/detail/CVE-2018-8729 39. NVD: CVE-2022-2522. https://nvd.nist.gov/vuln/detail/CVE-2022-2522 40. NVD: CVE-2023-35852. https://nvd.nist.gov/vuln/detail/CVE-2023-35852 41. NVD: CVE-2023-40173. https://nvd.nist.gov/vuln/detail/CVE-2023-40173 42. NVD: CVE-2024-1394. https://nvd.nist.gov/vuln/detail/cve-2024-1394 43. Pornprasit, C., Tantithamthavorn, C.K.: Jitline: A simpler, better, faster, finergrained just-in-time defect prediction. In: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). pp. 369–379. IEEE (2021) 44. Ray, B., Posnett, D., Filkov, V., Devanbu, P.: A large scale study of programming languages and code quality in github. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering. pp. 155–165 (2014) 45. Śliwerski, J., Zimmermann, T., Zeller, A.: When do changes induce fixes? ACM sigsoft software engineering notes 30(4), 1–5 (2005) 46. Software Freedom Conservancy: Git. https://git-scm.com/ 47. Tan, X., Zhang, Y., Cao, J., Sun, K., Zhang, M., Yang, M.: Understanding the practice of security patch management across multiple branches in oss projects. In: Proceedings of the ACM Web Conference 2022. pp. 767–777 (2022) 48. Tang, L., Bao, L., Xia, X., Huang, Z.: Neural szz algorithm. In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). pp. 1024–1035. IEEE (2023) 49. The MITRE Corporation: CVE: Common Vulnerabilities and Exposures. https://www.cve.org/ 50. The RedMonk Programming Language Rankings: June 2024: https://redmonk. com/sogrady/2024/09/12/language-rankings-6-24/ (2024)
20
W. Qi et al.
51. Wang, S., Wang, X., Sun, K., Jajodia, S., Wang, H., Li, Q.: Graphspd: Graph-based security patch detection with enriched code semantics. In: 2023 IEEE Symposium on Security and Privacy (SP). pp. 2409–2426. IEEE (2023) 52. Wang, W., Li, Y., Li, A., Zhang, J., Ma, W., Liu, Y.: An empirical study on noisy label learning for program understanding. In: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. pp. 1–12 (2024) 53. Wang, X., Sun, K., Batcheller, A., Jajodia, S.: Detecting" 0-day" vulnerability: An empirical study of secret security patch in oss. In: 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). pp. 485– 492. IEEE (2019) 54. Wang, X., Wang, S., Feng, P., Sun, K., Jajodia, S., Benchaaboun, S., Geck, F.: Patchrnn: A deep learning-based system for security patch identification. In: MILCOM 2021-2021 IEEE Military Communications Conference (MILCOM). pp. 595– 600. IEEE (2021) 55. Woo, S., Choi, E., Lee, H.: A large-scale analysis of the effectiveness of publicly reported security patches. Computers & Security 148, 104181 (2025) 56. Wu, Y., Lin, Z., Chen, Y., Le, D.K., Mu, D., Xing, X.: Mitigating security risks in linux with {KLAUS}: A method for evaluating patch correctness. In: 32nd USENIX Security Symposium (USENIX Security 23). pp. 4247–4264 (2023) 57. Xiao, Y., Chen, B., Yu, C., Xu, Z., Yuan, Z., Li, F., Liu, B., Liu, Y., Huo, W., Zou, W., et al.: {MVP}: Detecting vulnerabilities using {Patch-Enhanced} vulnerability signatures. In: 29th USENIX Security Symposium (USENIX Security 20). pp. 1165–1182 (2020) 58. xiaoge1001: Heap-based Buffer Overflow in function ins_compl_infercase_gettext() in vim/vim. https://huntr.com/bounties/3a2d83af-9542-4d93-8784-98b115135a22 59. Xu, C., Chen, B., Lu, C., Huang, K., Peng, X., Liu, Y.: Tracking patches for open source software vulnerabilities. In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. pp. 860–871 (2022) 60. Xu, Z., Chen, B., Chandramohan, M., Liu, Y., Song, F.: Spain: security patch analysis for binaries towards understanding the pain and pills. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). pp. 462–472. IEEE (2017) 61. Yan, M., Xia, X., Fan, Y., Hassan, A.E., Lo, D., Li, S.: Just-in-time defect identification and localization: A two-phase framework. IEEE Transactions on Software Engineering 48(1), 82–101 (2020) 62. Zhang, Z., Li, H., Li, X., Zhang, H., Qian, Z.: Llmbisect: Breaking barriers in bug bisection with a comparative analysis pipeline. arXiv preprint arXiv:2510.26086 (2025) 63. Zhong, H., Su, Z.: An empirical study on real bug fixes. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. vol. 1, pp. 913–923. IEEE (2015) 64. Zhou, Y., Liu, S., Siow, J., Du, X., Liu, Y.: Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems 32 (2019) 65. Zhou, Y., Siow, J.K., Wang, C., Liu, S., Liu, Y.: Spi: Automated identification of security patches via commits. ACM Transactions on Software Engineering and Methodology (TOSEM) 31(1), 1–27 (2021)