VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats SOFIA DELLA PENNA, LORENZO PARRACINO, LUCIANO PIANESE, and VITTORIO ORBINATO, Università degli Studi di Napoli Federico II, Italy ROBERTO NATELLA, Gran Sasso Science Institute, Italy Enterprise networks are continuously targeted by Advanced Persistent Threats (APTs), attack campaigns exploiting software vulnerabilities to compromise critical assets over time. As disclosed vulnerabilities grow, resource-constrained organizations must prioritize which ones to patch. Existing prioritization standards score vulnerabilities individually and cannot capture how a patching policy performs against an adversary that progresses through the network over time. Previous tools have simulated attack campaigns through Reinforcement Learning (RL), but either omit vulnerability management, leaving the attacker unopposed, or rely on synthetic networks disconnected from real threat data, and so cannot assess how a policy would fare against a realistic adversary. To fill this gap, we propose VulnGym, a simulation tool to evaluate vulnerability management policies. VulnGym simulates an RL-trained attacker,
arXiv:2607.24552v1 [cs.CR] 27 Jul 2026
calibrated on real APT profiles, against a defender executing a configurable patching policy over a network with real Common Vulnerabilities and Exposures (CVEs). Both agents act on a shared, evolving network representation, so the attacker’s progress is directly shaped by the defender’s patching activity, allowing a given policy to be stress-tested against a realistic attack campaign. Experiments based on real-world vulnerabilities and two APTs show that vulnerability management must be tailored to organizational context, adversarial behavior, network topology, and asset criticality. CCS Concepts: • Security and privacy → Vulnerability management; • Computing methodologies → Reinforcement learning.
1
Introduction
In the current cybersecurity landscape, organizations are increasingly confronted with sophisticated attacks, known as Advanced Persistent Threats (APTs) [27, 33]. APTs apply a variety of techniques to stealthily propagate across computer networks, to evade detection, to maintain persistence over extended periods, and ultimately to disrupt services and exfiltrate sensitive data [32, 39, 59]. In particular, APT attacks are enabled by software vulnerabilities in the network, which can be exploited to obtain initial access, gain privileges, and move laterally across the network. Notable examples of high-impact vulnerabilities include the Microsoft Server Message Block (SMB) vulnerability exploited by the WannaCry ransomware through the EternalBlue exploit [64], the Apache Struts vulnerability exploited in the large data breach from the Equifax agency [65], and the Log4Shell vulnerability in the popular Log4j open-source library [19]. Therefore, software vulnerabilities represent a critical risk faced by organizations today. Despite the continuous release of security patches for newly discovered vulnerabilities, a substantial proportion of cyberattacks still exploit known vulnerabilities for which patches have long been available but not applied [7, 13, 29]. This problem is caused by the overwhelming volume of reported vulnerabilities that IT teams must manage: the number of vulnerabilities reported in the National Vulnerability Database (NVD) was close to 50, 000 new entries in 2025 [20, 25]. As a matter of fact, organizations have a limited capacity to deploy patches [54], since vulnerability management requires significant resources in terms of time and personnel, and since patches can disrupt business operations, e.g., due to human errors and broken dependencies [46, 62]. This trend is expected to be exacerbated by the emergence of Authors’ Contact Information: Sofia Della Penna, [email protected]; Lorenzo Parracino, [email protected]; Luciano Pianese, [email protected]; Vittorio Orbinato, [email protected], Università degli Studi di Napoli Federico II, Naples, Italy; Roberto Natella, [email protected], Gran Sasso Science Institute, L’Aquila, Italy.
1
2
Della Penna et al.
LLM-based tools capable of autonomously discovering and exploiting software vulnerabilities, a capability recently demonstrated at scale by frontier AI models, which is likely to substantially increase the volume of vulnerability reports organizations must handle [3]. To keep up with the pace of vulnerabilities, several standards and metrics have been developed to guide vulnerability management, by prioritizing efforts on the most critical vulnerabilities [9, 10, 22, 23, 38, 42, 62], such as vulnerabilities that are easier to exploit or that have already been exploited in recent attacks. However, these guidelines only analyze vulnerabilities individually, and do not consider how vulnerabilities can be leveraged in the wider context of a complete attack campaign. As a matter of fact, attack campaigns from APTs combine vulnerability exploitation with several other post-exploitation techniques, such as lateral movement, credential stealing, and persistence. Moreover, multiple vulnerabilities can be exploited by the same attack campaign. Therefore, the prioritization of vulnerability management should take into account multiple factors, including the attack patterns of APTs, and the topology and assets of the victim network. In this paper, we present VulnGym, a novel simulation tool1 for the quantitative analysis of vulnerability management strategies. VulnGym models the dynamics of complex attack campaigns in IT networks, by taking into account that: (i) new vulnerabilities are being discovered over time; (ii) IT teams continuously scan for vulnerabilities and deploy patches to fix them, with a limited budget; (iii) adversaries can exploit vulnerabilities towards compromising sensitive assets. In particular, the tool uses Reinforcement Learning (RL) to train an adversarial agent able to orchestrate multiple attack techniques and to maximize impact on assets. Moreover, the tool enables the modeling of vulnerability management policies, such as policies based on severity scores (e.g., CVSS), and policies that prioritize specific assets and network segments. The analysis brings insights on how effective such policies are at mitigating attack campaigns as a whole, providing valuable feedback for vulnerability management. The paper includes an analysis of vulnerability management policies with respect to real vulnerabilities from the NVD [43]. The analysis shows the shortcomings of vulnerability management based on severity scores of individual vulnerabilities, and the benefits of holistic policies that account for asset criticality. This result motivates the need to consider more contextual information in defining prioritization policies, including the expected adversarial behaviors, the budget for vulnerability management, and the topology and assets of the network. The remainder of the paper is organized as follows. Section 2 provides background on vulnerability management frameworks and Reinforcement Learning. Section 3 reviews the related work. Section 4 presents the proposed simulation approach. Sections 5 and 6 describe the experimental design and evaluate the proposed vulnerability management policies. Section 7 discusses the threats to validity. Finally, Section 8 concludes the paper. 2
Background
2.1
Vulnerability Management Frameworks
Vulnerability management is a cornerstone of cybersecurity. It is based on frameworks for Risk Assessment, Vulnerability Identification and Prioritization, and Severity Scoring. Risk management frameworks provide structured approaches for securing information systems [45]. Among these, NIST Special Publication 800-40 [62] offers recommendations for operationalizing vulnerability management. It advocates for patch prioritization based on cybersecurity impact, integration with change management processes, secure acquisition and validation of patches, and thorough testing in staging environments prior to deployment. 1 Available at: https://github.com/dessertlab/vulnGym
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
3
For vulnerability identification and classification, the Common Vulnerabilities and Exposures (CVE) registry [38] serves as the de facto standard. Each CVE entry receives a unique identifier and description from designated CVE Numbering Authorities (CNAs). The Known Exploited Vulnerabilities (KEV) Catalog [9] by CISA extends the CVE framework by tracking vulnerabilities that were exploited in the wild. Severity scoring and exploit likelihood are commonly estimated using the Common Vulnerability Scoring System (CVSS) [23], which quantifies vulnerability impact across multiple technical dimensions. The Exploit Prediction Scoring System (EPSS) [22] complements CVSS by leveraging machine learning models trained on threat intelligence data to estimate the probability that a vulnerability will be exploited in the wild. Despite their widespread adoption, these scoring systems have repeatedly been questioned for their ability to reflect real exploitation risk. Younis et al. [69] compared CVSS Base metrics and Microsoft’s rating system over 813 vulnerabilities, revealing high false-positive rates and a strong dependency on software-specific factors. Several subsequent studies confirmed that many low-scoring vulnerabilities are still exploited in the wild [11, 26, 30, 53]. For instance, ransomware campaigns have repeatedly exploited CVEs classified as medium/low severity [15], revealing that CVSS lacks contextual awareness [21, 57]. Similarly, EPSS, despite improving on CVSS through machine learning, remains an opaque approach unable to replace systematic risk analysis [63]. Taken together, these findings show that individually-scored vulnerabilities are a poor proxy for the risk posed by a real attack campaign, which motivates research in the field as discussed in Section 3. 2.2
Reinforcement Learning
We adopt Reinforcement Learning (RL) [2] to create automated agents that emulate attackers. RL is a machine learning paradigm in which a decision-making agent interacts actively with an uncertain environment. The key components of a RL system are as follows [2]: • Agent: seeks goals by interacting with the environment. • Environment: everything the agent interacts with. The agent reads the state of the environment to choose actions. • Policy: a map between the states in the environment and the actions that are taken from that state. • Reward Signal: defines the goal of the problem. At each time step, RL assesses the current state and action of the agent, and sends a reward value to the agent. • Value Function: is defined for a state. When the agent starts from a state and accumulates a total reward over time, it is known as the value of the state. Q-Learning (QL) [66] is a model-free RL algorithm, based on the Q-function 𝑄 (𝑠, 𝑎), which estimates the expected reward of taking action 𝑎 in state 𝑠 to guide the agent and is updated iteratively according to Eq. 1: 𝑄 (𝑠𝑡 , 𝑎𝑡 ) ← 𝑄 (𝑠𝑡 , 𝑎𝑡 ) + 𝛼 𝑅𝑡 +1 + 𝛾 max𝑎 𝑄 (𝑠𝑡 +1, 𝑎) − 𝑄 (𝑠𝑡 , 𝑎𝑡 )
(1)
where: • 𝑄 (𝑠𝑡 , 𝑎𝑡 ) is the current estimate of the Q-value for state 𝑠𝑡 and action 𝑎𝑡 . • 𝑅𝑡 +1 is the immediate reward received after taking action 𝑎𝑡 in state 𝑠𝑡 . • 𝛾 is the discount factor, a parameter between 0 and 1 that represents the importance of future rewards relative to immediate rewards. • 𝛼 is the learning rate, which controls the magnitude of updates to the Q-values based on new experiences.
4
Della Penna et al. • max𝑎 𝑄 (𝑠𝑡 +1, 𝑎) is the maximum Q-value over all possible actions in the next state 𝑠𝑡 +1 , representing the optimal future value. Deep Q-Learning or Deep Q-Network (DQN) [16, 40] is an extension of QL, where the Q-function is approximated by
a Deep Neural Network 𝑄 (𝑠, 𝑎; 𝜃 ), where 𝜃 denotes the network weights. It implements experience replay at each time step 𝑡, by storing the agent’s experiences (𝑠𝑡 , 𝑎𝑡 , 𝑟𝑡 , 𝑠𝑡 +1 ) in a replay buffer. The DQN update at iteration 𝑖 is performed by minimizing the loss function in Eq. 2: 𝐿𝑖 (𝜃 𝑖 ) = E (𝑠𝑡 ,𝑎𝑡 ,𝑟𝑡 ,𝑠𝑡 +1 ) (𝑦𝑖 − 𝑄 (𝑠𝑡 , 𝑎𝑡 ; 𝜃 𝑖 )) 2
(2)
𝑦𝑖 = 𝑟𝑡 + 𝛾 max 𝑄 (𝑠𝑡 +1, 𝑎𝑡 +1 ; 𝜃 𝑖− ) ′
(3)
𝑎
where: • E (𝑠𝑡 ,𝑎𝑡 ,𝑟𝑡 ,𝑠𝑡 +1 ) is the average on replay buffer samples. • 𝑄 (𝑠𝑡 , 𝑎𝑡 ; 𝜃 𝑖 ) is the predicted Q-value. • 𝑄 (𝑠𝑡 +1, 𝑎𝑡 +1 ; 𝜃 𝑖− ) is the target Q-value for next state. 3
Related Work
Several tools have been proposed to simulate attack campaigns and vulnerability management processes, ranging from optimization-based patch prioritization to graph-based attack analysis and, more recently, RL-based simulation environments. Among optimization-based tools, Farris et al. [18] proposed VULCON, a vulnerability management framework that uses a mixed-integer goal-programming model to select which vulnerabilities to patch at each remediation cycle, jointly minimizing vulnerability exposure and remediation delay under a personnel-hour budget. However, the framework focuses exclusively on the defender and does not model attacker behavior or vulnerability exploitation. VulnGym extends this perspective by pairing a budget-constrained defender with an RL-trained attacker calibrated on real APT attack profiles, enabling defensive policies to be evaluated against realistic attack campaigns. Attack graphs have also been proposed to analyze the impact of vulnerabilities on network security. MulVAL [48] is a logic-based framework that models vulnerabilities, host configurations, and network connectivity using Datalog, to enumerate all potential attack paths across the network graph. Similarly, the Topological Vulnerability Analysis (TVA) [60] and Bayesian attack graph models [41] apply this approach for risk quantification. These tools analyze how vulnerabilities can be combined in complex attack scenarios, but they rely on static snapshots of the network, and do not jointly model the temporal dynamics of defenders that perform vulnerability management, of new vulnerabilities discovered over time, and of the adaptive behavior of adversaries. In contrast, VulnGym focuses on the evaluation of vulnerability management in a dynamic setting, explicitly modeling the interaction between defenders and attackers over time, and leverages RL to model sophisticated APT behaviors. Closer to our approach, RL has been used to train autonomous attacker agents and, in some cases, defender agents in simulated network environments. Becker et al. [6] extended NASim [55], a lightweight Gymnasium-based network attack simulator, to train RL agents for penetration-testing scenarios involving exploitation, credential theft, and wiretapping, comparing their performance against a rule-based penetration-testing baseline. However, their evaluation is limited to small network topologies of four to five hosts and excludes active defense and vulnerability remediation, allowing the attacker to operate without opposition. CyberBattleSim [34] is an OpenAI Gym environment developed by Microsoft that models an enterprise network as a graph of abstract nodes, in which a trainable attacker exploits local
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
5
and remote vulnerabilities and performs lateral movement to compromise network assets. However, the simulator relies on synthetic, abstract vulnerabilities rather than real CVEs and does not model the vulnerability management lifecycle. In contrast to NASim and CyberBattleSim, VulnGym grounds both the network and the vulnerabilities in real-world data from the NVD, models the defender through configurable vulnerability management policies operating under a limited patching budget, and trains the attacker on behavior profiles derived from real APT threat intelligence, allowing a direct, quantitative evaluation of vulnerability prioritization strategies against realistic adversarial campaigns. 4
Methodology
The basic idea of our approach is to simulate the interactions between an attacker and a defender (agents) within a network with vulnerable assets, in order to analyze the impact of vulnerability management policies against attack campaigns.
Fig. 1. Architecture of the simulated environment.
Figure 1 summarizes the elements of the simulation. The agents perform actions on the nodes of the simulated network. Actions are selected based on the current state of the network (e.g., nodes that have already been compromised) and modify the state of the affected nodes. After executing an action, the attacker agent receives a reward, a numerical representation of its outcome that can be either positive or negative. The reward is used to train it via RL. Through this process, the agent learns to orchestrate a sequence of attack techniques that maximize the overall impact of the attack on the network. In parallel, the defender agent follows a policy to mitigate vulnerabilities across the network, representing a vulnerability management strategy to be evaluated (e.g., based on exploitability scores). Figure 2 illustrates the execution flow of a single episode, representing a simulated time period (e.g., one year). At the beginning of the episode, the network environment and the agents are initialized. Both agents then interact with the environment: the attacker selects a target node and an action, then executes it; the defender progresses through a vulnerability management cycle, alternating between discovery and remediation phases while maintaining a vulnerability queue. Multiple episodes are simulated to account for the stochastic nature of the simulation, including probabilistic agent behavior and uncertain action outcomes. 4.1
Network
The simulated network is the environment in which agents operate and interact dynamically, and is designed to emulate realistic enterprise conditions in terms of both topology and vulnerabilities. The network is modeled as an undirected
6
Della Penna et al.
Fig. 2. Execution flow of an episode.
graph, where nodes represent potentially vulnerable assets, and edges define the connections between them, shaping the possible paths for attacker movement. The network is logically divided into zones, which are typical of IT enterprise networks [5]. The network is fully customizable and supports the definition of multiple zones (e.g., DMZ, Database) as well as intra-zone and inter-zone connections. Examples of network topologies are provided in the experimental evaluation of the paper (Section 5). Each node represents a network resource and is characterized by the attributes listed in Table 1. Asset criticality is quantified through two dedicated attributes: Importance, reflecting the operational value of the node within the network, and Centrality, a graph-theoretic measure of how pivotal the node is with respect to the overall network topology. Each node is also associated with one or more operating system and application products, selected from those commonly used in enterprise environments. Examples include OS products such as Microsoft Windows, Microsoft Windows Server, and Red Hat Linux, and applications from Adobe, Oracle, and Microsoft. Moreover, we define the Attacker Access attribute to track whether the node has been successfully compromised. VulnGym allows the user to configure which vulnerabilities to introduce in the simulation, and when to introduce them. This information can be obtained from public sources, such as the NVD [43]. Table 2 lists the attributes that can be configured for each vulnerability. Among these, Vulnerability Type is assigned by evaluating the impact of the vulnerability on the CIA triad as reported in the CVSS vector string. If the vulnerability reports a non-zero impact on Confidentiality or Integrity, it is labelled as Remote Control, as it may allow an attacker to access or manipulate data on the node. If the impact on Availability is non-zero, it is labelled as Denial of Service, reflecting the attacker’s ability to disrupt the normal operation of the node. A vulnerability can be assigned both types when it affects multiple dimensions
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
7
Table 1. Node attributes.
Name ID ZONE IMPORTANCE CENTRALITY PRODUCTS VULNERABILITIES STATE ATTACKER ACCESS
Description Unique identifier of the node. Network zone to which the node belongs. Importance level of the node within the network. Node’s structural relevance in the network. List of software products installed on the node. List of CVEs affecting the node. Current operational state of the node. Whether the attacker has gained access to the node.
of the triad simultaneously. The simulation models the progressive discovery and exploitation of vulnerabilities within the network. Initially, each vulnerability is considered undisclosed, and it is gradually enabled according to its actual publication date. When a vulnerability is disclosed, it becomes active on all nodes that host a product affected by that vulnerability. Attackers must adapt their behavior as new vulnerabilities emerge, while defenders must decide how to prioritize their mitigation efforts. Table 2. Vulnerability attributes.
Name
4.2
Description
CVE ID
Unique identifier of the vulnerability.
CVSS SCORE
Severity score assigned to the vulnerability according to the CVSS.
VULNERABILITY TYPE
Remote Control and/or Denial of Service.
RELEASE TIME
Date on which the vulnerability was publicly disclosed.
ATTACK COMPLEXITY
Level of effort required for a successful exploitation.
Attacker
The tool simulates an attacker agent based on DQN, which follows the policy learned through RL. The attacker operates over the action set listed in Table 3. This set is designed to reflect the intrusion process as described by the Cyber Kill Chain [32], and it is aligned with the tactics and techniques defined in the MITRE ATT&CK framework [39]. Figure 3 provides a graphical overview of the attacker’s actions, showing the targets and the effects on the simulation.
Fig. 3. Propagation of the attack across the network.
8
Della Penna et al. Table 3. Attacker’s actions.
Action SCAN EXPLOIT DOS
Target non-compromised node vulnerable node vulnerable node
Description The attacker scans the node for exploitable vulnerabilities. The attacker launches a remote-control attack on the node. The attacker launches a DoS attack on the node.
CREDENTIALS THEFT
compromised node
The attacker steals credentials from the node.
PERSISTENCE
compromised node
The attacker establishes persistence on the node.
PRIVILEGE ESCALATION
compromised node
The attacker escalates privileges on the node.
DATA EXFILTRATION
compromised node
The attacker exfiltrates data from the node.
WIPER
compromised node
The attacker deletes or corrupts the node’s data.
LATERAL MOVEMENT
non-compromised node adjacent to stolen credentials
The attacker moves to a connected node using stolen credentials.
Post-Condition The attacker knows the node’s exploitable vulnerabilities. The attacker gains access to the node. The node is unavailable. Increases exploitation success and enable Lateral Movement on neighbouring nodes. If the defender patches the node, the attacker still retains access to it. Increases the probability of successfully stealing credentials and achieving persistence on the node. The attacker is rewarded when targeting database nodes with sensitive assets (Database layer); otherwise, no effect. The attacker is rewarded when targeting database nodes with sensitive assets (Database layer); otherwise, no effect. The attacker gains access to the node.
The attacker is characterized by a list of products they are capable of targeting and a list of CVEs they are able to exploit. This reflects real APT behavior, as such groups are typically specialized in attacking products by specific vendors, and leverage publicly-available exploits, including those supported by tools such as Metasploit [52] and Nuclei [50], and against vulnerabilities in the KEV catalog [9]. The attacker can be configured to start from outside or inside the network, modeling exploitation of Internet-exposed vulnerable services and phishing-derived initial access, respectively. These correspond to the two most common vectors for achieving Initial Access [35] in enterprise networks [4, 12, 58]. If outside the network, the initial node is selected from those exposed to the Internet. If the attacker has phishing capabilities, the initial node of the attack is selected from within the network. Once the attacker is inside the network, it attempts to move laterally across neighboring nodes. The set of actions available to the attacker depends on whether a node has already been compromised. On noncompromised nodes, the attacker may perform one of the following actions: Scan, Exploit, or DoS. If the exploit is successful, the attacker gains access to the node, which becomes compromised. On compromised nodes, the attacker can execute post-exploitation actions, which include Credential Theft, Privilege Escalation, Persistence, Data Exfiltration, or Wiper, each representing a distinct tactic aligned with common adversarial behaviors in real-world attacks. If Credential Theft is successful on a compromised node, the attacker extracts authentication material (e.g., credentials or tokens) that can be reused to perform the Lateral Movement action. This enables the attacker to access other nodes in the network that are reachable from the compromised node, even if they are not directly vulnerable to exploitation. If the defender applies a patch to a compromised node, the attacker is evicted from that node, unless a persistence action has been performed on the node. Actions have a probabilistic outcome, reflecting real-world uncertainty. The probability of success can be configured based on Cyber Threat Intelligence (CTI) describing the attacker’s capabilities; for instance, the likelihood of successfully exploiting a vulnerability may vary depending on whether the attacker is specialized on the vulnerable product, whether the vulnerability is known to be exploited in the wild, and other characteristics (e.g., complexity of the vulnerability). The selection of the target node, the action to perform, and the vulnerability to exploit is not predefined, but learned over time from simulation episodes. The attacker progressively constructs a policy that guides actions to maximize the impact of the attack campaign. The learning process is driven by a reward structure, which assigns numerical values to the outcomes of different actions. The reward is defined as: 𝑅 =𝐴×𝐼
(4)
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
9
More specifically, 𝐴 is the importance of the action, which reflects the relevance of a specific action with respect to the attacker’s objective. For instance, actions that directly achieve the attacker’s goals (e.g., impact operations such as Data Exfiltration and Wiper) are assigned higher importance values, while information-gathering actions (e.g., Scan and Credential Theft) are associated with lower values; other actions fall between these extremes. 𝐼 denotes the importance of the node, defined in terms of its proximity to sensitive assets, with higher values assigned to nodes closer to critical resources. Rewards are positive for successful actions. Conversely, a fixed negative reward is given for failed actions, since they do not contribute to the attacker’s objective and represent wasted effort. The agent adopts an 𝜖-greedy action selection strategy to balance exploration and exploitation. With probability 𝜖, the attacker explores the action space by selecting a random move; with probability 1 − 𝜖, it exploits its current knowledge by choosing the action that maximizes the estimated Q-value. To manage the transition from exploration to exploitation, an 𝜖-decay mechanism is employed.
4.3
Defenders
The objective of the defender agent is to protect the network by patching vulnerabilities, by following a vulnerability management policy under evaluation. Due to resource limitations, the defender must allocate efforts strategically. The defender performs periodic vulnerability management cycles, as typical of enterprise systems [51, 56, 68]. A cycle consists of two phases: (i) the discovery phase (which takes a fixed amount of time), in which the defender performs a scan to identify new vulnerabilities, reviews the results, and updates a priority queue of vulnerabilities (which includes both previous vulnerabilities that are still pending, and the new ones); and (ii) the resolution phase (which takes a variable amount of time, as discussed in this section), in which the defender performs patching, by following the prioritization established by the discovery phase. The cost required to patch a vulnerability represents the amount of time to apply a patch, in discrete time steps of the simulation (e.g., days). We model this cost as a function of multiple factors: 𝑇fix = 𝛼 · 𝛽 · 𝛾 · 𝛿
(5)
where: • 𝛼 represents the baseline patching time, i.e., the average time to patch a vulnerability in the simplest case, on a single node. This parameter is calibrated by the user based on the vulnerability management capabilities of the organization. 1 • 𝛽 = defender effort , where defender effort is a scale parameter that captures the amount of resources allocated to
vulnerability management, e.g., in terms of man-days. Higher effort corresponds to lower values of 𝛽, resulting in faster patching times. In this work, we consider three levels for the defender effort (Low < 1, Regular = 1, High > 1). • 𝛾 is a scale parameter that represents an additional patching effort due to the type of vulnerability. In this work, we adopt two levels: 𝛾 APP for application-level vulnerabilities (𝛾 = 1) and 𝛾 OS (𝛾 > 1) for operating-system vulnerabilities. The latter are assumed to require longer patching times due to higher complexity. • 𝛿 = 𝜅 · |N𝑣 | is a scale parameter that represents an additional patching effort due to the number of affected nodes. |N𝑣 | is the number of nodes affected by the vulnerability. 𝜅 is a weight that reflects the impact of the
10
Della Penna et al. number of nodes on 𝑇fix . For example, it can be calibrated such that 𝑇fix matches the worst-case expected cost for deploying a patch over the entire network (N𝑣 = N ). The cost to patch has been designed to be easy to interpret and configure by the user of VulnGym. Since the time to
apply a patch depends on the specific organization, we designed the factors above such that the user can apply knowledge from past vulnerability management activities, such as the average time to patch a simple one-node vulnerability, and to patch a vulnerability that spans the entire network. The formula can be easily modified to reflect different criteria, such as increasing 𝛿 with a logarithmic trend instead of a linear one. 4.4
Implementation Details
VulnGym is implemented in Python and builds on NetworkX [44] for network modeling and Gymnasium [17] for the RL environment. The framework is modular, with configuration files defining the network topology, vulnerability distribution, and agent settings. Experiments are run via Jupyter notebooks, which are used to produce all reported figures and tables. 5
Experiments Design
In the rest of the paper, we present experiments based on VulnGym, in order to show how the tool can be used for analyzing vulnerability management policies. We consider popular network topologies, and analyze real APT profiles according to threat intelligence sources. We remark that VulnGym is a customizable tool, which can be configured by security analysts to simulate specific network environments and threats. Thus, experimental results should be interpreted in the light of the specific context of interest. We release the full source code of the tool and the configuration of the experiments, in order to support reproducibility and customizations. In particular, VulnGym produces the following metrics, which will be discussed in Section 6: • Goal Achievement Rate: the percentage of episodes in which the attacker successfully reaches the campaign goal; • Network Vulnerability Index (NVI): the percentage of network nodes to which the attacker has gained access, relative to the total number of nodes in the network; • Time to Reach Goal (TTRG): the number of simulated days required for the attacker to reach its goal; • Time to Patch Vulnerability(TTPV): the average number of days a policy takes to remediate a vulnerability since it is known; • Vulnerabilities in Backlog (VIB): the average number of vulnerabilities awaiting remediation at any given time.
5.1
Simulated Network
We conducted the experiments on two network topologies of equal size, each comprising 100 nodes, differing only in their structural organization, in order to enable a controlled comparison between topologies. This network size is consistent with typical Small and Medium Enterprise (SME) deployments, which are generally reported to range from a few dozen to a few hundred networked devices [31, 61]. The network is organized into multiple security layers (i.e., External, DMZ, Internal, and Database), which are interconnected through routing and firewall devices. These components regulate and filter traffic between layers. As a result, only a subset of nodes in each layer is reachable from other layers, reflecting realistic network segmentation and restricted connectivity patterns.
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
11
Fig. 4. Layered network topology.
Layered topology (Figure 4). In this configuration, nodes are organized into multiple layers, where only a subset of nodes in each layer is reachable from the previous one. These nodes act as shared entry points between layers, such as web servers or other exposed services. This topology reflects real-world scenarios such as enterprise networks composed of multiple subnets (e.g., DMZ, employee network, and shared resources network) [47], as well as industrial environments with segmented architectures (e.g., DMZ, IT, and OT networks) [8]. Tree topology (Figure 5). In this configuration, the network is structured hierarchically, with a primary network connected to multiple secondary networks, and optionally to additional lower-level networks. These networks are interconnected through a subset of nodes belonging to the higher-level network. This structure models real-world scenarios with geographically-distributed enterprise networks, such as in the IBM Hierarchical Tree Topology [28]. The simulation models an enterprise environment in which each zone is associated with a set of representative software products. For each layer, products are sampled from predefined product categories to ensure diversity while maintaining a realistic distribution. The External zone includes commonly used client-side applications and operating systems (e.g., Adobe products, Microsoft browsers and tools, and Windows). The DMZ comprises network-facing services, including web and application servers, remote access components, virtualization platforms, and Linux-based operating systems. The Internal zone encompasses a broad range of enterprise technologies, including operating systems
12
Della Penna et al.
(e.g., Windows, Debian, and Red Hat), and infrastructure and virtualization services (e.g., VMware). Finally, the Database zone consists of database management systems, including relational solutions (e.g., MariaDB, PostgreSQL, Oracle, and SQLite) as well as NoSQL systems (e.g., MongoDB). Vulnerabilities affecting these products are collected from the NVD [43]. We considered the ones disclosed in the year 2020, resulting in 4,137 vulnerabilities. To ensure scalability and computational efficiency, stratified sampling is applied to the set of vulnerabilities, preserving their distribution across products and severity levels. The resulting subset is mapped onto a fixed network of 100 nodes, where each node may host multiple vulnerabilities, with a maximum of 10 vulnerabilities per node. Simulation episodes represent a full one-year time window.
Fig. 5. Tree network topology.
5.2
Agents
5.2.1 Attacker. In the following experiments, we evaluate the impact of APT campaigns on the target network. We consider two common types of campaign: Exfiltration/Wiper and Denial of Service (DoS). In the Exfiltration/Wiper campaign, the goal is to perform data exfiltration and destructive actions on database nodes; in the DoS campaign, the objective is to disrupt availability of database nodes. We use VulnGym to measure the likelihood of success of the attack
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
13
campaigns over multiple runs of the simulation. In these experiments, we define campaign success as Exfiltration/Wiper or DoS on at least three database nodes, to represent multi-node impact on critical assets. The threshold is configurable. The attacker’s behavior is modeled after two distinct APT profiles: APT41 [37] for the Exfiltration/Wiper campaign, and APT28 [36] for the DoS campaign. In each campaign, the attacker is assumed to be specialized in a set of products, and can therefore exploit only vulnerabilities affecting those products, as indicated by CTI sources. We derive this information from the CTI-HAL dataset [14]. We configure the attacker to also exploit vulnerabilities listed in the KEV catalog [9]. Furthermore, both APT41 and APT28 possess phishing capabilities; consequently, Initial Access can occur on both external and internal nodes. Hyperparameters for RL were configured as in Table 4, based on prior research applying RL to cybersecurity domains [67]. The attacker agent was trained in a standalone setting (i.e., without a defender) through a dedicated training phase of 1,000 simulation episodes. Training was performed separately for each combination of APT and network topology, resulting in independent training runs for all APT–topology pairs.
Table 4. RL hyperparameters.
Parameter alpha gamma epsilon epsilon_decay epsilon_min memory_size batch_size
Description Learning rate. Discount factor. Exploration probability. Epsilon decay rate. Minimum value for epsilon. Replay buffer size. Number of experiences sampled to update the neural network.
Set Value 0.001 0.99 1.0 0.995 0.01 10, 000 64
5.2.2 Defender. The defender alternates between discovery and remediation activities. Periodically every week, the defender performs discovery, in which new vulnerabilities are identified. The following days of the cycle are dedicated to remediation, in which the defender sequentially patches vulnerabilities, according to the priority defined by the policy under evaluation. If a higher-priority vulnerability arises while another one is being patched, the remediation activity is not preempted in order to avoid wasting efforts. The tool also allows the user to enable preemption. We considered three vulnerability prioritization policies. Policy-Severity prioritizes vulnerabilities based on their CVSS severity score, as reported in the NVD [23]. Policy-Importance ranks vulnerabilities according to the average importance of the nodes they affect, prioritizing those impacting more critical assets. Policy-Centrality prioritizes vulnerabilities affecting nodes with higher network centrality, capturing structural importance within the system graph and reflecting alternative interpretations of asset criticality [18]. We configured the parameters of the time-to-patch as follows (Eq. 5). The baseline patching time 𝛼 is set to 1 day. The defender efficiency factor 𝛽 is set to 0.5 for high effort, 1 for regular effort and 1.5 for low effort. The parameter 𝛾 is set to 1 for application-level vulnerabilities, and to 2 for OS ones, reflecting the higher complexity and operational impact of patching OS components. The weight 𝜅 that defines 𝛿 is set to 0.3 with linear growth, accounting for parallelization effects and operational efficiencies. Overall, this parameterization results in patching times ranging approximately from 1 day up to about 12 days in the worst case.
14 6
Della Penna et al. Evaluation
We analyze multiple scenarios, by considering every combination of the two APTs and of the two topologies. For each case, we analyze the three patching policies (Severity, Importance, Centrality), with the three configurations of the defender effort, and the case of no defender (None). Each scenario was executed over 100 independent episodes to ensure statistical significance. Experiments were run on a MacOS machine with M2 Pro processor and 16 GB RAM. On this hardware, agent training took approximately 4 minutes, while the full execution took approximately 12 minutes. Table 5 reports the percentage of episodes in which the attacker successfully reaches its goal (respectively, Exfiltration/Wiper and DoS on at least 3 nodes in the Database layer). In the absence of a defender, both attackers consistently achieve their objectives in all settings. When defenses are introduced, Policy-Importance provides the strongest mitigation effect. This is particularly evident for APT41 in the Layered topology, where increasing defender effort leads to a near-complete suppression of attacker success. For APT28, the reduction is less pronounced, with the attacker remaining more effective even under higher effort levels. This difference can be partly explained by the nature of the two campaigns, since APT28 pursues a simpler objective that requires only a single action (DoS) on the database nodes, while APT41 entails multiple actions to impact on these nodes, making it more sensitive to defensive interventions. In contrast, Severity and Centrality policies show limited impact across both campaigns, even at higher effort levels, with only marginal reductions in attacker performance. This is likely due to their focus on criteria not directly aligned with protecting the assets targeted by the attacker. Conversely, Importance explicitly prioritizes those assets that are most relevant to the attacker’s objectives, which explains its higher effectiveness. Table 5. Attacker Goal Achievement (%).
APT Defender
Effort
None
Low Regular High Low Regular High Low Regular High
Importance
Severity
Centrality
APT41 Layered 100 56 30 3 100 100 83 100 100 92
APT28 Topology Tree Layered Tree 100 100 100 48 98 98 16 94 93 0 81 57 100 99 98 100 99 97 72 94 90 100 100 96 100 100 96 81 99 95
To provide a deeper qualitative view of attacker and defender behavior, VulnGym produces a heatmap of the network in which the color intensity of each node reflects the average number of times that node was selected as the target of an action across episodes (Figure 6). As shown in Figure 6a, the attacker tends to traverse specific intermediate nodes to progress toward the final objective. For example, node 68 emerges as a key pivot point. This suggests that certain nodes are systematically exploited as stepping stones during the attack progression. Furthermore, several nodes within the database zone exhibit higher intensities, confirming that the attacker successfully reaches its objective. Figure 6b illustrates the behavior of the defender under Policy-Importance strategy. The defender indeed prioritizes critical assets, by focusing patching efforts on the database zone. The color intensity gradually fades as the distance
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
15
(a) Attacker vs. Policy-Importance
(b) Defender with Policy-Importance
(c) Defender with Policy-Severity Fig. 6. Heatmaps of attacker and defender node targets in a layered network topology.
from the database zone increases, showing that nodes distant from critical assets are patched less frequently. In contrast, Figure 6c depicts the defender’s behavior under Policy-Severity strategy, which targets vulnerabilities with the highest CVSS scores. The comparison highlights that addressing the most “severe” vulnerabilities does not necessarily protect critical resources. In this scenario, the database zone and pivotal nodes (e.g., node 68) receive less emphasis from the defender, which ultimately facilitates the attacker in reaching its goal. This limited effectiveness can be attributed to the fact that severity-based prioritization does not fully capture the characteristics of real-world exploited vulnerabilities,
16
Della Penna et al.
as reflected in the KEV catalog, where factors beyond CVSS scores play a crucial role. Moreover, we note that the policy gives higher priority to nodes in the external layer, which is less effective at protecting from attacks that originate from within the network due to phishing. VulnGym also reports several other metrics to support a deeper analysis of policies. The Network Vulnerability Index (NVI) represents the percentage of network nodes to which the attacker has gained access, relative to the total number of nodes in the network. Table 6 reports the corresponding values for each configuration. The results show that NVI generally decreases as the defender’s effort increases, reflecting a reduction in the overall fraction of compromised nodes. However, minimizing the number of compromised nodes does not necessarily make the attack more difficult, as patching may protect nodes that are of limited relevance to the attacker’s objective. This indicates that NVI captures the overall exposure of the network, rather than the attacker’s progress toward its specific target. Accordingly, policy effectiveness should be assessed with respect to the goal achievement rate, rather than the raw number of protected nodes. Table 6. Mean Network Vulnerability Index (%)
APT Defender
Effort
None
Low Regular High Low Regular High Low Regular High
Importance
Severity
Centrality
APT41 Layered 34 21 17 19 23 22 15 25 23 16
APT28 Topology Tree Layered Tree 39 29 22 19 18 17 14 13 14 12 6 8 26 19 21 25 19 19 13 7 9 29 18 19 25 16 18 12 8 8
The Time to Reach Goal (TTRG) represents the number of simulated days required for the attacker to reach its goal. Table 7 reports the corresponding values for each configuration. Higher TTRG values indicate that the attacker faces greater difficulty in succeeding, thereby providing defenders with a wider time window to detect the ongoing attack. The results show that, as the defender’s effort increases, the system’s defensive capability improves, requiring the attacker a greater number of days to reach the goal. In particular, Policy-Importance yields a significant increase in TTRG compared to the other policies. The difference in the number of days between the two campaigns is due to their different nature, as already discussed previously. The Time to Patch Vulnerability (TTPV) measures the average number of days that a policy takes to remediate a vulnerability since it is known. Table 8 reports the corresponding values for each configuration. The results show that, as the defender’s effort increases, fewer days are required to patch a vulnerability: with Low effort, approximately 10 days are needed on average, whereas with High effort this decreases to approximately 4 days. These results can be used as Key Performance Indicators for guiding vulnerability management: if the observed patching time falls within this range, the process can be considered adequate; otherwise, it may indicate an issue to be addressed.
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
17
Table 7. Mean Time To Reach the Goal (days)
APT Defender
Effort
None
Low Regular High Low Regular High Low Regular High
Importance
Severity
Centrality
APT41 Layered 125 215 243 315 165 178 236 150 158 216
APT28 Topology Tree Layered Tree 128 104 108 237 152 163 244 171 178 270 190 202 180 120 133 204 135 154 242 158 187 160 118 132 175 125 144 223 138 173
Table 8. Mean Time To Patch Vulnerability (days)
APT Defender
Importance
Severity
Centrality
Effort Low Regular High Low Regular High Low Regular High
APT41 Layered 13 10 4 10 7 4 10 7 4
APT28 Topology Tree Layered Tree 13 9 8 10 6 6 4 3 3 10 8 7 7 5 5 3 3 3 10 8 7 7 5 5 3 3 3
The Vulnerabilities in Backlog (VIB) reports the average number of vulnerabilities awaiting remediation at any given time. Table 9 reports the corresponding values for each configuration. The results show that, as the defender’s effort increases, the number of vulnerabilities in the backlog decreases: with Low effort, approximately 15 vulnerabilities remain in the backlog on average, whereas with High effort this decreases to approximately 5. Notably, the average VIB remains well above zero across all configurations, indicating that the defender does not have sufficient budget to promptly patch all known vulnerabilities. This observation is consistent with what is commonly reported in practice by security analysts, who remark the need for effective policies for vulnerability prioritization [1, 24, 49]. However, a low backlog length is not synonymous with defensive effectiveness: even when few vulnerabilities remain in the backlog, the attacker may still be able to reach the goal in a large fraction of cases. For instance, under Policy-Centrality with the APT41 campaign in the layered topology, the attacker achieves the goal in 92% of cases despite an average backlog of only 4 vulnerabilities. Taken together, NVI, TTRG, TTPV, and VIB provide complementary evidence on policy behavior. Low values of TTPV and VIB indicate a well-managed backlog and fast remediation, but they do not necessarily translate into a lower
18
Della Penna et al. Table 9. Mean Vulnerability In Backlog (#)
APT Defender
Importance
Severity
Centrality
Effort Low Regular High Low Regular High Low Regular High
APT41 Layered 17 14 5 15 11 4 15 12 4
APT28 Topology Tree Layered Tree 17 19 20 14 16 17 4 11 10 14 17 16 11 3 12 3 6 5 15 17 16 11 14 13 13 8 7
goal achievement rate for the attacker, since the effectiveness of a patching strategy also depends on how well it aligns with the attacker’s specific objective. Therefore, assessing policy effectiveness requires jointly considering NVI, TTRG, TTPV, and VIB together with the goal achievement rate and the nature of the attacker’s objective. 7
Threats to validity
We here analyze threats to the validity of our study, and the methodological countermeasures to mitigate adverse effects on our findings. As for external validity, the main threat is related to the realism of the modeled attacker. To mitigate this, we designed VulnGym to be highly configurable, allowing it to strictly align with data from Cyber Threat Intelligence (CTI), which provides information about exploited vulnerabilities and attack techniques adopted by real-world attackers. This approach leverages the best knowledge available from expert-analyzed CTI about APTs, which prioritizes welldocumented APTs over mere chronological recency; specifically, we focused on vulnerabilities from 2020, as they coincide with the peak activity period of the public-documented APTs. A second fundamental challenge relates to the representativeness of the network scenarios. Public CTI withholds sensitive details (e.g., network topology, node configurations) to preserve confidentiality of affected organizations. For example, about the Equifax breach mentioned before, only the exploited vulnerabilities are publicly documented, but the underlying network structure and configuration remain undisclosed. Therefore, in our experiments, we considered typical enterprise network architectures. Furthermore, we mitigated this threat by designing VulnGym to be fully configurable by practitioners to reflect their own organizations. As for internal validity, it concerns whether the underlying design choices are sound and complete. In particular, we adopted RL to orchestrate the ordering and timing of actions by attackers, with the action space grounded in MITRE ATT&CK. Similarly, the defender’s action space is currently focused on well-known policies for vulnerability management to orchestrate the actions of defenders. A direct consequence of this modelling choice is that the RL attacker is explicitly rewarded for targeting critical assets. While our abstraction might not capture every possible real-world attack or defense vector, we preserved this alignment because it realistically reflects goal-oriented APTs targeting an organization’s pivotal assets. A second potential threat to internal validity relates to parameter sensitivity. To mitigate this, we derived the RL hyperparameters from previous research in the field. Furthermore, we conducted
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
19
evaluations across different defender effort levels to ensure that the relative performance of the prioritization remains consistent across different operational constraints. Stochasticity in RL algorithms poses a threat to internal validity, as the attacker’s converged policy might depend on the random seed initialization. To mitigate this, the reported likelihood of success of the attack campaigns was measured over multiple independent simulation runs, ensuring that our evaluation of the defender policies is not biased by a single, potentially anomalous, RL training outcome. We designed VulnGym to be extendable in order to enable future research, as discussed in the next section. 8
Conclusion
This work presents VulnGym, a novel simulation tool for the quantitative evaluation of vulnerability management strategies. It captures the dynamics of complex attack campaigns in the context of continuous vulnerability management. By modeling different vulnerability management policies and the constraints imposed by a limited budget, it provides actionable insights into their effectiveness at mitigating APTs. Our experiments show highlight the need for vulnerability management strategies tailored to the organizational context, accounting for adversarial behavior, network topology, and asset criticality. VulnGym is a useful resource for security analysts, which can configure their own specific network environments and threat profiles. New APT profiles, network topologies, and vulnerabilities can be customized through configuration files, e.g., by specifying attack actions and targeted products for a specific APT, or by populating the network topology from existing IT asset management tools, such as CMDB systems. The tool is also a resource for further research on vulnerability management policies. The emergence of LLMbased vulnerability discovery will increase the volume of exploited vulnerabilities, which will require more accurate prioritization based on the organizational and threat context. New vulnerability management policies, such as policies based on the semantic analysis of the context and of the vulnerabilities through LLMs, are open research opportunities in the field. VulnGym can be easily extended with new prioritization policies to be evaluated. More broadly, VulnGym is open to further extensions to model additional aspects that interact with vulnerability management, including SOC teams and intrusion detection processes, honeypots, and moving target defense strategies. The agent-based architecture of VulnGym readily supports the integration of additional agents, such as a threat hunter agent performing periodic scans, a moving target defense agent relocating critical assets, or a topology update agent, which simulates network dynamism over time. These extensions do not require structural changes to the framework. We devised VulnGym as a basis to enable further research in vulnerability management. We release the full source code and experimental setup for future research. References [1] Saeed Abbasi and Qualys Threat Research Unit (TRU). 2026. The Broken Physics of Remediation. Technical Report. Qualys. https://cdn2.qualys.com/ docs/mktg/qualys-tru-the-broken-physics-of-remediation.pdf [2] Amrin Maria Khan Adawadkar and Nilima Kulkarni. 2022. Cyber-security and reinforcement learning - A brief survey. Eng. Applications of Artificial Intelligence 114 (2022). [3] Anthropic. 2026. Project Glasswing: Securing Critical Software for the AI Era. https://www.anthropic.com/glasswing. Accessed: 2026-07-05. [4] Arctic Wolf. 2025. 2025 Threat Report. https://www.arcticwolf.com/resource/arctic-wolf-threat-report-2025-lp/arctic-wolf-threat-report-2025 [5] Aman Bansal. 2020. Security Zoning in Network Architecture. https://medium.com/@aman.bansal93/security-zoning-in-network-architectureff7693b91556 [6] Norman Becker, Daniel Reti, Evridiki V. Ntagiou, Marcus Wallum, and Hans D. Schotten. 2024. Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN. arXiv:2407.15656 [cs.CR] https://arxiv.org/abs/2407.15656 [7] BitSight Technologies, Inc. 2017. A Growing Risk Ignored: Critical Updates. Technical Report. BitSight. https://info.bitsight.com/bitsight-insights-agrowing-risk-ignored-critical-updates.
20 [8] [9] [10] [11]
Della Penna et al.
Charles J. Brooks and Philip A. Craig Jr. 2022. Practical Industrial Cybersecurity: ICS, Industry 4.0, and IIoT. Wiley. CISA. 2026. Known Exploited Vulnerabilities Catalog (KEV). https://www.cisa.gov/known-exploited-vulnerabilities-catalog CISA. 2026. Stakeholder-Specific Vulnerability Categorization (SSVC). https://www.cisa.gov/stakeholder-specific-vulnerability-categorization-ssvc Cisco and Cyentia Inst. 2023. Prioritization to Prediction, Vol. 9: Role of the Known Exploited Vulnerability Catalog in Risk-Based Vulnerability Management. Technical Report. Cisco Systems. [12] Coalition, Inc. 2025. Cyber Threat Index 2025: Deciphering the Ransomware Playbook. https://web.coalitioninc.com/DLC-Cyber-Threat-Index2025.html [13] Cyber Security Works. 2024. Spotlight Report 2024: Ransomware Through the Lens of Threat and Vulnerability Management. Technical Report. [14] Sofia Della Penna, Roberto Natella, Vittorio Orbinato, Lorenzo Parracino, and Luciano Pianese. 2025. CTI-HAL: A Human-Annotated Dataset for Cyber Threat Intelligence Analysis. In 8th Workshop on Attackers and Cyber-Crime Operations (WACCO). [15] Ben Edwards. 2024. A Global View of the CISA KEV Catalog: Prevalence and Remediation. Technical Report. Bitsight Technologies. [16] Jianqing Fan, Zhaoran Wang, Yuchen Xie, and Zhuoran Yang. 2020. A theoretical analysis of deep Q-learning. In Learning for Dynamics and Control. [17] Farama Foundation. 2026. Gymnasium: An API standard for reinforcement learning with a diverse collection of reference environments. https: //gymnasium.farama.org/ [18] Katheryn A Farris, Ankit Shah, George Cybenko, Rajesh Ganesan, and Sushil Jajodia. 2018. VULCON: A System for Vulnerability Prioritization, Mitigation, and Management. ACM Trans. on Priv. and Sec. 21, 4 (2018). [19] Sylvia Feng and Muharman Lubis. 2022. Defense-in-depth security strategy in LOG4J vulnerability analysis. In Intl. Conf. Adv. in Data Science, E-learning and Inf. Sys. (ICADEIS). [20] Dave Ferguson. 2026. 2017 - 2026 Vulnerability Severity By Year. https://securityvulnerability.io/. [21] Fernao Group. 2024. CVSS: The 5 Biggest Misconceptions in Risk Assessment. https://www.fernao.com/de/blog/cvss-the-5-biggest-misconceptionsin-risk-assessment. [22] FIRST. 2025. Exploit Prediction Scoring System (EPSS). https://www.first.org/epss/ [23] FIRST. 2026. Common Vulnerability Scoring System (CVSS). https://www.first.org/cvss/ [24] FIRST.org. 2026. Mid-Year Vulnerability Forecast 2026. Technical Report. Forum of Incident Response and Security Teams (FIRST). https: //www.first.org/newsroom/releases/20260615 [25] Jerry Gamblin. 2026. CVE Publications by Year. https://cve.icu/. [26] Patrick Garrity. 2023. Insights Into Vulnerability Management. https://nucleussec.com/blog/insights-into-vulnerability-management-v1/. [27] Google LLC. 2026. APT groups and threat actors. https://cloud.google.com/security/resources/insights/apt-groups. [28] IBM Corp. 2026. Hierarchical Tree Topology. https://www.ibm.com/docs/en/informix-servers/15.0.x?topic=systems-high-availability-clusters-inhierarchical-tree-topology. [29] Insikt Group. 2024. Patterns and Targets for Ransomware Exploitation of Vulnerabilities: 2017–2023. Technical Report. Recorded Future. https: //www.recordedfuture.com/research/patterns-targets-ransomware-exploitation-vulnerabilities-2017-2023. [30] Syed Islam and Ankur Sand. 2024. The CVSS Deception: How We’ve Been Misled on Vulnerability Severity. https://i.blackhat.com/EU-24/ Presentations/EU-24-Islam-The-CVSS-Deception.pdf. [31] Caiyan Liu, Wentao Shen, Wenling Lyu, Xiaojie Xu, and Xufeng Ling. 2025. A Study on network architectures and security for small and mediumsized enterprises. In Proceedings of the 2025 8th International Conference on Computer Information Science and Artificial Intelligence (CISAI ’25). Association for Computing Machinery, New York, NY, USA, 1514–1519. doi:10.1145/3773365.3773603 [32] Lockheed Martin Corp. 2025. Cyber Kill Chain®. https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html. [33] Microsoft Corp. 2026. What is an advanced persistent threat (APT)? https://www.microsoft.com/en-us/security/business/security-101/what-isadvanced-persistent-threat-apt. [34] Microsoft Defender Research Team. 2021. CyberBattleSim. https://www.microsoft.com/en-us/research/project/cyberbattlesim/ [35] MITRE. [n. d.]. Initial Access. https://attack.mitre.org/tactics/TA0001/. [36] MITRE. 2026. APT28. https://attack.mitre.org/groups/G0007/. [37] MITRE. 2026. APT41. https://attack.mitre.org/groups/G0096/. [38] MITRE. 2026. CVE - Common Vulnerabilities and Exposures. https://www.cve.org. [39] MITRE Corp. 2025. MITRE ATT&CK: Adversarial Tactics, Techniques, and Common Knowledge. https://attack.mitre.org/. [40] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. arXiv preprint arXiv:1312.5602 (2013). [41] Luis Muñoz-González, Daniele Sgandurra, Martín Barrère, and Emil C Lupu. 2017. Exact inference techniques for the analysis of Bayesian attack graphs. IEEE Transactions on Dependable and Secure Computing (2017). [42] National Cyber Security Centre. 2023. Verify and regularly review your vulnerability management process. https://www.ncsc.gov.uk/collection/ vulnerability-management/guidance/verify-review-process. [43] National Institute of Standards and Technology (NIST). 2026. National Vulnerability Database (NVD). https://nvd.nist.gov. [44] NetworkX Developers. 2024. NetworkX: Software for Complex Networks. https://networkx.org/ [45] NIST. 2024. The NIST Cybersecurity Framework (CSF) 2.0. Technical Report CSWP 29. NIST. [46] Jason RC Nurse. 2025. To Patch or Not to Patch: Motivations, Challenges, and Implications for Cybersecurity. arXiv preprint arXiv:2502.17703 (2025).
VulnGym: Evaluating Vulnerability Management Strategies against Advanced Persistent Threats
21
[47] Oracle Corp. 2018. Deploying Services Gatekeeper in a Demilitarized Zone. https://docs.oracle.com/communications/E81149_01/doc.70/e95424/ sgsec_dmz.htm#SGSEC215 [48] Xinming Ou, Sudhakar Govindavajhala, Andrew W Appel, et al. 2005. MulVAL: A logic-based network security analyzer. In USENIX Security Symp. [49] Ponemon Institute LLC. 2019. Costs and Consequences of Gaps in Vulnerability Response. Technical Report. Ponemon Institute, sponsored by ServiceNow. https://media.bitpipe.com/io_15x/io_152272/item_2184126/ponemon-state-of-vulnerability-response-.pdf [50] ProjectDiscovery. 2026. Nuclei: A fast and customisable vulnerability scanner. https://docs.projectdiscovery.io/opensource/nuclei/. [51] PurpleSec. 2024. What is Vulnerability Management? (The Definitive Guide). https://purplesec.us/learn/what-is-vulnerability-management/. [52] Rapid7. 2026. Metasploit - Penetration Testing Software. https://www.metasploit.com. [53] Red Hat. 2019. Why CVSS Does Not Equal Risk: How to Think About Risk in Your Environment. https://www.redhat.com/en/blog/why-cvss-doesnot-equal-risk-how-think-about-risk-your-environment. [54] Michael Roytman. 2026. Capacity is King. https://research.empiricalsecurity.com/research/capacity-is-king. [55] Jonathon Schwartz and Hanna Kurniawati. 2019. Autonomous Penetration Testing using Reinforcement Learning. arXiv:1905.05965 [cs.CR] https://arxiv.org/abs/1905.05965 [56] SentinelOne. 2025. What is Continuous Vulnerability Management? https://www.sentinelone.com/cybersecurity-101/cybersecurity/what-iscontinuous-vulnerability-management/. [57] Deana Shick. 2018. Towards Improving CVSS. https://insights.sei.cmu.edu/blog/towards-improving-cvss/. [58] John Shier, Angela Gunn, and Hilary Wood. 2025. Sophos Active Adversary Report. https://news.sophos.com/en-us/2025/04/02/2025-sophosactive-adversary-report/ [59] Saurabh Singh, Pradip Kumar Sharma, Seo Yeon Moon, Daesung Moon, and Jong Hyuk Park. 2019. A comprehensive study on APT attacks and countermeasures for future networks and communications: Challenges and solutions. J. of Supercomp. 75, 8 (2019). [60] Anoop Singhal and Xinming Ou. 2017. Security risk analysis of enterprise networks using probabilistic attack graphs. In Network security metrics. Springer. [61] Xining Song, Yuting Li, and Decai Zhang. 2025. Design and Implementation of Building a Small and Medium sized Enterprise Office LAN. In Proceedings of the 2025 11th Annual International Conference on Network and Information Systems for Computers (ICNISC ’25). Association for Computing Machinery, New York, NY, USA, 221–225. doi:10.1145/3776942.3777008 [62] Murugiah Souppaya and Karen Scarfone. 2022. Guide to Enterprise Patch Management Planning: Preventive Maintenance for Technology. Technical Report SP 800-40r4. NIST. [63] Jonathan Spring. 2022. Probably Don’t Rely on EPSS Yet. https://www.sei.cmu.edu/blog/probably-dont-rely-on-epss-yet/. [64] UK National Audit Office. 2017. Investigation: WannaCry cyber attack and the NHS. https://www.nao.org.uk/reports/investigation-wannacrycyber-attack-and-the-nhs/ [65] Ping Wang and Christopher Johnson. 2018. Cybersecurity incident handling: A case study of the Equifax data breach. Issues in Inf. Systems 19, 3 (2018). [66] Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning 8 (1992), 279–292. [67] Jacob Wiebe, Ranwa Al Mallah, and Li Li. 2023. Learning cyber defence tactics from scratch with multi-agent reinforcement learning. arXiv preprint arXiv:2310.05939 (2023). [68] Wiz, Inc. 2025. What is Continuous Vulnerability Management? https://www.wiz.io/academy/continuous-vulnerability-management. [69] Awad A Younis and Yashwant K Malaiya. 2015. Comparing and evaluating CVSS base metrics and Microsoft rating system. In IEEE Intl. Conf. on Software Quality, Reliability and Security (QRS).