arXiv CS open access

ADAPTD: Adaptive Detection and Proactive Threat Defense for Autonomous APT attacks

2026 · arxiv_cs

arXiv CS · Papers · 2026

cryptography cybersecurity privacy security

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

ADAPTD: Adaptive Detection and Proactive Threat Defense for Autonomous APT Attacks Yeongwoo Kim∗ , Quanyan Zhu† and György Dán∗ ∗ Department of Network and Systems Engineering, KTH Royal Institute of Technology, Stockholm, Sweden † Department of Electrical and Computer Engineering, New York University, New York, USA

arXiv:2608.17251v1 [cs.CR] 18 Aug 2026

Email: ∗ {yeongwoo, gyuri}@kth.se, † qz494@nyu.edu

Abstract—Advanced persistent threat (APT) actors increasingly employ sophisticated techniques to propagate laterally through segmented enterprise networks. Timely detection and defense depend on cross-subnetwork coordination, yet maintaining global situational awareness generates substantial communication overhead. To manage this tradeoff, flexible monitoring and adaptable containment are imperative. This paper presents ADAPTD, a communication- and computation-efficient, decisiontheoretic framework integrating: (i) compact kill chains for identifying diverse attack vectors, (ii) an immediate blocking mechanism for timely containment, and (iii) a predictive eviction strategy to restore system security. Our experiments validate ADAPTD’s effectiveness across diverse threat scenarios. First, our decentralized belief update scheme outperforms state-ofthe-art diffusion HMM. Second, ADAPTD substantially reduces false evictions compared to transformer-based detection. Third, under noisy environments, adaptive blocking contains attackers while minimizing unnecessary disruption. Lastly, the ablation study confirms that combining two defensive actions significantly reduces the defender’s total cost. Index Terms—Advanced persistent threat, hidden Markov model, incident response

I. I NTRODUCTION An advanced persistent threat (APT) is a threat actor that employs various techniques and tools to gain illegitimate access to networked systems, often over an extended period of time [1]. APT may involve human threat actors, but there have also been cases of self-propagating malware (e.g., Stuxnet autonomously compromised and destroyed nuclear facilities), causing significant financial loss to organizations [2]. Such autonomous threats have then been reported in critical infrastructures, such as financial institutions [3], government agencies [4], the healthcare industry [5], and the energy sector [6]. A key challenge posed by such attacks is their ability to exploit diverse attack vectors ranging from social engineering to USB flash drives [7], enabling attacks even against well protected corporate infrastructures. Once compromised, an entire networked system can be infected autonomously through lateral movement via connected subnetworks, without the need to communicate with a command-and-control server, i.e., without any form of remote coordination. The diversity of attack vectors implies that, from the perspective of the defender, the starting point of an attack is hard to predict. As attacks propagate, defenders have to i) account for all possible initial attack vectors and ii) strategically combine

SOC

3 1 2

Corporate Subnetwork (𝑛2 )

4 Control Subnetwork (𝑛4 )

DMZ Subnetwork (𝑛1 )

Substation Subnetwork (𝑛5 )

Corporate Subnetwork (𝑛3 )

Fig. 1. Illustration of diverse attack vectors of a ransomware that autonomously propagates to a targeted critical asset. In the example, 1. The malware compromises the DMZ subnetwork by using unauthorized USB access. 2. It penetrates the firewalls between DMZ and control subnetworks. 3. The corporate network detects the attack. 4. The detection leads to the blocking of all traffic from DMZ and corporate subnetworks. 5. The attack compromises the substation subnetwork.

diverse defensive actions to prevent the compromise of critical assets, while maintaining a certain level of business continuity. Effective threat response relies on real-time situational awareness (SA), which depends on efficient processing of alerts triggered by intrusion detection systems (IDSs). In centralized approaches, beliefs are computed by aggregating alerts at the machine or host level [8], [9]. Concurrently, another line of work utilizing provenance- or behavior-based detection approaches (e.g., HOLMES [10]) has demonstrated remarkable capabilities in detecting APTs by aggregating system events. However, this aggregation introduces substantial communication overhead, underscoring the need for a bandwidth-efficient belief update mechanism. Designing such a decentralized scheme, however, poses a significant technical challenge, as maintaining global SA without incurring prohibitive communication cost among distinct subnetworks is inherently difficult. SA is a prerequisite for threat response, but it is not sufficient on its own. Given a belief about the attacker’s state, existing intrusion response systems (IRSs) typically deploy lightweight defensive actions, such as revoking credentials or blocking access, to delay the attacker’s progression [11]– [13]. While such actions are attractive due to their low cost and limited disruption, relying on them exclusively can lead to prolonged engagement, eventual compromise of critical assets, or continuous defensive overhead that impacts business continuity. To address this, effective threat response must combine lightweight actions with heavyweight defenses, such

as evicting compromised components. However, integrating these two classes of defenses in a principled and adaptive manner, particularly under uncertainty about the attacker’s progression, remains an open decision-theoretic problem. Fig. 1 shows an attack scenario in which an autonomous APT compromises the demilitarized zone (DMZ), and subsequently it compromises the substation subnetwork. The defender could have protected the substation subnetwork from the attacker if it had blocked outgoing traffic from the corporate subnetwork in a timely manner. Nonetheless, to do so with minimal disruption to business continuity, the defender needs SA, i.e., an accurate belief about the attack’s progression, as well as a cost-aware decision rule for when to isolate subnetworks and when to shut down parts of the system for evicting the attacker. In this paper, we propose a decision-theoretic defense framework with decentralized belief update, called ADAPTD, that i) decomposes large-scale kill chains into smaller, subnetwork-specific chains; ii) blocks lateral movement for attack mitigation and for more accurate decision-making; and iii) estimates the future cost of mitigation actions for deciding when to evict the attacker from the system. Unlike provenanceor behavior-based detection systems (e.g., HOLMES [10]) that aim to generate high-level alerts using low-level system events, ADAPTD operates as a decision-theoretic engine that ingests these high-level alerts and optimizes proactive defense under system-wide resource constraints. Consequently, the primary objective of our work is to establish a rigorous framework for defensive decision-making under uncertainty, rather than refining passive detection algorithms. Given this decisiontheoretic focus on the systemic evaluation of dynamic defense policies, comprehensive numerical simulations serve as a rigorous environment to mathematically validate our framework across diverse attacker behaviors and system sizes. The contributions of our work are as follows: • Decentralized situational awareness: We propose to decompose the attacker’s kill chain into subnetworkspecific kill chains, allowing each subnetwork to maintain its own belief state based on local alerts and share it only with neighboring subnetworks. This decentralized approach reduces communication overhead compared to a centralized approach at comparable estimation accuracy. • Proactive Defense: Our proposed framework proactively isolates suspicious subnetworks to contain the propagation of APTs until the attack is confirmed. Outgoing traffic from subnetworks is blocked based on the belief about the attacker’s progression taking into account its impact on business continuity. • Minimum Cost Eviction: We employ Monte Carlo (MC) simulations based on the belief to find the optimal time for evicting the attacker from the system taking into account the expected future cost of blocking actions. • Numerical Evaluation: We use extensive numerical simulations to evaluate the proposed policies against state-ofthe-art baselines in segmented environments. Our results show that ADAPTD outperforms the baselines in terms

of defense cost, demonstrates robust performance against diverse attack vectors, and has lower computational and communication overhead. The rest of the paper is organized as follows. We discuss related work in Section II. We formulate the system model in Section III and present the ADAPTD in Section IV. Section V provides an evaluation of ADAPTD, and Section VI concludes the paper. II. R ELATED W ORK IDSs generate alerts to identify the attacker’s malicious activities [10], [14]–[20]. Phillip et al. used signatures of known attacks to generate alerts [14], which, however, cannot detect unknown attacks. The authors of [15] used a data mining algorithm to learn rules to detect anomalous activities. However, this approach raises numerous false alerts, causing alert fatigue. In [16], the authors correlated communication between internal and external devices to reduce the number of alerts. Pei et al. correlated dispersed malicious activities in log data using community detection [17]. The authors in [10], [18] constructed a graph to correlate suspicious system calls, aiming to detect APT attacks. Alrubayyi et al. used a negative-positive-selection algorithm to detect the novel malware attack [19]. Kim et al. considered the investigation of alerts with dynamic prioritization of alerts [20]. While these state-of-the-art tools achieve high detection accuracy, they operate as passive monitors and lack a cost-aware mechanism for active defense. Therefore, they are complementary to our framework; they serve as upstream alert generators, whereas ADAPTD functions as a top-level decision-theoretic engine that optimizes mitigation actions under uncertainty. IRSs choose defense actions autonomously upon detecting malicious activity [11]–[13], [21]–[29]. Xiao et al. [12] formulated a Stackelberg game to choose lightweight defensive actions aiming to mitigate malware propagation, but do not consider that malware may use different attack vectors. Kreidl et al. [21] used a partially observable Markov decision process for maintaining SA regarding the security state of the host and to select defense actions, considering the defender’s cost [11]. The authors in [22] developed a hierarchical approach comprised of local and global engines. The local engine maintains the SA on an attack-response chain, while the global engine aggregates the SA from local engines and chooses the defense actions. Iannucci et al. [23], [24] chose a response action to counteract the attacks, considering a response that may mitigate multiple attacks. Miehling et al. [11], [25] used Bayesian attack graphs to choose defensive actions where the action changes the attacker’s available exploits, improving SA. Considering strategic attackers [29], [30], Jia et al. [29] formulated a Bayesian–Stackelberg game to model attackerdefender interactions under incomplete information, whereas Singh et al. [30] formulated the attacker–defender interaction as a linear quadratic differential game to design defensive strategies under asymmetric information. Similarly, the authors in [13], [27] modeled networked systems or individual devices using Markov models to guide defensive decision-making. The belief state, updated via Bayes’ rule, was then used to select

Subnetwork 𝒊

: State transition Reconnaissance 𝒔𝒊,𝟐

Access 𝒔𝒊,𝟏

Alert 1 (𝑦𝑡𝑖,1 )

: Observation Lateral movement 𝒔𝒊,𝟑

Alert 1 (𝑦𝑡𝑖,1 )

…

Alert 𝑘 (𝑦𝑡𝑖,𝑘 )

…

Alert 8 (𝑦𝑡𝑖,8 )

Fig. 2. A subnetwork kill chain for i ∈ {1, . . . , I}. The row of gray nodes represents the attack states in subnetwork i ∈ I\I c . Black arrows denote state transitions, while blue arrows indicate observations conditioned on the state. The attacker repeatedly conducts lateral movements across transient networks in an effort to reach one of the critical subnetworks.

appropriate defensive actions (e.g., blocking IP addresses, allocating honeypots). In a related line of work, [26], [28] employed epidemiological models to determine mitigation or remediation strategies, respectively. However, these approaches do not consider the combination of lightweight and heavyweight defenses; lightweight defenses can be effective in delaying the attacker’s progression, whereas heavyweight defenses (e.g., eviction) are necessary for restoring system integrity. Our proposed solution is closely related to the zero trust architecture that considers hosts outside a corporate firewall malicious and requests verification to access a network [31]. In contrast to this work, our work considers that the attacker is already in the corporate network, and firewalls between subnetworks have default rules to enable regular communications, which leads to more effective blocking of subnetworks. III. S YSTEM AND ATTACK M ODEL We consider a system that consists of a set I = {1, . . . , I} of subnetworks separated by firewalls, in accordance with state-of-the-art practice in industrial control systems [32], as illustrated in Fig. 1. We denote by I ↑ (i) the set of upstream subnetworks relative to subnetwork i (e.g., I ↑ (4) = {1, 2, 3}), by Id↑ (i) ⊆ I ↑ (i) the set of directly connected upstream subnetworks relative to subnetwork i (e.g., Id↑ (4) = {2, 3}), by I ↓ (i) the set of downstream subnetworks relative to subnetwork i (e.g., I ↓ (3) = {4, 5}), and by Id↓ (i) ⊆ I ↓ (i) the set of directly connected downstream subnetworks from subnetwork i (e.g., I(3) = {4}). We denote the set of critical subnetworks by I c ⊂ I; subnetworks i ∈ I c contain critical assets, making it imperative to secure these subnetworks. We make the reasonable assumption that the firewalls between neighboring subnetworks filter traffic based on rules that can be updated in real-time. A. Attack model 1) Attack states: We model the attacker’s progression within a subnetwork by transitions between attack states, where each attack state corresponds to a tactic in the MITRE ATT&CK framework [33]. Let us denote the set of attack state indices by J = {1, . . . , j, . . . , J} and the set of attack states in subnetwork i by S i = {si,1 , . . . , si,j , . . . , si,J }. We adopt the convention that si,1 corresponds to access to subnetwork i and si,J corresponds to lateral movement from subnetwork i to a downstream subnetwork. Since all subnetworks are initially

clean, we denote this completely clean state by s0,0 . The overall state space is therefore S = ∪i∈I S i ∪ {s0,0 }. The objective of the attacker is to compromise assets within one of the critical subnetworks, which can be done by reaching state si,1 for some i ∈ I c . Fig. 2 illustrates three attack states in a kill chain (i.e., J = 3): initial access si,1 , reconnaissance si,2 , and lateral movement si,3 . The initial access state si,1 involves the use of various exploits, such as exploiting external remote services and public-facing applications, compromised removable media, and abused valid accounts. The reconnaissance state si,2 includes malicious activities such as vulnerability and IP scanning, aiming at identifying vulnerable hosts in neighboring subnetworks. Finally, the lateral movement state si,3 uses tactics such as exploiting remote services, abusing the and hijacking remote sessions to compromise neighboring subnetworks. Notably, techniques such as remote-service exploitation and the abuse of valid accounts can be used in both initial access and lateral movement. To compare these two phases, we analyze the techniques and their associated procedure examples in the ATT&CK Enterprise matrix [33]. The analysis shows that 58% of the procedure examples associated with lateral movement also appear among the procedure examples for initial access. At the technique level, these overlapping procedures correspond to 89% of all lateral-movement techniques. This substantial overlap motivates our modeling choice: access to a downstream subnetwork i′ ∈ Id↓ (i) following lateral movement in subnetwork i is modeled as an instance of initial access. Given that the attacker’s goal is to compromise a critical subnetwork, in Fig. 2, we omit minor attack steps unrelated to the lateral movement. This is consistent with the notion of a high-level kill chain [8], which demonstrated that some attack steps can be excluded while detecting attacks successfully. 2) Attacker progression: Time is considered to be slotted and is represented by t ∈ Z+ , where Z+ denotes the set of positive integers. We denote the attack state at time t by St ∈ S, and we consider that S0 = s0,0 , i.e., all subnetworks are initially not compromised. The attacker then selects a subnetwork and initiates an attack. Let us denote by q i the probability that the attacker selects subnetwork i and by P(St+1 = si,1 | St = s0,0 ) = 1 − P(St+1 = s0,0 | St = s0,0 ) that the probability of initiating attack at time t. We make the reasonable assumption that the initialPcompromise does not happen in a critical subnetwork, i.e., i∈I\I c q i = 1, that is, ′ q i = 0 if i′ ∈ I c . This assumption is reasonable considering the stringent security controls mandated by standards such as IEC62443 [32]. We denote the probability that the attacker progresses from ′ ′ state si,j to state si ,j between subsequent time slots by ′ ′ ′ ′ ai,j→i ,j = P(St+1 = si ,j | St = si,j ). As we consider an APT attacker, we make the reasonable assumption that the attacker aims to compromise a critical subnetwork without re-compromising subnetworks that have already been compromised. Therefore, the attacker tries to transition to a

ADAPTD Framework

Attack

Subnet 1

Subnet 2

Subnet 3

Adaptive Detection

Proactive Threat Defense

Decentralized Belief Update

Blocking of Lateral Movement SOC

SOC

Belief aggregation

𝑖,𝑗|ℎ

𝜋𝑡,𝑝

Subnet 1

Subnet 2

Subnet 3

S O C

3,𝑗|ℎ

𝜙𝑡

Inter-subnet belief sharing

Sample 1:

… …

2,𝑗|ℎ

𝜙𝑡

Subnet 2

Predictive eviction … …

1,𝑗|ℎ

𝜙𝑡

Subnet 1

Sample 𝑚: Sample 𝑀:

Fig. 3. Illustration of the ADAPTD framework. The attacker initiates the intrusion from subnetwork 1. As the attack propagates, each subnetwork independently maintains its partitioned belief to track the attacker’s progression, while the SOC aggregates these beliefs. Based on the aggregated belief, threat blocking contains the attacker, and predictive eviction removes the attacker from the system when the expected cost of continued blocking exceeds the predefined eviction cost.

directly connected downstream subnetwork i′ ∈ Id↓ (i) from subnetwork i. Recall that state si,J corresponds to lateral movement, thus a transition from subnetwork i ∈ I to a direct downstream subnetwork i′ ∈ Id↓ (i) would correspond to a ′ transition to si ,1 , i.e., initial access on subnetwork i′ . B. Defender model 1) Observations and Belief: The defender can observe alerts generated by the IDS, but cannot directly observe the attacker’s underlying state. Considering that the IDS does not trigger true alerts when the attack is not present in subnetwork i, the defender extends the kill chain in Fig. 2 by prepending the clean state si,0 and appending the foothold state si,J+1 , which represent an uncompromised subnetwork and inactive exploits after the attacker has left the subnetwork, respectively. Thus, we denote the defender’s states by S ′ = ∪i∈I S i ∪i∈I {si,0 , si,J+1 }, the set of alert indices in a subnetwork by K = {1, . . . , k, . . . , K}, and the alerts from subnetwork i observed at time slot t by Yti = [Yti,k ] ∈ {0, 1}K , where Yti,k = 1 indicates an alert, and Yti,k = 0 indicates no alert. The alerts across all subnetworks at time t are denoted by the matrix YtI = [Yti ] ∈ {0, 1}I×K . Finally, the tensor of alerts available to the defender at time t is denoted by Y1:t = [YtI′ ] ∈ {0, 1}t×I×K for t′ ∈ {1, . . . , t}. ′ We denote by δ i ,k|i,j the probability of a true alert k in ′ subnetwork i ∈ I given state si,j . We denote the probability ′ of a false alert k in subnetwork i′ by ζ i ,k , which is independent of the attack state. The probability of observing alert k in subnetwork i′ given attack state si,j at time t can thus be expressed as ′ ′ P(Yti ,k = yti ,k |St = si,j ) ( ′ ′ ′ (1 − δ i ,k|i,j )(1 − ζ i ,k ), yti ,k = 0, = ′ ′ ′ 1 − (1 − δ i ,k|i,j )(1 − ζ i ,k ), yti ,k = 1. Since the alerts are conditionally independent given the attack state, we can express the probability of observing a particular alert vector for subnetwork i′ ∈ I given a state at time t as: Y ′ ′ ′ ′ P(Yti = yti |St = si,j ) = P(Yti ,k = yti ,k |St = si,j ). k∈K

Then, the probability of observing alert vector ytI in state si,j can be expressed as Y ′ ′ P(YtI = ytI |St = si,j ) = P(Yti = yti |St = si,j ). i′ ∈I

2) Actions: In every time slot, the defender can update the network access control lists (ACLs) (e.g., firewall rules), which it can use for blocking and unblocking traffic between subnetworks. We use Dt ∈ P(I) to denote the set of subnetworks from which the defender blocks traffic towards their downstream subnetworks. That is, for a subnetwork i and its direct downstream subnetwork i′ ∈ Id↓ (i), the transition probability becomes ( 0, i ∈ Dt , i,J→i′ ,1 at = ′ ai,J→i ,1 , i ∈ / Dt , i.e., the attacker cannot access subnetwork i′ through lateral movement from subnetwork i ∈ Dt . To limit the impact on business continuity, we consider that the defender’s blocking decision is subject to the constraint |Dt | ≤ B where B ∈ N is the blocking budget. As an alternative, the defender can decide to escalate the incident to the emergency response team for immediate eviction of the attacker. We refer to this as the decision to evict, and we denote the decision at time t by τt ∈ {0, 1}, where τt = 0 means continuing regular operation, and τt = 1 means eviction of the attacker. Upon eviction, computers in all subnetworks are reinstalled to evict the attacker. 3) Belief update: Based on the observations and its actions, i,j the defender can maintain a belief πt+1 about the attacker’s progression using the forward algorithm [34] as i,j I πt+1 = F i,j (πt , yt+1 , Dt ), i,j I I = ηt · αt+1 · P(Yt+1 = yt+1 |St = si,j ),

(1)

I where F i,j (πt , yt+1 , Dt ) is the forward algorithm using the I from all subnetworks, previous belief πt , current alerts yt+1 and the previous blocking decision Dt . Each term in (1) is X ′ ′ ′ ′ i,j = πti ,j · ait ,j →i,j , αt+1 (2) i′ ∈I,j ′ ∈J

ηt = P

i,j I I i,j i∈I,j∈J αt+1 P(Yt+1 = yt+1 |St = s )

It is worth noting that the forward algorithm has to collect alerts from multiple subnetworks to compute belief. This can make it challenging to maintain the belief πt in real-world systems. 4) Cost Model: Our cost model captures four aspects: the cost of a compromise, the cost of network reconfiguration, its impact on business continuity, and the cost of false eviction.

i∈I i′ ∈I ↓ (i)

−

X d X

′

ri,i + τt cτ (1−Π0,0 t ),

(3)

i∈I\Dtρ i′ ∈I ↓ (i) d

where ρ is a policy for blocking and eviction. In (3), the first term is the risk of the attacker compromising another subnetwork at time t + 1. The second term is the cost of reconfiguring the firewall. The third term is the value of connectivity between non-blocked subnetworks. Finally, the fourth term is the cost incurred by falsely evicting the attacker. The defender’s objective is to minimize the cost of a networked system, subject to a constraint on the blocking budget B. We denote by γ ∈ [0, 1] the discount factor, by ρD ∈ PD the policy for blocking subnetworks, and by ρτ ∈ Pτ the policy for eviction. Then, we express the defender’s objective (∞ ) as X E γ t c(Πt , DtρD , τt )1Pt−1 min τt′ =0 |Π0 = π0 , ′ ρD ∈PD ,ρτ ∈Pτ

s.t

t=1

t =1

i,j I Πi,j t+1 = F (Πt , Yt , Dt ),

DtρD = ρD (Πt ), τt = ρτ (Πt ), |DtρD | ≤ B, where 1 is an indicator function. The defender’s problem is thus a partially observable Markov decision process (POMDP) with an infinite horizon, whose length depends on the defender’s policy. A straightforward approach to solving the above problem would be to train a defender policy using deep reinforcement learning (RL) [35]. However, this approach would not provide explainability [36] and would require extensive training of |I \ I c | RL models due to the random initial compromise. Instead, our objective is to develop a solution that (i) minimizes the defender’s cost, and at the same time (ii) is explainable and (iii) has low communication overhead. In what follows we present ADAPTD, a framework and algorithms for achieving the above design objectives. IV. ADAPTD F RAMEWORK Fig. 3 illustrates the interaction between the attacker and the defender in our proposed framework. The framework is

Clean 𝑠 𝐢,𝟎|𝟎

Attack (ℎ = 1):

Clean 𝑠 𝐢,𝟎|𝟏

Access 𝒔𝒊,𝟏|𝟏

Recon’ 𝒔𝒊,𝟐|𝟏

Attack (ℎ = 𝑖 −1):

Clean 𝑠 𝐢,𝟎|𝒊−𝟏

Access 𝒔𝒊,𝟏|𝒊−𝟏

Attack (ℎ = 𝑖):

Clean 𝑠 𝐢,𝟎|𝒊

Access 𝒔𝒊,𝟏|𝒊

Dynamic transition

Static transition

Lateral move’ 𝒔𝒊,𝟑|𝟏

Foothold 𝒔𝒊,𝟒|𝟏

Recon’ 𝒔𝒊,𝟐|𝒊−𝟏

Lateral move’ 𝒔𝒊,𝟑|𝒊−𝟏

Foothold 𝒔𝒊,𝟒|𝒊−𝟏

Recon’ 𝒔𝒊,𝟐|𝒊

Lateral move’ 𝒔𝒊,𝟑|𝒊

Foothold 𝒔𝒊,𝟒|𝒊

…

We consider that the defender is a rational entity. We can express the perceived cost of the defender at time t as X X ′ ρ i,J→i′ ,1 cic Πi,J +cd |Dtρ \ Dt−1 | c(Πt , Dtρ , τt ) = t a

Hypotheses for the partition belief filtering Clean (ℎ = 0):

…

We denote the cost of the compromise of subnetwork i by cic . We consider that the act of reconfiguring the ACL incurs cost cd per subnetwork. Furthermore, we model business continuity ′ in terms of the value ri,i of connectivity between pairs of ′ subnetworks i and i , hence blocking traffic will result in corresponding loss of revenue. Finally, we denote the cost of false eviction by cτ , representing the cost of unnecessary reinstallation of system components even though they were not compromised. C. Problem Formulation

Fig. 4. Hypotheses in subnetwork i for the partitioned belief filtering HMM. Each oval represents a state, and each subnetwork contains a sequence of states. Hypothesis h corresponds to an attacker initiating access from subnetwork h and subsequently reaching subnetwork i.

designed to enable agile attack detection and to mitigate lateral movement and the compromise of critical assets. ADAPTD consists of three core modules: • Decentralized Belief Update: This module enables to maintain real-time SA at low overhead. The core idea is that each subnetwork maintains a belief about the attacker progression in the subnetwork based on the locally generated alerts and based on the beliefs of neighboring subnetworks. Decentralized belief update allows low communication overhead compared to sharing alerts across all subnetworks (see Sec. IV-A). • Blocking of Lateral Movement: This module isolates subnetworks to block the attacker’s progression. The decision to isolate subnetworks is taken based on the current belief, computing the cost reduction by blocking one-step progression of the attack and greedily selecting subnetworks to block (see Sec. IV-B). • Predictive Eviction: This module estimates the expected blocking cost via MC simulations and determines whether eviction would be beneficial. Eviction is triggered when the expected blocking cost exceeds the termination cost. (see Sec. IV-C). In what follows, we present the three modules in detail. A. Decentralized belief update The centralized belief update in Sec. III-B3 requires aggregating all raw alerts at a single node. Such a scheme is vulnerable to link failures and is constrained by bandwidth and energy resources [37], [38]. These limitations motivate the use of decentralized belief update, which mitigates both communication and reliability concerns. However, existing decentralized approaches typically rely on consensus among subnetworks, incurring additional communication rounds. To address this issue, we propose partitioned belief filtering, which employs dynamic transition probabilities and eliminates the consensus phase to reduce communication overhead. We capture diverse attack vectors through a set of hypotheses. Let us denote by H = {0, . . . , I} the set of hypotheses representing the initiation of attack from subnetwork h ∈ H where h = 0 corresponds to the clean scenario, by Hi = I ↑ (i) ∪ {i} ∪ {0} the set of hypotheses that can compromise subnetwork i (e.g., H4 = {0, 1, 2, 3, 4}), by I h = {h}∪I ↓ (h) the set of reachable subnetworks starting from subnetwork h (e.g., I 2 = {2, 4, 5} in Fig. 1), by si,j|h the attack state si,j

i,j→i′ ,j ′ |h

under hypothesis h ∈ H, at the dynamic transition i,j|h probability under hypotehsis h at time t, and by ϕt the partitioned belief for state si,j|h under hypothesis h ∈ H. For intuitive understanding of partitioned belief filtering, we consider a networked system comprised of two subnetworks where |Id↓ (1)| = 1, i′ ∈ Id↓ (2) and |Id↑ (2)| = 1. The probability of the attacker transitioning to the next subnetwork is a2,0→2,1 = 0 (.resp a2,0→2,1 = a1,J→2,1 ) when the t t attacker’s state satisfies St−1 ̸= s1,J (.resp St−1 = s1,J ). However, because the defender cannot observe the attacker’s true state, we instead rely on the partitioned belief in the 1,J|h upstream subnetwork, ϕt . This motivates modeling the transition probability from the clean state to the access state as a dynamic quantity that depends on the defender’s belief about lateral movement in upstream subnetworks. Each subnetwork i,J|h therefore shares only its lateral-movement belief, ϕt , with ↓ its direct downstream subnetworks i′ ∈ Id (i). Fig. 4 illustrates a clean hypothesis in subnetwork i and |{i} ∪ I ↑ (i)| attack hypotheses. Each kill chain follows the same structure as Fig. 2, except that transitions from the clean state si,0|h to the access states si,1|h are dynamically modified for hypotheses h ̸= i. For h = i, no dynamic adjustment is required since the attacker initiates its threat directly from subnetwork h. For h ̸= i, the attacker can transition into subnetwork i from any of its direct upstream subnetworks i′ ∈ Id↑ (i) ∩ I h . Therefore, the dynamic transition probability has to aggregate the contributions from all such upstream subnetworks. For hypothesis h and subnetwork i, a subnetwork-level path is a finite sequence p = (p0 , p1 , . . . , pl , . . . pL ), where L is the length of the sequence. This sequence satisfies p0 = h,

pL = i,

p ,J→pl+1 ,1

at l

> 0.

The set of all such directed paths is denoted by p →p P(h → i) = {p|p0 = h, pL = i, at l,J l+1,1 > 0}. The probability of a path p is L−1 Y p →p ,1 P (p) = at l,J l+1 . l=1

For an upstream subnetwork i′ ∈ Id↑ (i) ∩ I h , the transition weight under hypothesis Ph is defined as ′ p∈P(h→i) 1{i′ ∈p} P (p) P . ω i |i,h = p∈P(h→i) P (p) This quantity represents the probability that the attacker reaches subnetwork i via upstream subnetwork i′ under the given hypothesis h. For a subnetwork i, combining upstream beliefs and transition weights, the dynamic clean-to-access transition probability for subnetwork i is ( a0,J→i,1 , for h = i, i,0→i,1|h ′ at = P i′ |i,h i ,J|h i′ ,J→i,1 ϕt−1 a , otherwise, i′ ∈I h ∩I ↑ (i) ω d

i,j ′ →i,j|h

′

while, for j ′ ̸= 0 and j ̸= 1, at = ai,j →i,j . Then, we then model the attacker’s transition as.  i,0|h i,0→i,0|h  ϕt−1 · at ,     i,j|h for′ j = 0, ′ P άt = i,0|h i,0→i,j|h i,j |h i,j →i,j|h  ϕ · a + ϕ · a , ′  t−1 t t t−1 j ∈J    otherwise, (4) which uses the prior belief and the corresponding transition probabilities. Next, we compute the likelihood using the probability of local alerts, i,j|h i,j|h (5) αt = P(Yti = yti |St = si,j )άt . The belief in subnetwork i can be obtained by normalization: i,j|h i,h i,j|h ϕt = ηt,p αt , (6) i,h where ηt,p = P

1 i,j|h j∈J i αt,p

. Then, we can combine the local

beliefs and obtain the aggregated belief as, Y i′ ,J+1|h Y i′ ,0|h i,j|h i,j|h h πt,p = ηt,p ϕt ϕt ϕt ,

(7)

i′ ∈I h \I ↑ (i)

i′ ∈I h ∩I ↑ (i)

h is a normalization factor for hypothesis h. Now, where ηt,p let us analyze the similarity between the central and the aggregated partitioned beliefs using Kullback–Leibler (KL) divergence. To this end, we introduce two assumptions for the partitioned belief as follows:

Assumption 1 (Lower-Bounded Belief). There exists ϵ > 0 such that for all i ∈ I and j ∈ J , i,j|h

i,j|h

ϕt−1,p ≥ ϵ, and πt−1 ≥ ϵ. This assumption implies that we use ϵ in place of 0. This is a necessary choice to ensure numerical stability, as the KL divergence involves a logarithm, which is undefined at 0. Assumption 2 (Bounded Transition Probability). The transition probabilities satisfy: ′

′

q i ,j →i,j ∈ [qmin , qmax ] ⊂ (0, 1). Under these two assumptions, we can compute the upperbound of the KL divergence between central and partitioned belief filtering beliefs as, Proposition 1. The KL divergence satisfies ·,·|h ·,·|h DKL (πt ∥ πt,p ) ≤ log

qmax P

j ′ ∈J i|h ϵ

|I h |+1 q |I | min

Proof. See appendix. B. Threat blocking The computational cost of predictive blocking based on MC simulation is significant [11], [25], and poses a fundamental challenge for real-time decision-making in large-scale networks. Instead, we propose a computationally simple approach to subnetwork blocking, based on sequential hypothesis testing. Since, in our problem, there are multiple hypotheses

and corresponding beliefs (in different subnetworks), the challenge lies in identifying the hypothesis and the belief to be used. The proposed approach selects the subnetwork to be used by comparing their likelihood ratios at time t and then determines the most likely hypothesis within that subnetwork. For subnetwork i, the likelihood ratio is defined as PJ+1 i,j|h j=0 αt i,h , Rt = i,0|0 αt where this term represents the likelihood ratio between the attack hypothesis and the clean hypothesis. We then denote the most likely subnetwork by

This implies that previously blocked subnetworks may be restored if the current belief suggests that blocking is no longer beneficial, i.e., when cred (Πt+t′ = πt+t′ , Dt+t′ = Dt+t′ ) ≤ 0 for t′ ∈ N. This restoration mechanism is essential, as belief estimates are subject to error due to false alerts, and incorrect blockings must be reversed to maintain operational continuity. While the greedy decision strategy may not yield globally optimal blocking, it substantially reduces computational complexity, enabling real-time response and ensuring computational tractability in large-scale networks.

î = arg max max Rti,h ,

Deep RL-based approaches [13], [39] have to be retrained whenever the underlying Markov model is modified, and such training is costly. To keep computational cost manageable, we propose to employ MC simulation to estimate the expected cost of blocking and that of eviction. To further reduce the computational cost, we perform a MC simulation only if the maximum likelihood ratio exceeds the MC-trigger threshold θM . That is, for î = arg maxi∈I maxh∈H Rti,h , we trigger a MC simulation if the maximum likelihood ratio satisfies

i∈I

h∈H

and the most likely hypothesis within that subnetwork by ĥ = arg max Rtî,h . h∈H

Accordingly, the belief associated with hypothesis ĥ is used for threat blocking. Let us denote by D≤B ∈ {D′ |D′ ∈ P(I), |D′ | ≤ B} the set of subnetworks that can be blocked under the blocking budget B, by Dt′ ∈ D≤B the set of subnetworks blocked at time t′ ∈ N, and by D1:t−1 = [D1 , . . . , Dt′ . . . , Dt−1 ] the history of blocking decisions up to time t − 1. Once our ·,·|ĥ framework chooses the belief πt , the defender evaluates the expected cost at time t without blocking. The per-time-step cost without blocking is given by: X X ′ i,J|ĥ ′ ·,·|ĥ c(Πt =πt ,D1:t−1 =D1:t−1 ) = cic πt ai,J→i ,1 i∈Ii′ ∈I ↓ (i) d ′

− ri,i . With blocking Dt ∈ D≤B , the cost is X X ′ i,J|ĥ ′ ′ ·,·|ĥ c(Πt =πt ,D1:t =D1:t )= cic πt ai,J→i ,1−ri,i i∈I\Dt i′ ∈I ↓ (i) d

+ cd |Dt \ Dt−1 |. We define the cost reduction achieved by blocking subnetworks Dt as: ·,·|ĥ

cred (Πt = πt

, Dt =Dt ) ·,·|ĥ

= c(Πt = πt

, D1:t−1 = D1:t−1 )

·,·|ĥ − c(Πt = πt , D1:t = D1:t ),

X X ′ i,J|ĥ ′ ′ cic πt ai,J→i ,1 −ri,i −cd |Dt\Dt−1 |. i∈Dti′ ∈I ↓ (i)

C. Predictive eviction

max Rî,h > θM .

(8)

h∈H

If this is the case, then we perform a MC simulation as follows. We sample M particles, each represented as a tuple (hm , im , jm ) where hm is the hypothesis, im is the subnetwork, and jm is the attack state of mth particle. To sample a hypothesis, we compute the posterior distribution over hypotheses. Since the number of hypotheses in subnetwork î may be smaller than H, the computation of the posterior distribution in subnetwork î requires the prior distribution of hypotheses from the perspective of subnetwork î. Recall that q h = P(H = h) is the prior probability of hypothesis h. The prior distribution from the perspective of subnetwork î is then obtained as  P h′  for h = 0, 1 − h′ ∈Hî \{0} q ′ î h P (H = h) = q for h′ ∈ Hî \ {0},   0 otherwise. We can then express the posterior distribution as î î Pî (H = h|Y1:t = y1:t )= P

i i Pî (H = h, Y1:t = y1:t )

î î î h∈Hi P (H = h, Y1:t = y1:t ) î î = y1:t |H = h)Pî (H = h) Pî (Y1:t =P . î î î î h∈Hî P (Y1:t = y1:t |H = h)P (H = h)

The greedy blocking at time t selects an action that maximizes the cost reduction, i.e., D̂t = arg maxD∈D≤B cred (Πt = ·,·|ĥ πt , Dt = D). We make the blocking decision as, ( ·,·|ĥ D̂t , if cred (Πt = πt , Dt = D̂t ) > 0, Dt = ∅, otherwise.

Once hypothesis hm is sampled from the posterior distribution î î = y1:t )), we subsequently sample (i.e., hm ∼ Pî (H = h|Y1:t i,j|hm im and jm from the belief πt . As we perform MC simulation under blocking actions, using a large simulation horizon introduces bias. Once an attack hypothesis is selected, the simulation may result in continuous blocking, which artificially increases the expected blocking

TABLE I T RANSITION PROBABILITIES ASSOCIATED WITH THE COMPOSITE KILL CHAIN CONSTRUCTED FROM F IG . 1 AND F IG . 2. Next subnetwork (i′ )

Current subnetwork (i)

Next state (j ′ )

2 3 4 5

0.10 0.90 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.10 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.25 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.05 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.05 0.97 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.25 0.00 0.00 0.00 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.97 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.03 0.98 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.98 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.98 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.98 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.98 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 1.00

1 i,j→i′′ ,j ′′ (i′′ ,j ′′ )∈S ↓ (i,j) a

Thus, the expected time to reach the next subnetwork from state si,j can be computed as ( t̄i,J , if j = J, i,j T̄ = i,J PJ−1 i,j t̄ + j ′ =j t̄ , otherwise. Then, the cost of the mth MC simulation is computed as cm =

m ,jm T̄ iX

tm =0

γ tm

0.90 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

cost. To avoid this unfair effect, we restrict the simulation horizon to the expected time required for the attacker to reach the next subnetwork from the sampled state sim ,jm . This horizon corresponds to the expected hitting time in Markov chains [40]. Given that each state in our attack graph has two outgoing transitions (i.e., one to itself and one to the next state) the expected time to reach the next subnetwork is estimated by recursively enumerating the expected time to reach the next state. For a given state si,j , we define the set of directly ′ ′ reachable next states as Sd↓ (i, j) = {(i′ , j ′ )|ai,j→i ,j > ′ ′ 0, si,j ̸= si ,j }. The expected time to reach the next state is then computed as t̄i,j = P

Current state (j)

1 2 3 1 2 3 1 2 3 1 2 3 1 2 3

ri,i

′

+ cd |Dtm \ Dtm −1 |.

i∈Dtm i′ ∈I ↓ (i) d

PM 1

We choose eviction if M m=1 cm > cτ , as it indicates that immediate eviction is more cost-efficient than continued operation given the current belief. D. Model Parameterization and Feasibility ADAPTD requires an attack graph to model a networked system and probabilities to describe the attacker’s behaviors The attack graph can be built based on known exploits since finding zero-day vulnerabilities is difficult and developing exploits is time-consuming [41]. For example, one can use MulVAL for creating the attack graph [42], and given the attack graph, the transition and observation probabilities can be obtained using automated penetration testing methodologies [8], [43].

V. N UMERICAL R ESULTS In what follows, we evaluate ADAPTD using simulations on a composite kill chain that integrates Fig. 1 and Fig. 2. A. Evaluation Methodology For the evaluation, we employ a composite kill chain constructed from Fig. 1 and Fig. 2. The attack originates from any subnetwork i ∈ I \ I c excluding the subnetworks containing critical assets. We assume a uniform prior distribution over 1 the choice of the initial subnetwork, given by q i = |I\I c| . Once the initial subnetwork is selected, the probability of initiating malicious access is set to a0,0→i,1 = 0.9, reflecting a random initiation of the malicious access into the system. Table I shows the transition probabilities of the full networked system. Once the initial subnetwork is accessed, the attacker proceeds according to the defined transition probabilities. Table II presents the true and false alert probabilities for each state si,j , with a higher false alert rates assumed for DMZ and corporate subnetworks due to their higher traffic volumes. The discount factor is γ = 0.97. The operational reward for maintaining connectivity between subnetworks i TABLE II T RUE AND FALSE ALERT RATES USED IN THE EVALUATION . False alert rate Subnet

Alert index (k)

Index (i)

1 2 3 4 5

0.6 0.5 0.5 0.4 0.3

0.4 0.3 0.3 0.2 0.1

0.5 0.4 0.4 0.3 0.2

0.4 0.3 0.3 0.2 0.1

0.5 0.4 0.4 0.3 0.2

0.4 0.3 0.3 0.2 0.1

0.5 0.4 0.4 0.3 0.2

0.4 0.3 0.3 0.2 0.1

Subnetwork i ∈ {1, ..., 4} True alert rate Stage

Alert index (k)

index (j)

1 2 3

0.4 0 0

0 0.3 0

0.3 0.2 0

0.1 0.2 0

0 0 0.3

0 0.2 0.3

0 0 0.4

Subnetwork 5 True alert rate Stage

Alert index (k)

index (j)

1 2 3

0.4 0 0

0 0 0

0.3 0 0

0.2 0 0

0 0.6 0

0 0.5 0

0 0 0.7

0 0 0.6

Centralized Partitioned Transformer-based Autoencoder

Diffusion Likelihood Ratio

Centralized Partitioned Transformer-based Autoencoder Likelihood ratio 600

Operating point

600

Mean Total Cost with Attack

625

575 550 525 500 475

500 400 300 200 100 0

20 30 Mean Total Cost without Attack

Fig. 5. Mean total cost with attack vs. mean total cost without attack. ′

and i′ is ri,i = |I ↓2(i)| . The cost of reconfiguring ACL is d cd = 1, and the cost of compromise for the 5 subnetworks is [c1c , c2c , c3c , c4c , c5c ] = [250, 500, 500, 750, 1500]. Each simulation lasts 1, 000 time slots, with each time slot corresponding to a wall clock time of five minutes. For belief update, we use Partitioned as a shorthand for partitioned belief filtering. As baselines for comparison, we consider two belief-update algorithms: Centralized and Diffusion [44]. Using the first approach, the SOC aggregates alerts from all subnetworks and maintains a global belief state. While this method offers comprehensive threat visibility, it incurs substantial communication overhead. We refer to this as Centralized. Using the second baseline approach, each subnetwork independently computes its belief from the full transition matrix based on local alerts, and then shares its complete belief with neighboring subnetworks for aggregation (see Diffusion HMM in Appendix). We refer to this as Diffusion. For predictive eviction, we employ the likelihood-ratio test in (8) and we used grid search to find the threshold θM . As a baseline for comparison, we use a transformer-based autoencoder trained on sequences of benign likelihoods, and trigger a MC simulations when the reconstruction error exceeds a threshold (details are provided in the Appendix). The results shown are the averages of 200 simulation runs. As the main evaluation metric, we use the incurred average total cost, computed akin to (3), but based on the actual progression of the attacker. As additional evaluation metrics we use the eviction delay, the false-eviction rate, and the empirical frequency of single blocking events. Let us denote by t0,0→i,1 the starting time of the attack, i.e., the time when the attacker transitions from a clean state to access state and te = min{t|τt = 1, t ∈ N} the time of evicting the attacker. We then define the eviction delay de = max(te − t0,0→i,1 , 0). We then define the false-eviction rate as the fraction of simulation runs that incorrectly choose eviction when the scenario is not under attack. Finally, we define the empirical frequency of single-blocking events PT 1|Dt |=1 single , f = PTt=1 t=1

1|Dt |>0

which captures how often a single blocking action is taken despite a potentially larger blocking budget.

100 Time (t)

125

150

175

200

Fig. 6. Mean total cost under attack vs. time including 95% confidence intervals, for different belief update schemes and MC triggering methods.

For the computations we used a server with an Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz and 32G memory. All algorithms were implemented in Python with Numpy and Pandas. B. Mean cost with attack vs. mean cost without attack We start with exploring the relationship between the mean total cost with attack and the mean total cost without attack. Fig. 5 shows the mean total cost with attack as a function of the mean total cost without attack. The mean total cost with attack decreases as the mean total cost without attack increases, since a higher total cost without attack corresponds to lower MC-trigger thresholds that result in higher falseeviction rates and faster true evictions. Regardless of the trigger method employed, Partitioned exhibits performance comparable to Centralized belief aggregation. Recalling that Centralized incurs substantial communication overhead [37], [38], we can conclude that Partitioned is well suited for belief update in the considered setting. Since Partitioned substantially outperforms Diffusion, we omit Diffusion from further analysis. The figure also shows that the transformerbased autoencoder achieves results similar to the likelihood ratio trigger. We will compare their performance later in more detail. For the subsequent analysis, we choose an operating point where the mean total cost without attack is 8.07, i.e., the minimum achievable cost for Partitioned. C. Mean total cost with attack vs. time Fig. 6 shows the mean total cost as a function of time, for a mean total cost without attack of 8.07. The figure confirms that likelihood ratio-based triggering of MC simulations outperforms the transformer-based autoencoder, indicating that the likelihood ratio offers an effective, training-free MC triggering mechanism, in contrast to a data-driven approach that requires extensive training. Comparing the results for the different belief update schemes, we observe that the Partitioned scheme achieves a cost comparable to the Centralized scheme, i.e., similar performance at substantially reduced communication overhead, especially towards the end of the time horizon. At the beginning of an episode, however, we observe a sharp cost increase due to the 0.1 transition probability from clean state to initial access, regardless of MC-triggering method and the belief update scheme, yielding an 88% chance of transition by t = 20 (i.e., 1 − (0.9)20 ). Given the randomness and unpredictability of the initial compromise, the sharp increase is

Mean Eviction Delay (de)

1000

Transformer-based Autoencoder

Likelihood ratio

Centralized

Partitioned

E. Impact of Blocking Budget

800 600 400 200 0

With Attack

Without Attack

Fig. 7. Mean eviction delay (de ) with 95% confidence intervals. A comparative analysis between two scenarios (i.e., one with attack and one without attack).

unavoidable. However, after about t = 25, the performance of the different methods becomes clearly different, underscoring the importance of likelihood-ratio triggering for effective MC simulation activation. D. Mean time to evict Fig. 7 shows the mean time to eviction under different belief-update algorithms and triggering methods. We can observe the results are significantly different in scenarios with and without an attack, indicating that our framework responds appropriately to genuine threats while minimizing unnecessary evictions. Without attack, the maximum mean time to eviction reaches 1, 000 time steps, corresponding to the simulation horizon. Comparing belief-update algorithms, Partitioned exhibits a mean time to eviction comparable to Centralized, further confirming the superior scalability of Partitioned without a significant degradation in eviction delay. Notably, Partitioned with likelihood ratio demonstrates longer mean eviction delay without attack compared to the transformer-based autoencoder, indicating its suitability as a robust decision-theoretic engine that effectively minimizes false-eviction cost while maintaining threat responsiveness. To analyze the differences introduced by the two MC triggering methods, we next examine the frequency of MC simulations and the false-eviction rate. Table III shows the frequency of performing MC simulations and the false-eviction rate without attack. The figure shows that Partitioned with likelihood ratio results in substantially less MC simulations compared to Centralized, i.e., lower computational burden, as well in a lower false-eviction rate. The reason why Partitioned results in less MC simulations is that it considers a reduced hypothesis set (|Hî | ≤ H ′ where î = arg maxi′ ∈I maxh∈H Rti ,h ), which in turn reduces the probability of falsely triggering an MC simulation due to noisy alerts. Overall, the proposed likelihood ratio-based approach demonstrates superior performance compared to the transformer-based approach across configurations, confirming the effectiveness of our proposed framework. TABLE III AVERAGE NUMBER OF MC SIMULATIONS WITH 95% CONFIDENCE INTERVALS AND THE CORRESPONDING FALSE - EVICTION RATE . Count of MC simulations

Centralized Partitioned

False-eviction rate

Transformer

Likelihood ratio

Transformer

Likelihood ratio

2.58±1.06 1.39±0.85

0.63±0.22 0.17±0.09

22.0% 9.0%

22.0% 5.5%

In what follows, we examine the impact of the blocking budget. When the belief is accurate, a single blocking action suffices. To introduce more uncertainty into the belief, we increase the false positive rate by 0.3 and rerun the grid search to compare the resulting total costs under attack with those obtained without attack at the same mean total cost. It is important to note that this scenario is extremely noisy, as all false alert rates become comparable to or even exceed the highest true alert rate. Fig. 8 shows the mean total cost with attack as a function of blocking budget. The likelihood ratio consistently achieves lower total costs compared to the transformer-based autoencoder. Considering three aspects (i.e., the lower cost, the inherent explainability of the likelihood ratio, and the transformer’s substantial training requirements), it confirms the likelihood ratio as the proper choice for our framework. Interestingly, Partitioned exhibits a small cost reduction with increasing budget, while Centralized remains largely unchanged; this stems from Partitioned’s reliance on a smaller alert set for belief maintenance, which introduces belief inaccuracies absent in the more comprehensive Centralized scheme. However, given that the increase in total cost from lateral movement ranges from 500 to 1, 500, the additional cost incurred when using Partitioned with the likelihood ratio is small, indicating that only a small fraction of simulations successfully achieve lateral movement. This highlights that our framework remains robust in defending the networked system even in a noisy environment. To further investigate the small difference in total cost, we examine the frequency of single blocking actions. Fig. 9 shows the frequency of single blocking actions as a function of the blocking budget. The figure shows that single blocking occurs in at least 95% of cases across all beliefupdate algorithms and MC-triggering methods, demonstrating the greedy algorithm’s efficiency. Compared to Centralized, Partitioned shows a slightly lower single blocking frequency, reflecting belief uncertainty that prompts more extensive blocking; however, this blocking delays attacker progression and refines subsequent beliefs, ultimately making single blocking sufficient. In contrast, Centralized maintains a higher single blocking frequency due to its more precise beliefs, albeit at greater communication cost. The cost differences observed in Fig. 8 primarily arise from single blocking dominance, underscoring the greedy blocking mechanism’s flexibility in subnetwork selection based on belief states. F. Ablation Study Finally, we investigate the importance of the components in our proposed framework. Table IV shows the mean total cost with attack for different combinations of components. ADAPTD (blocking + termination) outperforms Eviction, indicating the importance of integrating greedy containment and predictive recovery actions. With ADAPTD (blocking + termination), Partitioned shows performance comparable to Centralized, demonstrating the scalability and stability of our proposed decentralized

Centralized Partitioned Transformer-based Autoencoder Likelihood ratio Frequency of Single Blocking (f single)

Centralized Partitioned Transformer-based Autoencoder Likelihood ratio

Mean Total Cost with Attack

850 800 750 700 650 600

1.00 0.99 0.98 0.97 0.96 0.95 0.94 1

550 1

Blocking Budget (B)

Fig. 8. Mean total cost as a function of the blocking budget with 95% confidence intervals.

Fig. 9. Frequency of single blocking events (f single ) as a function of the blocking budget, with 95% confidence intervals.

solution. Without blocking, there is a large performance gap between Centralized and Partitioned, as blocking particularly benefits the decentralized approach: likelihood ratios computed locally in each subnetwork require complete re-accumulation upon attacker movement to new subnetworks. In contrast, Centralized maintains cumulative likelihood ratios irrespective of attacker progression, enabling faster detection but incurring substantial communication overhead. VI. C ONCLUSION

on the network environment and its belief about the attacker´s progression. R EFERENCES

We addressed the challenge of threat response against autonomous APTs by formulating the problem as a POMDP. The key innovations of our framework include: i) the flexible detection of threats, ii) the immediate blocking for containment, and iii) proactive threat defense to prevent the attacker from reaching critical assets. We introduced a method to maintain a belief about the attacker’s progression in a distributed manner, facilitating timely blocking and proactive threat response. Our numerical results demonstrate that the proposed approach significantly reduces the total cost while efficiently managing communication overhead. While the central focus of this work is on the decision-theoretic framework and the distributed belief update rather than engineering a practical detection tool, our evaluation is grounded in numerical simulations conforming to the MITRE ATT&CK matrix. Validating the framework against large-scale operational traces and heterogeneous real-world testbeds remains an important direction for future work. In addition to a real world deployment, there are several promising directions for theoretical and structural future research. First, one could explore malware that evolves by interacting with compromised hosts. Second, the framework could be extended to account for multiple simultaneous actions by the attacker, where each compromised machine can independently target other subnetworks. Third, incorporating strategic attackers that dynamically adapt their strategy based TABLE IV A BLATION STUDY. M EAN COST WITH ATTACK WITH 95% CONFIDENCE INTERVAL . Total cost with Attack Method

Belief algorithm

Eviction

Blocking + Eviction

Transformer-based Autoencoder

Centralized Partitioned

560.60±31.30 628.75±36.26

540.66±30.73 555.69±29.84

Likelihood Ratio

Centralized Partitioned

534.80±32.20 596.40±34.98

523.12±29.74 545.60±28.66

[1] P. Hu, H. Li, H. Fu, D. Cansever, and P. Mohapatra, “Dynamic defense strategy against advanced persistent threat with insiders,” in IEEE Conf. Comput. Commun. (INFOCOM). IEEE, 2015, pp. 747–755. [2] J. P. Farwell and R. Rohozinski, “Stuxnet and the future of cyber war,” Survival, vol. 53, no. 1, pp. 23–40, 2011. [3] C. Cimpanu, “Chilean bank shuts down all branches following ransomware attack,” accessed: 2024-07-05. [Online]. Available: https: //www.zdnet.com/article/chilean-bank-shuts-down-all-branches-followi ng-ransomware-attack/ [4] L. Abrams, “Netwalker ransomware hits Argentinian government, demands $4 million,” accessed: 2024-07-05. [Online]. Available: https://www.bleepingcomputer.com/news/security/netwalker-ransomwar e-hits-argentinian-government-demands-4-million/ [5] K. Collier, “Major hospital system hit with cyberattack, potentially largest in us history,” accessed: 2024-07-05. [Online]. Available: https://www.nbcnews.com/tech/security/cyberattack-hits-major-u-s-hos pital-system-n1241254 [6] T. Tsvetanov and S. Slaria, “The effect of the colonial pipeline shutdown on gasoline prices,” Econ. Lett. (EL), vol. 209, p. 110122, 2021. [7] A. Zimba, “Malware-free intrusion: a novel approach to ransomware infection vectors,” Int. J. Comput. Sci. Inf. Secur. (IJCSIS), vol. 15, no. 2, p. 317, 2017. [8] P. Holgado, V. A. Villagrá, and L. Vazquez, “Real-time multistep attack prediction based on hidden Markov models,” IEEE Trans. Dependable Secur. Comput. (TDSC), vol. 17, no. 1, pp. 134–147, 2017. [9] D. Ourston, S. Matzner, W. Stump, and B. Hopkins, “Applications of hidden Markov models to detecting multi-stage network attacks,” in Proc. Annu. Hawaii Int. Conf. Syst. Sci. (HICSS). IEEE, 2003. [10] S. M. Milajerdi, R. Gjomemo, B. Eshete, R. Sekar, and V. Venkatakrishnan, “HOLMES: real-time apt detection through correlation of suspicious information flows,” in IEEE Symp. Secur. (SP). IEEE, 2019, pp. 1137–1152. [11] E. Miehling, M. Rasouli, and D. Teneketzis, “A POMDP approach to the dynamic defense of large-scale cyber networks,” IEEE Trans. Inf. Forensics Secur. (TIFS), vol. 13, no. 10, pp. 2490–2505, 2018. [12] K. Xiao, C. Zhu, J. Xie, Y. Zhou, X. Zhu, and W. Zhang, “Dynamic defense strategy against stealth malware propagation in cyber-physical systems,” in IEEE Conf. Comput. Commun. (INFOCOM). IEEE, 2018, pp. 1790–1798. [13] K. Hammar and R. Stadler, “Intrusion prevention through optimal stopping,” IEEE Trans. Netw. Serv. Manag. (TNSM), vol. 19, no. 3, pp. 2333–2348, 2022. [14] P. A. Porras and R. A. Kemmerer, “Penetration state transition analysis: A rule-based intrusion detection approach,” in Annu. Comput. Secur. Appl. Conf. (ACSAC). IEEE Computer Society, 1992, pp. 220–221. [15] W. Lee, S. J. Stolfo, and K. W. Mok, “A data mining framework for building intrusion detection models,” in IEEE Symp. Secur. (SP). IEEE, 1999, pp. 120–132. [16] G. Gu, P. A. Porras, V. Yegneswaran, M. W. Fong, and W. Lee, “Bothunter: Detecting malware infection through ids-driven dialog correlation.” in USENIX Security, Aug 2007.

[17] K. Pei, Z. Gu, B. Saltaformaggio, S. Ma, F. Wang, Z. Zhang, L. Si, X. Zhang, and D. Xu, “Hercule: Attack story reconstruction via community discovery on correlated log graph,” in Proc. Annu. Comput. Secur. Appl. Conf. (ACSAC). ACM, 2016, pp. 583–595. [18] M. N. Hossain, S. M. Milajerdi, J. Wang, B. Eshete, R. Gjomemo, R. Sekar, S. Stoller, and V. Venkatakrishnan, “SLEUTH: Real-time attack scenario reconstruction from COTS audit data,” in USENIX Secur. Symp. (USENIX Security), 2017, pp. 487–504. [19] H. Alrubayyi, G. Goteng, M. Jaber, and J. Kelly, “A novel negative and positive selection algorithm to detect unknown malware in the IoT,” in Proc. IEEE INFOCOM Workshops. IEEE, 2021, pp. 1–6. [20] Y. Kim and G. Dán, “Dynamic alert prioritization for real-time situational awareness: a hidden markov model framework with active learning,” IEEE Trans. Dependable Secur. Comput. (TDSC), 2025. [21] O. P. Kreidl and T. M. Frazier, “Feedback control applied to survivability: a host-based autonomic defense system,” IEEE Trans. Reliab. (T-R), vol. 53, no. 1, pp. 148–166, 2004. [22] S. A. Zonouz, H. Khurana, W. H. Sanders, and T. M. Yardley, “RRE: A game-theoretic intrusion response and recovery engine,” IEEE Trans. Parallel Distrib. Syst. (TPDS), vol. 25, no. 2, pp. 395–406, 2013. [23] S. Iannucci, Q. Chen, and S. Abdelwahed, “High-performance intrusion response planning on many-core architectures,” in Proc. Int. Conf. Comput. Commun. Netw. (ICCCN), 2016, pp. 1–6. [24] S. Iannucci and S. Abdelwahed, “A probabilistic approach to autonomic security management,” in Proc. IEEE Int. Conf. Auton. Comput. (ICAC), 2016, pp. 157–166. [25] E. Miehling, M. Rasouli, and D. Teneketzis, “Optimal defense policies for partially observable spreading processes on bayesian attack graphs,” in Proc. of ACM Workshop on Moving Target Defense (MTD), 2015, pp. 67–76. [26] S. Shen, J. Wu, Y. Shen, X. Wu, J. Dong, T. Wang, and R. Li, “Privacy-aware drl for differential games-assisted malware defense in edge intelligence-enabled social iot,” IEEE Trans. Netw. Serv. Manag. (TNSM), 2026. [27] A. H. Anwar, C. A. Kamhoua, N. O. Leslie, and C. Kiekintveld, “Honeypot allocation for cyber deception under uncertainty,” IEEE Trans. Netw. Serv. Manag. (TNSM), vol. 19, no. 3, pp. 3438–3452, 2022. [28] B. Peng, J. Liu, and J. Zeng, “Propagation dynamics of network system in response to the resource-consuming attack,” IEEE Transactions on Control of Network Systems, 2025. [29] P. Jia, J. Lian, and W. Wang, “A novel cross-layer resilient control approach of data-based switched systems with time delay induced by dos attacks,” IEEE Transactions on Control of Network Systems, 2025. [30] S. K. Singh and P. V. Reddy, “Dynamic network analysis of a target defense differential game with limited observations,” IEEE Transactions on Control of Network Systems, vol. 10, no. 1, pp. 308–320, 2022. [31] S. Rose, O. Borchert, S. Mitchell, and S. Connelly, “Zero trust architecture,” NIST special publication, vol. 800, no. 207, pp. 1–52, 2020. [32] A. Bicaku, M. Tauber, and J. Delsing, “Security standard compliance and continuous verification for industrial internet of things,” Int. J. Distrib. Sens. Networks (IJDSN), vol. 16, no. 6, p. 1550147720922731, 2020. [33] “MITRE ATT&CK®,” accessed: 2024-07-05. [Online]. Available: https://attack.mitre.org/ [34] J. L. Fan, “Constrained coding and soft iterative decoding,” in Proc. IEEE Inf. Theory Workshop. IEEE, 2001, pp. 18–20. [35] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” J. Artif. Intell. Res. (JAIR), vol. 4, pp. 237–285, 1996. [36] Y. Bekkemoen, “Explainable reinforcement learning (XRL): a systematic literature review and taxonomy,” Mach. Learn., vol. 113, no. 1, pp. 355– 441, 2024. [37] N. Ghasemi, S. Dey, and J. S. Baras, “Stochastic average consensus filter for distributed hmm filtering: Almost sure convergence,” IFAC Proc. Vol., vol. 43, no. 19, pp. 335–340, 2010. [38] A. Tamjidi, R. Oftadeh, S. Chakravorty, and D. Shell, “Efficient recursive distributed state estimation of hidden Markov models over unreliable networks,” Auton. Robots, vol. 44, no. 3, pp. 321–338, 2020. [39] M. N. Kurt, O. Ogundijo, C. Li, and X. Wang, “Online cyber-attack detection in smart grid: A reinforcement learning approach,” IEEE Trans. on Smart Grid (TSG), vol. 10, no. 5, pp. 5174–5185, 2018. [40] J. R. Norris, Markov chains. Cambridge university press, 1998, no. 2. [41] L. Ablon and A. Bogart, “Zero days, thousands of nights,” RAND Corporation, Santa Monica, CA, 2017.

[42] X. Ou, S. Govindavajhala, A. W. Appel et al., “MulVAL: A logic-based network security analyzer.” in Proc. of USENIX Secur. Symp. (USENIX Security), vol. 8, 2005, pp. 113–128. [43] H. Holm, “Lore a red team emulation tool,” IEEE Trans. Dependable Secur. Comput. (TDSC), vol. 20, no. 2, pp. 1596–1608, 2022. [44] M. Kayaalp, V. Bordignon, S. Vlaski, and A. H. Sayed, “Hidden markov modeling over graphs,” in Proc. Data Sci. Learn. Workshop (DSLW). IEEE, 2022, pp. 1–6. Yeongwoo Kim received his M.Sc. degree in Electrical Engineering from both KTH Royal Institute of Technology, Sweden, and the Technical University of Berlin, Germany, in 2020. He is currently pursuing a Ph.D. in the Division of Network and Systems Engineering at KTH Royal Institute of Technology in Stockholm, Sweden. His research interests icnludes human-in-the-loop systems, machine learning algorithms, and analytic approaches for cybersecurity.

Quanyan Zhu Quanyan Zhu (Senior Member, IEEE) received the B.Eng. degree (Hons.) in electrical engineering from McGill University in 2006, the M.A.Sc. degree from the University of Toronto in 2008, and the Ph.D. degree from the University of Illinois at Urbana-Champaign (UIUC) in 2013. After stints at Princeton University, he is currently an Associate Professor with the Department of Electrical and Computer Engineering, New York University (NYU), where he is an Affiliated Faculty Member with the Center for Urban Science and Progress (CUSP). He is the coauthor of two recent books published by Springer: Cyber-Security in Critical Infrastructures: A Game-Theoretic Approach (with S. Rass, S. Schauer, and S. König) and A Game- and Decision-Theoretic Approach to Resilient Interdependent Network Analysis and Design (with J. Chen). His current research interests include game theory, machine learning, cyber deception, network optimization and control, smart cities, the Internet of Things, and cyber-physical systems. He was a recipient of many awards, including the NSF CAREER Award, the NYU Goddard Junior Faculty Fellowship, the NSERC Postdoctoral Fellowship (PDF), the NSERC Canada Graduate Scholarship (CGS), and the Mavis Future Faculty Fellowships. He spearheaded and chaired the INFOCOM Workshop on Communications and Control on Smart Energy Systems (CCSES), the Midwest Workshop on Control and Game Theory (WCGT), and the ICRA workshop on Security and Privacy of Robotics. He served as the General Chair or the TPC Chair for the Seventh and the 11th Conference on Decision and Game Theory for Security (GameSec) in 2016 and 2020, the Ninth International Conference on NETwork Games, COntrol and OPtimization (NETGCOOP) in 2018, the Fifth International Conference on Artificial Intelligence and Security (ICAIS 2019) in 2019, and the 2020 IEEE Workshop on Information Forensics and Security (WIFS). He also spearheaded the IEEE Control System Society (CSS) Technical Committee on Security, Privacy, and Resilience, in 2020. György Dán (Senior Member, IEEE) received the M.Sc. degree in computer engineering from the Budapest University of Technology and Economics, Hungary, in 1999, the M.Sc. degree in business administration from the Corvinus University of Budapest, Hungary, in 2003, and the Ph.D. degree in telecommunications from KTH in 2006. He was a Consultant in the field of access networks, streaming media, and videoconferencing from 1999 to 2001. He was a Visiting Researcher at the Swedish Institute of Computer Science in 2008, a Fulbright Research Scholar at the University of Illinois at Urbana–Champaign from 2012 to 2013, and an Invited Professor at EPFL in from 2014 to 2015. He is a Professor with the KTH Royal Institute of Technology, Stockholm, Sweden. His research interests include the design and analysis of content management and computing systems, game theoretical models of networked systems, and cyber-physical system security and resilience. He was area editor of Computer Communications from 2014 to 2021 and of IEEE Trans. on Mobile Computing 2019-2023.

Hypotheses for Diffusion HMM Clean 𝑠 𝟎,𝟎|𝒊,𝟎

Attack (ℎ = 1):

Clean 𝑠 𝟏,𝟎|𝒊,𝟏

Subnetwork 1

Subnetwork 4

Subnetwork 5

Subnetwork 3

…

Clean 𝑠 𝟑,𝟎|𝒊,𝟑

Attack (ℎ = 4):

Clean 𝑠 𝟒,𝟎|𝒊,𝟒

Upper bound

10 3

Subnetwork 2

… Attack (ℎ = 3):

|h t, c)

100

Subnetwork 3

Subnetwork 4

Subnetwork 5

Subnetwork 4

Subnetwork 5

KL divergence

Clean (ℎ = 0):

DKL( t|h

10 6 10 9 10 12 10 15 10 18

Fig. 10. Hypotheses in subnetwork i for the diffusion HMM. Each oval and rectangle represents a state and a subnetwork, respectively. Each subnetwork contains three states: access, reconnaissance, and lateral movement. Attack hypothesis h corresponds to an attacker initiating access from subnetwork h.

A PPENDIX : B ELIEF ESTIMATION BASED ON DIFFUSION HMM We adopt the distributed belief update approach from [44]. We modify [44] by allowing a) that each subnetwork maintains up to |I \ I c | + 1 hypotheses, while the diffusion HMM uses a single hypothesis, and b) that each subnetwork maintains likelihoods while the diffusion HMM maintains belief. Fig. 10 shows |I \ I c | + 1 hypotheses considered in subnetwork i to model the random initial compromise of an attack, along with a clean hypothesis representing a fully clean network. Let us denote by H = {0} ∪ I \ I c the set ′ ′ i′ ,j ′ |i,h of hypotheses, where h ∈ H, by st the state si ,j in ′ the kill chain of subnetwork i′ under hypothesis h, by i ,j ′ |i,h i ,j ′ |i,h αt,d the likelihood of state st maintained by the i′ ,j ′ |i,h

diffusion HMM, and πt,d the corresponding belief. The clean hypothesis h = 0 consists of a single benign state {s0,0|i,0 }, and the attack hypothesis h ∈ Hi consists of the set of states {s0,0|i,0 } ∪i∈I h S i . An attack hypothesis h > 0 corresponds to an initial compromise ′originating from 0,0→h,1|i,h 0,0→i ,j|i,h subnetwork h (at > 0, while at = 0 for all ′ i ∈ I \ {h} and j ∈ J ). Each subnetwork updates its belief using transition probabilities:  0,0|i,h 0,0→0,0|i,h ,  αt−1,d a    for i′ = 0, j ′ = 0,    ′ α0,0|i,h a0,0→i ,1|i,h + αi′ ,1|i,h ai′ ,1→i′ ,1 , i′ ,j ′ |i,h t−1,d t−1,d άt,d = ′  for i = h, j ′ = 1,    P P i′′ ,j ′′ |h i′′ ,j ′′ →i′ ,j ′   , ′′ h  j ′′ ∈J i′′ αt−1,d at   i ∈I otherwise, i′ ,j ′ |i,h

where αt−1,d ′

′

100

150 Time (t)

200

250

300

Fig. 11. Average KL divergence between central and partitioned belief filtering with 95% confidence intervals. The simulation is conducted without blocking or termination.

To enable consensus across subnetworks, we introduce a symmetric likelihood-sharing matrix B satisfying B · 1 = 1 ′ and BT = B. Let bi,i denote the (i, i′ ) entry of the beliefsharing matrix B, representing the weight that subnetwork i assigns to the local likelihoods of subnetwork i′ . The updated likelihood is computed as:  ′

′

i ,j |i,h

αt,d

= exp 

 ′ ′ ′′ i ,j |i ,h  bi,i ln α̂t,d . ′

i′′ ∈I h

We normalize the aggregated likelihood to obtain the belief distribution as follows: i′ ,j ′ |i,h

i′ ,j ′ |i,h πt,d =

αt,d 0,0|i,h

αt,d

i′ ∈I h

i′ ,j ′ |i,h j ′ ∈J αt,d

resulting belief distribution is then used to determine which subnetworks to block (see Section IV-B). A PPENDIX : KL D IVERGENCE B OUND B ETWEEN C ENTRALIZED AND PARTITIONED BELIEF FILTERING B ELIEFS We evaluate the discrepancy between the centralized belief ·,·|h ·,·|h πt and the partitioned belief filtering belief πt,p using the KL divergence. i,j|h

·,·|h ·,·|h DKL (πt ∥ πt,p ) =

X X

i,j|h πt log

i∈I h j∈J i

′

πt

i,j|h

. (9)

πt,p

is the likelihood of state si ,j |i,h at time

i ,j |i,h

t − 1, and άt,d captures the transition-evolved likelihood. Next, each subnetwork incorporates locally observed alerts to compute the state likelihood: ′ ′ κ i′ ,j ′ |i,h i′ ,j ′ |i,h α̂t,d = P Yti = yti |St = si ,j άt,d , where κ ∈ N.

Plugging (5) and (6) into (7), we can rewrite it as: i,j|h

πt,p

i,j|h

h = ηt,p P(YtI = ytI |St = si,j )άt,p

i′ ∈I h Y i′′ ,J+1|h i′′′ ,0|h · άt,p άt,p . i′′ ∈I h ∩I ↑ (i) i′′′ ∈I h \I ↑ (i)

′

i ,h ηt,p

(10)

h where ηt,p is expressed as

sion, leading to inequality (b) as follows: ·,·|h

DKL (πt

·,·|h

∥ πt,p ) 

1 h ηt,p = i,0|h P(YtI = ytI |St = si,0 )άt,p P

i,j|h I I i,j j∈J P(Yt = yt |St = s )άt,p

+ i∈I h Q i′′ ,J+1|h i′ ,h Q · i′ ∈I h ηt,p i′′ ∈I h ∩I ↑ (i) άt,p Q i′′′ ,0|h · i′′′ ∈I h \I ↑ (i) άt,p

i′ ,h ηt,p

where is the normalization factor used in (6). Plugging (10) into (9), we derive:

This indicates that the upper bound of the KL divergence at time t depends solely on the belief at time t − 1, independent of the alert observations. By applying (2) and (4), we can reformulate the above expression as follows:

·,·|h

DKL (πt

·,·|h ·,·|h DKL (πt ∥ πt,p )





i′′ ∈I h ∩I ↑ (i)

·,·|h

∥ πt,p )

X X

≤ log

i,j|h πt

′ ′ i′ ,j ′ |h ,j →i,j · ait−1 i′ ∈I,j ′ ∈J πt−1

i′′′ ,0|h άt,p

X i,j ′ |h i,j ′ →i,j|h ϕt−1,p · at−1

βt−1 =

j ′ ∈J i|h

Y i′′ ,J|h i′′ ,J→i′′ ,J+1|h i′′ ,J+1|h ϕt−1,p · at−1 + ϕt−1,p

i′′ ∈I h ∩I ↑ (i)

Y i′′′ ,0|h i′′′ ,0→i′′′ ,1|h ϕt−1,p · (1 − at−1 ) .

i′′′ ∈I h \I ↑ (i)

The upperbound of right-hand side of (11) can be expressed as, ·,·|h

DKL (πt ≤ log

·,·|h

∥ πt,p )

X X

i,j|h πt

′ ′ i′ ,j ′ |h ,j →i,j · ait−1 i′ ∈I,j ′ ∈J πt−1

We observe that the last two terms are non-positive, since the arguments of the logarithm are bounded by one. Consequently, we obtain an upper bound by omitting these terms, which yields inequality (a). Next, we apply the arithmetic–geometric mean inequality to further relax the expres-

≤ log

max

βt−1

i′ ,j ′ i′ ,j ′ →i,j i′ ∈I,j ′ ∈J πt−1 · at−1

i′′′ ∈I h \I ↑ (i)

(11) .

where

i∈I h j∈J i i′′ ,J+1|h άt,p

βt−1

i∈I h j∈J i

i∈I h j∈J



βt−1

! .

Recalling assumptions 1 and 2, we proceed by maximizing the numerator and minimizing the denominator. The maximum value of the numerator is qmax . We obi,0→i,1|h serve that: i) The minimum of 1 − at−1 is given by P ′ min 1 − i′ ∈I h ∩I ↑ (i) wi ,i|h · 1 · qmax = 1 − qmax since the d weights sum to one. ii) Consequently, we define qmin = 1−qmax based on the assumption that each state transitions either to itself or to the next state. Therefore, the transition probability

i,j ′ →i,j|h

satisfies at−1 ∈ [qmin , qmax ]. Using these bounds, we now the validation performance fails to improve for more than five rewrite the denominator as follows: consecutive evaluations. This entire procedure from generating X Y Y new episodes to training the model is repeated ten times to βt−1 = ϵ2 qmin (ϵ · qmin + ϵ) (ϵ · (1 − qmax )) , obtain a robust model. ′′ h ↑ ′′′ h ↑ j ′ ∈J i|h

i ∈I ∩I (i)

ϵ2 qmin

(ϵqmin )) ,

i′′′ ∈I h \I ↑ (i)

(ϵqmin )

(ϵqmin )) ,

i′′ ∈I h ∩I ↑ (i) i′′′ ∈I h \I ↑ (i)

j ′ ∈J i|h

∈I \I (i)

(ϵ · qmin + ϵ)

i′′ ∈I h ∩I ↑ (i)

j ′ ∈J i|h

≥

|I h |

ϵ|I |+1 qmin .

j ′ ∈J i|h

Therefore, the upperbound of the KL divergence is ·,·|h ·,·|h DKL (πt ∥ πt,p ) ≤ log

qmax P

j ′ ∈J i|h ϵ

|I h |+1 q |I | min

Let ϵ = 0.001. We conduct 100 simulations without applying blocking actions to evaluate the average KL divergence between the central belief and the partitioned belief filtering belief. Fig. 11 shows the KL divergence as a function of time. This figure shows that the mean KL divergence remains within the theoretical upper bounds, demonstrating that the proposed belief update mechanism exhibits stable performance and converges toward the central belief by time t = 300. A PPENDIX : T RANSFORMER - BASED AUTOENCODER The transformer-based autoencoder architecture consists of an embedding layer, a stack of transformer encoder layers, and a final expansion layer. Let nlikelihood denote the input likelihood dimensionality and nembed the embedding size. The embedding layer applies a linear transformation to compress nlikelihood into nembed , incorporating a causal mask to ensure reconstructions at time t exclude future inputs t′ > t, and a positional encoder to preserve temporal order. The transformer employs 4 encoder layers with 8 multi-head attention heads, 0.1 dropout, and feed-forward dimension 2nembed to capture temporal correlations. The expansion layer linearly maps back to nlikelihood . Input dimensionality varies by belief mode: nlikelihood = 85 for Centralized and 85·I for Partitioned. Grid search over nembed ∈ {8, 16, 32, 64} identifies nembed = 16 as optimal. The input to the model consists of the likelihoods of all hypotheses and subnetworks. Because these likelihoods are computed by repeatedly multiplying values smaller than 1 at each time step, they naturally decay toward zero over the course of a simulation. To prevent this collapse and avoid trivial zero-valued predictions near the end of an episode, the likelihoods are normalized at every time step. The model is trained to reconstruct these normalized likelihoods, using KL divergence as the reconstruction loss to reflect the probabilistic structure of the inputs. We simulate 100 episodes without attack and divide them into training and validation sets using a 90/10 split. The model is trained with early stopping, which terminates training when

ADAPTD: Adaptive Detection and Proactive Threat Defense for Autonomous APT attacks

Analyze & explore related research in Conceptio

Related documents