arXiv CSopen access

Dynamic Cyber Ranges

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

cryptography cybersecurity privacy security

cryptography, security, privacy, cybersecurity

Dynamic Cyber Ranges Vı́ctor Mayoral-Vilches1,† , Marı́a Sanz-Gómez1,† , Francesco Balassone1,2,† , Maite Del Mundo De Torres1 , George Nicolaou3 , Samuel Rodriguez Borines3 , Almerindo Graziano3 , Paul Zabalegui1 and Endika Gil-Uriarte1 1 Alias Robotics, Vitoria-Gasteiz, Álava, Spain

arXiv:2604.24184v1 [cs.CR] 27 Apr 2026

2 University of Naples Federico II, Naples, Italy 3 CYBER RANGES, Limassol, Cyprus

As LLM-driven agents advance in cybersecurity, Jeopardy CTF benchmarks are approaching saturation and cyber ranges, the natural next evaluation frontier, offer diminishing resistance under their current static design. We validate this observation by deploying an LLM-driven Advanced Persistent Threat (APT) agent across three tiers of increasingly realistic infrastructure (PRO Labs, MHBench, military-grade CYBER RANGES). To counteract this trend, we propose Dynamic Cyber Ranges: cyber range environments augmented with LLM-driven Defender agents that harden infrastructure, monitor for intrusions, and respond in real time. Across evaluated scenarios, Defender agents reduce attacker success to 0–55%, achieving complete prevention on multiple configurations. Since attacker and defender agents draw from the same underlying model capabilities, Dynamic Cyber Ranges preserve evaluation headroom as models improve. Notably, a smaller, specialized on-premise model (alias2-mini) matched the frontier model’s defensive outcomes on multiple scenarios under identical, untuned prompts, and detected the attacker 10× faster on a complex enterprise scenario, suggesting that privacy-preserving on-premise models can serve as competent defenders against frontier-class attackers. The experiments further surface emergent agent behaviors, including scope expansion and prompt exfiltration, with implications for AI benchmark integrity and agentic system design.

Introduction

CTFs saturated (>80%)

Cyber Ranges

Dynamic Cyber Ranges

adversarial Advanced Persistent Threats (APTs) represent the most equilibrium consequential class of cyber adversaries, conducting multi41–100% stage intrusion campaigns that combine reconnaissance, exploitation, lateral movement, and data exfiltration Advancing Cybersecurity AI research −→ across enterprise networks [1, 2]. Defending against APTs requires testing security postures under realistic adversarial pressure, a practice formalized as adversary Cyber Ranges Dynamic Cyber Ranges simulation [3]. Cyber ranges provide the controlled environments where such simulations take place, replicating + defender enterprise topologies, Active Directory domains, and monitoring stacks. However, conventional cyber ranges AI APT AI Defender AI APT are static: vulnerable machines sit idle, services run hosts compromised (100%) attacker success reduced (0–55%) unpatched, and the environment state changes only in Same infrastructure → different outcomes response to the participant’s actions. No defender adapts, no attacker persists, and the exercise reduces to a puzzle Figure 1: From static to dynamic cyber ranges. The upper row with a fixed solution. presents three progression stages: in CTFs (left), multiple model Recent advances in cybersecurity AI have produced families converge toward a saturation ceiling; in Cyber Ranges agents capable of independent multi-step offensive oper- (center), attacker success rates span 41–100% across scenarios ations [4, 5, 6, 7]. On jeopardy-style CTF benchmarks, but remain bounded by fixed challenge sets; in Dynamic Cyber solve rates have increased from 6% to over 75% within Ranges (right), attacker and defender co-evolve toward adversarial twenty months [8], raising questions about the long-term equilibrium. The lower row contrasts the same infrastructure under both conditions: in a Cyber Range (left), the APT agent discriminative power of fixed challenge sets [9]. Mayoral- compromises all hosts (100%); in a Dynamic Cyber Range (right), Vilches et al. [10] demonstrated state-of-the-art models a Defender agent reduces attacker success to 0–55%. The already saturate (>80%) reference CTF benchmarks like probabilistic nature of LLM-driven agents ensures that each Cybench [11], Wei et al. [12] demonstrated that adversaries execution produces different adversarial dynamics. can improve agent capability by over 40% within a fixed compute budget, underscoring that static audits We test this hypothesis by deploying an LLM-driven underestimate real-world risk. These results suggest that APT agent across three tiers of increasingly realistic cyber AI agents are ready to move beyond CTFs, and that range infrastructure. On Hack The Box PRO Labs [13], the environments used to evaluate them must evolve the agent fully solved one lab (5/5 flags in 77 minutes) and accordingly. captured 52% of flags on a 15-machine enterprise network. † These authors contributed equally. Corresponding author: On MHBench [14], an open-source OpenStack-based cyber [email protected] range platform, it captured 67 of 164 flags (41%) across

eight evaluated scenarios, solving all small-to-medium These results indicate that information asymmetry alone configurations. is insufficient to overcome the capability gap when the To further validate these results, we partner with scenario demands active defensive judgment, though CYBER RANGES [15], the official cyber range provider the MHBench parity suggests that smaller, on-premiseof the UN’s International Telecommunication Union capable models can serve as viable defenders in structured (ITU) for the delivery of national, regional, and global environments. cyber drills. CYBER RANGES delivers simulation-based Our contributions are: cybersecurity exercises for military, government, and 1. We demonstrate that LLM-driven APT agents enterprise clients. We evaluated on professional- and can conduct end-to-end intrusion campaigns across military-grade cyber range exercises with 20+ hosts and professional-grade cyber ranges, from CTF-adjacent multi-segment topologies, where our APT agent achieved labs through enterprise networks to military-grade full domain compromise of all evaluated scenarios. These exercises, without predetermined attack scripts. results confirm that static cyber ranges, regardless of complexity, offer diminishing resistance to frontier AI 2. We formalize the concept of Dynamic Cyber Ranges agents, with attacker success rates spanning 41–100% and show that the introduction of LLM-driven across evaluated scenarios (Figure 1, upper center). Defender agents reduces the success rates of APT To restore adversarial pressure, we introduce the conattackers to 0%–55% across all evaluated scenarcept of dynamic cyber ranges: environments augmented ios, achieving complete attack prevention in two with LLM-driven Defender agents that monitor, harden, MHBench configurations and in the military-grade and respond to intrusions while the attacker operates. By intelligence scenario. comparing attacker performance with and without active defenders across the same scenarios, we isolate the effect 3. We evaluate three defensive deployment strategies of adversarial dynamism. On MHBench, the APT agent’s (chokepoint, per-machine, hostmanager) and report capture flags rate dropped from 100% to 0% on both a their cost-effectiveness trade-offs at two network 6-host and a 30-host scenario when the most effective scales. defensive strategy was deployed. On CYBER RANGES, 4. We document emergent agent behaviors, including the APT agent went from conquering all 16 hosts to scope expansion, prompt exfiltration, and writeup zero on a military-grade intelligence scenario, and on an retrieval, that have implications for AI benchmark enterprise scenario its progression slowed by a factor of integrity and agentic system design. three while the number of compromised hosts decreased by 45%. Beyond these measured effects, dynamic cyber The remainder of the paper is organized as follows. ranges present a structural property absent in static Section 2 surveys related work across CTF benchmarks, evaluation environments: both attacker and defender cyber ranges, RL-based agents, and LLM-driven cyberagents draw from the same AI capability improvements. security agents. Section 3 presents the methodology, As models advance, static benchmarks and static ranges including the APT and Defender agent configurations, saturate because the challenge is fixed (Figure 1, upper three defensive deployment strategies, and measurement row), whereas in dynamic ranges, capability gains on the criteria. Section 4 describes the experimental setup across attack side are matched by equivalent gains on the defense the three infrastructure tiers. Section 5 reports results side, reducing attacker success to 0–55% (Figure 1, lower on Hack The Box PRO Labs, MHBench, and CYBER row) and preserving evaluation headroom without manual RANGES, comparing static and dynamic conditions. Secscenario redesign. tion 6 analyzes emergent agent behaviors and limitations. We further evaluate alias2-mini, a smaller model de- Section 7 concludes. signed for on-premise deployment where data sovereignty and privacy constraints preclude the use of cloud-hosted frontier models. As an attacker, alias2-mini exhibits 2 Related Work limited offensive capability, capturing 0% of flags on Prior work on AI-driven cybersecurity spans three pillars: MHBench and 1 of 27 on Dante. However, as a defender benchmarks that measure agent capability, cyber ranges on MHBench, it matched Opus 4.6 in flag denial rates that provide evaluation environments, and agents that across both evaluated strategies while completing initial operate within them. Recent surveys confirm that these hardening in 3–5 minutes compared to 4–15 minutes pillars have developed largely in isolation [16, 17, 18, 19]. for the frontier model. On CYBER RANGES, results We review each in turn and position the contributions of were scenario-dependent: alias2-mini matched Opus this paper. 4.6 on the intelligence scenario (Scenario B), where preCTF benchmarks and evaluation saturation. existing infrastructure hardening sustained the defense, CTF-style benchmarks have driven measurable progress, but on the enterprise scenario (Scenario A), despite including Cybench [11], NYU CTFBench [20], and Cyberdetecting the attacker 10× faster and performing malware Gym [21]. However, CAIBench [8] documents that solve remediation the frontier model omitted, critical failures rates on jeopardy-style CTFs increased from 6% to over (absent credential rotation, self-lockout) resulted in 11 of 75% within twenty months, revealing rapid saturation 13 hosts compromised, identical to undefended conditions. of fixed challenge sets and a knowledge-capability gap:

LLMs achieve approximately 70% on knowledge metrics Box PRO Labs [13] and MHBench [14] provide increasingly yet drop to 20–40% on multi-step adversarial tasks. realistic multi-machine environments, however, in all cases Anthropic’s Claude Opus 4.6 system card reports 93% no defender adapts and no attacker persists beyond the pass@1 on Cybench [22], further confirming that fixed participant’s session. CTF benchmarks are approaching ceiling performance RL-based attack and defense agents. Reinforcefor frontier models. This pattern has motivated several ment learning has been the dominant approach for training responses: CTFusion [23] streams evaluation through red and blue agents. CybORG [37, 38] provides a gym for live competitions, Honarvar et al. [24] generate challenge attacker and defender agents in simulation and emulation. families through semantics-preserving transformations, CyGIL [39] aligns its action space with the MITRE and ZeroDayBench [25] tests agents on novel vulnera- ATT&CK framework. CyberBattleSim [40] was extended bilities ported from production CVEs. On the defensive to support blue agents trained jointly with red agents, side, CTIBench [26] and CyberSOCEval [27] show that yielding defenders that better withstand sophisticated reasoning models offer smaller gains in cybersecurity adversaries through co-evolution. Cyberwheel [41] offers than in mathematics or coding. Balassone et al. [28] a high-fidelity training environment with configurable moved beyond jeopardy-style CTFs by deploying LLM- reward, observation, and action spaces. Shashkov et driven attackers and defenders simultaneously in attack- al. [42] compared deep RL, evolutionary strategies, and and-defense CTFs, finding no statistically significant Monte Carlo Tree Search for adversarial co-training, difference between offensive and defensive agent per- finding that combined DRL/ES approaches produce the formance. However, attack-and-defense CTFs remain most capable attackers. Farooq and Kunz [43] combined constrained environments with limited infrastructure RL with supervised learning to build a generic blue complexity, motivating evaluation on cyber ranges. agent across network topologies. Multi-agent actor-critic Cyber ranges as evaluation platforms. Cyber methods [44, 45] demonstrate that collaborating RL ranges provide controlled environments beyond CTFs. agents can more effectively detect and mitigate threats The DARPA Cyber Grand Challenge [29] ran Cyber through learned communication protocols. Closer to live Reasoning Systems on DECREE, a purpose-built air- execution, Yamin and Katt [46] introduced an agent-based gapped network where automated agents scanned hosts, system that performs attack and defense actions during deployed patches, and competed head-to-head, enabling exercises using formal execution plans, and Santorsola exploit generation and defense at machine speed beyond et al. [47] used RL agents to simulate both normal and single binaries. Eckroth et al. [30] proposed procedural malicious user activities. A systematic review by Vyas topology generation. Oak Ridge National Laboratory et al. [19] confirms that these environments are necessary assembled a range to evaluate AI/ML security tools at preconditions for real-world deployment, however, it flags scale [31]. The paradigm has expanded toward digital unresolved gaps in continuous learning, explainability, and twins for cybersecurity education [32] and AI-enhanced realism. These works share two fundamental limitations: ranges for cyber-physical systems [33]. On scenario RL agents operate within abstract, discrete action spaces orchestration, Hannay et al. [34] proposed machine that do not transfer to production infrastructure, and reasoning for exercise management, Lupinacci et al. [35] the environments are simulations with tens of nodes at used Agentic RAG to generate range configurations from most, not infrastructure with Active Directory domains, natural-language descriptions, and Rizos et al. [36] reduced monitoring stacks, and multi-segment topologies. scenario development time with an LLM-based assistant. LLM-driven cybersecurity agents. LLM-driven These works focus on building and orchestrating ranges, agents have progressed from assistive tools like Pennot on populating them with persistent agents. Hack The testGPT [6] to independent multi-step operators. On

Cyber Range

ResultS

∆(S, D)

Dynamic Cyber Range

ResultD

Figure 2: Methodology overview. In the static condition, the APT agent operates alone against the cyber range, where no defensive agents are present and the environment state changes only in response to the attacker’s actions. In the dynamic condition, an LLM-driven Defender agent is introduced into the same range, actively monitoring, hardening, and responding to intrusions while the APT agent operates. Comparing the two results isolates the effect of adversarial dynamism on attacker success.

the offensive side, CAI [4, 5] demonstrated end-toend intrusion campaigns, D-CIPHER [48] introduced a Planner-Executor framework, Zhu et al. [49] showed that hierarchical LLM teams can exploit zero-day vulnerabilities, and ARTEMIS [7] deployed multi-agent penetration testing on an 8,000-host university network, outperforming 9 of 10 human participants. Further architectures include VulnBot [50], PentestAgent [51] with RAG integration, and PENTEST-AI [52] grounded in MITRE ATT&CK. Tool-augmented agents such as EnIGMA [53] integrate interactive debugging, while knowledge-augmented systems such as CRAKEN [54] and HackSynth [55] leverage writeup retrieval. CurriculumPT [56] applies curriculum learning for progressive skill acquisition. On the defensive side, BlueCodeAgent [57] automates blue team operations, CyberSleuth [58] performs forensics from packet traces, and Dijk et al. [59] explored LLM-based blue team automation in NATO Locked Shields. Balassone et al. [28] found no statistical offensive advantage when defenders also employ AI in attack-and-defense CTFs. However, all existing work deploys agents as evaluation subjects or one-shot tools, not as persistent environmental actors that modify range state while an adversary operates. Across these four areas, three limitations persist. First, benchmarks and cyber ranges remain static, with no concept of persistent agents that alter the environment during an exercise. Second, RL-based agents that do coevolve are confined to abstract simulations with discrete action spaces that do not transfer to real infrastructure. Third, LLM-driven agents operate on real systems but are deployed as evaluation subjects or one-shot tools, never as concurrent attacker-defender pairs on multi-tier infrastructure. This work addresses all three by embedding LLM-driven attacker and defender agents within cyber range infrastructure to create Dynamic Cyber Ranges.

Methodology

conditions to isolate the effect of this adversarial dynamism (Figure 2).

3.1

Evaluation approach Phase 1

PRO Labs

AI APT

Phase 2 MHBench

AI Def.

CYBER RANGES

AI APT

AI Def.

AI APT

validate attacker

static vs. dynamic

static only

8 scenarios, 6–30 hosts

2 scenarios, up to 22 hosts

Figure 3: Experimental design. Phase 1 validates the APT agent on PRO Labs under static conditions (all hosts compromised). Phase 2 compares attacker performance with and without AI Defender agents on MHBench and CYBER RANGES, isolating the effect of adversarial dynamism on the same infrastructure.

The evaluation proceeds in two phases (Figure 3): Phase 1: APT agent validation (PRO Labs). We first deploy the APT agent against Hack The Box PRO Labs [13], multi-machine environments with realistic enterprise topologies classified at Red Team Operator Level I. PRO Labs are opaque (no host-level introspection, no custom agent deployment), which makes them suitable for validating the APT agent’s capability against static, undefended ranges. Two PRO Labs were selected: Dante (27 flags, 15 machines, two subnets) and P.O.O. (5 flags, Active Directory environment). This phase also serves to compare five frontier LLMs as APT agents and to evaluate three agent configurations. Phase 2: Static vs. dynamic comparison (MHBench, CYBER RANGES). We then evaluate the effect of introducing Defender agents into the cyber range. On MHBench [14], an open-source OpenStack-based platform that permits deployment of custom Defender agents alongside the APT agent, enabling attacker-versusdefender experiments. We evaluate defensive strategies of increasing sophistication (chokepoint, per-machine, hostmanager) to measure how Defender agents affect attacker success rates. However, MHBench scenarios are limited in complexity compared to enterprise or military environments. To address this, we validate the Dynamic Cyber Range concept on undisclosed, non-public scenarios provided by CYBER RANGES [15], an industry provider of military and government cyber ranges. These exercises span threat emulation scenarios replicating specific APT campaigns, and military intelligence scenarios modeling state-actor operations. Each scenario is evaluated first without a Defender agent (static condition), then with a Defender agent (dynamic condition), to isolate the effect of adversarial dynamism.

We prototype an LLM-driven APT agent that operates as an adversary simulation agent, following the Cyber Kill Chain and MITRE ATT&CK framework (full system prompt in Appendix A.1). The agent receives an entry point and a set of objectives, then conducts multi-stage intrusion campaigns, including reconnaissance, exploitation, lateral movement, privilege escalation, and data exfiltration, without predetermined attack scripts. This APT agent serves as the evaluation instrument: by deploying it against cyber range environments of increasing complexity, we measure both attacker capability and the effect of introducing LLM-driven Defender agents into the range. In a conventional (static) cyber range, the environment state evolves only in response to the primary participant’s actions. We define a Dynamic Cyber Range as an environment in which additional LLM driven agents, acting as attackers and defenders, that modify the environment state independently, creating adversarial 3.2 APT Agent configurations interactions characteristic of real-world cyber operations. Three agent configurations are evaluated as ablations over The evaluation compares attacker performance under both the attacker architecture (Figure 4): Single, where one

agent operates with direct tool access; Multi-Agent, where a primary agent spawns additional agents that operate in parallel with isolated contexts [60]; and Team, where a primary agent spawns additional agents that share a communication channel for coordinated operations [60]. In the Multi-Agent and Team configurations, the number of spawned agents is not predetermined by the experimenter; the primary agent decides how many teammates to create based on its own assessment of the scenario. Early experiment sessions served a dual purpose: conquering the scenario and comparing configurations. Based on these evaluations, the team configuration was selected for dynamic condition experiments on CYBER RANGES, as it enabled coordinated operations across network segments.

3.3

To measure the effect of defensive agents on attacker success, three deployment strategies of increasing sophistication are evaluated (Figure 5): • S1 (Chokepoint). A single Defender agent is placed on one host that sits on a critical network path. This strategy tests whether a minimal defensive investment can block or slow the attacker at a topological bottleneck. • S2 (Per-machine). One independent Defender agent is deployed on every host in the network. Each agent hardens its own machine without coordination with the others, providing defense in depth at the cost of running many parallel agents [60]. • S3 (Hostmanager). A single Defender agent runs on the Hostmanager with root-level access to all virtual machines. It can SSH into every host, deploy hardening scripts in parallel, and apply a uniform security policy across the entire network.

Single APT Agent

nmap

privesc

exploit

creds

In all strategies, the Defender operates concurrently with the APT agent.

lateral One agent, direct tool access

3.4

Multi-Agent APT Agent

spawn

isolated

tools

tools Parallel agents, isolated contexts

tools

Experimental Setup

This section describes the implementation details of the APT and Defender agents, the infrastructure used across all three evaluation tiers, and the protocol for dynamic condition experiments.

Team APT Agent

4.1 shared channel

tools

Measurement criteria

Success metrics vary by tier. PRO Labs and MHBench use discrete flags placed on target machines, with capture rates reported as fractions of total flags. CYBER RANGES scenarios do not define discrete flags; instead, success is measured by the number of hosts compromised to root (Linux) or SYSTEM (Windows) level. Session durations are uncapped for PRO Labs and CYBER RANGES (with an approximate 8-hour guideline for PRO Labs), and fixed at 60 minutes for MHBench.

tools

Defender configurations strategies

tools

Parallel agents, shared state

Figure 4: Agent configurations evaluated [60]. Single: one APT agent with direct tool access. Multi-Agent: a primary APT agent spawns additional APT agents that operate in parallel with isolated contexts. Team: a primary APT agent spawns additional APT agents that share a communication channel for coordinated operations. All configurations use the same pentesting tool suite.

APT agent implementation

The APT agent is built on the CAI scaffold (v0.7.4+)1 [4], an LLM-driven agent loop that maps observations to tool calls in a observe → plan → act cycle. The agent operates from a single attack platform (Kali GNU/Linux Rolling 2025.4) with root access, connected to the target environment via VPN and SSH. It receives only the IP address or subnet of the entry point and, where applicable, SSH credentials for its own attack platform. No internal topology maps, credentials, or network diagrams are provided. The agent discovers internal infrastructure 1 The + suffix denotes custom modifications to support the multiagent and team configurations evaluated in this work (Figure 4).

S1: Chokepoint Subnet A

Host 1

Host 2

Subnet B APT Agent

Host 3

Host 4

Single defender on critical network path

S2: Per-machine

The tool suite available to the agent includes standard penetration testing utilities present on the attack platform: nmap (network scanning), metasploit (exploitation framework), crackmapexec and impacket (credential tools and lateral movement), linpeas/winpeas (privilege escalation enumeration), and general-purpose shell access for arbitrary command execution. The agent selects tools based on its LLM-driven planning, not from a predefined playbook. A single operator message initiates each session (example operator messages in Appendix A.2); all subsequent operations are performed by the agent. In practice, during long sessions the agent occasionally paused waiting for instructions, in which case the operator sent a single “continue” message to resume execution. No tactical guidance was provided.

Subnet A

4.2 Host 1

Host 2

Subnet B APT Agent

Host 3

Host 4

Independent defender on every host

S3: Hostmanager Hostmanager (root)

Subnet A

Host 1

Defender agent implementation

The Defender agent monitors system logs and network traffic via periodic observation cycles. Upon detecting indicators of compromise (failed authentication attempts, port scanning activity, unauthorized process execution), the Defender selects from a repertoire of responses: adding firewall rules, terminating suspicious processes, rotating credentials, isolating compromised hosts, or deploying decoy services. Response selection is mediated by an LLM, introducing probabilistic variation in defensive behavior. In dynamic experiments, the Defender agent operates from a separate machine with full knowledge of the infrastructure: topology documentation, administrative credentials, and access to the monitoring stack (Wazuh API, Velociraptor console, Elasticsearch indices). This asymmetry reflects realistic conditions where defenders operate with knowledge of their own infrastructure while attackers must discover it. On CYBER RANGES, the Defender agent was started 30 minutes before the APT agent to perform initial reconnaissance and hardening.

Host 2

4.3

APT Agent

Models and scaffold

Subnet B

Host 3

Host 4

Single defender with Hostmanager root access

Defender agent placement

Figure 5: Defender deployment strategies evaluated. S1 (Chokepoint): a single Defender agent is placed on one host at a critical network path. S2 (Per-machine): one independent Defender agent is deployed on every host. S3 (Hostmanager): a single Defender agent operates from the Hostmanager with root-level access to all virtual machines. The Defender icon ( ) indicates Defender agent placement.

through network scanning, DNS enumeration, and analysis of compromised hosts.

Three LLMs were tested driving our APT agents: Claude Opus 4.5, Claude Opus 4.6 and alias2-mini. Claude Opus 4.6, the current state-of-the-art model on the Cybench benchmark [11], was the only model capable of sustained multi-step intrusion campaigns across all tiers, consistent with its benchmark ranking. It was therefore used for all dynamic (attacker vs. defender) experiments. All models were accessed through the CAI scaffold (v0.7.4+).

4.4

Infrastructure details

PRO Labs. Hack The Box offers several PRO Labs of increasing difficulty. We selected the two simplest environments, both classified as Red Team Operator Level I, as an initial validation of APT agent capabilities before proceeding to more controlled platforms. Dante is an enterprise penetration testing environment comprising 15 machines (8 Linux, 5 Windows, 1 FreeBSD

firewall, and 2 domain controllers) distributed across are fixed at 60 minutes. multiple network segments, with 27 flags. The topology CYBER RANGES. To instrument each range for includes web servers, SQL servers, workstations, and the experiment, we provisioned two Kali Linux 2024.2 an administrative subnet that is only reachable through endpoints (2 vCPUs, 8192 MB vRAM, 41 GB disk each) lateral movement and pivoting. The attack surface in a dedicated subnet (private net) attached to the spans web application vulnerabilities, buffer overflows, infrastructure router. One endpoint serves as the attack credential reuse, Active Directory exploitation, and platform, the other as the defender platform. The defender privilege escalation on both Linux and Windows. P.O.O. endpoint is configured with multiple network interfaces (Professional Offensive Operations) is a compact Active connected to the private subnets of the range, allowing it Directory environment with 2 Windows machines (a to run diagnostics and deploy defensive measures across domain controller and a compatibility server) and 5 flags. all network segments. The only modifications applied The goal is to compromise the perimeter host, escalate to these machines were setting a root password (passwd privileges, and ultimately take over the domain. The root) and enabling SSH root login (PermitRootLogin attack chain requires web application enumeration, Active yes). Each agent begins from its own Kali VM, connected Directory attacks, lateral movement, and local privilege to the CYBER RANGES VPN that is attached to the escalation. Hack The Box provides a textual description target range. The agents receive SSH credentials for of each lab and supplies either a subnet (Dante) or a single their respective machines and are prompted to conduct endpoint (P.O.O.) as the entry point. Each exercise is all operations from within them. The defender prompt divided into stages, where each stage corresponds to one includes visibility of all subnets, the endpoints to be flag. The platform assigns a descriptive hint to each stage defended, and administrative credentials for each host (e.g., “I am nuts and bolts about you,” referencing a flag (see Appendix A.2). Attacker and defender run on located in robots.txt). These hints were not provided to isolated machines with separate filesystems to prevent the agent. No modifications to the lab environment are prompt exfiltration (see Section 6). Session durations possible. Session durations were capped at approximately are uncapped, with scenarios designed for 4–24 hour 8 hours per experiment to enable comparison across engagements. configurations. Internet

Router

Ext. firewall

DMZ

Web server(s)

Email server

Int. firewall

DNS server

Switch

Internal network

Domain controller

Workstation 1

Experiments and Results

5.1

Hack The Box PRO Labs

Switch

Three agent configurations were tested (Figure 4), with an approximate time cap of 8 hours per experiment to enable comparison across configurations. Table 1 reports the best result per model and agent configuration across all PRO Lab experiments. P.O.O. was fully solved by Claude Opus 4.5 in single-agent mode (5/5 flags, 77 minutes), indicating that small static ranges provide limited discriminative value for frontier AI agents. Table 1: PRO Labs results. Best flags captured per model and agent configuration. Dante: 27 flags, 15 machines; P.O.O.: 5 flags, 2 machines.

Monitoring (SIEM)

Workstation 2

Database server

Workstation 3

Figure 6: Generic network topology of a cyber range exercise. The environment replicates a production network with a DMZ hosting public-facing services, an external and internal firewall, and a protected internal network containing application servers and workstations. This layered architecture is representative of the scenarios used across all three experimental platforms.

Lab

Model

Config

Flags

Duration

P.O.O.

Opus 4.5

Single

5/5

1h 17m

Dante

alias2-mini Opus 4.5 Opus 4.5 Opus 4.6 Opus 4.6 Opus 4.6

Team Multi-Agent Single Single Multi-Agent Team

1/27 6/27 8/27a 14/27 14/27 19/27b

7h 45m 8h 27m 8h 24m 8h 03m 8h 03m 8h 16m

a Best of 10 runs; 8 terminated at ≤1/27 due to infrastructure failures (VPN disconnections, tool hangs). b Run reached 19/27 but was invalidated: the agent retrieved publicly available writeups instead of solving flags through exploitation. 14 flags verified as legitimate.

MHBench. MHBench [14] runs on OpenStack, provisioning network topologies ranging from 2 to 30 hosts. Three findings emerge from the Dante experiments The APT agent connects via SSH to a designated attack machine within the virtual network. Defender agents are (Figure 7). First, model capability is the dominant deployed on separate virtual machines with administrative factor: Opus 4.6 in single-agent mode (14/27) outperaccess to the monitoring infrastructure. Session durations forms Opus 4.5 at its best configuration (6/27), while

alias2-mini captures only 1 flag. Second, the team configuration [60] did not yield a verified advantage over single-agent mode, as the only team run with higher flag counts (19/27) was invalidated after the agent retrieved publicly available writeups (Section 6). Third, infrastructure fragility on opaque platforms is a persistent obstacle: 8 of 10 Opus 4.5 runs terminated prematurely due to VPN disconnections or scaffold errors. Opus 4.5

alias2-mini

Opus 4.6

Invalidatedb

27 total

19b

Flags Captured

15 14 10

8 6

5.2

MHBench

Experiments proceeded in two phases, all sessions constrained to 60 minutes. First, two LLMs were compared as attackers across all eight MHBench scenarios (Section 4.4) without any defense deployed, establishing baseline attacker performance. Second, the three defensive strategies defined in Section 3.3 were deployed on two scenarios, EquifaxSmall (6 hosts, 3 subnets) and EnterpriseA (30 hosts, 4 subnets), to measure how attacker performance degrades under active defense. These two scenarios were selected because they span different network scales while both producing complete attack chains in the baseline, providing a measurable surface for evaluating the effect of dynamic defenders. Attacker model comparison. Two LLMs were evaluated as APT agents (Figure 8). Claude Opus 4.6 captured 67 of 164 flags (41%), capturing all flags in the four small-to-medium scenarios and partially compromising Star (8/25) and EnterpriseA (20/30). The remaining alias2-mini model captured none across all scenarios, showing its shortcomings for acting as an attacker in scenarios of this complexity.

Opus 4.6: 67/164 (41%)

alias2-mini: 0/163 (0%)

0 2/2

6/6

26/26

8/25

0/30

20/30

0/40

alias2-mini03/26

0/5

0/2

0/6

0/25

0/30

0/40

■

■ En te

rp ris

■

■ En te

bb e um D

rp ris e

■

St ar

iu m

xS m

■

■ ts

n2 H

en Ch ai

ro ev En

The PRO Labs results establish a static baseline. P.O.O. was fully solved within a single session, while Dante was 52% solved by the best verified configuration. However, PRO Labs present structural limitations for reproducible AI evaluation. The platform is opaque: it does not permit host-level introspection, system log collection, or deployment of custom agents, preventing detailed analysis of agent behavior. Infrastructure fragility further undermines reproducibility, as 8 of 10 Opus 4.5 runs on Dante terminated prematurely due to VPN disconnections or tool hangs. Commercial platform availability is also a concern: Hack The Box previously offered Battlegrounds, a real-time attack-and-defense environment used in prior work [28], which was discontinued, rendering all dependent benchmarks non-reproducible. These limitations motivate the transition to open-source and CYBER RANGES infrastructure, where full control over the environment enables systematic, reproducible evaluation.

Figure 7: Dante PRO Lab (27 flags): best verified flags captured per model and agent configuration. Model capability is the dominant factor, with Claude Opus 4.6 reaching 14/27 verified flags regardless of agent configuration. b The red dashed region on the rightmost bar indicates 5 additional flags invalidated after the agent retrieved publicly available writeups instead of solving through exploitation (14 flags verified as legitimate). Higher is better.

■

5/5

l■

Claude Opus 4.6

.6 s4 pu ) O eam (T

ul (M

.6 s4 pu ) O ent Ag ti-

Eq ui

.6 s4 pu ) O ngle i (S

.5 s4 pu ) O ngle i (S

Eq ui

.5 s4 pu ) O ent Ag tiul (M

■

i in m 2- ) as am i l a Te (

MHBench Scenarios, flags captured (60 min)

Figure 8: MHBench model comparison: flags captured per model and scenario. Each cell shows the fraction of flags captured (found/total) with color intensity proportional to the capture rate. Sessions limited to 60 minutes of compute time. Scenario size increases left to right: ■ Small, ■■ Medium, ■■■ Large.

Attacker performance under defense. Since Claude Opus 4.6 was the only model capable of sustained attacks, it served as the attacker in all subsequent experiments. Two defender models were deployed against it: Claude Opus 4.6 and alias2-mini, a model meant to be served on-prem in embedded devices. The three defensive strategies defined in Section 3.3 (S1 Chokepoint, S2 Per-machine, S3 Hostmanager), plus a no-defense baseline, were tested on EquifaxSmall (6 hosts, 3 subnets) and EnterpriseA (30 hosts, 4 subnets). For S1, the chokepoint host must be selected per topology. In EquifaxSmall, the topological chokepoint is WS-0 (the primary webserver). In EnterpriseA, the topology offers several candidates, so three placements were tested: S1-A on webserver WS-0, S1-B on the management database Mng-DB, and S1-C on employee host Emp0. For S2, 6 and 30 independent defender agents were deployed in EquifaxSmall and EnterpriseA, respectively. Network topologies and defensive deployments for both scenarios are shown in Figures 9 and 10. Table 2 reports attacker performance under each configuration. Detailed

Table 2: Defensive strategy comparison for EquifaxSmall (6 hosts, 3 subnets) and EnterpriseA (30 hosts, 4 subnets) with Claude Opus 4.6 and alias2-mini as defenders. Flags obtained: number of target flags the attacker captured; Def. success: percentage of flags the defender prevented from being captured; 1st flag : time in minutes until the attacker captured the first flag; Cost: API cost in USD for the defender agent(s) (prices rounded for values greater than 10). Baseline: no defender deployed. S1: single defender on a chokepoint host. S2 (Per-machine): one independent defender per host (6 agents for EquifaxSmall, 30 for EnterpriseA). S3 (Hostmanager): single agent with Hostmanager root access. All sessions used Claude Opus 4.6 as attacker. Both defender models achieve identical flag denial rates across all strategies, however, costs vary substantially. Detailed timelines are provided in Appendix B. Network topologies and strategy placements are depicted in Figures 9 and 10. Defender: Opus 4.6 Scenario

Strategy

EquifaxSmall

EnterpriseA

∗

Flags obt.

Def. (%)

1st flag (min)

Baseline (no defense) S1 (Chokepoint: WS-0) S2 (Per-machine, 6×) S3 (Hostmanager)

6/6 4/6 0/6 0/6

—† 33 100 100

29 53 —∗ —∗

Baseline (no defense) S1-A (Chokepoint: WS-0) S1-B (Chokepoint: DB-0) S1-C (Chokepoint: Emp-0) S2 (Per-machine, 30×) S3 (Hostmanager)

20/30 18/30 20/30 21/30 0/30 0/30

—† 40 33 30 100 100

22 34 26 23 —∗ —∗

No flag captured by attacker.

†

Defender: alias2-mini Cost ($)

Flags obt.

Def. (%)

1st flag (min)

— 10.39 11 2.81

6/6 4/6 0/6 0/6

—† 33 100 100

29 38 —∗ —∗

Cost ($) — 7.88 51 6.41

— 2.10 2.3 1.85 63 1.31

20/30 18/30 20/30 20/30 0/30 0/30

—† 40 33 33 100 100

22 47 51 32 —∗ —∗

— 11 15 2.65 2.15 2.05

No defender deployed.

attacker–defender timelines and attack/defense chains for 2017-5638) on port 8080 yielded RCE as tomcat on all EquifaxSmall experiments are provided in Appendix B, 10 webservers, privilege escalation via kernel exploits failed organized by strategy (Figures 15, 16, 17). (PwnKit CVE-2021-4034 and OverlayFS CVE-2023-2640, both patched) but sudo brute-force succeeded with the EquifaxSmall (6 hosts). With no defense, the attacker default password ubuntu (the tomcat user belonged to captured all 6 flags in under 30 minutes. Against S1 the admin group with full sudo privileges), and SSH keys (single defender on WS-0), the attacker’s capture rate stored in /home/tomcat/.ssh/ provided lateral movement dropped to 4/6: it pivoted through the undefended WS-1, to all 10 employee hosts where the same sudo credential which offered an equivalent entry point, but lost access applied. The attacker never discovered the netcat listener to two flags. Under S2 (one defender per machine, on port 4444 needed to pivot into the database subnet, 6 agents) and S3 (a single hostmanager), the attacker leaving 10 flags uncaptured. Chokepoint placement had limited impact on attacker captured zero flags across the full 60-minute session, a complete neutralization of the attack chain. The performance. Against S1-A (defender on WS-0), the attacker obtained the same number of flags regardless of attacker captured 18/30 flags: the defender rendered whether Opus 4.6 or alias2-mini was defending (Table 2). WS-0 unreachable (applying an iptables default DROP Figure 11 provides a visual breakdown of defender success policy before inserting allow rules, locking itself out), rate, attacker-captured flags, and cost per strategy for which removed WS-0 and its associated employee from the both defender models. The correlated attacker–defender attack surface but left the remaining 9 webservers fully timelines (Appendix B, Figures 15–17) illustrate the exploitable through the identical Struts2 vulnerability. contrast: under S1, the defender hardens WS-0 within Against S1-B (defender on DB-0), the attacker captured 5 minutes (SSH restrictions, iptables rules, Tomcat shell 20/30, unchanged from baseline: the defender applied removal), however, the attacker bypasses it entirely by UFW rules and SSH hardening on database-0, however, exploiting the identical Struts2 S2-048 vulnerability on the attacker never reached the database subnet, confirming WS-1, then pivots via stolen SSH keys to all four databases that DB-0 sits outside the primary attack path. Against where default credentials (password ubuntu) yield root S1-C (defender on Emp-0), the attacker still captured 20– access and 4 flags. Under S3, the centralized hostmanager 21/30 flags, since defending 1 of 10 equivalent employee hardens all 6 hosts within 3 minutes (SSH lockdown, hosts provides insufficient coverage. Under S2 and S3, iptables DROP policies, SUID removal on all machines defenders applied iptables default DROP policies across simultaneously), leaving the attacker unable to exploit all machines before the attacker could establish persistence, any entry point across the full 60-minute session. Both rendering every server unreachable and reducing the defenders apply qualitatively similar hardening sequences, attacker from 20/30 flags to zero with both defender however, alias2-mini completes full hardening in 3– models. 5 minutes versus 4–15 minutes for Opus. On-premise defender viability. Across both scenarEnterpriseA (30 hosts). At enterprise scale, the ios and all evaluated strategies, alias2-mini achieved dynamics change. Without defenses, the attacker captured identical flag denial rates to Opus 4.6 (Table 2). Under 20 of 30 flags in 22 minutes. The attack chain proceeded S2 and S3, both models reduced the attacker from full through three stages: S2-045 OGNL injection (CVE- compromise to zero captured flags, regardless of network

1 ATTACKER

S2-048 RCE

WS-0

Attacker

SSH (stolen key)

WEBSERVER

CRITICAL COMPANY

WS-1

DB-0

DB-1

DB-2

DB-3

CVE-2017-9791 192.168.202.0/24

192.168.200.0/24

192.168.201.0/24

(a) Network topology and attack chain. The EquifaxSmall scenario spans three subnets: an attacker network, two webservers vulnerable to CVE-2017-9791 1 the attacker exploits S2-048 on port 8080 to gain RCE as tomcat on (Struts2 S2-048 OGNL injection), and four database servers holding PII. Phase ○: 2 stolen SSH keys from /home/tomcat/.ssh/id rsa enable lateral movement to all four databases. both webservers. Phase ○:

Hostmanager (root) DB-0 WS-0

WS-0

DB-1

DB-0..3 WS-1

WS-1

WS-0

DB-0

DB-2

WS-1

DB-1

DB-3

DB-2 DB-3

(b) Strategy 1: a single defender agent is deployed on the primary webserver (WS-0). WS-1 and all databases remain undefended.

(d) Strategy 3: a single defender agent as a Hostmanager with root access deploys hardening scripts to all 6 machines simultaneously.

Figure 9: EquifaxSmall scenario (6 hosts, 3 subnets). (a) Network topology and phase attack chain, (b–d) Three defensive deployment strategies evaluated. Defender icons ( ) indicate LLM-driven defender agent placement.

scale. The hardening sequences were qualitatively similar an enterprise network (Scenario A), and one military (SSH restrictions, iptables DROP policies, SUID removal), intelligence scenario modelling two critical infrastructure however, alias2-mini completed initial hardening in 3– organizations (Scenario B). Success is measured by hosts 5 minutes compared to 4–15 minutes for Opus 4.6. This compromised to root or SYSTEM level (Section 3). For convergence in defensive outcomes between a frontier each scenario, we first deployed the APT agent alone cloud model and a smaller on-premise-capable model has (static condition) to establish that the scenario could be practical implications. Data sovereignty regulations such conquered, then re-deployed it against a Defender agent as the EU’s NIS2 Directive and GDPR, sector-specific (dynamic condition). mandates for defense and critical infrastructure, and Both scenarios are post-compromise environments national cybersecurity frameworks increasingly restrict containing traces of prior adversary activity (pre-existing the transmission of sensitive infrastructure telemetry to malware, persistence mechanisms, indicators of previous external cloud providers. Organizations subject to these intrusions), reflecting the reality that military cyber constraints currently face a gap between the AI capabilities range exercises often simulate environments where an available through cloud APIs and the models deployable initial breach has already occurred. Following common within their own perimeter. The MHBench results indicate convention, when testing with defenders, we provide that this gap may not extend to defensive operations in defender a 30 minute grace period before the attackers are structured environments: when the defensive task consists activated. of hardening known infrastructure, a smaller model can Scenario A: Enterprise Network. A threat match frontier performance. Whether this parity holds emulation exercise modelling a technology company’s under more complex conditions is examined in Section 5.3, corporate network. The infrastructure spans seven where CYBER RANGES experiments reveal scenario- network segments (DMZ, public zone, backend servers, dependent limitations. firewall transit, server/Active Directory zone, workstation zone, and a management network) totaling approximately 20 hosts. The environment includes an Active Directory 5.3 CYBER RANGES domain with 22 user accounts, a mail server stack Two exercises were selected from CYBER RANGES (Postfix, Dovecot, Roundcube), edge and internal Linux (Section 4.4): one threat emulation scenario modelling

S2-048 :8080

ATTACKER

SSH (key)

WEBSERVER

3 EMPLOYEE A

nc :4444

DATABASE 4

SSH

MgmtDB WS-0 Attacker

WS-1

Emp-0

Emp-1

. . . ×10 webservers

. . . ×10 employees

CVE-2017-9791

random assignment

nc :4444

.202.100

DB-0

. . . ×9 databases PII + flags

192.168.202.0/24

192.168.200.0/24

192.168.201.0/24

192.168.203.0/24

1 Struts2 S2-048 RCE on any of 10 webservers via port 8080. Phase ○: 2 (a) Network topology and four-phase attack chain (30 hosts, 4 subnets). Phase ○: 3 netcat connection on port 4444 to the SSH lateral movement to a randomly assigned employee using stolen keys from tomcat/.ssh/. Phase ○: 4 SSH pivot from MgmtDB to all 9 remaining databases containing PII and flags. management database. Phase ○:

Hostmanager (root) A:

Emp

Mgmt

DBs

Emp

Mgmt

DBs

Emp

Mgmt

DBs

Emp

×10

Emp

×10

30 agents total

1 agent, 30 hosts (b) S1: three placement options. A: defend one webserver (blocks 1/10 entry points). B: defend the MgmtDB hub (blocks pivot to all databases). C: defend one employee (blocks 1/10 SSH paths).

(c) S2: one independent defender agent deployed on each of the 30 hosts (10 webservers + 10 employees + 10 databases). Each agent hardens its machine.

(d) S3: a single defender agent as a Hostmanager with root access deploys hardening scripts to all 30 machines simultaneously.

Figure 10: EnterpriseA scenario (30 hosts, 4 subnets). (a) Network topology and four-phase attack chain. (b–d) Three defensive deployment strategies. Defender icons ( ) indicate LLM-driven defender agent placement.

firewalls running Webmin, a load balancer, three Win- persists via Group Policy on the government agency’s dows 10 workstations, and a centralized monitoring stack domain controller, simulating a prior state-actor intrusion comprising Wazuh Manager 4.3.10, Velociraptor 0.6.7, with C2 callbacks to external domains. Figure 13 provides Arkime, and Elasticsearch, with Wazuh agents deployed an abstract representation. on all eight managed endpoints. The attack surface Agent configurations. Both the APT agent and includes default credentials on network appliances and the the Defender agent used Claude Opus 4.6 through monitoring infrastructure, password reuse across services, the CAI scaffold (v0.7.4+) in team-based multi-agent and reversible encryption enabled on the domain controller. configuration [60] (Figure 4). The APT agent received Figure 12 provides an abstract representation. only SSH credentials for its attack platform and DMZScenario B: Dual-Organization Critical Infras- facing IP addresses; the Defender received full topology tructure. A military intelligence scenario modelling two documentation, administrative credentials for all hosts, organizations, a healthcare provider and a government and access to the monitoring stack. The attacker discovagency, connected through separate DMZ segments and ered internal infrastructure through DNS enumeration, protected by independent firewall chains. Each organi- routing table analysis, and firewall NAT rule inspection. zation maintains its own Active Directory forest, mail This information asymmetry and the dynamic condition infrastructure (iRedMail), and monitoring endpoints. The protocol (including the Defender’s 30-minute head start) environment comprises approximately 15 hosts across follow the methodology described in Section 3. six segments including external firewalls, DMZ mail Static condition. In the static condition, both servers (with Roundcube webmail), internal domain scenarios were conquered by the APT agent without controllers, workstations, and centralized monitoring defensive opposition. (Wazuh, Velociraptor, Elasticsearch). The monitoring Scenario A was conquered in 4h 11m (best of two stack provides visibility across both organizations through sessions), with 11 of 13 discovered hosts compromised Wazuh agents installed on all endpoints. The scenario to root or SYSTEM level. The attack chain Table 4) is pre-compromised: a malware dropper (Chicken1.bat) progressed through discovered credentials on the edge

the single most impactful vulnerability in the scenario. Table 3 summarizes all experiment iterations across both scenarios, including the static sessions used for agent 80% configuration evaluation and the dynamic sessions with the Defender agent. 60% Dynamic condition. Having established that both $0 $7.88 $2.65 40% scenarios could be conquered, we re-deployed the APT agent against an active Defender agent following the dy20% 4/6 20/30 $0 20/30 namic condition protocol (Section 3). Table 5 summarizes 0/6 0/30 0/6 0/30 6/6 the results across both conditions. 0% e 1 2 3 The first two dynamic experiments (both using Claude n S S S i sel Ba Opus 4.6 as Defender) produced divergent outcomes, providing a natural comparison of effective versus ineffective (a) Defender: alias2-mini defensive strategies. Two additional dynamic experiments replaced the Defender model with alias2-mini, one per $11 $63 $2.81 $1.31 100% scenario, to assess cross-model generalization. EquifaxSmall Scenario A: Attacker wins. The Defender agent EnterpriseA 80% x/y = captured flags rotated all domain passwords, performed two krbtgt $ = defender cost resets, removed a user from the Domain Admins group, 60% and deployed monitoring scripts and firewall rules. The $2.10 $0 $10.39 40% defender detected 6,192 SSH brute-force attempts and blocked the attacker’s IP via iptables. However, the 18/30 20% 4/6 $0 20/30 Defender failed to change default credentials on the moni0/6 0/30 0/6 0/30 6/6 toring infrastructure: the Wazuh API retained its factory 0% e 2 3 1 credentials and the Velociraptor console retained default n S S S i sel Ba credentials. The attacker accessed the Wazuh API with these unchanged credentials, queried the Elasticsearch (b) Defender: Claude Opus 4.6 indices backing the SIEM, and discovered the Defender’s Figure 11: Defender success rate and attacker-captured flags per own password rotation commands logged as WinRM defensive strategy across EquifaxSmall (6 hosts) and EnterpriseA execution events. The attacker extracted the newly (30 hosts). Bar height indicates defender success (percentage of rotated password for a domain administrator account flags the attacker was prevented from capturing); labels inside from these command logs and used it to perform a or above each bar show the number of flags captured by the DCSync attack, dumping all domain hashes. The timeattacker (x/y); green values above bars report the defender agent cost in USD. (a) alias2-mini as defender, with S1-C (Emp-0) to-compromise tripled from 4h 11m (static) to 12h 09m placement for EnterpriseA. (b) Claude Opus 4.6 as defender, with (dynamic), and hosts compromised decreased from 11 to 6, S1-A (WS-0) placement. Both defender models achieve identical but the attacker ultimately achieved domain compromise. flag denial under S2 and S3, however, costs differ substantially: This result demonstrates a critical operational security S3 provides full denial at a fraction of S2’s cost. All sessions used failure: the Defender secured the credentials it rotated but Claude Opus 4.6 as the attacker model. Higher is better. failed to secure the monitoring infrastructure that recorded those operations, enabling the attacker to recover the new firewall (Webmin) through a configuration file, then passwords from SIEM event data. Figure 12 shows the pivoted to the internal firewall via SSH, a SOCKS proxy correlated timeline (full version in Appendix, Figure 18). Scenario B: Defender wins. The Defender agent chain to reach internal networks, unchanged factory credentials on both the Wazuh SIEM API and Velociraptor used its 30-minute head start to perform network reconEDR, and a DCSync attack dumping all 22 domain naissance across both organizations, discovering and remeaccounts (reversible encryption was enabled). The entire diating the pre-existing malware dropper (Chicken1.bat attack chain exploited no software vulnerabilities; every deployed via GPO on the government agency’s domain pivot relied on default or reused credentials and legitimate controller, with C2 callbacks to external domains). The Defender restored an expired Velociraptor TLS certifiadministrative tools. Scenario B required approximately 48 hours cumu- cate, deployed eight firewall rules on the healthcare lative across five sessions (which also served to evaluate provider’s external firewall and nine rules on the govagent configurations) to achieve full domain compromise ernment agency’s firewall, restricted the www-data shell to of both Active Directory forests. The agent obtained root /usr/sbin/nologin, bound Elasticsearch to localhost on or SYSTEM access on 16 hosts across 12 subnets. The the OpenEMR server, and deployed cron-based monitoring longer time-to-completion reflects the dual-organization scripts (every 3–5 minutes). When the APT agent’s scans topology requiring independent exploitation chains for reached the DMZ mail servers, the Defender detected each AD forest. A critical inflection point occurred when 4,157 Wazuh SIEM alerts within 32 minutes and blocked the agent cracked a shared administrative credential that the attacker’s IP on both external firewalls and all three provided SSH access to six previously inaccessible hosts, DMZ mail servers via host-level iptables rules within an $51

Defender Success (%)

100%

EquifaxSmall EnterpriseA x/y = captured flags $ = defender cost

$2.15

$6.41 $2.05

APT

DMZ

Recon Cred. rotation, 2× krbtgt

DEF DNS

FW rules

Proxy

Brute-force detected IP blocked

Edge FW

DCSync detected Recon

Backend

Brute-force Blocked Mail

Web

SIEM compromised DCSync

APT

Post-exploitation

Int. FW

Conquered Servers

DEF

Workstations

12h 09m (3× static) SIEM

WS01

WS02

10h

12h

Figure 12: Scenario A (Enterprise Network). Left: abstract topology with seven segments, centralized SIEM/EDR (Wazuh, Velociraptor, Elasticsearch). Right: dynamic experiment timeline. The vertical dashed line marks the attacker start, 30 minutes after the Defender. Despite credential rotation, the Defender failed to change monitoring stack defaults, and the attacker extracted rotated passwords from SIEM logs. Outcome: Attacker wins. Hostnames omitted per non-disclosure requirements. APT

Recon, malware removal Healthcare

Government

FW hardening (17 rules)

DEF

Service hardening DMZ

DMZ

Detected (4,157 alerts) IP blocked (5 hosts)

Mail

Web

Continuous monitoring

Mail

Recon FW

Brute-force Blocked Evasion attempts (failed)

APT Internal

Guacamole (failed)

Internal

Degradation DC

Monitor OpenEMR

Monitor

DEF

32 min

16 min

Figure 13: Scenario B (Dual-Organization Critical Infrastructure). Left: abstract topology with two organizations, separate AD forests, DMZ mail servers, firewalls, and monitoring (Wazuh, Velociraptor). Pre-existing malware on government DC. Right: dynamic experiment timeline. The vertical dashed line marks the attacker start, 30 minutes after the Defender. Defender detected attacker in 32 min, full containment in 48 min. Outcome: Defender wins. Hostnames omitted. Table 3: Experiment iterations across both CYBER RANGES scenarios. Static sessions served a dual purpose: conquering the scenario and evaluating agent configurations. The team configuration was selected for the dynamic condition based on its superior performance in static evaluations. Scenario A (Enterprise Network)

B (Military, Critical Infra.)

a b

Condition

Duration

Hosts ROOT

Static (session 1) Static (session 2) Dynamic Dynamic (DEF: alias2-mini)

5h 04m 4h 11m 12h 09m 4h 45m

6/13 11/13 6/13 11/13

Partiala Conquered Conquered (ATK wins) Conquered (ATK wins)

Static (session 1) Static (session 2) Static (session 3) Static (session 4) Dynamic Dynamic (DEF: alias2-mini)

4h 10m 11h 24m 10h 05m 22h 18m 6h 53m 4h 39m

3/15 8/15 3/15 16/15b 0/15 0/15

Partial (FWs only) Partial (1 AD forest) Re-confirmed access Conquered Not conquered (DEF wins) Not conquered (DEF wins)

Workstation hosts experienced BSOD during the exercise, preventing full compromise. 16 hosts includes adversary infrastructure hosts discovered beyond the initial 15-host topology.

Outcome

Scenario

Tactic

Technique

Tool

Target

Result

A (static)

Reconnaissance Initial Access Lateral Movement Lateral Movement Credential Access Execution Execution Credential Access

T1046 Network Service Discovery T1078.001 Default Accounts T1021.004 Remote Services: SSH T1572 Protocol Tunneling T1078.001 Default Accounts T1059.001 PowerShell T1059.004 Unix Shell T1003.003 NTDS

nmap Webmin ssh ssh -D (SOCKS) curl (Wazuh API) Velociraptor Velociraptor impacket-secretsdump

All subnets Edge Firewall Internal Firewall Internal networks SIEM/EDR All Windows hosts All Linux hosts Domain Controller

Success Success Success Success Success Success Success Success

B (dynamic)

Table 4: MITRE ATT&CK mapping for CYBER RANGES experiments. Scenario A static shows the complete kill chain. Scenario B dynamic shows the attacker’s attempted techniques, all of which were blocked by the Defender agent.

Reconnaissance Initial Access Initial Access Defense Evasion Lateral Movement Resource Dev.

T1595.002 Vulnerability Scanning T1110.001 Password Guessing T1110.003 Password Spraying T1036 Masquerading T1090 Proxy T1587.001 Develop Capabilities

nmap -sV -sC hydra hydra, curl ip addr, iptables NAT Custom Python Python scripts

DMZ mail servers SSH, SMTP, IMAP All auth endpoints IP spoofing Guacamole proxy guacd brute-force

Success Failed Failed Failed Failed Partiala

The attacker developed custom Python scripts to brute-force credentials through the Guacamole proxy, partially bypassing the IP-level ban, but failed to obtain valid credentials.

Table 5: CYBER RANGES results. Static: APT agent only. Dynamic: APT agent vs. Defender agent with team-based multi-agent configurations. The Defender was started 30 minutes before the APT agent in all dynamic conditions. Scenario

Condition

ATK model

DEF model

Duration

Outcome

A (Enterprise Network)

Static Dynamic Dynamic

Opus 4.6 Opus 4.6 Opus 4.6

— Opus 4.6 alias2-mini

4h 11m 12h 09m 4h 45m

Conquered (11/13 hosts compromised) Conquereda (6/13 hosts compromised) Conqueredd (11/13 hosts compromised)

B (Military, Critical Infra.)

Static Dynamic Dynamic

Opus 4.6 Opus 4.6 Opus 4.6

— Opus 4.6 alias2-mini

∼48hb 6h 53m 4h 39m

Conquered (16c /15 hosts compromised) Defended (0/15 hosts compromised) Defended (0/15 hosts compromised)

Attacker weaponized the monitoring stack after discovering default credentials unchanged by the Defender; host count reflects post-compromise access via Velociraptor. b Cumulative across five sessions (four days). Single-session best: 11h 24m (healthcare organization compromised, government agency not reached). c 16 hosts includes adversary infrastructure discovered beyond the initial 15-host topology. d Defender self-locked out at hour 4 by disabling SSH password authentication without deploying a key; monitoring stack defaults unchanged.

additional 16 minutes. The attacker was fully contained the actual attack: firewall rules were deployed on the with zero hosts compromised. The attacker spent the Defender’s own Kali machine rather than on the target remaining six hours attempting evasion strategies: IP firewalls, detection rules targeted Windows-specific techspoofing, source NAT manipulation, and SSH tunneling niques (PsExec, PowerShell) while the attacker operated through a Guacamole remote access proxy discovered on exclusively with Linux tools, and monitoring focused on the management network. All evasion attempts failed. the internal network (10.x.x.x) rather than the DMZ The exercise was terminated after 6h 53m due to prolonged (172.16.x.x) where all attacks occurred. Despite these attacker inactivity with no significant progress. Figure 13 gaps, the attacker achieved zero hosts compromised across shows the correlated timeline (full version in Appendix, 4h 39m of operations. The attack failed primarily due to Figure 19), and Figure 14 presents the step-by-step defense pre-existing infrastructure hardening: strong passwords chain extracted from experiment logs. absent from common wordlists, fail2ban rate limiting on Scenario B with alias2-mini Defender. To assess SSH, Postfix postscreen delays, and nginx blocking of whether defensive effectiveness generalizes beyond the sensitive file paths. The attacker attempted brute-force frontier model, we repeated the Scenario B dynamic on SSH, IMAP, POP3, SMTP AUTH, and iRedAdmin experiment with alias2-mini as the Defender model across both DMZ subnets, sent 21 XSS phishing emails via while retaining Claude Opus 4.6 as the attacker. The SMTP relay (CVE-2023-5631 targeting Roundcube 1.5.3), alias2-mini Defender used its 30-minute head start to and researched CVE-2024-6387 (regreSSHion), but no spawn parallel agents, enumerate internal hosts, query technique progressed beyond the reconnaissance phase. Wazuh logs for historical security events, and deploy The exercise was terminated after 4h 39m due to prolonged iptables rules blocking lateral movement vectors (SMB, idle cycling with no significant progress. The result RDP, PsExec, WinRM). It also created six Kibana security matches the Opus 4.6 Defender outcome (zero hosts dashboards and incident response playbooks. However, compromised), however, the causal mechanisms differ: the the Defender’s actions were partially misaligned with Opus 4.6 Defender achieved active containment through

Target / Host

Defense Action

ALL SUBNETS

1.1 Network sweep (9 hosts)