DSLE: A Learning Environment for Dark Souls Boss Encounters

DSLE: A Learning Environment for Dark Souls Boss Encounters Derin Gezgin* , Jim O’Connor* , Tanner Goodwin, Gary B. Parker

arXiv:2608.09902v1 [cs.AI] 10 Aug 2026

Autonomous Agent Learning Lab Connecticut College New London, Connecticut, USA {dgezgin, joconno2, tgoodwin, parker}@conncoll.edu

Figure 1: Overview of the boss encounters from Dark Souls: Remastered supported in DSLE. These encounters span a diverse range of environments, enemy designs, and combat dynamics, providing a challenging benchmark for real-time control policies. Abstract We introduce the Dark Souls Learning Environment (DSLE), a containerized platform that presents all 22 boss encounters of Dark Souls: Remastered as game-playing agent benchmarks through a Gymnasium-style interface. DSLE combines real-time combat, high-dimensional visual input, and sparse terminal rewards, with each environment step being a real action executed against the running game. To support controlled comparison, we define DSLE-5, a representative fiveboss subset, spanning a melee fight, a spatially constrained arena, an environmental-hazard fight, a multi-target fight, and a fast final-boss fight, that we recommend as the starting suite for agents built on DSLE. On DSLE-5 we evaluate a random policy, an expert system, an evolutionary baseline, and PPO and DQN agents trained from visual input. The expert system and the evolutionary baseline each defeat the Asylum Demon, the game’s tutorial boss (63% and 43% peak win rates), but none of the five methods defeats the other four DSLE-5 bosses; PPO and DQN show no measurable learning (≤0.33% win rate on the tutorial boss, 0% elsewhere) within a budget that already costs tens of wall-clock hours per run. A broader study running the evolutionary baseline across all 22 encounters under advantaged all level-50 stats yields wins on only a handful of additional early-game bosses and leaves the rest unwon. The failure cases range from sub-10second deaths in cramped, multi-target encounters to minutelong stalemates that inflict almost no damage, and we report them through survival time and damage dealt rather than win rate alone.† * These authors contributed equally. Copyright © 2026, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. † Code available at github.com/ConnAALL/dsle

1

Introduction

Game playing has long served as a testbed for artificial intelligence, providing controlled environments for evaluating decision-making under well-defined rules and objectives. Early work focused on deterministic, fully observable domains such as chess and checkers (Shannon 1950; Campbell, Hoane, and Hsu 2002; Samuel 1959), and later on complex board games such as Go (Silver et al. 2017), while more recent research has expanded to settings with stochasticity, partial observability, and high-dimensional observations. A key development in this transition was the introduction of standardized evaluation environments. The Atari Learning Environment (ALE) provides a diverse suite of games with a unified interface, enabling direct comparison across algorithms (Machado et al. 2018). ALE has been widely used in both reinforcement learning and neuroevolution research, including Deep Q-Networks (Mnih et al. 2015) and large-scale evolutionary approaches (Salimans et al. 2017), demonstrating that agents can learn directly from pixel input while highlighting challenges related to sample efficiency and exploration. More recent environments have extended this paradigm to increasingly complex domains. ViZDoom (Kempka et al. 2016) introduces a three-dimensional partially observable setting, while the StarCraft II Learning Environment (Vinyals et al. 2017) provides a large-scale real-time strategy domain with long horizons and hierarchical structure. However, these platforms rely on purpose-built interfaces that expose structured state information. Many modern commercial games do not provide such interfaces, limiting their

use in reproducible evaluation pipelines. Dark Souls: Remastered presents one such domain. The game is a third-person action role-playing game characterized by real-time combat, delayed and sparse rewards, and visually complex environments. Progression is organized around boss encounters, each of which can be treated as a distinct task with unique dynamics, including differences in timing, spatial constraints, and interaction structure. Prior work introduced the Dark Souls API (DSAPI), which enabled agent interaction through screen capture and computer vision (O’Connor, Parker, and Bugti 2025). While this demonstrated the feasibility of learning from pixel input, the system relied on visual estimation of game state, required manual reset procedures, and supported only a single boss encounter. In this work, we introduce the Dark Souls Learning Environment (DSLE), a containerized system designed to support reproducible experimentation in Dark Souls: Remastered. DSLE replaces vision-based state estimation with direct memory access, enabling reliable measurement of player and boss state. The environment includes automated reset through save-state swapping, headless execution, and support for distributed training. All 22 boss encounters are supported as independent tasks within a unified interface, each verified to load and reset correctly through the same pipeline. To establish DSLE as a benchmark and to assess its difficulty, we evaluate five representative boss encounters that we call DSLE-5 with a random policy, a scripted expert system, PPO and DQN agents trained from visual input, and Sparse Cosine Optimized Policy Evolution. Only the tutorial boss, the Asylum Demon, is ever defeated: the expert system wins 63% of episodes and SCOPE up to 43% of its best generation, while PPO and DQN never exceed a 0.33% win rate and show no measurable learning within a budget that already costs tens of wall-clock hours per run. None of the five methods defeats any of the four harder DSLE5 bosses, and this difficulty extends across the full 22-boss suite.

2 2.1

Related Work

Learning Environments for Games

Standardized learning environments have been central to progress in reinforcement learning for games, both by enabling controlled comparisons across algorithms and by introducing new axes of difficulty that expose the limits of prior methods. The Arcade Learning Environment is one of the first examples of a standardized learning environment for games, supporting dozens of Atari 2600 games behind a common observation, action, and reward interface, enabling agents to be evaluated across diverse tasks without game-specific engineering (Machado et al. 2018). This shared benchmark quickly became a standard testbed for both reinforcement learning (Mnih et al. 2015) and black-box evolutionary search (Salimans et al. 2017; Such et al. 2018). Later environments retained the same interfacecentric design while targeting properties Atari does not address. ViZDoom renders a first-person, partially observable

three-dimensional world from the raw screen buffer, on the argument that the two-dimensional, third-person perspective of arcade games does not resemble real-world tasks (Kempka et al. 2016), and the StarCraft II Learning Environment adds imperfect information, a combinatorial action space, and long-horizon rewards (Vinyals et al. 2017). Following the same interface-centric design, MineRL applies it to the open world of Minecraft, pairing a common agent interface with a large dataset of human demonstrations to study sample-efficient learning on long-horizon, hardexploration tasks (Guss et al. 2019). Extending this horizon further, the NetHack Learning Environment spans procedurally regenerated runs of tens of thousands of steps (Küttler et al. 2020), while Procgen isolates generalization by scoring agents on held-out instances drawn from the same procedural distribution rather than the levels on which they trained (Cobbe et al. 2020). More recently, this benchmarking paradigm has extended to large language model agents, which are evaluated as game players across large collections of procedurally generated games through a common interface (Li et al. 2026). Across this lineage, a purpose-built interface exposing structured state, rewards, and resets is what makes each environment reproducible as a benchmark. A parallel line of work studies unmodified commercial games, but these systems are often difficult to reuse as general benchmarks. AlphaStar achieved grandmaster-level StarCraft II play using league-based multi-agent training initialized from human replays (Vinyals et al. 2019), while OpenAI Five defeated world champions in Dota 2 through large-scale self-play (OpenAI et al. 2019). In games without such interfaces, agents have instead been trained from screen observations. For example, behavioral cloning in Counter-Strike shows that a commercial first-person shooter can be controlled from pixels alone, but through an offline imitation-learning pipeline rather than a live environment with automated resets (Pearce and Zhu 2022). This leaves an important gap: a reproducible benchmark for a real-time, third-person commercial action role-playing game with close-range reactive combat, visual observations, automated resets, and repeatable episodes. DSLE addresses this gap by exposing Dark Souls: Remastered as a standardized benchmark rather than a single-use integration.

2.2

Prior Dark Souls Learning Systems

The most directly related prior system is DSAPI (O’Connor, Parker, and Bugti 2025), a Python framework that exposes a limited agent interface to Dark Souls: Remastered through real-time screen capture, computer vision, and keyboard input emulation. Using this interface, O’Connor, Parker, and Bugti (2025) trained NEAT agents to fight the Asylum Demon from raw visual input rather than an official game API. Screenshots were downscaled to a compact RGB observation, and evolved policies selected from a discrete set of movement, attack, defensive, and healing actions. The result was an important first demonstration that learning agents could interact with Dark Souls: Remastered through the same visual channel available to a player, and that evolved policies could discover useful combat behavior in a commercial game without privileged simulator access.

Table 1: Comparison between DSAPI by O’Connor, Parker, and Bugti (2025) and the Dark Souls Learning Environment (DSLE). Feature Primary platform Headless execution Instances per machine State extraction Boss coverage Player configuration

DSAPI Windows ✗ 1 Template matching 1 boss Default setup

DSLE Cross-platform (Docker) ✓ Multiple isolated instances Screen capture, template matching, memory reading All 22 bosses Scenario-specific save states

In the published experiment, computer vision was used to extract player and enemy health from the game UI, recognize menu and terminal-state elements, and support reset through a predefined save file. This made Dark Souls: Remastered accessible as a research target, but the demonstrated system was still organized around a single boss encounter and relied on visual estimation of game state rather than direct access to structured environment variables. DSLE builds on this line of work by changing the role of Dark Souls: Remastered from a single visually controlled experiment into a benchmark environment. Instead of using computer vision as the primary state-estimation mechanism, DSLE combines rendered observations with direct processmemory instrumentation for reliable reward computation, termination checks, and diagnostic logging. It also adds containerized execution, automated reset through scenario save states, isolated parallel instances, and boss-specific initialization procedures. The resulting system preserves the central challenge of the earlier Dark Souls work, while providing a much wider task coverage and more structured execution infrastructure needed for systematic comparison. Table 1 summarizes the main differences between the earlier DSAPI and DSLE.

3

Dark Souls Learning Environment

Dark Souls: Remastered (FromSoftware 2018) is an action role-playing game widely known for its challenging combat system and reputation for difficulty (Gaither and Lee 2024). Player progression in the game depends on frequent boss fights, where the player must defeat powerful enemies with distinct attack patterns and behaviors. These fights require sustained decision-making under strict timing constraints, as mistakes can result in significant damage, immediate defeat, or loss of progress. From an AI perspective, boss fights provide a controlled but challenging setting for evaluating game-playing agents. Each fight is a self-contained task with clear objectives, bounded spatial limits, and explicit termination conditions. At the same time, the combat system demands reactive control, spatial awareness, and temporal coordination between offensive and defensive actions. These properties make boss encounters in Dark Souls: Remastered a useful benchmark for studying adaptive control policies in real-time game environments. The Dark Souls Learning Environment (DSLE) provides a scalable benchmark and software interface for training and evaluating agents in these encounters. Rather than expos-

ing the full game campaign, which includes long-horizon navigation, strategic decisions, item progression, and many smaller fights with non-player characters, DSLE isolates boss fights as repeatable combat scenarios. Each task begins from a preconfigured save state that places the agent in the boss encounter, gives the agent direct control of the player character, and ends when the boss is defeated, the player dies, or a step limit is reached. This focus keeps the task definition compact enough for controlled experimentation while retaining the core difficulty of Dark Souls: Remastered.

3.1

Available Boss Encounters

Each DSLE task is specified by four components: a boss identifier, a save state, a boss-specific initialization procedure, and an episode interface. The save state fixes the player build, inventory, equipment, and location. The standard DSLE-5 protocol reuses a single low-level starting character across all five encounters so that scores are comparable across bosses of very different intended difficulty, while the exploratory 22-boss study in Section 5.3 uses one uniform advantaged build for every encounter and is reported separately for that reason. Users can create additional scenario save files to study alternate character configurations, equipment loadouts, and inventory contents. These custom saves make it possible to vary the difficulty of an encounter, for example by changing health, stamina, armor, weapon damage, or starting position while retaining the same control interface. Scenario saves can also be paired with alternate game and display configurations, including resolutions up to 3840 × 2160. The initialization procedure loads the save, reaches the boss entrance, performs any encounter-specific setup, and then transfers control to the agent. After control is transferred, the same observation, action, reward, and termination interface is used across bosses. DSLE supports all 22 boss encounters in Dark Souls: Remastered. We verified that every encounter can be loaded, reset, and stepped through the interface so any of them can serve as an evaluation task, although the learning experiments in this paper use the DSLE-5 subset defined in Section 4. These encounters differ in spatial layout, number of enemies, vulnerability windows, phase structure, environmental hazards, and the ideal strategy to beat them. A single method can perform well on short reactive melee fights while failing on bosses that require waiting, target switching, long-horizon positioning, or distanced attacks. For instance, the Asylum Demon, the game’s opening boss, is designed to be beaten on a first attempt: it signals every slow hammer swing and only requires basic dodging and attacking. The Capra Demon is fought in what players widely consider the smallest arena in the game and is accompanied by two fast dogs that rush the player the instant the fight begins, making opponent selection and positioning crucial. Other encounters move the difficulty onto entirely different axes. Chaos Witch Quelaag continuously floods her arena with lava that lingers on the ground and shrinks the safe areas over time; Ornstein and Smough is a two-on-one fight in which defeating either boss heals and empowers the other, punishing the intuitive strategy of splitting damage; and the Bed of Chaos is not a regular fight at

Listing 1: Minimal single-episode interaction loop for a DSLE boss encounter. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

import dsle env = dsle.make( boss="asylum_demon", instance="dsr-1", obs_mode="grayscale", max_steps=7200, ) try: obs, info = env.reset() ret = 0.0 done = False while not done: action = env.action_space.sample() obs, reward, term, trunc, info = env.step(action) ret += reward done = term or trunc finally: env.close()

all, but a fight in which the player must destroy environmental targets while the floor collapses into instantly lethal pits. All bosses in Dark Souls: Remastered require different skills, from reactive timing to hazard avoidance, target prioritization, and multi-stage planning, allowing DSLE to function as a benchmark with a built-in difficulty gradient rather than a single repeated challenge. Table 2 summarizes the full supported boss set, listing the design and the control skills each encounter emphasizes.

3.2

Environment Interface

DSLE follows a Gymnasium-style interface (Towers et al. 2025): ot+1 , rt , terminated, truncated, info = step(at ).

(1)

Here at is the discrete action selected by the agent, ot+1 is the next observation, rt is the scalar reward, terminated and truncated are Boolean episode flags, and info is a dictionary for logging and evaluation. Listing 1 shows a minimal single-episode Python interaction loop for a boss instance, and Table 3 summarizes the default observation, action, termination, and reward configuration. Policy Input Space. The default policy input is a captured game frame from the player’s view. In the standard visual mode, DSLE converts this frame to a single-channel grayscale tensor with observation space Box(0, 255, (1, 600, 800), uint8). This image includes the same rendered arena, player character, boss, animation state, camera perspective, and game HUD that are visible in the running game, as shown in Figure 2. The environment also includes preprocessing wrappers for resizing observations, stacking frames, clipping rewards, changing time limits, and recording episodes. The environment records metrics such as player health, boss health, death count, lock-on state to the boss, and whether the boss has been defeated. These

Figure 2: In-game frame from Dark Souls: Remastered showing the player-view observation used by DSLE, including the arena, the player, the boss (in this case, Asylum Demon), and the game HUD with the player and boss health bars.

measurements are used for reward computation, termination checks, and logging, and are provided to the user in the info dictionary. Policy Output Space. The agent outputs one integer from Discrete(14). The 14 actions are four directional moves (forward, backward, left, and right), a light attack, a strong attack, a heal, a backstep, four directional rolls (forward, backward, left, and right), and two raw mouse-click actions (left- and right-click) for parrying. Each selected action is injected into the game and held for a configured duration, 250 ms by default, before the next action is queried; the agent therefore issues roughly four decisions per second of wall-clock game time. This fixed hold makes the control rate explicit and reproducible rather than tying it to the rendering frame rate. Lock-on to the boss is handled automatically by the environment at a fixed interval when enabled, as maintaining camera focus is a low-level bookkeeping action rather than a strategic decision in most boss encounters. Keeping it automatic leaves the 14 actions focused on movement, attacks, healing, and dodging, while the agent still has to learn combat timing and positioning. Automatic lock-on is configurable and can be disabled, and the current lock-on state is read from memory and logged for every step. Environment Outputs. After executing at , DSLE returns the next observation ot+1 in the selected observation space, a scalar reward rt , two Boolean episode flags, and an info dictionary. The terminated flag is true when the boss is defeated or the player dies. The truncated flag is true when the episode reaches the step limit. The reset and step metadata include diagnostic fields such as the boss identifier, instance name, current step, raw player and boss health, damage dealt and taken during the step, and whether the episode ended in a win.

Table 2: The 22 supported DSLE boss encounters. The “Characteristics” column summarizes each encounter’s design, and “Benchmark Focus” uses a fixed, color-coded tag set to mark the control skills and pressures each boss emphasizes: Melee : reactive close combat; Positioning : spatial positioning; Multi-target : multiple simultaneous targets; Phase/state : phase or state transitions; Ranged : ranged or projectile threats; Hazards : environmental hazards; Perception : hard-to-perceive targets; Objective : objective completion beyond pure combat; and Transfer : reuse of behavior learned on a related boss. Boss

Characteristics

Benchmark Focus

Asylum Demon Tutorial boss with slow, heavily telegraphed attacks and a simple arena. Melee Positioning Bed of Chaos Puzzle boss focused on destroying environmental targets while the arena collapses. Objective Hazards Phase/state Multi-target Phase/state Bell Gargoyles Rooftop boss where a second gargoyle joins mid-fight. Capra Demon Early boss fought in a cramped arena with two aggressive dogs and limited room. Multi-target Positioning Ceaseless Discharge Large boss fought near lava, with long-reaching attacks and narrow attack windows. Ranged Hazards Centipede Demon Lava-area boss with long-reaching attacks, restricted safe ground, and severable limbs. Hazards Positioning Chaos Witch Quelaag Area denial boss with sword attacks, lava pools, and explosive fire area attacks. Hazards Melee Crossbreed Priscilla An invisible boss that is tracked mainly through footprints in the snow. Perception Positioning Dark Sun Gwyndolin Ranged magic boss fought down a long corridor with repeated projectile attacks. Ranged Positioning Demon Firesage Late-game Asylum-style demon with heavy melee swings and explosive leap attacks. Transfer Hazards Melee Multi-target Phase/state Four Kings Abyss boss where additional Kings spawn over time, creating escalating pressure. Gaping Dragon Large boss with slow, telegraphed slams, charges, and long recovery windows. Positioning Melee Gravelord Nito Boss supported by numerous skeleton enemies, with Nito relying on slow melee attacks. Multi-target Positioning Melee Great Grey Wolf Sif Large sword-wielding wolf boss fought in an open arena with wide sweeping attacks. Melee Positioning Gwyn, Lord of Cinder Final boss with fast, aggressive sword attacks and high pressure at close range. Melee Positioning Iron Golem Large fortress boss fought on a narrow high platform where falls are a major threat. Hazards Positioning Melee Ranged Positioning Moonlight Butterfly Flying ranged boss that attacks with magic projectiles before landing briefly. Multi-target Phase/state Ornstein and Smough Dual-boss dual-phase fight with a fast spear user and a slow heavy attacker. Pinwheel Weak boss that teleports, casts fire magic, and creates clones of itself. Perception Multi-target Seath the Scaleless Large crystal puzzle boss with sweeping attacks, and an initial vulnerability mechanic. Objective Positioning Phase/state Stray Demon Stronger Asylum Demon variant with heavy melee attacks and large magic explosions. Transfer Melee Hazards Taurus Demon Early boss encounter. A large minotaur-like melee enemy fought on a narrow bridge. Melee Positioning Hazards

600

800

500

700

400

Player HP

Table 3: DSLE interface summary for the default observation, action, termination, and reward configuration. Component

Interface

Default observation Alternative observations Action space Default action hold Default step limit Termination Truncation Default reward terms

Grayscale frame, 1 × 600 × 800, uint8 RGB frame, 3 × 600 × 800; diagnostic state only mode Discrete(14) player-control actions 250 ms per selected action 7200 steps per episode Boss defeated or player death Step limit reached Boss damage, player damage, step penalty, win bonus, death penalty

3.3

Stochasticity and Evaluation Protocol

Although every episode begins from a fixed save state, DSLE should not be treated as a fully deterministic environment. To measure this directly, we ran a test with Asylum Demon in which each trial executed the same set of actions: attack, move forward, attack, and roll forward, repeated for up to 200 steps. The test logged the selected action, boss health, and player health after each step, while leaving the game process, timing, animation system, and boss behavior unseeded by the environment. Across ten trials, the trajectories first diverged at step 12 and terminated after 134– 200 steps. Figure 3 shows the resulting boss-health and player-health trajectories for these 10 runs. An agent can exploit early regularities in a fight such as the boss entrance sequence, but reliable performance still requires feedback from the current episode state.

Boss HP

600

300

500

200

400 300

100 0

50

Run 1 (134 steps) Run 6 (199 steps)

100

Step

150

Run 2 (159 steps) Run 7 (200 steps)

200

0

Run 3 (174 steps) Run 8 (200 steps)

0

50

100

Step

Run 4 (189 steps) Run 9 (200 steps)

150

200

Run 5 (195 steps) Run 10 (200 steps)

Figure 3: Asylum Demon stochasticity replay test. Ten runs begin from the same save state and execute the same set of actions. The two panels show boss-health and player-health trajectories. Despite identical replayed inputs, the health trajectories diverge by step 12.

3.4

Runtime Architecture

Running Dark Souls: Remastered as an RL environment requires an orchestration layer over the game. DSLE executes the unmodified Windows game inside an Ubuntubased Docker container through Wine. Rendering is provided through a virtual X11 display, and DXVK translates DirectX calls to Vulkan for GPU-backed execution. Each game instance receives its own display number, VNC port, Wine prefix, save directory, desktop name, and XDG runtime directory. This isolation avoids collisions between Wine processes and allows several independent encounters to run in a single container on a single machine. Episode resets in the environment are automated. Before

an episode, DSLE copies the target scenario save into the active game save directory, navigates the game’s menus using template matching, loads the character, waits for the bossentry prompt, and enters the arena. Some encounters require additional setup operations, such as walking to a trigger, waiting for a template, or skipping a cutscene. These operations are part of the environment reset procedure and are performed before the agent-controlled combat phase. The action layer sends keyboard and mouse events through X11 input injection. From the game’s perspective, the agent is pressing the same keys and mouse buttons as a human player. The observation layer captures frames from the virtual display. DSLE combines visual templates with process-level instrumentation of the running game process. At runtime, DSLE locates the game process associated with the active instance and opens the Linux /proc/<pid>/mem interface for reading the memory state. The reader resolves fixed base pointers for the player, boss, event-flag, lock-on, and menu structures, then follows boss-specific pointer chains declared in the configuration files. Through these memory reads, DSLE retrieves player health, maximum player health, death count, boss health, boss-defeated flags, and lock-on state. This instrumentation is used to compute rewards, recover from menus, return to the title screen, and verify terminal states.

3.5

Runtime Scalability and Reproducibility

In DSLE, each environment step is bounded by the real execution of the game and the selected action-hold duration. Consequently, the environment is not intended to match the throughput of JAX-native environments (Matthews et al. 2024; Freeman et al. 2021). This creates a trade-off between single-instance throughput and the fidelity of running an unmodified commercial game. DSLE is designed to be scalable through parallelism rather than high single-instance throughput. It can generate multiple game instances, assign them separate displays and Wine prefixes, and run them concurrently inside the same container or across multiple machines. DSLE runs on any Linux host with an NVIDIA GPU and Docker. Each concurrent game instance adds roughly 0.3 GB of GPU memory and 2–3 GB of system RAM on top of a shared ∼1.2 GB GPU baseline, measured at the default 600 × 800 observation size, and a single game installation occupies about 30 GB of disk that is shared across all instances on a machine. Because system RAM rather than GPU memory is the binding constraint at scale, the number of concurrent instances is set by host memory: our experiments ran five instances per machine, but we were able to run up to 30 instances on a system with an NVIDIA RTX 5090 (32 GB of VRAM) and 128 GB of RAM, where the instances together used roughly 10 GB of VRAM and about 80 GB of RAM. The variation is driven almost entirely by episode resets rather than by the steps themselves: encounters in which the agent dies within a few seconds produce thousands of short episodes, and each reset must copy a save file, navigate the menus, and re-enter the arena. For reproducibility, DSLE includes Docker build files, setup scripts, scenario save states, environment checks, eval-

uation scripts, and baseline agents. The container can also expose VNC sessions and episode recordings, allowing researchers to inspect live agent behavior directly instead of relying only on scalar logs. The game itself is not redistributed with the code: users provide their own legally obtained copy of Dark Souls: Remastered, while the opensource artifact supplies the environment wrapper, automation, and benchmark code around that dependency. This follows the same distribution model as other game-based environments that ship only the research interface and require the user to supply the game, such as the StarCraft II Learning Environment, which requires a separately installed copy of the game (Vinyals et al. 2017). A single user-provided installation is reused across an entire instance pool, so scaling to many parallel environments does not require a separate purchased copy per running instance.

4

Experiments

DSLE is a benchmark environment for all 22 bosses supported by Dark Souls: Remastered. Our experiments aim to characterize environment stochasticity rather than treat it as a fixed open-loop puzzle, provide non-learning reference points through random and expert-system controllers, and establish learning baselines from both reinforcement learning and neuroevolution. We evaluate five baselines on a representative subset of five bosses that we call DSLE-5: a random policy, a scripted expert system, and PPO and DQN agents under a single shared protocol, together with the neuroevolutionary policy. The DSLE-5 subset is chosen to cover the distinct pressures of the benchmark: it includes a tutorial melee fight (Asylum Demon), a spatially constrained arena (Capra Demon), an environmental-hazard fight (Chaos Witch Quelaag), a multi-target fight (Ornstein and Smough), and a fast final-boss fight (Gwyn, Lord of Cinder). We recommend DSLE-5 as the starting suite for anyone building an agent on DSLE. Running all 22 bosses is costly in both wall-clock time and compute, so this representative subset gives researchers a lower-cost target on which to develop and compare methods before committing to the full set. DSLE-5 is not the only possible suite that can be constructed from the 22 bosses. The encounter tags in Table 2 can be used to construct further boss suites with specific characteristics.

4.1

Evaluation Metrics and Episode Protocol

Each episode begins by loading a boss-specific scenario save, navigating into the encounter, and transferring control to the agent at the start of combat. The policy observes grayscale visual frames and selects from the shared Discrete(14) action space described in Section 3. An episode terminates when the boss is defeated or the player dies, and it is truncated when the configured step limit is reached. Because the game process has its own internal randomness, agent seeds control only agent-side randomness such as action sampling, parameter initialization, and any other random initializations throughout the pipeline. Setting the seeds in agent scripts does not make the game dynamics deterministic, as we demonstrate in Section 3.3.

4.2

Random and Expert-System Baselines

We evaluate two non-learning baselines on DSLE-5: a random policy and a rule-based expert system. These baselines provide non-learning reference points: uniform random action selection and a simple hand-written reactive policy. The random policy samples uniformly from the action space defined in Section 3 at every step. The expert-system baseline is a small rule-based controller. It observes the diagnostic state exposed in the info dictionary, heals when player health falls below 40% of the maximum player health observed so far, waits through a short heal cooldown, and otherwise alternates between moving forward and attack. This controller is not meant to approximate expert human play; it is a simple reactive combat policy for the benchmark. After the scenario setup procedure, the boss is initialized in front of the agent, so the simplest hand-written combat prior is to move forward, attack, and spend heals only when health is low. This tests whether a minimal closedloop combat heuristic is sufficient for any encounter while revealing how quickly the benchmark becomes harder when timing, positioning, target management, or hazard avoidance matter.

4.3

Reinforcement Learning Baselines

We train PPO (Schulman et al. 2017) and DQN (Mnih et al. 2015) baselines on DSLE-5 using visual observations only. The policies never receive the memory-extracted game-state scalars; those are used solely for reward computation, termination, and logging. Both agents observe the grayscale (1, 600, 800) frame, which the preprocessing pipeline downsamples to (1, 84, 84) before it is passed to the convolutional architecture introduced by Mnih et al. (2015), consisting of three convolutional layers followed by a fully connected layer. For PPO, the network has a shared convolutional trunk with separate policy and value heads. For DQN, the network predicts action values and uses epsilon-greedy exploration. The reinforcement learning agents receive the shaped environment reward rt = −0.001+∆hb −0.25∆hp +100·1win −10·1death , (2) where ∆hb is the fraction of boss health removed during the step and ∆hp is the fraction of player health lost during the step. This reward provides intermediate feedback for damage dealt and damage received while preserving the terminal win bonus as the main success signal. We use a single fixed evaluation protocol for both methods and all five bosses, with no per-boss or per-method tuning of the experimental setup: each method and boss is run with the same five agent-side seeds, the same target budget of 100,000 environment steps per run, and the same network architecture and reward. The protocol is fixed across bosses; PPO and DQN differ only in their algorithm-specific hyperparameters, as described below. This keeps the comparison reproducible and avoids tailoring the protocol to any individual encounter. PPO uses rollouts of 128 steps, four optimization epochs, minibatches of 32, learning rate 2.5×10−4 with annealing, γ = 0.99, generalized advantage estimation with λ = 0.95, clipping coefficient 0.1, and entropy coefficient 0.01. DQN uses a replay buffer of 100,000 transitions,

minibatches of 32, learning rate 10−4 , γ = 0.99, epsilon annealed from 1.0 to 0.01 over the first 10% of training, learning starts after 10,000 steps, and target-network updates every 1000 steps.

4.4

Neuroevolutionary Baseline

We also include a neuroevolutionary baseline: Sparse Cosine Optimized Policy Evolution (SCOPE), a lightweight derivative-free policy for high-dimensional visual game inputs (O’Connor et al. 2025, 2026). SCOPE is motivated by the fact that evolutionary methods become harder to optimize as visual inputs increase the number of free policy parameters. It addresses this by applying a two-dimensional discrete cosine transform to each grayscale observation, retaining the low-frequency and high-energy coefficient block, sparsifying low-magnitude coefficients, and mapping the resulting sparse coefficient matrix to action preferences with a bilinear affine policy. We optimize this compact policy with CMA-ES (Hansen and Ostermeier 2001). Five independent SCOPE trials are reported for each boss in DSLE-5. SCOPE runs use grayscale 600 × 800 observations, with the SCOPE policy used to select one of the 14 policy actions exposed by the runtime. SCOPE uses truncation parameter k = 100, sparsification parameter p = 90, CMA-ES initial step size σ = 0.5, and no explicit population-size override. With 14 output actions, this gives a 1,514-parameter chromosome. The default CMA-ES population under this configuration contains 25 candidates, so each 40-generation training trial evaluates 1,000 candidate episodes. Candidate evaluations are parallelized across Docker/Wine game instances locally, which significantly reduces the wall-clock time required to train a single SCOPE policy. SCOPE uses the fitness f = (Hp + (1 − Hb )) × 100,

(3)

where Hp is the final player-health ratio and Hb is the final boss-health ratio. A perfect episode in which the player defeats the boss without taking damage obtains a fitness of 200. Because fitness adds the survival and damage terms, an intermediate value does not by itself indicate a win: an agent can score highly by either removing most of the boss’s health or retaining high player health, even without a kill. Wins are therefore recorded separately, from the boss-death flag rather than from fitness.

4.5

Computational Cost

Because every environment step executes a real action against a running copy of the game, the computational overhead of DSLE is limited by the game’s execution speed. At the default 250 ms action hold the agent makes roughly four decisions per second of game time. The 100,000-step budget is close to seven hours of game time per run before a single reset is counted. Episodes end on a win or a death and truncate at 7,200 steps, and because episode length is determined by how long the agent survives rather than by a fixed schedule, the number of episodes inside that budget varies by boss and by method: pooled over five seeds, PPO completes between 1,684 and 6,423 episodes per boss and DQN between 2,284 and 12,549, as shown in Table 4. A single run takes

Wins / episodes Method

Asylum Tut.

Capra Early

Quelaag Mid

O&S Hard

Gwyn Hard

Random 0/100 0/100 0/100 0/100 0/100 Expert 63/100 0/100 0/100 0/100 0/100 PPO 5/1971 0/6423 0/1684 0/5309 0/5892 DQN 8/2469 0/12549 0/2284 0/8203 0/9942 SCOPE 1065/5000 0/5000 0/5000 0/5000 0/5000

Table 4: Per-episode results on DSLE-5 under the standard save states, reported as wins / episodes to make sample sizes explicit. Random and Expert are evaluated for 100 episodes per boss. PPO and DQN are pooled over five seeds trained for 100,000 steps each, and SCOPE is pooled over five seeds of 40 generations with 25 candidates each (5,000 episodes per boss). Only the tutorial boss is ever beaten; all four harder bosses hold every method to zero wins.

9–21 hours for PPO and 12–41 hours for DQN. This cost can be reduced through parallelism by running five game instances per machine across several machines.

5

Results

In this section, we present results primarily from DSLE-5. Table 4 reports the per-episode win rate of all five baselines on DSLE-5. On the tutorial boss, Asylum Demon, the methods separate cleanly: the scripted expert wins 63% of episodes, SCOPE wins 21%, and PPO and DQN win under 0.4%, while the random policy never wins. At its best generation, SCOPE reaches a 43% win rate, which exceeds the 35% reported at the same best-generation granularity for the NEAT agent of O’Connor, Parker, and Bugti (2025), the only prior neuroevolutionary result on Dark Souls: Remastered; this makes SCOPE the strongest neuroevolutionary policy reported on this boss to date. For the four harder bosses, every method records zero wins across thousands of episodes. In Section 5.1, we show that the tested gradient-based RL baselines show no sign of learning within a budget that already costs tens of wall-clock hours per run. They are in fact the weakest family we tested, beaten on an early boss by a simple derivative-free evolutionary policy and an expert system. In Section 5.2, we show that agents fail in distinct regimes, and these failure modes can be inferred from survival time and boss damage. Finally, in Section 5.3, we look beyond DSLE-5 to the rest of the benchmark’s 22 bosses, and find that the difficulty extends well past the five we studied in depth.

5.1

Reinforcement Learning Baselines Cannot Learn Within Budget

PPO and DQN show no measurable learning on any boss within the 100,000-step budget. Figure 4(b,c) plots episode reward against environment steps. Every curve is pinned at the death floor near −10 and never trends upward. As a win contributes +100 and this is never reached, the only signal is a small shaped-damage term set against a −10 death

penalty, leaving a near-flat, sparse reward landscape. A single run takes 9–21 hours for PPO and 12–41 hours for DQN (Capra Demon, whose short episodes force constant savestate reloads, is the most expensive), i.e. tens of wall-clock hours per run across the 50-run matrix. The evolutionary baseline does better only on the tutorial: SCOPE’s best-ofgeneration fitness (Figure 4(a)) climbs steadily on Asylum but stays flat near the floor on all four harder bosses, and it registers kills only on Asylum. With the given compute budget, gradient-based RL methods were only able to defeat the tutorial boss, and did so less frequently than both SCOPE and the expert-system agents.

5.2

How Agents Fail

The failures split into two regimes, both visible in how long an agent survives (Figure 4(d)) and how much boss health it removes. In the small-arena boss encounters the agent dies almost immediately: median survival is 7–19 s on Capra and 7–19 s on Gwyn across methods. This is very little time to land meaningful hits, so even the expert system removes only ∼ 7 out of 771 boss HP on Capra Demon. In the environmental-hazard fight the opposite happens: on Quelaag the agents survive longest of all (medians 52–68 s) yet still deal almost no damage, because the boss approaches slowly and the agent is killed by lava and ranged attacks before the agent engages.

5.3

Difficulty Beyond DSLE-5

The evaluation above characterizes DSLE-5, but DSLE supports all 22 encounters. To understand how far the difficulty extends, we ran an exploratory study over the full set, asking not how well an agent scores but whether each boss can be won at all under conditions that favor the agent. In Dark Souls: Remastered, a character’s overall strength is summarized by its stats: the total of the points invested across attributes such as Vitality, Endurance, and Strength. While the experiments above use the game’s low-level starting character, here we evaluated SCOPE on every encounter under advantaged save states built at all level-50 stats, which give the player substantially more health, stamina, and weapon damage. These advantaged saves are intentionally easier than the standard save states used in Section 4 and Table 4; the aim is not comparability with those results but a lower bound on which encounters are solvable under any configuration. Under this eased setting, SCOPE won only a handful of early-game encounters: 96% of episodes on the Capra Demon, 88% on the Taurus Demon, 16% on Pinwheel, and 8% on the Bell Gargoyles; while the large majority of the 22 bosses remained unwon. Compared to Table 4, SCOPE wins most advantaged episodes in Capra Demon, showing that per-encounter difficulty in DSLE can be tuned through the save state while the observation, action, and reward interface is held fixed. Several encounters outside DSLE-5 are not harder versions of the Asylum-style melee task that can be solved with better equipment. The Bed of Chaos has a single unit of health and cannot be beaten by dealing damage, the player must destroy two environmental targets while the arena floor

(a) SCOPE

(b) PPO

(c) DQN

100

50

0

0 −2 −4 −6 −8 −10 −12

0

10

20

30

Generation

40

150

Seconds per episode

150

(d) Survival time

2

Mean episode reward

2

Mean episode reward

Best fitness (0-200)

200

0 −2 −4 −6 −8 −10 −12

0

25

50

75

100

Environment steps (k) Asylum Demon

Capra Demon

Random

PPO

Expert

DQN

125 100 75 50 25 0

0

25

50

75

100

m

ylu As

Environment steps (k) Chaos Witch Quelaag

Ornstein & Smough

a pr Ca

ag

ela

Qu

S

O&

yn

Gw

Gwyn, Lord of Cinder

Figure 4: Learning curves and failure modes on DSLE-5 (five seeds; mean ± std in (a)–(c)). (a) SCOPE best-of-generation fitness value outlined in Section 4.4. (b,c) PPO and DQN mean episode reward stay pinned at the death floor (≈ −10) on every boss for the full 100k-step budget; the boss curves overlap because no method makes measurable progress. (d) Per-episode survival time by baseline (boxes span the IQR, whiskers 1.5×IQR): agents die within seconds in the cramped Capra and fast Gwyn fights but survive longest on Quelaag while dealing almost no damage, so long survival is not progress. collapses into instantly lethal pits. Seath the Scaleless is invulnerable until the player turns away from the boss to find and destroy a separate crystal that regenerates on each attempt. None of these fights is a reactive melee task; they demand objective completion, target switching, multi-stage planning, and adaptation to non-stationary dynamics.

6

Conclusion

We introduced the Dark Souls Learning Environment (DSLE), a containerized benchmark for game-playing agents in Dark Souls: Remastered. DSLE exposes all 22 of the game’s boss encounters through a single Gymnasiumstyle interface, runs the unmodified game inside Docker over Wine, and resets each encounter automatically from a scenario save state so that experiments run headlessly and in parallel without manual intervention. The encounters span a wide range of control problems, from a tutorial boss to multi-target, hazardous, and objective-based fights that demand qualitatively different skills. By bringing standardized benchmarking to a real-time commercial action RPG, DSLE opens an evaluation setting whose domain and deployment constraints are not covered by existing learning environments. Our baselines establish that Dark Souls: Remastered has a multi-tiered difficulty structure, with the harder environments being unsolved. An expert controller wins the majority of its Asylum Demon episodes but removes less than 1% of the health of every harder DSLE-5 boss; the evolutionary SCOPE policy wins only the tutorial Asylum Demon; and PPO and DQN show no measurable learning within a budget that already costs tens of wall-clock hours per run. Across all five baselines, the mid- and hard-tier bosses go unwon and agents fail in interpretable, encounter-specific ways. Reading survival time alongside the fraction of boss health removed, we find fights that kill agents within seconds, fights that agents survive but cannot meaningfully damage, and fights whose two-on-one or multi-stage structure defeats any fixed reactive behavior.

6.1

Future Work

DSLE opens the door to a broader set of baselines and agent families because each can be evaluated through the same interface, save-state protocol, and boss suite. The most immediate direction is methods built for sparse, long-horizon, real-time control (Lu, Hu, and Clune 2025). The flat learning curves of our model-free baselines point toward agents that can exploit denser intermediate structure, whether through reward shaping, recurrent or transformer-based policies that track combat state across frames, model-based rollouts that amortize the high per-step cost of a real game (Schrittwieser et al. 2020; Hafner et al. 2025), or imitation from human or scripted demonstrations (Guss et al. 2019; Baker et al. 2022; Aytar et al. 2018) to overcome the sparse success signal. The structured layout of DSLE also allows future work in curriculum and transfer learning. Because the encounters form a graded sequence and share several mechanics, such as the Asylum Demon and its later, tougher variants, agents could be trained from easy to hard rather than on each boss in isolation, and skills learned on one fight could transfer to related ones. The difficulty of any encounter can be tuned by changing the player’s health, stamina, equipment, or starting position while keeping the interface fixed, which supports both curriculum design and controlled studies of robustness. Because DSLE presents many distinct tasks under one interface, it is also a natural setting for multi-task and continual learning. Finally, as most of the architecture of DSLE is game-agnostic, the same methodology can be applied to other souls or soulslike games.

6.2

Limitations

DSLE executes a real action against the real game at a fixed control rate, so single-instance throughput is far below that of emulator or simulator-based benchmarks, and a single training run can take tens of hours. We address this through parallelism: instances are isolated and run concurrently within a container and across machines, and a single game installation is shared across an entire pool of instances. This makes wall-clock cost a function of how many instances a researcher can run, but it does not make DSLE

a high-throughput environment, and methods that need hundreds of millions of steps will find it expensive. The cost is also uneven across bosses, since encounters that end in fast deaths spend proportionally more time on resets than on agent control. Using DSLE requires a legally obtained copy of Dark Souls: Remastered, which the open-source artifact treats as an external dependency in the same way that the StarCraft II Learning Environment relies on a separately installed game client (Vinyals et al. 2017). A single purchased copy is reused across all instances on a machine, so scaling does not require buying additional copies. All the baselines were run on the five-boss DSLE-5 subset under a single fixed protocol of five seeds and a 100,000-step budget, chosen so that the comparison is reproducible and free of per-boss tuning rather than to chase the best possible score. These results bound what current methods achieve at a realistic compute budget; they do not establish the asymptotic difficulty of any encounter, and they leave most of the 22 bosses, as well as memory-based, model-based, and curriculum-driven approaches, open for future work.

References Aytar, Y.; Pfaff, T.; Budden, D.; Paine, T.; Wang, Z.; and de Freitas, N. 2018. Playing Hard Exploration Games by Watching YouTube. In Advances in Neural Information Processing Systems, volume 31. Baker, B.; Akkaya, I.; Zhokhov, P.; Huizinga, J.; Tang, J.; Ecoffet, A.; Houghton, B.; Sampedro, R.; and Clune, J. 2022. Video pretraining (VPT): learning to act by watching unlabeled online videos. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781713871088. Campbell, M.; Hoane, A. J.; and Hsu, F. 2002. Deep Blue. Artificial Intelligence, 134(1): 57–83. Cobbe, K.; Hesse, C.; Hilton, J.; and Schulman, J. 2020. Leveraging procedural generation to benchmark reinforcement learning. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org. Freeman, C. D.; Frey, E.; Raichuk, A.; Girgin, S.; Mordatch, I.; and Bachem, O. 2021. Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation. FromSoftware. 2018. Dark Souls Remastered. Video game. Gaither, S.; and Lee, N. 2024. Dark Souls RPG Through the Lens of Challenge, 517–519. Cham: Springer International Publishing. ISBN 978-3-031-23161-2. Guss, W. H.; Houghton, B.; Topin, N.; Wang, P.; Codel, C.; Veloso, M.; and Salakhutdinov, R. 2019. MineRL: a large-scale dataset of Minecraft demonstrations. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI’19, 2442–2448. AAAI Press. ISBN 9780999241141. Hafner, D.; Pasukonis, J.; Ba, J.; and Lillicrap, T. 2025. Mastering diverse control tasks through world models. Nature, 640(8059): 647–653.

Hansen, N.; and Ostermeier, A. 2001. Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation, 9(2): 159–195. Kempka, M.; Wydmuch, M.; Runc, G.; Toczek, J.; and Jaśkowski, W. 2016. ViZDoom: A Doom-based AI research platform for visual reinforcement learning. In 2016 IEEE Conference on Computational Intelligence and Games (CIG), 1–8. IEEE Press. Küttler, H.; Nardelli, N.; Miller, A. H.; Raileanu, R.; Selvatici, M.; Grefenstette, E.; and Rocktäschel, T. 2020. The NetHack Learning Environment. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Li, Y.; Lin, C.; Nasir, M. U.; Bontrager, P.; Liu, J.; and Togelius, J. 2026. GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games. arXiv:2508.08501. Lu, C.; Hu, S.; and Clune, J. 2025. Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models. In The Thirteenth International Conference on Learning Representations. Machado, M. C.; Bellemare, M. G.; Talvitie, E.; Veness, J.; Hausknecht, M. J.; and Bowling, M. 2018. Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research, 61: 523–562. Matthews, M.; Beukman, M.; Ellis, B.; Samvelyan, M.; Jackson, M.; Coward, S.; and Foerster, J. 2024. Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning. In International Conference on Machine Learning (ICML). Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-level control through deep reinforcement learning. Nature, 518(7540): 529–533. O’Connor, J.; Nash, J. B.; Gezgin, D.; and Parker, G. B. 2025. Playing Atari Space Invaders with sparse cosine optimized policy evolution. In Proceedings of the Twenty-First AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE ’25. AAAI Press. ISBN 157735-904-6. O’Connor, J.; Nash, J. B.; Gezgin, D.; and Parker, G. B. 2026. SCOPE for Hexapod Gait Generation. In Marcelloni, F.; Madani, K.; van Stein, N.; and Filipe, J., eds., Computational Intelligence, 236–247. Cham: Springer Nature Switzerland. ISBN 978-3-032-15635-8. O’Connor, J.; Parker, G. B.; and Bugti, M. 2025. Learning Dark Souls Combat Through Pixel Input with Neuroevolution. In 2025 IEEE Conference on Games (CoG), 1–4. OpenAI; Berner, C.; Brockman, G.; Chan, B.; Cheung, V.; Debiak, P.; Dennison, C.; Farhi, D.; Fischer, Q.; Hashme, S.; Hesse, C.; Józefowicz, R.; Gray, S.; Olsson, C.; Pachocki, J.; Petrov, M.; d. O. Pinto, H. P.; Raiman, J.; Salimans, T.; Schlatter, J.; Schneider, J.; Sidor, S.; Sutskever, I.; Tang, J.; Wolski, F.; and Zhang, S. 2019. Dota 2 with Large Scale Deep Reinforcement Learning. arXiv:1912.06680.

Pearce, T.; and Zhu, J. 2022. Counter-Strike Deathmatch with Large-Scale Behavioural Cloning. In 2022 IEEE Conference on Games (CoG), 104–111. IEEE Press. Salimans, T.; Ho, J.; Chen, X.; Sidor, S.; and Sutskever, I. 2017. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv:1703.03864. Samuel, A. L. 1959. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development, 3(3): 210–229. Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; Lillicrap, T.; and Silver, D. 2020. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588(7839): 604–609. Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347. Shannon, C. E. 1950. XXII. Programming a computer for playing chess. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 41(314): 256– 275. Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A.; Guez, A.; Hubert, T.; Baker, L.; Lai, M.; Bolton, A.; Chen, Y.; Lillicrap, T.; Hui, F.; Sifre, L.; van den Driessche, G.; Graepel, T.; and Hassabis, D. 2017. Mastering the game of Go without human knowledge. Nature, 550(7676): 354–359. Such, F. P.; Madhavan, V.; Conti, E.; Lehman, J.; Stanley, K. O.; and Clune, J. 2018. Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv:1712.06567. Towers, M.; Kwiatkowski, A.; Terry, J.; Balis, J. U.; Cola, G. D.; Deleu, T.; Goulão, M.; Kallinteris, A.; Krimmel, M.; KG, A.; Perez-Vicente, R.; Pierré, A.; Schulhoff, S.; Tai, J. J.; Tan, H.; and Younis, O. G. 2025. Gymnasium: A Standard Interface for Reinforcement Learning Environments. arXiv:2407.17032. Vinyals, O.; Babuschkin, I.; Czarnecki, W. M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D. H.; Powell, R.; Ewalds, T.; Georgiev, P.; Oh, J.; Horgan, D.; Kroiss, M.; Danihelka, I.; Huang, A.; Sifre, L.; Cai, T.; Agapiou, J. P.; Jaderberg, M.; Vezhnevets, A. S.; Leblond, R.; Pohlen, T.; Dalibard, V.; Budden, D.; Sulsky, Y.; Molloy, J.; Paine, T. L.; Gulcehre, C.; Wang, Z.; Pfaff, T.; Wu, Y.; Ring, R.; Yogatama, D.; Wünsch, D.; McKinney, K.; Smith, O.; Schaul, T.; Lillicrap, T.; Kavukcuoglu, K.; Hassabis, D.; Apps, C.; and Silver, D. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782): 350–354. Vinyals, O.; Ewalds, T.; Bartunov, S.; Georgiev, P.; Vezhnevets, A. S.; Yeo, M.; Makhzani, A.; Küttler, H.; Agapiou, J.; Schrittwieser, J.; Quan, J.; Gaffney, S.; Petersen, S.; Simonyan, K.; Schaul, T.; van Hasselt, H.; Silver, D.; Lillicrap, T.; Calderone, K.; Keet, P.; Brunasso, A.; Lawrence, D.; Ekermo, A.; Repp, J.; and Tsing, R. 2017. StarCraft II: A New Challenge for Reinforcement Learning. arXiv:1708.04782.

Related documents

• The eater of souls#65384
• Captains of souls#65111
• The Dark Sigil Will Guide Thee: The Hollowing Mechanic in FromSoftware's _Souls_ Games#77272
• Dark Destiny#61043
• Priest#618390