Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

arXiv:2608.28364v1 [cs.SE] 28 Aug 2026

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget Hiroki Mukai

Yusaku Kato

Norihiro Yoshida

is0692vs@ed.ritsumei.ac.jp Ritsumeikan University Ibaraki, Osaka, Japan

is0610ks@ed.ritsumei.ac.jp Ritsumeikan University Ibaraki, Osaka, Japan

norihiro@fc.ritsumei.ac.jp Ritsumeikan University Ibaraki, Osaka, Japan

Erina Makihara

Katsuro Inoue

makihara@fc.ritsumei.ac.jp Ritsumeikan University Ibaraki, Osaka, Japan

inoue-k@fc.ritsumei.ac.jp Ritsumeikan University Ibaraki, Osaka, Japan

Abstract Software testing often relies on assumptions such as reproducible executions and stable correctness criteria. However, many modern software systems exhibit non-deterministic executions and large behavior spaces, making exhaustive exploration impractical and single-run judgments unreliable. These characteristics make it difficult to identify where acceptable behavior ends and problematic behavior begins. Competitive multiplayer games represent a challenging instance of such systems, where balance must be maintained so that no single strategy dominates. Even small parameter changes can trigger abrupt balance disruption, yet detecting such failures requires repeated simulations under non-deterministic outcomes and high-dimensional parameter spaces. In this paper, we formulate game balance regression testing as a boundary-discovery problem under a finite simulation budget. The objective is to efficiently identify inputs near the boundary that separates balanced and unbalanced regions. To address this problem, we propose BBExplorer, which combines multi-directional candidate generation, budgetaware two-stage screening, and adaptive step-size shrinkage for boundary refinement. Experimental results on two games with different levels of complexity show that the approach is strong in lowdimensional settings and remains effective in higher-dimensional ones. It also exhibits stable boundary behavior across unseen random seeds and threshold settings. These results indicate that BBExplorer is effective for practical balance regression testing and, more broadly, for boundary-oriented testing in non-deterministic, budget-constrained systems.

CCS Concepts • Software and its engineering → Search-based software engineering; Empirical software validation.

Keywords Game balance, Search-based testing, Boundary discovery

This work is licensed under a Creative Commons Attribution 4.0 International License. ASE ’26, Munich, Germany © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2882-2/2026/10 https://doi.org/10.1145/3832783.3834407

ACM Reference Format: Hiroki Mukai, Yusaku Kato, Norihiro Yoshida, Erina Makihara, and Katsuro Inoue. 2026. Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget. In Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE ’26), October 12–16, 2026, Munich, Germany. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3832783.3834407

1

Introduction

Software testing has traditionally relied on assumptions such as reproducible executions and sufficiently stable correctness criteria, enabling systematic evaluation through coverage and test adequacy [1, 4, 14, 16]. However, many modern software systems exhibit large behavior spaces and non-deterministic executions, making exhaustive exploration impractical and single-run judgments unreliable [7, 10, 12, 23, 28]. In such systems, small variations in inputs or configurations can lead to disproportionately different behaviors, further complicating the identification of acceptable system behavior [15, 52]. Competitive games represent a particularly challenging instance of such systems, where large behavior spaces and non-deterministic executions arise from complex player interactions. Even small changes in parameters such as attack power or speed can disrupt balance, making certain strategies dominant and degrading player experience [21, 50]. With over three billion players worldwide [6, 39], such imbalances can affect a large user base. This challenge is especially pronounced in live-ops games, where patches and updates continuously introduce new parameter configurations [11, 31]. For example, League of Legends releases balance patches roughly every two weeks [40], requiring developers to repeatedly assess the impact of parameter changes under time constraints. To support regression testing during frequent game updates, GameRTS uses a state-transition-graph-based representation to select regression tests affected by game software changes [54]. These characteristics make balance evaluation inherently challenging under practical testing constraints. In such settings, exhaustive evaluation of candidate inputs becomes unrealistic, while repeated simulations remain necessary because outcomes vary across runs. As a result, developers need to understand not only which configurations perform well, but also where acceptable configurations stop and problematic ones begin. From this perspective, game balance regression testing can be viewed as a boundary-discovery

ASE ’26, October 12–16, 2026, Munich, Germany

problem under non-deterministic evaluation and limited simulation budgets. Existing research has addressed game balance from multiple directions, including restricted-play evaluation [19], player-behavior modeling [32, 33], and simulation-driven optimization [5, 18, 41]. In practice, however, developers need more than a single recommended configuration. They need to understand where balance disruption begins, namely, the boundary between balanced and unbalanced regions. Boundary-near inputs provide this information. They allow developers to verify whether a candidate patch remains on the safe side and to estimate how far parameters can be changed without introducing imbalance. In practice, developers first determine an acceptable win-rate range from their design goals and use its upper or lower limit as the target depending on whether they are testing overpowered or underpowered behavior. The discovered boundary-near inputs can then be retained as a patch-level regression suite and rerun after updates; inputs that move into the unbalanced region become candidates for inspection, rebalancing, or CI guardrail tests. In contrast to an optimum, which provides a single solution, boundary information characterizes the extent and shape of the safe region, including directions in which updates become risky. Identifying such boundaries is challenging. The parameter space is multi-dimensional, and boundaries may extend in multiple directions, making them difficult to capture with a single search trajectory. Moreover, game testing is already timeconsuming in practice [31, 35, 43], and each balance evaluation requires repeated simulations to obtain reliable win-rate estimates. As a result, exhaustive evaluation of candidate configurations is impractical within a single patch cycle. From a software testing perspective, this boundary-discovery task is better viewed as a test-input selection and generation problem [2, 4, 30] within a finite simulation budget, rather than an optimization problem that seeks a single best configuration. The input space is high-dimensional, win-rate measurement is nondeterministic [28], and each candidate input requires repeated executions. As a result, dense evaluation of all configurations is infeasible [15, 35], making efficient budget allocation across candidate inputs a central concern [20, 26]. In this paper, we formulate game balance regression testing [53] as a boundary-discovery problem under non-deterministic evaluation and a finite simulation budget. The objective is to efficiently explore the neighborhood of the boundary that separates balanced and unbalanced regions. Unlike prior optimization-based approaches, which focus on identifying a single optimal configuration, our goal is to explicitly characterize the boundary at which acceptable behavior breaks down. To address this objective, we introduce BBExplorer (Balance Boundary Explorer), which combines multi-directional candidate generation, two-stage candidate screening, and adaptive step-size shrinkage for boundary refinement. The contributions of this paper are as follows:

(1) We formulate game balance regression testing as a boundarydiscovery problem for generating test inputs under nondeterministic evaluation and a finite simulation budget.

Mukai et al.

(2) We propose BBExplorer, which integrates multi-directional candidate generation, two-stage candidate screening, and adaptive step-size shrinkage for boundary refinement. (3) Through experiments on a custom turn-based game (Turnbased) and a real-time strategy game (Generals), we demonstrate that BBExplorer achieves effective boundary localization in both low- and higher-dimensional settings and remains robust under seed and threshold variations.

2 Motivating Example 2.1 Win-Rate Terrain To motivate our approach, we use a simple turn-based competitive game with a two-dimensional parameter space as a running example. In this game, a focal character competes against a fixed opponent in a turn-based duel. The focal character has two adjustable parameters: attack power (ATK), which influences damage, and speed (SPD), which affects turn order and evasion. Figure 1 visualizes how the focal character’s win rate changes over the ATK–SPD space. We refer to this heatmap of estimated win rates as the win-rate terrain. To obtain it, we fix the opponentside evaluation condition and sweep the focal character’s ATK and SPD over a two-dimensional grid. At each grid point, repeated simulations are executed to obtain an estimated win rate, and the resulting values are visualized over the ATK–SPD grid. From this figure, several intuitive characteristics of boundary-near behavior can be observed. First, between balanced and unbalanced regions, we observe a transitional boundary band where the win rate can easily cross the acceptable threshold. Second, within parts of this band, small parameter changes produce abrupt shifts in win rate. In this subsection, we refer to such locally steep regions as cliffs in an observation-based sense. Third, the transitional region is curved rather than linear, suggesting that ATK and SPD interact in shaping the boundary. We use these terms as descriptive labels for the visible phenomena here, and formalize them later. These observations suggest that balance disruption begins along a curved and locally fragile boundary-near region rather than at an isolated point. They also suggest why discovering such regions is difficult: simple single-axis exploration or coarse evaluation may fail to capture where the balance judgment actually changes. The next subsection explains this difficulty from the perspectives of interaction, non-determinism, and budget constraint.

2.2

Why Boundary Discovery Is Hard

The terrain in Figure 1, introduced in Section 2.1, suggests why boundary discovery is difficult. This difficulty can be understood from three aspects: interaction, non-determinism, and budget constraints. First, interactions make it difficult to infer the boundary shape from a single axis. When multiple parameters (e.g., ATK and SPD) jointly affect the win rate, the boundary can take on a curved shape. As a result, single-axis exploration is insufficient and can miss regions near the boundary. Second, win-rate measurement is non-deterministic. Even with the same parameter configuration, match results vary due to randomness and opponent behavior. Reliable win-rate estimation, therefore, requires repeated executions rather than a single run. This

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

80

1.0

70

60

SPD

0.6 50 0.4 40

Estimated Win Rate

0.8

0.2

30

20

0.0 20

30

40

50

60

70

80

ATK

Figure 1: Estimated win-rate terrain over ATK and SPD. Colors indicate estimated win rates; the dashed band marks the acceptable range (0.4–0.6) separating balanced and unbalanced regions.

instability becomes especially important near the boundary, where small estimation errors can easily flip the judgment between balanced and unbalanced. Third, dense evaluation used to produce Figure 1 is realistic only in a low-dimensional example. Because each point requires repeated executions for reliable win-rate estimation, a dense grid can become prohibitively expensive as dimensionality increases. Under a finite simulation budget, the central question is therefore how to allocate evaluation effort across candidate inputs, rather than how to evaluate the whole space exhaustively. These difficulties directly translate into the design requirements introduced next.

2.3

Requirements

In this subsection, we formulate three design requirements (R1, R2, and R3) for a practical boundary discovery method. Each requirement addresses one of the difficulties discussed above; because these difficulties are closely connected, the three requirements must ultimately be considered together. R1: Multi-directional exploration. Due to interactions, the boundary may bend in complex ways in a multi-dimensional space. As a result, exploration along only a single axis may capture the boundary only partially. Both its position and its shape may therefore be characterized incompletely. A boundary discovery method is therefore required to generate candidate points in multiple directions and explore the space from multiple perspectives. R2: Sampling for judgment reliability. Near the boundary, an estimated win rate obtained from only a small number of executions can easily move across the threshold, making pass/fail judgment unstable. What matters here is not simply increasing the number of evaluations, but ensuring sufficient samples at each candidate point to support a reliable judgment. In other words, R2 is needed to make boundary detection reliable. R3: Efficient budget allocation. In practice, balance testing already consumes substantial development effort [31, 35], and the

ASE ’26, October 12–16, 2026, Munich, Germany

per-candidate cost of repeated match evaluation further tightens this constraint. At the same time, it is unrealistic to apply such a reliable evaluation uniformly to all candidate points under a limited computational budget. The method must therefore allocate more budget to points that appear promising and likely near the boundary, while suppressing excessive evaluation of points that are likely far from it. In other words, R3 is not about improving judgment reliability itself, but about deciding where limited resources should be spent. These three requirements must be satisfied simultaneously. Multidirectional exploration (R1) increases the number of candidate points, while sampling for judgment reliability (R2) increases the evaluation cost per point. Trying to satisfy both naively would quickly exhaust the computational budget. Because R1, R2, and R3 must be satisfied simultaneously under a limited budget, budget allocation (R3) becomes the central design problem: it determines which candidates should be screened coarsely and which should be examined in detail. To address this trade-off, our proposed method, BBExplorer, combines multi-directional candidate generation, budget-aware two-stage screening, and step-size shrinkage, as detailed in Section 4.

3 Problem Formulation 3.1 Parameter Space and Win Rate In this section, we formally define the boundary-discovery problem in game balance. Let 𝑝 denote the complete set of parameters that determine the outcome of a competitive game match. We decompose 𝑝 into controllable parameters 𝑃𝑐 and uncontrollable parameters 𝑃𝑐¯ , and write 𝑝 = (𝑃𝑐 , 𝑃𝑐¯ ). Here, 𝑃𝑐 ⊆ R𝑑 is the controllable parameter space, where each dimension corresponds to a game parameter (e.g., attack power ATK, speed SPD, or defense DEF). A point in this (𝑖 ) (𝑖 ) space, 𝑃𝑐(𝑖 ) = (𝑝𝑐,1 , . . . , 𝑝𝑐,𝑑 ), represents one concrete parameter configuration. By contrast, 𝑃𝑐¯ ⊆ R𝑚 denotes environment-side factors that affect match outcomes but are not themselves search variables, such as the opponent’s parameter configuration, map layout, and random seed governing stochastic match outcomes. For brevity, in the remainder, the simplified notation 𝑃𝑐 = 𝑃, ¯ 𝑃𝑐(𝑖 ) = 𝑃 (𝑖 ) , and 𝑝𝑐,𝑘 = 𝑝𝑘 is used. Under a fixed 𝑃, ¯ the 𝑃𝑐¯ = 𝑃, induced win-rate function over the controllable space is written as ¯ : 𝑃 → [0, 1] 𝑓𝑝 (· | 𝑃)

(1)

where · is a placeholder for the controllable configuration to be evaluated. Accordingly, the win rate at a configuration 𝑃 (𝑖 ) is written ¯ This notation makes explicit that environment-side as 𝑓𝑝 (𝑃 (𝑖 ) | 𝑃). ¯ while 𝑃 (𝑖 ) is factors are treated as a fixed evaluation condition 𝑃, the object explored by a developer. This interpretation also matches the operational meaning used in the experiments. Within one search run, 𝑃¯ should be read as a fixed evaluation condition in the simulator: the opponent-side setup, the simulation condition, and the distribution of stochastic outcomes are treated as fixed, while BBExplorer updates only 𝑃. Accordingly, the search changes only the controllable parameters,

ASE ’26, October 12–16, 2026, Munich, Germany

Mukai et al.

Win-Rate Terrain (3D Surface)

and the environment-side condition remains the fixed evaluation condition. ¯ is generally unavailable In practice, the true value of 𝑓𝑝 (𝑃 (𝑖 ) | 𝑃) analytically. For a given parameter configuration 𝑃 (𝑖 ) , we therefore ¯ run 𝑁 simulations and define the estimated win rate 𝑓ˆ𝑝 (𝑃 (𝑖 ) | 𝑃) as 𝑁 ∑︁ ¯ = 1 𝑓ˆ𝑝 (𝑃 (𝑖 ) | 𝑃) I[win in match 𝑗] (2) 𝑁 𝑗=1

3.2

Balanced Set and Boundary

In this subsection, we define the balanced set and its boundary ¯ In the remainder, 𝑃¯ denotes the fixed evaluation under a fixed 𝑃. condition for one search run, whereas the threshold and evaluation budget are testing-side settings. Let [𝑙, ℎ] ⊆ [0, 1] be the acceptable win-rate interval. Under a ¯ we define the balanced set as given 𝑃, ¯ ≤ ℎ} B𝑃¯ = {𝑃 (𝑖 ) ∈ 𝑃 : 𝑙 ≤ 𝑓𝑝 (𝑃 (𝑖 ) | 𝑃)

0.8

0.8

0.6 0.4

0.6

0.2 0.4

0.0 80 70 60 50 SPD 40 30 20

0.2

20

30

40

50

60

70

80

0.0

ATK

Figure 2: Three-dimensional view of the win-rate terrain. Steep surface regions correspond to cliff-like areas with large local sensitivity.

marks where the judgment switches between balanced and unbalanced configurations. A cliff, in contrast, denotes a fragile region where the win rate changes sharply under small parameter perturbations. These two notions are often close in practice, but they play different conceptual roles. Developers directly observe and modify only the controllable ¯ the visible win-rate terrain can therefore space 𝑃. Under a fixed 𝑃, be treated as one slice of the full space. To capture local change on this slice, we define the sensitivity under a given 𝑃¯ as ¯ = ∥∇𝑃 𝑓𝑝 (𝑃 (𝑖 ) | 𝑃)∥ ¯ 𝑆 (𝑃 (𝑖 ) | 𝑃)

(4)

where ∇𝑃 denotes the gradient with respect to the controllable parameters. This quantity measures how strongly the expected win rate responds to small parameter changes. Next, let 𝜏 > 0 be a cliff threshold. We define the cliff set as

(3) ¯ ≥ 𝜏} C𝑃¯ = {𝑃 (𝑖 ) ∈ 𝑃 : 𝑆 (𝑃 (𝑖 ) | 𝑃)

This is the set of parameter configurations regarded as balanced under that evaluation condition. The corresponding boundary is given by 𝜕B𝑃¯ , where 𝜕 denotes the boundary operator. This boundary marks where the judgment switches between balanced and unbalanced configurations. The boundary band introduced in Section 2.1 should be understood not as the formal boundary itself, but as an informal transition region around the formal boundary. In practice, the goal of this paper is not exhaustive characterization of all balanced configurations, but efficient discovery of boundary-near inputs under a finite simulation budget. Thus, while ¯ the practhe balanced set and boundary are defined under a fixed 𝑃, tical interest lies in capturing the neighborhood of that boundary efficiently. The next two subsections refine this view: Section 3.3 introduces sensitivity and cliffs near the boundary, and Section 3.4 formulates the resulting testing goal under budget.

3.3

Estimated Win Rate

Win Rate

Here, I[·] is the indicator function, which returns 1 when the condition holds and 0 otherwise. This is an empirical estimate of the win ¯ Under standard repeated-sampling rate under the fixed condition 𝑃. ¯ becomes a more stable approximation assumptions, 𝑓ˆ𝑝 (𝑃 (𝑖 ) | 𝑃) ¯ as 𝑁 increases. When 𝑁 is small, however, the estiof 𝑓𝑝 (𝑃 (𝑖 ) | 𝑃) mate can vary substantially, making the resulting balance judgment unstable, especially near a decision threshold. The robustness analyses in Section 5.3 follow the same distinction. A seed shift tests what happens when the stochastic aspect of the fixed condition 𝑃¯ changes. A threshold shift, by contrast, changes the boundary definition. The evaluation budget is a separate testing-side setting: it determines how thoroughly the search can be conducted but does not alter the boundary or the environment condition. Throughout this paper, we therefore distinguish environment-side conditions represented by 𝑃¯ from testing-side settings such as the threshold and the evaluation budget.

1.0 1.0

Sensitivity and Cliff

In this subsection, we introduce the notions of sensitivity and cliff to describe steep local changes near the boundary. The boundary

(5)

This set represents regions where small parameter changes can induce large win rate shifts. In this paper, the cliff concept is used primarily to explain why progressively finer boundary localization is needed near steep local transitions, rather than as a directly optimized or separately evaluated target. Figure 2 provides an intuitive view of a cliff. The steep regions suggested by the sharp color transitions in Figure 1 correspond to the formal cliff concept introduced here. Later experiments revisit these steep regions through the boundary-near inputs discovered by BBExplorer.

3.4

Testing Goal under Budget

Building on the preceding definitions, we now clarify the testing goal addressed in this paper. Unlike conventional software testing, where expected outputs can often be specified before execution, in game balance testing the expected win rate for a test input ¯ 𝑃 (𝑖 ) is unavailable a priori. The estimated win rate under a fixed 𝑃, (𝑖 ) ˆ ¯ written as 𝑓𝑝 (𝑃 | 𝑃), can only be obtained after actually executing simulations. This execution-based judgment forms the basis for the boundary-discovery problem addressed in this paper.

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

We first introduce the balance judgment condition only to fix the decision criterion. For a test input 𝑃 (𝑖 ) ∈ 𝑃, ¯ ≤ℎ 𝑙 ≤ 𝑓ˆ𝑝 (𝑃 (𝑖 ) | 𝑃)

(6)

means that the input is judged balanced. The main goal of this paper is not to confirm whether an input is balanced. We seek to efficiently obtain boundary-near inputs within a finite simulation budget. Recovering the full boundary via dense-grid evaluation requires many simulations, even in lowdimensions, and becomes even less realistic in higher dimensions. Under a finite simulation budget, a practical objective is therefore to generate as many boundary-near inputs as possible. From this perspective, this paper formulates boundary discovery as a test input generation problem. Given a target boundary value 𝑏 ∈ {𝑙, ℎ} and a tolerance 𝜖 > 0, a test input 𝑃 (𝑖 ) is regarded as boundary-near if it satisfies ¯ − 𝑏| ≤ 𝜖 | 𝑓ˆ𝑝 (𝑃 (𝑖 ) | 𝑃)

(7)

boundary-discovery problem. Given a finite computational budget 𝐵 and a starting point set 𝑋 0 = {𝑥 0(1) , . . . , 𝑥 0(𝐾 ) }, starting from each point in 𝑋 0 , generate as many test inputs 𝑃 (𝑖 ) satisfying the boundary-near condition in Equation (7) as possible. BBExplorer aims to generate boundary-near test inputs automatically. The resulting set of inputs provides developers with practical information about the boundary’s position and shape, supporting balance tuning within a limited budget.

4

Approach

BBExplorer (Balance Boundary Explorer), introduced in Section 1, is an iterative approach for the boundary-discovery problem defined in Section 3 under a finite simulation budget. To address R1–R3 (Section 2.3), it combines (i) multi-directional candidate generation, (ii) budget-aware two-stage screening, and (iii) step-size shrinkage. These components play complementary roles: multi-directional candidate generation probes positive and negative directions along each dimension (R1); two-stage screening reduces stochastic uncertainty through repeated simulations (R2); and top-𝑘 full evaluation concentrates the finite simulation budget on candidates estimated to be closest to the target boundary (R3). Figure 3 and Algorithm 1 summarize the overall procedure. From each 𝑥 0 ∈ 𝑋 0 , BBExplorer uses an equal budget share 𝐵/|𝑋 0 |. At each iteration, it generates candidates (Line 5), selects the next center by two-stage screening (Line 6), and checks boundarynearness. If the selected center is boundary-near, it is added to D and that path terminates (Lines 8–10); otherwise, the step size is shrunk and the search continues (Line 13). We describe the rationale and details of each component.

4.1

Candidate Generation and Screening

This subsection describes one iteration from candidate generation to next-center selection (Algorithm 1, Lines 5–6). Multi-directional candidate generation. Let 𝑥𝑡 ∈ 𝑃 be the current search center at iteration 𝑡 and let Δ𝑡 be the current step size. For each basis vector 𝑒𝑖 , it considers 𝑥𝑡 + Δ𝑡 𝑒𝑖 and 𝑥𝑡 − Δ𝑡 𝑒𝑖 , and clips the result to the feasible range of 𝑃 if it falls outside. The number of candidates per iteration is therefore at most 2𝑑. This design directly

ASE ’26, October 12–16, 2026, Munich, Germany

addresses R1 (multi-directional exploration) in Section 2.3. Probing both positive and negative directions along every axis improves robustness to local boundary-shape changes, while keeping the number of candidates linear in dimension. It also enables direct comparison of directions by boundary proximity before choosing the next center. Budget-aware two-stage screening. Given the candidate set C, BBExplorer allocates simulation budget in two stages under the same fixed condition 𝑃¯ (Figure 3). As discussed in Section 3.1, reliable boundary detection under non-determinism requires a sufficient number of simulation runs per candidate. However, allocating that cost to every candidate is infeasible under a limited budget. BBExplorer addresses this trade-off in two stages. Stage A: Screening. The role of this stage is coarse screening by estimated boundary proximity, not the final judgment. For all 2𝑑 candidates, BBExplorer executes a small number of simulations, ¯ The 𝑁 scr , and obtains a preliminary estimated win rate 𝑓ˆ𝑝scr (c | 𝑃). screening score of a candidate c is defined as 1 (8) 𝑠 (c) = scr ¯ − 𝑏 + 𝜀𝑠 𝑓ˆ (c | 𝑃) 𝑝

where 𝑏 is the target boundary value and 𝜀𝑠 > 0 is a small constant to prevent division by zero. Based on these scores, BBExplorer selects the top-𝑘 candidates: C scr = Screen(C, 𝑠, 𝑘)

(9)

Only the top-𝑘 candidates advance to Stage B, concentrating the remaining budget on the most promising boundary-near directions. Stage B: Full evaluation. The role of this stage is to determine the next search center based on reliable win-rate estimates. For the selected 𝑘 candidates, BBExplorer executes 𝑁 full simulations ¯ The candidate closest (𝑁 full ≫ 𝑁 scr ) and computes 𝑓ˆ𝑝full (c | 𝑃). to the target boundary value is selected as the next search center. Because this decision rests on 𝑁 full runs rather than on the coarse screening estimate, the center update uses a more precise win-rate estimate than the screening stage. This two-stage design addresses R2 (sampling for judgment reliability) and R3 (efficient budget allocation) simultaneously. It allocates more simulations to promising candidates while avoiding expensive full evaluation of clearly non-promising directions. The computational advantage is also clear. While full evaluation of all candidates costs 2𝑑×𝑁 full simulations, the two-stage procedure costs 2𝑑 × 𝑁 scr + 𝑘 × 𝑁 full (10) which is substantially smaller when 𝑘 ≪ 2𝑑 and 𝑁 scr ≪ 𝑁 full .

4.2

Step-Size Shrinkage

In this subsection, we describe the update rule for shrinking the step size Δ as the search progresses (Algorithm 1, Line 13). The role of this step is to combine coarse exploration and fine localization within a single search process. A fixed step size involves a fundamental trade-off. A large step size may overshoot boundary-near fragile regions, especially clifflike regions (with sharp win-rate drops), leading to coarse boundary approximation. Conversely, a small step size requires many iterations to reach the boundary from distant starting points, wasting

ASE ’26, October 12–16, 2026, Munich, Germany

Mukai et al.

one search path from a starting point 𝑥 0

𝑃cur

Generate up to 2𝑑 candidates

Stage A: screening

±Δ/axis (§4.1)

𝑁 scr runs (§4.1)

top-𝑘 candidates

Δ ← 𝛾Δ

Stage B: full eval

𝑃 next

𝑁 full runs

no

Boundarynear ?

yes Add to D

Step-size shrink (§4.2) full control flow and budget handling are given in Algorithm 1

Figure 3: Overview of BBExplorer. The top row shows one iteration of a search path: candidate generation, screening, and selection of the next center. The bottom row shows the update loop: if the selected center is boundary-near, it is added to D; otherwise, the step size is shrunk and the search continues. Algorithm 1 BBExplorer ¯ target Require: Starting point set 𝑋 0 , fixed evaluation condition 𝑃, boundary value 𝑏, tolerance 𝜖, initial step size Δ0 , decay factor 𝛾, budget 𝐵, screening size 𝑘 Ensure: Set of discovered boundary-near inputs D 1: D ← ∅ 2: for each starting point 𝑥 0 ∈ 𝑋 0 do 3: 𝑃cur ← 𝑥 0 , Δ ← Δ0 , 𝑏 used ← 0 4: while 𝑏 used < 𝐵/|𝑋 0 | do 5: C ← GenerateCandidates(𝑃cur, Δ) ⊲ Section 4.1: 2𝑑 candidates ¯ 𝑏, 𝑘) 6: 𝑃 next ← TwoStageScreen(C, 𝑃, ⊲ Section 4.1 7: Update 𝑏 used with simulation cost ¯ − 𝑏| ≤ 𝜖 then 8: if | 𝑓ˆ𝑝 (𝑃next | 𝑃) 9: D ← D ∪ {𝑃 next } 10: break ⊲ move to next starting point 11: end if 12: 𝑃 cur ← 𝑃next 13: Δ ←𝛾 ·Δ ⊲ Section 4.2 14: end while 15: end for 16: return D

the budget. The desirable step size therefore differs between early coarse exploration and later fine localization. To resolve this trade-off, BBExplorer monotonically decreases the step size as iterations progress. Specifically, it applies exponential decay with a decay factor 𝛾 ∈ (0, 1), so that the step size is updated as Δ𝑡 +1 = 𝛾 Δ𝑡 , where Δ𝑡 is the step size at iteration 𝑡. This update allows BBExplorer to approach the boundary quickly in early iterations and then localize boundary-near regions more precisely in later iterations. Step-size shrinkage is more than decay. Its main effect is to avoid both overshooting and stagnation while enabling a transition from coarse exploration to fine-grained refinement. This transition is especially important near the cliff-like steep regions discussed in Section 3.3, where a coarse step can jump over relevant boundarynear regions. To prevent the step size from becoming excessively small, a lower bound Δmin can be introduced, and the step size can be clipped when

Δ𝑡 < Δmin . However, the essential point is the coarse-to-fine change in search resolution rather than the lower bound itself. The specific hyperparameter values are reported in Section 5.

5

Experiments

This section presents an experimental study of BBExplorer under finite simulation budgets from four perspectives: low-dimensional performance, higher-dimensional performance, robustness under seed and threshold shifts, and computational cost. Our experiments compare BBExplorer with Random and FixedDelta. Random is a uniform-sampling baseline, whereas Fixed-Delta is an ablation that removes step-size shrinkage from BBExplorer. This comparison allows us to separate the effect of structured search itself from the additional benefit of coarse-to-fine refinement. We refer to Random and Fixed-Delta collectively as the baselines throughout this section. We address the following research questions.

• RQ1: Is BBExplorer more effective than the baselines in terms of discovery rate and boundary proximity? • RQ2: To what extent does BBExplorer retain its discovery performance as dimensionality increases? • RQ3: Do the boundary-near inputs discovered by BBExplorer remain stable under different random seeds and boundary thresholds?

5.1

Experimental Setup

We use two contrasting environments in the experiments: Turnbased, a controlled low-dimensional environment, and Generals, a demanding high-dimensional environment. Together with the corresponding simulator settings and evaluation metrics, they allow us to evaluate both the basic behavior of BBExplorer in controlled low-dimensional settings and its robustness in more demanding ones. Turnbased is based on a custom turn-based competitive game in which the boundary shape is relatively simple. It is mainly used in 2D and 3D conditions to assess the basic performance of BBExplorer in low-dimensional settings.

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

Generals is based on the real-time strategy game of the same name1 [49], implemented on top of an adapted version of the opensource simulator2 . This environment is higher-dimensional, more computationally expensive, and subject to stronger parameter interactions, making it a more demanding test bed for BBExplorer. In this paper, we use it mainly in 3D, 4D, and 5D conditions to study higher-dimensional behavior. Across both environments, all methods are evaluated under the same environment-specific simulation budget and the same matchevaluation protocol. Each search run is conducted under a fixed ¯ this setting serves as the reference point for the seed-shift and 𝑃; threshold-shift analyses in Section 5.3. We do not claim that the following settings are optimal; they instantiate the testing target, cost–reliability trade-off, and search resolution used in our evaluation.3 We use the following evaluation metrics, organized into four roles: success, efficiency, boundary proximity, and search-process quality. Let |𝐷 | denote the number of discovered boundary-near inputs returned from the |𝑋 0 | search paths, and let 𝑀 denote the total number of matches consumed. As the success metric, we use discovery rate 𝑅 = |𝐷 |/|𝑋 0 |: the fraction of starting points from which the method reaches a point satisfying the boundary-near condition in Equation (7). As the efficiency metric, we use discovery efficiency 𝐸 = 104 |𝐷 |/𝑀: the number of boundary discoveries per 10,000 matches consumed. In sparse-hit settings, 𝐸 is not interpreted alone but used alongside 𝑅 and the proximity metrics. As the boundary-proximity metric, we use mean boundary dis¯ the mean absolute difference between the estimated win tance 𝐷: rate at the final point of each search path and the target boundary value. It measures how close the method gets to the boundary by the end of the search, even when it does not converge. As the search-process-quality metric, we use boundary distance AUC 𝐴: the area under the per-iteration boundary-distance ¯ which captures only the curve over the search process. Unlike 𝐷, endpoint proximity, 𝐴 reflects whether the search trajectory stays close to the boundary from early stages onward. For statistical comparison, we use the Mann–Whitney U test and report Vargha–Delaney’s 𝐴ˆ12 as the effect size. This combination allows us to evaluate not only whether a difference exists, but also how large it is.

5.2

Baselines

Random samples candidate points uniformly without directional guidance, serving as an unstructured baseline. In contrast, FixedDelta retains the overall search framework of BBExplorer but uses a fixed step size instead of step-size shrinkage. Together, these 1 https://generals.io 2 https://github.com/strakam/generals-bots 3 Boundary setting: 𝜖 = 0.01, 𝑏 = 0.60 in the main experiments, 𝑏 ∈ {0.40, 0.50, 0.60}

in the threshold-shift analysis, and 𝑘 = 1. Turnbased 3D: 𝑁 scr = 200, 𝑁 full = 2000, Δ0 = 15.0, 𝛾 = 0.90, Δmin = 1.0, and max_iter = 50. Generals: 𝑁 scr = 20, 𝑁 full = 200, 𝛾 = 0.92, and max_iter = 40; Δ0 /Δmin = [6.0, 1.6, 0.7]/[0.5, 0.1, 0.05] in 3D, with 2.0/0.15 and 1.2/0.05 appended in 4D and 5D, respectively. Starts and trials: fixed-seed C-LHS with seed 42 for Turnbased and seed 0 for Generals; 30 trials for Turnbased 2D/3D and Generals 5D, 20 for Generals 3D/4D, and 10 for the additional 𝑏 = 0.40 and 𝑏 = 0.50 settings. Implementation: max_iter capped each search path, total_games recorded the consumed matches corresponding to 𝐵 , and each candidate evaluation used 𝑁 scr or 𝑁 full repeated matches.

ASE ’26, October 12–16, 2026, Munich, Germany

baselines allow us to separate the effect of structured search from the additional contribution of progressive refinement. Scope of comparison. Two broader families of methods could in principle be applied to boundary-near input discovery: model-based optimization (e.g., Bayesian optimization) and population-based search (e.g., evolutionary algorithms). We exclude them for separate reasons. Model-based methods construct a surrogate predictor from observed evaluations and use it to guide subsequent queries. In the present setting, environment-side factors and stochastic match conditions are not explicit search variables, and predictor construction under a short release cycle budget would require additional representation and modeling choices, effectively reframing the task as surrogate-based optimization rather than direct boundary testing [44, 46]. Population-based methods maintain a diverse pool of candidates and are effective at covering extreme or disconnected regions of the design space. However, the scenario targeted here, boundary-onset localization under small, incremental parameter updates, calls for concentrating the evaluation budget near a known balanced region rather than distributing it across distant candidates [24, 25]. By restricting the comparison to Random and Fixed-Delta, we isolate two controlled factors, the value of structured search and the additional benefit of shrinkage, without confounding them with modeling or population management decisions [47]. Our focus is not on outperforming general-purpose optimizers, but on isolating the effectiveness of boundary-oriented search under practical testing constraints.

5.3

Results

We next report the results for low-dimensional performance, higherdimensional performance, robustness under seed and threshold shifts, and computational cost. Low-dimensional results. In Turnbased, we evaluate BBExplorer in 2D and 3D settings, each based on 30 independent trials. We first discuss the 2D case, where the boundary shape is relatively simple. In the 2D experiment (ATK × SPD, 30 trials), BBExplorer achieves a discovery rate of 𝑅 = 1.000, Fixed-Delta 𝑅 = 0.967, and Random 𝑅 = 0.200. BBExplorer significantly outperforms Random (𝑝 < 0.0001, 𝐴ˆ12 = 0.900), confirming the effectiveness of structured search. Compared with Fixed-Delta, the improvement is marginal, indicating that the additional benefit of shrinkage is limited in 2D. In the 3D experiment, where DEF is added to the search space alongside ATK and SPD, Table 1 summarizes the results. BBExplorer maintains 𝑅 = 1.000 ± 0.000, while Random degrades to 𝑅 = 0.067 ± 0.254, indicating a clear loss of efficiency for unstructured sampling as dimensionality increases. BBExplorer continues to significantly outperform Random (𝑝 < 0.0001, 𝐴ˆ12 = 0.967) and also ¯ and boundary distance achieves better mean boundary distance (𝐷) AUC (𝐴). Fixed-Delta remains competitive with 𝑅 = 0.967 ± 0.183, suggesting that the additional benefit of shrinkage remains limited in this setting. In summary, BBExplorer reliably discovers boundary-near inputs in low-dimensional settings, with most of the advantage coming from structured search, while shrinkage provides only limited additional benefit.

ASE ’26, October 12–16, 2026, Munich, Germany

Mukai et al.

Table 1: Turnbased 3D results (ATK × SPD × DEF, 30 trials). ¯ mean boundary 𝑅: discovery rate; 𝐸: discovery efficiency; 𝐷: distance; 𝐴: boundary distance AUC. Method

𝑅

𝐷¯

𝐸

𝐴

BBExplorer 1.000±0.000 0.76±0.85 0.0043±0.0027 0.0830±0.0810 Fixed-Delta 0.967±0.183 0.69±0.77 0.0059±0.0033 0.1250±0.1171 Random 0.067±0.254 0.03±0.11 0.0766±0.0783 0.3315±0.0790

Table 2: Generals results across dimensions. 𝑅: discovery rate; ¯ mean boundary distance (lower is better). Trials: 20 for 𝐷: 3D/4D, 30 for 5D. 3D Method BBExplorer Fixed-Delta Random

4D

Table 3: Generalization test summary (5D, 15 paths × 5 seeds × 1000 games). Mean diff

Diff SD

Mean inter-seed std

Worst degradation

+0.014

±0.027

0.012

−0.026

Table 4: Target boundary value variation in 5D. The additional settings 𝑏 = 0.40 and 𝑏 = 0.50 use 10 trials each, while ¯ the main 𝑏 = 0.60 setting uses 30 trials. 𝑅: discovery rate; 𝐷: mean boundary distance (lower is better).

Method

5D

𝑅

𝐷¯

𝑅

𝐷¯

𝑅

𝐷¯

0.550 0.000 0.006

0.033 0.120 0.232

0.600 0.200 0.017

0.049 0.092 0.226

0.500 0.067 0.017

0.078 0.108 0.223

Higher-dimensional results. For Generals, results for the higherdimensional settings (3D, 4D, and 5D) are summarized in Table 2. We first evaluate a 3D setting by selecting three dimensions from the parameter space of Generals and comparing methods over 20 independent trials. As shown in Table 2, BBExplorer achieves 𝑅 = 0.550 in 3D, confirming that structured search remains effective in this environment. We next evaluate 4D (20 trials) and 5D (30 trials) settings. As dimensionality increases, achieving boundary proximity becomes harder. As shown in Table 2, BBExplorer reaches 𝑅 = 0.600 in 4D and 𝑅 = 0.500 in 5D. BBExplorer maintains its discovery performance despite the increased difficulty. By contrast, Random degrades to 𝑅 = 0.017 in 5D, showing a near-complete loss of efficiency for unstructured sampling. Fixed-Delta also degrades more strongly than BBExplorer in higher dimensions. Boundary proximity further supports this pattern: across all dimensions, BB¯ Explorer achieves consistently lower mean boundary distance (𝐷) than Fixed-Delta, indicating tighter convergence to the boundary. The 5D condition serves as the main high-dimensional stress test in this paper. Although precise boundary localization becomes more difficult there, BBExplorer still preserves discovery performance and behaves more stably than Fixed-Delta. This pattern suggests that, in higher-dimensional settings, shrinkage becomes increasingly important alongside structured search itself. Overall, BBExplorer continues to outperform Random, indicating that structured search remains effective in higher dimensions, while its advantage over Fixed-Delta becomes more pronounced, suggesting that step-size shrinkage provides additional benefit as dimensionality increases. Robustness under seed and threshold shifts. Tables 3 and 4 summarize the robustness results. We consider two robustness analyses. In the seed-shift analysis, as defined in Section 3.1, the same discovered input 𝑃 (𝑖 ) in the controllable space is re-evaluated under different random seeds, while all other components of the fixed evaluation condition 𝑃¯ are held fixed. In the threshold-shift analysis, 𝑃¯ is kept fixed, and only the target boundary value 𝑏 used

BBExplorer Fixed-Delta Random

𝑏 = 0.40 𝑅 𝐷¯ 0.600 0.200 0.043

0.013 0.062 0.179

𝑏 = 0.50 𝑅 𝐷¯ 0.500 0.300 0.029

0.025 0.100 0.189

𝑏 = 0.60 𝑅

𝐷¯

0.500 0.067 0.017

0.078 0.108 0.223

to define boundary-near inputs in Section 3.4 is varied. Together, these analyses address RQ3. Table 3 summarizes the Seed shift results. For the 15 paths that converged in the 5D experiment (out of 30 trials at 𝑅 = 0.500), we reran 1,000 matches for each configuration under 5 different random seeds. The difference between the original search-time estimate and the re-evaluated win rate had a mean of +0.014 and a standard deviation of ±0.027. The mean inter-seed standard deviation was 0.012, 10 of the 15 paths showed positive diff values, and the worst degradation was only −0.026. These results indicate that the boundary-near inputs discovered by BBExplorer are not overly dependent on a particular seed and remain close to the boundary under shifted stochastic conditions. Table 4 summarizes the Threshold shift results. In Section 3.4, boundary-near inputs are defined relative to a target boundary value 𝑏 ∈ {𝑙, ℎ}. Here we compare the additional settings 𝑏 = 0.40 and 𝑏 = 0.50, each evaluated over 10 trials, with the main 𝑏 = 0.60 setting, evaluated over 30 trials, to assess stability under different boundary definitions. BBExplorer achieved a mean discovery rate of 𝑅 = 0.533 across the three thresholds, whereas Fixed-Delta and Random showed considerably lower and less consistent results. The mean boundary distance of BBExplorer decreased from 𝑏 = 0.60 to 𝑏 = 0.40, indicating that localization became easier at lower thresholds. These results confirm that BBExplorer remains stable across boundary definitions and that shrinkage continues to provide an additional benefit under threshold shifts. Computational cost. From a practical standpoint, BBExplorer also remained computationally efficient in the high-dimensional setting. In the main 5D experiment, the average number of matches per path was approximately 9,000 for BBExplorer and approximately 15,000 for Fixed-Delta. The total execution time for all additional experiments4 was approximately 2.7 hours: about 65 minutes for the 5D 30-trial experiment, about 58 minutes for the 3D and 4D experiments, and about 40 minutes for the threshold-shift experiments. These figures confirm that BBExplorer operates within 4 All experiments were conducted on a machine equipped with an AMD Ryzen 9 7900

CPU, 64GB RAM, and an NVIDIA GeForce RTX 4070 Ti SUPER GPU.

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

a practical computational budget while maintaining its discovery advantage over Fixed-Delta.

6 Discussion 6.1 Answers to Research Questions We answer the three research questions based on the results in Section 5. RQ1 (Effectiveness). Strongly supported. In the Turnbased 3D setting, BBExplorer achieves 𝑅 = 1.000 compared with 𝑅 = 0.067 for Random (𝑝 < 0.0001, 𝐴ˆ12 = 0.967), along with a substantially smaller mean boundary distance (𝐷¯ = 0.0043 vs. 0.0766). In the Generals 5D setting, BBExplorer maintains 𝑅 = 0.500, whereas Random degrades to 𝑅 = 0.017 (Tables 1 and 2). These results confirm that structured search significantly improves boundary discovery performance. RQ2 (Scalability). Largely supported. In the Generals environment, BBExplorer achieves 𝑅 = 0.550 in 3D, 𝑅 = 0.600 in 4D, and 𝑅 = 0.500 in 5D, while Fixed-Delta remains at 𝑅 ≤ 0.200 and Random at 𝑅 ≤ 0.017 across all dimensions (Table 2). These results indicate that BBExplorer retains meaningful discovery performance as dimensionality increases, although precise boundary localization becomes more challenging in 5D. RQ3 (Robustness). Supported. Re-evaluation of the 15 converged 5D paths under five different random seeds shows a mean winrate difference of +0.014 (±0.027) from the original search-time estimates, with a worst-case degradation of only −0.026 (Table 3). Under threshold shifts, BBExplorer achieves a mean discovery rate of 𝑅 = 0.533 across three boundary values (𝑏 = 0.40, 0.50, 0.60), while Fixed-Delta and Random remain substantially lower (Table 4). These results indicate that the discovered boundary-near inputs remain stable under both stochastic variation and changes in boundary definitions. Summary. Overall, the results support the effectiveness of structured search for boundary discovery, show that step-size shrinkage provides additional benefits as dimensionality increases, and demonstrate that the discovered boundary-near inputs remain robust under stochastic variation and changes in boundary definitions.

6.2

Why BBExplorer Works

The experimental results show that BBExplorer consistently outperforms the baselines, achieving strong performance even in the most demanding Generals 5D setting and maintaining stability under both seed and threshold shifts (Tables 2 to 4). We attribute these outcomes to three design elements, each addressing a distinct challenge. Multi-directional candidate generation. At each iteration, BBExplorer probes both directions of every axis simultaneously. This makes the search responsive to local boundary shape, regardless of which axis is most relevant at a given point. As dimensionality increases, boundary orientation becomes harder to predict, and searching along a single axis or at random is more likely to miss the locally informative direction. Multi-directional probing reduces this

ASE ’26, October 12–16, 2026, Munich, Germany

risk, contributing to the sustained discovery performance observed across dimensions. Two-stage screening. The screening stage filters candidates using a small number of matches before committing the full evaluation budget. This design allocates computation to the most promising candidates rather than distributing it evenly. It also provides robustness against stochastic match outcomes: a candidate that passes screening is less likely to be a noise artifact. This is consistent with the seed-shift analysis in Section 5.3, where re-evaluation under different random seeds produced only minor deviations from the original estimates. Step-size shrinkage. A fixed step size must compromise between rapid progression toward the boundary and precise localization. Shrinkage removes this trade-off by allowing large steps when the search is far from the boundary and small steps when it is close. This transition is particularly important in higher dimensions, where the distance to the boundary can vary more across axes. The contrast with Fixed-Delta, which shares the same search framework but lacks shrinkage, directly isolates this contribution. These three elements address different aspects of the problem, namely multidirectional exploration, budget allocation under non-determinism, and progressively finer boundary localization. Their combination enables the consistent performance observed across the experimental conditions.

6.3

Limitations

In this subsection, we examine the main limitations of BBExplorer. Whereas Section 6.2 focused on why BBExplorer works, this subsection centers on the main limitations observed in the demanding Generals setting. Non-convergence. When starting points are far from the boundary, or when screening-stage variance is high, promising directions may fail to advance to full evaluation. As a result, the search may not reach the boundary-near region within the available iterations. This non-convergence becomes more visible in higher-dimensional settings. Fixed simulation counts. The current implementation uses fixed simulation counts 𝑁 scr and 𝑁 full for screening and full evaluation. A natural extension is a confidence-interval-driven stopping rule that adapts these counts based on the estimated distance from the target boundary, using a looser interval during screening and a tighter interval during full evaluation. We leave this dynamic allocation for future work. Step-size overshooting. When a starting point already lies near the boundary, a large initial step size may jump over the boundarynear region in early iterations. Highly irregular boundary shapes can likewise cause the search to leave the useful neighborhood before the step size has sufficiently decayed. Axis-aligned probing. Strong parameter interactions may require joint movements across multiple axes that axis-aligned probing cannot represent directly. Although BBExplorer evaluates both directions of every axis at each iteration, it can still miss boundary structures whose locally informative direction is not aligned with

ASE ’26, October 12–16, 2026, Munich, Germany

any single axis. This limitation reflects the greedy, axis-aligned nature of the search and suggests extending candidate generation beyond axis-aligned moves. Comparing BBExplorer with nongreedy or population-based search strategies, therefore, remains an important direction for future work. Single-path coverage. BBExplorer follows a single convergent path from each starting point. Accordingly, the discovered boundarynear inputs are limited to the portion of the boundary reachable along those paths. This limitation should be interpreted as a coverage constraint rather than outright failure. The goal of the approach is not exhaustive boundary mapping, but practical acquisition of useful boundary-near inputs under a finite simulation budget. Incremental scope. This work primarily targets scenarios where incremental parameter updates to an existing design trigger balance disruption. It is less suited to extreme parameter configurations that lie far from the known balanced region, including highly skewed settings that occupy disconnected regions of the parameter space. Such cases may require search designs beyond continuous stepsize shrinkage alone. At the same time, increasing the diversity of the starting point set offers a practical way to extend coverage toward such regions. Despite these limitations, BBExplorer still demonstrates practical effectiveness in discovering useful boundarynear inputs across the Turnbased (2D/3D) and Generals (3D–5D) settings evaluated in Section 5. The cases discussed above also point to clear directions for extension, including richer candidate generation and broader multi-path coverage.

6.4

Threats to Validity

Internal validity. The results depend on the evaluation protocol used in Section 5. Hyperparameters such as the screening and full evaluation counts, the initial step size, and the decay factor can influence the search behavior. Moreover, win-rate estimation involves Monte Carlo variance, and this residual variance cannot be completely eliminated near the boundary. Although all methods were compared under the same conditions and we additionally performed seed-shift analysis, the quantitative results may still vary under other configurations. That said, the consistent ranking across both environments and the stability confirmed by the seedshift analysis suggest that the main conclusions are not merely an artifact of the specific experimental setup used here. External validity. The experiments are limited to two environments, Turnbased and Generals, and to settings of up to five dimensions. These two environments nonetheless cover a range of complexity, from a controlled synthetic setting to an environment based on a real-world online strategy game. Additional studies are needed before generalizing the conclusions to higher-dimensional spaces, mixed-type parameter spaces, or commercial live-ops games. Within this scope, BBExplorer is not tied to a specific game genre. As formulated in Section 3, it applies when developers can define controllable parameters 𝑃, a representative evaluation condition ¯ and a measurable balance objective such as win rate. For real𝑃, time, human-input-heavy, or many-parameter games, the main challenge is defining representative 𝑃¯ conditions and selecting or grouping controllable parameters, rather than changing the

Mukai et al.

boundary-discovery formulation itself. The comparison set is limited to unstructured sampling and a core ablation. This design isolates structured search and step-size shrinkage, but it does not establish superiority over population-based, quality-diversity, or surrogate-based optimizers. A fair comparison would require adapting those methods to near-boundary discovery under the same stochastic match budget. Under that setting, BBExplorer concentrates evaluations along individual search paths, whereas populationbased methods may cover a broader set of boundary-near regions. Such comparisons remain important future work. In addition, 𝑃¯ is treated as a fixed evaluation condition throughout the experiments. Although the seed-shift analysis in Section 5.3 re-evaluated convergence under different random outcomes, opponent-side strategy diversity was not systematically varied. Evaluation under broader opponent variation and strategy diversity remains an important direction for further validation. Separately, the seed-shift results provide initial evidence that the discovered boundary-near inputs are not overly dependent on a single stochastic condition.

7 Related Work 7.1 Game Balance Testing Various parameter optimization approaches that leverage reinforcement learning (RL) and genetic algorithms have been proposed in game balance studies. For instance, approaches such as LUDUS utilize evolutionary computation to optimize attribute values in card games, while GEEvo focuses on optimizing resource flows within game economies [5, 42]. Additionally, RL-based approaches for evaluating and optimizing map configurations and deep learning-based behavioral modeling to assess balance have been explored [32, 41]. These conventional approaches primarily aim to identify a single set of optimal parameter values that satisfy specific evaluation metrics defined by developers. However, they typically require a high number of iterations to converge and do not focus on identifying the specific boundaries at which game balance begins to break down. In contrast, BBExplorer aims to efficiently identify the boundaries at which game balance becomes disrupted, along with their surrounding regions. In Games-as-a-Service settings, where continuous post-launch updates are required [11], developers must repeatedly adjust parameters to prevent imbalance. Understanding these boundaries is therefore more critical than identifying a single optimal configuration.

7.2

Search-Based Software Testing

From a search-based software testing (SBST) perspective, game balance adjustment can be framed as a search problem over parameter configurations [4, 30]. Typical SBST techniques focus on efficiently exploring the input space to detect failures with a minimal number of test executions or to achieve high coverage, thereby improving software quality. In contrast, BBExplorer focuses on boundary-oriented exploration under non-deterministic evaluation. Rather than emphasizing failure detection or coverage, it targets parameter configurations near the boundary where acceptable behavior transitions into imbalance. This shift reflects the need to characterize regions of

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

behavioral change rather than locate isolated faults or maximize coverage.

7.3

Automated Game Testing

In game development testing, instead of performing low-level tests such as unit tests, higher-level tests such as playtesting, in which testers play the game, are given priority [31, 35]. Automated approaches to playtesting have been actively studied, though the gap between academic solutions and the actual needs of game developers remains significant [34]. Fuzzing is an automated testing approach in which an initial input is repeatedly mutated to generate new inputs, termed fuzz, that are likely to cause issues [29]. While it is generally used in the security field, in recent years it has been adopted in various fields, such as autonomous driving systems [17] and physics simulation engines [52]. AFL5 is a representative coverage-guided grey-box fuzzer with a proven track record of detecting issues in various software. It measures code coverage for each input and uses this information to guide the generation of subsequent fuzz. Aschermann et al. proposed IJON to enhance AFL and applied it to Super Mario Bros to automate playtesting [3]. They demonstrated that conventional fuzzing based on code coverage performs inadequately for software with complex state transitions, such as video games. Furthermore, they proposed an extended tool named IJON to improve AFL based on annotations. Consequently, they demonstrated the alleviation of state explosion. Also, BiFuzz is a two-stage fuzzer that automates playtesting for open-world games developed using game engines [22]. Both tools focus on playtesting automation using fuzzing. However, IJON aims to transition the game to a specific state, while BiFuzz aims to detect a character stuck issue as a common problem in open-world games. Agent-based and learning-based approaches offer alternatives to fuzzing. Fan et al. proposed a method using machine learning for automated playing in open-world games [13, 51]. This research aims to perform a variety of tasks within the game. Similarly, Chen et al. proposed MIMIC, which uses a large language model to simulate diverse player personality traits for automated game testing [9]. Prasetya et al. developed aplib for programming intelligent test agents [36] and later extended it to smart playtesting with dynamic goal structures [37, 45]. Stahlke et al. proposed PathOS, which simulates player navigation to support level design evaluation [48]. Prasetya et al. also demonstrated automated navigation and exploration in 3D game environments [38]. Other approaches operate at the pixel level or use RL. Zheng et al. proposed Wuji, which combines evolutionary algorithms with deep reinforcement learning for automatic online combat game testing [55]. Liu et al. proposed Inspector, a pixel-based tool that integrates exploration, detection, and investigation for automated game testing [27]. From a game software engineering perspective, Casamayor et al. addressed bug localization in video games using evolving simulations [8]. Beyond individual testing techniques, Kato et al. use open-world games as an extreme case to examine software testing under uncertainty. They identify inexhaustible behavior spaces, non-deterministic execution outcomes, elusive behavioral boundaries, and unstable 5 https://lcamtuf.coredump.cx/afl/

ASE ’26, October 12–16, 2026, Munich, Germany

test oracles as recurring challenges, and argue for selective exploration of behaviorally meaningful regions under cost constraints and explicit treatment of execution variability [23]. Within this broader perspective, BBExplorer focuses on a concrete objective: boundary-oriented game balance regression testing. Whereas prior automated game testing methods primarily focus on reaching specific game states, detecting anomalies, or executing gameplay scenarios, BBExplorer searches for parameter configurations near the boundary between balanced and unbalanced gameplay under a finite simulation budget. It therefore characterizes regions of behavioral transition through repeated stochastic evaluation rather than targeting predefined states or discrete issues.

8

Conclusion

This paper formulated balance regression testing in competitive games as a boundary-discovery problem under stochastic evaluation and limited simulation budgets. We introduced BBExplorer, which combines multi-directional candidate generation, two-stage screening, and adaptive step-size shrinkage. Our experiments show that BBExplorer consistently identifies boundary-near inputs more effectively than baseline methods, and maintains this advantage in higher-dimensional settings and under seed and threshold shifts. These results suggest that structured search combined with adaptive refinement is effective for boundary localization under non-deterministic conditions. Future work includes extending the search strategy and evaluating the approach in more complex parameter spaces and real-world game settings.

Data Availability Statement The replication package, including source code and experimental artifacts, is available at https://doi.org/10.5281/zenodo.21076109.

Acknowledgments We thank Dr. Masanari Kondo and Mr. Sota Nakashima at Kyushu University for their valuable comments. This work was supported by JSPS KAKENHI Grant Numbers JP24K02923, JP23K28065. The authors utilized generative AI tools, including ChatGPT and Claude, to assist with the English translation, text refinement, literature organization, and LaTeX formatting of this paper. All generated content was thoroughly reviewed and edited by the authors, who take full responsibility for the final manuscript.

References [1] P. Ammann and J. Offutt. 2016. Introduction to Software Testing. Cambridge University Press. https://books.google.de/books?id=bQtQDQAAQBAJ [2] Saswat Anand, Edmund K. Burke, Tsong Yueh Chen, John Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn. 2013. An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86, 8 (Aug. 2013), 1978–2001. doi:10.1016/j.jss.2013.02. 061 [3] Cornelius Aschermann, Sergej Schumilo, Ali Abbasi, and Thorsten Holz. 2020. Ijon: Exploring Deep State Spaces via Fuzzing. In 2020 IEEE Symposium on Security and Privacy (SP). 1597–1612. doi:10.1109/SP40000.2020.00117 [4] Antonia Bertolino. 2007. Software Testing Research: Achievements, Challenges, Dreams. In Future of Software Engineering (FOSE ’07). 85–103. doi:10.1109/FOSE. 2007.25 [5] Nathaniel Budijono, Phoebe Goldman, Jack Maloney, Joseph B. Mueller, Phillip Walker, Jack Ladwig, and Richard G. Freedman. 2022. Ludus: An Optimization

ASE ’26, October 12–16, 2026, Munich, Germany

Framework to Balance Auto Battler Cards. Proceedings of the AAAI Conference on Artificial Intelligence 36, 11, 12727–12734. doi:10.1609/aaai.v36i11.21550 [6] Michiel Buijsman, Tiago Reis, Tomofumi Kuzuhara, Brett Hunt, Michael Wagner, and Tianyi Gu. 2024. Global Games Market Report 2024 (Free Version). https://best-of-gaming.be/wp-content/uploads/2024/09/2024_Newzoo_ Global_Games_Market_Report.pdf. Accessed: 2026-1-28. [7] Radu Calinescu, Danny Weyns, Simos Gerasimou, Muhammad Usman Iftikhar, Ibrahim Habli, and Tim Kelly. 2018. Engineering Trustworthy Self-Adaptive Software with Dynamic Assurance Cases. IEEE Transactions on Software Engineering 44, 11 (2018), 1039–1069. doi:10.1109/TSE.2017.2738640 [8] Rodrigo Casamayor, Lorena Arcega, Francisca Pérez, and Carlos Cetina. 2022. Bug localization in game software engineering: evolving simulations to locate bugs in software models of video games. In Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems (Montreal, Quebec, Canada) (MODELS ’22). Association for Computing Machinery, New York, NY, USA, 356–366. doi:10.1145/3550355.3552440 [9] Yifei Chen, Sarra Habchi, and Lili Wei. 2025. MIMIC: Integrating Diverse Personality Traits for Better Game Testing Using Large Language Model. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). 39–51. doi:10.1109/ASE63991.2025.00012 [10] Rogério de Lemos, Holger Giese, Hausi A. Müller, Mary Shaw, Jesper Andersson, Marin Litoiu, Bradley Schmerl, Gabriel Tamura, Norha M. Villegas, Thomas Vogel, Danny Weyns, Luciano Baresi, Basil Becker, Nelly Bencomo, Yuriy Brun, Bojan Cukic, Ron Desmarais, Schahram Dustdar, Gregor Engels, Kurt Geihs, Karl M. Göschka, Alessandra Gorla, Vincenzo Grassi, Paola Inverardi, Gabor Karsai, Jeff Kramer, Antónia Lopes, Jeff Magee, Sam Malek, Serge Mankovskii, Raffaela Mirandola, John Mylopoulos, Oscar Nierstrasz, Mauro Pezzè, Christian Prehofer, Wilhelm Schäfer, Rick Schlichting, Dennis B. Smith, João Pedro Sousa, Ladan Tahvildari, Kenny Wong, and Jochen Wuttke. 2013. Software Engineering for Self-Adaptive Systems: A Second Research Roadmap. In Software Engineering for Self-Adaptive Systems II: International Seminar, Dagstuhl Castle, Germany, October 24-29, 2010 Revised Selected and Invited Papers, Rogério de Lemos, Holger Giese, Hausi A. Müller, and Mary Shaw (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1–32. doi:10.1007/978-3-642-35813-5_1 [11] Louis-Etienne Dubois and Johanna Weststar. 2022. Games-as-a-service: Conflicted identities on the new front-line of video game development. New Media & Society 24, 10 (October 2022), 2332–2353. doi:10.1177/1461444821995815 [12] Saikat Dutta, August Shi, Rutvik Choudhary, Zhekun Zhang, Aryaman Jain, and Sasa Misailovic. 2020. Detecting flaky tests in probabilistic and machine learning applications. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual Event, USA) (ISSTA 2020). Association for Computing Machinery, New York, NY, USA, 211–224. doi:10.1145/3395363. 3397366 [13] Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. 2022. MINEDOJO: building open-ended embodied agents with internet-scale knowledge. In Proceedings of the 36th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS ’22). Curran Associates Inc., Red Hook, NY, USA, Article 1333, 20 pages. [14] P.G. Frankl and E.J. Weyuker. 1988. An applicable family of data flow testing criteria. IEEE Transactions on Software Engineering 14, 10 (1988), 1483–1498. doi:10.1109/32.6194 [15] Gordon Fraser and Andrea Arcuri. 2013. Whole Test Suite Generation. IEEE Transactions on Software Engineering 39, 2 (2013), 276–291. doi:10.1109/TSE.2012. 14 [16] John B. Goodenough and Susan L. Gerhart. 1975. Toward a theory of test data selection. IEEE Transactions on Software Engineering SE-1, 2 (1975), 156–173. doi:10.1109/TSE.1975.6312836 [17] An Guo, Yang Feng, Yizhen Cheng, and Zhenyu Chen. 2024. Semantic-guided fuzzing for virtual testing of autonomous driving systems. J. Syst. Softw. 212, C (June 2024), 12 pages. doi:10.1016/j.jss.2024.112017 [18] Daniel Hernandez, Charles Takashi Toyin Gbadamosi, James Goodman, and James Alfred Walker. 2020. Metagame Autobalancing for Competitive Multiplayer Games. In 2020 IEEE Conference on Games (CoG). 275–282. doi:10.1109/CoG47356. 2020.9231762 [19] Alexander Jaffe, Alex Miller, Erik Andersen, Yun-En Liu, Anna Karlin, and Zoran Popović. 2012. Evaluating Competitive Game Balance with Restricted Play. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 8, 1 (2012), 26–31. doi:10.1609/aiide.v8i1.12513 [20] Kevin Jamieson and Ameet Talwalkar. 2016. Non-stochastic Best Arm Identification and Hyperparameter Optimization. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 51), Arthur Gretton and Christian C. Robert (Eds.). PMLR, Cadiz, Spain, 240–248. https://proceedings.mlr.press/v51/jamieson16.html [21] Hyunjae Kang, Changwoo Suh, and Huy Kang Kim. 2024. Match Experiences Affect Interest: Impacts of Matchmaking and Performance on Churn in a Competitive Game. Heliyon 10, 3 (2024), e24891. doi:10.1016/j.heliyon.2024.e24891

Mukai et al.

[22] Yusaku Kato, Norihiro Yoshida, Erina Makihara, and Katsuro Inoue. 2025. BiFuzz: A Two-Stage Fuzzing Tool for Open-World Video Games. In Proceedings of the 32nd Asia-Pacific Software Engineering Conference (APSEC 2025). 1017–1020. doi:10.1109/APSEC66846.2025.00120 [23] Yusaku Kato, Norihiro Yoshida, Erina Makihara, and Katsuro Inoue. 2026. Software Testing Beyond Closed Worlds: Open-World Games as an Extreme Case. In Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering (Concordia University, Montreal, QC, Canada) (FSE Companion ’26). Association for Computing Machinery, New York, NY, USA, 1356–1360. doi:10.1145/3803437.3805587 [24] Marcelo Lacerda, Fernando Lima Neto, Teresa Ludermir, and Herbert Kuchen. 2023. Out-of-the-box parameter control for evolutionary and swarm-based algorithms with distributed reinforcement learning. Swarm Intelligence 17 (01 2023). doi:10.1007/s11721-022-00222-z [25] Jian-Yu Li, Zhi-Hui Zhan, and Jun Zhang. 2022. Evolutionary Computation for Expensive Optimization: A Survey. Machine Intelligence Research 19 (02 2022), 3–23. doi:10.1007/s11633-022-1317-4 [26] L. Li, K. Jamieson, Giulia DeSalvo, A. Rostamizadeh, and A. Talwalkar. 2018. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research 18 (04 2018), 1–52. [27] Guoqing Liu, Mengzhang Cai, Li Zhao, Tao Qin, Adrian Brown, Jimmy Bischoff, and Tie-Yan Liu. 2022. Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation. In 2022 IEEE Conference on Games (CoG). 237–244. doi:10.1109/CoG51982.2022.9893630 [28] Qingzhou Luo, Farah Hariri, Lamyaa Eloussi, and Darko Marinov. 2014. An empirical analysis of flaky tests. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (Hong Kong, China) (FSE 2014). Association for Computing Machinery, New York, NY, USA, 643–653. doi:10.1145/2635868.2635920 [29] Valentin J.M. Manès, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J. Schwartz, and Maverick Woo. 2021. The Art, Science, and Engineering of Fuzzing: A Survey. IEEE Transactions on Software Engineering 47, 11 (2021), 2312–2331. doi:10.1109/TSE.2019.2946563 [30] Phil McMinn. 2004. Search-based software test data generation: a survey. Softw. Test., Verif. Reliab. 14 (06 2004), 105–156. doi:10.1002/stvr.294 [31] Emerson Murphy-Hill, Thomas Zimmermann, and Nachiappan Nagappan. 2014. Cowboys, ankle sprains, and keepers of quality: how is video game development different from software development?. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 1–11. doi:10.1145/2568225.2568226 [32] Johannes Pfau, Antonios Liapis, Georg Volkmar, Georgios N. Yannakakis, and Rainer Malaka. 2020. Dungeons & Replicants: Automated Game Balancing via Deep Player Behavior Modeling. In 2020 IEEE Conference on Games (CoG). 431–438. doi:10.1109/CoG47356.2020.9231958 [33] Johannes Pfau and Magy Seif El-Nasr. 2024. On Video Game Balancing: Joining Player- and Data-Driven Analytics. ACM Games 2, 3, Article 27 (Aug. 2024), 30 pages. doi:10.1145/3675807 [34] Cristiano Politowski, Yann-Gaël Guéhéneuc, and Fabio Petrillo. 2022. Towards automated video game testing: still a long way to go. In Proceedings of the 6th International ICSE Workshop on Games and Software Engineering: Engineering Fun, Inspiration, and Motivation (Pittsburgh, Pennsylvania) (GAS ’22). Association for Computing Machinery, New York, NY, USA, 37–43. doi:10.1145/3524494.3527627 [35] Cristiano Politowski, Fabio Petrillo, and Yann-Gaël Guéhéneuc. 2021. A Survey of Video Game Testing. In 2021 IEEE/ACM International Conference on Automation of Software Test (AST). 90–99. doi:10.1109/AST52587.2021.00018 [36] I. S. W. B. Prasetya, Mehdi Dastani, Rui Prada, Tanja E. J. Vos, Frank Dignum, and Fitsum Kifetew. 2020. Aplib: Tactical Agents for Testing Computer Games. In Engineering Multi-Agent Systems: 8th International Workshop, EMAS 2020, Auckland, New Zealand, May 8–9, 2020, Revised Selected Papers (Auckland, New Zealand). Springer-Verlag, Berlin, Heidelberg, 21–41. doi:10.1007/978-3-03066534-0_2 [37] I. S. W. B. Prasetya, Mehdi Dastani, Rui Prada, Tanja E. J. Vos, Frank Dignum, Fitsum Kifetew, Guido Mintjes, Samira Shirzadehhajimahmood, and Saba Gholizadeh Ansari. 2026. Programming Smart Playtesting. ACM Trans. Softw. Eng. Methodol. 35, 3, Article 79 (Feb. 2026), 47 pages. doi:10.1145/3742473 [38] I. S. W. B. Prasetya, Maurin Voshol, Tom Tanis, Adam Smits, Bram Smit, Jacco van Mourik, Menno Klunder, Frank Hoogmoed, Stijn Hinlopen, August van Casteren, Jesse van de Berg, Naraenda G.W.Y. Prasetya, Samira Shirzadehhajimahmood, and Saba Gholizadeh Ansari. 2020. Navigation and exploration in 3D-game automated play testing. In Proceedings of the 11th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation (Virtual, USA) (A-TEST 2020). Association for Computing Machinery, New York, NY, USA, 3–9. doi:10.1145/3412452.3423570 [39] Gil Press. 2024. How Many Gamers Are There? (2024 Statistics). What’s The Big Data. Accessed: 2026-1-28. https://whatsthebigdata.com/number-of-gamers/ [40] Riot Games. 2025. Patch 25.24 Notes. https://www.leagueoflegends.com/enus/news/tags/patch-notes/. Accessed: 2026-1-28.

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget

[41] Florian Rupp, Manuel Eberhardinger, and Kai Eckert. 2024. Simulation-Driven Balancing of Competitive Game Levels With Reinforcement Learning. IEEE Transactions on Games 16, 4 (2024), 903–913. doi:10.1109/TG.2024.3399536 [42] Florian Rupp and Kai Eckert. 2024. GEEvo: Game Economy Generation and Balancing with Evolutionary Algorithms. In 2024 IEEE Congress on Evolutionary Computation (CEC). 1–8. doi:10.1109/CEC60901.2024.10612054 [43] Ronnie E. S. Santos, Cleyton V. C. Magalhães, Luiz Fernando Capretz, Jorge S. Correia-Neto, Fabio Q. B. da Silva, and Abdelrahman Saher. 2018. Computer games are serious business and so is their quality: particularities of software testing in game development from the perspective of practitioners. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (Oulu, Finland) (ESEM ’18). Association for Computing Machinery, New York, NY, USA, Article 33, 10 pages. doi:10.1145/3239235.3268923 [44] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. 2016. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 104, 1 (2016), 148–175. doi:10.1109/JPROC.2015.2494218 [45] Samira Shirzadehhajimahmood, I. S. W. B. Prasetya, Frank Dignum, Mehdi Dastani, and Gabriele Keller. 2021. Using an agent-based approach for robust automated testing of computer games. In Proceedings of the 12th International Workshop on Automating TEST Case Design, Selection, and Evaluation (Athens, Greece) (A-TEST 2021). Association for Computing Machinery, New York, NY, USA, 1–8. doi:10.1145/3472672.3473952 [46] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In Advances in Neural Information Processing Systems, F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (Eds.), Vol. 25. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/ 2012/file/05311655a15b75fab86956663e1819cd-Paper.pdf [47] Lev Sorokin, Damir Safin, and Shiva Nejati. 2025. Can search-based testing with pareto optimization effectively cover failure-revealing test inputs? Empirical Software Engineering 30, 1 (2025), 26. doi:10.1007/s10664-024-10564-3 [48] Samantha Stahlke, Atiya Nova, and Pejman Mirza-Babaei. 2020. Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and

ASE ’26, October 12–16, 2026, Munich, Germany

World Design. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Virtual Event, Canada) (CHI PLAY ’20). Association for Computing Machinery, New York, NY, USA, 267–280. doi:10.1145/3410404.3414249 [49] Matej Straka and Martin Schmid. 2025. Artificial Generals Intelligence: Mastering Generals.io with Reinforcement Learning. In Reinforcement Learning and Video Games Workshop @ RLC 2025. https://openreview.net/forum?id=fyKzFiKaRd [50] Penelope Sweetser and Peta Wyeth. 2005. GameFlow: A Model for Evaluating Player Enjoyment in Games. Computers in Entertainment 3 (07 2005), 3. doi:10. 1145/1077246.1077253 [51] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023). doi:10. 48550/arXiv.2305.16291 [52] Dongwei Xiao, Zhibo Liu, and Shuai Wang. 2023. PhyFu: Fuzzing Modern Physics Simulation Engines. In Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (Echternach, Luxembourg) (ASE ’23). IEEE Press, 1579–1591. doi:10.1109/ASE56229.2023.00054 [53] Shin Yoo and Mark Harman. 2012. Regression testing minimization, selection and prioritization: A survey. Software Testing, Verification and Reliability 22 (03 2012). doi:10.1002/stvr.430 [54] Jiongchi Yu, Yuechen Wu, Xiaofei Xie, Wei Le, Lei Ma, Yingfeng Chen, Jingyu Hu, and Fan Zhang. 2023. GameRTS: A Regression Testing Framework for Video Games. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 1393–1404. doi:10.1109/ICSE48619.2023.00122 [55] Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yingfeng Chen, and Changjie Fan. 2019. Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 772–784. doi:10.1109/ASE.2019.00077

Received 2026-03-26; accepted 2026-06-18

Related documents

• Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget#792376
• Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget#788376
• Towards stratified sampling for redistricting plans#1012908
• Distributionally balanced sampling designs via tactical configurations#1076840
• What does a Computer Simulation prove? The case of plant modeling at CIRAD#775163