Adaptive Strategy Generation for Boundary Value Exploration Beyond Numeric Inputs

Adaptive Strategy Generation for Boundary Value Exploration Beyond Numeric Inputs Sabinakhon Akbarovaa,∗, Felix Dobslawb , Robert Feldta a Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden

arXiv:2608.28230v1 [cs.SE] 28 Aug 2026

b Mid Sweden University, Östersund, Sweden

Abstract Software behavior often changes abruptly at boundaries between input regions, and these transitions are known to be fault-prone. Boundary Value Exploration (BVE) automates boundary discovery by searching for pairs of similar inputs that nevertheless trigger different program behaviors. Existing automated BVE techniques rely on mutation operators hand-engineered for each input type, or even for each function under test, which has confined their use to numeric inputs. We present ABEX, an agentic LLM-based framework that replaces operator engineering with adaptive strategy generation: specialized LLM agents propose, select, and execute boundary-exploration strategies, guided by execution feedback and a quality-diversity (QD) archive. Because strategies are expressed in natural language, they can encode both type-level and function-specific knowledge, and effective strategies can even be stored and reused. We evaluate ABEX in a black-box setting on 20 functions with numeric, string, array, and mixed inputs. On numeric functions, ABEX outperforms a state-of-the-art QD method on 10 of 11 functions, with average QD-scores 11.7x higher. On non-numeric functions, addressed here for the first time in automated black-box BVE, ABEX discovers domain-aligned boundary behaviors for all subjects. Mutation testing shows the discovered boundaries are fault-revealing: with equally sized test suites, ABEX reaches an average mutation score of 86.2% versus 61.9% for the QD baseline, and kills nine times as many hard-to-detect stubborn mutants. An ablation study identifies adaptive strategy generation as the primary driver of these gains. Keywords: Boundary value exploration, automated software testing, large language models, quality-diversity optimization, AI4SE

1. Introduction Software systems often change behavior at the borders between input regions. These transitions can be especially fault-prone, which makes boundary-focused testing an efficient way to find defects with relatively few test cases [1, 2, 3]. Boundary Value Exploration (BVE) makes this idea concrete by casting boundary discovery as a search for pairs of nearby inputs that nevertheless trigger different program behaviors, thereby exposing behavioral boundaries [4]. Recent work has started to automate BVE. AutoBVA [5] searches for boundary-inducing input pairs by optimizing the program derivative (PD) [6], a metric that rewards pairs of inputs that are close in the input space but produce different outputs. This works well for sharp transitions, but optimizing a single metric can pull the search into a narrow part of the space and miss boundary areas that score lower yet still matter for testing. SETBVE [7] addresses this limitation with a quality-diversity (QD) approach based on MAP-Elites [8], maintaining an archive of boundary candidates that balances boundary strength against behavioral diversity. Despite this progress, automated BVE approaches still face a fundamental limitation: they depend on mutation operators tailored to the input data types of the function under test (FUT) and, for effective exploration, often to the FUT’s specific semantics. Designing generic operators is straightforward for numeric inputs, where boundary candidates ∗ Corresponding Author

Email addresses: sabina.akbarova@chalmers.se (Sabinakhon Akbarova), felix.dobslaw@miun.se (Felix Dobslaw), robert.feldt@chalmers.se (Robert Feldt)

can be generated by incrementing, decrementing, or sampling around numeric thresholds. However, this becomes harder for other and more complex data types such as strings, arrays, or mixed inputs. For example, consider a function that validates email addresses. A generic string mutation operator may randomly insert, delete, or replace characters, but such mutations are unlikely to systematically explore the meaningful boundary cases of the email format. Inputs such as alice@example.com, aliceexample.com, alice@@example.com, and alice@example..com differ by small syntactic changes, yet they exercise distinct boundary conditions related to the presence and position of @, dots, local-part structure, and domain structure. Generating such cases requires knowledge of the expected input grammar and of the validation rules implemented by the FUT. Similar challenges arise for file paths, URLs, identifiers, JSON objects, or other structured inputs whose boundary behavior depends on domain-specific constraints. Each new type, argument combination, or function therefore requires bespoke operator engineering, which is labor-intensive, error-prone, and difficult to scale. As a result, automated BVE has remained largely confined to numeric domains with generic operators, even though most real software operates on richer input structures whose boundary behavior depends on function-specific logic. This limitation motivates our work, which aims to reduce the need for manually engineered, type- and function-specific mutation operators. We present ABEX, an agentic LLM-based framework designed to remove this bottleneck. The central concept in ABEX is a boundary-exploration strategy: a reusable, natural-language procedure that instructs the system how to generate candidate input pairs for a given FUT. A strategy is not a single test case or a fixed low-level mutation operator; rather, it specifies an exploration policy, such as which input structures to vary, whether to generate new pairs from scratch or mutate existing boundary candidates. For example, for an email-validation function, a strategy may instruct the system to vary the position and number of @ symbols, alter dots in the local part or domain, and pair syntactically similar valid and invalid addresses to expose format-validity boundaries. Building on work on autonomous testing agents [9, 10] and Automated Design of Agentic Systems (ADAS) [11], ABEX combines a QD archive and the program derivative metric with specialized agents that propose, refine, select, and execute such strategies. Because these strategies are expressed in natural language, they can capture both type-level and FUT-specific knowledge derived from docstrings, specifications, or other semantic information without requiring changes to the framework. The resulting strategies are human-readable and, in principle, reusable: effective strategies discovered for one function or input type can be stored and applied to others, allowing the framework to accumulate testing knowledge over time. Although the use of LLMs leads to higher per-iteration cost than optimized numeric search, ABEX compensates by finding high-quality boundaries early and by working on input types that existing methods cannot handle. We evaluate ABEX on 20 FUTs with integer, string, array, and mixed inputs. On numeric inputs, ABEX outperforms both the QD-based baseline SETBVE [7] and a single-prompt LLM baseline on 10 of the 11 evaluated functions. Under the same budget as SETBVE, 100 iterations with 10 candidates generated per iteration, with a total of 1000 generated candidate input pairs, ABEX achieves an average QD-score 11.7× higher. The single-prompt baseline serves as a check on whether simply asking an LLM for boundary candidates, without iteration, feedback, or agents, is already sufficient; following prior work [12], it generates 50 candidate pairs in a single call, and ABEX achieves an average QD-score 2.8× higher (Section 4.2 details both budgets). For non-numeric inputs, which to the best of our knowledge are explored here for the first time in automated black-box BVE, ABEX discovers meaningful, domain-aligned boundary behaviors across all tested functions. Mutation testing further shows that the boundary candidates discovered by ABEX are effective as fault-detecting test cases: under equally sized test suites, ABEX achieves an average mutation score of 86.2% on numeric functions, compared to 61.9% for SETBVE, and kills roughly nine times as many stubborn mutants. For non-numeric functions, ABEX-generated suites reach an average mutation score of 76.3%. An ablation study identifies adaptive strategy generation as the primary driver of these gains: on a six-FUT subset, the full framework achieves an average per-FUT QD-score improvement of 6.8× over a variant that iterates a single fixed strategy. The contributions of this paper are: • ABEX, an adaptive boundary exploration framework that autonomously generates and accumulates reusable exploration strategies, achieving 2.8× and 11.7× higher QD-scores than single-prompt LLM and QD-based baselines, respectively, on numeric inputs. • The first automated BVE results in a black-box setting for non-numeric inputs: ABEX discovers domain-aligned boundary behaviors across all 10 string, array, and mixed-input functions. 2

• Mutation-testing evidence that the discovered boundary candidates are effective at detecting seeded faults: ABEX reaches an average mutation score of 86.2% versus 61.9% for the QD-based baseline on numeric FUTs, and kills roughly nine times as many hard-to-detect stubborn mutants. • An open replication package1 containing the ABEX framework and evaluation dataset. Section 2 presents background and related work. Section 3 describes the ABEX architecture with a running example. Section 4 outlines the experimental setup. Section 5 reports results, followed by discussion and threats to validity in Section 6. Section 7 concludes. 2. Background and Related Work This section introduces key concepts behind boundary discovery and reviews related work on quality-diversity optimization and LLM-based testing. 2.1. Boundary Discovery Boundary Value Analysis (BVA) and Boundary Value Testing (BVT) focus on inputs near transitions between behavioral domains (equivalence partitions), where faults are more likely to occur [2, 3]. Traditionally, identifying such boundaries has been largely manual. Early work by White and Cohen proposed domain testing, which identifies boundaries between mutually exclusive subdomains to detect control-flow errors [1]. Subsequent efforts aimed to reduce the manual effort involved in boundary identification. For example, Jeng et al. [13] introduced a semi-automated approach that combines dynamic search with algebraic manipulation of boundary conditions. Fully automated approaches have also been proposed, but they typically rely on explicit program specifications or access to internal program structure. Pandita et al.[14] presented a white-box technique that instruments code to identify boundary values near decision points, while Zhang et al.[15] leveraged symbolic execution to extract boundary conditions and generate test cases through constrained combinatorial testing. Although effective, these methods depend on well-defined partitions or specifications, which limits their applicability when such information is incomplete, ambiguous, or unavailable. Boundary Value Exploration (BVE) was introduced to address this limitation by treating boundary discovery as a systematic search problem [4]. AutoBVA [5], the first automated black-box framework evaluated on numeric-input functions, uses the concept of the program derivative (PD) [6] as a search objective. The PD measures how sensitive program behavior is to changes in input by computing the ratio of output distance to input distance2 PD(i1 , i2 ) = ddoi(o(i11,i,o22)) . Input pairs with PD > 0 are considered boundary candidates, and larger PD values indicate sharper behavioral transitions. While effective at identifying strong boundaries, optimizing solely for PD can bias the search toward high-contrast regions and overlook other relevant boundary areas with lower PD values due to program-specific characteristics [7]. Moreover, PD depends on the chosen distance metrics, and no single metric fully captures boundariness across domains. Consequently, PD values are not directly comparable across the entire input space, and smaller PD values in some regions may be more informative than larger ones in others. Boundary testing beyond numeric inputs has received some attention. Jain et al. [16] manually apply classical BVA schemes to character inputs by treating letters as ordered values within a known range, without automation or boundary discovery. Zhao et al. [17] automatically generate test points for string predicate borders, but their technique is white-box. Neither approach discovers behavioral boundaries of a black-box function whose partitions are unknown. Automated BVE for non-numeric inputs in a black-box setting has, to our knowledge, not been addressed. 1 https://zenodo.org/records/21354853 2 Input and output distances calculation depend on datatype (e.g., Euclidean for numeric values and Jaccard for strings).

3

2.2. Quality-Diversity Optimization Quality-Diversity (QD) algorithms aim to discover diverse sets of high-performing solutions rather than a single optimum. A widely used QD algorithm is MAP-Elites, introduced by Mouret and Clune [8]. MAP-Elites maintains a grid-based archive of elite solutions organized by behavioral descriptors that represent different regions of the behavioral space. Each cell stores the best solution discovered for that region. QD methods have been applied to software testing across several domains. Marculescu et al. [18] showed that exploration-focused algorithms including MAP-Elites cover a broader portion of the behavior space than objectivebased approaches, with over 80% of their outputs not found by the objective-based method, demonstrating their suitability for high-dimensional behavioral spaces under limited resources. Feldt and Poulding [19] further investigated diversity-driven test input generation inspired by Novelty Search [20] and MAP-Elites. Xiang et al. [21] applied MAP-Elites to test suite generation for Software Product Lines. QD algorithms have also been applied to deep learning system testing: Riccio and Tonella [22] developed DeepJanus to map the behavioral frontier of deep learning systems using input pairs with similar features but differing behaviors, and Zohdinasab et al. [23] developed DeepHyperion, which uses Illumination Search to explore interpretable feature spaces of deep learning inputs. More recently, Weißl et al. [24] proposed Mimicry, which shows that targeted boundary exploration outperforms the untargeted perturbations used by earlier deep learning testing techniques. While some of these tools operate in a black-box manner, they still assume specific input/output types and domain-specific quality signals [22], limiting their generalization to traditional software with heterogeneous input types. Most closely related to our work, Akbarova et al. proposed SETBVE [7], a QD-based framework for automated BVE. Their results show that balancing solution quality with behavioral diversity enables the discovery of a wider range of edge-case behaviors than approaches that optimize for quality alone. SETBVE guides the search using a MAP-Elites archive defined by four behavioral descriptors derived from the input and output characteristics of a FUT. ABEX builds on this formulation by adopting the same behavioral descriptors. 2.3. Software Testing with LLMs Large Language Models (LLMs) have recently demonstrated strong capabilities in software engineering tasks, including automated testing [25, 26, 27]. A recent survey by Wang et al. [28] reviews over 100 studies applying LLMs to software testing, highlighting their effectiveness across tasks such as test case generation, debugging, and program repair. These works show that LLMs are particularly well-suited for generating test inputs by leveraging their understanding of programming patterns and semantics [29, 30]. However, most existing approaches rely on single-shot generation or fixed prompting schemes without incorporating execution feedback [12, 31], which might lead to redundant, low-quality, or invalid test cases [32]. While some recent work explores iterative or feedback-driven testing [33], these approaches typically operate within predefined generation patterns and do not adapt how the input space is explored over time. Agent-based systems have recently emerged as a promising paradigm for structuring LLM-driven workflows. In software testing, DroidAgent introduced a multi-agent architecture for Android GUI testing, where agents maintain system knowledge and generate high-level exploration goals [10], building on earlier work on LLM-based autonomous testing agents [9]. Inspired by these advances, ABEX integrates LLM-driven strategy generation within a quality-diversity search framework, enabling adaptive boundary discovery guided by dynamically generated exploration strategies. 3. Framework ABEX follows a three-layer agent architecture inspired by hierarchical frameworks in management science that separate strategic, managerial (tactical), and operational levels [34] (see Figure 1). Strategic decisions in (1) determine how the input space is explored. Tactical decisions in (2) materialize as the construction or selection of an input space exploration strategy. Finally, the operational layer (3) generates candidate inputs according to the chosen strategy and executes them on the FUT. This separation supports efficient resource allocation and allows different LLM configurations to be assigned to each layer [35]: strategic decisions are infrequent but high-impact, while operational actions are frequent and constrained. ABEX iterates in a loop until the search budget is exhausted. 4

1

Coordinator select

repeat request

2

Strategy Library

Strategy Pool

Ideator Strategy Generator

archive data flow feedback 1 strategic 2 tactical 3 operational

3

Explorer

Figure 1: High-level overview of ABEX architecture. The Coordinator (strategic layer) decides whether to repeat, select, or request a strategy; the Ideator and Strategy Generator (tactical layer) create new strategies; the Explorer (operational layer) executes strategies and updates the MAP-Elites archive.

The following subsections describe the role of each component and walk through one iteration using a running example on a date FUT, taking three integer inputs (year, month, day) and returning an ISO date string 3 . 3.1. Coordinator The Coordinator implements the strategic layer of ABEX and directs the overall exploration process. Exploration is driven by strategies — textual, step-by-step instructions for generating candidate input pairs. During the search, available strategies are maintained in a strategy pool, which is initialised from a strategy library: an offline repository of reusable strategies. In this study, the library contains a single pre-seeded baseline strategy that instructs the system to “generate diverse boundary value input pairs for the given function”. This baseline receives the FUT signature and docstring — a one- to two-sentence natural-language description of the function’s purpose — but specifies no structured procedure. Since it receives the signature and docstring, it is formally a context-aware generation strategy (see Section 3.2 for more details), but we track it separately in our analysis (Section 5.3) because it is pre-seeded rather than produced by the tactical layer during the search. The first iteration executes this baseline strategy directly. The Coordinator is invoked only after the active strategy’s stopping condition is met (Section 3.3). Once activated, the Coordinator decides among three exploration directions. First, it may continue using the most recently executed strategy if it appears promising or insufficiently explored. Second, it may switch to another strategy from the strategy pool, selecting the specific strategy to activate. Third, it may request the creation of a new strategy by specifying the desired strategy mode, type, and the search gap that the strategy should address (Section 3.2). To support this decision-making process, the Coordinator receives contextual information about the search. This includes the current iteration number, allowing it to reason about the stage of exploration, as well as information about the FUT, such as its signature and docstring. The Coordinator also has access to tools that summarize search progress, such as recent archive updates and stagnation indicators. Stagnation occurs when no new archive cells are discovered and no improvements are observed in existing cells for several consecutive iterations. In addition, the Coordinator can inspect the pool of available strategies, including a short description of each strategy and its usage statistics. These statistics include both total success rate (new boundary candidates and quality improvements over all attempts) and a time-decayed success rate that weights recent performance more heavily. Comparing these two metrics enables the Coordinator to assess not only a strategy’s overall effectiveness but also whether its performance is improving or declining. This adaptive decision process allows ABEX to dynamically adjust its search behavior as exploration progresses, either by revisiting strategies that may still be promising or by requesting new strategies when experiencing stagnation. 3 Prompt examples are truncated for readability and complete prompts are available in the replication package.

5

Coordinator [System] Choose the best next action to maximize the discovery of diverse boundary test cases: strategy, repeat current strategy or select another available strategy. [...truncated...] [Input] Iteration progress: Function signature:

Iteration 12 of 100 date(year: int, month:

int, day:

request new

int) -> str

[Coordinator calls tool: stagnation_check] Progress slowing - 3 consecutive rounds with no progress... [Coordinator calls tool: strategies] Strategies sorted by recent success rate: --- ID: strategy1 (CURRENT) --Summary: Generate pairs testing integer boundary values like 0, -1, MAX_INT. Mode: generation, Type: generic Used in 8 of 11 past rounds. Last used 1 rounds ago. Success rate: 0.31, Recent success rate: 0.23 --- ID: strategy2 --Summary: Test month-day combinations at month boundaries (30/31 days, February). Mode: generation, Type: context-aware Used in 3 of 11 past rounds. Last used 4 rounds ago. Success rate: 0.22, Recent success rate: 0.2 [Output] option: request strategy_mode: generation strategy_type: context-aware search_gap: target year boundary transitions affecting leap years.

3.2. Ideator and Strategy Generator When a new strategy is required, the tactical layer generates one through the Ideator and the Strategy Generator. The Coordinator specifies three elements for the new strategy: the strategy mode, its type, and the search gap that the strategy should target. Definition 1 (Strategy). A strategy s is a tuple s = (m, t, P), where m ∈ {generation, mutation} is the strategy mode, t ∈ {generic, context-aware} is the strategy type, and P = ⟨p1 , p2 , . . . , pk ⟩ is an ordered sequence of natural-language steps that, given a function under test f and optionally a sampled input pair a = (i1 , i2 ) from the archive A, produces a set of candidate input pairs {(i1j , i2j )}Nj=1 . A generation strategy requires no sampled input pair, i.e., a = none, while a mutation strategy requires a ∈ A. The mode m determines how input pairs are obtained. Generation strategies produce new input pairs from scratch using the available FUT information. Mutation strategies instead operate on existing archive entries: they first sample4 an input pair from the archive (each archive cell stores one pair), and then apply small modifications to explore nearby regions of the input space. The type t determines what information the generation strategy may use; mutation strategies never receive the function signature or docstring and are therefore always of type generic. Generic generation strategies only know the number and data types of the input arguments, and therefore rely on general boundary heuristics. In contrast, context-aware generation strategies additionally have access to the function signature and its docstring. This enables the strategy to exploit domain-specific hints about the input space. For example, if the LLM only knows that a function accepts three integers, it may explore general numeric boundaries such as INT_MAX or INT_MIN. However, if it also knows that the function is date(year, month, day), 4 An archive cell is sampled uniformly at random.

6

it may instead focus on domain-specific transitions such as month changes or leap years. Both perspectives are useful for discovering different types of boundary behaviors. The Ideator bridges the Coordinator’s search gap and the executable strategy by producing a tactical idea: a one- to two-sentence conceptual direction that is more specific than a search gap but not yet a full strategy. Separating idea generation from strategy construction serves two purposes. First, it identifies a promising strategy direction, or, when a suitable direction already exists, selects an existing strategy rather than producing a redundant one. Second, it filters candidate directions that are likely to be low-yield before they are expanded into full strategies. The first purpose requires comparing candidate directions by the boundary behavior they are expected to explore, not only by the wording of their descriptions. Lexical or embedding-based similarity checks can identify strategies that are close in phrasing, but they may miss functional redundancy between differently worded directions. For example, "integer overflow boundaries" and "values near MAX_INT" may appear different as text, yet both target essentially the same numeric boundary. At the same time, textually similar directions can still be useful if they target different boundary conditions: "test empty string input" and "test single-character input" are close in wording but exercise different string-length boundaries. The Ideator therefore evaluates whether a candidate direction addresses the Coordinator’s requested search gap and adds functional novelty relative to the existing strategy pool, rather than merely checking whether its wording differs. The second purpose concerns the expected usefulness of a candidate direction before it is expanded into a full strategy. A direction may be novel but still low-yield if it is too broad to guide concrete input-pair generation, too narrow to produce diverse candidates, inconsistent with the requested generation or mutation mode, or likely to spend the search budget on invalid or behaviorally uninformative inputs. To filter such cases, the Ideator is instructed, within a single LLM call, to consider two to three candidate directions, compare them against the existing strategy pool in terms of search-gap relevance, functional novelty, and expected yield, and return the most promising one. Once an idea is selected, the Strategy Generator expands it into an executable exploration procedure by converting it into concrete steps describing how input pairs should be generated. The Strategy Generator also specifies the required output format and introduces variation within the steps so that repeated executions of the strategy produce diverse boundary candidates. The resulting strategy is then passed to the operational layer for execution. Ideator [System] Return one worthwhile new idea that is different from existing strategies and likely to discover new diverse boundary test cases. If none is worthwhile, return a fallback strategy. [...truncated...] [Input] existing_strategies: --- ID: strategy1 --Mode: generation, Type: generic Steps: 1. Pick a boundary value (0, 1, -1, MAX_INT, MIN_INT). 2. Create i1 at the boundary. 3. Create i2 by adding or subtracting 1. ... --- ID: strategy2 --Mode: generation, Type: context-aware Steps: 1. Select a month with 30 or 31 days. 2. Set day to the last valid day. 3. Create i2 by incrementing day by 1. ... function_signature: date(year: int, month: int, day: int) -> str requested_strategy_mode: generation requested_strategy_type: context-aware search_gap: target year boundary transitions affecting leap years. [Output] Idea: Focus on year values where leap year rules change: 28/29 transitions across these boundaries.

years divisible by 4, 100, and 400, testing February

7

Strategy Generator [System] Turn the given idea description into a concise strategy for generating boundary test cases. [...truncated...] [Input] requested_strategy_mode: generation requested_strategy_type: context-aware function_signature: date(year: int, month: int, day: int) -> str search_gap: Target year boundary transitions affecting leap years. idea: Focus on year values where leap year rules change: years divisible by 4, 100, and 400, testing February 28/29 transitions across these boundaries. [Output] Strategy: 1. Pick a leap year boundary: (a) divisible by 4, (b) divisible by 100, (c) divisible by 400. 2. Set month = 2 and day = 28 or 29. 3. For i1, use a year just before the boundary; for i2, use a year at or after. 4. Vary across boundary types and day values. 5. Output as {"i1": [year, month, day], "i2": [year, month, day]}

3.3. Explorer The Explorer implements the operational layer of ABEX and executes the strategies produced by the tactical layer; the name follows the similar component in SETBVE with a comparable role [7]. Given a textual strategy, the Explorer invokes the LLM to generate N input pairs per call5 . Each pair represents a potential boundary candidate and is evaluated using a MAP-Elites style archive. The archive follows a QD formulation similar to SETBVE. It is organized as a four-dimensional grid in which each occupied cell corresponds to an exact integer coordinate in the behavioral descriptor space. These descriptors capture observable program behaviors induced by an input pair. In this work, we adopt four descriptors [7]: the number of exceptions triggered by the pair, the output length difference, the total length of the stringified input pair, and the variance of the input lengths. We use the same four descriptors for all FUTs because they are type-agnostic: they depend on exceptions and stringified input/output properties rather than datatype-specific structure. Using this common descriptor space for both numeric and non-numeric FUTs also keeps archive-level measurements comparable across FUTs. As in SETBVE, ABEX applies no additional binning to these descriptors: each distinct integer descriptor vector defines a separate archive cell. For example, two candidates mapped to (0, 3, 12, 2) and (0, 4, 12, 2) occupy different cells rather than being coarsened into the same interval. We avoid such coarsening because small changes in output length, input length, or exception behavior can correspond to meaningful boundary differences. The archive is therefore implemented as a dynamically growing map: cells are created only when new descriptor coordinates are observed. Descriptor values are integer-valued, and their observed ranges are FUT-dependent: the exception-count descriptor takes values in {0, 1, 2}, while output-length difference, total input length, and rounded input-length variance are bounded below by 0 and, in principle, unbounded above. In practice, however, their observed ranges are limited by the generated inputs and the outputs produced by the FUT during a run. Each generated or mutated input pair is assigned to a corresponding archive cell based on the behavioral descriptors by executing the FUT on each input in the pair. Consider the pair i1 = "a@b.co" and i2 = "a@b.c" for the validateEmail FUT, which yields the outputs "Valid" and "ValueError: Invalid email format." respectively. This is a boundary where removing one character from the domain extension (.co → .c) makes the email invalid. The behavioral descriptors are computed as follows: (1) one output raises an exception giving an exception-count descriptor of 1; (2) the output lengths differ by |5 − 33| = 28; (3) the input lengths sum to 6 + 5 = 11; and (4) the variance of input lengths is ⌊0.5⌋ = 0. The pair is therefore assigned to the cell at coordinates (1, 28, 11, 0). A different boundary such as "user@example.com" and "userAexample.com", where replacing the @ symbol with a letter removes the required delimiter, maps to a different cell at (1, 28, 32, 0) due to the longer total input length (16 + 16 = 32). 5 In this study, N = 10.

8

The quality of the input pair is then evaluated using the program derivative (PD), computed following SETBVE. Output distances are computed using Jaccard distance on stringified outputs, which both aligns with SETBVE and makes the metric applicable across all output types in our study. For inputs, we use Euclidean distance when all arguments are numeric, and an average of per-element distances otherwise, using Jaccard distance for strings and arrays (stringified), and equality for numeric values (0 if identical, 1 otherwise)6 . The archive is updated following the MAP-Elites policy: if the cell is empty the input pair is inserted, if the cell is occupied but the new pair has higher quality (PD) it replaces the input pair occupying the cell, otherwise the archive remains unchanged. The Explorer repeatedly executes the same strategy until a stopping condition is reached, either after 10 iterations or after three consecutive iterations without archive updates, meaning that no new cells are discovered and no existing cells are improved. The iteration limit encourages exploration diversity under a limited search budget by allowing other strategies to be attempted. If a strategy remains effective, the Coordinator may later select it again. To further encourage diversity, the Explorer is instructed to avoid generating input pairs that already exist in the archive. The prompt therefore includes a sample of previously discovered pairs together with their input-related properties, consisting of both the most recently added pairs and randomly selected ones from the archive7 . Once a stopping condition is met, control returns to the Coordinator together with the updated archive statistics, which are used to determine the next exploration step. Explorer [System] Generate 10 diverse input pairs (i1 and i2) by following the given strategy. [Input] function_signature:

date(year:

int, month:

int, day:

[...truncated...]

int) -> str

Strategy: 1. Pick a leap year boundary: (a) divisible by 4, (b) divisible by 100, (c) divisible by 400. 2. Set month = 2 and day = 28 or 29. 3. For i1, use a year just before the boundary; for i2, use a year at or after. 4. Vary across boundary types and day values. 5. Output as {"i1": [year, month, day], "i2": [year, month, day]} Avoid existing input pairs: i1: [2023, 1, 31], i2: [2023, 2, 1] (small length, similar magnitudes) i1: [2023, 12, 31], i2: [2024, 1, 1] (small length, similar magnitudes)... [Output] {"i1": [1999, 2, 28], "i2": {"i1": [1900, 2, 28], "i2":

[2000, 2, 28]} [1900, 2, 29]} ...

4. Evaluation This section presents the experimental setup used to evaluate ABEX. 4.1. Research Questions Our evaluation addresses the following research questions. RQ1: How effective are LLM-generated strategies at discovering boundaries for functions with numeric inputs? This question evaluates whether LLM-generated strategies can discover boundary behaviors for FUTs with integer inputs. Integer FUTs enable direct comparison with the existing QD-based automated BVE framework. Effectiveness is measured using both the quality of discovered boundary candidates and the diversity of behaviors represented in the archive. RQ2: Can ABEX discover boundaries for functions with non-numeric inputs, and what types of boundaries are identified? This question investigates whether ABEX generalizes beyond numeric domains. We evaluate FUTs with 6We use bounded [0, 1] equality for numeric elements in mixed inputs to match Jaccard’s range; unbounded Euclidean would otherwise dominate the average. 7We include 10 recent and 30 randomly sampled input pairs; including the full archive would make the prompt excessively long.

9

string, array and mixed-type inputs and analyze the types of behavioral transitions discovered (e.g., domain-specific output properties). RQ3: How do different exploration strategies influence boundary discovery across input data types? This question analyzes the relationship between strategy characteristics and exploration performance. Specifically, we examine the usage frequency and effectiveness of different strategy modes and types across three FUT groups: numeric, string, and array or mixed inputs. RQ4: How effective are boundary candidates discovered by ABEX at detecting faults compared to SETBVE? This question evaluates fault-detection effectiveness using mutation testing. For FUTs with numeric inputs, we compare mutation scores and the number of uniquely killed mutants between test suites generated from ABEX- and SETBVEdiscovered boundary candidates. For non-numeric FUTs, where SETBVE is not applicable, we report mutation testing results for ABEX without a baseline. RQ5: How do ABEX’s components and underlying LLM affect its ability to discover boundary candidates? This question evaluates the contribution of individual ABEX components through an ablation study and examines how the choice of closed (proprietary) or open-weight LLM affects boundary candidate generation capability. 4.2. Experimental Setup Subjects. Table 1 lists the 20 FUTs used in our evaluation. The set includes 10 functions with integer inputs, 5 with string inputs, 3 with array inputs, and 2 with mixed input types (array–integer and integer–boolean). The subject set was chosen to provide a controlled benchmark that still exercises different boundary-search scenarios. In particular, the FUTs vary not only in input datatype and arity, but also in the form of the observable behavior that defines boundaries: the benchmark includes categorical classifiers, format checkers, string transformation functions, array transformation functions, and search-like functions with input preconditions. One function, complexCheck, is evaluated in two variants: one with a one- or two-sentence docstring consistent with the other FUTs and one with a more detailed description (complexCheck_full). This function is commonly used in white-box testing due to its multiple branching conditions with arbitrary thresholds; the richer description allows us to assess whether additional semantic information improves boundary discovery in our black-box setting. We prioritize functions from prior BVE and testing literature where available. Since automated BVE in a black-box setting has not been explored for string inputs and only minimally for arrays, we include additional FUTs to cover a broader range of program logic. All FUTs were implemented in Python, closely following the original implementations where available. Metrics. Since ABEX maintains a MAP-Elites archive, our primary metric follows the QD paradigm. We report the QD-score [40], which measures both the quality and diversity of the archive population. Diversity is represented by the number of occupied archive cells c, while quality is given by the fitness values stored in these cells. The QD-score P is defined as ci=1 Qi , where Qi is the fitness value stored in cell i. In our setting, fitness corresponds to the program derivative (PD) of the input pair stored in the cell. Because the QD-score does not explicitly indicate the number of discovered boundary candidates, we additionally report the number of archive cells that contain boundary candidates (i.e., archive cells with PD > 0), referred to as the BC count. Baselines. For RQ1, we compare ABEX with two baselines: SETBVE and a single-prompt LLM. All experiments for RQ1 and RQ2 are repeated 10 times per configuration to account for stochastic variation. SETBVE’s original implementation evaluates runs using fixed wall-clock budgets (30 seconds or 10 minutes). Because LLM-based exploration incurs higher computation time per iteration, we instead compare methods using a fixed iteration budget8 . Specifically, SETBVE is executed for 1000 iterations, producing 1000 candidate input pairs. The corresponding ABEX configuration is run for 100 iterations, with each Explorer iteration generating 10 input pairs, also yielding 1000 candidate input pairs. We use the default SETBVE configuration, where initial sampling occupies 10% of the total budget and mutation selects archive entries uniformly at random. ABEX follows the same uniform mutation policy. 8We use the open-source implementation and modify it to support an iteration-based budget.

10

Table 1: Subject programs used in evaluation. Name

Description

bmi [5, 7] bytecount [5, 7] calDate [15] circle [7] complexCheck† [15, 36] date [5, 7] english [37] findMiddle [15] nextDate [15] tritype [38, 15] passwordStrength printTokens replace stringPalindrome [16] validateEmail [39] insertionSort normalize [7] maxLexString [17] binarySearch tcas [15]

Classify BMI category Format bytes with SI prefix Convert to Julian day number Point inside/outside circle Nested conditional branches Format as ISO date string Determine pass/fail status Return median of three Compute next calendar date Classify triangle type Assign strength level Tokens classification Match pattern and substitute Check palindrome or not Email format validity Insertion sort algorithm Min-max array scale to [0, 1] Max lexicographical string Search for target value Collision avoidance system

Number of arguments

Input type

2 1 3 2 3 3 2 3 3 3 1 1 3 1 1 1 1 1 2 12

Integer Integer Integer Integer Integer Integer Integer Integer Integer Integer String String String String String Array[int] Array[int] Array[str] Mixed (arr[int], int) Mixed (int, bool)

† The FUT has two specification versions (docstrings): a short version consistent with other FUTs and a more detailed one.

The single-prompt baseline is a check on whether simply asking an LLM to generate boundary candidates, without iteration, feedback, or agents, is already sufficient. It uses the same LLM model and baseline strategy as the Explorer in ABEX, but without archive feedback or adaptive strategy selection. The baseline receives the function signature and docstring, and the model is prompted once to generate a diversified set of 50 boundary input pairs (i1 , i2 ), following the single-prompt setting of prior work [12]. The ExplorerOnly ablation (Section 5.5) complements this baseline: it iterates the same fixed strategy with archive feedback under a full search budget, which allows us to separate the effect of budget from the effect of adaptive strategy generation. Large Language Models. We employ a heterogeneous model configuration across components [41], assigning models based on cognitive demand and call frequency. The frequently invoked Explorer uses gpt-5-mini for cost-efficient instruction following, while the Coordinator and Ideator use reasoning-enhanced gpt-5.1 for strategic decisions and idea generation. The Strategy Generator uses gpt-5-mini to convert ideas into executable strategies. Sampling temperatures are also role-dependent: the Coordinator and Strategy Generator use a moderate temperature (0.5) for moderately stable decisions, whereas the Ideator and Explorer use higher temperatures (0.8) to promote diversity in ideas and generated input pairs. This temperature selection is consistent with the findings of Lemieux et al. [26], which show that higher sampling temperatures have a consistently positive effect on exploration effectiveness. Section 5.5 also evaluates two ABEX configurations based on the open-weight models Gemma-4 and Qwen-3.5. ABEX is implemented in Python using the DSPy framework [42], which enables modular LLM pipelines through declarative task specifications (signatures). DSPy replaces manually engineered prompts with a prompt-programming approach that automatically compiles and optimizes prompts for each pipeline component, simplifying the development of multi-step LLM systems. All experiments were conducted on a MacBook Pro equipped with an Apple M2 Pro chip, 16GB RAM, running macOS 26.3. Mutation Analysis. To answer RQ4, we evaluate whether the boundary candidates discovered by ABEX and SETBVE are effective as test cases for detecting faults. We do this using mutation testing. Mutation testing creates faulty versions of the program under test, called mutants, by applying small syntactic changes to the original program. A test case kills a mutant if it produces a different output on the mutant than on the original program. Otherwise, the mutant remains 11

live. We summarize test-suite effectiveness using the mutation score (MS), defined as the ratio of killed mutants to the total number of mutants. For each FUT, mutants were generated once using mutmut [43], a Python mutation testing tool that applies mutation operators such as arithmetic, relational, and logical operator replacements. The same mutant set was then reused for all runs and compared frameworks. This ensures that the mutation results are based on the same faults, so differences in mutation score reflect differences in the generated test suites rather than differences in the mutant sets. The mutation analysis follows the structure of the boundary-exploration experiment. Each framework is run 10 times per FUT, and each run produces one archive of discovered boundary candidates. From each archive, we construct one test suite and evaluate it against the fixed mutant set for that FUT. Therefore, for each FUT and framework, we obtain 10 mutation-analysis results, one for each boundary-exploration run, which are later averaged to account for stochastic variation. Since boundary candidates are input pairs, each candidate contributes two test cases. For numeric FUTs, where both ABEX and SETBVE are applicable, we balance the comparison within each run by first counting the found boundary candidates for each framework and then selecting the smaller of the two counts from both frameworks. This ensures that ABEX and SETBVE are compared using equally sized test suites without giving either framework an advantage due to test-suite size. When one framework discovers more boundary candidates than the selected suite size N, we sub-sample its candidates by sorting them in descending order of program derivative and taking the top N. Since higher PD indicates a stronger behavioral transition and thus a higher likelihood of representing a genuine boundary, this selection favors the framework with the surplus, typically SETBVE, by including its strongest available candidates rather than a random subset. To test whether SETBVE can compensate for lower per-candidate quality with larger test suites, we additionally evaluate a SETBVE-2x configuration that uses up to twice as many SETBVE boundary candidates as ABEX, selected by the same top-PD rule; for FUTs where SETBVE discovers fewer than 2N candidates, all available candidates are used. For non-numeric FUTs, SETBVE is not applicable. Therefore, we report mutation testing results only for ABEX, using the number of boundary candidates available to ABEX. For each run-specific test suite, every selected test case is executed on the original program and on each mutant in the fixed mutant set. The results are stored in a kill matrix, where each entry indicates whether a particular test case killed a particular mutant. We compute mutation scores and kill counts separately for each run-specific test suite and then average the results across the runs. In addition to mutation scores, we analyze whether the generated test suites can detect stubborn mutants, i.e., mutants that are relatively difficult to kill. We use the Relative Stubbornness Thresholding Model (RSTM) [44]. For each FUT, let si be the number of test cases from both methods (ABEX and SETBVE), pooled across all runs, that kill mutant mi , and let smax be the highest such kill count among all mutants of that FUT. Pooling across both methods ensures a neutral definition of stubbornness: a mutant is classified as stubborn based on its inherent difficulty, not on the limitations of any single method. A mutant is considered stubborn if si ≤ θ · smax , where θ ∈ (0, 1] is a configurable threshold. We use θ = 0.05, meaning a mutant is stubborn if it is killed by at most 5% of the number of test cases that kill the easiest-to-kill mutant. Once stubborn mutants are identified, we compare how many each method kills (averaged across runs). This defines stubbornness relative to each FUT’s own kill distribution instead of relying on a fixed global threshold. Following standard practice in empirical studies of mutation testing [45], we do not attempt to identify or remove equivalent mutants: reliable detection of equivalent mutants remains an open problem, and manual inspection does not scale to our corpus of 1158 mutants across 20 FUTs. Because identical mutant sets are used for all configurations of the same FUT, any equivalent mutants lower absolute mutation scores for all methods equally and do not bias the relative comparison. Statistical tests. To assess whether observed differences between configurations are due to chance, we compare per-run outcomes (QD-scores for RQ1, mutation scores for RQ4) using the two-sided Mann–Whitney U test, which makes no assumptions about the underlying distributions [46]. We report the Vargha–Delaney Â12 effect size [47], where Â12 = 1.00 indicates that every run of one configuration outperforms every run of the other. Given the small sample size (10 runs per configuration), we use the exact distribution of the U statistic, and we control for multiple comparisons across the FUTs within each pairwise comparison using Holm–Bonferroni correction at α = 0.05. FUTs where both configurations produce identical results in all runs are reported as ties and excluded from testing. 12

Table 2: QD-score and BC count (mean ± standard deviation over 10 runs) on numeric FUTs. ABEX runs 100 iterations (1000 input pairs), SETBVE runs 1000 iterations (1000 input pairs), and single-prompt issues one call generating 50 input pairs per run. Best QD-score and BC count per FUT in bold. QD-score

BC count

FUT

ABEX

Single-Prompt

SETBVE

ABEX

Single-Prompt

SETBVE

bmi bytecount calDate circle complexCheck complexCheck_full date english findMiddle nextDate tritype

13.7 ± 3.9 10.6 ± 2.2 19.3 ± 6.0 8.3 ± 0.7 3.0 ± 1.7 11.1 ± 1.6 25.0 ± 4.9 8.7 ± 1.3 41.4 ± 8.0 25.3 ± 7.5 39.0 ± 6.6

6.3 ± 1.9 6.8 ± 1.0 3.1 ± 0.9 2.7 ± 1.2 0.0 ± 0.0 10.6 ± 1.8 6.6 ± 2.3 3.0 ± 0.6 10.4 ± 2.1 6.9 ± 1.5 17.6 ± 2.6

0.5 ± 0.3 15.1 ± 2.5 0.3 ± 0.2 0.1 ± 0.1 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 1.6 ± 0.9 0.2 ± 0.2 0.0 ± 0.0 0.0 ± 0.0

65 ± 21 67 ± 17 84 ± 30 32 ± 10 34 ± 16 58 ± 27 106 ± 29 21 ± 9 88 ± 16 63 ± 11 61 ± 9

7±2 8±2 15 ± 2 3±1 0±0 14 ± 3 19 ± 2 3±1 18 ± 4 15 ± 3 20 ± 3

309 ± 23 228 ± 7 898 ± 11 251 ± 31 235 ± 17 235 ± 17 830 ± 15 30 ± 8 842 ± 15 11 ± 4 19 ± 6

18.7

6.7

1.6

62

11

353

Average

5. Results This section presents the results of our empirical evaluation. 5.1. RQ1: Numeric Boundary Discovery As summarized in Table 2, ABEX achieves the highest QD-score on 10 of 11 FUTs, with an average of 18.7, roughly 2.8× that of single-prompt (6.7) and 11.7× that of SETBVE (1.6), indicating that LLM-generated strategies are effective at discovering boundary candidates that combine high program derivative values with good behavioral diversity. The sole exception is bytecount, where SETBVE achieves a higher QD-score (15.1 vs. 10.6), which indicates that traditional search can still be advantageous for some simple numeric structures. These differences are statistically significant: ABEX’s advantage over SETBVE holds at Holm-corrected p < 0.05 with maximal effect sizes (Â12 = 1.00) on 10 of 11 FUTs, and over single-prompt on 10 of 11 FUTs (Â12 ≥ 0.94). The exceptions are bytecount, where SETBVE’s higher QD-score is significant (p = 0.001, Â12 = 0.09), and complexCheck_full, where the difference between ABEX and single-prompt is not significant, consistent with the convergence discussed below. SETBVE dominates in BC count on 9 of 11 FUTs (average 353 vs. ABEX’s 62), indicating that it is more effective at populating diverse archive regions within the same budget, but at the cost of lower per-cell quality as reflected in its lower QD-scores. This quality gap is substantial: across the 11 numeric FUTs, approximately 99% of SETBVE’s boundary candidates have PD < 0.001, whereas ABEX achieves a mean PD roughly 150× higher under the same iteration budget. Notably, on nextDate and tritype, ABEX outperforms SETBVE in both QD-score and boundary candidate count, indicating that LLM-guided exploration can simultaneously achieve higher coverage and boundary quality for certain FUTs. The complexCheck pair reveals a sensitivity to specification quality. Without a detailed docstring, single-prompt discovers zero boundary candidates, while ABEX still achieves a QD-score of 3.0 and 34 cells, highlighting its robustness to underspecified functions. With a more detailed specification, both methods improve (ABEX: 11.1; single-prompt: 10.6), and their QD-scores converge, suggesting that when rich context is available, a single prompt can approach the boundary quality achieved by iterative, feedback-driven exploration. Answer to RQ1: ABEX achieves higher boundary quality than both baselines on most numeric FUTs, demonstrating that LLM-generated strategies are effective for numeric boundary discovery.

13

Table 3: Boundary types discovered by ABEX for string-based FUTs, with a representative input pair and the number of runs (out of 10) in which each type was found. FUT

Boundary type

Example: input1 (output1) → input2 (output2)

passwordStrength (str)

Weak ↔ Acceptable Acceptable ↔ Strong Strong ↔ Very strong

onlylowercase (Weak) → onlylowercase! (Acceptable) Password123 (Acceptable) → Password123! (Strong) midlengthnoSym9 (Strong) → midlengthnoSym9! (Very strong) aaaaaaaaaaaaaaa (Weak) → !Aaaaaaaaaaaaaa (Strong) [empty] (Invalid) → a (Weak)

10/10 10/10 9/10

def (keyword: def) → ef (identifier: ef) 123.456 (number:123.456) → 123..456 (number:123.; number:.456) == (operator: ==) → != (operator: !=) ‘quote_here’ (string :quote_here) → ‘quote_here (err: unclosed string)

8/10 10/10

Non-adjacent transition Invalid ↔ Weak printTokens (str)

Token type changed Token count changed Token value changed Valid ↔ Error

replace (src, pattern, repl)

Match count changed Same match count, result changed No match ↔ Match

Found runs

9/10 10/10

10/10 10/10

‘a’, ‘a’, ‘aa’ (result:aa; matches:1) → ‘aa’, ‘a’, ‘aa’ (result:aaaa; matches:2) ‘a’, ‘a’, ‘A’ (result:A; matches:1) → ‘a’, ‘a’, ‘A ’ (result:A , matches:1) ‘A’, ‘a’, ‘b’ (result:A; matches:0) → ‘a’, ‘a’, ‘b’ (result:b; matches:1)

10/10 10/10 10/10

stringPalindrome (str)

Palindrome ↔ Not

123321 (Palindrome) → 1233210 (Not palindrome)

10/10

validateEmail (str)

Valid ↔ Invalid Invalid (different errors)

bob@trusted.co (Valid) → bob@trusted.cö (Invalid format) [empty] (Email must not be empty) → [space char] (Invalid format)

10/10 7/10

5.2. RQ2: Non-numeric Boundaries Tables 3 and 4 report the boundary types discovered by ABEX for string and array/mixed FUTs, together with a representative example for each type and the number of runs in which the type was found. Boundary types are derived from each FUT’s observable output partitions. For FUTs with discrete semantic outputs, boundary types correspond to transitions between output categories; for example, passwordStrength induces transitions among {Weak, Acceptable, Strong, Very strong}. For FUTs with structural inputs, boundary types capture input-level transitions that trigger different observable behaviors; for example, insertionSort distinguishes empty from non-empty inputs, changes in array length, changes in element order, and changes in element values. Once these categories are defined, assigning a discovered boundary candidate to a boundary type is deterministic and rule-based. We do not claim that the listed boundary types form an exhaustive taxonomy of all possible boundaries for each FUT. Different granularities are possible: for example, one could split a broad type into finer subtypes or merge several related transitions into a coarser category. We use these boundary types as a reproducible summarization of the main behavioral transitions observed in the discovered candidates. We additionally characterize boundaries using the validity groups from Dobslaw et al. [5]: VV, where both outputs are valid but different; VE, where one output is valid and the other is erroneous; and EE, where both outputs are erroneous but correspond to different errors. When domain-specific output semantics are unavailable, these validity groups provide a coarser fully automatic fallback characterization, because they are computed directly from ABEX’s execution outcomes and require no domain-specific boundary-type rules. String FUTs. ABEX consistently discovers meaningful behavioral transitions aligned with domain logic across all string-based programs. For passwordStrength, all five boundary types, including all adjacent strength-level transitions and non-adjacent jumps, are found in at least 9 of 10 runs, demonstrating reliable coverage of a multi-class output space. For printTokens, replace, stringPalindrome, and validateEmail, nearly all boundary types are discovered in 8–10 runs, with the exception of cross-error-type boundaries in validateEmail (7/10). Most discovered boundaries are VV (e.g., crossing password strength thresholds, toggling match counts, breaking palindrome symmetry), while VE boundaries, where one output is erroneous (e.g., unclosed string token, invalid email format), are 14

Table 4: Boundary types discovered by ABEX for array and mixed input FUTs, with a representative input pair and the number of runs (out of 10) in which each type was found. FUT

Boundary type

Example: input1 (output1) → input2 (output2)

insertionSort (arr[int])

Empty ↔ Non-empty Different array length Same elements different order

10/10 10/10 10/10

Different elements

[] (empty) → [0] (single:[0]) [42] (single:[42]) → [42,0,0] ([0,0,42]; comp:3; swaps:2) [-1,0,0,1,2,3] ([-1,...,3]; comp:5; swaps:0) → [-1,0,0,1,3,2] ([-1,...,3]; comp:6; swaps:1) [0] (single:[0]) → [1] (single:[1])

normalize (arr[int])

Empty ↔ Non-empty Different array length Different elements

[] (Error: Expected...) → [0] ([0.0]) [0.0] ([0.0]) → [0.0, 0.0] ([0.0, 0.0]) [1, 2] ([0.0, 1.0]) → [2, 2] ([0.0, 0.0])

4/10 10/10 10/10

maxLexString (arr[str])

Same length, different elements

[‘apple’, ‘banana’, ‘cherry’] (cherry) → [‘apple’, ‘banana’, ‘chery’] (banana) [‘file2.txt file10.txt’] (file2.txt file10.txt) → [‘file2.txt’, ‘file10.txt’] (file10.txt) [‘Z’, ‘z’] (z) → [‘Z’, ‘z’, ‘zz’] (zz)

10/10

([], 1) (empty) → ([0], 1) (not_found) ([7, 8, 9, 9, 9,...], 9) (found:2) → ([7, 8, 8, 9, 9...], 9) (found:3) ([1, 2, 3, 4, 5,...], 8) (not_found) → ([1, 2, 3, 4, 5,...8...], 8) (found:7) ([-3, -2, -1, 0,...], 0) (found:3) → ([-3, -2, -1, 5,...], 0) (error:not_sorted)

10/10 10/10

1024, False, False, 2... (Unresolved) → 1024, True, False, 2,... (IndexError) 601, True, False, 3000... (Climb) → 600, True, False, 3000... (Unresolved) 640, True, False,..., 1,... (Descend) → 640, True, False..., 3,... (Unresolved)

4/10

Single ↔ Multiple elements Different length, both multiple binarySearch (arr[int], int)

Empty ↔ Non-empty Found (diff index) Found ↔ Not found Sorted (Valid) ↔ Unsorted (Error)

tcas (int, bool, bool, int,...)

Valid ↔ IndexError Climb ↔ Unresolved Descend ↔ Unresolved

15

Found runs

10/10

7/10 8/10

10/10 5/10

10/10 6/10

Table 5: Strategy usage and effectiveness by FUT group: iterations selected, new cells per 10 generated pairs (NC/10), and better solutions in existing cells per 10 pairs (BS/10), aggregated over 10 runs per FUT. FUT Group

Strategy type

Iterations

NC/10

BS/10

Numeric (11 FUTs)

mutation gen/context-aware gen/generic gen/baseline

3321 3272 2282 2125

1.52 0.71 2.49 1.53

0.42 0.16 0.19 0.17

String (5 FUTs)

gen/context-aware gen/baseline mutation gen/generic

2454 1204 946 396

2.56 3.14 3.33 3.35

0.14 0.12 0.23 0.03

Array & Mixed (5 FUTs)

gen/context-aware gen/baseline gen/generic mutation

2089 1395 825 691

1.59 2.28 3.09 2.70

0.36 0.27 0.14 0.39

also discovered consistently, indicating that ABEX is not exclusively biased toward same-validity transitions. Array and Mixed FUTs. For array-based FUTs, ABEX reliably discovers structural boundaries such as emptyvs-non-empty and length-change transitions, with most types found in 8–10 runs. Notable exceptions include the empty-vs-non-empty boundary in normalize (4/10) and the unsorted-array error boundary in binarySearch (5/10), suggesting that VE-type conditions are harder to discover consistently. For the mixed-type tcas FUT, ABEX discovers all three boundary types, including a VE boundary (IndexError) in 4/10 runs, demonstrating that LLM-guided exploration can navigate complex mixed-type input spaces, albeit with reduced consistency on rarer error conditions. While the current configuration appears biased toward VV boundaries and may under-emphasize VE and EE transitions, the observed candidates suggest that ABEX can systematically identify rich and interpretable non-numeric boundaries, providing evidence that it generalizes beyond purely numeric domains. Answer to RQ2: ABEX consistently discovers meaningful, domain-aligned boundaries across FUTs with string, array, and mixed inputs, demonstrating generalization beyond numeric domains without input-type-specific engineering. 5.3. RQ3: Strategy Effectiveness by Input Type Table 5 reports the effectiveness of each strategy type broken down by FUT group, aggregating results across all 10 runs per FUT9 . We distinguish four categories: mutation, context-aware generation, generic generation, and the pre-seeded baseline strategy, which is reported separately from context-aware generation as explained in Section 3.1. For each strategy type, we report the total number of iterations in which it was selected (Iterations), the average number of newly discovered archive cells per 10 generated input pairs (NC/10), and the average number of better solutions found for already-occupied cells per 10 input pairs (BS/10). NC/10 reflects a strategy’s ability to broaden coverage of the behavioral space, while BS/10 reflects its ability to refine the quality of already-discovered archive regions. Numeric FUTs. Mutation is the most frequently selected strategy (3321 iterations), closely followed by contextaware generation (3272), while generic and baseline generation are selected less often (2282 and 2125). Despite its high selection rate, context-aware generation achieves the lowest NC/10 (0.71) and BS/10 (0.16) of all strategies. Generic generation leads in NC/10 (2.49), making it the strongest contributor to coverage breadth, while mutation leads in BS/10 (0.42), outperforming all generation strategies by more than a factor of two. These results reveal a clear division of labor: generic generation drives coverage breadth, while mutation is the dominant mechanism for refining boundary quality. 9 Strategies discovered per FUT are available in the replication package archive/ABEX_full/strategies.

16

String FUTs. The usage distribution shifts for string FUTs: context-aware generation dominates (2454 iterations), while generic generation is the least used (396). All four strategy types achieve higher NC/10 values than in the numeric group, suggesting the string behavioral space is broadly more amenable to diverse coverage. Mutation achieves the highest BS/10 (0.23), whereas generic generation, despite its leading NC/10 (3.35), contributes negligibly to refinement (0.03). The strong coverage performance of generic generation type indicates that simple string heuristics already capture relevant boundary patterns and that docstrings are not strictly necessary for effective exploration. Array and Mixed FUTs. Context-aware generation is again the most-used strategy type (2089 iterations), while mutation is the least used (691). Generic generation leads in NC/10 (3.09), followed by mutation (2.70) and baseline generation (2.28), mirroring the numeric group’s relative rankings. For refinement, mutation and context-aware generation lead with BS/10 of 0.39 and 0.36, respectively. The notably stronger BS/10 of context-aware generation here, compared to numeric and string FUTs, suggests that context captured in docstrings provides more actionable guidance when inputs are compositionally complex. Overall, generation strategies, especially generic and baseline variants, are critical for discovering diverse archive regions, while mutation is essential for quality refinement. However, none of these strategies operates in isolation during ABEX runs, and it is the coordinated mix of generation and mutation that ultimately yields strong boundary discovery performance. Answer to RQ3: Generation strategies drive coverage breadth while mutation refines boundary quality, forming a consistent division of labor whose coordinated interplay is key to boundary discovery performance. 5.4. RQ4: Fault Detection Table 6 reports the mutation testing results averaged across 10 runs. Overall, the results show that boundary candidates discovered by both BVE frameworks are effective at detecting faults. When ABEX and SETBVE are compared using test suites of the same size, ABEX achieves higher mutation scores on 8 of the 10 numeric FUTs and ties with SETBVE on the remaining two (date and findMiddle). On average, ABEX reaches a mutation score of 86.2%, compared to 61.9% for SETBVE. Excluding the two saturated FUTs, date and findMiddle, which have very few mutants, widens the average mutation-score gap between ABEX and SETBVE. ABEX achieves an average mutation score of 82.8%, compared with 52.4% for SETBVE, yielding a difference of about 30. The difference is especially large for complexCheck10 , english, nextDate, and tritype, where ABEX improves the mutation score by more than 29 percentage points. These results indicate that, for the same number of tests, ABEX-discovered boundary candidates have stronger fault-detection capability than those discovered by SETBVE. These differences are statistically significant with large effect sizes (Mann–Whitney U, Holm-corrected p < 0.05; Â12 ≥ 0.94) on all eight numeric FUTs with non-degenerate outcomes; date and findMiddle are ties, with all mutants killed by both methods in every run. To examine whether SETBVE can close this gap with a larger test suite, we also evaluate SETBVE-2x, using up to twice as many SETBVE boundary candidates as ABEX. This doubled configuration is possible for seven FUTs: bmi, bytecount, calDate, circle, complexCheck, date, and findMiddle. For the remaining three FUTs, english, nextDate, and tritype, SETBVE does not discover enough boundary candidates to construct a doubled suite; in these cases, we use all available SETBVE candidates, marked with † in the table. SETBVE-2x improves over the SETBVE setting on several FUTs, but the gains are generally small. Its average mutation score (computed over all 10 FUTs, including the four † cases) increases from 61.9% to 63.7%, and it still remains below ABEX on 8 of the 10 comparable numeric FUTs, tying only on date and findMiddle. This suggests that the gap is not primarily caused by test-suite size. Even with more boundary candidates, SETBVE does not reach the fault-detection effectiveness of ABEX, indicating that ABEX tends to discover candidates that are more fault-revealing rather than merely more numerous. The FUTs with non-numeric inputs further show that ABEX-discovered candidates are useful beyond the numeric domain. ABEX achieves high mutation scores for several string, array, and mixed-input functions, including 10We use the complexCheck_full variant, which has a more detailed docstring than complexCheck.

17

Table 6: Killed mutants and mutation scores (mean ± standard deviation over 10 runs) for equally sized test suites from ABEX and SETBVE, and for SETBVE-2x with up to twice as many candidates. SETBVE is not applicable to non-numeric FUTs (bottom block). FUT

Tests (avg)

Mutants

Killed

Mutation Score (%)

ABEX

SETBVE

SETBVE-2x

ABEX

SETBVE

SETBVE-2x

bmi bytecount calDate circle complexCheck date english findMiddle nextDate tritype

130 134 168 63 116 213 37 176 22 38

50 65 61 28 38 7 37 2 113 85

47.5 ± 1.3 42.5 ± 1.7 58.6 ± 2.5 26.0 ± 0.0 34.3 ± 1.3 7.0 ± 0.0 29.6 ± 1.2 2.0 ± 0.0 71.9 ± 14.4 66.9 ± 5.4

34.8 ± 4.9 30.0 ± 0.0 53.2 ± 1.0 20.2 ± 2.5 14.4 ± 0.9 7.0 ± 0.0 12.5 ± 1.7 2.0 ± 0.0 25.5 ± 1.7 42.1 ± 8.3

35.6 ± 5.0 35.5 ± 1.9 53.9 ± 0.7 21.3 ± 2.2 14.7 ± 1.2 7.0 ± 0.0 13.3 ± 2.2† 2.0 ± 0.0 25.5 ± 1.7† 42.1 ± 8.3†

95.0 ± 2.6 65.4 ± 2.7 96.1 ± 4.0 92.9 ± 0.0 90.3 ± 3.5 100.0 ± 0.0 80.0 ± 3.2 100.0 ± 0.0 63.6 ± 12.8 78.7 ± 6.3

69.6 ± 9.7 46.2 ± 0.0 87.2 ± 1.6 72.1 ± 9.0 37.9 ± 2.4 100.0 ± 0.0 33.8 ± 4.7 100.0 ± 0.0 22.6 ± 1.5 49.5 ± 9.8

71.2 ± 10.0 54.6 ± 2.9 88.4 ± 1.1 76.1 ± 8.0 38.7 ± 3.1 100.0 ± 0.0 35.9 ± 5.9† 100.0 ± 0.0 22.6 ± 1.5† 49.5 ± 9.8†

Average

109.7

48.6

38.6

24.2

25.1

86.2

61.9

63.7

binarySearch insertionSort maxLexString normalize passwordStrength printTokens replace stringPalindrome tcas validateEmail

343 504 359 572 289 641 588 90 39 236

58 49 40 4 47 205 161 11 71 26

51.2 ± 1.3 46.0 ± 0.0 12.7 ± 1.42 4.0 ± 0.0 45.0 ± 2.2 156.0 ± 7.3 73.6 ± 11.2 11.0 ± 0.0 38.0 ± 5.2 20.2 ± 1.9

– – – – – – – – – –

– – – – – – – – – –

88.3 ± 2.2 93.9 ± 0.0 31.8 ± 3.5 100.0 ± 0.0 95.7 ± 4.7 76.1 ± 3.5 45.7 ± 7.0 100.0 ± 0.0 53.5 ± 7.3 77.7 ± 7.5

– – – – – – – – – –

– – – – – – – – – –

Average

366

67.2

45.8

–

–

76.3

–

–

† Fewer than twice as many boundary candidates with PD > 0 were available, so all available candidates were used. These FUTs are included in the

SETBVE-2x average.

passwordStrength (95.7%), insertionSort (93.9%), binarySearch (88.3%), stringPalindrome (100.0%), and normalize (100.0%). The lower scores for maxLexString, replace, and tcas suggest that some non-numeric or mixed-input programs contain faults that are harder to expose with the discovered boundary candidates. Figure 2 further breaks down mutants according to whether they are killed by neither method, only SETBVE, only ABEX, or both methods. The two methods kill many of the same mutants: the overlap is approximately 70% for bmi, 72% for circle, 87% for calDate, and 100% for both date and findMiddle. The overlap for date and findMiddle is likely due to their small mutant sets, with only 7 and 2 mutants, respectively, reflecting their small code size. ABEX kills many mutants that SETBVE does not. The ABEX-only fraction is largest for complexCheck and english, at roughly 50% and 46% of mutants, respectively, followed by nextDate at about 44% and tritype at about 33%. In contrast, the SETBVE-only fraction is very small, appearing only for nextDate and tritype. Figure 3 and Table 7 compare the number of stubborn mutants killed by ABEX and SETBVE. Stubborn mutants are those killed by at most 5% as many test cases as the easiest-to-kill mutant for the same FUT (see Section 4.2 for details). ABEX kills more stubborn mutants than SETBVE for every FUT shown11 . The largest absolute numbers occur for nextDate, tritype, and complexCheck, where ABEX kills, on average over 10 runs, 24.5, 19.2, and 15.3 stubborn mutants, respectively. In contrast, SETBVE kills only a small number of stubborn mutants for these functions and kills almost none for complexCheck, english, and bytecount. Overall, ABEX kills 95.5 of the 190 stubborn mutants (50.3%), while SETBVE kills 10.6 (5.6%). Thus, SETBVE kills only about 11% as many stubborn mutants as 11 findMiddle and date are excluded because no stubborn mutants were detected for these FUTs.

18

100

n=85

Killed by Neither

SETBVE Only

ABEX Only

n=38

n=50

n=28

n=37

n=65

Both n=61

n=7

n=2

80 60 40 20

le idd fin dM

te da

lDa te ca

ou tec

cir cle

nt

i

sh gli

bm

by

lex C mp

en

he c

e yp co

trit

ne xtD

k

0

ate

Percentage of Mutants

n=113

Figure 2: Distribution of mutants killed by ABEX only, SETBVE only, both, or neither, for numeric FUTs (averaged over 10 runs, equal test suite sizes). n gives the number of mutants per FUT.

ABEX. The stubborn-mutant distribution also explains the notably high run-to-run variance of ABEX on nextDate (σ = 12.8 percentage points, the highest in Table 6). nextDate contains the largest concentration of stubborn mutants in our corpus (64 of 113, 57%), located primarily in its leap-year and month-boundary logic. Whether these mutants are killed appears to depend on whether a given run discovers strategies targeting those specific edge cases: runs that do reach mutation scores of up to 81%, while runs that do not fall to 37%. The other date-related functions lack this concentration, date has no stubborn mutants, and calDate has 16%, and correspondingly show low variance. For FUTs with string, array, and mixed input types, no SETBVE baseline exists. This group includes eight functions, excluding normalize and stringPalindrome, for which no stubborn mutants were detected. On these eight functions, ABEX kills an average of 74.9 out of 279 stubborn mutants in total, corresponding to a kill rate of 26.8%. The kill rate varies across FUTs: ABEX achieves over 35% on five FUTs (passwordStrength, printTokens, insertionSort, binarySearch, and validateEmail), but performs less effectively on maxLexString and tcas, with kill rates of 2.5% and 14.2%, respectively. These two functions contain 28 and 33 stubborn mutants, respectively. Overall, the kill rate for this group of functions is lower than for functions with integer inputs, which may be due to differences in function complexity and greater input variability. Answer to RQ4: ABEX-discovered boundary candidates are more fault-revealing than SETBVE’s, achieving higher mutation scores and killing more difficult-to-detect stubborn mutants. 5.5. RQ5: Ablation We conduct RQ5 on a representative subset of six FUTs selected from the initial set of 20: bmi, tritype, passwordStrength, replace, insertionSort, and binarySearch. These functions were selected to cover variation in input type, output type, arity, and boundary complexity. The subset includes numeric, string, and array-based inputs; categorical, transformed, and index outputs; one-, two-, and three-parameter functions; and different forms of boundary behavior, including threshold-based boundaries, multi-condition classification, pattern matching, structural array behavior, and precondition-dependent behavior. Unlike the RQ1 – RQ2 experiments, the ablation and LLM-comparison runs use 500 iterations rather than 100. Longer runs are needed here because differences between configurations that share components can take longer to manifest than differences between distinct frameworks: a weaker variant may keep pace early while the archive is sparse, and fall behind only once easy boundary candidates are exhausted (cf. Figure 5). Because of the associated cost, each configuration is run once per FUT on the representative subset.

19

SETBVE

Table 7: Number of stubborn mutants killed by ABEX and SETBVE per numeric FUT (mean over 10 runs, RSTM θ = 0.05), out of the total per FUT.

ABEX

FUT

complexCheck calDate bmi tritype english nextDate bytecount circle 0

20

40

60

80

Stubborn Mutants Killed (%)

Total

SETBVE

ABEX

bmi bytecount calDate circle complexCheck english nextDate tritype

10 35 10 3 19 15 64 34

1.2 0.0 2.4 0.0 0.4 0.4 2.1 4.1

7.5 12.5 7.6 1.0 15.3 7.9 24.5 19.2

Total

190

10.6

95.5

100

Figure 3: Comparison of stubborn mutant detection between ABEX and SETBVE averaged over 10 runs (RSTM θ = 0.05).

ExplorerOnly

80 60 40 22 77 20 80 26 6

94 8 62 6 82

10 13 36 04 20 6

12

07 70 2 14 9

73 77 20 04 18 76 62 5 96 4 11 7

17

73 65 41 17 0 20 3 46

29 17

8

20 0 100 80 60 40 71 72 3 17 6

11

tritype passwordStrength replace

98

bmi

79

0

3

20 52

BC Count (% of max)

NoIdeator

29 7 39 7 26

QD-Score (% of max)

ABEX 100

insertionSort binarySearch

Figure 4: Component ablation on six FUTs: QD-score (top) and BC count (bottom) for ABEX, NoIdeator, and ExplorerOnly, normalized per FUT to the best performing variant (bar labels show absolute values).

20

ABEX

ExplorerOnly

BC count

QD-Score

30 25 20 15 10 5 0 400

NoIdeator

300 200 100 0

0

100

200

Iteration

300

400

500

Figure 5: QD-score and BC count trend for the bmi FUT.

Components ablation. Figure 4 compares ABEX against two ablated variants. NoIdeator removes the Ideator: the Strategy Generator expands the Coordinator’s search gap directly into a strategy, without the intermediate step that compares candidate directions against the existing strategy pool for functional novelty, reuses existing strategies when a suitable one is available, and filters directions likely to be low-yield before expansion. ExplorerOnly removes the Coordinator, Ideator, and Strategy Generator entirely and executes only the pre-seeded baseline strategy throughout the run; it retains the same archive feedback as the full framework, with each Explorer prompt including recently added and randomly sampled input pairs to avoid duplicates. All three configurations generate the same number of candidate input pairs (10 per iteration over 500 iterations), so differences reflect the removed components rather than generation budget. Figure 5 shows QD-score and boundary candidate count over 500 iterations for bmi as a representative example; further plots are in the replication package. ABEX achieves the highest QD-score on all six FUTs, confirming that the full pipeline is necessary. ExplorerOnly performs worst relative to the ablation variants across all FUTs in both metrics, establishing adaptive strategy generation as the primary driver of effectiveness. Averaged over the six FUTs, ABEX improves QD-score by 6.8× over ExplorerOnly (per-FUT ratios ranging from 1.8× on tritype to 11.6× on insertionSort). The NoIdeator setup falls between the two, showing the Ideator yields measurable quality gains. NoIdeator achieves higher boundary cell counts than ABEX on several FUTs, suggesting that, for those FUTs, it discovers more diverse boundary candidates, but of lower quality. As shown in Figure 5, QD-score growth is non-monotonic, with sharp jumps coinciding with strategy switches, visible in ABEX and NoIdeator but absent in ExplorerOnly. Moreover, both ABEX and NoIdeator show no signs of stagnation even after almost 1 hour (500 iterations), suggesting that longer runs could yield further improvements. Cost. Runtime cost depends on FUT complexity and strategy turnover rate. The framework uses a two-tier model architecture: GPT-5.1 handles the Coordinator and Ideator, while GPT-5-mini powers the Explorer and Strategy Generator. On average, ABEX consumed 2.7M tokens (6.38 USD) per 500-iteration run, with 416K tokens allocated to GPT-5.1 and 2.3M to GPT-5-mini. NoIdeator reduced costs to 2.3M tokens (5.72 USD) by eliminating Ideator calls while retaining strategy generation overhead. ExplorerOnly, lacking both Coordinator and Ideator, required only 1.4M tokens (2.71 USD), representing cost reduction compared to ABEX, albeit with a trade-off in quality. Closed vs. open-weight LLMs. To evaluate whether the choice of LLM affects ABEX’s boundary candidate generation ability, we compare three LLMs: GPT-5.1/mini (closed-weight), Gemma-4 (open-weight), and Qwen-3.5 (open-weight). 21

ABEX-Qwen3.5

80 60 40

29 41 29

73 91 78

73 77 13 29 81 7

22 77 26 05 41 38

94 8 13 68 92 6

17 07 41 22 50 1

17 0 38 1 15 2

62 5 65 3 38 6

10 13 20 41 15 15

12 52 10 45 10 96

11 71 11 71 53 3

20 0

BC Count (% of max)

ABEX-Gemma4

29 7 40 5 15 2

QD-Score (% of max)

ABEX-GPT5.1/mini 100

100 80 60 40 20 0

bmi

tritype passwordStrength replace

insertionSort binarySearch

Figure 6: ABEX with three LLMs (GPT-5.1/mini, Gemma-4, Qwen-3.5) on six FUTs: QD-score (top) and BC count (bottom), normalized per FUT (bar labels show absolute values).

We run ABEX with each LLM on six functions for 500 iterations, measuring BC Count and QD-score. Note that ABEX-GPT5.1/mini uses a combination of GPT-5.1 and GPT-5-mini to reduce API costs, whereas ABEX-Gemma4 and ABEX-Qwen3.5 use the same full model for all ABEX components throughout. All three models successfully discovered boundary candidates across all six functions. Figure 6 shows the results. Gemma-4 achieved the highest total BC count (5696), followed by GPT-5.1/mini (4528) and Qwen-3.5 (3834). Gemma4 achieved the maximum BC count on four of six functions (bmi, tritype, passwordStrength, replace), while GPT-5.1/mini achieved the maximum on two functions (insertionSort, binarySearch, tied with Gemma-4 on the latter). The QD-Score results provide a complementary view of model performance. Gemma-4 obtains the highest QD-score on four functions12 : bmi, tritype, insertionSort, and binarySearch. This indicates that, for these functions, Gemma-4 not only discovers many candidates but also accumulates high-quality candidates overall. GPT5.1/mini achieves the highest QD-score on passwordStrength, despite discovering slightly fewer candidates than Gemma-4 for this function. This suggests that GPT-5.1/mini identifies fewer but much stronger candidates in this case. Qwen-3.5 achieves the highest QD-score on replace, where it outperforms both GPT-5.1/mini and Gemma-4 in accumulated quality, even though Gemma-4 discovers the largest number of candidates for this function. To further assess candidate quality, we compute the average program derivative (QD-score / BC Count)13 for each model-function combination (Table 8). GPT-5.1/mini achieved the highest mean average PD (2.80), compared to Gemma-4 (1.41) and Qwen-3.5 (1.22). GPT-5.1/mini’s advantage is driven primarily by passwordStrength, where it achieved an average PD of 11.80 compared to 2.04 (Gemma-4) and 2.12 (Qwen-3.5). The relationship between quantity and quality varies across functions. For replace, Gemma-4 discovered the most candidates (2041) but achieved the lowest average PD (1.28). For binarySearch, Gemma-4 and GPT-5.1/mini discovered identical BC counts (1171), but Gemma-4 achieved a higher average PD (3.52 vs. 1.46). Qwen-3.5 achieved the highest average PD on three functions (bmi: 0.19, tritype: 0.51, replace: 2.73) despite having lower BC counts. Both closed and open-weight LLMs generate meaningful boundary candidates with ABEX. Gemma-4 produces 12 Gemma-4 was released in April 2026, after the RQ1 – RQ4 experiments had been completed. The main ABEX configuration therefore uses GPT-5.1/mini. 13 Average PD is computed from raw program-derivative values, whose range is FUT-dependent.

22

Table 8: Average program derivative (QD-score / BC Count) by LLM over 500 iterations.

FUT

ABEX-GPT5.1/mini

ABEX-Gemma4

ABEX-Qwen3.5

bmi tritype passwordStrength replace insertionSort binarySearch

0.10 0.43 11.80 2.25 0.76 1.46

0.10 0.24 2.04 1.28 1.31 3.52

0.19 0.51 2.12 2.73 0.84 0.94

Average

2.80

1.41

1.22

the highest boundary candidate count, GPT-5.1/mini produces higher-quality candidates on average despite using a mixed-model configuration, and Qwen-3.5 shows competitive quality on specific functions but lower overall quantity of boundary candidates. Answer to RQ5: Adaptive strategy generation is the primary effectiveness driver. The absence of stagnation suggests further improvement is achievable with longer runs. ABEX remains effective across both closed and open-weight LLMs, although the magnitude of performance gains varies across functions and models.

6. Discussion ABEX shows that an agentic, LLM-based framework can automate boundary value exploration across heterogeneous input types without manually engineered mutation operators, while producing higher-quality boundaries than both baselines on numeric inputs. On 10 of 11 numeric FUTs, ABEX achieves the highest QD-scores, averaging 2.8× the single-prompt LLM baseline and 11.7× the QD-based search baseline. For non-numeric inputs, explored here for the first time in automated BVE in a black-box setting, ABEX discovers meaningful, domain-aligned boundary behaviors across all 10 string, array, and mixed input FUTs, again without datatype-specific operators. Taken together, these results show that adaptive, LLM-guided search can extend BVE beyond numeric domains. Adaptive strategy generation matters. The comparison between ABEX and the single-prompt baseline is revealing because both use the same underlying LLM to generate inputs. Single-prompt achieves an average QD-score of 6.7, whereas ABEX reaches 18.7. This gap therefore cannot be explained by model capability alone; it points instead to the agentic design and adaptive feedback loop as the main sources of boundary quality. Because the two configurations use different generation budgets (Section 4.2), part of this gap could in principle be attributed to budget rather than design. The ablation results let us separate the two factors. ExplorerOnly iterates the same fixed baseline strategy as single-prompt, with archive feedback, under a full search budget; yet even with more generated candidate pairs, it remains far below the full framework on the shared FUTs (bmi: 8 vs. 29; tritype: 41 vs. 73), with bmi improving only from 6.3 to 8 over the single call. Budget-matched iteration of a fixed strategy thus recovers only part of the difference; adaptive strategy generation accounts for the rest, with NoIdeator’s intermediate performance further showing that adaptive strategy selection and Ideator-mediated refinement make distinct contributions to quality. This interpretation aligns with Lemieux et al. [26], who show that feedback-driven LLM exploration outperforms one-shot generation, and with the multi-agent testing results of Yoon et al. [10]. The strategy-level analysis (RQ3) explains how adaptation helps: generation strategies broaden archive coverage while mutation strategies refine boundary quality. This pattern mirrors the classic exploration–exploitation trade-off in evolutionary search, so it is not surprising in direction. Its importance here is that it appears uniformly across numeric, string, and array domains, indicating that ABEX benefits from genuine complementarity among strategy types rather than from any one strategy dominating the search.

23

Complementarity with optimized QD search. ABEX and SETBVE [7] make different trade-offs. Under a fixed iteration budget, SETBVE discovers more boundary candidates than ABEX on average (353 vs. 62 BC count), confirming that optimized numeric search is more effective at broadly covering the behavioral space. ABEX, by contrast, prioritizes boundary quality. As reported in Section 5.1, approximately 99% of SETBVE’s candidates have PD < 0.001, whereas ABEX achieves a mean PD roughly 150× higher under the same budget. Since the BC count only requires PD > 0, many near-zero-PD boundary candidates may reflect different equivalence partitions rather than true boundary cases. The trade-off between quality and breadth is, moreover, not universal: on nextDate and tritype, ABEX outperforms SETBVE on both archive coverage and QD-score. The main exception in the other direction is bytecount, where SETBVE achieves a higher QD-score (15.1 vs. 10.6), suggesting that numerically simple functions with tightly bounded input structures remain well suited to optimized search. Overall, these results position ABEX as a complement to QD-based search rather than a replacement: ABEX provides higher boundary quality and supports non-numeric inputs, but at a higher compute cost per iteration. For numeric inputs, a natural hybrid strategy would use SETBVE for broad exploration and ABEX to refine promising regions. The role of specification quality. The complexCheck results reveal a robustness property with clear practical value. With only a short docstring, ABEX achieves a QD-score of 3.0 and discovers 34 boundary cells, whereas single-prompt produces no candidates at all. With a more detailed specification, both methods improve and their QD-scores converge (11.1 vs. 10.6), suggesting that richer context narrows the advantage of iteration but does not eliminate it. A natural next step is to relax the black-box assumption progressively and test whether richer context yields proportionate gains. The non-numeric results also show a tendency toward VV boundaries, which likely reflects the default prompting configuration. Because strategy generation is LLM-driven, explicitly targeting VE or EE validity groups through tailored prompts is a straightforward next step that requires no architectural changes. Fault-detection effectiveness. Mutation testing confirms that ABEX-discovered boundary candidates are effective as fault-detecting test cases. Compared with SETBVE under equal test-suite sizes, ABEX achieves higher mutation scores on 8 of 10 numeric FUTs, and SETBVE does not close the gap even when given up to twice as many boundary candidates (SETBVE-2x). The gap is therefore not primarily an artifact of test-suite size: ABEX tends to discover candidates that are more fault-revealing, not merely more numerous. The stubborn-mutant analysis sharpens this point. ABEX kills 50.3% of stubborn mutants on numeric FUTs versus 5.6% for SETBVE, indicating that the quality advantage translates into the ability to expose faults. No dependence on proprietary frontier models. The LLM comparison (RQ5) shows that ABEX’s effectiveness is not tied to a specific proprietary model: the open-weight Gemma-4 achieved the highest boundary candidate count and the highest QD-score on four of six FUTs, while GPT-5.1/mini produced higher average per-candidate quality. This matters for practical adoption. Organizations with confidentiality constraints that preclude sending code artifacts, signatures, or specifications to third-party APIs can run ABEX entirely on locally hosted open-weight models, and open-weight deployment also removes per-token API costs from the framework’s main cost driver (Section 5.5). The performance differences across FUTs suggest that model choice can be treated as a tunable deployment decision rather than a prerequisite for the approach. Implications for research and practice. The main implication for researchers is that adaptive, LLM-guided search is a viable general mechanism for testing problems in which operator engineering is costly or infeasible. We demonstrate this here for BVE, but the core ingredients should transfer to any setting in which exploration policies can be expressed and assessed through execution feedback. For practitioners, the clearest benefit lies in non-numeric inputs or settings where per-type operator design is substantial. For simple numeric FUTs, SETBVE remains faster and cheaper, though that gap will likely narrow as inference costs fall [48]. 6.1. Lessons Learnt and Future Directions We draw three main lessons from this study. First, behavioral descriptors strongly influence which boundaries the search discovers. We reused the four descriptors from SETBVE because they generalize across input types and worked well overall, but for array-based FUTs they often distinguish small input changes without capturing genuinely different types of behavioral transitions. For structured inputs, better FUT-specific descriptors may therefore be needed, potentially with LLM support. 24

Second, the Coordinator does not always know when the search has run out of useful directions. For simple or tightly constrained FUTs, such as circle or complexCheck, exploration saturates quickly, yet the system may still request new strategies. A dynamic stopping agent could improve efficiency and cost predictability by tracking saturation signals, such as QD-score change relative to Coordinator invocation frequency, and then halting, redirecting, or escalating to the tester for guidance. Third, the strategy library matters beyond initialization. In our runs, it provided the starting point through the baseline strategy, but it can also serve as a reusable knowledge base: effective strategies can transfer across FUTs, and testers can seed the library with domain-specific strategies when prior knowledge exists. We note that while the strategies produced during our runs are human-readable, we did not empirically evaluate their interpretability to testers, nor did we measure the benefit of transferring strategies across FUTs. Both are natural next steps for validating the strategy library as an accumulating knowledge asset. Taken together, these lessons suggest that ABEX is best viewed as a platform that can improve with use. As the strategy library grows and coordination becomes more selective, exploration knowledge can become a durable asset rather than something recreated in each run. 6.2. Validity Threats Construct validity. We use QD-score as the primary metric, following standard practice in QD evaluation [21], and complement it with boundary-candidate cell count. SETBVE [7] reports RAC and RPD, but these require substantial archive-cell overlap between the compared methods, and overlap in our study is too limited because ABEX and SETBVE explore the space differently. Both QD-score and BC count are, however, proxies for boundary quality and diversity rather than direct measures of fault-detection value. Moreover, the ingredients they are computed from are heuristic choices inherited from prior work [7]: the program derivative used as fitness relies on fixed distance metrics rather than ground-truth measures of boundariness. In particular, we compute output distances using Jaccard distance on stringified outputs, so transitions whose outputs share similar n-gram sets may receive low PD and be undervalued. The behavioral descriptors determine what the archive treats as “diverse”. Alternative fitness functions, output-distance measures, or descriptor sets could therefore lead to different archive structures and potentially shift the results. Since all configurations use the same metrics, these choices affect PD and QD-score magnitudes rather than the fairness of the comparison. To link the discovered candidates to fault detection through a metric-independent lens, we complement the archive-level analysis with mutation testing (RQ4), which confirms that the candidates are fault-revealing. Mutation testing has its own construct limitation: it measures fault detection against syntactic mutants produced by mutmut’s operator set, which are proxies for real faults [45], and since we do not filter equivalent mutants, absolute mutation scores should be read comparatively rather than as fault-detection rates for real defects. Internal validity. We compared ABEX and SETBVE under a fixed budget of generated input pairs rather than wallclock time. This keeps the comparison aligned at the level of exploration effort, but it also favors neither method’s runtime characteristics: SETBVE is faster per iteration because it does not invoke LLMs, so a time-budget comparison would likely improve its raw throughput. At the same time, LLM costs are falling rapidly; Gundlach et al. [48] estimate declines of 5–10× per year, with algorithmic efficiency alone contributing about 3× annually. We therefore report a hardware-agnostic comparison that readers can map to current runtimes and costs. Also, the single-prompt baseline uses a smaller generation budget than ABEX, following prior work [12]. In preliminary experiments, we attempted to scale the single call to larger candidate sets, but the model increasingly produced near-duplicate pairs, so a larger single-call budget does not yield a proportionally stronger baseline. The natural way to scale a single prompt is to invoke it repeatedly, which is precisely the ExplorerOnly configuration (Section 5.5): it executes the same fixed strategy with archive feedback under a full search budget, yet its QD-score and BC count saturate early (Figure 5), with generated pairs becoming repetitive despite archive-based duplicate avoidance. This indicates that the gap between ABEX and single-prompt is not primarily a budget artifact. Finally, both single-prompt and ExplorerOnly rely on a single fixed baseline strategy; a stronger baseline with several fixed strategies would better isolate the contribution of adaptive strategy selection. External validity. We evaluate 20 FUTs across four input categories and a range of program complexities. Although broader than prior BVE benchmarks, all FUTs are small, stateless, side-effect-free, single-function Python units with English-language docstrings; the behavior of ABEX on stateful code, multi-function APIs, larger real-world programs, 25

or non-English specifications remains untested. LLM’s training data contamination is a potential threat, since several subject programs (e.g., tritype, tcas, nextDate) are long-standing testing benchmarks and may appear in training data. This raises the possibility that some boundary knowledge is memorized rather than inferred from the signature and docstring. However, ABEX and the single-prompt baseline use the same model, so memorization alone cannot explain the 2.8× QD-score gap; moreover, the black-box protocol reveals only the signature and docstring, not the implementation. Evaluation on proprietary or newly written functions is still needed to fully rule out contamination. Results may also vary across LLM models and configurations, an underexplored source of instability in LLM-based systems [49]. To partially assess this threat, we evaluated ABEX with three different LLMs: GPT-5.1/mini as a closed-weight proprietary model, and Gemma-4 and Qwen-3.5 as open-weight models. In all cases, ABEX was able to generate boundary exploration strategies and identify boundary candidates. However, performance varied across FUTs, suggesting that different LLMs may be better suited to different types of functions. At minimum, ABEX requires an LLM that can reason about boundary testing and input data types well enough to generate meaningful input mutations. Future work should investigate which LLM characteristics are most effective for different categories of FUTs. Conclusion validity. The main threat here is stochastic variation. We repeated each configuration 10 times per FUT and assessed the main comparisons with non-parametric significance tests and effect sizes. The consistently large Â12 values suggest the reported differences are robust to run-to-run variation. Nevertheless, 10 runs may be too few to characterize variance fully for string, array, and mixed FUTs, where output variability is higher and where no baseline comparison was tested. The ablation and LLM-comparison experiments use a single 500-iteration run per configuration due to cost; their results should therefore be read as indicative. 7. Conclusion We presented ABEX, an agentic LLM framework for automated boundary value exploration that eliminates the need for manually engineered mutation operators and extends BVE beyond numeric inputs. By combining quality-diversity search with adaptive, LLM-driven strategy generation, ABEX enables boundary discovery across heterogeneous input types within a single unified framework. Our results show that ABEX consistently produces higher-quality boundary candidates than both traditional search-based methods and a baseline LLM approach. In numeric domains, it achieves the highest QD-scores on most FUTs, while for non-numeric inputs it discovers meaningful, domain-aligned boundary behaviors without requiring datatype-specific operators. Mutation testing confirms that these boundary candidates translate into fault-detection capability: ABEX achieves higher mutation scores than the QD-based baseline and kills more hard-to-detect stubborn mutants. The analysis further highlights a complementary interplay between strategies: generation expands coverage, while mutation refines boundary quality. ABEX demonstrates that boundary value exploration can move from handcrafted, datatype-specific algorithms to adaptive, strategy-driven systems. This suggests a broader shift: LLM-guided search can serve as a general mechanism for automating complex testing tasks that were previously difficult to scale across input domains. 8. Acknowledgments This work was supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP), funded by the Knut and Alice Wallenberg Foundation. ChatGPT and Claude.ai was utilized to improve the phrasing of some parts of the text, originally written by the authors. The authors reviewed and edited the output and take full responsibility for the content of the article. References [1] L. J. White, E. I. Cohen, A domain strategy for computer program testing, IEEE transactions on software engineering (3) (1980) 247–257. [2] L. A. Clarke, J. Hassell, D. J. Richardson, A close look at domain testing, IEEE Transactions on Software Engineering (4) (1982) 380–390. 26

[3] R. M. Hierons, Avoiding coincidental correctness in boundary value analysis, ACM Transactions on Software Engineering and Methodology (TOSEM) 15 (3) (2006) 227–241. [4] F. Dobslaw, F. G. de Oliveira Neto, R. Feldt, Boundary value exploration for software analysis, in: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE, 2020, pp. 346–353. [5] F. Dobslaw, R. Feldt, F. G. de Oliveira Neto, Automated black-box boundary value detection, PeerJ Computer Science 9 (2023) e1625. [6] R. Feldt, F. Dobslaw, Towards automated boundary value testing with program derivatives and search, in: International Symposium on Search Based Software Engineering, Springer, 2019, pp. 155–163. [7] S. Akbarova, F. Dobslaw, F. G. d. O. Neto, R. Feldt, Setbve: Quality-diversity driven exploration of software boundary behaviors, ACM Transactions on Software Engineering and Methodology (2025). [8] J.-B. Mouret, J. Clune, Illuminating search spaces by mapping elites, arXiv preprint arXiv:1504.04909 (2015). [9] R. Feldt, S. Kang, J. Yoon, S. Yoo, Towards autonomous testing agents via conversational large language models, in: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE, 2023, pp. 1688–1693. [10] J. Yoon, R. Feldt, S. Yoo, Intent-driven mobile gui testing with autonomous large language model agents, in: 2024 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE, 2024, pp. 129–139. [11] S. Hu, C. Lu, J. Clune, Automated design of agentic systems, in: ICLR, 2025. [12] X. Guo, C. Li, T. Tsuchiya, Boundary value test input generation using prompt engineering with llms: Fault detection and coverage analysis, arXiv preprint arXiv:2501.14465 (2025). [13] B. Jeng, I. Forgács, An automatic approach of domain test data generation, Journal of Systems and Software 49 (1) (1999) 97–112. [14] R. Pandita, T. Xie, N. Tillmann, J. De Halleux, Guided test generation for coverage criteria, in: 2010 IEEE international conference on software maintenance, IEEE, 2010, pp. 1–10. [15] Z. Zhang, T. Wu, J. Zhang, Boundary value analysis in automatic white-box test generation, in: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), IEEE, 2015, pp. 239–249. [16] A. Jain, S. Sharma, S. Sharma, D. Juneja, Boundary value analysis for non-numerical variables: Strings, Orient. J. Comp. Sci. and Technol 3 (2) (2010). [17] R. Zhao, M. R. Lyu, Y. Min, Automatic string test data generation for detecting domain errors, Software Testing, Verification and Reliability 20 (3) (2010) 209–236. [18] B. Marculescu, R. Feldt, R. Torkar, Using exploration focused techniques to augment search-based software testing: An experimental evaluation, in: 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), IEEE, 2016, pp. 69–79. [19] R. Feldt, S. Poulding, Searching for test data with feature diversity, arXiv preprint arXiv:1709.06017 (2017). [20] M. Boussaa, O. Barais, G. Sunyé, B. Baudry, A novelty search approach for automatic test data generation, in: 2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing, IEEE, 2015, pp. 40–43. doi:10.1109/SBST.2015.17. [21] Y. Xiang, H. Huang, S. Li, M. Li, C. Luo, X. Yang, Automated test suite generation for software product lines based on quality-diversity optimization, ACM Transactions on Software Engineering and Methodology 33 (2) (2023) 1–52. 27

[22] V. Riccio, P. Tonella, Model-based exploration of the frontier of behaviours for deep learning system testing, in: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pp. 876–888. [23] T. Zohdinasab, V. Riccio, A. Gambi, P. Tonella, Deephyperion: exploring the feature space of deep learning-based systems through illumination search, in: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2021, Association for Computing Machinery, New York, NY, USA, 2021, p. 79–90. doi:10.1145/3460319.3464811. URL https://doi.org/10.1145/3460319.3464811 [24] O. Weißl, A. Abdellatif, X. Chen, G. Merabishvili, V. Riccio, S. Kacianka, A. Stocco, Targeted deep learning system boundary testing, ACM Transactions on Software Engineering and Methodology 35 (7) (2026) 1–31. [25] M. Nass, E. Alégroth, R. Feldt, Improving web element localization by using a large language model, Software Testing, Verification and Reliability 34 (7) (2024) e1893. [26] C. Lemieux, J. P. Inala, S. K. Lahiri, S. Sen, Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models, in: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), IEEE, 2023, pp. 919–931. [27] S. Feng, C. Chen, Prompting is all you need: Automated android bug replay with large language models, in: Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, 2024, pp. 1–13. [28] J. Wang, Y. Huang, C. Chen, Z. Liu, S. Wang, Q. Wang, Software testing with large language models: Survey, landscape, and vision, IEEE Transactions on Software Engineering 50 (4) (2024) 911–936. [29] M. Boukhlif, N. Kharmoum, M. Hanine, M. Kodad, S. N. Lagmiri, Towards an intelligent test case generation framework using llms and prompt engineering, in: International Conference on Smart Medical, IoT & Artificial Intelligence, Springer, 2024, pp. 24–31. [30] Y. Chen, Z. Hu, C. Zhi, J. Han, S. Deng, J. Yin, Chatunitest: A framework for llm-based test generation, in: Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, 2024, pp. 572–576. [31] R. Pan, M. Kim, R. Krishna, R. Pavuluri, S. Sinha, Aster: Natural and multi-language unit test generation with llms, in: 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), IEEE, 2025, pp. 413–424. [32] S. Gu, Q. Zhang, K. Li, C. Fang, F. Tian, L. Zhu, J. Zhou, Z. Chen, Testart: Improving llm-based unit testing via co-evolution of automated generation and repair iteration, arXiv preprint arXiv:2408.03095 (2024). [33] J. Altmayer Pizzorno, E. D. Berger, Coverup: Effective high coverage test generation for python, Proceedings of the ACM on Software Engineering 2 (FSE) (2025) 2897–2919. [34] R. N. Anthony, Planning and Control Systems: A Framework for Analysis, Harvard Business School Press, 1965. [35] Q. Wu, G. Bansal, J. Zhang, et al., Autogen: Enabling next-gen llm applications via multi-agent conversation, in: COLM, 2024. [36] K. Ghani, Searching for test data, Ph.D. thesis, The University of York (2009). [37] X. Guo, H. Okamura, T. Dohi, Optimal test case generation for boundary value analysis, Software Quality Journal 32 (2) (2024) 543–566. [38] N. Williams, B. Marre, P. Mouy, M. Roger, Pathcrawler: Automatic generation of path tests by combining static and dynamic analysis, in: European Dependable Computing Conference, Springer, 2005, pp. 281–292.

28

[39] S. Akbarova, F. Dobslaw, R. Feldt, Understanding on the edge: Llm-generated boundary test explanations, arXiv preprint arXiv:2601.22791 (2026). [40] J. K. Pugh, L. B. Soros, K. O. Stanley, Quality diversity: A new frontier for evolutionary computation, Frontiers in Robotics and AI 3 (2016) 40. [41] R. Ye, X. Liu, Q. Wu, X. Pang, Z. Yin, L. Bai, S. Chen, X-mas: Towards building multi-agent systems with heterogeneous llms, arXiv preprint arXiv:2505.16997 (2025). [42] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, et al., Dspy: Compiling declarative language model calls into self-improving pipelines, arXiv preprint arXiv:2310.03714 (2023). [43] mutmut: Mutation testing system for python, https://pypi.org/project/mutmut/, python package, accessed 2026-07-02. [44] I. T. Elgendy, R. M. Hierons, P. McMinn, How effective are coverage-and diversity-based test selection at killing stubborn mutants?, in: 2026 IEEE Conference on Software Testing, Verification and Validation (ICST), IEEE, 2026. [45] M. Papadakis, M. Kintis, J. Zhang, Y. Jia, Y. Le Traon, M. Harman, Mutation testing advances: an analysis and survey, in: Advances in computers, Vol. 112, Elsevier, 2019, pp. 275–378. [46] A. Arcuri, L. Briand, A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering, Software Testing, Verification and Reliability 24 (3) (2014) 219–250. [47] A. Vargha, H. D. Delaney, A critique and improvement of the cl common language effect size statistics of mcgraw and wong, Journal of Educational and Behavioral Statistics 25 (2) (2000) 101–132. [48] H. Gundlach, J. Lynch, M. Mertens, N. Thompson, The price of progress: Algorithmic efficiency and the falling cost of ai inference, arXiv preprint arXiv:2511.23455 (2025). [49] F. Dobslaw, R. Feldt, J. Yoon, S. Yoo, Challenges in testing large language model based software: A faceted taxonomy, arXiv preprint arXiv:2503.00481 (2025).

29

Related documents

• Adaptive Strategy Generation for Boundary Value Exploration Beyond Numeric Inputs#784981
• Adaptive Strategy Generation for Boundary Value Exploration Beyond Numeric Inputs#783391
• Boundary Value Correction for the Boundary Element Method#1106254
• CAD algorithms for yield optimization of integrated circuits#334312
• Automatisierte Auswertung von DYMOLA-Simulationsergebnissen mit MATLAB#353583