arXiv CSopen access

IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues

IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues

arXiv:2607.17286v1 [cs.SE] 19 Jul 2026

JIAWEI LIU, Shanghai Jiao Tong University, China and Shanghai Innovation Institute, China YUN LIN, Shanghai Jiao Tong University, China CHENYAN LIU, National University of Singapore, Singapore YU QIAN, Shanghai Jiao Tong University, China YIMING LIU, Shanghai Jiao Tong University, China and Shanghai Innovation Institute, China JIAXIN CHANG, Shanghai Jiao Tong University, China WEINAN ZHANG, Shanghai Jiao Tong University, China and Shanghai Innovation Institute, China LINPENG HUANG, Shanghai Jiao Tong University, China Issue localization, which identifies code locations requiring modification from issue descriptions, is a critical step in automated software maintenance. Existing approaches predominantly attempt to directly align issue descriptions with code elements, yet often struggle due to the inherent abstraction gap between the issue description and code implementation. Seeking alternative signals, our theoretical analysis suggests that test suites can serve as executable proxies for requirements, reducing localization uncertainty by 7.73 bits of entropy on average. A large-scale empirical study on 18 repositories validates this premise: existing tests cover 96.98% of ground-truth files, and the two-hop pathway yields stronger semantic connectivity than direct matching in 82.4% of cases. Despite their potential, leveraging tests for localization faces two key challenges: the semantic gap separating issue descriptions from test identifiers, and the substantial noise in execution traces from infrastructure code. To address these, we propose IssueExec, which bridges the semantic gap through domain-knowledge-enhanced test representations and filters noise via hierarchical trace analysis. Experiments on SWE-bench Lite show that IssueExec achieves state-of-the-art performance, improving function-level Recall@1 by 41.57% over the strongest baseline. When integrated into the Agentless pipeline, IssueExec resolves 17.72% more issues, demonstrating practical downstream benefits. CCS Concepts: • Software and its engineering → Test-driven software engineering; • Computing methodologies → Information extraction. Additional Key Words and Phrases: Issue Localization, Test-driven Analysis ACM Reference Format: Jiawei Liu, Yun Lin, Chenyan Liu, Yu Qian, Yiming Liu, Jiaxin Chang, Weinan Zhang, and Linpeng Huang. 2026. IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues. Proc. ACM Softw. Eng. 3, ISSTA, Article ISSTA199 (October 2026), 22 pages. https://doi.org/10.1145/3832290 Authors’ Contact Information: Jiawei Liu, Shanghai Jiao Tong University, Shanghai, China and Shanghai Innovation Institute, Shanghai, China, [email protected]; Yun Lin, Shanghai Jiao Tong University, Shanghai, China, [email protected]; Chenyan Liu, National University of Singapore, Singapore, Singapore, [email protected]; Yu Qian, Shanghai Jiao Tong University, Shanghai, China, [email protected]; Yiming Liu, Shanghai Jiao Tong University, Shanghai, China and Shanghai Innovation Institute, Shanghai, China, [email protected]; Jiaxin Chang, Shanghai Jiao Tong University, Shanghai, China, [email protected]; Weinan Zhang, Shanghai Jiao Tong University, Shanghai, China and Shanghai Innovation Institute, Shanghai, China, [email protected]; Linpeng Huang, Shanghai Jiao Tong University, Shanghai, China, [email protected].

This work is licensed under a Creative Commons Attribution 4.0 International License. © 2026 Copyright held by the owner/author(s). ACM 2994-970X/2026/10-ARTISSTA199 https://doi.org/10.1145/3832290 Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:2

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

Issue #6671 from Moto: Aurora Postgres does not support ServerlessV2ScalingConfiguration (a) Direct Matching — Failure Case

(b) Test-Mediated Matching — Success Case

LLM Agent Issue Description

ServerlessV2 support

LLM Agent Issue Description

ServerlessV2 support serverless

No overlap di#erent abstraction levels

Target Location

_get_db_cluster_kwargs

terminology alignment

Test (Executable Requirement)

test_create_serverless_db_cluster execution coverage

db_cluster

Target Location

(parameter extraction)

_get_db_cluster_kwargs

Weak Semantic Alignment

Strong Semantic Alignment

Fig. 1. Motivation for test-mediated localization. (a) A failure case of direct issue-to-code matching due to weak semantic alignment. (b) A success case using tests as functional bridges to align requirements with target locations.

1

Introduction

Automating issue resolution is a longstanding goal in software maintenance, and recent LLM-based tools have renewed interest in practical automation [22, 24]. A central step is issue localization, which occupies approximately two-thirds of the total debugging time according to empirical studies [8]. Given an issue description and a repository (with its accompanying test suite), this task aims to identify the code locations that should be modified to implement the intended behavior. Recent solutions typically follow a direct issue–code alignment paradigm [12, 35, 40]. In practice, they either (i) treat localization as a retrieval task and rank code elements by similarity to the issue description [10, 20], or (ii) use a structured search procedure, often powered by an LLM agent, to traverse the repository hierarchy, invoke tools, and iteratively narrow down candidate files and functions [32, 57, 61]. Despite their progress, these approaches still hinge on matching requirementlevel language in issues to implementation-oriented identifiers in code, making them brittle when the issue describes behavior while the relevant code is organized by technical concerns [17, 19]. Concretely, issues express behavioral expectations while code identifiers reflect technical organization. As illustrated in Figure 1(a), the issue requests support for “ServerlessV2” (a functional capability), yet the target function _get_db_cluster_kwargs is named after its implementation role in parameter extraction, with no indication of the feature it serves. A natural alternative would be requirement-aware localization, but explicit requirement documentation drifts from implementation and rarely maps to code precisely [31]. Tests, however, must remain synchronized with implementation to pass [64]. As Figure 1(b) shows, the test test_create_serverless_db_cluster shares behavioral terminology with the issue (both reference “serverless” and “db_cluster”), providing a natural intermediate target for retrieval. Executing the retrieved tests further yields execution traces that connect the test intent to the exercised implementation, including the target function. This two-hop pathway (issue → tests → code) can provide a more reliable evidence chain than direct issue–code matching alone. This is not an isolated case: our theoretical analysis suggests that test-driven localization reduces uncertainty by 7.73 bits on average compared to direct retrieval (Section 2). To validate this premise, our empirical study on 18 high-quality open-source Python repositories (Section 3) shows that existing tests cover 96.98% of ground-truth files and 66.70% of Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:3

functions, while the two-hop pathway yields stronger semantic connectivity than direct matching in 82.4% of cases. This effectiveness stems from tests’ unique position in software development: they are both human-readable specifications and machine-executable validators [2, 7, 25]. Each test verifies specific functionality, with identifiers that encode requirement-level semantics [37, 42], while execution traces establish deterministic links to implementation [19, 26, 55]. This duality motivates our key insight: test suites can serve as executable requirements, as test names and assertions provide a requirement-level semantic channel, while execution traces provide a concrete dynamic link from tests to the exercised code. Leveraging test coverage for issue localization introduces two technical challenges: • C1. Domain-Specific Semantic Gap: Retrieving relevant tests from issues requires domain knowledge absent from generic embeddings, e.g., retrieving test_tz_aware_datetime from “connecting wrong time displayed for users in different countries” requires recognizing that “tz” denotes timezone [13, 39, 59]; • C2. Infrastructure Noise and Trace Dilution: Localizing target code from test coverage is nontrivial, as a single test may execute thousands of functions, including infrastructure code irrelevant to the requirement. Moreover, existing tests are all passing before issue resolution, leaving no failing-test information required by spectrum-based fault localization techniques [1, 29]. To address these challenges, we propose IssueExec (Issue Executable), a test-mediated localization framework that treats test suites as executable realizations of issue requirements. In particular, IssueExec tackles C1 by retrieving candidate tests that reflect the issue intent using test representations enhanced with project-specific domain knowledge (e.g., abbreviations and API aliases) mined from historical commits. It then tackles C2 by leveraging the runtime execution traces of the retrieved tests and modeling their execution hierarchy as a trace graph, which helps filter incidental infrastructure and highlight requirement-central code locations. IssueExec combines (1) domain knowledge enhancement via historical commit mining for robust issue–test alignment; and (2) dynamic trace graph modeling for hierarchy-aware trace denoising and localization. Together, these components construct a focused, requirement-centric search space that maintains high recall while substantially reducing context size. On SWE-bench Lite [28], IssueExec boosts the Recall@1 localization performance at the file, module, and function levels by 17.78%, 25.98%, and 41.57%, respectively. When integrated into the Agentless pipeline [57], our approach resolves 17.72% more issues, indicating benefits for downstream patch generation. Our contributions are as follows: • We formalize and operationalize the notion of tests as executable requirements for issue localization, showing how semantic alignment at the test level and execution grounding at runtime jointly bridge the issue–code gap. • We propose IssueExec, a procedure-based framework that enhances test representations with domain knowledge for effective issue-test alignment and leverages execution trace hierarchy to pinpoint suspicious locations. • We demonstrate state-of-the-art issue localization performance on SWE-bench Lite, with 17.72% improvement in end-to-end resolution when integrated into Agentless. The source code, prompts, and additional materials are available on the artifact page [14]. 2

Theoretical Motivation

We conduct a preliminary theoretical analysis and empirical validation to establish whether the test-driven localization paradigm yields meaningful uncertainty reduction over direct issue-to-code matching. We model how uncertainty evolves along the hierarchical retrieval process (issue → Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:4

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

tests → trace-guided code) and validate the predicted gain under method-agnostic conditions [21]. Our goal is to quantify the uncertainty reduction achievable by the test-driven paradigm itself, independent of specific implementation choices, thereby providing principled justification for the subsequent framework design. 2.1

Formal Preliminaries

Setup and Notation. Given an issue description 𝑑 and repository 𝑅, let 𝐿 be the universe of candidate locations (e.g., all functions/methods), with 𝑁 = |𝐿|. Let G ⊆ 𝐿 denote the ground-truth edit set for 𝑑, with |G| > 0. For any issue localization system that returns a ranked list of locations under a predefined budget 𝑘 ret ; the resulting set is denoted as 𝐿ret , where |𝐿ret | = 𝑘 ret . For the test-driven pipeline, we retrieve a fixed-size set of tests T𝑑 , based on the given issue description 𝑑, with a predefined budget 𝑘 test , i.e., |T𝑑 | = 𝑘 test . Each retrieved test 𝑡 ∈ T𝑑 has a dynamic coverage set Cov(𝑡) ⊆ 𝐿. The trace-constrained candidate subspace is the union coverage: Ø 𝐿cov = Cov(𝑡), 𝑁 cov = |𝐿cov |. (1) 𝑡 ∈ T𝑑

Entropy Proxy. We approximate uncertainty with Hartley entropy 𝐻 (C) ≈ log2 |C| [15, 23], where C denotes a candidate set and |C| represents the size of the search space. In our issue localization scenario, C corresponds to the candidate code locations under consideration. Our objective is not to estimate the intractable distribution 𝑃 (𝐿 | 𝑑), but to quantify the relative uncertainty reduction induced by hierarchical retrieval and trace constraints. 2.2

Quantifying Localization Uncertainty

We now formulate the localization uncertainty for two different paradigms to quantify the theoretical advantage of test-driven localization. Specifically, we define 𝐻 direct for the baseline approach that localizes code locations by directly matching the issue description to code, and 𝐻 indirect for our proposed two-stage hierarchical process that leverages tests as executable requirements to perform indirect localization via an issue-to-test and test-to-code mapping. Direct Retrieval Uncertainty. A direct method ranks the entire space 𝐿 and returns 𝐿ret . Let 𝑔ret = |G ∩ 𝐿ret | be the number of ground-truth locations already included in the returned set. We define a partition-based entropy proxy that accounts for whether ground-truth locations fall inside or outside the returned set:   𝑔ret  |G| − 𝑔ret  𝐻 direct = ⊮[𝑔ret > 0] · − log2 + ⊮[|G| − 𝑔ret > 0] · − log2 (2) |𝐿ret | 𝑁 − |𝐿ret | Intuitively, the first term measures the ambiguity among returned candidates that contain (some of) the true edits, while the second term captures the residual uncertainty when some ground-truth edits are not retrieved. Test-Driven Retrieval Uncertainty. Test-driven localization decomposes the search into two stages: (i) retrieving requirement-relevant tests, and (ii) localizing within a trace-constrained subspace. We model uncertainty as the sum of stage-wise entropies, accounting for realistic failure modes. Stage 1: Test selection entropy. Among the retrieved tests T𝑑 , we call a test effective if it covers at least one ground-truth location, i.e., Cov(𝑡) ∩ G ≠ ∅. Let 𝑛 be the number of effective tests in T𝑑 . We define:  𝑛  𝐻 stage1 = ⊮[𝑛 > 0] · − log2 . (3) 𝑘 test This term captures how confidently the pipeline selects tests that are functionally linked to the Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:5

𝑆"'$(

𝐿!"#

𝑆$%& 𝐿$%&

𝑆!"#

𝐿

Fig. 2. Venn diagram illustrating the hierarchical search space partition

true edits. When 𝑛 = 0, the pipeline fails to retrieve any effective test, and the second stage will necessarily incur a large penalty (defined below). Stage 2: Trace-constrained localization entropy. We partition the global space 𝐿 into three disjoint regions induced by the trace subspace and the returned top-𝑘 ret locations. Critically, in the test-driven pipeline, the final retrieved set 𝐿ret is a subset of the trace-covered space, i.e., 𝐿ret ⊆ 𝐿cov . We define the partitions as: 𝑆 ret = 𝐿ret,

𝑆 cov = 𝐿cov \ 𝐿ret,

𝑆 excl = 𝐿 \ 𝐿cov .

(4)

Here, 𝑆 excl represents the excluded regions that are not covered by any retrieved test traces, as shown in Figure 2. Let 𝑁𝑖 = |𝑆𝑖 | and 𝑔𝑖 = |G ∩ 𝑆𝑖 | for 𝑖 ∈ {ret, cov, excl}. We then define:  ∑︁ 𝑔𝑖  𝐻 stage2 = − log2 . (5) 𝑁𝑖 𝑖 ∈ {ret,cov,excl}, 𝑔𝑖 >0

This formulation makes two realistic phenomena explicit: (i) Space compression [50]: when 𝐿cov is much smaller than 𝐿, uncertainty shrinks because 𝑁 cov ≪ 𝑁 ; (ii) Coverage/retrieval penalty: if 𝑔excl > 0 (i.e., some true edits fall outside trace coverage), the term − log2 (𝑔excl /𝑁 excl ) can dominate, reflecting the high uncertainty caused by incomplete trace constraints. We refer to this condition as external failure hereafter. Finally, the hierarchical uncertainty is: 𝐻 indirect = 𝐻 stage1 + 𝐻 stage2 . 2.3

(6)

Entropy Gain Analysis

For each issue, we define the empirical uncertainty reduction: Δ𝐻 = 𝐻 direct − 𝐻 indirect .

(7)

A positive Δ𝐻 indicates that introducing tests and traces reduces the ambiguity of edit localization compared to direct matching, under the same budget of returning top-𝑘 ret locations. The decomposition further reveals two complementary directions for maximizing entropy gain: (1) improving test retrieval precision to increase 𝑛/𝑘 test and reduce the external failure rate (𝑔excl > 0), thereby lowering 𝐻 stage1 ; (2) refining trace analysis to shrink 𝑁 cov while preserving 𝑔ret , thereby lowering 𝐻 stage2 . These insights directly inform our framework design in subsequent sections. 2.4

Theoretical Validation

We compute the above entropy proxies on 340 real-world resolved issues. To ensure a fair comparison that isolates the effect of the paradigm itself, we employ a unified embedding model for both retrieval pathways: (i) direct retrieval, which ranks code locations in 𝐿 by similarity to the issue description, and (ii) indirect retrieval, which first retrieves tests by the same similarity measure, then constrains Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:6

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

the candidate space via execution coverage. Test coverage is obtained from standard execution traces without task-specific filtering. The validation results are shown in Table 1. A consistently positive Δ𝐻 confirms that the Table 1. Pipeline-aware entropy analysis under realistwo-hop pathway (issue → tests → trace- tic test retrieval. Δ𝐻 is the uncertainty reduction (Equaconstrained code) reduces localization uncer- tion 7). “External failure” reports the fraction of issues tainty on average. Notably, even with a high with 𝑔excl > 0 (some ground-truth edits outside trace external failure rate (24.41%), the overall Δ𝐻 coverage). Results use 𝑘 test = 5 and 𝑘 ret = 10. remains strongly positive. For issues without external failure, trace constraints yield substanMetric Value tial gains (Δ𝐻 = 9.21 bits) by compressing the Total Valid Samples 340 search space. For issues with external failure, diMean 𝐻 direct (bits) 11.90 rect retrieval often fails to assign high priority Mean 𝐻 indirect (bits) 4.17 to the ground-truth locations (𝐻 direct = 11.90 Mean 𝐻 stage1 (bits) 0.14 bits), while indirect retrieval still achieves parMean 𝐻 stage2 (bits) 4.04 tial hits within the trace-constrained subspace, Mean Entropy Gain Δ𝐻 (bits) 7.73 resulting in a positive Δ𝐻 of 3.15 bits. These External failure rate (𝑔excl > 0) 24.41% findings validate the theoretical advantage of test-driven localization and motivate the design of IssueExec, which realizes this paradigm through enhanced test retrieval and hierarchical trace analysis. 3

Empirical Study

Theoretical arguments in Section 2 suggest that tests can serve as executable requirements for issue localization by providing (i) requirement-level semantics and (ii) a dynamic link to exercised code. In this section, we validate this premise empirically on a large-scale dataset, and quantify when and how tests provide actionable signals for localization. We test the following hypotheses: • H1 (Coverage feasibility). Existing tests execute the ground-truth edit locations. • H2 (Retrievability). Tests that cover ground-truth locations are more semantically aligned with the issue than non-covering tests. • H3 (Bridging effect). The two-hop pathway Issue→Test→Location provides stronger semantic connectivity than direct Issue→Location matching. 3.1

Setup

The empirical investigation is conducted on a collection of 929 issues sourced from SWE-bench [28] and SWE-bench Gym [41], covering 18 widely-adopted Python repositories (each with ≥500 stars). These projects represent diverse domains including web frameworks, data processing, and scientific computing. To ensure the reliability of dynamic analysis, we employ a Docker-based execution framework. Each repository-issue pair is isolated within a container where the environment is reset to its pre-patch state. We retain only those instances where the test environment is sufficiently robust, defined as having at least 50% of test functions execute successfully, to prevent environment configuration errors from confounding the findings. Function-level coverage traces are collected via instrumentation using the Python sys.settrace mechanism. During the execution of the full test suite, the tracer records call and return events for every function invocation. We apply a filtering layer to exclude standard library calls and third-party dependencies, retaining only repository-internal functions identified by their fully-qualified paths. Ground-truth edit locations are extracted by parsing the pull request diffs associated with each issue, identifying the specific functions modified by developers to resolve the reported problem. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:7

Cumulative Distribution of Test Coverage and Search Space Reduction (H1)

Percentage of Instances (%)

100 80 60 40 20 0

File-level Module-level Function-level Space Reduction

0

20

40

60

Coverage Rate / Reduction Rate (%)

80

100

Fig. 3. Cumulative distribution of test coverage rates (H1). Existing tests cover 96.98% of ground-truth files and 66.70% of ground-truth functions on average, reducing search space to 58.39% of the repository.

Fig. 4. Semantic similarity of tests to issue descriptions (H2). Tests covering more ground-truth locations exhibit higher similarity to the issue, enabling retrieval-based selection.

For semantic analysis, the bge-large-en-v1.5 [58] model is utilized to generate dense vector embeddings. Each test case is transformed into a textual representation by concatenating its docstring, its fully-qualified module path, and its complete function body. These components are truncated to 512 tokens to fit the model context window. Code locations are similarly represented by their fully-qualified signatures. Semantic similarity is then quantified using the cosine similarity between the resulting embeddings. 3.2

Findings

H1. (Coverage feasibility). Figure 3 demonstrates the structural feasibility of using existing tests to reach target locations. By intersecting the dynamic traces of the entire test suite with the groundtruth edit sets, we observe that existing tests provide broad coverage, reaching 96.98% of files and 66.70% of functions requiring modification. This confirms that for the vast majority of issues, the necessary execution signals already exist within the repository infrastructure. Furthermore, the search space reduction is quantified by comparing the cardinality of the union of all test-covered functions against the total functions in the repository. This restriction narrows the candidate pool to 58.39% of the repository on average, achieving high recall while providing effective search space compression for subsequent localization stages. H2. (Retrievability). For test-based localization to succeed, covering tests must be distinguishable from non-covering tests via semantic signals. Our experimental setup compares the similarity of tests that cover ground-truth locations against a baseline of randomly sampled tests from the same repository. After verifying the normality of the similarity distribution via the Shapiro-Wilk test, a paired t-test is conducted [49]. As shown in Figure 4, covering tests exhibit significantly higher similarity to issue descriptions in 90.9% of instances (𝑝 < 0.001; Cohen’s 𝑑 = 0.56), indicating a medium-to-large statistical effect. Additionally, Spearman’s rank correlation analysis between the number of ground-truth locations covered and the issue similarity yields 𝜌 = 0.18 (𝑝 < 0.001). This suggests that semantic relevance can effectively prioritize tests with broader functional coverage of the requirement. H3. (Bridging effect). Our central hypothesis is that tests serve as semantic bridges between abstract issues and technical code. For each (issue, location) pair where coverage exists, we compare the direct similarity 𝑠 direct = sim(issue, location) against a test-mediated pathway strength. We formalize this mediated connectivity using a two-hop geometric mean formulation [21]: √︁ 𝑠 mediated = max sim(issue, 𝑡) · sim(𝑡, location) 𝑡 ∈ Tcover

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:8

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

This formulation ensures that a strong bridge requires balanced semantic associations across both the requirement-to-test and test-to-implementation links. As illustrated in the results, the mediated pathway is stronger in 82.4% of cases (paired 𝑡-test, 𝑝 < 0.001; Cohen’s 𝑑 = 0.93). The substantial effect size confirms that tests act as effective semantic relays, leveraging both requirement-level language and deterministic execution grounding to achieve connectivity that direct issue-to-code matching cannot. 3.3

Implications for IssueExec

These findings directly inform IssueExec’s design. H1 establishes that test coverage provides a high-recall, reduced-entropy search space. H2 confirms that semantic retrieval can identify requirement-relevant tests, though the moderate effect size (𝑑 = 0.56) motivates our domainknowledge enhancement (Section 4.2) to strengthen Issue→Test alignment. H3 validates the twohop localization strategy, while the gap between covering and non-covering code within traces motivates our hierarchical trace analysis (Section 4.4) to filter infrastructure noise. 4

The IssueExec Framework

The previous two sections provide complementary support for our central premise that tests can serve as executable requirements for issue localization: Section 2 motivates the premise from a theoretical perspective, and Section 3 validates it empirically at scale. Building on this premise, we introduce IssueExec, which operationalizes tests as an intermediate layer that connects issue descriptions to candidate code locations. 4.1

Problem Formulation

Given a natural-language issue description 𝑑 and a repository 𝑅 with code entities 𝑉 , issue localization aims to identify the code locations 𝐿 = {𝑙 1, 𝑙 2, . . . , 𝑙𝑘 } ⊆ 𝑉 that need to be modified to resolve the issue [39, 56]. Formally, we seek the optimal 𝐿 ∗ that minimizes |𝐿| such that Patch(𝑅, 𝐿) |= 𝑑. Guided by the preliminary study in Section 2, which confirms the uncertainty reduction achievable via test-driven localization, we instantiate a two-stage formulation. First, we retrieve tests T𝑑 semantically aligned with 𝑑 to maximize the effective test ratio 𝑛/𝑘 test . Second, we leverage the execution coverage Cov(T𝑑 ) to localize 𝐿 ∗ within a trace-constrained, low-entropy search space. Framework Overview. Figure 5 illustrates the framework. IssueExec first performs offline preprocessing to collect test execution traces and mine commit history for enriching test representations with domain knowledge (Section 4.2). At inference time, given an issue description, the localization pipeline proceeds as following stages: ❶ retrieve relevant tests T𝑑 through two-phase filtering (Section 4.3), implementing the first stage of our formulation; ❷ analyze execution traces to identify suspicious locations within Cov(T𝑑 ) (Section 4.4); ❸ refine and rerank candidates to produce the final 𝐿 ∗ (Section 4.5). 4.2

Domain Knowledge Enhancement

The semantic distance between the generic embeddings of the issue description and the tests remains substantial, as default test representations (e.g., function signatures and docstrings) fail to capture domain-specific associations and project-specific terminology. By incorporating domain knowledge 𝐷𝑡 , we bridge the terminology gap between issue descriptions and test identifiers, effectively reducing the semantic gap for requirement-relevant tests. To this end, we extract domain knowledge from the repository’s commit history, which captures the semantic context under which tests were introduced and evolved. For each test function 𝑡, we extract two types of signals. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

O"line Preprocessing

ISSTA199:9

Online Issue Localization 1. Relevant Test Retrieval (Two-Phase)

2. Trace-Guided Analysis LLM-Guided Trace Analysis

Issue Description ( 𝒅 ) OneVsRestClassifier yields

All Tests

incorrect class assignments when trained on ...

Repository & Test Suite

</> Enriched Test

Execution Traces

Enriched Test Representation

𝑮𝒕 Trace Collection (Containerized Execution)

test_ovr_multiclass

doc(𝒕): Check multiclass behavior with … 𝑫𝒕 : ovr→ OneVsRestClassifier

Domain (𝑫𝒕 )

1. test_ovr_ multiclass: # Check multi class ... X = ... y = ... Y = ... classes = ... ...

Stage 1: Lexical Filtering (BM25)

sig(𝒕): …/

sig(𝒕), doc(𝒕), 𝑫𝒕

Domain Knowledge

Relevant Tests 𝑻𝒅 (Top-k)

Stage 2: Semantic Selection

Trace Graph 𝑮𝒕

LLM

Suspicious Locations Critical High Medium

3. Contextual Refinement & Reranking LLM Code-Aware Reranking

Context Retrieval (Module Structure)

Suspicious Locations

Commit History Mining (AST-based Analysis)

Suspicious Locations

Code Context

OneVsRestCla ssifier.fit

Code Context Full Source Code

4. Final Ranked Edit Locations

LLM

Reasoning: Root Cause vs. Symptom Causal Relationships Architectural Appropriateness

Fig. 5. Overview of the IssueExec framework. Left: Offline preprocessing collects test execution traces 𝐺𝑡 via containerized execution and extracts domain knowledge 𝐷𝑡 from commit history to construct enriched test representations. Right: Online issue localization proceeds through 3 steps: (1) relevant test retrieval combining BM25 lexical filtering and LLM-based semantic selection to obtain T𝑑 , (2) trace-guided analysis leveraging execution traces and LLM reasoning to identify suspicious locations with confidence levels, (3) contextual refinement and reranking using module structure and full source code, and outputs final ranked edit locations 𝐿 ∗ for downstream patch generation.

First, we identify the commit that first introduced 𝑡 via AST-based diff analysis, filtering out lowquality commits (e.g., merge commits, bulk refactoring). The associated commit message provides requirement-level semantic context: 𝑚𝑡 = msg(arg min{time(ℎ) | 𝑡 ∈ added(ℎ)})

(8)

ℎ∈ H

where H denotes the commit history and added(ℎ) returns test functions introduced in commit ℎ. Second, we identify co-changed entities, i.e., code locations frequently modified together with 𝑡 across commits: A𝑡 = {𝑙 ∈ 𝑉 | cochange(𝑡, 𝑙) ≥ 𝜏 } (9) where 𝑉 is the set of all code entities, cochange(𝑡, 𝑙) counts the number of commits in which both 𝑡 and 𝑙 were modified, and 𝜏 is a frequency threshold. These entities typically represent requirement-related code that 𝑡 implicitly covers. Finally, we distill domain-specific tokens from these extracted signals to form the test-specific knowledge set 𝐷𝑡 : 𝐷𝑡 = tok(𝑚𝑡 ) ∪ ident(A𝑡 ) \ tok(sig(𝑡)) (10) where tok(·) extracts semantic tokens from the commit message and ident(A𝑡 ) extracts function and class names from co-changed entities. The set difference removes tokens already present in the test signature to avoid redundancy. We construct the enriched representation by concatenating three components for each test: repr(𝑡) = [sig(𝑡); doc(𝑡); 𝐷𝑡 ]

(11)

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:10

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

where sig(𝑡) denotes the fully qualified test identifier and doc(𝑡) is the docstring if present. By incorporating 𝐷𝑡 , domain-specific tokens provide additional semantic alignment between issues and tests beyond what generic embeddings can capture from sig(𝑡) alone, thereby more effectively bridging the terminology gap and reducing the semantic gap for requirement-relevant tests. 4.3

Relevant Test Retrieval

This section implements the first stage of our localization pipeline. Given an issue description 𝑑 and the enriched test representations constructed in Section 4.2, IssueExec employs a two-phase filtering process that retrieves a subset T𝑑 ⊆ T with |T𝑑 | ≪ |T | while maximizing information extraction for downstream localization from the issue. In the lexical filtering phase, we apply BM25 [46] to efficiently reduce the search space from potentially thousands of tests to a manageable candidate set: Tcand = Filter(𝑑, T , 𝑁 )

(12)

where Filter returns the top-𝑁 tests ranked by lexical similarity to the issue 𝑑. This phase prioritizes recall, ensuring that requirement-relevant tests are retained. In the semantic selection phase, we employ an LLM to carefully identify the most relevant subset from the candidates. Given the enriched representations repr(𝑡) that incorporate domain-specific tokens 𝐷𝑡 , the LLM explicitly reasons about the issue’s intent and the tests’ purposes: T𝑑 = Select(𝑑, {repr(𝑡) | 𝑡 ∈ Tcand }, 𝑘)

(13)

where 𝑘 ≪ 𝑁 controls the size of the final retrieved set. The LLM is prompted to analyze which components are likely responsible for the reported issue, which test modules exercise those components, and which specific tests are most likely to cover the root cause. This produces a small set of relevant tests T𝑑 whose execution traces will guide subsequent localization. The LLM is guided by manually designed examples illustrating common pitfalls and selection criteria; the full prompt is provided on the artifact page [14]. This two-phase design strikes a balance between efficiency and quality: BM25 filtering reduces the search space from |T | to 𝑁 candidates efficiently, while LLM selection further refines to 𝑘 ≪ 𝑁 high-quality tests, avoiding the prohibitive cost of applying LLM reasoning to all tests. 4.4

Trace-Guided Localization

The retrieved tests T𝑑 narrow the search space from the entire test suite to a small, requirementrelevant subset. However, test retrieval alone is insufficient for precise localization: the execution traces of these tests typically cover a large portion of the codebase, where the true edit locations constitute only a small fraction (|𝐿 ∗ | ≪ |Cov(T𝑑 )|). While a single test may execute hundreds of functions, its execution trace preserves caller-callee relationships that encode causal structure. We leverage this hierarchical information to distinguish requirement-central code from incidental infrastructure and utility functions. For each retrieved test 𝑡 ∈ T𝑑 , we extract its dynamic execution trace and construct a directed graph 𝐺𝑡 = (𝑉𝑡 , 𝐸𝑡 ) rooted at 𝑡. The nodes are functions covered by the test: 𝑉𝑡 = Cov(𝑡)

(14)

The edge set captures observed caller-callee relationships: 𝑡

𝐸𝑡 = {(𝑢, 𝑣) | 𝑢 → − 𝑣} 𝑡

(15)

where 𝑢 → − 𝑣 denotes that 𝑢 directly calls 𝑣 during execution of 𝑡. Since 𝐺𝑡 may exceed LLM context limits, we apply BFS-based pruning to retain only shallow layers of the call hierarchy. To ensure Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:11

trace quality, we collect complete execution paths only from tests that execute and pass successfully, discarding traces from tests that fail or raise errors to avoid noise from engineering failures. When the issue’s root cause lies outside the coverage of retrieved tests, localization may fail, and we discuss such cases in Section 6.1. We then analyze these execution traces in conjunction with the issue description 𝑑 and the enriched test representations repr(𝑡): 𝑆𝑡 = Analyze(𝑑, repr(𝑡), 𝐺𝑡 )

(16)

where 𝑆𝑡 ⊆ 𝑉𝑡 denotes the set of suspicious locations identified from trace 𝐺𝑡 . Guided by step-bystep instructions, the LLM generates diagnostic reports that analyze potential assertion deviations, hypothesize root causes, trace error propagation paths through the call chain, and identify systematic risks across related functions. Based on these analyses, each location 𝑙 ∈ 𝑆𝑡 is assigned a confidence level conf(𝑙) ∈ {critical, high, medium}. The aggregated suspicious set is: Ø 𝑆= 𝑆𝑡 (17) 𝑡 ∈ T𝑑

By analyzing execution traces with diagnostic reasoning, we aim to acquire the suspicious set 𝑆, which is designed to be sparse (|𝑆 | ≪ |Cov(T𝑑 )|) while maintaining high recall of ground-truth locations, filtering out infrastructure code that contributes little information about the issue’s root cause. 4.5

Refinement and Reranking

Building on the trace-guided suspicious set 𝑆 from Section 4.4, we further refine and rerank candidates by enriching each location with additional contextual information. The coverage-based candidate set may miss relevant locations that are structurally related but not directly covered. To address this, for each file 𝑓 containing at least one suspicious location, we retrieve its structure as context. For modules exceeding context limits, we provide a compressed skeleton that preserves function signatures and class hierarchies while omitting implementation details. Here 𝑆 ′ denotes the expanded candidate set after contextual refinement, obtained via step-by-step guided reasoning over module structure. 𝑆 ′ = 𝑆 ∪ Refine({𝑓 | 𝑓 ∩ 𝑆 ≠ ∅}, 𝑑)

(18)

To prioritize candidates and filter false positives, we retrieve the full source code for each candidate location and employ the LLM to perform code-aware reranking, guided by a designed example showing how to filter non-fix locations and retain coupled locations requiring joint changes, reasoning about root-cause distinctions, inter-candidate causalities, and the architectural appropriateness of each contextual refinement: 𝐿 ∗ = Rerank(𝑆 ′, {code(𝑙) | 𝑙 ∈ 𝑆 ′ }, 𝑑)

(19)

where code(𝑙) denotes the full source code of location 𝑙. The output 𝐿 ∗ is the final ranked list of edit locations for downstream patch generation. This refinement and reranking process ensures that 𝐿 ∗ includes not only directly covered locations but also structurally and semantically related entities, improving recall beyond execution coverage. The reranking stage then prioritizes the most plausible edit locations, improving top-𝑘 precision. 5

Experiments

We evaluate IssueExec on SWE-bench Lite to answer the following research questions: • RQ1 (Localization performance, Section 5.2) How effective is IssueExec for issue localization? Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:12

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

• RQ2 (Issue resolution performance, Section 5.3) Can improved localization benefit downstream issue resolution? • RQ3 (Cost Analysis, Section 5.4) What is the cost-efficiency of IssueExec compared to existing baselines? • RQ4 (Ablation study, Section 5.5) How does each component contribute to the overall performance of IssueExec? 5.1

Experimental Setup

Benchmarks. We evaluate on a widely-adopted benchmark for automated issue resolution. SWEbench Lite [28] contains 300 issues sampled from 11 popular Python repositories, filtered for self-contained problems solvable without extensive codebase knowledge. Baselines. We compare against three categories of methods. Retrieval-based approaches include BM25 [46], mGTE [66], CodeSage [65], and CodeRankEmbed [51]. Procedure-based methods include Agentless [57] and PatchPilot [32]. Agent-based approaches include LocAgent [12], SWE-Agent [61], OrcaLoca [63], OpenHands [54], and MoatlessTools [3]. Metrics. We evaluate localization at three granularities: file, module (class or top-level function), and function. We report Precision (Prec.) and Recall (Rec.) at cutoffs @1, @3 and @5, measuring the ability to identify ground-truth edit locations within the top-ranked predictions. We use resolved rate as the metric for the downstream repair task, and report F1-score for the ablation study. Implementation Details. We use GPT4o-2024-05-13 and Claude3.5-sonnet-20241022 as the base models for all methods to ensure fair comparison. For IssueExec, we set the co-change frequency threshold 𝜏 to 3, the candidate test set size |Tcand | to 200, and the final selected test set size |T𝑑 | to 5. 5.2

RQ1. Localization Performance

Table 2 summarizes localization performance on SWE-bench Lite at file, module, and function granularities, reporting Precision and Recall at @1/@3/@5. IssueExec achieves state-of-the-art performance across all granularities and metrics. Specifically, IssueExec with GPT-4o attains 70.07% file-level Recall@1, outperforming the best baseline Agentless by 17.78%. The improvements are more pronounced at finer granularities: IssueExec achieves 60.58% / 46.72% Precision@1 and 58.39% / 41.07% Recall@1 at module and function levels respectively, representing a notable boost of 25.98% and 41.57% in Recall@1 over the strongest baselines (Agentless at module level and MoatlessTools at function level); it further improves function-level Recall@3 by 23.38% over the strongest baseline. Similar gains hold with Claude, indicating that IssueExec is robust across backbone models, as test-mediated localization (issue→tests→code) reduces reliance on direct issue–code matching. 5.3 RQ2. Issue Resolution Performance To evaluate whether improved localization translates to better end-to-end performance, we integrate IssueExec into the Agentless pipeline [57], replacing its original localization module while keeping the rest unchanged. Table 3 presents the results on SWE-bench Lite. IssueExec-augmented Agentless achieves a resolution rate of 37.67%, compared to 32.00% for the original Agentless, representing 17 additional resolved issues. This improvement demonstrates that superior @1 localization performance, particularly the advantages of fine-grained localization at the function levels, directly benefits downstream patch generation by providing the repair model with a highly focused and relevant context. Unlike agentic search, which must discover relevant tests and infer their connections to implementation through heuristic exploration, IssueExec explicitly decomposes localization into Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:13

Table 2. RQ1. Performance comparison on SWE-bench Lite. Precision and Recall at @1, @3, and @5 are reported in %. Bold: best; Underline: second best. File (%) Type

Method

Precision @1

@3

Module (%)

Recall @1

@3

Precision @1

@3

Function (%) Recall

@5

@1

@3

Precision @5

@1

Recall

@3

@5

@1

BM25 27.37 14.72 27.37 43.43 19.71 12.04 10.35 19.34 29.87 32.79 13.87 7.54 mGTE 35.04 22.44 35.04 63.50 29.93 19.22 16.70 29.38 47.26 50.36 18.25 12.16 Retrieval CodeSage 32.85 19.34 32.85 51.09 26.28 15.88 14.43 25.91 35.95 38.87 14.23 7.91 CodeRankEmbed 28.83 18.31 28.83 52.19 24.82 17.52 15.85 24.27 37.41 40.33 16.42 8.88

5.40 8.91 6.20 6.57

13.26 21.41 25.36 17.15 34.25 41.42 12.77 21.44 27.86 15.82 25.30 30.47

@3

@5

GPT-4o-2024-05-13 Procedure

Agentless PatchPilot

59.49 38.38 59.49 78.27 47.75 25.18 21.30 46.35 62.59 66.42 23.36 19.04 16.53 21.78 43.25 47.26 53.70 39.57 53.70 64.44 47.78 34.38 33.16 45.37 55.86 57.35 30.74 22.78 21.54 26.98 34.51 35.25

Agent

LocAgent SWE-Agent OrcaLoca OpenHands MoatlessTools

57.30 33.21 57.30 77.74 18.61 16.91 13.14 18.43 48.36 57.30 13.87 12.47 10.15 12.90 33.64 43.43 54.74 54.01 54.74 57.66 41.24 40.82 40.77 40.15 43.61 43.61 30.29 29.50 29.45 28.25 32.00 32.00 55.11 53.53 55.11 60.58 44.16 41.91 41.72 43.43 51.45 51.45 10.95 22.16 21.85 10.58 42.03 44.28 31.39 30.41 31.39 31.75 28.83 27.86 26.41 28.28 28.83 29.75 24.09 24.21 23.13 21.90 24.15 24.64 53.65 38.99 53.65 54.74 43.43 40.82 40.62 42.52 44.71 44.71 30.66 27.74 27.41 29.01 30.60 31.25

Procedure IssueExec

70.07 59.43 70.07 78.10 60.58 43.98 43.55 58.39 68.86 70.32 46.72 26.95 25.94 41.07 53.36 55.67

Agentless Procedure PatchPilot

57.71 42.57 57.71 69.70 57.66 26.88 21.83 55.47 67.67 69.13 24.45 17.09 13.87 21.09 40.03 43.65 55.84 37.29 55.84 67.15 50.73 26.15 22.79 48.97 56.69 59.18 24.09 15.27 13.75 20.67 28.45 53.66

Claude-3-5-Sonnet-20241022

Agent

LocAgent SWE-Agent OrcaLoca MoatlessTools

Procedure IssueExec

53.65 27.80 53.65 68.25 34.67 15.39 11.49 34.12 39.05 40.51 10.95 5.96 5.02 10.40 11.86 12.23 47.45 47.45 47.45 70.07 36.86 38.75 37.90 35.58 58.12 61.19 30.66 29.99 25.91 28.22 44.59 47.26 65.69 40.08 65.69 70.80 45.62 27.74 27.12 44.83 48.60 48.60 43.07 32.97 32.41 40.45 51.89 52.25 51.82 46.78 51.82 71.90 42.34 39.96 40.29 41.24 61.68 64.96 33.58 32.36 31.70 30.93 51.19 52.65 69.71 59.85 69.71 76.64 60.22 47.75 47.29 57.30 68.19 68.73 48.18 34.37 33.49 41.74 52.93 53.66

Table 3. RQ2. End-to-end issue resolution on SWE-bench Lite. Method

Resolved

Rate (%)

Agentless (original) Agentless + IssueExec

96 113 (+17 ↑)

32.00 37.67 (+5.67 ↑)

the issue→tests→code pathway and uses execution traces as structured intermediate evidence. By pinpointing exact edit locations rather than providing entire files, IssueExec effectively avoids overwhelming the model with extraneous code, thereby reducing potential distractions during the reasoning process; detailed examples and cases illustrating this effect are provided on the artifact page [14]. 5.4

RQ3. Cost Analysis

We evaluate the cost-efficiency of IssueExec by measuring the average monetary cost per issue on SWE-bench Lite using GPT-4o, with results compared against representative baselines in Table 4. Compared with representative agent-based methods, IssueExec achieves significant cost savings by reducing the average expense per issue by approximately 17.54%, 41.98%, and 46.89% relative to OpenHands, SWE-Agent, and OrcaLoca, respectively, averaging 35.47% across these baselines. Although IssueExec incurs a slightly higher cost than the procedural baseline Agentless due to the overhead of dynamic trace analysis, this modest investment is justified by its superior performance. IssueExec significantly outperforms Agentless in localization accuracy (Section 5.2) and enables Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:14

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

the resolution of 17.72% more issues in downstream tasks (Section 5.3). In summary, IssueExec strikes an optimal balance between resource consumption and diagnostic effectiveness, providing state-of-the-art capabilities with much higher cost-efficiency than complex agents. Table 4. RQ3. Efficiency and cost comparison (per issue, GPT-4o). Bold: best; Underline: second best.

Table 5. RQ4. Ablation study on SWE-bench Lite. F1@3 (%)

Configuration Method

Cost ($)

Agentless OpenHands SWE-Agent OrcaLoca

0.70 1.14 1.62 1.77

IssueExec

0.94

File

Module

Function

IssueExec (full)

65.27

51.07

34.08

w/o Tests w/o Domain Knowledge w/o Trace Analysis w/o Refinement Module Refinement w/o Reranking

52.55 58.47 56.02 62.71 64.54 50.97

41.27 43.47 42.03 45.72 47.92 24.32

21.14 26.38 25.55 28.89 31.74 16.24

5.5 RQ4. Ablation Study We conduct ablation experiments to understand the contribution of each component, reporting F1@3 scores across three granularities in Table 5. Removing the test-based intermediate entirely (w/o Tests), i.e., replacing Relevant Test Retrieval and Trace-Guided Localization with direct retrieval, causes significant degradation, with F1@3 dropping by 12.72, 9.80, and 12.94 percentage points at the file, module, and function levels. This confirms that leveraging tests as structured intermediate evidence is central to IssueExec’s effectiveness. Removing trace analysis results in a performance decrease across all metrics, with an average drop of 8.94% and specific declines of 9.25%, 9.04%, and 8.53% at the file, module, and function levels. These results emphasize that execution-based call hierarchies are essential for establishing causal links between tests and code, effectively mitigating the noise inherent in static analysis. Excluding domain knowledge leads to an average 7.37% reduction in performance. This component facilitates the alignment of issue requirements with project-specific test identifiers by leveraging historical commit data to bridge terminology gaps. The removal of the refinement stage produces a smaller average decline of 4.37%, suggesting that in the majority, the IssueExec achieves robust localization performance within the space covered by existing tests alone. Replacing the full refinement process with module-level refinement limits the search space while preserving performance near the full IssueExec configuration. This observation suggests that when operating under restricted budgets, narrowing the refinement scope can further improve system efficiency with minimal impact on localization accuracy. The substantial performance loss in the absence of reranking is attributed to the F1@3 experimental setting, as this module provides more accurate importance-based ordering of entities. Furthermore, prioritizing the most probable edit locations at the top of the recommendation list reflects a method design that aligns with the requirements of real-world software maintenance tasks. 6

Failure Analysis

While IssueExec achieves promising performance in issue localization, it still fails to identify the correct edit locations in several recurring scenarios. This section focuses specifically on these failure modes. For each failure mode, we qualitatively analyze the underlying design choice or assumption that causes the failure, illustrate it with a representative example, and discuss potential mitigation directions for future work. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

6.1

ISSTA199:15

Localization Failure due to Missing Test Coverage

While IssueExec’s test-driven localization paradigm achieves notable effectiveness in bridging issue descriptions with target code locations, a specific failure mode arises when the repository’s existing test suite fails to adequately cover the buggy code regions. In such scenarios, the hierarchical trace graph 𝐺𝑡 constructed from retrieved test execution cannot propagate localization signals to uncovered code regions, manifesting as Cov(T𝑑 ) ∩ G = ∅, where the union of dynamic coverage from all retrieved tests T𝑑 exhibits no intersection with the ground-truth edit set G. This represents a boundary condition for the H1 (Coverage Feasibility) hypothesis established in Section 3. Even after filtering for repositories with at least 50% test function execution success (Section 3.1), incomplete coverage persists as a structural challenge. This reflects the varying test maturity levels found in real-world codebases, an inherent constraint of the test-driven localization paradigm when applied at scale. class FilePathField(Field): Target Location ... def formfield(self, **kwargs): ... ‘path’: self.path,000000000000000000000000000000000000000 + ‘path’: self.path() if callable(self.path) else self.path ...

Issue Detail

# Issue Title

Uncovered GT Locations

Allow FilePathField path to accept a callable.

# Issue Description:

•

I have a special case where I want to create a model containing the path to some local files ...

Covered Locations

file=models.FilePathField(path=os.path.join(settings.... ……

• •

Now when running manage.py makemigrations it will resolve the path based on the machine it is being run on. ……

•

• • •

PromiseTest::test_FilePathField FilePathFieldTest::test_fix_os_paths FilePathFieldTest::test_allow_folders

Selected Tests 𝑻𝒅

fields/__init__.py:FilePathField.formfield

fields/__init__.py::FilePathField.__init__ fields/__init__.py::FilePathField.get_prep _value forms/fields.py::FilePathField

Trace-Guided Analysis

Fig. 6. Illustration of the coverage gap failure mode on django__django-10924. The selected tests T𝑑 cover auxiliary methods of the target class but do not execute the ground-truth edit location (shown in red), causing it to be excluded from the trace-guided suspicious set 𝑆 throughout all pipeline stages.

Specifically, as depicted in Figure 6, although the retrieval stage surfaces tests that are semantically related to the target functionality, the selected tests T𝑑 collectively cover only auxiliary or adjacent locations, leaving the ground-truth edit method entirely absent from Cov(T𝑑 ). Consequently, the trace-guided analysis can only reason over the covered locations and receives no direct execution evidence for the uncovered ground-truth method, causing it to be excluded from the suspicious set and the final localization output. This case thus exemplifies how semantic relevance at the test-retrieval stage does not guarantee execution reachability at the method level, a distinction that is invisible to retrieval-only approaches but becomes a hard boundary when trace-guided localization is constrained by incomplete coverage. The scope of this limitation is empirically grounded in the H1 analysis of Section 3, which establishes that 33.30% of ground-truth functions fall outside the full test suite’s coverage, indicating that coverage gaps are a non-negligible structural risk in real-world instances. Critically, this boundary cannot be overcome by improving the semantic alignment between issues and tests Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:16

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

(H2) or by strengthening the two-hop bridging pathway (H3), since all downstream stages are fundamentally constrained by what the selected tests execute at runtime. A natural direction for future work is to combine IssueExec with issue-to-test generation techniques [38] that synthesize bug-triggering tests directly from issue descriptions, thereby relaxing the dependency on preexisting test coverage and extending the reach of trace-guided localization to repositories with lower test maturity. 6.2

Localization Failure under Long Call Chains

Even when the ground-truth edit location is covered by the retrieved tests’ execution traces, localization can still fail when the ground-truth node is reachable only through a deep and complex call chain. The execution path from the selected tests to the ground-truth location passes through a large number of intermediate orchestration layers, whose high invocation frequency causes them to dominate the trace graph 𝐺𝑡 and obscure the true fix site. As a result, the localization pipeline fails to recover G in 𝐿 ∗ not due to an absence of execution evidence, but because the signal-to-noise ratio within 𝐺𝑡 degrades with increasing chain depth, rendering the ground-truth node statistically indistinguishable from higher-level infrastructure components. This constitutes a qualitatively distinct failure mode from the coverage gap discussed in Section 6.1. class StandardDomain(Domain): Target Location ... def resolve_numref_xref(self,...): ... logger.warning(_(“no number is assigned for %s: %s)..... + logger.warning(_(“Failed to create a cross reference ... ...

# Issue Title

Issue Detail

v3.3 started generating “WARNING: no number is assigned for table” warnings

# Issue Description: We’ve updated to Sphinx 3.3 in our documentation. and suddenly the following warning started popping up in our builds when ... WARNING: no number is assigned for table: I looked through the changelog but it didn’t seem like there was anything related to numref that was changed,... ……

• • •

Deep Covered Call Trace tests/test_build_latex.py::test_numref -> sphinx/builders/__init__.py::LaTeXBuilder.build_all -> sphinx/builders/__init__.py::LaTeXBuilder.build -> sphinx/builders/__init__.py::LaTeXBuilder.read ... --------------------- truncation boundary -------------------... -> sphinx/domains/std.py::StandardDomain.resolve_xref -> sphinx/domains/std.py::StandardDomain._resolve_numref_xref

tests/test_build_latex.py::test_numref_with_prefix1 -> sphinx/builders/__init__.py::LaTeXBuilder.build_all -> sphinx/builders/__init__.py::LaTeXBuilder.build ... --------------------- truncation boundary -------------------... -> sphinx/domains/std.py::StandardDomain.resolve_xref -> sphinx/domains/std.py::StandardDomain._resolve_numref_xref

Selected Tests 𝑻𝒅

tests/test_build_latex.py::test_numref tests/test_build_latex.py::test_numref_with_prefix1 tests/test_build_latex.py::test_numref_with_language_ja

Trace-Guided Analysis

Fig. 7. Illustration of the deep call chain failure mode on sphinx-doc__sphinx-8474. Although the groundtruth edit location (shown in red) is reachable within Cov(T𝑑 ), it is buried beneath multiple layers of orchestration nodes that dominate the trace-guided suspicious set S, preventing its recovery as a final candidate 𝐿 ∗ . The truncation boundary marks the depth limit imposed by BFS-based trace pruning; nodes below this boundary, including the ground-truth location, are omitted from the trace provided to the trace-guided analysis stage.

As illustrated in Figure 7, the selected tests are semantically well-aligned with the issue and their execution traces do reach the ground-truth edit location through the full build pipeline. However, the trace is dominated by high-level orchestration components that are invoked far more frequently and appear structurally more prominent within 𝐺𝑡 . As a result, the contextual refinement stage retrieves class-level context around these upstream nodes, and the subsequent reranking elevates pipeline-level candidates over the precise domain-level resolver where the actual Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:17

fix resides. This exposes a characteristic difficulty in trace-guided localization, where high-level pipeline components are both highly visible in the trace and semantically plausible given the issue description, making it difficult to distinguish them from the true root-cause location without reasoning over the depth and role of the covered call path. The prevalence of this failure mode is closely tied to the architectural complexity of the target codebase. Projects with deeply layered execution pipelines, such as build systems, compilers, or documentation frameworks, are particularly susceptible, as their runtime behavior is inherently dominated by high-traffic orchestration components that sit far above the actual logic being modified. A promising mitigation direction is to incorporate call-chain depth as an explicit signal during candidate ranking, discounting locations that appear primarily as transitive callers rather than direct implementors of the relevant behavior. Complementarily, future work could leverage static call graph analysis to prune orchestration-layer noise from the trace prior to suspicious set construction, thereby improving the ability to surface deep but precise edit locations. 7

Threats to Validity

Internal Validity The effectiveness of IssueExec’s domain knowledge enhancement depends on the availability and quality of historical commit data. If a repository has insufficient commit history or maintains messy, non-atomic commit logs, the AST-based mining process may fail to extract meaningful domain tokens, potentially weakening the semantic alignment between issues and tests. While our framework implements filtering mechanisms to exclude low-quality commits, the reliance on historical signals remains a potential internal threat. Future work could involve incorporating external documentation or API references to supplement missing local history. External Validity A primary external threat is the generalizability of IssueExec to issues that cannot be connected to runnable tests or partially executable behaviors. IssueExec is designed for the runnable and test-accessible issue localization setting, where tests can serve as executable proxies for requirements. For non-runnable issues, or cases where the issue behavior is not exercised by existing tests, the ground-truth edit locations may not be captured within execution traces. However, our empirical study suggests that even in large-scale repositories, existing tests often exercise core functional paths, providing a high-recall starting point. Future work could combine IssueExec with issue-to-test generation techniques to synthesize bug-triggering tests from issue descriptions, thereby relaxing the dependency on existing test coverage. Another external threat is language and framework support. Currently, our implementation and evaluation focus exclusively on Python repositories and the sys.settrace instrumentation. The characteristics of dynamic tracing and caller-callee relationship extraction may differ in statically typed languages like Java or C++. While the core methodology of test-mediated localization is language-agnostic, the current lack of cross-language validation remains a threat to external validity. Generalizing IssueExec to other languages mainly requires adapting language-specific components, including test collection, execution tracing, and the language frontend, while the core issue-to-testto-code pipeline remains unchanged. We plan to implement support for additional programming languages in future iterations. Construct Validity The process of collecting full execution traces and constructing trace graphs can be computationally expensive for massive software systems. This overhead might raise concerns regarding the practical utility of the tool in rapid CI/CD environments. To mitigate this threat, we emphasize that trace collection is primarily an offline preprocessing step. Furthermore, this process can be optimized through incremental trace updates where only affected tests are re-executed and traced. Such engineering optimizations will ensure that the system remains scalable as the project evolves. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:18

8

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

Related Work

Information Retrieval and Bug Localization. Retrieval-based approaches compute similarity between issues and code to bridge the gap between natural language and formal syntax [4, 67]. Early works relied on sparse retrievers like BM25 [46] utilizing lexical overlap. To improve accuracy, subsequent studies introduced query reformulation with contextual cues [45] or composed richer evidence sources such as code change histories, metadata, and project-specific edit patterns [33, 34, 53, 62]. While effective, these enhancements introduce sensitivity to modeling choices, where configuration substantially impacts localization performance [52]. More recently, dense retrievers such as CodeSage [65] and CodeRankEmbed [51] have emerged to capture semantic relationships beyond keyword matching. This paradigm has been further adapted to specialized scenarios: CoRet [20] incorporates call graph dependencies, BLAZE [10] addresses cross-project settings, and other models target concurrent programs [48]. However, as noted in classic studies, IR-based methods remain largely static proxies for relevance and continue to struggle with the "vocabulary mismatch" problem where informal issue descriptions and formal code syntax lack sufficient overlap [36]. Traditional Fault Localization and Traceability. Our work is fundamentally grounded in the long-standing challenges of Spectrum-based Fault Localization (SBFL) and Traceability Link Recovery (TLR). SBFL techniques, such as Tarantula [29] and Ochiai [1], utilize program spectra from passing and failing tests to rank suspicious code entities. While foundational, empirical studies show that SBFL effectiveness varies significantly across scales and often requires combination with additional signals [16, 27, 30]. Crucially, these methods rely on pre-existing failing tests, limiting their applicability in general issue resolution where such tests are often absent. Complementary research has explored project-specific test generation for requirement validation and test generalization for broader scenario coverage [43, 44]. Traceability Link Recovery (TLR) aims to link requirements to code but remains costly to maintain and recover automatically [4, 6]. Related research in Feature Location [17] and dynamic analysis has demonstrated that execution scenarios and traces can provide direct evidence of functional behavior [18, 47]. However, interpreting these dynamic traces has historically relied on manual effort or complex static analysis. Bridging this gap by automating diagnostic reasoning on execution traces—potentially via LLMs—remains an open challenge for recovering dynamic traceability links at scale. Procedure-based Approaches employ hierarchical pipelines to narrow the search space. Agentless [57] pioneered a three-phase paradigm, while SWE-Fixer [60] streamlines this with BM25 retrieval and reranking. BugCerberus [11] extends to statement-level localization with program slicing, and PatchPilot [32] adds reproduction and refinement stages. Related work also uses formal refinement to guide and verify LLM-generated code [9]. While efficient, these rigid pipelines are prone to irreversible error propagation, particularly when the initial retrieval step relies on suboptimal queries or configurations [45, 52]. Agent-based Approaches frame localization as sequential decision-making. SWE-Agent [61] designs an Agent-Computer Interface for navigation, while OpenHands [54] provides Dockersandboxed environments. Multi-agent systems like MASAI [5] and SWE-Search [3] distribute tasks across sub-agents. Graph-based methods have also emerged: LocAgent [12] constructs heterogeneous graphs with multiple edge types; RepoGraph [40] supports k-hop retrieval; CodexGraph [35] enables Cypher-based querying. Despite their sophistication, these methods navigate structural rather than functional relationships, missing disconnected targets.

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

9

ISSTA199:19

Conclusion

This paper revisits issue localization from a requirement-centric perspective and argues that test suites can serve as executable requirements that bridge the abstraction gap between naturallanguage issues and code. Motivated by both theoretical analysis and large-scale empirical evidence, we propose IssueExec, a test-driven localization framework that leverages domain-knowledgeenhanced test representations for effective issue–test alignment and hierarchical execution trace analysis to filter infrastructure noise and pinpoint requirement-central code. Extensive experiments on SWE-bench Lite demonstrate that IssueExec achieves state-of-the-art localization performance across file, module, and function levels. When integrated into an endto-end issue resolution pipeline, the improved localization quality translates into tangible gains in downstream patch generation, confirming that precise, requirement-aware localization is a critical enabler for automated software maintenance. Our in-depth analysis further reveals that the advantages of test-driven localization are most pronounced for complex, multi-file issues, while also highlighting limitations arising from incomplete test coverage and noisy execution traces. Data Availability The source code of IssueExec, LLM prompts, detailed analysis from the theoretical motivation and empirical study, illustrative case studies, and all other supplementary materials are available at [14]. Acknowledgments This research is supported in part by the National Natural Science Foundation of China (62572300), the Minister of Education, Singapore (MOE-T2EP20124-0017, MOET32020-0004), the National Research Foundation, Singapore and the Cyber Security Agency under its National Cybersecurity R&D Programme (NCRP25-P04-TAICeN), DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-GC-2023-008-1B), and Cyber Security Agency of Singapore under its National Cybersecurity R&D Programme and CyberSG R&D Cyber Research Programme Office, and partially by HUAWEI’s Al Hundred Schools Program using the Ascend AI technology stack. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore, Cyber Security Agency of Singapore as well as CyberSG R&D Programme Office, Singapore. References [1] Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2006. An evaluation of similarity coefficients for software fault localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06). IEEE, 39–46. [2] Gojko Adzic. 2011. Specification by example: how successful teams deliver the right software. Simon and Schuster. [3] Antonis Antoniades, Albert Örwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Wang. 2024. Swe-search: Enhancing software agents with monte carlo tree search and iterative refinement. arXiv preprint arXiv:2410.20285 (2024). [4] Giuliano Antoniol, Gerardo Canfora, Gerardo Casazza, Andrea De Lucia, and Ettore Merlo. 2002. Recovering traceability links between code and documentation. IEEE transactions on software engineering 28, 10 (2002), 970–983. [5] Daman Arora, Atharv Sonwane, Nalin Wadhwa, Abhav Mehrotra, Saiteja Utpala, Ramakrishna Bairi, Aditya Kanade, and Nagarajan Natarajan. 2024. Masai: Modular architecture for software-engineering ai agents. arXiv preprint arXiv:2406.11638 (2024). [6] Thazin Win Win Aung, Huan Huo, and Yulei Sui. 2020. A literature review of automatic traceability links recovery for software change impact analysis. In Proceedings of the 28th International Conference on Program Comprehension. 14–24. [7] Kent Beck. 2003. Test-driven development: by example. Addison-Wesley Professional. [8] Marcel Böhme, Ezekiel O Soremekun, Sudipta Chattopadhyay, Emamurho Ugherughe, and Andreas Zeller. 2017. Where is the bug and how is it fixed? an experiment with practitioners. In Proceedings of the 2017 11th joint meeting on foundations of software engineering. 117–128. Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:20

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

[9] Yufan Cai, Zhe Hou, David Sanán, Xiaokun Luan, Yun Lin, Jun Sun, and Jin Song Dong. 2025. Automated program refinement: Guide and verify code large language model with refinement calculus. Proceedings of the ACM on Programming Languages 9, POPL (2025), 2057–2089. [10] Partha Chakraborty, Mahmoud Alfadel, and Meiyappan Nagappan. 2025. BLAZE: Cross-language and cross-project bug localization via dynamic chunking and hard example learning. IEEE Transactions on Software Engineering (2025). [11] Jianming Chang, Xin Zhou, Lulu Wang, David Lo, and Bixin Li. 2025. Bridging Bug Localization and Issue Fixing: A Hierarchical Localization Framework Leveraging Large Language Models. arXiv preprint arXiv:2502.15292 (2025). [12] Zhaoling Chen, Robert Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor Prasanna, Arman Cohan, and Xingyao Wang. 2025. Locagent: Graph-guided llm agents for code localization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 8697–8727. [13] Agnieszka Ciborowska and Kostadin Damevski. 2022. Fast changeset-based bug localization with BERT. In Proceedings of the 44th international conference on software engineering. 946–957. [14] code-philia. 2025. IssueExec Project Website. https://code-philia.github.io/IssueExec.github.io/. Accessed: 2026-07-17. [15] Thomas M Cover. 1999. Elements of information theory. John Wiley & Sons. [16] Higor A de Souza, Marcos L Chaim, and Fabio Kon. 2016. Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347 (2016). [17] Bogdan Dit, Meghan Revelle, Malcom Gethers, and Denys Poshyvanyk. 2013. Feature location in source code: a taxonomy and survey. Journal of software: Evolution and Process 25, 1 (2013), 53–95. [18] Thomas Eisenbarth, Rainer Koschke, and Daniel Simon. 2001. Aiding program comprehension by static and dynamic feature analysis. In Proceedings IEEE International Conference on Software Maintenance. ICSM 2001. IEEE, 602–611. [19] Thomas Eisenbarth, Rainer Koschke, and Daniel Simon. 2003. Locating features in source code. IEEE Transactions on software engineering 29, 3 (2003), 210–224. [20] Fabio James Fehr, Luca Franceschi, Giovanni Zappella, et al. 2025. CoRet: Improved Retriever for Code Editing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 775–789. [21] Farid Feyzi and Saeed Parsa. 2019. Inforence: effective fault localization based on information-theoretic analysis and statistical causal inference. Frontiers of Computer Science 13, 4 (2019), 735–759. [22] GitHub. 2025. Copilot: Faster, Smarter, and Built for How You Work Now. Accessed: 2025. https://github.blog/ai-andml/github-copilot/copilot-faster-smarter-and-built-for-how-you-work-now/ [23] Ralph VL Hartley. 1928. Transmission of information 1. Bell System technical journal 7, 3 (1928), 535–563. [24] Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology 33, 8 (2024), 1–79. [25] DM Hutton. 2009. Clean code: a handbook of agile software craftsmanship. Kybernetes 38, 6 (2009), 1035–1035. [26] Marko Ivanković, Goran Petrović, René Just, and Gordon Fraser. 2019. Code coverage at Google. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 955–963. [27] Jiajun Jiang, Ran Wang, Yingfei Xiong, Xiangping Chen, and Lu Zhang. 2019. Combining spectrum-based fault localization and statistical debugging: An empirical study. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 502–514. [28] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 (2023). [29] James A Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering. 273–282. [30] Fabian Keller, Lars Grunske, Simon Heiden, Antonio Filieri, Andre van Hoorn, and David Lo. 2017. A critical evaluation of spectrum-based fault localization techniques on a large-scale software system. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS). IEEE, 114–125. [31] Stanislav Levin and Amiram Yehudai. 2017. The co-evolution of test maintenance and code maintenance through the lens of fine-grained semantic changes. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 35–46. [32] Hongwei Li, Yuheng Tang, Shiqi Wang, and Wenbo Guo. 2025. PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verification. arXiv preprint arXiv:2502.02747 (2025). [33] Chenyan Liu, Yufan Cai, Yun Lin, Yuhuan Huang, Yunrui Pei, Bo Jiang, Ping Yang, Jin Song Dong, and Hong Mei. 2024. CoEdPilot: Recommending code edits with learned prior edit relevance, project-wise awareness, and interactive nature. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 466–478. [34] Chenyan Liu, Yun Lin, Yuhuan Huang, Jiaxin Chang, Binhang Qi, Bo Jiang, Zhiyong Huang, and Jin Song Dong. 2025. Learning Project-wise Subsequent Code Edits via Interleaving Neural-based Induction and Tool-based Deduction. In

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

IssueExec : A Test-Driven Approach for Localizing Software Engineering Issues

ISSTA199:21

2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1377–1389. [35] Xiangyan Liu, Bo Lan, Zhiyuan Hu, Yang Liu, Zhicheng Zhang, Fei Wang, Michael Qizhe Shieh, and Wenmeng Zhou. 2025. Codexgraph: Bridging large language models and code repositories via code graph databases. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 142–160. [36] Andrian Marcus and Jonathan I Maletic. 2003. Recovering documentation-to-source-code traceability links using latent semantic indexing. In 25th International Conference on Software Engineering, 2003. Proceedings. IEEE, 125–135. [37] Nachiappan Nagappan, E Michael Maximilien, Thirumalesh Bhat, and Laurie Williams. 2008. Realizing quality improvement through test driven development: results and experiences of four industrial teams. Empirical Software Engineering 13, 3 (2008), 289–302. [38] Noor Nashid, Islem Bouzenia, Michael Pradel, and Ali Mesbah. 2025. Issue2test: Generating reproducing test cases from issue reports. arXiv preprint arXiv:2503.16320 (2025). [39] Feifei Niu, Chuanyi Li, Kui Liu, Xin Xia, and David Lo. 2025. When Deep Learning Meets Information Retrieval-based Bug Localization: A Survey. Comput. Surveys 57, 11 (2025), 1–41. [40] Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, and Dong Yu. 2024. Repograph: Enhancing ai software engineering with repository-level code graph. arXiv preprint arXiv:2410.14684 (2024). [41] Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. 2024. Training software engineering agents and verifiers with swe-gym. arXiv preprint arXiv:2412.21139 (2024). [42] Leandro Sales Pinto, Saurabh Sinha, and Alessandro Orso. 2012. Understanding myths and realities of test-suite evolution. In Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering. 1–11. [43] BINHANG QI, Y Lin, XINYI WENG, YUHUAN HUANG, CHENYAN LIU, HAILONG SUN, Z Jin, and JIN SONG DONG. 2018. Generating Project-Specific Test Cases with Requirement Validation Intention. (2018). [44] Binhang Qi, Yun Lin, Xinyi Weng, Chenyan Liu, Hailong Sun, Gordon Fraser, and Jin Song Dong. 2026. Generalizing Test Cases for Comprehensive Test Scenario Coverage. Proceedings of the ACM on Software Engineering 3, FSE (2026), 4760–4781. [45] Mohammad Masudur Rahman and Chanchal K Roy. 2018. Improving ir-based bug localization with context-aware query reformulation. In Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. 621–632. [46] Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Vol. 4. Now Publishers Inc. [47] Maher Salah, Spiros Mancoridis, Giuliano Antoniol, and Massimiliano Di Penta. 2006. Scenario-driven dynamic analysis for comprehending large software systems. In Conference on Software Maintenance and Reengineering (CSMR’06). IEEE, 10–pp. [48] Shuai Shao and Tingting Yu. 2023. Information retrieval-based fault localization for concurrent programs. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1467–1479. [49] Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3-4 (1965), 591–611. [50] Marius Smytzek, Martin Eberlein, Lars Grunske, and Andreas Zeller. 2025. How Execution Features Relate to Failures: An Empirical Study and Diagnosis Approach. ACM Transactions on Software Engineering and Methodology (2025). [51] Tarun Suresh, Revanth Gangi Reddy, Yifei Xu, Zach Nussbaum, Andriy Mulyar, Brandon Duderstadt, and Heng Ji. 2024. CoRNStack: High-quality contrastive data for better code retrieval and reranking. arXiv preprint arXiv:2412.01007 (2024). [52] Chakkrit Tantithamthavorn, Surafel Lemma Abebe, Ahmed E Hassan, Akinori Ihara, and Kenichi Matsumoto. 2018. The impact of IR-based classifier configuration on the performance and the effort of method-level bug localization. Information and Software Technology 102 (2018), 160–174. [53] Shaowei Wang and David Lo. 2016. Amalgam+: Composing rich information sources for accurate bug localization. Journal of Software: Evolution and Process 28, 10 (2016), 921–942. [54] Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. 2024. Openhands: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407.16741 (2024). [55] Robert White, Jens Krinke, and Raymond Tan. 2020. Establishing multilevel test-to-code traceability links. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 861–872. [56] W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707–740.

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.

ISSTA199:22

J. Liu, Y. Lin, C. Liu, Y. Qian, Y. Liu, J. Chang, W. Zhang, and L. Huang

[57] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489 (2024). [58] Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-Pack: Packaged Resources To Advance General Chinese Embedding. arXiv:2309.07597 [cs.CL] [59] Yan Xiao, Jacky Keung, Kwabena E Bennin, and Qing Mi. 2019. Improving bug localization with word embedding and enhanced convolutional neural networks. Information and Software Technology 105 (2019), 17–29. [60] Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. 2025. Swe-fixer: Training open-source llms for effective and efficient github issue resolution. arXiv preprint arXiv:2501.05040 (2025). [61] John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems 37 (2024), 50528–50652. [62] Klaus Changsun Youm, June Ahn, and Eunseok Lee. 2017. Improved bug localization based on code change histories and bug reports. Information and Software Technology 82 (2017), 177–192. [63] Zhongming Yu, Hejia Zhang, Yujie Zhao, Hanxian Huang, Matrix Yao, Ke Ding, and Jishen Zhao. 2025. Orcaloca: An llm agent framework for software issue localization. arXiv preprint arXiv:2502.00350 (2025). [64] Andy Zaidman, Bart Van Rompaey, Arie Van Deursen, and Serge Demeyer. 2011. Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining. Empirical Software Engineering 16, 3 (2011), 325–364. [65] Dejiao Zhang, Wasi Ahmad, Ming Tan, Hantian Ding, Ramesh Nallapati, Dan Roth, Xiaofei Ma, and Bing Xiang. 2024. Code representation learning at scale. arXiv preprint arXiv:2402.01935 (2024). [66] Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. arXiv preprint arXiv:2407.19669 (2024). [67] Jian Zhou, Hongyu Zhang, and David Lo. 2012. Where should the bugs be fixed? more accurate information retrievalbased bug localization based on bug reports. In 2012 34th International conference on software engineering (ICSE). IEEE, 14–24.

Received 2026-01-30; accepted 2026-04-16

Proc. ACM Softw. Eng., Vol. 3, No. ISSTA, Article ISSTA199. Publication date: October 2026.