arXiv CSopen access

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports QUANJUN ZHANG, School of Computer Science and Engineering, Nanjing University of Science and Technology, China

YI ZHENG and YE SHANG, State Key Laboratory for Novel Software Technology, Nanjing University Nanjing University, China

WEIFENG SUN, Singapore Management University, Singapore HAICHUAN HU, School of Computer Science and Engineering, Nanjing University of Science and Technology, China

arXiv:2607.09123v1 [cs.SE] 10 Jul 2026

CHUNRONG FANG and ZHENYU CHEN, State Key Laboratory for Novel Software Technology, Nanjing University, China LIANG XIAO, School of Computer Science and Engineering, Nanjing University of Science and Technology, China Reproduction tests help developers confirm reported issues and provide executable feedback for issue resolution, yet issue reports in open-source projects rarely include such tests. Recent studies have explored generating issue reproduction tests from issue reports with large language models, but existing approaches largely rely on prompt-based pipelines that retrieve textual context and generate tests. This limits their ability to understand how reported issues behave in repository-scale codebases and to flexibly organize the construction of reproduction tests. In this paper, we propose ReProAgent, a multi-stage agent framework for reproduction test generation from issue reports. Inspired by how developers manually reproduce reported issues, ReProAgent decomposes the task into four agen stages: bug localization, root cause analysis, test planning, and test generation. To support these stages, ReProAgent integrates task-specific tools for task decomposition and reflection, context retrieval from both textual sources and repository graphs, and runtime interaction with the execution environment. Experiments on SWT-bench-lite and SWT-bench-verified show that ReProAgent successfully reproduces 58.43% and 70.30% of issues, outperforming all baselines, with an average cost of $0.14 per instance. For example, when equipped with GPT-5-mini, ReProAgent exceeds OpenHands with the same backbone by 20.43 and 7.90 percentage points, respectively. ReProAgent also generalizes across multiple backbone LLMs and improves downstream issue resolution performance when integrated with existing repair approaches. CCS Concepts: • Software and its engineering → Software testing and debugging. Additional Key Words and Phrases: LLMs, Reproduction Test Generation, Agents, Knowledge Graph Authors’ Contact Information: Quanjun Zhang, [email protected], School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China; Yi Zheng, [email protected]; Ye Shang, yeshang@ smail.nju.edu.cn, State Key Laboratory for Novel Software Technology, Nanjing University and Nanjing University, Nanjing, China; Weifeng Sun, [email protected], Singapore Management University, Singapore; Haichuan Hu, huhaichuan2024@ gmail.com, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China; Chunrong Fang, [email protected]; Zhenyu Chen, [email protected], State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China; Liang Xiao, [email protected], School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2026 ACM. ACM XXXX-XXXX/2026/1-ART1 https://doi.org/1 , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:2

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

ACM Reference Format: Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao. 2026. ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports. 1, 1, Article 1 (January 2026), 23 pages. https://doi.org/1

1

Introduction

Issue reports are the primary mechanism for submitting bugs on open-source platforms (e.g., GitHub), yet they rarely include executable tests that reproduce the reported issues [19]. The absence of such tests makes it difficult for developers to confirm issues before fixing and to validate patches afterward, thereby significantly increasing the cost of software debugging and maintenance [55, 57]. Although software testing has been extensively studied, most existing test generation techniques focus on objectives such as coverage maximization and defect detection [14, 54]. In contrast, relatively little attention has been paid to generating tests that faithfully reproduce real-world issues described in natural language, which often involve complex execution contexts and implicit assumptions. The importance of issue reproduction has been increasingly recognized in recent work, as exemplified by benchmarks such as SWT-Bench [32]. Unlike prior benchmarks that focus on automated program repair [24], SWT-Bench is specifically designed to evaluate the ability of models to generate tests that reproduce real-world issues from natural language descriptions. Each instance is constructed from a real-world pull request and consists of an issue description, the corresponding pre-fix codebase, and a reference patch. The task is to generate fail-to-pass tests, i.e., tests that fail on the original buggy version but pass after applying the ground-truth fix, thereby serving as executable specifications of the reported issue. Such reproduction tests can further facilitate downstream tasks, including but not limited to patch validation and automated repair. These benchmarks underscore the critical role of issue reproduction test in verifying patch correctness and enabling iterative repair workflows [47, 50]. Recently, the community has witnessed growing interest in automatically generating issue reproduction tests from issue reports using Large Language Models (LLMs) [56]. Representative approaches include LIBRO [26], which leverages few-shot prompting with example issue–test pairs, and Issue2Test [34], which incorporates meta-prompting to infer project-specific testing conventions followed by iterative refinement based on execution feedback. AssertFlip [27] adopts a different strategy by first generating passing tests and then inverting assertions, based on the observation that LLMs are more effective at producing valid passing tests. Despite these advances, existing prompt-based approaches largely follow a similar pipeline of context retrieval and test generation, which introduces two fundamental limitations. First, reproducing an issue often requires understanding behavior along execution paths that span multiple modules and files, whereas existing approaches primarily rely on text-based retrieval, making it difficult to capture the crossfile behavioral dependencies involved in the reported issue. Second, generating tests is inherently a multi-step task that requires bug localization, root cause analysis, test setup construction, and test verification, whereas existing approaches largely treat it as direct generation from retrieved context, without explicitly modeling the intermediate planning process. To address these challenges, we propose ReProAgent, a multi-stage agent framework for issue reproduction test generation. Rather than treating reproduction test generation as a fixed prompting pipeline, ReProAgent decomposes the task into multiple agent stages and equips each stage with task-specific tools for autonomous exploration and decision-making. This design enables ReProAgent to support both repository-scale issue understanding and the structured construction of reproduction tests. In particular, ReProAgent integrates three categories of tools: task decomposition and reflection tools for progressively reasoning, code retrieval tools for collecting , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:3

issue-relevant context from textual sources and repository graph, and runtime interaction tools for interacting with the runtime environment. Built upon this toolset, ReProAgent organizes the generation process into four stages: bug localization via hierarchical analysis, root cause analysis via execution path, assertion-aware test planning, and test generation via triadic review. This design is inspired by the workflow developers typically follow when manually reproducing reported issues: they identify issue-relevant code, reason about how the failure is triggered, design the test setup and assertions, and execute the test to check whether the observed failure matches the report. Experiments on SWT-bench-lite and SWT-bench-verified demonstrate that ReProAgent successfully reproduces 58.43% and 70.30% of issues, outperforming all baselines, e.g., improving AssertFlip by 53.76% and 54.51%. ReProAgent also generalizes across multiple backbone LLMs, achieving average rates of 64.37%, 58.47%, 52.11%, and 50.43% with GPT-5-mini, Qwen3-Coder, DeepSeek-V3.2, and GLM-4.6, respectively. Moreover, additional discussions demonstrate that ReProAgent can be integrated with existing repair approaches, increasing the number of successfully resolved issues from 143 to 153 for SWE-agent on SWT-bench-lite. In summary, this paper makes the following contributions: • Multi-stage Agents. We propose ReProAgent, a multi-stage agent framework for repositorylevel issue reproduction test generation, which decomposes the task into bug localization, root cause analysis, test planning, and test generation. • Task-specific Tools. We design a dedicated toolset to support this framework, including task decomposition and reflection tools, context retrieval tools, and runtime interaction tools. • Extensive Evaluation. We conduct extensive experiments on two benchmark datasets against several state-of-the-art baselines. The results show that ReProAgent achieves significant improvements. Ablation studies validate the effectiveness of individual components, and integration with existing repair frameworks further demonstrates its practical value. 2 2.1

Background and Motivation Test Generation

Automated test generation has been extensively studied in software engineering [14]. Existing approaches can be broadly grouped into three paradigms: traditional, learning-based, and LLMbased. Traditional Approaches. These studies are mainly represented by heuristic search and random testing techniques [18, 28, 31, 38, 39]. Representative examples include EvoSuite [17], which uses genetic algorithms to evolve test suites toward predefined coverage objectives, and Randoop [37], which generates method invocation sequences through feedback-directed random testing. These approaches are effective at improving structural coverage, although prior studies have shown that high coverage does not necessarily imply strong bug detection capability [10, 22]. Learning-based approaches. These studies typically formulate test generation as a sequenceto-sequence learning problem over given focal methods. For example, AthenaTest [44] constructs large-scale focal method–test pairs and reports strong performance on Defects4J [25]. A3Test [5] further improves assertion correctness through knowledge injection and test validity verification. LLM-based approaches. Recent studies have shown that LLM can generate unit tests with promising effectiveness, while also motivating a shift from direct one-shot prompting to more structured generation pipelines [12, 20, 23, 40, 48, 52, 53, 59]. Representative approaches further improve this paradigm from different angles. ChatUniTest [11] enhances generation with adaptive focal-context construction and automated validation, CoverUp [6] introduces coverage-guided iterative refinement, and IntUT [33] improves generation by explicitly modeling test intentions such as inputs, mocks, and expected outcomes. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:4

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

Despite their effectiveness, these approaches are not tailored to reproduction test generation. Existing approaches typically assume a given testing target and a functionally correct implementation, with the goal of improving coverage or validating expected behavior. By contrast, our work starts from a natural-language issue report over a buggy repository, and the objective is to generate a failing test that reproduces the reported issue. This setting requires understanding issue semantics, localizing relevant elements, and determining whether the resulting failure truly corresponds to the described issue, making the task substantially challenging. 2.2

Bug Reproduction Test Generation

As a special form of test generation, this task focuses on constructing tests to reproduce issues described in software issue trackers. Existing work can be discussed from three perspectives: benchmarks and task-specific approaches. Benchmarks. SWE-bench [24] is a a representative early benchmark to evaluate the ability of LLMs to resolve real-world issue reports from software repositories. It is built from real-world issues collected from 12 Python repositories and has become a widely used testbed for repository-level software engineering tasks. Built on top of SWE-bench, SWT-bench [32] focuses specifically on bug reproduction test generation. It includes two subsets, SWT-bench-lite and SWT-bench-verified, where the latter is a manually curated subset with verified correctness. These benchmarks provide a common evaluation basis for studying reproduction test generation in realistic repository settings. Task-specific Approaches. These approaches mainly use LLMs to transform issue descriptions into executable reproduction tests through prompting and feedback-based refinement [1, 16, 21, 45, 46]. For example, LIBRO [26] is among the earliest studies in this direction. It adopts few-shot prompting with issue-test pairs, followed by post-processing and reranking to improve generation quality, but makes limited use of repository-specific context. Issue2Test [34] further incorporates root-cause analysis, meta-prompting for project-specific testing conventions, and iterative refinement with execution feedback. AssertFlip [27] explores a complementary strategy. It first generates passing tests that capture the buggy behavior, and then flips assertions to obtain bug-reproducing tests. However, most existing methods still rely on fixed prompting pipelines, primarily textual context construction, or execution feedback without explicit semantic review. Although some methods, such as Issue2Test, incorporate root-cause analysis, and execution feedback, these capabilities are mostly organized as predefined pipeline steps rather than as a task-specific agentic loop. In contrast, ReProAgent organizes reproduction test generation as a multi-stage agentic process that explicitly integrates bug localization, root-cause analysis, assertion-aware test planning, and triadic review with execution feedback. 2.3

General SE Agents

More recent SE agents are designed for broad repository-level software engineering tasks, such as issue resolution [58]. Within these general-purpose workflows, reproduction test generation is often incorporated as an intermediate step for understanding the issue or validating candidate patches. For example, OpenHands [47] employs ReAct-style agents [51] to navigate repositories and construct verification tests. SWE-agent [50] defines an issue-resolution workflow in which bug localization is followed by reproduction test generation. Agentless [49] further reformulates repository-level repair as a fixed workflow, including fault localization, patch generation, and filtering with regression and reproduction tests. These systems demonstrate the value of agentic or workflow-based reasoning for repository-level software engineering. However, because their primary goal is issue resolution rather than faithful reproduction test generation, reproduction tests are usually treated as auxiliary artifacts. As a result, they place limited emphasis on systematically verifying whether the generated tests faithfully reproduce the reported bug. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:5

Title: Messages framework incorrectly serializes/deserializes extra_tags when it's an empty string Description When a message is serialised and then deserialised with any of the built in storage backends, then extra_tags=="" is converted to extra_tags==None. This is because MessageEncoder checks for the truthyness of extra_tags rather than checking it is not None. To replicate the bug

Effect of the bug in application behavior This error occurred in the wild with a template tag similar to the following:

When the message was displayed as part of a redirect, it had been serialised and deserialized which meant that extra_tags was None instead of the empty string. This caused an error. …(Shorten for the paper)

Fig. 1. The django-15347 issue from SWT-bench

2.4

Motivating Example

We use a real-world issue django-15347 to illustrate the challenges in repository-level issue reproduction test generation. As shown in Figure 1, the issue concerns the handling of the extra_tags field in Django’s messages framework. When a message with the field extra_tags="" is serialized and deserialized, the resulting object incorrectly sets extra_tags to None. Although the issue appears simple from its description, reproducing it requires reasoning about the semantic difference between an empty string and a missing field, as well as tracing how this discrepancy propagates through the underlying serialization and deserialization logic. This issue is challenging because the issue report does not explicitly reveal the faulty code location or the bug-triggering execution path. To reproduce it correctly, the model must identify the relevant repository context, connect the serialization and deserialization logic across functions, and construct an assertion that captures the semantic mismatch between "" and None. In our experiments, both LIBRO and Issue2Test fail to generate a correct reproduction test for this issue. This failure highlights two core challenges in reproduction test generation: understanding repository-scale issues and constructing generation stages. ReProAgent is designed to address these challenges through a multi-stage process that combines bug localization, root cause analysis, test planning, and test generation. For this issue, ReProAgent successfully generates the reproduction test shown in Listing 1, which creates a Message object with extra_tags="", serializes and deserializes it, and asserts that the empty string should be preserved rather than converted to None. This example shows why effective issue reproduction test generation requires repository exploration, semantic issue understanding, and failure-aware validation rather than test generation alone. 3 3.1

Approach Overview

As illustrated in Figure 2, ReProAgent is designed as a multi-stage agent framework for repositorylevel issue reproduction test generation. Particularly, ReProAgent equips LLMs with three categories of tools: task decomposition and reflection tools, hybrid context retrieval tools, and runtime , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:6

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

Listing 1. Test generated by ReProAgent for Django-15347 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

import pytest from django . conf import settings from django . contrib . messages . storage . base import Message from django . contrib . messages . storage . cookie import MessageEncoder , MessageDecoder def test_reproduce_issue () : if not settings . configured : settings . configure () original_message = Message (10 , " Here is a message " , extra_tags = " " ) encoder = MessageEncoder () encoded_message = encoder . encode ( original_message ) decoder = MessageDecoder () decoded_message = decoder . decode ( encoded_message ) assert original_message . extra_tags == " " , " Original extra_tags should be empty string " assert decoded_message . extra_tags == " " , " Decoded extra_tags should be empty string " assert original_message . extra_tags == decoded_message . extra_tags , " extra_tags should be preserved "

interaction tools. These tools allow the agent to break down complex objectives, capture relevant code and dependencies, interact with the repository environment, and refine its decisions based on intermediate feedback. Built on top of this toolset and inspired by how developers manually reproduce reported issues, ReProAgent formulates issue reproduction test generation as a four-stage agentic workflow: bug localization via hierarchical analysis, root cause analysis via execution path, assertion-aware test planning, and test generation via triadic review. In the bug localization stage, the agent identifies suspicious files, classes, functions, and code regions related to the reported issue. In the root cause analysis stage, it analyzes relevant implementations, call paths, and dependencies to infer the bugtriggering execution path and underlying cause. In the test planning stage, it examines existing tests and project-specific testing conventions to derive an appropriate reproduction strategy, including setup, invocation patterns, and expected failure conditions. In the test generation stage, it generates candidate reproduction tests, executes them in a Docker-based sandbox, and iteratively refines them based on execution feedback and triadic review. 3.2

Agent Toolset Construction

Issue reproduction test generation over repository-level codebases is inherently challenging, as it requires understanding complex implementations, cross-file dependencies, and dynamic execution behavior. Unlike prior approaches that largely follow a fixed prompting pipeline, ReProAgent adopts a multi-stage agent framework that performs iterative exploration and decision-making throughout the process. To support this framework, we equip the agent with a multi-dimensional toolset at each stage, enabling it to decompose tasks, retrieve relevant context, interact with the repository environment, and refine its decisions based on intermediate feedback. As shown in Table 1, the toolset consists of three categories: task decomposition and reflection tools, hybrid context retrieval tools, and runtime interaction tools. These tools provide complementary support across the pipeline. Task decomposition and reflection tools help the agent break , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

Execution Path-based Root Cause Analysis

Hierarchical Bug Localization Thinking/ Reflection

Thinking/ Reflection

Bug Reproduction Test Generation

Root Cause

Thinking Reflection

Tool Invocation

Iterative Refine

Bug Reproduction Test Planning

Issue

Test

Review Failed Bug Location

Thinking/ Reflection

Repository Code Analysis

Planning

Static Analysis

KG Construction

Developer

Review Passed

Review Result

Ex e Re cuti su on lt

Final Test

Sandbox Execution Environment

Agent Tool Set

val etrie ph R Gra

tree-sitter Codebase

Tool Invocation

Test Execution

Tool Invocation

Tool Invocation

1:7

Exec ution

KG Retrieval

Python

Bash Tool

Docker

Python Env

Shell

Sequential-thinking Tool

Fig. 2. Overview of the ReProAgent framework

down complex objectives and revise intermediate reasoning. Context retrieval tools enable the agent to inspect relevant code and identify dependencies among modules, classes, and functions. Runtime interaction tools allow the agent to observe execution behavior and validate hypotheses in the actual repository environment. Table 1. Equipped Toolset in ReProAgent Tool Name

Tool Category

Function Description

Sequential-Thinking

Task Decomposition and Reflection Tool

Decomposes complex tasks and supports process-level reflection

Grep Glob Read-File Graph-Search

Context Retrieval Tool Context Retrieval Tool Context Retrieval Tool Context Retrieval Tool

Searches file contents based on keywords Matches file names based on keywords Reads file contents with optional range selection Retrieves dependency relations based on a code knowledge graph

Bash Interactive-Python

Runtime Interaction Tool Runtime Interaction Tool

Executes command-line operations Executes interactive Python scripts at runtime

3.2.1 Task Decomposition and Reflection Tools. Issue reproduction test generation requires multistage reasoning, long-horizon planning, and iterative adaptation. Unlike prior approaches that follow a fixed pipeline, ReProAgent adopts a multi-stage agent framework in which each stage performs fine-grained task decomposition and dynamically adjusts its actions according to intermediate results and environmental feedback. To support this process, we incorporate sequential-thinking [8, 9] as an explicit task decomposition and reflection tool. It allows the agent to break down each stage into manageable reasoning steps, revise earlier decisions when necessary, and adapt its strategy as new evidence becomes available. Within ReProAgent, sequential-thinking serves as a high-level controller for structured decomposition and dynamic reflection. At each step, the agent maintains an explicit reasoning trajectory with step indices and estimated total steps, while using explicit control signals to determine whether to continue, revise previous thoughts, or branch into alternative reasoning paths. Concrete actions such as context retrieval, execution, and test generation are delegated to other , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:8

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

tools, and their outputs are fed back into subsequent reasoning steps. This design forms a closed loop of thinking–acting–observing–revising, which improves the stability and interpretability of long-horizon reasoning. 3.2.2 Hybrid Context Retrieval Tools. ReProAgent provides two complementary retrieval mechanisms: command-line keyword-based retrieval and knowledge-graph-based dependency retrieval. Command-line keyword-based retrieval. ReProAgent provides Linux-style retrieval tools that mimic how developers explore repositories from the command line. These tools support repository navigation, file localization, keyword lookup, and line-level source inspection through operations analogous to ls, find, grep, and glob. They allow the agent to quickly build an understanding of repository layout, module organization, and implementation details relevant to the reported issue. To avoid wasting the LLM context window, excessively long outputs are truncated or summarized. In addition, potentially dangerous commands are filtered before execution, and all tool calls are executed inside a Docker-based sandbox to ensure safety. Knowledge-graph-based dependency retrieval. To support deeper semantic understanding and cross-file reasoning, ReProAgent also incorporates a code knowledge graph. The graph is constructed by recursively parsing the repository with Tree-sitter [43] to extract program entities such as classes, function definitions, and function calls from source files. These entities are represented as nodes, while relations such as file membership, class references, and function calls are encoded as edges, forming a cross-file semantic dependency network. Built on top of this graph, ReProAgent provides dependency-oriented retrieval tools such as entity lookup, class structure inspection, cross-file relation querying, and import analysis. Unlike keyword-based retrieval, these tools directly exploit semantic dependencies and return structurally relevant context, improving the agent’s ability to recover call chains, dependency paths, and behaviorally related code regions. 3.2.3 Runtime Interaction Tools. While context retrieval tools focus on static repository information, runtime interaction tools expose the repository’s dynamic behavior during execution. Their purpose is to enable the agent to validate hypotheses, inspect runtime behavior, and revise its decisions based on concrete execution feedback. To support this process, ReProAgent provides two types of runtime interaction tools. The first is a command execution tool, which allows the agent to run repository-level commands inside a Docker-based sandbox. It supports environment inspection, dependency checking, build execution, and test invocation. The second is an interactive execution tool, which allows the agent to execute Python snippets directly within the sandboxed repository. This tool enables fine-grained inspection of function behavior, dependency interactions, and data-flow changes under concrete inputs. 3.3

Bug Localization via Hierarchical Analysis

Given an issue report and a target repository, this module attempts to identify a ranked set of suspicious program elements that are likely related to the reported failure. To this end, we design a multi-stage hierarchical analysis procedure that progressively narrows the search space from suspicious files to fine-grained code lines. At the core of this procedure is a chain-of-thoughtstyle reasoning process, in which the agent incrementally forms, refines, and verifies localization hypotheses based on evidence collected from the issue report and the repository context. Starting with the issue report, ReProAgent first constructs an initial query by extracting key symptoms, error messages, and referenced entities, such as file names, class names, and method names from stack traces. Guided by this initial query, ReProAgent gathers contextual evidence through syntactic and semantic retrieval mechanisms. Specifically, it uses command-line-based text retrieval to search for issue-related files using keywords such as exception names, function names, , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:9

log fragments, line numbers, and configuration parameters. In parallel, it performs knowledgegraph-based dependency retrieval to analyze call chains across files and inter-module dependency structures, thereby inferring how faults may propagate through the system. After identifying suspicious files associated with the issue, ReProAgent reads the relevant implementations using file-reading tools for issue understanding. The retrieved contexts are then iteratively refined through a reasoning-and-retrieval loop, where intermediate hypotheses about potential root causes are generated and used to reformulate queries for subsequent retrieval steps. Once candidate files has been identified, ReProAgent performs hierarchical localization within each file. It first ranks functions or classes according to their semantic relevance to the issue and their structural proximity to previously identified elements, such as those connected through call relations or dependency links. It then further refines the search to the line level by examining code snippets, execution-relevant statements, and context-specific cues, including conditionals, error-handling logic, and data-flow usage. This coarse-to-fine localization strategy enables efficient exploration of large codebases while maintaining localization precision. To further encourage explicit reasoning, ReProAgent requires the agent to provide a naturallanguage rationale for each selected suspicious region, together with a confidence score. These rationales externalize the agent’s reasoning trajectory and make the localization process more interpretable, while the confidence scores offer an additional signal for prioritizing candidates in downstream stages. Finally, the module outputs a ranked list of suspicious locations, each accompanied by a confidence estimate and supporting rationale. These results are then passed to the downstream root cause analysis and test generation stages. 3.4

Root Cause Analysis via Execution Path

After bug localization, ReProAgent performs root cause analysis to infer the failure mechanism behind the reported issue. Given the suspicious locations from the previous stage and the issue description, this stage traces how the fault is triggered and propagated along the execution path, and produces a structured root cause analysis report for downstream issue reproduction test generation. ReProAgent first analyzes the issue report together with the suspicious locations to identify the reported failure symptoms, such as exception types, error messages, incorrect return values, and abnormal state changes. Based on these symptoms, the agent then performs execution path analysis by incrementally retrieving relevant program context, including dependent functions, related classes, and other critical code regions, and reconstructing the execution path from the entry point to the failure point. After reconstructing the execution path, the agent identifies the underlying logical defect. This step focuses on determining what is wrong in the code logic and why it leads to the unexpected behavior described in the issue report. By comparing the intended behavior implied by the issue report with the actual behavior exhibited by the code, the agent identifies common bug patterns such as incorrect condition checks and improper boundary handling. The agent then formulates a root cause hypothesis based on symptom interpretation, execution path analysis, and logical defect inspection. This hypothesis provides an abstract explanation of the failure mechanism: it not only identifies where the defect lies, but also explains why that defect produces the reported symptoms. To improve reliability, ReProAgent further refines this hypothesis through an iterative reason–validate–refine loop. During this process, the agent continuously collects supporting evidence, including relevant code snippets, complete method implementations, call dependencies, and module relationships, and revises the hypothesis until it consistently explains both the observed symptoms and the retrieved program context. Finally, as shown in Figure 3, ReProAgent outputs a structured root cause analysis report with five components: (1) root cause summary, a concise description of the underlying fault mechanism; (2) error category, the defect type, such as logic error, boundary-handling defect, or API misuse; (3) execution path, the key , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:10

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

execution steps from the entry point to the failure point, annotated with corresponding code locations and behaviors; (4) trigger conditions, the environmental constraints required to reproduce the failure; and (5) confidence score, a measure of the reliability of the inferred root cause. Root Cause The bug occurs in MessageEncoder.default() where it checks if obj.extra\_tags: instead of if obj.extra\_tags is not None:. This causes empty string values to be treated as falsy and excluded from serialization. Error Category logic_error Execution Path • Step 1: cookie.py:22 MessageEncoder checks if obj.extra_tags:, which evaluates to False for empty strings. • Step 2: cookie.py:21—24 Because the condition is False for empty strings, extra_tags is not appended to the message list. • Step 3: cookie.py:38 During deserialization, Message(*obj[2:]) is called without extra_tags. • Step 4: base.py:14 Message constructor uses default value None for extra_tags Trigger Condition • Creating a Message object with extra_tags set to an empty string ’’ • Serializing the Message object using MessageEncoder, • Deserializing the Message object using MessageDecoder, • Accessing the extra_tags attribute which will be None instead of the original empty string Confidence: HIGH(95)

Fig. 3. Root Cause Analysis result of Django-15347

3.5

Assertion-aware Test Planning

After root cause analysis, ReProAgent performs test planning to derive structured guidance for downstream test generation. Given the root cause analysis report, this stage identifies the trigger conditions, reproduction setup, and assertions to reproduce the reported issue, thereby reducing structural errors and information omissions that often arise in direct test generation. Specifically, this stage consists of three connected steps: trigger condition analysis, reproduction setup construction, and assertion derivation. In trigger condition analysis, ReProAgent first parses the root cause analysis report to extract key information, such as the root cause summary, issue category, and execution path. Based on this information, it derives the minimal trigger conditions and key constraints required to expose the issue. In reproduction setup construction, ReProAgent determines the preconditions and environmental constraints required for reproduction, such as project initialization procedures, dependency loading mechanisms, necessary context states, and version-specific requirements. It further identifies the minimal test inputs, parameter combinations, and boundary values sufficient to reproduce the issue, using retrieval and runtime interaction tools to ground the plan in the actual repository context. In assertion derivation, ReProAgent determines how the reproduced bug should be verified. It analyzes observable failure signals and runtime outputs that characterize the issue behavior, such as exception messages, stack traces, incorrect return values, and abnormal internal states. Based on these observations, ReProAgent specifies the expected behavior differences , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:11

between buggy and intended execution, and determines the assertions that should be encoded in the reproduction test. Finally, ReProAgent outputs a structured test plan with three components: (1) environment constraints, which specify the setup and initial states for reproduction; (2) test input construction, which defines the minimal inputs and key parameter configurations needed to trigger the bug; and (3) expected assertion, which describes the deviations between buggy and intended behavior and specifies how they should be asserted. 3.6

Test Generation via Triadic Review

After test planning, ReProAgent enters the test generation stage. This stage takes as input the intermediate artifacts produced in the previous stages, including the issue description, suspicious locations, root cause analysis report, and test plan, and outputs an executable issue reproduction test together with its execution result. Its goal is to translate the structured analysis results into a runnable test and, through iterative execution and review, ensure that the generated test can reliably reproduce the reported issue. To generate the reproduction test, ReProAgent assembles the issue description, suspicious locations, root cause analysis report, and test plan into a unified prompt for the agent. Guided by these inputs, the agent constructs a single reproduction test that targets the relevant suspicious code regions, instantiates the necessary setup and input conditions for issue manifestation, and encodes assertions against the expected faulty behavior. The generated test is required to expose the actual issue behavior, that is, it should fail on the buggy version and pass once the issue is fixed. Once a candidate test is generated, ReProAgent writes it into the sandboxed repository and executes it in a Docker-based environment. The execution outputs are then used as feedback for subsequent refinement. If the test passes, it fails to reproduce the issue, and the agent must revise the test based on the observed execution feedback. A failing test, however, is not immediately accepted as a valid reproduction. Instead, ReProAgent further examines whether the observed failure is semantically consistent with the reported issue. Only tests whose failures are confirmed to match the issue description are accepted as valid reproduction tests; otherwise, the agent continues to refine the test through further iterations until the review succeeds or the maximum iteration budget is reached. Issue Reproduced Successfully: True Confidence Score: 0.95 Reasoning The test correctly creates a Message with extra_tags='', serializes it with MessageEncoder, and deserializes it with MessageDecoder. The assertion failure occurs at the exact point expected: comparing decoded_message.extra_tags to an empty string ” Feedback Summary Successfully reproduced the bug - the test correctly fails with decoded_message.extra_tags being None instead of empty string. Recommended Action: Report success.

Fig. 4. Review result of Django-15347

To determine whether a failing test truly reproduces the reported issue, we introduce a triadic review procedure over the issue report, the generated test, and the execution result, as shown in Figure 4. The procedure evaluates: (1) whether the generated test is semantically aligned with the issue report in its targeted functionality, triggering conditions, and assertions; (2) whether the observed failure is consistent with the test logic rather than caused by incidental factors such as syntax errors, missing dependencies, or environment misconfiguration; and (3) whether the , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:12

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

execution result matches the issue report in terms of error message, exception type, or overall failure pattern. If a generated test passes review, it is retained as a valid issue reproduction test and can be used by developers for inspection, validation, and regression checking. Otherwise, the agent uses the review result together with the prior execution feedback and intermediate analysis artifacts to further refine the test, including its preconditions, input construction, and assertions, and then re-executes the revised version. In this way, ReProAgent forms a closed-loop framework based on feedback-driven iteration and triadic review, enabling the agent to progressively converge to a high-quality test that reliably reproduces the target issue. 4 4.1

Experimental Setup Research Questions

To evaluate the effectiveness of ReProAgent, we conduct experiments to answer the following research questions (RQs): • RQ1: How does ReProAgent perform compared to state-of-the-art approaches? • RQ2: What is the impact of different LLMs on the generalizability of ReProAgent? • RQ3: What are the contributions of different components in ReProAgent? 4.2

Datasets

We evaluate ReProAgent on two subsets from SWT-Bench [32]: SWT-Bench-Lite and SWT-Bench-Verified. SWT-Bench is a large-scale benchmark for evaluating the capability of models and code agents to generate reproducible tests for real-world issues collected from GitHub repositories. SWT-Bench-Verified is a curated subset comprising human-verified tasks with clear descriptions and well-specified reproduction criteria, providing a higher-confidence evaluation set. SWT-Bench-Lite is a smaller, lightweight subset designed for rapid iteration and efficient benchmarking. Both subsets have been widely adopted in recent work on reproducible test generation for real-world issues [2, 26, 27, 34]. 4.3

Baselines

We compare ReProAgent with several representative baselines from two categories. First, we include task-specific reproduction-test generation methods. Zero-Shot-Plus [32] is a zero-shot prompting baseline introduced by SWT-Bench. LIBRO [26] adopts few-shot prompting with issue–test pairs, followed by post-processing and reranking to select high-quality candidate tests. Issue2Test [34] employs a multi-stage pipeline with root-cause analysis, meta-prompting for project-specific testing conventions, and iterative refinement with execution feedback. Otter [2] performs bug localization and uses reflection-based planning to iteratively generate and refine tests. Otter++ [2] extends Otter with heterogeneous prompt sampling and ensemble-style selection. eOtter and e-Otter++ [3] further incorporate execution-feedback-guided generation and selection. AssertFlip [27] first generates passing tests that capture buggy behavior and then inverts their assertions to obtain bug-reproducing tests. Second, we include general-purpose software engineering agents. AutoCodeRover [60] is an LLMbased issue repair framework adapted in SWT-Bench to produce bug reproduction tests through modified instructions. SWE-Agent [50], SWE-Agent+ [32], Aider [4], and OpenHands [47] are coding agents that can be adapted to generate reproduction tests. Amazon Q [7] is an LLM-based development assistant, and we use its publicly reported results on the SWT-Bench leaderboard. 4.4

Evaluation Metrics

Following the evaluation protocols widely adopted in prior reproduction test generation work [32], we assess the effectiveness of ReProAgent using the Fail-to-Pass (FP) Rate. Given one generated , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:13

test per issue, the FP Rate measures the proportion of tests that both reproduce the reported issue and validate its corresponding patch, i.e., tests that fail on the pre-patch (buggy) version of the code and pass after the issue-resolving patch is applied. This metric directly captures whether a generated test correctly characterizes the buggy behavior and serves as a reliable test after the fix. 4.5

Implementation Details

To implement ReProAgent, we adopt GPT-5-mini [36] as the primary backbone model. GPT-5 mini belongs to OpenAI’s GPT-5 model family, which is designed to support coding and agentic tasks involving long-context reasoning and tool interaction. This makes it suitable for our multistage workflow, where the agent must localize bugs, analyze execution paths, plan assertions, and iteratively refine reproduction tests based on runtime feedback. We also instantiate ReProAgent with other advanced code-capable LLMs, including DeepSeek-V3.2, GLM-4.6, and Qwen3-Coder, to evaluate whether the proposed framework generalizes across different backbone models. The overall framework is built upon LangChain [29] and LangGraph [30], which are used to orchestrate multi-stage reasoning and tool interactions. We configure the LLM with temperature = 0.7 and top_p = 0.8; for open-source backbones, we additionally set top_k = 20 and repetition_penalty = 1.05 when these decoding parameters are supported. To control computational cost, we enforce a bounded interaction budget. Each stage allows up to 50 LLM calls. In the test generation stage, which incorporates feedback-driven refinement and automated validation, the maximum number of feedback iterations is limited to 20. All experiments are conducted on a server running Ubuntu 22.04. For knowledge graph construction, we utilize Neo4j [35], which enables efficient representation and querying of code structure and dependencies. 5 5.1

Evaluation and Results RQ1: Comparison with State-of-the-Arts

Experimental Design. We evaluate the effectiveness of ReProAgent on two widely used benchmarks, SWT-bench-lite and SWT-bench-verified. Following prior SWT-bench studies, we adopt FP Rate as the primary evaluation metric, which measures whether a generated test fails on the buggy version and passes on the patched version. We compare ReProAgent with representative prompting-based and agent-based baselines, including LIBRO, Issue2Test, Otter, Otter++, OpenHands, and AssertFlip, as well as additional systems with publicly reported SWT-bench leaderboard results. Unless otherwise specified, the baseline results are taken from the corresponding original papers or the public SWT-bench leaderboard. To control for backbone-model effects, we further include a same-backbone comparison with the strongest baseline OpenHands under GPT-5-mini. Specifically, on SWT-bench-lite, we reproduce OpenHands using the recommended SWT-bench configuration [41], while on SWT-bench-verified, we use the result reported on the SWT-bench leaderboard. Experimental Results. Table 2 and 3 present the overall comparison results. ReProAgent achieves the best FP Rate on both benchmarks, reaching 58.43% on SWT-bench-lite and 70.30% on SWT-bench-verified. On SWT-bench-lite, this result is substantially higher than all compared baselines, where the strongest baseline reaches 38.0%. On SWT-bench-verified, ReProAgent also outperforms the best baseline result of 62.4%. The advantage of ReProAgent is also consistent when compared with recent issue-to-test generation methods. For example, it improves over Issue2Test by 92.01% on SWT-bench-lite and 110.92% on SWT-bench-verified, and over Otter++ by 119.08% and 118.32%, respectively. Compared with AssertFlip, which generates passing tests before inverting assertions, ReProAgent still yields clear improvements on both datasets. Overall, these results indicate that the proposed , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:14

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

Table 2. Comparison of ReProAgent with baselines on SWT-bench-verified. Method

Backbone

Zero-Shot-Plus LIBRO OpenHands Otter Issue2Test Otter++ AssertFlip Amazon Q OpenHands

GPT-4/GPT-4o GPT-4o Claude 3.5 Sonnet GPT-4o GPT-4o-mini GPT-4o GPT-4o Amazon Bedrock GPT-5-mini

ReProAgent

GPT-5-mini

FP Rate 14.3% (↑ 391.61%) 17.8% (↑ 294.94%) 27.7% (↑ 153.79%) 31.6% (↑ 122.47%) 33.33% (↑ 110.92%) 37.4% (↑ 87.97%) 45.5% (↑ 54.51%) 51.0% (↑ 37.84%) 62.4% (↑ 12.66%) 70.30%

Table 3. Comparison of ReProAgent with baselines on SWT-bench-lite. Method

Backbone

AutoCodeRover Zero-Shot-Plus SWE-Agent SWE-Agent Aider LIBRO SWE-Agent SWE-Agent+ Otter OpenHands Otter++ e-Otter Otter++ Issue2Test e-Otter AssertFlip OpenHands Amazon Q e-Otter++

GPT-4 GPT-4/GPT-4o GPT-4o mini Claude 3.5 Sonnet GPT-4 GPT-4o GPT-4 GPT-4 GPT-4o Claude 3.5 Sonnet GPT-4o GPT-4o Claude 3.7 Sonnet GPT-4o-mini Claude 3.7 Sonnet GPT-4o GPT-5-mini Amazon Bedrock GPT-4o

ReProAgent

GPT-5-mini

FP Rate 9.1% (↑ 542.09%) 9.4% (↑ 521.60%) 9.8% (↑ 496.22%) 12.3% (↑ 375.04%) 12.7% (↑ 360.08%) 14.1% (↑ 314.40%) 15.9% (↑ 267.48%) 18.5% (↑ 215.84%) 23.33% (↑ 150.45%) 28.3% (↑ 106.47%) 29.0% (↑ 101.48%) 29.0% (↑ 101.48%) 30.4% (↑ 92.20%) 30.43% (↑ 92.01%) 36.0% (↑ 62.31%) 38.0% (↑ 53.76%) 38.0% (↑ 53.76%) 39.9% (↑ 46.44%) 40.2% (↑ 45.35%) 58.43%

multi-stage framework is more effective than directly generating tests from retrieved context or relying primarily on post-hoc refinement. Effectiveness across repositories. Tables 4 and 5 present the results across different projects. ReProAgent shows stable performance across a diverse set of projects rather than concentrating its gains on only a small subset of instances. On SWT-bench-lite, it achieves reproduction rates above 50% on several major repositories, including django (64.6%), sympy (63.4%), and scikit-learn (68.4%). On SWT-bench-verified, the same trend remains, with particularly strong results on django (74.1%), sympy (74.0%), matplotlib (71.9%), scikit-learn (91.7%), and astropy (75.0%). The results also show that performance is not uniform across repositories. Some low rates should be interpreted cautiously because they are computed from very few instances. For example, flask has only one instance in SWT-Bench-Lite, and Issue2Test also gets 0% on it. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:15

Table 4. Distribution of results across different projects on SWT-bench-lite Repository

Total Issues

Reproduced

Rate

django/django sympy/sympy matplotlib/matplotlib scikit-learn/scikit-learn pytest-dev/pytest sphinx-doc/sphinx pydata/xarray astropy/astropy mwaskom/seaborn pylint-dev/pylint pallets/flask psf/requests

113 71 23 19 11 11 5 4 4 3 2 1

73 45 11 13 3 2 2 3 2 1 0 1

64.6% 63.4% 47.8% 68.4% 27.3% 18.2% 40.0% 75.0% 50.0% 33.3% 0.0% 100.0%

Total

267

156

58.43%

Table 5. Distribution of results across different projects on SWT-bench-verified Repository

Total Issues

Reproduced

Rate

django/django sympy/sympy matplotlib/matplotlib sphinx-doc/sphinx scikit-learn/scikit-learn astropy/astropy pydata/xarray pytest-dev/pytest pylint-dev/pylint psf/requests mwaskom/seaborn pallets/flask

216 73 32 28 24 16 15 15 5 4 2 1

160 54 23 6 22 12 10 11 2 3 0 0

74.1% 74.0% 71.9% 21.4% 91.7% 75.0% 66.7% 73.3% 40.0% 75.0% 0.0% 0.0%

Total

431

303

70.30%

Overlap Analysis. To further understand whether the gain of ReProAgent mainly comes from solving the same easy instances more reliably, we analyze the overlap of successful cases between ReProAgent and representative baselines. Figure 5 presents a overlap analysis among ReProAgent, AssertFlip, OpenHands, LIBRO, and Issue2Test on SWT-bench-lite. The results show that ReProAgent uniquely reproduces 36 issues that are not reproduced by any of the four baselines. This indicates that the improvement of ReProAgent is not limited to solving alreadyeasy cases more reliably; instead, it expands the set of issues for which valid reproduction tests can be generated. Answer to RQ1: ReProAgent performs best on both SWT-bench-lite and SWT-benchverified, reaching 58.43% and 70.30%, respectively, and it uniquely reproduces 36 issues not reproduced by any of the four representative baselines on SWT-bench-lite.

, Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:16

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

AssertFlip 14 2 ReProAgent

18 36

4

2

0 0

6

OpenHands

1 3

7

6

1

12

12 9

2

14

3

0

16 10

7

2

1

1

0

6 0

1

Issue2Test

LIBRO

Fig. 5. Overlap analysis against baselines

Table 6. Performance across different backbone LLMs

FP Rate LLM

5.2

SWT-bench-lite

SWT-bench-verified

Average

GPT-5-mini DeepSeek-V3.2 GLM-4.6 Qwen3-Coder

58.43% 49.64% 48.19% 56.88%

70.30% 53.58% 52.66% 60.05%

64.37% 52.11% 50.43% 58.47%

Average

53.29%

59.15%

56.35%

RQ2: Generalizability across Different LLMs

Experimental Design. To evaluate whether ReProAgent generalizes across different backbone models, we further implement ReProAgent with three open-source LLMs: DeepSeek-V3.2 [15], GLM-4.6 [42], and Qwen3-Coder. These models differ in scale and training emphasis, while all provide strong support for code generation and agentic reasoning. We run the same ReProAgent pipeline with each backbone on both SWT-bench-lite and SWT-bench-verified, and use FP Rate as the evaluation metric. Experimental Results. Table 6 reports the results across different LLM backbones. Overall, ReProAgent achieves strong performance with all four models, obtaining average FP Rates of 64.37%, 58.47%, 52.11%, and 50.43% with GPT-5-mini, Qwen3-Coder, DeepSeek-V3.2, and GLM-4.6, respectively. Among them, GPT-5-mini achieves the best performance, reaching 58.43% on SWTbench-lite and 70.30% on SWT-bench-verified. Qwen3-Coder ranks second with 56.88% and 60.05%, while DeepSeek-V3.2 and GLM-4.6 also obtain competitive results on the two benchmarks. These results indicate that the effectiveness of ReProAgent does not depend on a single backbone model. Instead, the proposed framework remains effective across LLMs with different model sizes and training characteristics. In particular, even the weaker backbones still achieve competitive FP Rates, suggesting that the gains of ReProAgent mainly come from the framework design rather than from a specific model alone. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:17

Table 7. Impact of different stages in ReProAgent with Qwen3-Coder

SWT-bench-lite

SWT-bench-verified

w/o Hierarchical Bug Localization w/o Execution Path-based Root Cause Analysis w/o Test Generation Planning w/o Feedback Iteration and triadic review

47.83% 53.26% 52.17% 18.84%

55.89% 56.58% 57.74% 22.86%

Ours (Full Method, Qwen3-Coder)

56.88%

60.05%

Variant

The stronger performance of GPT-5-mini is likely due to its stronger coding, long-context reasoning, and tool-use capabilities, which better support repository understanding, structured reasoning, and execution-guided test generation. Nevertheless, the overall trend shows that ReProAgent transfers well across different advanced LLMs. Answer to RQ2: ReProAgent generalizes well across different backbone LLMs, achieving average FP Rates of 64.37%, 58.47%, 52.11%, and 50.43% with GPT-5-mini, Qwen3-Coder, DeepSeekV3.2, and GLM-4.6, respectively. Across the four backbones, the overall average FP Rate reaches 56.35%. 5.3

RQ3: Contribution of Different Components

Experimental Design. To examine the contribution of each stage in ReProAgent, we conduct an ablation study by removing each stage in turn from the full framework. Since full ablation with GPT-5-mini would require substantially higher inference cost, we perform this study with Qwen3-Coder, the strongest open-source backbone in RQ2. Specifically, we evaluate four core stages, including hierarchical bug localization, execution path-based root cause analysis, test generation planning, and feedback iteration with triadic review. The resulting variants are evaluated on both SWT-bench-lite and SWT-bench-verified using FP Rate. Experimental Results. Table 7 presents the ablation results. Removing any stage leads to a decline in FP Rate on both benchmarks, indicating that all components contribute to the effectiveness of ReProAgent. Among them, feedback iteration with triadic review has the largest impact: removing this stage causes FP Rate to drop sharply from 56.88% to 18.84% on SWT-bench-lite and from 60.05% to 22.86% on SWT-bench-verified. This result highlights the importance of execution-based refinement and review in correcting invalid tests and ensuring that generated tests faithfully capture the intended fail-to-pass behavior. The other three stages also provide consistent benefits. Removing hierarchical bug localization reduces FP Rate to 47.83% on SWT-bench-lite and 55.89% on SWT-bench-verified, suggesting that fine-grained localization helps identify code regions that are more relevant to the reported issue. Removing execution path-based root cause analysis lowers FP Rate to 53.26% and 56.58%, respectively, showing that structured reasoning over fault propagation improves downstream test construction. Removing test generation planning further decreases FP Rate to 52.17% and 57.74%, indicating that explicit planning provides useful guidance even after the preceding analysis stages. As bug localization plays an important role in providing suspicious code for subsequent stages, we further analyze how its correctness affects the final fail-to-pass performance. Table 8 groups instances according to whether ReProAgent correctly localizes the buggy code. The results show that correct localization substantially improves the final FP rate, increasing it from 50.7% to 67.8% on SWT-Bench-Lite and from 60.8% to 85.3% on SWT-Bench-Verified. This confirms that localization , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:18

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

Table 8. Sensitivity of ReProAgent to localization correctness.

Dataset

Localization

FP Rate

SWT-Bench-Lite SWT-Bench-Lite SWT-Bench-Verified SWT-Bench-Verified

Correct Incorrect Correct Incorrect

67.8% (82/121) 50.7% (74/146) 85.3% (145/170) 60.8% (158/260)

Table 9. Effect of reproduction tests on existing repair work

Patch Pass Rate Framework

Dataset Before

After

SWE-agent + Qwen3-Coder

SWE-bench-lite SWE-bench-verified

143 / 300 153 / 300 317 / 500 327 / 500

Agentless + Qwen3-Coder

SWE-bench-lite SWE-bench-verified

93 / 300 188 / 500

100 / 300 201 / 500

quality is an important factor in generating effective reproduction tests. Meanwhile, incorrectly localized cases still achieve non-trivial FP rates, suggesting that later stages can partially recover through additional context retrieval and execution-path reasoning. These results also indicate that ReProAgent can further benefit from stronger localization modules in future work. Answer to RQ3: All four stages contribute positively to ReProAgent, e.g., with feedback iteration with triadic review having the largest effect. Removing this stage causes the FP Rate to drop from 56.88% to 18.84% on SWT-bench-lite and from 60.05% to 22.86% on SWT-benchverified. 6 6.1

Discussion Reproduction Tests in Issue Repair

Issue reproduction tests are valuable for automated repair because they serve as a critical criterion for assessing whether a generated patch is actually correct and supporting feedback-driven patch refinement. Motivated by this role, we further investigate how issue reproduction tests generated by ReProAgent can support downstream repair from two perspectives: improving existing repair frameworks and enabling feedback-driven iterative repair. Benefit to Existing Repair Frameworks. We integrate ReProAgent into two representative repair frameworks: agent-based SWE-agent [50], and workflow-based Agentless [49]. For SWEagent, we inject tests from ReProAgent into its prompts. For Agentless, we use tests from ReProAgent in the patch filtering stage alongside its original regression tests. Considering the cost of running full repair pipelines, we conduct this analysis with Qwen3-Coder, the open-source backbone in RQ2. As shown in Table 9, the generated reproduction tests consistently improve repair performance across both frameworks. On SWE-bench-lite, the number of successfully resolved issues increases from 143 to 153 for SWE-agent and from 93 to 100 for Agentless. On SWE-bench-verified, the corresponding numbers rise from 317 to 327 and from 188 to 201, respectively. These improvements , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:19

Table 10. Effect of reproduction tests on iterative repair

Patch Pass Rate Method SWE-bench-lite

SWE-bench-verified

Total

Localization + Repair

99/300 (33.0%)

194/500 (38.8%)

293

Localization + Repair + Feedback

120/300 (40.0%)

226/500 (45.2%)

346

suggest that reproduction tests provide useful signals, helping better align generated patches with issue semantics and offering stronger behavioral criteria for patch selection. Feasibility of End-to-End Repair with Reproduction Tests. We further investigate whether reproduction tests can be used as feedback signals in an end-to-end repair loop. To this end, we construct a simple iterative repair pipeline in which the model first identifies suspicious files, then generates candidate patches, verifies them against the generated reproduction tests, and performs another repair iteration if verification fails, up to 5 iterations. Table 10 shows that reproduction-test feedback consistently increases the number of successful repair outcomes on both benchmarks. On SWE-bench-lite, the number of successful cases increases from 99 to 120. On SWE-bench-verified, the corresponding number rises from 194 to 226. In total, the number of successful cases across the two benchmarks increases from 293 to 346. These results indicate that reproduction tests can serve not only as final validators, but also as actionable execution-time feedback for repair. Although the current pipeline is intentionally simple, the observed gains highlight the practical promise of coupling repair with reproduction-test feedback. Summary: Issue reproduction tests generated by ReProAgent consistently improve the effectiveness of existing repair frameworks and also serve as actionable feedback in iterative repair, increasing the number of successful cases from 99 to 120 on SWE-bench-lite and from 194 to 226 on SWE-bench-verified. 6.2

Cost Analysis

Since ReProAgent is an agentic framework with iterative tool use, we further analyze its inference cost under the primary GPT-5-mini setting. For comparison, we collect the reported average perinstance costs of representative baselines if such information is available in the original papers. As shown in Table 11, ReProAgent costs $0.14 per instance on average. This is slightly higher than OpenHands with GPT-5-mini, but lower than several GPT-4o- or Claude-based reproductiontest generation baselines. Because different methods use different backbone models and pricing schemes, we do not use monetary cost as a direct superiority claim. Instead, the result shows that ReProAgent remains affordable in practice while providing substantially higher FP Rate under the same GPT-5-mini backbone. We also examine the number of feedback iterations used by successful cases. Among all successfully reproduced instances, the average number of feedback iterations is 1.29, and 90% of successful cases are resolved within five iterations. This observation suggests that future work can explore adaptive iteration budgeting or early-stopping strategies to further reduce cost while preserving reproduction effectiveness. Summary: ReProAgent remains affordable in practice, costing $0.14 per instance on average. Most successful cases require only a small number of feedback iterations, with an average of 1.29 iterations and 90% completed within five iterations. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:20

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

Table 11. Average cost per instance

Method

Backbone

Cost

Otter++ GPT-4o $1.80 Otter++ Claude-3.7-Sonnet $2.75 AssertFlip GPT-4o $1.00 Issue2Test Claude-3.5-Sonnet $0.66 OpenHands GPT-5-mini $0.11 ReProAgent GPT-5-mini $0.14 6.3

Threats to Validity

Internal Validity. Internal validity concerns potential biases that may affect evaluation fairness and consistency. First, LLM inference is stochastic, so repeated runs may yield different results. Second, backbone LLMs may respond differently to the same prompts due to differences in reasoning and coding ability. To mitigate these threats, we use standardized Docker environments, unified prompts, consistent tool settings, and evaluate ReProAgent across multiple backbone LLMs under the same framework. External Validity. External validity concerns whether our findings generalize beyond the current setting. First, our experiments are limited to Python repositories in the SWT-bench datasets, so the results may not generalize to other programming languages. However, ReProAgent’s high-level workflow is language-agnostic, and its graph-construction component can be extended using existing tools (e.g., CodeGraphContext [13] supporting 19 languages). Second, benchmark issues may not fully represent the complexity and diversity of real-world industrial projects. To reduce these threats, we evaluate ReProAgent on two widely used benchmark subsets covering multiple repositories and LLMs. 7

Conclusion

This paper presents ReProAgent, a multi-stage agent framework for issue reproduction test generation. By integrating task decomposition and reflection, hybrid code retrieval, and runtime interaction, ReProAgent organizes the generation process into four stages: bug localization, root cause analysis, test planning, and test generation. Experiments on SWT-bench-lite and SWTbench-verified show that ReProAgent consistently outperforms existing baselines, achieving reproduction rates of 58.43% and 70.30%, respectively. ReProAgent also generalizes across multiple LLM backbones and improves downstream issue resolution. For example, when integrated with SWE-agent on SWT-bench-lite, it increases the number of successfully resolved issues from 143 to 153. These results demonstrate the effectiveness of structured multi-stage agent design for repository-level issue reproduction test generation. References [1] Toufique Ahmed, Jatin Ganhotra, Rangeet Pan, Avraham Shinnar, Saurabh Sinha, and Martin Hirzel. 2025. Otter: Generating Tests from Issues to Validate SWE Patches. In ICML (Proceedings of Machine Learning Research). PMLR / OpenReview.net. [2] Toufique Ahmed, Jatin Ganhotra, Rangeet Pan, Avi Shinnar, Saurabh Sinha, and Martin Hirzel. 2025. Otter: Generating Tests from Issues to Validate SWE Patches. In International Conference on Machine Learning. [3] Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar, and Martin Hirzel. 2026. Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection. arXiv:2508.06365 [cs.SE] https://arxiv.org/abs/2508.06365 Accepted to ICSE 2026. [4] Aider-AI. 2026. Aider: AI Pair Programming in Your Terminal. https://github.com/Aider-AI/aider. Accessed: 2026-07-05. , Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:21

[5] Saranya Alagarsamy, Chakkrit Tantithamthavorn, and Aldeida Aleti. 2024. A3Test: Assertion-Augmented Automated Test case generation. Inf. Softw. Technol. 176, C (Dec. 2024), 15 pages. doi:10.1016/j.infsof.2024.107565 [6] Juan Altmayer Pizzorno and Emery D Berger. 2025. Coverup: Effective High Coverage Test Generation for Python. Proceedings of the ACM on Software Engineering 2, FSE (2025), 2897–2919. [7] Amazon Web Services. 2026. Amazon Q: Generative AI Assistant for Software Development and Enterprise Data. https://aws.amazon.com/q/. [8] Anthropic. 2025. Model Context Protocol Specification. https://modelcontextprotocol.io/specification/2025-11-25. Accessed: 2025-12-08. [9] Anthropic. 2025. Sequential Thinking. https://github.com/modelcontextprotocol/servers/tree/main/src/ sequentialthinking. Accessed: 2025-12-08. [10] Thierry N. D. Chekam, Mike Papadakis, Yue Jia, and Mark Harman. 2017. An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption. In Proceedings of the 39th International Conference on Software Engineering. 597–608. doi:10.1109/ICSE.2017.61 [11] Yinghao Chen, Zehao Hu, Chen Zhi, Junxiao Han, Shuiguang Deng, and Jianwei Yin. 2024. Chatunitest: A Framework for LLM-Based Test Generation. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 572–576. [12] Xiang Cheng, Fan Sang, Yizhuo Zhai, Xiaokuan Zhang, and Taesoo Kim. 2025. Rug: Turbo LLM for Rust Unit Test Generation. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 634–634. [13] CodeGraphContext Contributors. 2026. CodeGraphContext: Turn Code Repositories into a Queryable Graph for AI Agents. https://github.com/CodeGraphContext/CodeGraphContext. [14] Ermira Daka and Gordon Fraser. 2014. A Survey on Unit Testing Practices and Problems. In 2014 IEEE 25th International Symposium on Software Reliability Engineering. 201–211. [15] DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL] https://arxiv.org/abs/2412.19437 [16] Zhiwei Fei, Yue Pan, Federica Sarro, Jidong Ge, Marc Liu, Vincent Ng, and He Ye. 2026. Echo: Graph-Enhanced Retrieval and Execution Feedback for Issue Reproduction Test Generation. arXiv preprint arXiv:2603.07326 (2026). [17] Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic Test Suite Generation for Object-oriented Software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (Szeged, Hungary) (ESEC/FSE ’11). Association for Computing Machinery, New York, NY, USA, 416–419. doi:10.1145/2025113.2025179 [18] Gordon Fraser and Andrea Arcuri. 2013. Whole Test Suite Generation. IEEE Transactions on Software Engineering 39, 2 (2013), 276–291. doi:10.1109/TSE.2012.14 [19] Danielle Gonzalez, Joanna C.S. Santos, Andrew Popovich, Mehdi Mirakhorli, and Mei Nagappan. 2017. A Large-Scale Study on the Usage of Testing Patterns That Address Maintainability Attributes: Patterns for Ease of Modification, Diagnoses, and Comprehension. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). 391–401. doi:10.1109/MSR.2017.8 [20] Siqi Gu, Quanjun Zhang, Kecheng Li, Chunrong Fang, Fangyuan Tian, Liuchuan Zhu, Jianyi Zhou, and Zhenyu Chen. 2024. Testart: Improving llm-based unit testing via co-evolution of automated generation and repair iteration. arXiv preprint arXiv:2408.03095 (2024). [21] Andre Hora and Gordon Fraser. 2026. Understanding Bug-Reproducing Tests: A First Empirical Study. arXiv preprint arXiv:2602.02965 (2026). [22] Laura Inozemtseva and Reid Holmes. 2014. Coverage is Not Strongly Correlated with Test Suite Effectiveness. In Proceedings of the 36th International Conference on Software Engineering. 435–445. [23] Kush Jain, Gabriel Synnaeve, and Baptiste Rozière. 2025. Testgeneval: A Real World Unit Test Generation and Test Completion Benchmark. In ICLR. OpenReview.net. [24] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=VTF8yNQM66 [25] René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: a Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (San Jose, CA, USA) (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 437–440. doi:10.1145/ 2610384.2628055 [26] Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. Large Language Models are Few-Shot Testers: Exploring LLM-Based General Bug Reproduction. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 2312–2323. doi:10.1109/ICSE48619.2023.00194 [27] Lara Khatib, Noble Saji Mathews, and Meiyappan Nagappan. 2025. AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests. arXiv:2507.17542 [cs.SE] https://arxiv.org/abs/2507.17542 , Vol. 1, No. 1, Article 1. Publication date: January 2026.

1:22

Quanjun Zhang, Yi Zheng, Ye Shang, Weifeng Sun, Haichuan Hu, Chunrong Fang, Zhenyu Chen, and Liang Xiao

[28] Bohdan Korel. 1990. Automated Software Test Data Generation. IEEE Transactions on Software Engineering 16, 8 (1990), 870–879. doi:10.1109/32.57624 [29] LangChain. 2026. LangChain: Building Applications with LLMs. https://www.langchain.com/ Accessed: 2026-03-27. [30] LangChain. 2026. LangGraph: Agent Orchestration Framework for Reliable AI Agents. https://www.langchain.com/ langgraph Accessed: 2026-03-27. [31] Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa Ling Liu, and Arno Fiva. 2011. Automatic Testing of Object-Oriented Software. Proceedings of the 33rd International Conference on Software Engineering (2011), 114–124. doi:10.1145/1985793. 1985812 [32] Niels Mündler, Mark Niklas Mueller, Jingxuan He, and Martin Vechev. 2024. SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. https://openreview.net/forum?id=9Y8zUO11EQ [33] Zifan Nan, Zhaoqiang Guo, Kui Liu, and Xin Xia. 2025. Test Intention Guided LLM-Based Unit Test Generation. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 1026–1038. [34] Noor Nashid, Islem Bouzenia, Michael Pradel, and Ali Mesbah. 2025. Issue2Test: Generating Reproducing Test Cases from Issue Reports. arXiv:2503.16320 [cs.SE] https://arxiv.org/abs/2503.16320 [35] Neo4j, Inc. 2026. Neo4j: The Graph Intelligence Platform. https://neo4j.com/ Accessed: 2026-03-27. [36] OpenAI. 2025. GPT-5 is here. https://openai.com/gpt-5/. Accessed: 2026-06-24. [37] Carlos Pacheco and Michael D. Ernst. 2007. Randoop: Feedback-directed Random Testing for Java. In Companion to the 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications Companion (Montreal, Quebec, Canada) (OOPSLA ’07). Association for Computing Machinery, New York, NY, USA, 815–816. doi:10.1145/ 1297846.1297902 [38] Annibale Panichella, Fitsum Kifetew, and Paolo Tonella. 2015. Reformulating Branch Coverage as a Many-Objective Optimization Problem. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation. 1–10. doi:10.1109/ICST.2015.7102604 [39] Annibale Panichella, Fitsum Kifetew, and Paolo Tonella. 2018. Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets. IEEE Transactions on Software Engineering 44, 2 (2018), 122–158. doi:10.1109/TSE.2017.2663435 [40] Gabriel Ryan, Siddhartha Jain, Mingyue Shang, Shiqi Wang, Xiaofei Ma, Murali Krishna Ramanathan, and Baishakhi Ray. 2024. Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting Using LLM. Proceedings of the ACM on Software Engineering 1, FSE (2024), 951–971. [41] SWT-bench Developers. 2026. SWT-bench Recommended Configuration for OpenHands. https://github.com/logicstar-ai/swt-bench/issues/38. Accessed: 2026-06-24. [42] GLM Team. 2025. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models. arXiv:2508.06471 [cs.CL] https://arxiv.org/abs/2508.06471 [43] Tree-sitter. 2026. Tree-sitter: A Parser Generator and Incremental Parsing Library. https://tree-sitter.github.io/treesitter/ Accessed: 2026-03-27. [44] Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. 2020. Unit Test Case Generation with Transformers and Focal Context. arXiv:2009.05617 [cs.SE] [45] Junyi Wang, Jialun Cao, and Zhongxin Liu. 2026. iCoRe: An Iterative Correlation-Aware Retriever for Bug Reproduction Test Generation. Proceedings of the ACM on Software Engineering 3, FSE (2026), 4231–4252. [46] Xinchen Wang, Pengfei Gao, Xiangxin Meng, Chao Peng, Ruida Hu, Yun Lin, and Cuiyun Gao. 2025. Aegis: An Agent-Based Framework for Bug Reproduction from Issue Descriptions. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 331–342. [47] Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. 2025. OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv:2407.16741 [cs.SE] https://arxiv.org/abs/2407.16741 [48] Zejun Wang, Kaibo Liu, Ge Li, and Zhi Jin. 2024. Hits: High-Coverage LLM-Based Unit Test Generation Via Method Slicing. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1258–1268. [49] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. Demystifying LLM-Based Software Engineering Agents. Proc. ACM Softw. Eng. 2, FSE, Article FSE037 (June 2025), 24 pages. doi:10.1145/3715754 [50] John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer Interfaces Enable Automated Software Engineering. Advances in Neural Information Processing Systems 37 (2024), 50528–50652. [51] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR).

, Vol. 1, No. 1, Article 1. Publication date: January 2026.

ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

1:23

[52] Xin Yin, Chao Ni, Xinrui Li, Liushan Chen, Guojun Ma, and Xiaohu Yang. 2025. Enhancing LLM’s Ability to Generate More Repository-Aware Unit Tests Through Precise Context Injection. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 623–635. [53] Zhiqiang Yuan, Mingwei Liu, Shiji Ding, Kaixin Wang, Yixuan Chen, Xin Peng, and Yiling Lou. 2024. Evaluating and Improving ChatGPT for Unit Test Generation. Proc. ACM Softw. Eng. 1, FSE, Article 76 (July 2024), 24 pages. doi:10.1145/3660783 [54] Quanjun Zhang, Chunrong Fang, Siqi Gu, Ye Shang, Zhenyu Chen, and Liang Xiao. 2025. Large language models for unit testing: A systematic literature review. arXiv preprint arXiv:2506.15227 (2025). [55] Quanjun Zhang, Chunrong Fang, Yang Xie, Yuxiang Ma, Weisong Sun, Yun Yang, and Zhenyu Chen. 2026. A Systematic Literature Review on Large Language Models for Automated Program Repair. ACM Transactions on Software Engineering and Methodology (2026). doi:10.1145/3799693 Just Accepted. [56] Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Shengcheng Yu, Weisong Sun, Yun Yang, and Zhenyu Chen. 2026. A Survey on Large Language Models for Software Engineering. Science China Information Sciences 69, 4 (2026), 141102. [57] Quanjun Zhang, Chunrong Fang, Tongke Zhang, Bowen Yu, Weisong Sun, and Zhenyu Chen. 2023. Gamma: Revisiting Template-based Automated Program Repair via Mask Prediction. In Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering. 535–547. [58] Quanjun Zhang, Chengyu Gao, Yu Han, Ye Shang, Chunrong Fang, Zhenyu Chen, and Liang Xiao. 2026. SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair. ACM Transactions on Software Engineering and Methodology (2026). doi:10.1145/3818617 Just Accepted. [59] Quanjun Zhang, Ye Shang, Chunrong Fang, Siqi Gu, Jianyi Zhou, and Zhenyu Chen. 2027. TestBench: Evaluating Class-level Test Case Generation Capability of Large Language Models. Frontiers of Computer Science 21 (2027), 2106202–. doi:10.1007/s11704-025-50078-9 [60] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria) (ISSTA 2024). Association for Computing Machinery, New York, NY, USA, 1592–1604. doi:10.1145/ 3650212.3680384

, Vol. 1, No. 1, Article 1. Publication date: January 2026.