arXiv CSopen access

Refploit: Facilitating Exploit Construction via Code-Agent Trajectory Repair

Refploit: Facilitating Exploit Construction via Code-Agent Trajectory Repair Zirui Chen∗ , Zhipeng Xue∗ , Jiayuan Zhou† , Xing Hu∗ ‡ , Xin Xia∗ , and Xiaohu Yang∗ ∗ The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China

arXiv:2607.01760v1 [cs.SE] 2 Jul 2026

{chenzirui, zhipengxue, xinghu, yangxh}@zju.edu.cn, [email protected] † Queen’s University, Kingston, Canada [email protected]

Abstract—Vulnerability exploits play a crucial role in assessing the downstream impact of Java library vulnerabilities. While some vulnerabilities are accompanied by disclosed exploit references, automatically reproducing such references into runnable exploits remains challenging because they are often incomplete, unstructured, or only describe partial reproduction steps. Recent code agents provide a promising way to automate this process, but our study shows that their generated exploits often appear successful without triggering the actual vulnerable logic, such as replacing vulnerable APIs with self-implemented functions. To address this, we propose R EFPLOIT, an LLM-based trajectory recovery framework for facilitating vulnerability reproduction from public exploit references. The key insight is that a failed agent trajectory is not entirely useless. It may have already completed some reproduction subtasks while also revealing misleading directions that should be avoided. R EFPLOIT first validates an agentgenerated exploit through differential execution. When the exploit is ineffective, R EFPLOIT analyzes its reproduction progress, locates the trajectory segments associated with the reproduction progress, and derives constraints to guide focused recovery. We evaluate R EFPLOIT on three open-source Java vulnerability datasets, covering 172 exploit references for 143 vulnerabilities. Under DeepSeek-V4-Flash, R EFPLOIT successfully reproduces 138 exploits, achieving a reproduction rate of 80.2%. It achieves a 64.3% relative improvement over the initially generated trajectories and outperforms both the SOTA exploit-generation method P O CG EN and advanced code agents such as Codex with GPT5.4. Our ablation study shows that differential execution reduces false reproductions, while progress-guided trajectory analysis and constraint-guided recovery improve repair effectiveness and efficiency. We further adapt R EFPLOIT to another code agent and observe consistent improvements, demonstrating its generality. Index Terms—Vulnerability Reproduction, Agent Trajectory

I. I NTRODUCTION With the widespread adoption of open-source libraries [1], [2], [3], [4], [5], [6], [7], [8], the exploitability of upstream vulnerabilities in downstream projects has drawn increasing attention [9], [10], [11], [12], [3], [13], [14]. This issue is particularly important in the Java ecosystem, where Maven [15] hosts ten million packages, making downstream impact assessment a critical task. Recent research employs exploits as evidence for assessing the impact of upstream vulnerabilities [16], such as determining affected library versions [17], [18], [19] and providing domain knowledge for generating exploit tests in downstream projects [20], [9], [21], [22]. ‡ Corresponding Author

However, manually reproducing vulnerabilities is often time-consuming and error-prone [23]. Existing studies mainly attempt to generate exploits based on vulnerability descriptions by designing validation strategies for specific vulnerability types [24], [25]. However, such strategies have limited generality because vulnerability behaviors vary substantially across vulnerabilities and libraries, making it difficult to design general validation oracles. For example, P O CG EN [24] introduces five static analysis queries and reproduction patterns for path traversal (CWE-22/35), prototype pollution (CWE-1321), command injection (CWE-77/78), code injection (CWE-94 to CWE-99), and ReDoS (CWE-1333). This design limits the generality of P O CG EN when the vulnerability requires specific validation beyond these predefined patterns. To support reproduction across a broader range of vulnerability types, we examine reusable exploit knowledge beyond predefined patterns and find that public exploit references are available for some Java vulnerabilities. Specifically, among the 151 vulnerabilities in CWE-Bench-Java [26] and VISION [27], 72 vulnerabilities include exploit links in NVD. However, our empirical study shows that these disclosed exploits are rarely directly executable. They are often incomplete or unstructured, providing only textual reproduction steps, partial code snippets, or advisory-level descriptions. As a result, additional effort is required to reproduce the expected behavior, including completing the project, locating the vulnerable API, preparing the environment, and designing verification logic. On average, each reproduction requires 2.15 manual interventions, highlighting the need for automation. Code agents have demonstrated promising capabilities in automatically building software repositories [28], [29]. Motivated by this progress, we explore their potential for automating vulnerability reproduction from exploit reference. However, we find that commonly used code agents (such as mini-sweagent) often produce outputs that appear to reproduce the vulnerability but do not exercise the actual required logic. For instance, when reproducing CVE-2018-1002200, the agent utilizes its own implementation of the ZIP extraction logic instead of invoking the vulnerable API, and for CVE-202139144, it triggers the vulnerability by adjusting configurations rather than using the intended payload. These observations motivate approaches that repair ineffective agent-generated

trajectories to facilitate vulnerability reproduction. By inspecting these ineffective agent trajectories, we find that a failed reproduction attempt is not entirely useless. The trajectory may have already completed some tasks required for reproduction, while exposing misleading directions that should be avoided in subsequent attempts. These signals can be distilled into preservation and repair constraints for trajectory refinement. Motivated by this insight, we propose R EFPLOIT, an LLM-based trajectory recovery framework for facilitating vulnerability reproduction. ❶ R EFPLOIT first executes the agent-generated exploit on both the selected and patched versions to determine whether the trajectory produces the expected differential behavior. ❷ When the trajectory is ineffective, R EFPLOIT analyzes its reproduction progress and maps the resulting judgments back to trajectory segments. ❸ Based on this analysis, R EFPLOIT derives constraints from the failed trajectory to guide recovery, specifying which directions should be preserved and which actions should be repaired. Guided by these constraints, R EFPLOIT resumes the agent with focused recovery tasks and iteratively re-validates the repaired exploit until it achieves effective reproduction. We evaluate R EFPLOIT on three open-source datasets with 172 exploits for 143 vulnerabilities. These vulnerabilities span 53 CWE types, covering at least one CWE associated with 61.9% of Maven vulnerabilities in GitHub Advisory. Overall, R EFPLOIT successfully reproduces 138 exploits, achieving a success rate of 80.2%. It outperforms all baselines, including P O CG EN on its predefined CWE types, the base agent miniswe-agent (48.8%), and Codex with GPT-5.4 (69.2%). Our ablation results further show that each component contributes to the performance of R EFPLOIT. Differential analysis reduces false reproductions, while progress assessment and constraint guidance reduce unnecessary repair attempts. Finally, we evaluate the generality of R EFPLOIT by adapting R EFPLOIT to another code agent, which shows that R EFPLOIT can consistently improve the initial trajectories across agents. The main contributions of this paper are as follows. • We conduct a systematic study on the gap between disclosed exploit references and runnable exploits, showing that reproduction requires additional effort in environment preparation, harness assembly, and exploit behavior adaptation. • We propose R EFPLOIT , a trajectory recovery framework for facilitating vulnerability reproduction by analyzing initial trajectories and deriving constraints to guide recovery. • On three vulnerability datasets with 172 exploits, R EFPLOIT achieves a reproduction rate of 80.2%, outperforming both code-agent baselines and the exploit-generation baseline. II. P RELIMINARY S TUDY In this section, we introduce the usage scenario, present our empirical study, and discuss a motivating example. A. Usage Scenario In real-world vulnerability management, our method is designed to support two practical usage scenarios.

TABLE I M ANUAL I NTERVENTIONS DURING E ACH S TAGE . Source Form

Intervention Category

Stage

Count

Textual Steps (14/17 success)

Exploit Project Setup Exploit Harness Construction Exploit Resource Setup Exploit Script Modification

Harness Harness Environment Reproduce

14 9 7 6

Test Snippet (27/27 success)

Exploit Project Setup Test Snippet Completion Test Environment Setup Payload Modification

Harness Harness Environment Reproduce

27 17 2 4

Exploit Script (17/20 success)

Exploit Project Setup Exploit Harness Construction External Tool Configuration

Harness Harness Environment

6 2 7

Advisory (16/20 success)

Exploit Project Setup Exploit Harness Construction

Harness Harness

16 42

Total Interventions

159

(1) Constructing runnable reproductions from incomplete disclosed exploit references. Public exploit references are important for validating whether a reported library vulnerability can be reproduced in practice [17], [9], [20], [19], but they may contain only a textual description or a partial test snippet rather than a complete runnable project. In such cases, security analysts still need to assemble the harness, prepare the environment, and make the vulnerability observable. R EFPLOIT supports this process by guiding a code agent to construct a valid reproduction attempt through differential execution. (2) Refining ineffective vulnerability reproduction trajectories. When a code agent fails to generate a valid exploit, its trajectory is not entirely useless and may still contain segments that support vulnerability reproduction. R EFPLOIT analyzes such trajectories to identify what can be reused, what remains missing, and what may mislead reproduction. These signals guide constrained trajectory repair rather than restarting the reproduction process, thereby improving efficiency. B. Reproduction Process Study To understand the process of reproducing exploit references, we conduct a manual reproduction study on NVD exploit references associated with vulnerabilities from two Java vulnerability datasets, CWE-Bench-Java [26] and VISION [27], which together contain 151 unique vulnerabilities. After filtering inaccessible and irrelevant links, we obtain 84 exploit references. Two researchers with over five years of experience in vulnerability reproduction then attempt to convert each reference into a runnable exploit project, with a time limit of two hours per case. During this process, the researchers record the encountered issues and the remediation steps. To ensure reliability, the two researchers cross-check both the constructed exploits and the recorded interventions, and resolve disagreements through discussion. This process involves 159 interventions and results in 74 valid exploits. We use these interventions to characterize the reproduction process rather than only measuring manual cost. During manual reproduction, we observe that a valid exploit must run under a suitable environment, invoke the vulnerability-

error propagation

Initial Trajectory for CVE-2024-23833 1. Info Collection · Load Exploit Reference

· Detect Workspace Structure · Search JDBC/MySQL clues

S1

S2

S3

S4

S5

S6

4. Exploit Validation · Directly read target file via

3. Harness Construction · Directly copy payload · Mock `buildJdbcUrl()` as

2. Environment Setup · Create Maven Project · Select JDK 11

`Files.readAllBytes` · No differential analysis

˝

vulnerable API

S7

S8

S9

S10

S11

S12

S13

S14

Trajectory Segments

Trajectory Segments

Trajectory Segments

Trajectory Segments

summarize s

summarize

summarize s invoke the target Does not API in OpenRefine

summarize s Simulated reproduction

Missing vulnerable OpenRefine library

Finished as expected

Invalid Exploit · Selected:

- (Not Used)

· Patched:

3.8.0

· The target file is accessed in both versions due to simulated exploit logic. Not Reproduced

Bypassing real exploit path

Reproduction Progress Analysis (Recovery Tasks / Constraints) 1) Environment Dimension

· Add the library openrefine:database · Keep JDK 11 · Keep Maven

JDK Selection

MySQL Detection Library Setup

Recovered Trajectory Load Exploit Reference

Detect Workspace Structure

Fix Search JDBC/MySQL clues

Preserve S1

S2

S3

Original Trajectory Segments

Project Skeleton Use Target API

R4

R3

R2

R1

2) Harness Dimension

3) Reproduce Dimension · Invoke real API `setDatabaseHost` instead of mock

R5

R6

Trigger Behavior Differential Test

R7

· Resolve Existing Library · Add Required Library

· Direct-JDBC Test · Confirm `LOCAL INFILE`

· Locate API with `javap` · Adapt to target API

· Initial Payload Attempt · Refine Host Payload

Initial Introduction of the Target Library

Exploit Environment Validation via Direct-JDBC

Alignment to OpenRefine `setDatabaseHost` API

Payload Refinement to Enable `LOCAL INFILE`

+ <dependency> + <groupId>org.openrefine</groupId> + <artifactId>database</artifactId> + <version>3.7.2</version> + </dependency> pom.xml

- … host="127.0.0.1:3306,(host=127.0.0.1,port=3306,allowLoadLocalInfile=true … + … host="127.0.0.1?allowLoadLocalInfile=true&localInfile=true"; + String maliciousSql = "LOAD DATA LOCAL INFILE '" + TARGET_FILE … + databaseConfig.setDatabaseHost(host); + boolean hasResultSet = stmt.execute(maliciousSql); Exploit.java

· Successfully load file via MySQL · Patch version should not reproduce Fixed Exploit · Selected: 3.7.2 Sensitive file accessed · Patched: 3.8.0 `Invalid host supplied`

· The target file is accessed

only in vulnerable version.

· The patched version rejects this host due to fix logic. Reproduced!

Fig. 1. Refining the initial trajectory of reproducing CVE-2024-23833 by R EFPLOIT.

relevant library logic through a runnable harness, and produce observable behavior that confirms the vulnerability. To systematically derive these requirements, the two researchers perform an open card-sorting process over the recorded remediation steps. Each intervention is first coded according to the concrete requirement it addresses, and semantically similar interventions are then discussed and merged into intervention categories. Finally, as illustrated in Table I, these categories are mapped to three reproduction dimensions: Environment Preparation, Harness Assembly, and Exploit Reproduction. ❶ Environment Preparation captures whether the runtime prerequisites required by the exploit are available. Some reproductions depend on specific operating systems, JDK versions, network settings, or external tools. Since these prerequisites are often underspecified in public references, a reproduction attempt may fail even when the exploit logic is correct. ❷ Harness Assembly captures whether the fragmented exploit information has been transformed into a runnable harness that invokes the vulnerability-relevant library logic. The required actions include preparing the target project, locating the vulnerable API, constructing malicious inputs, and generating executable exploit code. This dimension is crucial because an exploit may appear runnable while bypassing the actual vulnerable library path. ❸ Exploit Reproduction captures whether the exploit triggers observable vulnerability behavior in the prepared environment. Even when the project and harness are ready, payloads or verification logic may still require adaptation. For example, path-traversal payloads may need to match the local file-system layout, and GUI-based indicators may need to be replaced with effects that are easier to observe automatically. These observations suggest that exploit reproduction should not be judged only by whether the final exploit runs success-

fully. Instead, it should be assessed according to whether the environment is prepared, the harness invokes the corresponding logic, and the exploit produces expected behavior. Findings: Vulnerability reproduction follows a multidimensional process involving environment preparation, harness assembly, and exploit reproduction. These dimensions provide perspectives for assessing reproduction progress and diagnosing failed agent trajectories.

C. Motivation Example As illustrated in Figure 1, we present an example where a code agent fails to reproduce a vulnerability even though the agent states that the generated exploit “clearly shows the vulnerability behavior” before finishing the task. CVE2024-23833 is a JDBC attack vulnerability in OpenRefine, which arises because OpenRefine builds the MySQL JDBC URL by directly placing the user-provided host value into the URL without sufficient validation. For example, a host value containing (allowLoadLocalInfile=true) can be interpreted by the driver as enabling local file loading, which may allow reading sensitive files. The vulnerability has been addressed since version 3.7.7. We find that the exploit does not exhibit differential behavior after the library is updated. In the Initial Trajectory shown in Figure 1, we segment the trajectory into 14 segments based on two types of actions (reading the reference and running exploits). We then manually categorize each segment according to its purpose and examine whether it achieves the expected goal. We find that the initial trajectory has already achieved several goals, such as reading the exploit reference and constructing the Maven project skeleton. Meanwhile, we identify two reasons for the ineffective attempt. ❶ First, the trajectory does not follow

I. Differential-Guided Reproduction State Analysis Exploit Reference

Vulnerability Description Initial Prompt

Read the vulnerability details from the following link. Your task is to construct a working exploit as a test case. Reference: {Exploit Reference} Description: {Description}

𝑆! Framework

mini-swe-agent

Base LLM

Qwen3.5-27B

LLM Judge

𝑆#

Selected Version (Vselect)

Execute

Initial Exploit

Logs

Exceptions

Exploit Command

Patched Version (Vpatch)

Reproduction e.g., trigger capture, side effects

Mapped Segments Subtask Judgments

Preserve

Search Items

Repair

Progress Judgment Prompt

LLM Process

Go to Stage II

Remaining Tasks

𝑡&

Select Task

𝑡%

Recovery

Fixed Exploit

Updated Exploit

Recovery Sub-Agent Setup Selected Subtask

𝑡$

Current Exploit

Relevant Constraints

Updated Trajectory

Recovery Prompt

Constraint Generation Prompt

…You are given the differential execution results of an exploit. Your task is to review the exploit across three dimensions: Env ready, Test harness, and Reproduce. For each dimension, analyze whether the relevant subtask is complete and provide search items to locate…{description}, exploit:{workspace}, behavior: {execution result}

No

Trajectory Recovery

𝑡!

…

Trajectory Mapping

Subtask State

Recovery Constraints

Witness Effects

Valid Exploit Yes Expected Behavior

Loop back to Differential Execution

…

Progress Judgment

𝑆"

…

III. Constraint-Oriented Trajectory Recovery

Progress Dimensions Harness e.g., API usage, object initialization

…

Agent Stack

II. Progress-Guided Trajectory Analysis Environment e.g., platform library setup

Differential Execution

Initial Trajectory

Initial Generation Agent

…For every completed task, exact evidence that must be reintroduced if applicable. For every incomplete task, the constraint must concretely explain how to complete the subtask… details: {selected subtask}, {related segments}

…You are debugging a previously generated vulnerabilityreproduction trajectory. Your current submission must select exactly one recovery task… Previously reusable constraints from the old trajectory: {constraints}…{reused trajectories}

Fig. 2. The overall framework and prompt details of R EFPLOIT.

the exploit path mentioned in the reference, which requires invoking the vulnerable library API setDatabaseHost with a crafted MySQL host string. Instead, the generated project does not include the OpenRefine library at all (Summary for Environment Setup in Figure 1). It directly implements a local mock connection manager whose buildJdbcUrl() method manually inserts the user input into a JDBC URL to simulate the behavior of OpenRefine (Summary for Harness Construction in Figure 1). ❷ Second, to demonstrate the expected malicious behavior, the harness directly reads a local file using standard Java file I/O with Files.readAllBytes(...), instead of triggering file access through the JDBC attack path described in the reference (Summary for Exploit Validation in Figure 1). This example shows that ❶ the initial trajectory is not entirely useless, even though it fails to produce an effective differential result. It already contains reusable components, such as selecting MySQL/JDBC as the attack direction, creating a Maven project, and targeting the correct vulnerable version. We further find that ❷ the expected workflow of vulnerability reproduction provides useful dimensions for diagnosing trajectory progress, which is crucial for determining whether each trajectory segment satisfies the requirements of vulnerability reproduction. By checking environment preparation, harness construction, and reproduction observation, we can determine which segments have achieved their intended goals, which segments introduce misleading behavior, and which reproduction requirements remain missing. Based on this analysis, we can extract refinement constraints that the next reproduction attempt should follow. These observations motivate our progress-guided trajectory analysis. Based on these observations, we design R EFPLOIT to ❶ generate constraints from reproduction progress analysis and ❷ recover the trajectory based on these constraints. After extracting constraints from the initial trajectory, R EFPLOIT first preserves reusable components from the initial trajectory. It then repairs the ineffective parts according to the constraints.

For the environment dimension, R EFPLOIT introduces the OpenRefine:3.7.2 library. For the harness dimension, R EF PLOIT uses setDatabaseHost to configure the crafted host. For the reproduction dimension, R EFPLOIT constructs a payload based on the LOAD DATA LOCAL INFILE command to load the target file, and modifies the payload to trigger sensitive file access. R EFPLOIT confirms recovery by verifying that the repaired exploit reads the target file on the vulnerable version but is blocked by the patched version. III. M ETHODOLOGY As illustrated in Figure 2, R EFPLOIT employs a three-stage workflow to evaluate and repair vulnerability reproduction trajectories generated by code agents. First, it evaluates the initially generated exploit through differential execution on the selected and patched versions. Second, when the differential result is ineffective, R EFPLOIT decomposes the trajectory into segments and aligns the reproduction process with trajectory segments to identify satisfied, missing, or misleading progress. Third, based on the completion status of each reproduction task in the original trajectory, R EFPLOIT generates preservation and repair constraints and uses them to resume the agent with focused recovery guidance. The recovery loop iteratively applies constrained repairs and differential validation until the exploit reaches the expected reproduction behavior or the recovery budget is exhausted. A. Differential-Guided Reproduction State Analysis Given an initial reproduction trajectory generated by the code agent, this stage evaluates the trajectory through differential execution. R EFPLOIT executes the generated exploit on both the target and patched versions, compares the observed behaviors with the expected vulnerability behavior, and collects differential evidence that indicates whether the trajectory reaches a successful reproduction state.

1) Initial Trajectory Generation: Given a vulnerability description d and an exploit reference r, R EFPLOIT first employs the code agent to generate an initial reproduction trajectory based on its ReAct loop: Tinit = ⟨S1 , S2 , . . . , Sn ⟩. In each iteration, the agent generates the next environment command based on the vulnerability information (d, r) and the previously generated trajectory prefix ⟨S1 , . . . , Si−1 ⟩, executes the command within the environment, and integrates the resulting observation into its subsequent decisions. Each node Si records the intent for this step, the command generated by the agent, and the execution result. This information supports our subsequent trajectory analysis, constraint extraction, and trajectory recovery. As a result, Tinit captures both the generated initial exploit project einit and the step-bystep process by which the agent constructs the exploit. 2) Differential Execution: After generating einit , R EFPLOIT evaluates it on both the vulnerable version Vvul used during its construction and the corresponding patched version Vpat : Rvul = E(einit , Vvul ),

Rpat = E(einit , Vpat )

Here, E denotes the execution process, and Rvul and Rpat summarize the observable execution behaviors under the two versions, including execution logs, exceptions, and witness side effects. R EFPLOIT then analyzes the differential behaviors to determine whether the observed execution matches the expected reproduction behavior mentioned in d. To perform this assessment, the LLM receives the differential evidence (Rvul , Rpat ) together with the vulnerability description d and contextual information of einit , including the project structure, generated exploit files, and the exploit command. The analysis focuses only on whether the exploit exhibits the intended behavior, while ignoring unrelated environmental, compilation, or execution issues. For cases that do not satisfy the expected behavior, R EFPLOIT further assesses the reproduction progress to identify directions for trajectory refinement, repairs the trajectory accordingly, and re-validates the repaired exploit via differential execution. B. Progress-Guided Trajectory Analysis Given the differential evidence, R EFPLOIT analyzes the current reproduction trajectory to extract evidence that can support reproduction and steer subsequent repairs away from repeated errors. We first define evidence-oriented progress dimensions that capture the necessary conditions for vulnerability reproduction, and then describe how R EFPLOIT judges these dimensions and identifies reproduction-supporting and misleading trajectory components based on the evidence. 1) Evidence-Oriented Progress Dimensions: After differential execution, R EFPLOIT does not immediately repair the ineffective trajectory. Instead, it first estimates how far the current attempt has progressed toward a valid reproduction. This distinction is important because an ineffective attempt does not mean that the whole trajectory is useless. Analyzing this

progress allows R EFPLOIT to classify the trajectory according to reproduction progress, identifying which components support reproduction, which requirements remain missing, and which components may mislead subsequent repair. Following prior work on subgoal-based agent evaluation [30], we decompose the vulnerability reproduction process into three progress dimensions based on the reproduction workflow summarized from our preliminary study: • Environment. This dimension checks whether the reproduction conditions are available, including the Java runtime, library setup, and external services. • Harness. This dimension checks whether the generated exploit forms a runnable harness that invokes the vulnerable library API and preserves the exploit semantics. • Reproduction. This dimension checks whether the execution triggers the expected behavior and shows meaningful divergence between vulnerable and patched versions. 2) Reproduction Progress Judgment: To perform this assessment, R EFPLOIT instructs the LLM to conduct a constrained reproduction progress judgment over the three dimensions. The analyzer reuses the same evidence context from differential execution, including (Rvul , Rpat ), d, r, and the workspace context of einit . R EFPLOIT requires the LLM to expand each dimension δj into concrete reproduction subtasks based on r and judge the progress of each subtask separately. For each dimension δj , R EFPLOIT derives a subtask nj , where each tj,k denotes a reproduction set Tj = {tj,k }k=1 subtask under δj . For example, environment subtasks may involve preparing the correct Java runtime or external service, harness subtasks may involve invoking the vulnerable API as required, and reproduction subtasks may involve changing the payload to make the behavior observable. For each tj,k , R EFPLOIT analyzes its completion status and the search items κj,k used to localize the trajectory step that introduced the corresponding completed capability or issue. The completion status indicates whether the subtask has been satisfied, remains incomplete, or has been incorrectly completed. This judgment is directly usable for later trajectory repair. Subtasks marked as satisfied identify steps that can support reproduction and should be considered for preservation. Incomplete subtasks indicate missing requirements that should guide repair, while misleading subtasks identify directions that should be avoided in subsequent recovery. 3) Progress-to-Trajectory Mapping: Rather than being completed by a single command, a subtask may be supported by several related trajectory steps, such as inspecting the required environment information before editing the exploit harness. Therefore, R EFPLOIT first decomposes the trajectory into segments based on two key actions: ❶ reading the initial vulnerability reference and ❷running the exploit commands. These actions indicate two important milestones in the trajectory: ❶ the agent is collecting exploit-relevant information and ❷ the agent considers the current exploit sufficiently complete for a reproduction attempt. For each tj,k , R EFPLOIT uses its search items κj,k to map the progress judgment back to the trajectory. If κj,k is non-

empty, R EFPLOIT locates the earliest trajectory steps that introduce these items (i.e., the commands that write κj,k into source files). Segments containing these steps are treated as the related trajectory evidence for tj,k . If an incomplete subtask has no supporting evidence, R EFPLOIT marks it as missing, indicating that the required capability has not been established in the current trajectory. After mapping, each tj,k is associated with its completion status and related trajectory segments. In this way, R EF PLOIT transforms the initial trajectory that is difficult to directly analyze into structured components categorized by their contribution to the reproduction process, which provides preservation, repair, and avoidance guidance that constrain the search space for recovery. C. Constraint-Oriented Trajectory Recovery After progress-guided trajectory analysis, R EFPLOIT uses the mapped trajectory segments and judgments to drive recovery. This stage derives recovery constraints, selects a replay entry point, iteratively chooses repair tasks, and resumes the trajectory under the corresponding constraints. 1) Recovery Constraint Generation: Based on the mapped progress judgments, R EFPLOIT converts trajectory segments into actionable recovery constraints that provide explicit instructions for the resumed agent during recovery. ❶ For each completed subtask, R EFPLOIT derives preservation constraints from its related trajectory segments. These constraints specify validated capabilities that should be retained instead of rediscovered. For example, if the initial trajectory has already selected a compatible JDK, constructed a Maven project skeleton, or identified MySQL/JDBC as the relevant attack direction, R EFPLOIT instructs the LLM to distill these elements into preservation constraints, requiring the resumed agent to keep these capabilities when replaying the corresponding subtask during recovery. ❷ For each incomplete subtask, R EFPLOIT derives repair constraints that describe the missing requirement needed to advance reproduction. For example, it may require adding the real vulnerable library or adapting the payload so that the malicious behavior is triggered through the expected path. When the mapped trajectory evidence indicates that some segments drift away from the specified reproduction requirements, repair constraints further prevent the resumed agent from repeating misleading actions. For example, if the initial trajectory simulates vulnerable behavior through a locally implemented function, R EFPLOIT forbids these directions in subsequent recovery during harness construction. Together, these constraints transform the failed trajectories into recovery guidance. Preservation constraints encode validated progress, repair constraints encode missing requirements, and avoidance constraints encode failure-inducing directions that should not be repeated. 2) Constraint-Guided Trajectory Recovery: Given the generated recovery constraints, R EFPLOIT first selects a replay entry point for recovery by scanning the initial trajectory from the beginning and replaying segments as long as they

correspond to completed subtasks. Once R EFPLOIT encounters a segment associated with incomplete or incorrectly completed subtasks, it resumes recovery from that point. Otherwise, when no reliable completed prefix can be identified, R EFPLOIT starts a new recovery trajectory from the initial workspace state to avoid inheriting misleading trajectory content. During recovery, R EFPLOIT organizes the reproduction subtasks into a constraint-guided recovery plan, where each subtask is associated with the constraints derived from the trajectory analysis. In each recovery iteration, R EFPLOIT selects one tj,k as the current repair target and launches a recovery sub-agent to work on this target under the associated constraints. The sub-agent receives the vulnerability context, the current workspace, and the constraints relevant to tj,k . It is required to focus on tj,k , preserve validated capabilities, and avoid repeating misleading actions identified from the initial trajectory according to the constraints. After the recovery sub-agent submits the repair for a selected tj,k , R EFPLOIT integrates the updated workspace and continues with the remaining subtasks. Once the current recovery round finishes, R EFPLOIT returns to the differential execution stage and re-evaluates the repaired exploit on the selected and patched versions. If the differential result becomes effective, the recovery terminates successfully. Otherwise, R EFPLOIT starts the next recovery round by performing progress-guided trajectory analysis again until the exploit is successfully reproduced or the recovery budget is exhausted. IV. E XPERIMENT S ETUP This section presents the experimental design used to evaluate R EFPLOIT. We first introduce the research questions (RQs) and then describe the dataset, implementation details, baselines, and ground-truth construction during evaluation. Our evaluation aims to answer the following RQs: • RQ1 (Effectiveness): How effective is R EFPLOIT in recovering ineffective reproduction trajectories, and how does it compare with existing vulnerability reproduction methods? • RQ2 (Ablation Study): How does each component contribute to the overall performance of R EFPLOIT? • RQ3 (Generality): Can R EFPLOIT achieve similar improvements when adapted to other code agents? A. Dataset To ensure sufficient dataset coverage, we conduct experiments on three open-source Java library vulnerability datasets [26], [27], [31]. Together, these three datasets cover a total of 278 CVEs. We collect all exploit links disclosed on NVD for these CVEs, obtaining 199 links for 160 CVEs. After excluding inaccessible links, links unrelated to exploit reproduction, links whose key information is available only in images, and two CVEs whose patches do not directly eliminate the vulnerable behavior (CVE-2021-43859 and CVE-202222885, as reported by Chen et al. [31]), we obtain 172 exploit reference links corresponding to 143 CVEs. These CVEs cover 53 CWE types, which cover at least one CWE for 61.9% of

TABLE II E FFECTIVENESS C OMPARISON OF R EFPLOIT AND OTHER AGENTS ACROSS F IRST- LISTED CWE C ATEGORIES . Method

Model

Overall # Exploits=172

CWE-22 (34)

CWE-787 (18)

CWE-502 (16)

CWE-79 (11)

CWE-611 (10)

CWE-770 (8)

CWE-94 (6)

CWE-78 (5)

CWE-444 (4)

CWE-776 (4)

Other (56)

R EFPLOIT

DeepSeek-V4 Qwen3.5-27B

138 (80.2%) 96 (55.8%)

25 (73.5%) 22 (64.7%)

15 (83.3%) 11 (61.1%)

13 (81.2%) 8 (50.0%)

10 (90.9%) 7 (63.6%)

10 (100.0%) 10 (100.0%)

8 (100.0%) 2 (25.0%)

4 (66.7%) 1 (16.7%)

5 (100.0%) 3 (60.0%)

4 (100.0%) 3 (75.0%)

4 (100.0%) 3 (75.0%)

40 (71.4%) 26 (46.4%)

Codex (xhigh)

GPT-5.4 DeepSeek-V4

119 (69.2%) 90 (52.3%)

26 (76.5%) 15 (44.1%)

12 (66.7%) 12 (66.7%)

8 (50.0%) 9 (56.2%)

8 (72.7%) 3 (27.3%)

10 (100.0%) 4 (40.0%)

6 (75.0%) 4 (50.0%)

2 (33.3%) 2 (33.3%)

0 (0.0%) 4 (80.0%)

4 (100.0%) 2 (50.0%)

3 (75.0%) 3 (75.0%)

40 (71.4%) 32 (57.1%)

mini-swe-agent

DeepSeek-V4 Qwen3.5-27B

84 (48.8%) 47 (27.3%)

20 (58.8%) 9 (26.5%)

8 (44.4%) 5 (27.8%)

10 (62.5%) 3 (18.8%)

2 (18.2%) 6 (54.5%)

4 (40.0%) 2 (20.0%)

6 (75.0%) 2 (25.0%)

2 (33.3%) 1 (16.7%)

2 (40.0%) 0 (0.0%)

2 (50.0%) 2 (50.0%)

3 (75.0%) 2 (50.0%)

25 (44.6%) 15 (26.8%)

AutoGPT

DeepSeek-V4 Qwen3.5-27B

48 (27.9%) 39 (22.7%)

11 (32.4%) 4 (11.8%)

9 (50.0%) 8 (44.4%)

5 (31.2%) 2 (12.5%)

4 (36.4%) 1 (9.1%)

1 (10.0%) 3 (30.0%)

1 (12.5%) 1 (12.5%)

2 (33.3%) 0 (0.0%)

0 (0.0%) 0 (0.0%)

1 (25.0%) 2 (50.0%)

1 (25.0%) 2 (50.0%)

13 (23.2%) 16 (28.6%)

Maven library vulnerabilities in GitHub Advisory. Therefore, we consider this dataset sufficiently general for evaluation. B. Implementation In RQ1, we evaluate R EFPLOIT with two open-weight models to support reproducible comparison. ❶ DeepSeek V4Flash [32] is selected as a strong open-weight model with competitive coding and reasoning capabilities. ❷ Qwen3.527B [33] is selected because it can be locally deployed for reproducible evaluation, while also providing strong toolcalling capability and competitive performance on coding tasks. For RQ2 and RQ3, we use Qwen3.5-27B as the default model setting, because local deployment enables stable and cost-controllable evaluation. We deploy Qwen3.5-27B on a machine with eight A100 GPUs, while accessing the remaining models through APIs. We set the temperature to 0.2 for all models. For settings that require other models, such as Codex, we provide the details in the baseline description. For our choice of code agent, we adopt mini-swe-agent (v2.2.8) [34] as the base agent, as it is a lightweight and efficient code agent that facilitates reproducibility and supports multiple foundational models. For cross-agent generalizability evaluation, we include a widely used open-source code agent, OpenHands [35] (74.5K stars on GitHub). For each reproduction, we launch an Ubuntu 22.04 container with four LTS versions of Java. In all experiments, we provide only the vulnerability description and the exploit reference without supplying additional guidance. Following prior work [24], we allocate a token budget of 5M input tokens and 1M output tokens, which corresponds to an upper-bound cost of approximately $1.03 per case (DeepSeek-V4-Flash), excluding cache hits. To prevent unbounded exploration, we limit the agent execution trajectory to at most 200 steps. C. Baselines To evaluate the performance of R EFPLOIT in vulnerability reproduction, we consider the following baselines under the same budget and runtime environment: • Exploit Generation Baselines. We include P O CG EN [24] because it represents the SOTA approach for automated exploit generation. We adapt P O CG EN to Java and evaluate P O CG EN under DeepSeek-V4-Flash for a fair comparison. • Advanced Code-Agent Baseline. To compare R EFPLOIT with an advanced code-agent setting, we evaluate Codex (0.141.0) [36] under the xhigh-context setting using its

recommended model GPT-5.4 and the model used by R EF PLOIT, DeepSeek V4-Flash to ensure a fair comparison. • Underlying code-agent baseline. To quantify the improvement of R EFPLOIT over its underlying code agent, we directly evaluate mini-swe-agent under DeepSeek V4-Flash and Qwen3.5-27B, which measures the performance of the base agent without trajectory recovery. • Open-source Code-Agent Baselines. Following Simsek et al. [24], we select AutoGPT Classic [37] as a codeagent baseline and evaluate it under DeepSeek V4-Flash and Qwen3.5-27B, which can navigate the codebase by browsing files, reading and writing files, and executing commands. D. Ground Truth Due to the wide variety of vulnerability types in our dataset, it is impractical to design vulnerability-specific validation rules for every case. Existing strategies, such as rule-based validation used by Simsek et al. [24], cover only five CWEs. To evaluate exploit reproduction, we adopt a differential-based strategy. Specifically, we execute each exploit on both the vulnerable version and the patched version. After collecting the execution behaviors, two researchers with over five years of experience in software security independently analyze the results. To mitigate potential bias, we adopt a blinded annotation strategy. For each reproduction task, all generated exploits from different settings, including R EFPLOIT, baselines, ablation variants, and cross-agent variants, are randomly ordered and presented to the annotators without revealing which configuration produced them. An exploit is considered reproduced if it triggers the intended behavior in the vulnerable version while failing to do so in the patched version. After the independent analyses, the two researchers cross-verify their results and resolve discrepancies through discussion. The two researchers achieve a Cohen’s kappa of 0.828, indicating strong agreement. After resolving discrepancies, we identify 1,114 confirmed reproductions in total. V. E VALUATION R ESULTS We evaluate the performance of R EFPLOIT in terms of effectiveness, component contribution, and cross-agent generality. A. Effectiveness Table II and Table III present the effectiveness across CWE categories. Overall, R EFPLOIT with DeepSeek-V4-Flash

TABLE III C OMPARISON WITH P O CG EN ACROSS P REDEFINED CWE T YPES .

TABLE IV A BLATION S TUDY R ESULTS (BASED ON Q WEN 3.5-27B).

Vulnerability Type

CWE

Exploits

R EFPLOIT

P O CG EN

Variants

Effectiveness

Steps

Variants

Effectiveness

Steps

Path Traversal Command Injection Code Injection ReDoS Prototype Pollution

CWE-22/35 CWE-77/78 CWE-94 to 99 CWE-1333 CWE-1321

35 7 10 3 N/A

26 (74.3%) 5 (71.4%) 8 (80.0%) 3 (100.0%) N/A

11 (31.4%) 2 (28.6%) 3 (30.0%) 2 (66.7%) N/A

R EFPLOIT Diff-Feedback SWE-Agent

96 (55.8%) 54 (31.4%) 47 (27.3%)

109.3 64.3 47.6

w/o Differential w/o Progress w/o Constraint

69 (40.1%) 85 (49.4%) 72 (41.9%)

103.1 147.6 129.3

achieves the best performance, reproducing 138 out of 172 exploits with a success rate of 80.2%. It outperforms all evaluated baselines, including the exploit-generation baseline P O CG EN on predefined categories, the widely used AutoGPT (27.9%), the base agent mini-swe-agent with the same model setting (48.8%), and Codex with GPT-5.4 (69.2%). This result shows that R EFPLOIT can achieve competitive effectiveness against advanced code agents and exploit reproduction methods. To further control the effect of backbone models, we compare all code-agent baselines under DeepSeek-V4-Flash. R EFPLOIT achieves a success rate of 80.2%, outperforming Codex, mini-swe-agent, and AutoGPT by 27.9%, 31.4%, and 52.3%, respectively. This result shows that the improvement does not simply come from using a stronger backbone model. The comparison with the underlying models further shows that R EFPLOIT is effective across models. Under DeepSeek-V4Flash, REFPLOIT improves over the base agent from 84 to 138 reproductions, yielding 54 additional exploits and a 64.3% relative increase. Under Qwen3.5-27B, R EFPLOIT improves reproductions from 47 to 96, a 104.3% relative increase. These improvements show that R EFPLOIT can effectively recover trajectories generated by different models. Across CWE categories, R EFPLOIT also demonstrates broad effectiveness compared with code-agent baselines. With DeepSeek-V4-Flash, R EFPLOIT achieves the best performance in most listed CWEs, outperforming most baselines across diverse vulnerability categories such as deserialization, XXE, resource management, and HTTP parsing. This result suggests that trajectory recovery is effective across different vulnerability behaviors, rather than being limited to specific categories. As illustrated in Table III, we compare R EFPLOIT with P O CG EN on the CWEs covered by P O CG EN, including path traversal, command injection, code injection, and ReDoS. Prototype Pollution is not applicable because it mainly arises in prototype-based languages such as JavaScript rather than Java. On these predefined CWE types, R EFPLOIT achieves better performance than P O CG EN, outperforming it by 42.9%, 42.8%, 50.0%, and 33.3%, respectively. This comparison shows that R EFPLOIT outperforms the SOTA exploit generation method without CWE-specific designs. We further find two major limitations of P O CG EN. First, for vulnerabilities that require complex gadget construction, such as XStream deserialization vulnerabilities, P O CG EN often fails to construct the required gadget chain. Second, P O CG EN follows a fixed construction workflow and lacks project-level environment adaptation. As a result, it cannot modify the reproduction environment when the exploit depends on specific runtime

conditions, such as switching to a compatible JDK version. In contrast, R EFPLOIT performs project-level trajectory recovery and refines the exploit toward a valid reproduction. Answer to RQ1: R EFPLOIT achieves the best overall effectiveness and consistently improves the initial trajectories generated by the base agent. R EFPLOIT also performs well across diverse CWE categories.

B. Ablation Study As illustrated in Table IV, we evaluate R EFPLOIT from two perspectives: whether the underlying agent can recover ineffective reproductions without guidance, and whether each component of R EFPLOIT contributes to trajectory recovery. We first consider two agent-only settings. ❶ SWE-AgentOnly directly relies on mini-swe-agent to reproduce the vulnerabilities from the references without any guidance such as differential execution. ❷ Diff-Feedback-Only provides the differential execution results to mini-swe-agent when the initial reproduction is finished, and asks it to repair any invalid exploits. These settings evaluate whether the base agent can either generate exploits directly or use differential feedback by itself to repair ineffective trajectories. We then design three variants to evaluate the contribution of each component to R EFPLOIT. ❸ No-Differential-Analysis uses the vulnerable version execution result for subsequent recovery, without comparing it against the patched version. ❹ No-ProgressAssessment analyzes the reproduction process directly without guidance from the three reproduction process dimensions. ❺ No-Constraint-Guidance requires the agent to continue the repair from Tinit based on the reproduction process without deriving preservation and repair constraints. R EFPLOIT achieves the best performance among all variants, reproducing 96 exploits (55.8%). We first compare R EF PLOIT with the agent-only settings. SWE-Agent-Only reproduces only 47 exploits (27.3%), showing that the base agent cannot reliably construct valid exploits from references. Providing differential feedback to the base agent alone only yields a limited improvement, increasing the number of reproduced exploits from 47 (27.3%) to 54 (31.4%). This comparison indicates that the base agent cannot effectively interpret differential feedback or guide exploit repairs. Therefore, the improvement of R EFPLOIT comes from its trajectory analysis and recovery design, rather than the underlying agent or model. We compare R EFPLOIT with its ablation variants to understand the contribution of each component. All three variants perform worse than R EFPLOIT, indicating that differential analysis, progress assessment, and constraint-guided recovery

all contribute to the overall effectiveness. The No-DifferentialAnalysis variant achieves only 69 reproductions (40.1%). Although it uses fewer average steps than R EFPLOIT, the lower step count mainly results from the absence of patched-version evidence, which causes some simulated exploits to be incorrectly treated as valid reproductions and prevents the agent from further refining ineffective trajectories. The No-ProgressAssessment variant achieves 85 reproductions (49.4%) but requires the most steps. This suggests that progress assessment helps reduce exploratory repairs by decomposing reproduction into predefined dimensions. The No-Constraint-Guidance variant drops to 72 reproductions (41.9%), showing that unfinished tasks alone are insufficient without explicit constraints. Answer to RQ2: Each component contributes to the effectiveness of R EFPLOIT, as differential analysis avoids unreliable recovery decisions and progress assessment with constraint guidance keeps recovery focused.

C. Cross-agent Generality To evaluate whether R EFPLOIT can be adapted to other code-agent frameworks, we migrate R EFPLOIT from miniswe-agent to OpenHands. This adaptation only affects the agent interaction layer, including how R EFPLOIT segments the agent trajectory, replays a selected trajectory prefix, and launches recovery sub-agents. The core analysis and recovery logic, including differential execution, progress assessment, and constraint generation, remains unchanged. Under Qwen3.5-27B, OpenHands successfully reproduces 51 exploit references before applying R EFPLOIT. After integrating R EFPLOIT, the number increases to 89, corresponding to a 74.5% relative increase. This improvement indicates that R EFPLOIT can effectively repair ineffective trajectories generated by OpenHands. These results suggest that the trajectory analysis and constraint-guided recovery strategy of R EFPLOIT can be transferred to another code-agent framework and still bring effective improvement in vulnerability reproduction through trajectory recovery. Answer to RQ3: The improvement on OpenHands shows that R EFPLOIT can be adapted to other code agents with limited changes. Its trajectory analysis and constraint-guided recovery are not tied to the base agent.

VI. D ISCUSSION In this section, we qualitatively analyze the recovery behaviors of R EFPLOIT and discuss the main threats to validity. A. Qualitative Analysis 1) Successful Recovery Patterns: To better understand how R EFPLOIT recovers ineffective agent trajectories, we inspect representative recovery patterns. Aligning the Harness with Real Attack Surface. A particularly important recovery pattern is to redirect a misleading harness to the real attack surface exposed by the library. For example, in CVE-2020-27216, the initial exploit reproduced the race condition through a manually implemented helper

that creates the target file rather than invoking the vulnerable temporary-directory creation logic. R EFPLOIT repairs it by constructing a real WebAppContext and racing to create the temporary directory after Jetty releases the path inside WebInfConfiguration.makeTempDirectory(). These cases show that R EFPLOIT succeeds in modifying the harness to the required surface. Ensuring Valid Payload Semantics. Some initial trajectories fail to reproduce the vulnerability because the payload, trigger condition, or validation oracle is semantically misaligned with the reference exploit. In such cases, R EFPLOIT realigns the generated exploit with the expected vulnerability behavior. For example, in CVE-2022-25845, the initial harness used generic Fastjson auto-type payloads java.util.Properties and JdbcRowSetImpl rather than the required @type bypass during the deserialization of the Exception or Throwable class. R EFPLOIT therefore refines the exploit to the specified bypass payload and validates whether the vulnerable and patched versions diverge under the refined payload. A similar issue appears in CVE-2021-29061, where the initial input fails to trigger the ReDoS because it is both too short and structurally inconsistent with the vulnerable regex path, so matching never reaches the repeated (.+:.+@)* region where the slowdown occurs. R EFPLOIT repairs this by realigning the URI with the required exploit pattern and increasing its length until the vulnerable run reliably times out. These cases show that R EFPLOIT succeeds in restoring the semantics and structure of the payload to trigger the vulnerability. Repairing Configuration Prerequisites. Some initial trajectories fail because settings are missing. For example, in CVE-2022-23457, the initial harness already invoked ESAPI.validator().getValidDirectoryPath() with the correct sibling-path bypass input, but the ESAPI initialization failed because the required ESAPI.properties configuration is incomplete. R EFPLOIT repairs this by supplying a valid ESAPI configuration. A similar issue appears in CVE-2022-25845, where the exploit includes unsupported –add-opens flags. R EFPLOIT repairs this by removing the incompatible runtime options and aligning the environment with the required fastjson and JDK combination. These cases show that R EFPLOIT succeeds in satisfying the prerequisite configuration and runtime environment needed for vulnerability reproduction. 2) Failure Case Analysis: Not all ineffective trajectories can be recovered. We observe two failure patterns in which R EFPLOIT substantially improves the trajectory, yet still cannot obtain the expected divergence. Misclassified Progress in Payload Adaptation. Some failures persist not because the vulnerability path is completely absent, but because the recovered payload is still not fully adapted to the execution environment. In CVE-202139144, the recovered payload reaches an XStream deserialization path and raises a CannotResolveClassException for sun.tracing.NullProvider. This exception should be interpreted as a partial-progress signal during reproduction, suggesting that the payload could be further adapted by adjusting the command used to produce the observable RCE. However,

R EFPLOIT misclassifies it as an invalid attempt and therefore fails to further adapt the payload. Complex Runtime Configuration Failure. Some failures are dominated by complex runtime prerequisites that consume recovery effort before exploit semantics can even be validated. CVE-2021-43113 is a representative example. The exploit depends on Ghostscript being visible to the CompareTool. Although Ghostscript itself is installed, ITEXT GS EXEC is not propagated correctly, leading to a failed recovery with “Ghostscript command is not specified”. B. Threats to Validity Our conclusions may be affected by the following threats. 1) External validity: One threat comes from our exclusion of exploits whose key information is only available in images. For example, CVE-2021-40660 contains exploit code in image form. Excluding such cases may limit the generalizability of our results to image-heavy exploit references. We partially reduce this concern by evaluating R EFPLOIT with Qwen3.527B, a multimodal-capable model, and confirming that R EF PLOIT remains effective under this model. However, we have not systematically adapted R EFPLOIT to process image-only exploit references or evaluated its performance on such cases, and we leave this task to future work. Although our evaluation uses three large-scale public Java vulnerability datasets, the covered CWEs are associated with only 61.9% of historical Maven library vulnerabilities. This may limit the generalizability of our results to vulnerability categories that are underrepresented or absent from our datasets. We further find that the uncovered Top CWE categories are mainly associated with authorization issues, such as CWE-862, CWE-284, and CWE-639, or with vulnerabilities that more often arise at the platform or application level rather than the library level, such as CWE-352 and CWE-200. Therefore, the performance of R EFPLOIT on these vulnerability categories has not yet been evaluated. 2) Internal validity: A major threat comes from our adaptation of P O CG EN to Java vulnerabilities. This adaptation may not fully capture all implementation details or design assumptions of P O CG EN, and may therefore affect its performance. To mitigate this threat, we preserve the core workflow of P O CG EN as much as possible and only modify ecosystemspecific components required for Java reproduction. Our evaluation environment pre-installs four LTS JDK versions and tools such as curl. This setting standardizes the execution environment and allows us to focus on trajectory recovery. However, it also reduces the effort of environment preparation and therefore does not fully evaluate the ability of R EFPLOIT to install JDKs or prepare tools. Another internal threat comes from the LLM-based trajectory analysis in R EFPLOIT. R EFPLOIT relies on the LLM to judge reproduction progress, map progress judgments to trajectory segments, and generate recovery constraints. Incorrect judgments may lead R EFPLOIT to preserve misleading trajectory components, which can affect the final recovery outcome. To mitigate this threat, R EFPLOIT grounds its analysis

in differential execution evidence and uses structured progress dimensions to constrain the judgment process. VII. R ELATED W ORK In this section, we review prior research on software building, vulnerability exploits, and trajectory analysis. Software Building. Prior studies have extensively investigated the challenges of automatic software building [38], [39], [40], [29], [41], [28]. Hassan et al. [39] revealed that nearly half of Java projects fail to build with default commands, while Lou et al. [40] analyzed Stack Overflow discussions, identifying predominant causes of failure. Recently, Yu et al. [29] proposed CXXCrafter, an LLM-based agent to resolve build errors for C/C++ open-source software. Bouzenia and Pradel [28] proposed ExecutionAgent, an LLM-based agent that automatically prepares build scripts across diverse programming languages, build systems, and testing frameworks. However, these works focus on repairing existing, well-formed repositories, while disclosed exploits often exist as isolated fragments without a build environment. Library Vulnerability Exploits. Library vulnerability exploits are extensively employed in various downstream security tasks. Existing studies have leveraged exploits to assess vulnerability reachability in client projects [9], [20] or to identify affected library versions [19], [17]. However, these works typically rely on manual effort to construct executable exploits. While recent studies have explored automated oneday exploit generation [42], [24], [25], they are often restricted to specific vulnerability types. To enhance generality, we explore leveraging LLMs to reproduce vulnerabilities from unstructured disclosure information (e.g., textual advisories). Trajectory Analysis. Recent research in agent systems has increasingly focused on understanding failures by analyzing execution trajectories. MAST [43] first characterized agent failures across system design, coordination, and task execution, while TRAIL [44] refined this view with finergrained categories including reasoning and planning errors. To address these failures, researchers have developed complementary approaches that combine direct inspection, active intervention, and statistical reasoning. AGDebugger [45] allows developers to steer agent behaviors by editing message histories. DoVer [46] actively validates and repairs failures through targeted interventions instead of relying solely on logs. FAMAS [47] replays trajectories and uses spectrum analysis to estimate which agent actions are responsible for failures. However, in the context of vulnerability reproduction, these approaches often fail because execution outcomes are highly diverse and cannot be reliably attributed through singleshot inspection, interventions, or repeated pattern analysis. To address this, we propose a differential analysis approach at both the version and trajectory levels, which evaluates whether a generated intervention is truly effective. VIII. C ONCLUSION In this paper, we present R EFPLOIT, a trajectory recovery framework for facilitating agent-based vulnerability reproduc-

tion from publicly disclosed exploit references. R EFPLOIT analyzes ineffective agent-generated trajectories through differential execution and progress assessment, and derives constraints to guide focused recovery. Evaluated on three opensource Java vulnerability datasets, R EFPLOIT achieves a reproduction success rate of 80.2%, outperforming both codeagent baselines and the exploit-generation baseline. Ablation and generality studies further show that its core components contribute to trajectory recovery and that R EFPLOIT can be applied across different model and agent settings. In future work, we will evaluate R EFPLOIT with more models and codeagent frameworks to further assess its generality. IX. DATA AVAILABILITY The dataset and source code of R EFPLOIT will be made publicly available upon acceptance. R EFERENCES [1] Synopsys. OPEN SOURCE SECURITY AND RISK ANALYSIS REPORT 2023. [Online]. Available: https://www.synopsys.com/software-integrity/resources/analyst-reports/ open-source-security-risk-analysis.html [2] Y. Na, S. Woo, J. Lee, and H. Lee, “Cneps: A precise approach for examining dependencies among third-party c/c++ open-source components,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/3597503.3639209 [3] R. G. Kula, D. M. German, A. Ouni, T. Ishio, and K. Inoue, “Do developers update their library dependencies? an empirical study on the impact of security advisories on library migration,” Empirical Software Engineering, vol. 23, pp. 384–417, 2018. [4] Z. Zhang, Y. Hao, W. Chen, X. Zou, X. Li, H. Li, Y. Zhai, Z. Qian, and B. Lau, “Symbisect: accurate bisection for fuzzer-exposed vulnerabilities,” in Proceedings of the 33rd USENIX Conference on Security Symposium, ser. SEC ’24. USA: USENIX Association, 2024. [5] L. Zhang, J. Wu, C. Liu, K. Li, X. Sun, L. Zhao, C. Wang, and Y. Liu, “Fixing outside the box: Uncovering tactics for open-source security issue management,” Proc. ACM Softw. Eng., vol. 2, no. ISSTA, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3728977 [6] D. Cassel, N. Sabino, M.-C. Hsu, R. Martins, and L. Jia, “Nodemedicfine: Automatic detection and exploit synthesis for node. js vulnerabilities.” in NDSS, 2025. [7] M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel, “Smallworld with high risks: a study of security threats in the npm ecosystem,” in Proceedings of the 28th USENIX Conference on Security Symposium, ser. SEC’19. USA: USENIX Association, 2019, p. 995–1010. [8] S. Wu, W. Song, K. Huang, B. Chen, and X. Peng, “Identifying affected libraries and their ecosystems for open source software vulnerabilities,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/3597503.3639582 [9] Z. Chen, X. Hu, X. Xia, Y. Gao, T. Xu, D. Lo, and X. Yang, “Exploiting library vulnerability via migration based automating test generation,” in 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). Los Alamitos, CA, USA: IEEE Computer Society, apr 2024, pp. 2820–2831. [Online]. Available: https://doi.ieeecomputersociety.org/10.1145/3597503.3639583 [10] R. He, H. He, Y. Zhang, and M. Zhou, “Automating dependency updates in practice: An exploratory study on github dependabot,” IEEE Trans. Softw. Eng., vol. 49, no. 8, p. 4004–4022, Aug. 2023. [Online]. Available: https://doi.org/10.1109/TSE.2023.3278129 [11] S. Liu, J. Zhou, X. Hu, F. R. Cogo, X. Xia, and X. Yang, “An empirical study on vulnerability disclosure management of open source software systems,” ACM Trans. Softw. Eng. Methodol., vol. 34, no. 7, Aug. 2025. [Online]. Available: https://doi.org/10.1145/3716822 [12] G. Bavota, G. Canfora, M. Di Penta, R. Oliveto, and S. Panichella, “How the apache community upgrades dependencies: an evolutionary study,” Empirical Software Engineering, vol. 20, pp. 1275–1317, 2015.

[13] S. Li, Y. Wang, C. Dong, S. Yang, H. Li, H. Sun, Z. Lang, Z. Chen, W. Wang, H. Zhu, and L. Sun, “Libam: An area matching framework for detecting third-party libraries in binaries,” ACM Trans. Softw. Eng. Methodol., vol. 33, no. 2, Dec. 2023. [Online]. Available: https://doi.org/10.1145/3625294 [14] N. Zahan, T. Zimmermann, P. Godefroid, B. Murphy, C. Maddila, and L. Williams, “What are weak links in the npm supply chain?” in Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice, ser. ICSE-SEIP ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 331–340. [Online]. Available: https://doi.org/10.1145/3510457.3513044 [15] MvnRepository, “Mvnrepository.” [Online]. Available: https: //mvnrepository.com/repos [16] T. Avgerinos, S. K. Cha, A. Rebert, E. J. Schwartz, M. Woo, and D. Brumley, “Automatic exploit generation,” Commun. ACM, vol. 57, no. 2, p. 74–84, feb 2014. [Online]. Available: https: //doi.org/10.1145/2560217.2560219 [17] J. Dai, Y. Zhang, H. Xu, H. Lyu, Z. Wu, X. Xing, and M. Yang, “Facilitating vulnerability assessment through poc migration,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 3300–3317. [Online]. Available: https://doi.org/10.1145/3460120.3484594 [18] Z. Jiang, Y. Zhang, J. Xu, X. Sun, Z. Liu, and M. Yang, “Aem: Facilitating cross-version exploitability assessment of linux kernel vulnerabilities,” in 2023 IEEE Symposium on Security and Privacy (SP), 2023, pp. 2122–2137. [19] Z. Chen, Z. Xue, J. Zhou, X. Hu, X. Xia, and X. Yang, “Diffploit: Facilitating cross-version exploit migration for open source library vulnerabilities,” 2025. [Online]. Available: https: //arxiv.org/abs/2511.12950 [20] Z. Zhou, Y. Yang, S. Wu, Y. Huang, B. Chen, and X. Peng, “Magneto: A step-wise approach to exploit vulnerabilities in dependent libraries via llm-empowered directed fuzzing,” in Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1633–1644. [Online]. Available: https://doi.org/10.1145/3691620.3695531 [21] H. J. Kang, T. G. Nguyen, B. Le, C. S. Păsăreanu, and D. Lo, “Test mimicry to assess the exploitability of library vulnerabilities,” in Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2022. New York, NY, USA: Association for Computing Machinery, 2022, p. 276–288. [Online]. Available: https://doi.org/10.1145/3533767.3534398 [22] P. Deng, L. Zhang, Y. Meng, Z. Yang, Y. Zhang, and M. Yang, “Chainfuzz: exploiting upstream vulnerabilities in open-source supply chains,” in Proceedings of the 34th USENIX Conference on Security Symposium, ser. SEC ’25. USA: USENIX Association, 2025. [23] M. H. M. Bhuiyan, A. S. Parthasarathy, N. Vasilakis, M. Pradel, and C.-A. Staicu, “Secbench.js: An executable security benchmark suite for server-side javascript,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023, pp. 1059–1070. [24] D. Simsek, A. Eghbali, and M. Pradel, “Pocgen: Generating proof-ofconcept exploits for vulnerabilities in npm packages,” 2025. [Online]. Available: https://arxiv.org/abs/2506.04962 [25] V. Nitin, B. Ray, and R. Z. Moghaddam, “Faultline: Automated proof-of-vulnerability generation using llm agents,” 2025. [Online]. Available: https://arxiv.org/abs/2507.15241 [26] Z. Li, S. Dutta, and M. Naik, “IRIS: LLM-assisted static analysis for detecting security vulnerabilities,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=9LdJDU7E91 [27] S. Wu, R. Wang, K. Huang, Y. Cao, W. Song, Z. Zhou, Y. Huang, B. Chen, and X. Peng, “Vision: Identifying affected library versions for open source software vulnerabilities,” in Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1447–1459. [Online]. Available: https://doi.org/10.1145/3691620.3695516 [28] I. Bouzenia and M. Pradel, “You name it, i run it: An llm agent to execute tests of arbitrary projects,” Proc. ACM Softw. Eng., vol. 2, no. ISSTA, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3728922

[29] Z. Yu, Y. Zhang, M. Wen, Y. Nie, W. Zhang, and M. Yang, “Cxxcrafter: An llm-based agent for automated c/c++ open source software building,” Proc. ACM Softw. Eng., vol. 2, no. FSE, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3729386 [30] C. Ma, J. Zhang, Z. Zhu, C. Yang, Y. Yang, Y. Jin, Z. Lan, L. Kong, and J. He, “Agentboard: An analytical evaluation board of multi-turn LLM agents,” in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. [Online]. Available: https://openreview.net/forum?id=4S8agvKjle [31] Z. Chen, Q. Zhan, J. Zhou, X. Hu, X. Xia, and X. Yang, “A large-scale empirical study on the generalizability of disclosed java library vulnerability exploits,” 2026. [Online]. Available: https: //arxiv.org/abs/2603.25997 [32] A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling et al., “Deepseek-v4: Towards highly efficient milliontoken context intelligence,” arXiv preprint arXiv:2606.19348, 2026. [33] Qwen Team, “Qwen3.5: Towards native multimodal agents,” February 2026. [Online]. Available: https://qwen.ai/blog?id=qwen3.5 [34] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. R. Narasimhan, and O. Press, “SWE-agent: Agent-computer interfaces enable automated software engineering,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available: https://arxiv.org/abs/2405.15793 [35] X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y. Shao, N. Muennighoff, Y. Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig, “Openhands: An open platform for ai software developers as generalist agents,” 2025. [Online]. Available: https://arxiv.org/abs/2407.16741 [36] OpenAI, “Introducing codex,” 2025. [Online]. Available: https: //openai.com/index/introducing-codex/ [37] S. Gravitas, “Autogpt,” 2024. [Online]. Available: https://github.com/ Significant-Gravitas/AutoGPT [38] F. Hassan and X. Wang, “Hirebuild: an automatic approach to historydriven repair of build scripts,” in Proceedings of the 40th International Conference on Software Engineering, ser. ICSE ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 1078–1089. [Online]. Available: https://doi.org/10.1145/3180155.3180181 [39] F. Hassan, S. Mostafa, E. S. L. Lam, and X. Wang, “Automatic building of java projects in software repositories: a study on feasibility and challenges,” in Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement,

ser. ESEM ’17. IEEE Press, 2017, p. 38–47. [Online]. Available: https://doi.org/10.1109/ESEM.2017.11 [40] Y. Lou, Z. Chen, Y. Cao, D. Hao, and L. Zhang, “Understanding build issue resolution in practice: symptoms and fix patterns,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2020. New York, NY, USA: Association for Computing Machinery, 2020, p. 617–628. [Online]. Available: https://doi.org/10.1145/3368089.3409760 [41] C. Zhang, B. Chen, X. Peng, and W. Zhao, “Buildsheriff: changeaware test failure triage for continuous integration builds,” in Proceedings of the 44th International Conference on Software Engineering, ser. ICSE ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 312–324. [Online]. Available: https://doi.org/10.1145/3510003.3510132 [42] S. Yang, Y. He, K. Chen, Z. Ma, X. Luo, Y. Xie, J. Chen, and C. Zhang, “1dfuzz: Reproduce 1-day vulnerabilities with directed differential fuzzing,” in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 867–879. [Online]. Available: https://doi.org/10.1145/3597926.3598102 [43] M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran et al., “Why do multi-agent llm systems fail?” Advances in Neural Information Processing Systems, vol. 38, 2026. [44] D. Deshpande, V. Gangal, H. Mehta, J. Krishnan, A. Kannappan, and R. Qian, “Trail: Trace reasoning and agentic issue localization,” 2025. [Online]. Available: https://arxiv.org/abs/2505.08638 [45] W. Epperson, G. Bansal, V. C. Dibia, A. Fourney, J. Gerrits, E. E. Zhu, and S. Amershi, “Interactive debugging and steering of multi-agent ai systems,” in Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, ser. CHI ’25. New York, NY, USA: Association for Computing Machinery, 2025. [Online]. Available: https://doi.org/10.1145/3706598.3713581 [46] M. Ma, J. Zhang, F. Yang, Y. Kang, Q. Lin, S. Rajmohan, and D. Zhang, “Dover: Intervention-driven auto debugging for llm multiagent systems,” arXiv preprint arXiv:2512.06749, 2025. [47] Y. Ge, L. Xie, Z. Li, Y. Pei, and T. Zhang, “Who is introducing the failure? automatically attributing failures of multi-agent systems via spectrum analysis,” 2025. [Online]. Available: https://arxiv.org/abs/ 2509.13782