B OOSTAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
Yuanhao Li 1 Hongbo Wang 1 Xiaotang Shang 1 Xunzhu Tang 2 Yiming Cao 1 Xuhong Chen 1
arXiv:2605.09134v1 [cs.AI] 9 May 2026
Abstract
2023a; Yang et al., 2024; Xia et al., 2024). Despite this progress, several fundamental challenges continue to limit the effectiveness of LLM-based APR systems. First, execution feedback is inherently sparse: a patch either resolves all tests or it does not, providing a binary signal that offers limited guidance for learning. Unlike domains such as text generation where partial success can be meaningfully assessed, program repair admits no natural intermediate reward structure—a patch that passes 99% of tests but fails one critical assertion receives the same negative signal as a syntactically malformed attempt. Second, reward signals in reinforcement learning for APR are typically assigned at the sequence level, creating a severe credit assignment problem. When a 50-line patch succeeds or fails, the model receives no information about which specific edits were beneficial or harmful, leading to highvariance gradient estimates and inefficient learning. Third, the distribution shift between training and evaluation data poses persistent challenges, as models trained on curated datasets often struggle to generalize to the diverse bug patterns encountered in real-world repositories.
Reinforcement learning for program repair is hindered by sparse execution feedback and coarse sequence-level rewards that obscure which edits actually fix bugs. We present B OOSTAPR, a three-stage framework addressing these challenges: (1) supervised fine-tuning on executionverified demonstrations with reasoning traces, (2) training dual reward models—a sequence-level assessor and a line-level credit allocator—from execution outcomes, and (3) PPO optimization where the line-level model redistributes rewards to critical edit regions. This line-level credit assignment operates at an intermediate granularity naturally suited to code changes. Trained on SWEGym and evaluated on four benchmarks, B OOSTAPR achieves 40.7% on SWE-bench Verified (+22.9pp over base model), 24.8% on Defects4J (Python→Java transfer), 84.5% on HumanEvalJava, and 95.0% on QuixBugs, achieving competitive results among open-source models with strong cross-language generalization.
We address these challenges with B OOSTAPR, a principled three-stage training framework that combines executiongrounded learning with fine-grained credit assignment. Our approach integrates: (i) execution-verified reasoning transfer that warm-starts the repair policy with high-quality demonstrations containing both reasoning traces and validated patches; (ii) offline reward learning from strict execution outcomes using a hybrid objective that combines regression for calibrated absolute scores with pairwise preferences for correct relative rankings; and (iii) online PPO with dual reward models—a sequence-level model Rseq that assesses overall patch quality and a novel line-level credit allocator Rline that identifies critical edit regions and redistributes rewards accordingly.
1. Introduction Automated program repair (APR) represents one of the most consequential applications of artificial intelligence to software engineering, with the potential to dramatically reduce the substantial human effort devoted to debugging and maintenance (Le Goues et al., 2019; Monperrus, 2018). The emergence of large language models (LLMs) has catalyzed remarkable progress in this domain, enabling systems that can generate patches conditioned on rich contextual signals including bug descriptions, failing test outputs, and repository-wide code structure (Xia et al., 2023; Jiang et al.,
The central insight motivating our approach is that not all parts of a patch contribute equally to its success or failure. In a typical multi-line patch, some edits directly address the bug’s root cause while others handle edge cases, update documentation, or make stylistic improvements. By learning to score edit-line spans, the Rline component redistributes sequence-level reward to informative regions during
1
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China 2 University of Luxembourg. Correspondence to: Yuanhao Li <[email protected]>. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
1
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
Figure 1. Overview of the B OOSTAPR training framework. Our approach consists of three stages: Stage I performs supervised finetuning on execution-verified demonstrations with reasoning traces; Stage II trains dual reward models using a hybrid regression-preference objective on execution outcomes; Stage III optimizes the policy via PPO with token-level rewards derived from the combination of Rseq and Rline . The line-level allocator redistributes reward to edit-line spans, reducing reward sparsity without requiring counterfactual patch evaluation.
PPO training. This addresses the credit assignment problem that plagues RL for code generation, where assigning identical rewards to all tokens produces noisy gradients and slow convergence.
preference objective, and online policy optimization with token-level rewards derived from structured credit allocation. • Controlled RL comparisons under the same Qwen2.5Coder-32B backbone, SWE-Gym training data, and evaluation protocol, showing that dual rewards improve over GRPO, rejection-sampling RL, and PPO with sequence-level rewards alone.
Critically, our line-level allocator operates at an intermediate granularity between pure token-level and sequencelevel rewards. While token-level rewards (Yoon et al., 2024) can be overly fine-grained for code and sequencelevel rewards are too coarse, line-oriented unified diff spans provide a robust, language-agnostic unit for program repair. We do not claim that line spans are always semantically optimal; rather, they offer a practical compromise that is finer than hunk-level allocation and more stable than language-specific statement parsing under malformed or cross-language patches.
• Extensive ablation studies showing that PPO with Rseq provides the primary accuracy gains, while Rline provides complementary improvements in outof-distribution generalization, training efficiency, and gradient quality. The remainder of this paper is organized as follows. Section 2 situates our work within the broader landscape of LLM-based program repair and reinforcement learning for code. Section 3 presents the technical details of the B OOSTAPR framework, including our formulation of the dual reward architecture and the training objectives for each stage. Section 4 describes our experimental setup and presents comprehensive results across four benchmarks. Section 5 concludes with a discussion of limitations and directions for future work.
We train B OOSTAPR exclusively on SWE-Gym (Pan et al., 2025), a benchmark providing executable training environments for repository-level repair tasks, and evaluate on four diverse benchmarks spanning both repository-level (SWE-bench Verified, Defects4J v2.0) and function-level (HumanEval-Java, QuixBugs) repair scenarios. Our main contributions are: • A dual reward model architecture that combines sequence-level quality assessment with line-level editspan credit allocation, providing a practical mechanism for fine-grained reward distribution during PPO training in code editing tasks.
2. Related Work LLM-Based Program Repair. Early work showed that zero-shot prompting of LLMs could produce meaningful patches (Xia et al., 2023; Sobania et al., 2023). Agentic systems subsequently decomposed repair into localization
• A comprehensive three-stage training pipeline that integrates execution-verified supervised fine-tuning, offline reward learning with a novel hybrid regression2
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
correctness. The system outputs a candidate patch y—a git-applicable unified diff for repository-level benchmarks (SWE-bench, Defects4J) or complete repaired code for function-level benchmarks (HumanEval-Java, QuixBugs).
and generation phases: SWE-Agent (Yang et al., 2024) introduced iterative repair through tool use, Agentless (Xia et al., 2024) achieved competitive performance via hierarchical localization, and AutoCodeRover (Zhang et al., 2024) combined code search with fault localization. Fine-tuning approaches including SWE-Llama (Jimenez et al., 2024), Lingma-SWE-GPT (Ma et al., 2024), RepairLLaMA (Silva et al., 2025), and MORepair (Yang et al., 2025) demonstrated gains from training on repair data, but rely on supervised learning rather than directly optimizing execution success. SWE-RL (Wei et al., 2025) recently applied RL to repository-level repair (41.0% on SWE-bench Verified with 70B parameters). B OOSTAPR is complementary to such scaling and data choices: it studies execution-grounded reward redistribution under a controlled 32B code-model backbone.
We formalize repair as conditional generation where policy πθ (y|x) generates patches given bug contexts. The optimization objective is to maximize expected execution success: max Ex∼D,y∼πθ (·|x) [E(x, y)] , θ
(1)
where E(x, y) ∈ {0, 1} indicates whether patch y resolves all tests for instance x. A key challenge is that this objective provides only sparse binary feedback—a patch either passes all tests or fails, with no intermediate signal about partial correctness. During PPO training, we enforce a patch-only format constraint: the model outputs only unified diff text without natural-language explanations. This ensures rewards reflect patch quality rather than explanation quality, and simplifies credit assignment by focusing on actual code changes.
RL for Code Generation. CodeRL (Le et al., 2022) pioneered actor-critic methods for program synthesis with execution feedback. RLEF (Gehring et al., 2024) extended this to competitive programming, achieving strong results on APPS (Hendrycks et al., 2021) and CodeContests. However, these methods use sparse sequence-level rewards: when a patch passes or fails, the model learns nothing about which edits mattered. This causes high-variance gradients that impede learning—a problem B OOSTAPR addresses through Rline .
Training Data. We train exclusively on SWE-Gym (Pan et al., 2025), which provides executable environments for repository-level repair with real GitHub issues and test suites. We apply strict length filtering (dropping rather than truncating examples exceeding 28K tokens) to prevent learning artifacts from incomplete inputs. For contamination control, we verify that no evaluation instance IDs or patches appear in training data.
Credit Assignment in Sequence Models. Token-level reward models (Yoon et al., 2024) provide dense signals but may be too fine-grained for code where individual tokens lack semantic significance. Process reward models (Lightman et al., 2024) assign step-level feedback for mathematical reasoning, but “steps” do not map naturally to code edits. Attention-based allocation (Chan et al., 2024) and DPO-based rewards (Rafailov et al., 2023) operate at token level using model internals. Our Rline differs by operating at an intermediate granularity—edit lines—that matches the semantic structure of code modifications.
3.2. Stage I: Execution-Verified Reasoning Transfer The first stage initializes the repair policy through supervised fine-tuning on high-quality demonstrations that include explicit reasoning traces and pass strict execution verification. This differs from standard SFT in two key aspects: we require demonstrations to contain diagnostic reasoning, and we retain only patches that actually work. Demonstration Generation. We query a strong teacher model (Claude 3.5 Sonnet) with structured prompts requiring both a reasoning trace and a final patch in unified diff format. The reasoning trace explains the bug diagnosis process: identifying relevant code locations, understanding the root cause, and justifying the chosen fix. This traceand-patch format enables reasoning transfer—the student learns not only what patches to generate but also how to think about repair problems.
3. Method We present B OOSTAPR, a three-stage training framework for automated program repair that combines executionverified reasoning transfer, offline reward learning, and online PPO with dual reward models. Figure 1 illustrates the overall pipeline. 3.1. Problem Formulation
Execution Filtering. Each generated patch is executed against the full test suite using the strict SWE-Gym runner. Only demonstrations achieving resolved=True (all tests pass) are retained; approximately 35% pass this filter. This execution verification is crucial: it ensures the
Automated program repair takes as input a bug instance x consisting of: (i) a repository snapshot containing the buggy code, (ii) a natural-language issue description specifying the desired behavior, and (iii) a test harness that can verify 3
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
where (y + , y − ) is a preference pair with r∗ (y + ) > r∗ (y − ) and w weights by reward gap magnitude. The regression term ensures calibrated absolute scores for proper gradient scaling in PPO; the preference term ensures correct relative rankings for action selection.
model learns from patches that actually work, avoiding noise from plausible-looking but incorrect solutions that often fool surface-level evaluation. Training Objective. We fine-tune with standard nexttoken prediction loss, masking prompt tokens: |y| X LSFT (θ) = −E(x,y)∼DSFT log πθ (yt |x, y<t ) . (2)
3.3.3. L INE -L EVEL C REDIT A LLOCATOR Rline The line-level credit allocator Rline assigns credit over editline spans. Unlike Rseq , which provides a scalar assessment of overall patch quality, Rline learns a distribution over edited regions and enables fine-grained reward assignment during PPO training.
t=1
Training runs for 3 epochs with learning rate 2 × 10−5 and batch size 32. This stage transfers both repair knowledge and diagnostic reasoning patterns from teacher to student.
The central insight is that not all parts of a patch contribute equally to success or failure. In a typical multi-line patch, some edits directly address the bug’s root cause while others handle edge cases or make stylistic improvements. By learning which edits matter most, Rline enables principled redistribution of sequence-level rewards to informative regions.
3.3. Stage II: Dual Reward Learning from Execution The second stage trains dual reward models from execution feedback: Rseq for sequence-level quality assessment and Rline for line-level credit allocation. These models will guide policy optimization in Stage III.
Architecture. Given patch y, we parse the unified diff into edit-line spans—contiguous regions of added or deleted lines, excluding headers and context. For each span ℓ, Rline produces a score sℓ by encoding: (i) the edit content itself, (ii) surrounding context lines, (iii) file path, and (iv) position within the patch. The model is a causal language model with a span-level value head.
3.3.1. R EWARD DATA C OLLECTION For each training instance x, we sample K = 4 diverse candidates from the SFT policy using nucleus sampling (temperature 0.7, top-p 0.9). Each candidate is executed to obtain detailed feedback including application status, test outcomes, and failure traces. We convert execution results into scalar targets: r∗ (x, y) = renv (x, y) + γdiff · rdiff (y),
Allocation Mechanism. Span scores are converted to nonnegative allocation weights via temperature-controlled softmax: exp(sℓ /τ ) , (5) wℓ = P j exp(sj /τ )
(3)
where the environment reward renv = wapply · rapply + wtest · rtest combines patch application success (rapply ∈ {0, 1}) with test pass rate (rtest ∈ [0, 1]), and rdiff = − min(η · |∆(y)|, rmax ) penalizes large edits to encourage minimal patches. This decomposition provides partial credit for patches that apply successfully but fail some tests, offering richer signal than binary success/failure.
where τ = 0.5 provides moderate concentration on highscoring spans while maintaining signal for all regions.
Training and Span Supervision. We train Rline with a contrastive objective derived from execution outcomes and stack-trace-derived span labels. First, we parse each unified 3.3.2. S EQUENCE -L EVEL R EWARD M ODEL Rseq diff into maximal contiguous edit-line spans, excluding diff Rseq (x, y; θ) predicts overall patch quality using a causal headers and context lines. Each candidate patch is then language model with a scalar value head. A key design executed to collect application status, test outcomes, and choice is patch-only scoring: the model receives only the failure traces. For passing patches, all edit spans are treated unified diff without bug context. This prevents learning as positive spans. For failing patches, we apply a priority spurious correlations (e.g., preferring patches for “easier” cascade: when a failing assertion can be identified, we issues) and forces direct evaluation of patch quality. parse the traceback call chain and intersect it with editline spans; spans on the failure path receive negative credit We train with a hybrid objective combining regression and while unrelated spans remain neutral. When the traceback pairwise preference: is available but no clear assertion can be identified, edited h i 2 ∗ functions appearing in the traceback receive lower scores. Lseq (θ) = λreg · E(x,y) (Rseq (y; θ) − r (x, y)) When the patch fails to apply, we assign a uniform fallback + E(y+ ,y− ) −w log σ Rseq (y + ; θ) − Rseq (y − ; θ) , label. In our training data, 9,248 failing patch-span pairs are (4) labeled in this way: 62% use direct stack-trace attribution, 4
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
Algorithm 1 B OOSTAPR Training Pipeline
27% use function-level heuristics, and 11% use the uniform fallback.
Require: Training data D, evaluator E, base policy πbase 1: // Stage I: Execution-Verified SFT This procedure is execution-grounded stack-trace supervi2: Generate demonstrations with reasoning traces from sion rather than counterfactual patch evaluation, attention teacher attribution, or ground-truth-patch matching. Let ℓ+ denote − 3: Filter to execution-verified demonstrations (E = 1) spans from successful repairs and ℓ denote failing spans 4: π0 ← SFT(πbase , DSFT ) selected by the cascade: 5: // Stage II: Dual Reward Learning Lline (ϕ) = E(ℓ+ ,ℓ− ) − log σ Rline (ℓ+ ; ϕ) − Rline (ℓ− ; ϕ) . 6: for each x ∈ D do (6) 7: Sample candidates {yk }K k=1 ∼ π0 (·|x) This teaches Rline to rank beneficial spans above harmful 8: Execute and compute r∗ (x, yk ) for each ones. The labels are necessarily noisy, so we treat Rline as 9: end for a complementary reward redistribution mechanism rather 10: Train Rseq with hybrid objective (Eq. 4) than the sole source of repair performance. 11: Train Rline with contrastive objective (Eq. 6) 12: // Stage III: Online PPO 13: for step = 1, . . . , N do 3.4. Stage III: Online PPO with Dual Rewards 14: Sample rollouts (x, y) ∼ π(·|x) The final stage performs online policy optimization us15: Compute token rewards via Rseq , Rline (Eq. 8) ing Proximal Policy Optimization (PPO) (Schulman et al., 16: Update π with clipped PPO (Eq. 9) 2017), with token-level rewards derived from combining 17: end for Rseq and Rline . We implement training using VERL (Sheng 18: returnπfinal et al., 2024) with vLLM (Kwon et al., 2023) for efficient parallel rollout generation. This preserves total sequence reward while distributing it according to learned edit importance.
3.4.1. T OKEN -L EVEL R EWARD S HAPING Stage III distributes the sequence-level reward from Rseq across tokens using Rline ’s credit allocation. Given rollout (x, y) with y = (y1 , . . . , yT ), we construct token rewards through the following procedure:
3.4.2. P OLICY O PTIMIZATION We optimize using the standard clipped PPO objective. Let ρt (θ) = πθ (yt |x, y<t )/πθold (yt |x, y<t ) be the importance ratio and At the advantage estimated via Generalized Advantage Estimation (GAE) (Schulman et al., 2016):
(1) Sequence scoring: Compute overall quality score s = Rseq (y). (2) Span extraction: Parse y into edit-line spans. If parsing fails (malformed diff), fall back to assigning the full score to the final token.
Jclip (θ) = Et [min (ρt At , clip(ρt , 1 − ϵ, 1 + ϵ)At )] , (9) with clip ratio ϵ = 0.2. To prevent the policy from deviating too far from the initial distribution, we employ KL regularization against a frozen reference policy πref , with adaptive coefficient β targeting KL divergence of 0.1. PPO runs for 300 steps with batch size 64 and 4 rollouts per instance, using LoRA (rank 64) for parameter-efficient updates. The complete procedure is summarized in Algorithm 1.
(3) Credit allocation: For each span ℓ, compute allocation weight wℓ via Eq. 5. Map span weights to tokens: tokens within span ℓ receive weight wℓ /nℓ (where nℓ is token count); tokens outside edit spans (headers, context lines) receive zero weight. (4) Format penalty: Apply deterministic penalty rfmt (y) to the final token based on output structure: 0 if valid unified diff −0.4 if recoverable format rfmt (y) = (7) −1.0 if malformed diff −1.5 if not a diff
4. Experiments We evaluate B OOSTAPR across four diverse benchmarks and conduct extensive ablations to understand the contribution of each component. 4.1. Experimental Setup
The combined token reward is: rt = s · at + I[t = T ] · rfmt (y), where at is the normalized allocation weight (
Training Configuration. We train all components on SWE-Gym (train split) with an 8:2 train/dev partition. SFT runs for 3 epochs with learning rate 2 × 10−5 and batch size 32. Reward models train for 5 epochs with learning rate
(8) P
t at = 1).
5
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
1 × 10−5 and batch size 64. PPO runs for 300 steps with batch size 64 and 4 rollouts per instance, using LoRA (rank 64) for parameter-efficient updates.
The training stages contribute additively: Stage I (executionverified SFT) adds +5.6pp through reasoning transfer, Stage III with Rseq alone adds +14.9pp through direct policy optimization, and the Rline credit allocator contributes +2.4pp through fine-grained reward shaping. PPO with Rseq alone accounts for 65% of total improvement, confirming that RL from execution feedback is the primary driver of performance gains.
Model Architecture. Our base policy is Qwen2.5-Coder32B-Instruct (Hui et al., 2024), a strong open-source code model. For efficiency, Rseq and Rline use Qwen2.5-Coder7B-Instruct backbones with scalar value heads.
Defects4J v2.0 (Repository-Level Java). On the standard Java APR benchmark, B OOSTAPR achieves 24.8% (207/835 bugs), more than doubling the base model’s performance (11.3%) despite Java being absent from training data. The strong cross-language transfer suggests that B OOSTAPR learns repair strategies that generalize beyond Pythonspecific surface patterns.
Evaluation Benchmarks. We evaluate on four benchmarks spanning different repair granularities and programming languages: • SWE-bench Verified (Jimenez et al., 2024): 500 human-validated repository-level Python bugs from real GitHub issues.
Notably, the Rline component provides larger relative gains on Defects4J (+5.6pp) compared to SWE-bench Verified (+2.4pp), suggesting that fine-grained credit assignment is particularly valuable when generalizing to out-ofdistribution scenarios.
• Defects4J v2.0 (Just et al., 2014): 835 bugs across 17 Java projects, the standard Java APR benchmark. • HumanEval-Java (Chen et al., 2021): 164 functionlevel repair tasks adapted from the HumanEval benchmark.
HumanEval-Java (Function-Level). B OOSTAPR achieves 84.5% on function-level Java repair, a +20.5pp improvement over the base model. This indicates that repository-level training can transfer to simpler, isolated function repair tasks, likely because the model learns debugging patterns applicable across granularities.
• QuixBugs (Lin et al., 2017): 40 classic algorithmic bugs with known fixes. Evaluation Protocol. All results use strict evaluation on raw model outputs—no patch post-processing, syntax correction, or multiple-attempt filtering. We report pass@1 (greedy decoding) and pass@4 (best of 4 samples with temperature 0.2, top-p 0.95). A candidate is solved if: (i) it produces a git-applicable unified diff (for SWE-bench/Defects4J) or valid code (for HumanEvalJava/QuixBugs), and (ii) all tests pass.
QuixBugs (Classic Algorithmic Bugs). On the QuixBugs benchmark of 40 classic single-line bugs, B OOSTAPR achieves 95.0% (38/40). The base model already performs well (90.0%) on these relatively simple bugs; our training closes most of the remaining gap. Summary. Across all four benchmarks, B OOSTAPR improves the base model by +5.0 to +22.9pp. For concurrent or larger-scale systems using different base models, data scales, context lengths, or test-time scaling, we report results as contextual references rather than absolute leaderboard comparisons. The controlled same-backbone comparisons below isolate the contribution of our execution-grounded RL and line-level reward redistribution.
4.2. Main Results Table 1 presents comprehensive results across four benchmarks spanning repository-level and function-level program repair. We compare B OOSTAPR against three categories of baselines: agentic systems that leverage proprietary models through sophisticated scaffolding, fine-tuned models trained via supervised learning, and RL-based methods that optimize for execution outcomes.
4.3. Controlled RL Baselines
SWE-bench Verified (Repository-Level Python). B OOSTAPR achieves 40.7% resolve rate, representing a +22.9 percentage point improvement over the base Qwen2.5-Coder-32B model. This result is comparable to SWE-RL (Wei et al., 2025) (41.0%) while using a different and smaller backbone (32B vs. 70B). We therefore treat cross-backbone results as contextual references and rely on the controlled comparisons in Table 2 for apples-to-apples conclusions.
To control for confounding from model capacity, training data, and evaluation protocol, we compare representative RL baselines under the same Qwen2.5-Coder-32B backbone and SWE-Gym training pool. Table 2 isolates the training algorithm: B OOSTAPR outperforms GRPO by +4.6pp and rejection-sampling RL by +3.2pp on SWE-bench Verified, with larger gains on the out-of-distribution Defects4J benchmark. 6
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 1. Main results across four benchmarks. All numbers are pass@1 percentages under strict evaluation. For repository-level benchmarks (SWE-bench Verified and Defects4J v2.0), we report resolve rate. For function-level benchmarks (HumanEval-Java and QuixBugs), we report the percentage of correctly fixed bugs. Results for external baselines are taken from original papers where available; entries marked with ∗ are from our reproduction using the same evaluation protocol. All B OOSTAPR results use identical settings across benchmarks. Method
Backbone
Params
SWE-V
D4J v2.0
HE-Java
QuixBugs
Agentic / Prompting Systems: Agentless (Xia et al., 2024) SWE-agent (Yang et al., 2024) AutoCodeRover (Zhang et al., 2024) ChatRepair (Xia & Zhang, 2024)
GPT-4o Claude 3.5 Sonnet GPT-4o GPT-3.5-turbo
– – – –
38.8 33.6 28.8 18.2∗
12.4∗ 10.8∗ 9.6∗ 14.4
71.3∗ 68.9∗ 65.2∗ 72.0∗
87.5∗ 85.0∗ 82.5∗ 100.0
Fine-tuned Models (Supervised Learning): SWE-Gym (Pan et al., 2025) Qwen2.5-Coder-32B Lingma SWE-GPT (Ma et al., 2024) Qwen2.5-72B SWE-Fixer (Xie et al., 2025) Qwen2.5-72B RepairLLaMA (Silva et al., 2025) CodeLlama-7B KNOD (Jiang et al., 2023b) CodeT5-base
32B 72B 72B 7B 220M
32.0 30.2 33.0 8.6∗ 2.4∗
13.1∗ 14.5∗ 15.2∗ 17.2 6.0
70.7∗ 72.6∗ 73.8∗ 66.5 58.5∗
90.0∗ 92.5∗ 92.5∗ 75.0∗ 62.5
RL-based Methods: CodeRL (Le et al., 2022) RLEF (Gehring et al., 2024) SWE-RL (Wei et al., 2025)
CodeT5-large Llama-3-8B Llama-3-70B
770M 8B 70B
3.2∗ 12.6∗ 41.0
5.8∗ 8.4∗ 16.8∗
63.0 74.3 76.2∗
67.5∗ 80.0∗ 90.0∗
Our Method: Qwen2.5-Coder-32B (base) + Stage I (SFT) + Stage III (PPO, Rseq only) B OOSTAPR (+ Rline )
– – – –
32B 32B 32B 32B
17.8 23.4 38.3 40.7
11.3 14.9 19.2 24.8
64.0 73.1 79.4 84.5
90.0 92.5 95.0 95.0
+22.9 +2.4
+13.5 +5.6
+20.5 +5.1
+5.0 +0.0
Improvement over base model Gain from Rline over PPO+Rseq
Notes: SWE-V = SWE-bench Verified (500 instances); D4J v2.0 = Defects4J version 2.0 (835 bugs); HE-Java = HumanEval-Java (164 tasks); QuixBugs = QuixBugs-Java (40 bugs). Entries marked with ∗ are reproduced under our evaluation protocol and are therefore contextual rather than direct controlled comparisons. SWE-RL uses a larger 70B backbone. Table 2. Controlled same-backbone RL comparison. All methods use Qwen2.5-Coder-32B, SWE-Gym training data, and the same evaluation protocol.
Table 3. Component ablation. Each component contributes additively to final performance. Variant
Method SFT + GRPO SFT + RS-RL PPO + Rseq Full B OOSTAPR
SWE-V
D4J
HE-Java
36.1 37.5 38.3 40.7
16.4 17.6 19.2 24.8
75.2 77.8 79.4 84.5
Base model (no training) SFT only SFT + Rseq (reranking only) PPO + Rseq (w/o Rline ) PPO + Rseq + Rline (full)
Stability. Across three training seeds on SWE-bench Verified, PPO+Rseq obtains 38.3±0.3%, while full B OOSTAPR obtains 40.7±0.5%. A paired bootstrap test gives p = 0.012 with a 95% confidence interval of [+1.2,+3.6] percentage points for the Rline gain. On Defects4J, the +5.6±0.7pp gain is also significant (p < 0.001). These results indicate that Rline provides a modest but stable improvement on the primary benchmark and a larger improvement under cross-language transfer.
pass@1
pass@4
17.8 23.4 26.1 38.3 40.7
20.2 25.7 28.3 40.1 44.3
ablations evaluate on SWE-bench Verified with pass@1 and pass@4 metrics. Component Contributions. Table 3 ablates key components of B OOSTAPR. Execution-verified SFT improves pass@1 from 17.8% to 23.4% (+5.6pp), and online PPO with Rseq provides the largest gain, reaching 38.3%. The line-level allocator Rline then adds +2.4pp on SWE-bench Verified and +5.6pp on Defects4J (Table 1). Thus, PPO with sequence-level execution rewards is the primary accuracy driver, while Rline provides complementary gains through finer reward redistribution, especially under out-ofdistribution transfer.
4.4. Ablation Studies We conduct extensive ablations to understand the contribution of each component. Unless otherwise noted, all 7
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 4. Credit assignment granularity. Edit-line spans provide a robust intermediate unit for diff-based repair. Granularity Token-uniform Hunk-level Statement-level Edit-line spans (Rline )
Table 5. Reward model input mode. Patch-only scoring outperforms context-conditioned variants.
SWE-V
D4J
Rseq Input Mode
pass@1
pass@4
37.6 39.1 40.3 40.7
17.8 21.3 22.1 24.8
Patch-only scoring Full context scoring Issue + patch scoring
40.7 38.9 39.4
44.3 42.1 42.8
Reward Model Training Objective. We compare training objectives for Rseq . Pure regression achieves 39.2% pass@1, while pure preference learning (Bradley–Terry) achieves 38.8%. The hybrid objective performs best (40.7%), combining calibrated absolute scores (useful for PPO scaling) with reliable relative rankings. Pure regression is noise-sensitive, while pure preference can yield uncalibrated scores that destabilize PPO.
45 40.7%
pass@1 (%)
40 39.1%
35
35.2%
30 25
PPO Training Dynamics. Figure 2 shows performance during PPO training: accuracy improves quickly early on and reaches 40.7% by step 300, after which gains plateau (< 0.3pp) and additional training risks overfitting. With Rline , training reaches the no-Rline plateau of 38.3% around step 200 rather than around step 300, and gradient signalto-noise ratio improves from 1.42 to 1.83 (+29%). These diagnostics support the view that Rline primarily improves reward allocation and optimization stability rather than replacing sequence-level execution rewards.
SFT Baseline PPO w/o Rline
20
BoostAPR (Ours)
0
100
200
300
Training Steps Figure 2. PPO training dynamics. Performance improves steadily until approximately step 250, then plateaus. Shaded region shows standard deviation across 3 seeds.
5. Conclusion
Credit Assignment Strategies. Table 4 compares reward redistribution units. Token-uniform rewards dilute signal across non-informative formatting and context tokens. Hunk-level rewards are more stable but too coarse for multiedit patches. Statement-level rewards perform competitively on Python but require language-specific parsing and fall back when parser-based extraction is unavailable. Editline spans are line-oriented, robust to malformed diffs, and language-agnostic, giving the strongest results on both SWEbench Verified and Defects4J.
We presented B OOSTAPR, a three-stage framework that addresses sparse execution feedback and coarse reward signals in program repair through execution-verified SFT, dual reward learning, and PPO with line-level edit-span credit allocation. Our approach achieves 40.7% on SWE-bench Verified (+22.9pp over base model), 24.8% on Defects4J, 84.5% on HumanEval-Java, and 95.0% on QuixBugs. Controlled same-backbone comparisons show that PPO with sequence-level execution rewards provides the primary accuracy gain, while Rline provides complementary benefits in out-of-distribution generalization, training efficiency, and gradient quality. Limitations remain: Rline supervision is partially heuristic, edit-line spans are an approximate semantic unit, and comparisons with concurrent large-scale systems are confounded by base model, data scale, and testtime scaling. Future work may combine line-level credit allocation with stronger fault localization and larger-scale RL pipelines.
Reward Model Input. Table 5 examines the impact of what context Rseq receives during scoring. Counterintuitively, patch-only scoring (where Rseq sees only the unified diff) outperforms variants that include bug context. Full context scoring achieves only 38.9%, and including just the issue description with the patch achieves 39.4%. We hypothesize that providing context enables the reward model to learn spurious correlations—for instance, assigning higher scores to patches for “easier” issues regardless of actual patch quality, or learning to recognize patterns in issue descriptions that correlate with success in the training set but do not generalize. Patch-only scoring forces the model to evaluate patch quality directly based on code change patterns, leading to better generalization. 8
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
Impact Statement
Just, R., Jalali, D., and Ernst, M. D. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA), pp. 437–440, 2014.
This work advances automated program repair to reduce maintenance costs and accelerate debugging. By learning directly from execution feedback, B OOSTAPR moves toward AI systems that can assist developers in fixing real-world bugs. Potential risks include over-reliance on automated repair, patches that pass tests yet introduce subtle bugs, and misuse for malicious code changes. We encourage responsible deployment with human oversight and rigorous testing beyond unit tests.
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), 2023. Le, H., Wang, Y., Gotmare, A. D., Savarese, S., and Hoi, S. C. CodeRL: Mastering code generation through pretrained models and deep reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. 21314–21328, 2022.
References Chan, A. J., Sun, H., Holt, S., and van der Schaar, M. Dense reward for free in reinforcement learning from human feedback. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. arXiv:2402.00782.
Le Goues, C., Pradel, M., and Roychoudhury, A. Automated program repair. Communications of the ACM, 62(12):56– 65, 2019. doi: 10.1145/3318162.
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
Gehring, J., Zheng, K., Copet, J., Mella, V., Carbonneaux, Q., Cohen, T., and Synnaeve, G. RLEF: Grounding code LLMs in execution feedback with reinforcement learning. arXiv preprint arXiv:2410.02089, 2024.
Lin, D., Koppel, J., Chen, A., and Solar-Lezama, A. QuixBugs: A multi-lingual program repair benchmark set based on the Quixey challenge. In Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH Companion), pp. 55–56, 2017.
Hendrycks, D., Basart, S., Kadavath, S., Mazeika, M., Arora, A., Guo, E., Burns, C., Puranik, S., He, H., Song, D., and Steinhardt, J. Measuring coding challenge competence with APPS. In Advances in Neural Information Processing Systems, 2021. Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Dang, K., et al. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186, 2024.
Ma, Y., Cao, R., Cao, Y., Zhang, Y., Chen, J., Liu, Y., Liu, Y., Li, B., Huang, F., and Li, Y. Lingma SWE-GPT: An open development-process-centric language model for automated software improvement. arXiv preprint arXiv:2411.00622, 2024.
Jiang, N., Liu, K., Lutellier, T., and Tan, L. Impact of code language models on automated program repair. In Proceedings of the 45th International Conference on Software Engineering (ICSE), pp. 1430–1442, 2023a. doi: 10.1109/ICSE48619.2023.00125.
Monperrus, M. Automatic software repair: A bibliography. ACM Computing Surveys (CSUR), 51(1):1–24, 2018.
Jiang, N., Lutellier, T., Lou, Y., Tan, L., Goldwasser, D., and Zhang, X. KNOD: Domain knowledge distilled tree decoder for automated program repair. In Proceedings of the 45th IEEE/ACM International Conference on Software Engineering (ICSE), pp. 1–13. IEEE, 2023b.
Pan, J., Wang, X., Neubig, G., Jaitly, N., Ji, H., Suhr, A., and Zhang, Y. Training software engineering agents and verifiers with SWE-Gym. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., and Zhu, J. (eds.), Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 47717–47737. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/ v267/pan25g.html.
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. SWE-bench: Can language models resolve real-world GitHub issues? In The Twelfth International Conference on Learning Representations (ICLR), 2024. arXiv:2310.06770. 9
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems (NeurIPS), 2023.
LLMs to repair code via multi-objective fine-tuning. ACM Transactions on Software Engineering and Methodology, 2025. doi: 10.1145/3735129. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K. R., and Press, O. SWE-agent: Agentcomputer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), 2024. arXiv:2405.15793.
Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. High-dimensional continuous control using generalized advantage estimation. In Proceedings of the International Conference on Learning Representations (ICLR), 2016.
Yoon, E., Yoon, H. S., Eom, S., Han, G., Nam, D. W., Jo, D., On, K.-W., Hasegawa-Johnson, M. A., Kim, S., and Yoo, C. D. TLCR: Token-level continuous reward for fine-grained reinforcement learning from human feedback. In Findings of the Association for Computational Linguistics: ACL 2024, 2024.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. HybridFlow: A flexible and efficient RLHF framework. arXiv preprint arXiv:2409.19256, 2024. Accepted to EuroSys 2025.
Zhang, Y., Ruan, H., Fan, Z., and Roychoudhury, A. AutoCodeRover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pp. 1–13, 2024. arXiv:2404.05427.
Silva, A., Fang, S., and Monperrus, M. Repairllama: Efficient representations and fine-tuned adapters for program repair. IEEE Transactions on Software Engineering, 2025. doi: 10.1109/TSE.2025.3581062. Sobania, D., Briesch, M., Hanna, C., and Petke, J. An analysis of the automatic bug fixing performance of ChatGPT. In 2023 IEEE/ACM International Workshop on Automated Program Repair (APR), pp. 23–30, 2023. Wei, Y., Duchenne, O., Copet, J., Carbonneaux, Q., Zhang, L., Fried, D., Synnaeve, G., Singh, R., and Wang, S. I. SWE-RL: Advancing LLM reasoning via reinforcement learning on open software evolution. In Advances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2502.18449. Xia, C. S. and Zhang, L. Automated program repair via conversation: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pp. 1–13. ACM, 2024. Xia, C. S., Wei, Y., and Zhang, L. Automated program repair in the era of large pre-trained language models. In Proceedings of the 45th International Conference on Software Engineering (ICSE), pp. 1482–1494, 2023. Xia, C. S., Deng, Y., Dunn, S., and Zhang, L. Agentless: Demystifying LLM-based software engineering agents. arXiv preprint arXiv:2407.01489, 2024. Xie, C., Li, B., Gao, C., Du, H., Lam, W., Zou, D., and Chen, K. SWE-Fixer: Training open-source LLMs for effective and efficient GitHub issue resolution. arXiv preprint arXiv:2501.05040, 2025. Yang, B., Tian, H., Ren, J., Zhang, H., Klein, J., Bissyandé, T. F., Le Goues, C., and Jin, S. MORepair: Teaching 10
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
A. Theoretical Analysis This appendix provides formal analysis of the credit assignment problem in sequence-level RL for code generation and establishes conditions under which the line-level allocator Rline reduces gradient variance compared to standard approaches. A.1. Variance Analysis of Policy Gradient Estimators Consider a policy πθ generating sequences y = (y1 , . . . , yT ) given context x. The policy gradient for maximizing expected reward R(x, y) is: " # T X ∇θ J(θ) = Ex,y∼πθ R(x, y) ∇θ log πθ (yt |x, y<t ) . (10) t=1
Proposition A.1 (Variance of Sequence-Level Rewards). Under sequence-level reward assignment where rt = R(x, y)·I[t = T ], the variance of the policy gradient estimator scales as: h i ˆ θ J = O (T · Var[R]) , Var ∇ (11) where T is the sequence length. Proof. The policy gradient estimator using a single sample is: ˆ θ J = R(x, y) ∇
T X
∇θ log πθ (yt |x, y<t ).
(12)
t=1
Taking the variance: h
i
ˆ θJ = E R Var ∇
2
X
∇θ log πθ (yt |x, y<t )
− ∥E[·]∥2
(13)
t
≤ E R2
2
X
∇θ log πθ (yt |x, y<t )
.
(14)
t
Under standard assumptions of bounded gradients and approximate independence across time steps: 2 X E ∇θ log πθ (yt |x, y<t ) = O(T ),
(15)
t
ˆ θ J] = O(T · Var[R]). yielding Var[∇ This linear scaling with sequence length explains why sequence-level rewards lead to high-variance gradients for long code patches. P Proposition A.2 (Variance Reduction via Credit Allocation). Let a = (a1 , . . . , aT ) be an allocation scheme with t at = 1 and at ≥ 0. Define token-level rewards rt = R · at . If there exists a subset S ⊂ {1, . . . , T } of “informative” tokens with |S| = k < T such that the optimal allocation concentrates on S, then: i h ˆ θJ Var ∇ = O (k · Var[R]) , (16) allocated
achieving a variance reduction factor of T /k. Proof. With allocation at , the policy gradient becomes: ˆ θJ = ∇
T X
rt ∇θ log πθ (yt |x, y<t ) = R
t=1
T X t=1
11
at ∇θ log πθ (yt |x, y<t ).
(17)
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
If allocation concentrates on S, i.e., at ≈ 0 for t ∈ / S and ˆ θJ ≈ R ∇
X
P
t∈S at ≈ 1:
at ∇θ log πθ (yt |x, y<t ).
(18)
t∈S
Following the same analysis as Proposition A.1: h i ˆ θ J = O(|S| · Var[R]) = O(k · Var[R]). Var ∇
(19)
Remark A.3. For code patches, edit lines typically constitute a small fraction of the total output. If a 100-token patch contains 20 tokens of actual edits (with the rest being headers, context, and formatting), a perfect allocator achieves 5× variance reduction. A.2. Optimal Allocation and the Role of Rline The theoretical analysis above assumes access to an oracle allocation. In practice, Rline learns to approximate this allocation from execution feedback. We now characterize the properties of an optimal allocator. Definition A.4 (Causal Attribution). For a patch y with outcome R(x, y), define the causal attribution of token t as: CAt (y) = R(x, y) − Eyt′ ∼πθ [R(x, y<t , yt′ , y>t )],
(20)
measuring the expected change in outcome from replacing token t. Proposition A.5 (Optimal Allocation). The variance-minimizing allocation, subject to proportional to causal attribution magnitude: a∗t ∝ |CAt (y)|.
P
t at
= 1 and at ≥ 0, is (21)
Computing exact causal attributions requires counterfactual evaluation, which is prohibitively expensive. Rline approximates this through learned span-level scores that correlate with causal importance, as evidenced by its ability to identify critical edit regions. A.3. Convergence Analysis We analyze the convergence properties of PPO with dual reward models under standard assumptions. Assumption A.6 (Smoothness). The policy πθ is L-smooth in θ: ∥∇θ πθ (y|x) − ∇θ πθ′ (y|x)∥ ≤ L∥θ − θ′ ∥. Assumption A.7 (Bounded Rewards). |R(x, y)| ≤ Rmax and |rfmt (y)| ≤ Fmax for all x, y. Theorem A.8 (Convergence Rate). Under Assumptions 1-2, PPO with dual reward models converges to a stationary point at rate: 1 σ2 2 √ min E ∥∇θ J(θt )∥ = O + , (22) t≤T B T where T is the number of update steps, B is batch size, and σ 2 is the gradient variance (reduced by Rline per Proposition A.2). The proof follows standard PPO convergence analysis (Schulman et al., 2017) with the observation that Rline reduces σ 2 , improving the second term.
B. Implementation Details B.1. Supervised Fine-Tuning Hardware and Software. We train on 8 NVIDIA A100 80GB GPUs using DeepSpeed ZeRO-3 for memory-efficient distributed training. The training framework is built on Hugging Face Transformers with custom data loading for unified diff format. 12
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
Hyperparameters. • Learning rate: 2 × 10−5 with cosine decay • Warmup: 100 steps (linear) • Batch size: 32 (4 per GPU × 8 GPUs) • Epochs: 3 • Maximum sequence length: 32K tokens • Weight decay: 0.01 • Gradient clipping: 1.0 Data Format.
Each training example consists of:
1. System prompt: Instructions for generating reasoning traces and unified diffs 2. Bug context: Repository path, issue description, relevant code snippets 3. Response: Reasoning trace followed by unified diff patch The reasoning trace follows a structured format: (1) Bug Analysis, (2) Root Cause Identification, (3) Fix Strategy, (4) Implementation. This structure enables consistent reasoning transfer. B.2. Reward Model Training Rseq Architecture. We use Qwen2.5-Coder-7B-Instruct as the backbone, adding a scalar value head (linear layer) on top of the final hidden state. The value head is initialized with small random weights (N (0, 0.01)). Rline Architecture.
Rline uses the same backbone but with a span-level value head. For each edit span, we:
1. Extract the hidden states corresponding to span tokens 2. Apply mean pooling across the span 3. Pass through a two-layer MLP (hidden dim 512, ReLU activation) to produce a scalar score Training Details. • Learning rate: 1 × 10−5 • Batch size: 64 • Epochs: 5 • Optimizer: AdamW (β1 = 0.9, β2 = 0.999) • Hybrid loss weight: λreg = 0.5 B.3. PPO Training Infrastructure. We use VERL (Sheng et al., 2024) for distributed PPO training with vLLM (Kwon et al., 2023) as the inference engine. The setup includes: • 8 A100 GPUs for policy rollouts (vLLM) • 8 A100 GPUs for policy/critic updates • Separate reward model inference servers 13
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
PPO Hyperparameters. • Batch size: 64 instances • Rollouts per instance: 4 • PPO epochs per batch: 4 • Clip ratio ϵ: 0.2 • GAE λ: 0.95 • Discount γ: 0.99 • Entropy coefficient: 0.01 • KL coefficient β: 0.001 (initial), adaptively controlled • Target KL: 0.1 • LoRA rank: 64 • LoRA alpha: 128 • Training steps: 300 Critic Training. The critic (value function) uses the same architecture as the policy with a value head. It is trained with squared TD error: h i 2 LV (ψ) = E (Vψ (x, y≤t ) − Gt ) , (23) where Gt is the return computed from shaped rewards.
C. Additional Experimental Results C.1. Per-Project Breakdown on Defects4J Table 6 provides detailed results by project. Performance varies across projects, with higher success rates on projects with cleaner test suites and more localized bugs (Chart, Lang, Csv) and lower rates on large, complex projects (Closure, JacksonDatabind). C.2. Reward Model Quality Metrics Table 7 presents detailed evaluation of both reward models. C.3. Line-Level Supervision Quality The Rline labels combine direct stack-trace attribution with heuristic fallback. Table 8 reports a preliminary manual audit of 150 sampled spans. Direct stack-trace labels agree with manual judgments more often than function-level heuristic labels, confirming that heuristic supervision is useful but noisier. We therefore treat span supervision quality as a limitation and a concrete direction for future improvement. The stratified result suggests that Rline is most effective when failure traces provide direct span-level evidence. As an upper-bound diagnostic, upweighting 80 manually annotated high-quality Rline labels by 5× improves SWE-bench Verified from 40.7% to 42.1%, suggesting that cleaner supervision could provide an additional +1.4pp. 14
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 6. Per-project results on Defects4J v2.0.
Project
# Bugs
pass@1
pass@4
Chart Closure Lang Math Mockito Time Codec Collections Compress Csv Gson JacksonCore JacksonDatabind JacksonXml Jsoup JxPath Others
26 133 64 106 38 27 18 4 47 16 18 26 112 6 93 22 79
30.8% 18.8% 32.8% 28.3% 21.1% 25.9% 27.8% 25.0% 21.3% 31.3% 22.2% 19.2% 20.5% 16.7% 24.7% 22.7% 32.9%
34.6% 22.6% 35.9% 31.1% 26.3% 29.6% 33.3% 25.0% 25.5% 37.5% 27.8% 23.1% 24.1% 16.7% 28.0% 27.3% 30.4%
Total
835
24.8%
27.9%
Table 7. Reward model quality on held-out validation data.
Metric
Rseq
Rline
Pairwise accuracy ROC-AUC (success vs. failure) Spearman correlation Top-1 hit rate Top-3 hit rate
82.4% 0.891 0.743 – –
78.6% – – 67.3% 84.1%
C.4. Additional Controlled Analyses Across backbones, Rline contributes +2.2pp, +2.7pp, and +2.4pp, respectively. This supports the claim that edit-line credit allocation is not tied to a single model size. Execution-grounded rewards outperform lexical overlap because many valid repairs differ substantially from the reference patch. In our SWE-bench analysis, 41% of correctly fixed bugs have less than 50% token-level Jaccard overlap with the reference patch. C.5. Statistical Testing Details For the three-seed comparison in Section 4, SFT obtains 23.4±0.2%, PPO+Rseq obtains 38.3±0.3%, and full B OOSTAPR obtains 40.7±0.5%. We compute paired bootstrap intervals over benchmark instances and use McNemar’s test for instancelevel changes. For Seed 3 on SWE-bench Verified, Rline newly resolves 30 instances, degrades 18, and leaves 452 unchanged, giving McNemar’s p = 0.049. We report these tests as stability diagnostics rather than as evidence of universal superiority over systems trained with different backbones or data. C.6. Worked Example of Span Label Derivation Given a candidate unified diff, we first extract contiguous edit-line spans from added and deleted lines. If execution fails with an assertion trace that includes a function modified by one span, that span receives negative credit and unrelated spans are neutral. If several edited spans occur on the failing call path, all such spans are treated as failing candidates for contrastive pairing. If execution succeeds, all edit-line spans from the patch are positive. If the patch cannot be applied, no 15
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 8. Preliminary audit of Rline span labels.
Source
N
Agreement
Cohen’s κ
Stack-trace attribution Function-level heuristic Uniform fallback
50 50 50
82% (41/50) 64% (32/50) 94% (47/50)
0.72 0.56 N/A
Table 9. Stratified analysis by supervision quality.
Partition
N
PPO+Rseq
Full
∆
Strong trace analogues in training Heuristic-only analogues
312 188
39.7 35.6
43.3 36.7
+3.6pp +1.1pp
reliable trace exists, and the uniform fallback label is used. This example illustrates why Rline is best understood as noisy execution-grounded credit redistribution rather than exact causal attribution. C.7. Sensitivity Analysis Number of Candidates K.
Table 13 shows the impact of candidate count during reward model training.
K = 4 provides a good balance between diversity and computational cost. Temperature τ for Rline .
Table 14 examines allocation sharpness.
Moderate sharpness (τ = 0.5) works best, balancing focused credit assignment with sufficient gradient signal across edit regions. C.8. Computational Cost Table 15 summarizes computational requirements. PPO training dominates the computational budget due to the need for rollouts, reward computation, and policy updates.
D. Extended Qualitative Analysis D.1. Successful Repair Examples Issue ID: django django-15790 Issue Description: check for template tags with the same name fails with AttributeError Ground Truth Patch: --- a/django/core/checks/templates.py +++ b/django/core/checks/templates.py @@ -61,7 +61,8 @@ def check for template tags with the same name(app configs, **kwargs): libraries[module name] = module path + if module path not in libraries.values(): + libraries[module name] = module path B OOSTAPR Patch: --- a/django/core/checks/templates.py +++ b/django/core/checks/templates.py @@ -61,7 +61,9 @@ def check for template tags with the same name(app configs,
16
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 10. Rline improves more than one RL algorithm, suggesting that edit-line credit allocation is composable rather than PPO-specific.
Base RL
SWE w/o Rline
SWE w/ Rline
∆ SWE
∆ D4J
GRPO PPO+Rseq
36.1 38.3
38.4 40.7
+2.3pp +2.4pp
+3.7pp +5.6pp
Table 11. Additional backbone results on SWE-bench Verified.
Backbone
Base
+SFT
+PPO(Rseq )
+Rline
Total ∆
Qwen2.5-Coder-7B DeepSeek-Coder-V2-16B Qwen2.5-Coder-32B
8.6 12.4 17.8
12.8 17.1 23.4
22.4 28.6 38.3
24.6 31.3 40.7
+16.0pp +18.9pp +22.9pp
**kwargs): libraries[module name] = module path + # Avoid duplicate entries that cause AttributeError + if module path not in libraries.values(): + libraries[module name] = module path Analysis: B OOSTAPR produces a functionally equivalent patch with an additional comment explaining the fix rationale. Both patches prevent duplicate template tag entries that cause the AttributeError. Issue ID: scikit-learn scikit-learn-25570 Issue Description: ColumnTransformer with pandas output fails when transformers return DataFrames with unnamed columns B OOSTAPR Analysis: The model correctly identifies that the issue is in the wrap method output function, which fails to handle transformers returning DataFrames with unnamed columns. The fix adds a check for empty column names and generates default names. Outcome: Pass@1 success with minimal 4-line patch. D.2. Failure Case Analysis Failure Mode: Incomplete Localization Issue ID: matplotlib matplotlib-23964 Description: The model correctly identifies the primary bug location in the colorbar module but misses a secondary location in the figure module that requires a corresponding update. B OOSTAPR Attempt: Modified colorbar.py to fix the spacing calculation, but did not update figure.py to propagate the new parameter. Root Cause: The bug spans multiple files with implicit dependencies. Without explicit cross-file references in the issue description, the model struggles to identify all relevant locations.
17
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 12. Ground-truth-overlap rewards underperform execution-grounded rewards.
Reward PPO + RGT (BLEU with reference patch) PPO + RGT + Rline PPO + Rseq (execution-grounded) Full B OOSTAPR
SWE-V
D4J
35.6 36.8 38.3 40.7
13.8 15.1 19.2 24.8
Table 13. Impact of candidate count K on final performance.
K
pass@1
pass@4
2 4 8
38.9 40.7 40.5
42.1 44.3 44.1
Failure Mode: Semantic Misunderstanding Issue ID: sympy sympy-21379 Description: The issue describes a subtle difference between Piecewise behavior with ITE vs. direct evaluation. The model interprets this as a type coercion issue rather than a logical evaluation order problem. B OOSTAPR Attempt: Added explicit type conversion which passes syntax checks but produces incorrect numerical results on edge cases. Root Cause: Ambiguous natural language in the issue description led to misinterpretation of the expected behavior.
E. Benchmark Details E.1. SWE-bench Verified SWE-bench Verified contains 500 instances selected from the full SWE-bench dataset based on human validation. Each instance includes: • Repository snapshot at the commit immediately before the fix • Issue description from GitHub • Test files that exercise the bug • Ground truth patch Evaluation uses the official SWE-bench harness, which: 1. Applies the candidate patch to the repository 2. Runs the test suite in an isolated Docker container 3. Reports success only if all relevant tests pass 18
B OOSTAPR: Execution-Grounded RL for Automated Program Repair Table 14. Impact of allocation temperature τ .
τ 0.25 (sharp) 0.5 (default) 1.0 (smooth) 2.0 (uniform)
pass@1
pass@4
39.4 40.7 39.8 38.6
43.2 44.3 43.5 41.2
Table 15. Computational cost breakdown (A100 GPU-hours).
Stage
GPU-hours
% of Total
Demonstration generation SFT training Candidate generation Reward model training PPO training
8 6 12 8 30
12.5% 9.4% 18.8% 12.5% 46.9%
Total
64
100%
E.2. Defects4J v2.0 Defects4J v2.0 extends the original benchmark to 835 bugs across 17 Java projects. We use the official Defects4J infrastructure for patch application and test execution. Key differences from SWE-bench: • Java rather than Python • Build system integration (Maven/Ant) required • Generally larger test suites per bug • No natural language issue descriptions (only failing tests) For evaluation, we adapt our unified diff format to Java conventions and use the Defects4J defects4j test command for validation. E.3. Function-Level Benchmarks HumanEval-Java. We use the Java translation of HumanEval (Chen et al., 2021), containing 164 function-level coding problems. For repair evaluation, we introduce bugs into the canonical solutions using mutation operators (statement deletion, operator replacement, boundary changes) and task the model with fixing them. QuixBugs. The QuixBugs benchmark (Lin et al., 2017) contains 40 classic algorithmic bugs (e.g., off-by-one errors, incorrect comparisons) with known minimal fixes. Both Python and Java versions are available; we evaluate on both but report the Java results for consistency with Defects4J.
F. Broader Impact and Ethical Considerations F.1. Potential Positive Impacts Automated program repair has significant potential to benefit software development: • Reduced maintenance burden: Developers spend substantial time (estimates range from 25-50%) on debugging and maintenance. Effective APR tools could redirect this effort toward new feature development. • Improved code quality: Automated repair can catch and fix bugs earlier in the development cycle, reducing the cost and risk of defects reaching production. 19
B OOSTAPR: Execution-Grounded RL for Automated Program Repair
• Accessibility: APR tools could help less experienced developers fix complex bugs, democratizing software development expertise. F.2. Potential Negative Impacts We acknowledge several risks associated with this technology: • Skill atrophy: Over-reliance on automated tools could reduce developer debugging skills over time. • Security concerns: Automated code modification could potentially be exploited to introduce vulnerabilities if the repair system is compromised. • Job displacement: While we believe APR will augment rather than replace developers, economic impacts on software maintenance roles should be monitored. • Misplaced trust: Users may over-trust automated repairs without adequate verification, leading to deployment of incorrect fixes. F.3. Mitigations We recommend the following practices for responsible deployment: 1. Human oversight: All automated repairs should be reviewed by human developers before deployment. 2. Confidence calibration: Systems should provide calibrated confidence estimates to help users identify repairs requiring extra scrutiny. 3. Audit trails: Maintain detailed logs of automated repairs for accountability and debugging. 4. Gradual adoption: Introduce APR tools incrementally, starting with low-risk fixes and expanding as trust is established.
G. Reproducibility Checklist To facilitate reproduction of our results, we provide: ✓ Complete hyperparameter specifications (Appendix B) ✓ Training data construction details (Section 3.1) ✓ Evaluation protocol and metrics (Section 4) ✓ Computational requirements (Appendix C) ✓ Random seed specification (3 seeds for stability analysis) Code and trained models will be released upon publication. The public repository URL will be added to the camera-ready metadata once finalized.
20