From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs

[2506.13182] From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs Skip to main content Search Submit Donate Log in Computer Science > Software Engineering arXiv:2506.13182 (cs) [Submitted on 16 Jun 2025 ( v1 ), last revised 13 Oct 2026 (this version, v4)] Title: From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs Authors: Anh Ho , Thanh Le-Cong , Bach Le , Christine Rizkallah View PDF HTML (experimental) Abstract: [...] Since then, various APR approaches, especially those leveraging the power of large language models (LLMs), have been rapidly developed to fix general software bugs. Unfortunately, the effectiveness of these advanced techniques in the context of regression bugs remains largely unexplored. This gap motivates the need for an empirical study evaluating the effectiveness of modern APR techniques in fixing real-world regression bugs. In this work, we conduct an empirical study of APR techniques on regression bugs. To facilitate our study, we introduce RegressionBug4APR, a high-quality benchmark of Java and Python regression bugs integrated into a framework designed to facilitate APR research. The current benchmark includes 200 regression bugs collected from widely used real-world GitHub repositories. We begin by conducting an in-depth analysis of the benchmark, demonstrating its diversity and quality. Building on this foundation, we empirically evaluate the capabilities of APR to regression bugs by assessing both traditional APR tools and advanced LLM-based APR approaches. Our experimental results show that classical APR tools fail to repair any bugs, while LLM-based APR approaches exhibit promising potential. Motivated by these results, we investigate impact of incorporating bug-inducing change information into LLM-based APR approaches for fixing regression bugs. We further conduct an ablation study to disaggregate the contribution of each contextual element within the bug-inducing change information. Our results highlight that this context-aware enhancement significantly improves the performance of LLM-based APR, yielding 1.6x more successful repairs compared to using LLM-based APR without such context. Moreover, our findings are consistent across both Java and Python benchmarks, providing preliminary evidence for the generalizability of our findings. Comments: This paper has been peer-reviewed and accepted for publication in ACM Transactions on Software Engineering and Methodology (TOSEM) Subjects: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI) Cite as: arXiv:2506.13182 [cs.SE] (or arXiv:2506.13182v4 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2506.13182 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Anh Ho [ view email ] [v1] Mon, 16 Jun 2025 07:49:18 UTC (2,389 KB) [v2] Sat, 20 Jun 2026 16:37:07 UTC (2,735 KB) [v3] Tue, 25 Aug 2026 01:22:56 UTC (2,704 KB) [v4] Tue, 13 Oct 2026 03:32:20 UTC (2,660 KB) Full-text links: Access Paper: View PDF HTML (experimental) TeX Source view license Additional Features Audio Summary Current browse context: cs.SE < prev | next > new | recent | 2025-06 Change to browse by: cs cs.AI References & Citations SciX Google Scholar Semantic Scholar export BibTeX citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ( What is the Explorer? ) Connected Papers Toggle Connected Papers ( What is Connected Papers? ) Litmaps Toggle Litmaps ( What is Litmaps? ) scite.ai Toggle scite Smart Citations ( What are Smart Citations? ) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ( What is alphaXiv? ) Links to Code Toggle CatalyzeX Code Finder for Papers ( What is CatalyzeX? ) DagsHub Toggle DagsHub ( What is DagsHub? ) GotitPub Toggle Gotit.pub ( What is GotitPub? ) Huggingface Toggle Hugging Face ( What is Huggingface? ) ScienceCast Toggle ScienceCast ( What is ScienceCast? ) Demos Demos Replicate Toggle Replicate ( What is Replicate? ) Spaces Toggle Hugging Face Spaces ( What is Spaces? ) Spaces Toggle TXYZ.AI ( What is TXYZ.AI? ) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower ( What are Influence Flowers? ) Core recommender toggle CORE Recommender ( What is CORE? ) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs . Which authors of this paper are endorsers? | Disable MathJax ( What is MathJax? ) We gratefully acknowledge support from our major funders , member institutions , and all contributors. About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab)

Related documents

• From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs#670350
• The Missing Memory Hierarchy: Demand Paging for LLM Context Windows#2729
• Supplementary file 1_Application-driven pedagogical knowledge optimization of open-source LLMs via reinforcement learning and supervised fine-tuning.pdf#325675
• TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunications#175331
• The Invitation Trap: Proactive Availability Backdoor in LLMs via Conversational Induction#246462