Evaluating Agentic Code Repair Capabilities in Distributed Systems

[2608.14863] Evaluating Agentic Code Repair Capabilities in Distributed Systems Skip to main content Search Submit Donate Log in Search arXiv Press Enter to search · Advanced search Computer Science > Software Engineering arXiv:2608.14863 (cs) [Submitted on 14 Aug 2026] Title: Evaluating Agentic Code Repair Capabilities in Distributed Systems Authors: Yibo Yan , Huijuan Wang , Junzhou He , Yizhuo Liang , Shaoyu Wang , Huanchen Sun , Seo Jin Park View a PDF of the paper titled Evaluating Agentic Code Repair Capabilities in Distributed Systems, by Yibo Yan and 6 other authors View PDF HTML (experimental) Abstract: LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified. Distributed-system debugging, however, remains an under-explored regime: bugs span processes, nodes, and protocol interactions, with root causes rarely recoverable from source alone and brute-force exploration intractable across non-deterministic interleavings. This leaves two gaps in LLM and agent evaluation: no code-repair benchmark targets distributed-system bugs, and no controlled study isolates how much externally provided debugging context changes agent success on them. We introduce DDBench, a code-repair benchmark of 60 historical bugs mined from 13 open-source distributed systems, partitioned into three difficulty tiers. DDBench evaluates every case under two matched conditions: a symptom-only condition where the agent receives only the bug symptom and repository, and a context-augmented condition where it additionally receives a bounded debugging context (logs, traces, runtime state, and targeted code-investigation notes), isolating the effect of debugging context from model capability. The evaluation of ten LLMs on DDBench reveals several findings. First, distributed debugging exercises a reasoning dimension that single-process benchmarks do not surface: models' pass rates span 61 pp, and pairwise bootstrap separates 9 of 15 top-tier model pairs at p < 0.05 on DDBench's hardest case-set. Second, bounded debugging context lifts aggregate pass rate by +18.1 pp, and the lift is asymmetric: weaker models gain pass rate, while stronger models gain efficiency. Third, debugging context requires careful curation, as even faithful debugging context can sometimes mislead LLMs. Comments: Under submission Subjects: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) Cite as: arXiv:2608.14863 [cs.SE] (or arXiv:2608.14863v1 [cs.SE] for this version) https://doi.org/10.48550/arXiv.2608.14863 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Yibo Yan [ view email ] [v1] Fri, 14 Aug 2026 19:59:56 UTC (4,374 KB) Full-text links: Access Paper: View a PDF of the paper titled Evaluating Agentic Code Repair Capabilities in Distributed Systems, by Yibo Yan and 6 other authors View PDF HTML (experimental) TeX Source view license Current browse context: cs.SE < prev | next > new | recent | 2026-08 Change to browse by: cs cs.AI cs.DC References & Citations NASA ADS Google Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ( What is the Explorer? ) Connected Papers Toggle Connected Papers ( What is Connected Papers? ) Litmaps Toggle Litmaps ( What is Litmaps? ) scite.ai Toggle scite Smart Citations ( What are Smart Citations? ) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ( What is alphaXiv? ) Links to Code Toggle CatalyzeX Code Finder for Papers ( What is CatalyzeX? ) DagsHub Toggle DagsHub ( What is DagsHub? ) GotitPub Toggle Gotit.pub ( What is GotitPub? ) Huggingface Toggle Hugging Face ( What is Huggingface? ) ScienceCast Toggle ScienceCast ( What is ScienceCast? ) Demos Demos Replicate Toggle Replicate ( What is Replicate? ) Spaces Toggle Hugging Face Spaces ( What is Spaces? ) Spaces Toggle TXYZ.AI ( What is TXYZ.AI? ) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower ( What are Influence Flowers? ) Core recommender toggle CORE Recommender ( What is CORE? ) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs . Which authors of this paper are endorsers? | Disable MathJax ( What is MathJax? ) We gratefully acknowledge support from our major funders , member institutions , , and all contributors. About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab) Major funding support from

Related documents

• Evaluating Agentic Code Repair Capabilities in Distributed Systems#459635
• When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems#997554
• To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study#168373
• Rebuild Dossier: Mechanically-Enforced Specs for Agentic App Rebuilds, and What Model-Tier Failures Reveal#1048793
• Agentic Hardware Design as Repository-Level Code Evolution#319700