arXiv:2607.17211v1 [cs.SE] 19 Jul 2026
Prefactory: Automated Discovery and Application of Library-Adoption Refactorings Islem Bouzenia
Michael Pradel
CISPA Helmholtz Center for Information Security Germany
CISPA Helmholtz Center for Information Security Germany
Abstract—Replacing hand-written code with library API calls is a common refactoring that can reduce code size, make code more idiomatic, and reuse well-tested implementations. Yet many library-adoption opportunities are hard to find automatically: the original code often does not mention the target library and may resemble the library API only in behavior, with little syntactic overlap. Existing tools, such as linters and static modernizers, cover only a small set of manually specified patterns. LLMs and LLM-based agents, on the other hand, can generalize to more patterns, but they are costly, difficult to reproduce and to apply systematically at scale. This paper introduces Prefactory, an automated approach for library-adoption refactoring in Python. The key idea is to use an LLM to synthesize executable search heuristics rather than relying on repeated LLM prompting over a codebase. Given a target project and a target library name, Prefactory collects library metadata and project vocabulary, then generates lexical and structural detectors. Prefactory executes the detectors during a scan phase to find candidate functions. It then heuristically ranks the candidate functions, generates refactorings for the highest-ranked ones using an LLM, and validates the results with project tests and newly generated differential tests. We evaluate Prefactory on PrefactoryBench, a benchmark of 100 real-world library-adoption refactorings from 61 opensource Python projects and 18 libraries. Prefactory detects 75 instances at the file level and 56 at the function level, compared with 35 and 32 for the strongest baseline (Codex CLI). From the 56 detected functions, Prefactory produces 40 test-validated refactorings. Generated detectors make scanning inexpensive: scanning a project takes 1.3 seconds on average, requires no LLM calls, and keeps end-to-end LLM cost below $0.05 per instance. These results show that detector synthesis can turn LLM semantic knowledge into a practical, scalable workflow for library-adoption refactoring.
I. I NTRODUCTION Library-adoption refactoring replaces hand-written implementations with calls to existing library APIs, preserving program behavior. This refactoring is common in Python projects, where developers often reimplement functionality already provided by libraries such as numpy, pandas, scipy, or sklearn. Replacing such code with library calls can reduce code size, make implementations more idiomatic, and reuse code maintained and tested by library developers. Figure 1 shows a representative example from the mage_ai project. The original implementation manually builds histogram buckets by iterating over a series, computing a bucket index, and updating the selected bucket. The developer’s refactoring replaces the hand-written histogram computation
BEFORE (mage_ai/.../statistics.py, source commit) import math def build_histogram_data(col1, series, column_type): ... if bucket_interval == 0: return for value in series.values: index = math.floor((value - min_value) / bucket_interval) if index >= len(buckets): index = len(buckets) - 1 buckets[index][’values’].append(value) ... AFTER (same file, refactoring commit) import math import numpy as np def build_histogram_data(col1, series, column_type): max_value = series.max() min_value = series.min() buckets, bucket_interval = build_buckets( min_value, max_value, BUCKETS, column_type) if bucket_interval == 0: return counts, edges = np.histogram( series.values, bins=BUCKETS, range=(min_value, max_value)) ...
Fig. 1. Motivating library-adoption refactoring from mage_ai.
with np.histogram. Although the refactoring changes only a specific region of code, automatically finding such opportunities in a large project is non-trivial. The original code in this example does not mention the target library near the refactored region. Its relation to np.histogram is visible only through implementation-level signals: a loop over values, a bucket-index computation, updates to histogram buckets, and the domain-specific term “histogram” in the name of the surrounding function. Existing tools provide limited support for this kind of refactoring. Linters and static modernizers, such as ruff [1], refurb [2], and pylint [3], are effective when a refactoring opportunity can be encoded as a precise rule, but cover only patterns defined in advance. Prior work on custom-to-API replacement [4], including RETIWA [4] and AKIRA [5], also replaces hand-written implementations with calls to existing APIs. However, these approaches typically assume predefined transformation patterns or API mappings and have primarily targeted Java. In contrast, library-adoption refactoring in Python requires detecting opportunities for a target library even when the code does not mention the library, the relevant API, or a fixed syntactic pattern. LLMs and LLM agents are more flexible and can recognize more varied implementations,
but using them directly to explore entire projects is costly, difficult to control, and hard to reproduce. These limitations make detection the first bottleneck for library-adoption refactoring. An approach to this problem must address two detection challenges. First, it must localize relevant code without relying on explicit library mentions: the hand-written implementation may not import the target library or name the relevant API. Second, it must generalize beyond fixed syntactic templates: libraries span many domains and expose APIs for many kinds of behavior, and developers can implement the same functionality in different local forms. Once a candidate is found, a complementary task remains: the approach must generate a library-based replacement and reject edits that change the behavior of the original implementation. The histogram example in Figure 1 illustrates the two detection challenges. The original code does not call np.histogram, but it contains lexical cues, such as domain terms including histogram, bucket, and bin. It also contains structural cues, such as a loop over values, arithmetic used to compute a bucket index, and accumulation into a data structure. These cues are not tied to a single syntax: another developer could implement the same histogram computation using different variable names, control flow, or helper functions. At the same time, they recur across implementations of the same underlying behavior, which makes them useful as reusable detection cues. Prefactory uses an LLM to synthesize detector families that capture such lexical and structural cues while tolerating variation in local implementation details. To address these challenges, this paper introduces Prefactory, an automated approach for library-adoption refactoring in Python. The key insight of our approach is to use an LLM to synthesize reusable executable detectors rather than repeatedly prompt an LLM to inspect a codebase. These detectors turn the LLM’s semantic knowledge about library functionality into deterministic search mechanisms that can be applied efficiently across a project. Prefactory is organized as a four-stage pipeline. First, in the detector-generation stage, Prefactory collects target-library metadata and project vocabulary, then uses an LLM to generate two detector families: lexical detectors, implemented as regular expressions, and structural detectors, implemented as AST matchers. Second, in the project-scanning stage, Prefactory executes the generated detectors over the target project without LLM calls, producing candidate files and functions that may contain library-adoption opportunities. Third, in the candidateranking and filtering stage, Prefactory prioritizes candidates using signals such as detector-match strength, matched-code size, and the presence of basic Python implementation patterns, such as loops, branches, arithmetic, and updates to data structures. These patterns help identify hand-written code that may implement functionality available through higherlevel library APIs. This stage limits LLM-based rewriting to a small set of localized functions. Fourth, in the refactoringgeneration and validation stage, Prefactory invokes an LLM to generate a library-based edit for the highest-ranked functions, then validates the edit with LLM-generated differential tests
followed by project tests. The differential tests compare the original and refactored implementations on generated inputs, using the original implementation as the oracle for observed behavior. To evaluate our technique, we created PrefactoryBench, a benchmark of 100 real-world library-adoption refactorings mined from 61 open-source Python projects and spanning 18 libraries, including both third-party libraries, such as numpy and pandas, and standard-library targets, such as collections and itertools. Each benchmark instance includes a reference execution environment and a test setup that pairs the project tests (the project’s own test suite) with LLM-generated held-out tests targeting the refactored region. During its final stage, Prefactory runs only the project tests; held-out tests are never seen by Prefactory and exist solely for benchmarking. Both are separate from the differential tests that Prefactory generates internally during its final stage. On this benchmark, Prefactory detects 75 instances at the file level and 56 at the function level, outperforming linterbased tools, vanilla LLM prompting, Semgrep+LLM, and Codex CLI. The strongest baseline, Codex CLI, detects 35 instances at the file level and 32 at the function level while using the same underlying model. Prefactory also produces 40 validated refactorings from the 56 detected functions sent to generation. The approach scans projects in 1.3 seconds on average and costs less than $0.05 per instance to detect and refactor; Codex CLI costs roughly three times more. In an open-source study on the latest versions of five benchmark projects, Prefactory surfaces multiple opportunities per repository. Two of five submitted pull requests have already been accepted by maintainers, providing initial evidence that the pipeline can surface useful refactorings beyond historical benchmark commits. In summary, this paper makes the following contributions: 1) PrefactoryBench: a benchmark of 100 real-world Python library-adoption refactorings across 18 libraries and 61 projects, with reference execution environments, project tests, and held-out tests for evaluating both detection and transformation. 2) Prefactory: an automated approach based on the idea that LLMs can synthesize reusable executable detectors for library functionality. Starting from a target library name, Prefactory collects library metadata and project vocabulary, synthesizes lexical and structural detectors, and scans projects without LLM calls during detection. 3) Evidence that detector synthesis outperforms direct prompting and agentic search: a controlled comparison against linter-based tools, vanilla LLM prompting, Semgrep+LLM, and Codex CLI on the 100-instance benchmark, with an end-to-end evaluation and an open-source study on current project versions. II. A PPROACH Figure 2 shows Prefactory as a four-stage narrowing pipeline. Stage 1, detector generation, uses an LLM to synthesize lexical and structural detectors and discards detectors
LLM
Target project + target library
No LLM
No LLM
1
Detector Generation
2
Project Scanning
3
Ranking & Filtering
1.1
Generate lexical detectors (regex)
2.1
Filter non-source files
3.1
1.2
Generate structural detectors (AST)
2.2
Run detectors on project
Prioritize highmatch, pure Python code
3.2
Keep top files
1.3
Check syntax and exclude noisy detectors
Validated Detectors
Record matched 2.3 files, functions & code lines Detected Candidates
Keep top 3.3 functions Top Candidates
LLM 4
Refactoring/Validation
4.1
Refactor candidate func.
4.2
Run LLM tests
4.3
Run project tests
Validated refactoring
Accepted Edits
Fig. 2. Overview of the Prefactory four-stage pipeline. Stages 1 and 4 invoke an LLM; Stages 2 and 3 are deterministic and require no LLM calls.
that are malformed, fail basic execution checks, or produce overly broad matches. Stage 2, project scanning, executes the remaining detectors over the target project, without relying on LLM calls, to find candidate code regions. Stage 3, candidate ranking and filtering, ranks the detected files and functions and keeps only a small set of high-ranked candidates. Stage 4, refactoring generation and validation, uses an LLM to rewrite the selected functions and validates the resulting edits with differential tests and project tests. This design lets Prefactory search broadly with deterministic detectors before spending LLM refactoring effort on a much smaller set of localized functions. A. Stage 1: Detector Generation A detector is an executable search program that looks for hand-written code that may be replaceable by a call to a library API. Prefactory generates two detector families. Lexical detectors are regular expressions that match source-code text, such as API-related names, domain terms, local identifiers, and textual patterns associated with hand-written implementations. Structural detectors are AST matchers that match code structure, such as loops, branches, arithmetic expressions, indexed updates, accumulator variables, and exception-handling patterns. Prefactory uses separate LLM prompts to generate the two detector families. Both prompts include target-library context: package metadata from PyPI for third-party libraries, importable submodules, exposed functions and classes, usage examples, and mined examples of prior library-adoption refactorings. We obtain usage examples from the target library’s documentation by asking a cost-efficient LLM (gpt-5-nano) to extract examples from the package-level documentation. The mined refactoring examples are used as detector-generation hints. We collect them by searching GitHub commit histories for commits that introduce an import of the target library using its known import aliases, e.g., import numpy as np, and remove hand-written implementations. These examples are drawn from projects that do not overlap with the benchmark projects. The full detector-generation prompts are included in the supplementary material.
For lexical detectors, Prefactory additionally provides project vocabulary to the generation prompt. It extracts variable names, function names, class names, and imported names from all Python files in the target project, including tests, selects the 100 most frequent names, and includes up to three sampled source lines per name. This project context helps the LLM produce regular expressions that reflect local names, domain terms, and naming conventions. Structural-detector generation does not use project vocabulary, because AST matchers are intended to capture algorithmic and functionality shapes rather than project-specific names. Figure 3 shows detector examples for the motivating np.histogram refactoring. The lexical detector searches for histogram-related terms, index computations, and bucket updates. The structural detector abstracts away from exact variable names and instead matches the computation shape: iterating over values, deriving a bin index, optionally adjusting bounds, and updating an indexed container. Before scanning, Prefactory removes detectors that are malformed, crash during execution, or produce overly broad matches. Lexical detectors must compile and must not match more than 20% of project files for that project. Structural detectors are generated together with positive and negative Python examples; Prefactory parses these examples, runs the generated matcher on their ASTs, and keeps the detector only if it matches all positive examples and none of the negative examples. B. Stage 2: Project Scanning Project scanning runs the Stage 1 detectors over the target project at a specific commit. Before scanning, Prefactory removes files that are unlikely to contain relevant source code. The approach identifies these files using path, filename, and extension heuristics, e.g., vendored-code directories such as _vendor or _vendoring, documentation directories and non-source extensions, virtual-environment paths, generated-file locations, common test directories, and test_*.py naming patterns. Test files are retained only when the target library is testrelated, which Prefactory determines from PyPI metadata and a fixed list of test-related libraries.
Lexical detector (regular expression) target numpy.histogram name histogram_bucket_terms pattern (histogram|bucket|bin).{0,120} (floor|int|//|round).{0,120} (append|\+=|count) # The .{0,120} terms allow up to 120 # characters between related cues. Structural detector (Python AST matcher, simplified) def match(stmts, i, params): loop = stmts[i] if not isinstance(loop, ast.For): return None if _for_iter_attr(loop.iter) is None: # for v in X. values return None bin_idx = bucket_update = False for s in loop.body: bin_idx |= _assigns_floor_div(s) # idx = floor(...) bucket_update |= _subscript_update(s) # buckets[idx] += v if bin_idx and bucket_update: return Match("numpy", "np.histogram", span=(loop.lineno, loop.end_lineno)) return None
Fig. 3. Detector examples. Top: a lexical detector for np.histogram; Bottom: a simplified AST matcher for np.histogram; each shipped matcher is a per-pattern Python match() function that walks AST nodes and returns the matched span and target API.
For each remaining file, Prefactory runs the lexical and structural detectors produced in Stage 1. When a detector matches, Prefactory records the matched lines, the detector that matched them, the enclosing function, and the containing file. A candidate region may be detected by lexical detectors, structural detectors, or both. The matched lines and matching detectors are retained to guide ranking and to give the generator context on why the candidate was flagged. In the running example, scanning the mage_ai project with numpy as the target library identifies the file containing build_histogram_data. The generated detectors match histogram-related terms and bucketization-like code structure. The resulting candidate region, together with the detectors that matched it, is passed to the ranking stage. C. Stage 3: Candidate Ranking and Filtering Scanning may surface many candidates, not all of them equally strong. Prefactory therefore ranks them and keeps only the top-ranked candidates before generation. For each candidate file or function c, Prefactory computes a ranking tuple (D(c), B(c), −S(c)). The first component, D(c), is detector-match strength: D(c) = M (c) + U (c) · P (c), where M (c) is the weighted number of detector matches, U (c) is the weighted number of unique matching detectors, and P (c) is the fraction of source lines in c matched by at least one detector. Structural detector matches receive weight 2 and lexical detector matches receive weight 1. We use this fixed 2:1 weighting because structural detectors encode control-flow and data-flow shapes and are therefore more specific than lexical cues; the weights are fixed across all experiments. Including U (c) in the score reduces the effect of size: a large candidate
with many matches from a single detector is weaker evidence than one matched by several independent detectors. The second component, B(c), measures the density of basic Python implementation patterns that often appear in handwritten implementations of higher-level library functionality. These patterns include pure-Python loops, arithmetic operations, branches, updates to simple data structures, and imports from modules that are lower-level with respect to the target library. For a third-party target library, Prefactory treats the library’s declared dependencies and Python standard-library modules as lower-level imports with respect to that library. For example, when ranking candidates for numpy, code that combines arithmetic-heavy loops with imports from standardlibrary modules such as math receives higher priority. The final component, S(c), is candidate size, measured as the number of source lines in the file or function. Using −S(c) in the tuple breaks ties by preferring shorter candidates. Given the ranking tuple, Prefactory compares candidates lexicographically: it first compares detector-match strength D(c), then the basic-implementation score B(c), and finally candidate size S(c) as a tie breaker. The approach applies the same ranking rule at two levels: it first keeps the top five files, then ranks functions within those files and keeps the top ten functions overall. This fixed budget bounds LLM generation cost in Stage 4. We study the sensitivity of this budget in Section III. D. Stage 4: Refactoring Generation and Validation The refactoring-generation and validation stage takes the top-ranked candidates and attempts to rewrite each toward using the target library. Each attempt receives the candidate function, 100 lines of surrounding file context, target-library API hints, and the detector information for the candidate. The API hints come from the same target-library context used in Stage 1, including package metadata, importable submodules, exposed functions and classes, and usage examples. The detector information includes the detector names, target API families, detector patterns, and exact code lines that matched. The result is either a validated refactoring or a rejected candidate. First, Prefactory prompts an LLM to rewrite the candidate function using APIs from the target library. The prompt contains three kinds of information: i) the candidate function and local file context, consisting of surrounding code before and after the function; ii) library hints describing the target library and relevant APIs; and iii) the detectors that matched the candidate function and the exact code lines they matched. If the target library exposes at most 100 importable functions and classes, Prefactory passes the full list to the refactoring prompt. If it exposes more than 100 APIs, Prefactory first provides the full API list to an LLM and asks it to select up to 100 APIs that are likely relevant to the candidate function, then passes only that selected list to the refactoring prompt. The LLM is then asked to return either a concrete refactoring of the candidate function, together with any necessary imports, or to report that no suitable refactoring is possible.
After the LLM suggests a rewrite, Prefactory applies a relevance filter. The filter is AST-based: it rejects edits that only add an import without changing the implementation, and edits that do not call the target library in the rewritten function. Candidates that pass this filter are then checked with differential tests and project tests. To compare the original and rewritten implementations, Prefactory uses a second LLM call to generate a standalone differential test script [6]. The script places the original function and the rewritten function side by side, includes required dependencies available from the file context when possible, mocks remaining dependencies when needed, generates test inputs, and compares the outputs using the original function as the behavioral oracle, a well-established choice under testoracle scarcity [7]. The differential test passes when none of the generated inputs reveal a behavioral difference, fails when at least one produces diverging outputs, and is inconclusive when the script cannot execute reliably, for example because of dependency or mocking failures. The prompt asks the LLM to generate more than ten inputs, including edge cases. A passing differential test is therefore evidence of behavioral preservation on the generated inputs, not a proof of equivalence. If the differential test passes, Prefactory runs project tests on the refactored version and accepts the refactoring if no regressions are found. If the differential test fails, Prefactory feeds the failing inputs and observed outputs back to the refactoring prompt and requests a corrected edit, following the feedback pattern of conversational program repair [8]. If the differential test is inconclusive because the generated test script errors during setup, mocking, or execution, Prefactory asks the LLM to repair the test script instead of changing the refactoring edit. Prefactory allows up to three additional generation attempts. Candidates that still fail relevance filtering, differential testing, or project tests are rejected. In the running example, this stage rewrites the detected build_histogram_data function using np.histogram. Prefactory validates the rewritten function against the original with differential tests, then accepts the edit after the project tests pass without new regressions. III. E VALUATION In evaluating Prefactory, we aim to answer five questions: RQ1: End-to-End Effectiveness. How often does Prefactory detect real-world library-adoption opportunities and produce validated refactorings, and where does it fail? RQ2: Comparison to Baselines. How does Prefactory compare with the baselines on detection and refactoring? RQ3: Efficiency. What are the runtime, LLM-call, and monetary costs of Prefactory? RQ4: Ablation Study and Design Contributions. How does Prefactory’s design affect detection and refactoring? RQ5: Open-Source Usefulness. Can Prefactory surface actionable refactorings on current project versions? We describe the benchmark, baselines, metrics, and model configuration in turn.
TABLE I C OMPOSITION OF P REFACTORY B ENCH .
Property
Value
Library-adoption refactorings Open-source projects Target libraries Median refactored-region size
100 61 18 30 LOC
Project-test coverage of refactored region (avg. / med.) Benchmark-suite coverage after held-out tests (avg.) Instances with 100% benchmark-suite coverage
3% / 4% 94% 28
A. Experimental Setup 1) Benchmark: Existing refactoring benchmarks do not directly support our task end to end. Some focus on specific classes of Python refactorings, such as replacing nonidiomatic constructs with predefined Pythonic idioms [9]. Recent repository-level benchmarks such as SWE-Refactor [10] provide high-quality real-world refactoring instances, but target general refactoring scenarios in Java rather than Python library-adoption refactorings. Our setting instead requires instances where a hand-written Python implementation is replaced by a library API call, together with project context to detect the opportunity, generate a replacement, and validate the resulting edit. To fill this gap, we construct PrefactoryBench, a benchmark of 100 real-world library-adoption refactorings mined from 61 open-source Python projects and spanning 18 libraries. The benchmark is designed to facilitate a reliable end-to-end evaluation of refactoring techniques such as Prefactory: each instance is manually checked, tied to the developer’s original refactoring, paired with a reference execution environment, and accompanied by tests that exercise the refactored region. Each instance consists of a parent commit, a developer refactoring commit, the reference file and changed region modified by the developer, and a reference execution environment. The benchmark covers both third-party libraries, such as numpy and pandas, and standard-library targets, such as collections and itertools. We mine instances from open-source Python repositories on GitHub. We select third-party target libraries by starting from the top 200 PyPI packages by download count, reranking them by in-the-wild import frequency measured through GitHub code search, and keeping libraries with well-documented idioms that replace recognizable hand-written implementations. We supplement these libraries with five widely used standardlibrary modules. For each target library, we collect active Python repositories using library-specific import queries, such as import numpy as np or calls through common aliases such as np.*. We restrict the search to projects with at least 100 stars, scan commits from 2016 to 2026 using PyDriller [11], cap each repository at its 500 most recent commits, and skip merge commits. A diff hunk becomes a candidate when the post-commit file imports the target library and the added lines contain a call to that library API, which we validate using AST-based checks.
We deliberately avoid requiring the deleted lines to match any predefined hand-written pattern, so that the benchmark does not favor specific refactoring shapes. We then apply an LLM filter to remove cosmetic substitutions, deprecation swaps between library versions, and changes bundled with broader rewrites that alter functionality or behavior beyond the refactoring. Each remaining instance is manually audited to confirm that it represents a genuine library-adoption refactoring and that the reference file and changed region correspond to the developer’s edit. Finally, we ensure that each benchmark instance is paired with a working test setup. We combine the project tests (i.e., the project’s own test suite) with LLM-generated held-out tests that target the refactored region and increase line coverage; together they form the benchmark test suite. Held-out tests use the original implementation as the behavioral oracle and pass on both the original and developer-refactored code. During Stage 4 validation, Prefactory runs only the project tests; heldout tests are never seen by Prefactory and exist solely to raise test coverage of the target region for benchmark quality assessment. Across the benchmark, the project tests alone cover the refactored region with an average of 3% and a median of 4% line coverage, which is insufficient for reliable behavioral validation. Held-out tests targeting the refactored region raise the average benchmark-suite coverage to 94%, and 28 instances reach 100% coverage under the full suite. 2) Baselines: We compare Prefactory against four baseline families. a) Linters and static modernizers: We run three Python refactoring and modernization tools: ruff, refurb, and pylint. These tools represent manually authored rule-based detection. We take the union of locations reported by the three tools and calculate file-level and function-level detection rates (see definitions below). This baseline is expected to be precise when a supported rule matches, but it cannot detect opportunities that require rules not implemented by the tools. Linters are evaluated only for detection because they do not provide the same end-to-end refactoring generation workflow as Prefactory. b) Vanilla LLM: This baseline tests whether a general LLM can localize and refactor library-adoption opportunities without Prefactory’s generated detectors. The model receives the project file tree and the target library name and is asked to select up to 10 files likely to contain refactoring opportunities, using filenames and paths only. This file budget is larger than Prefactory’s top-five cutoff because the model has no codelevel signal at this step. The model then receives function signatures from the selected files and selects up to 10 candidate functions in total. The code of those functions is then passed again to the LLM to refactor them. This baseline avoids repository-wide code ingestion to keep cost comparable. We validate its suggested refactorings using the same validation setup as for our approach. c) Semgrep+LLM: This baseline uses an LLM to generate rules for the Semgrep [12] pattern-matching engine, validates the generated rules, and applies them to the project.
We evaluate it for detection only. It tests whether a standard pattern-matching engine combined with LLM-written rules can substitute for Prefactory’s detector generation and ranking. d) Codex CLI agent: We use Codex CLI as an agentic code-editing system to explore a codebase and suggest up to 10 refactorings. We give it the same input as Prefactory and we ask it to find and refactor candidate opportunities. Codex CLI uses the same underlying model as Prefactory, allowing us to compare a purpose-built pipeline against an agent using comparable model capability on the same benchmark. 3) Metrics: a) Detection metrics: We report detection at two granularities under each approach’s fixed reporting budget. A file-level detection is correct when the file containing the developer’s refactoring appears in the approach’s reported candidate files. A function-level detection is correct when the function containing the developer-refactored region appears in the approach’s reported candidate functions. For Prefactory, this means the gold file must appear among the top five ranked files, and the gold function must appear among the top ten ranked functions selected from those files. For baselines, we apply the corresponding reporting budget described in Section III-A2. Linters and Semgrep+LLM are scored as a boolean per file with no location cap, so firing once or fifty times counts equally; this gives them the most forgiving possible file-level scoring. Additionally, for linters and Semgrep+LLM, function-level detection is scored by mapping each matched line to its enclosing function with no function-count cap applied. Function-level detection is stricter than file-level detection, but it is more useful for refactoring generation. Passing an entire file to a generator increases LLM cost, makes it harder to create focused behavioral tests, and increases the risk that the model changes irrelevant code. Function-level localization therefore measures whether a tool can narrow the search to a concrete region that is practical to refactor. b) Proposed Refactoring and Pass Diff Test: We count an instance as a proposed refactoring (Refac.) when the approach detects the correct function and generates a non-trivial edit that passes the relevance filter. We count it as Pass Diff Test (PDT) when that edit additionally passes the differential tests with up to three repair attempts for Prefactory. c) Validated Refactoring: We count an instance as a validated refactoring if the approach detects the correct function, surfaces it among the top ten candidate functions, and produces a non-trivial refactoring that passes Stage 4 validation, meaning it passes the differential test and the project tests. A nontrivial refactoring must replace hand-written logic with a call to the target library rather than only adding an import or making cosmetic changes. The generated edit must overlap with the developer-changed lines recorded in the benchmark; an edit that refactors a completely separate region is not counted as a validated refactoring, even if the edit is otherwise useful. The validated refactoring must also pass on the held-out tests of the benchmark.
TABLE II RQ1 END - TO - END EFFECTIVENESS OF P REFACTORY . Inst.= INSTANCES ; File Det.= FILES DETECTED ; Func. Det.= FUNCTIONS DETECTED ; Refac.= PROPOSED REFACTORING ; PDT = PASSES DIFFERENTIAL TEST ( AFTER UP TO THREE REPAIRS ); Val.= VALIDATED ( PASSES PDT AND BENCHMARK TESTS ). Library click collections dataclasses fastapi functools git httpx numpy pandas psutil pydantic pytest pytorch re requests scipy shutil yaml Total
TABLE III FAILURE MODES AMONG THE 60 INSTANCES THAT DO NOT REACH A VALIDATED REFACTORING . Failure mode
Pass Diff Test Val.
Inst.
File Det.
Func. Det.
Refac.
6 4 7 3 6 7 6 9 5 3 4 4 4 4 9 7 7 5
6 2 6 2 4 7 5 4 1 2 2 2 3 3 8 7 6 5
6 2 5 2 3 5 5 1 0 1 2 2 3 3 5 7 1 3
5 2 5 2 3 5 5 1 0 0 1 1 2 3 3 7 1 3
5 2 5 2 3 5 5 1 0 0 1 1 2 3 3 7 1 3
5 2 5 2 3 1 0 1 0 0 1 1 2 3 3 7 1 3
100
75
56
49
49
40
4) LLM: Prefactory uses the same model, gpt-5-mini, for detector generation, refactoring generation, and differential-test generation. We run Codex CLI version v0.139.0 with gpt-5-mini, default temperature, and a single run per benchmark instance. B. Results 1) RQ1: End-to-End Effectiveness: We first evaluate whether Prefactory can complete the full library-adoption refactoring task end to end: detect a real opportunity, localize it to a concrete function, generate a library-based rewrite, and validate that the rewrite preserves behavior. Table II summarizes the results with a per-library breakdown. Starting from 100 benchmark instances, Prefactory detects the target file in 75 instances and surfaces the correct function among the top ten candidate functions for generation in 56 instances. Of these 56 functions, Prefactory proposes a candidate refactoring for 49; in the seven remaining instances the generator concludes that no suitable target-library refactoring is available. All 49 proposed refactorings pass the differential test (with up to three repair attempts). Nine subsequently fail on held-out tests, leaving 40 validated refactorings. Table II also shows that effectiveness is spread across many target libraries. Prefactory produces validated refactorings for 15 of the 18 libraries in the benchmark and surfaces the reference function in 17 of 18 libraries. The strongest results occur for libraries where the benchmark refactorings expose both API-related vocabulary and recognizable implementation structure, such as scipy, dataclasses, click, requests, yaml, and re. For example, Prefactory validates all seven scipy refactorings, all five dataclasses refactorings that reach generation, and five of six click instances.
No detector match Not in Top-10 No refactoring Validation failure
Count Interpretation 9 No generated detector matched the target 35 Target outside file or function budget 7 LLM does not suggest a refactoring 9 Generated refactoring fails benchmark tests
The weaker cases reveal where the pipeline loses candidates. For numpy and pandas, Prefactory reaches only four of nine numpy files and one of five pandas files, surfacing only one correct function for numpy and none for pandas. Manual inspection suggests that these libraries expose broad and heterogeneous API surfaces: opportunities may involve array construction, aggregation, indexing, reshaping, missing-value handling, type conversion, or numerical kernels, each with different vocabulary and implementation structure. Treating such libraries as single targets may therefore be too coarse; a more specialized pipeline could generate detectors for narrower API families or submodules of a target library. Other libraries fail mainly at ranking rather than initial detection. For example, shutil has six file-level detections but only one functionlevel detection, indicating that Prefactory often reaches the right file but does not rank the target function highly enough. To understand the 60 instances that do not reach a validated refactoring, we assign each one to the first pipeline stage it is lost. Table III summarizes these mutually exclusive categories. No detector match means no generated detector matched the target. Not in Top-10 means that at least one detector matched the project, but the reference function was not selected among the ten functions sent to generation; this includes both file-budget losses (target file not in the top five) and function-budget losses (target function not ranked high enough within the selected files). No refactoring and Validation failure cover instances where the reference function was surfaced but generation or validation did not succeed. Detection misses arise when no generated detector matches the target file or function. These cases are most visible for large, heterogeneous libraries such as numpy and pandas, where opportunities span many API families and implementation styles. Figure 4a shows a pandas miss from sktime: the code performs generic regular-expression parsing and tuple unpacking, while the API used by the developer, pandas.tseries.frequencies.to_offset, is a specialized frequency-parsing utility. The generated detectors therefore did not capture this specific part of the pandas API. This suggests that, for large libraries, detector generation may need to target narrower API families or submodules, for example, generating detectors specifically for pandas.tseries rather than for pandas as a whole. A second source of failures is ranking quality. In these cases, Prefactory may detect the relevant file or even match part of the reference function, but the function is not surfaced within the top candidates. This can happen when the target region receives only weak detector matches. It can also
BEFORE (sktime/.../frequencies.py) def _get_intervals_count_and_unit(freq: str): if freq is None: raise ValueError("frequency is missing") m = re.match(r"(?P<count>\d*)(?P<unit>[a-zA-Z]+)$", freq) if not m: raise ValueError(f"pandas frequency {freq} not understood.") count, unit = m.groups() count = 1 if not count else int(count) return count, unit
TABLE IV C OMPARISON AGAINST BASELINES ON THE 100- INSTANCE BENCHMARK .
AFTER (intended refactoring; not detected by Prefactory) def _get_intervals_count_and_unit(freq: str): if freq is None: raise ValueError("frequency is missing") offset = pandas.tseries.frequencies.to_offset(freq) return offset.n, offset.name (a) Detection miss: sktime opportunity for a rare pandas API.
highest detection effectiveness, detecting 75 instances at the file level and 56 at the function level. The strongest baseline is Codex CLI, which detects 35 instances at the file level and 32 at the function level. The vanilla LLM baseline detects 32 files but only 7 functions, showing that direct prompting can identify some relevant files but struggles to narrow the search to the correct function. Semgrep+LLM reaches 30 filelevel and 23 function-level detection, while linter-based tools detect only 11 and 8, respectively. Rule-based tools are precise for the patterns they encode but cannot generalize to new ones. Semgrep+LLM attempts to close this gap by generating rules with an LLM, but LLMwritten rules remain syntactically rigid and cannot generalize to the implementation variations present in real-world handwritten code; none of its 23 function-level detections translate to validated refactorings because the approach has no refactoring phase. The vanilla LLM baseline reveals a different bottleneck: file selection works because choosing likely files requires only coarse judgment over a project structure, but function localization collapses without a code-level search signal. 3) RQ3: Efficiency and Cost: To evaluate efficiency, we report scan time, LLM calls, and monetary cost. Scan time is the CPU cost of the scan stage alone; it differs from end-to-end wall-clock time, which is dominated by LLM-call latency across all four stages. All experiments are on a 20-core virtual machine (31 GiB RAM, Linux x86-64). For Prefactory, we distinguish detector-generation cost from per-scan cost, because generated detectors can be reused across project scans. We break down costs across three scopes: detector generation is charged once per project-library pair; scanning and ranking are charged per project scan and require no LLM calls; refactoring generation is charged per selected candidate function. Unless otherwise stated, the end-to-end per-instance cost includes detector generation, scanning, ranking, refactoring generation, differential-test generation, and any repair attempts for that benchmark instance. Detector generation costs $0.03 per project-library pair on average and takes about 80 seconds. Using those detectors, scanning a target project takes 1.3 seconds on average, with no LLM calls, while ranking and filtering complete in less than a second. As a result, after detector generation, Prefactory can rescan a project in only a few seconds without spending additional LLM budget. Refactoring generation uses LLM calls only for selected candidate functions, keeping this stage inexpensive: $0.01 per candidate function and 2.5 LLM calls per candidate function on average. The strongest baseline, Codex CLI, spends about $0.06 per
BEFORE (Megatron-LM/.../clip_grad.py) def _clip_grad_norm(parameters, max_norm, norm_type=2): is_not_tp_duplicate = param.tensor_model_parallel or \ (mpu.get_tensor_model_parallel_rank() == 0) ... total_norm_cuda = torch.cuda.FloatTensor([float( total_norm)]) torch.distributed.all_reduce( total_norm_cuda, op=torch.distributed.ReduceOp.SUM, group=mpu.get_model_parallel_group()) total_norm = total_norm_cuda[0].item() ... AFTER (attempted refactoring; validation rejected) def _clip_grad_norm(parameters, max_norm, norm_type=2): return torch.nn.utils.clip_grad_norm_( parameters, max_norm, norm_type=norm_type) (b) Refactoring failure: Megatron-LM behavior exceeds the PyTorch API.
Fig. 4. Representative failure examples.
happen when other functions in the same file are plausible library-adoption opportunities and rank above the benchmark target. In a manual inspection of 10 higher-ranked nonreference candidates sampled from ranking failures, six were genuine library-adoption opportunities, just not the developerrefactored instances recorded in the benchmark. The remaining failures happen during generation and validation. In seven cases, the model reports that no suitable library-based refactoring is available, often because the prompt does not identify the specific API needed for the rewrite. In another nine cases, the model generates a rewrite that passes the differential test but fails on project tests. Figure 4b shows a PyTorch example from Megatron-LM. The candidate function resembles torch.nn.utils.clip_grad_norm_, but the hand-written implementation also includes tensormodel-parallel filtering and distributed all-reduce behavior. A naive replacement with the library call would remove this project-specific behavior, so Prefactory rejects the generated edit instead of accepting a behavior-changing rewrite. The differential tests, which run on generated inputs, do not expose the parallel and distributed paths; the project tests do, causing the refactoring to be rejected. 2) RQ2: Comparison to Baselines: The primary comparison is candidate detection, because every baseline can report files or functions that may contain a library-adoption opportunity. For approaches that produce edits, we evaluate suggested refactorings using the same validation as for Prefactory. Table IV summarizes the results. Prefactory achieves the
Approach Linters Vanilla LLM Semgrep+LLM Codex CLI Prefactory
File Det.
Func. Det.
Val.
11 32 30 35 75
8 7 23 32 56
– 4 – 22 40
Detector type Lexical only Structural only Lexical + structural
File Det.
Func. Det.
47 48 75
31 30 56
TABLE VI E FFECT OF RANKING BUDGET ON FUNCTION - LEVEL DETECTION . Ranking budget Top 1 file, top 10 func. Top 3 files, top 10 func. Top 5 files, top 10 func. Top 3 files, top 20 func. Top 5 files, top 20 func.
Func. Det.
Gen. cost
43 54 56 56 58
1× 1× 1× 2× 2×
instance while detecting fewer correct instances. It also pays this search cost every time it is run: reapplying Codex CLI to the same project requires a new agentic search, which can produce different candidate sets across runs making results harder to reproduce and coverage of a project less predictable. In contrast, Prefactory turns LLM output into reusable detectors, so repeated scans are cheap and deterministic. 4) RQ4: Ablation Study and Design Contributions: We study which parts of Prefactory contribute most to detection and end-to-end refactoring. The ablations follow the main design choices in the pipeline: the detector type used in Stage 1, the ranking budget used in Stage 3, and the repair loop used during validation. a) Detector type: We first ablate the two detector types generated in Stage 1: lexical detectors (regex-based detection) and structural detectors (AST-based matching). Table V reports the results. Lexical detectors alone find opportunities in 47 files and 31 functions. Structural detectors alone cover a similar number of files, 48, but slightly fewer functions, 30. Combining the two detector types substantially improves detection, reaching 75 file-level and 56 function-level detections. This shows that the two detector types are complementary: each family alone detects roughly 30 functions, while together they detect 56, indicating that lexical and structural cover different opportunities. b) Ranking budget: We next vary the number of files and functions retained before refactoring generation. The default configuration keeps the top five files and the top ten functions overall from those files. Table VI shows the recall-cost tradeoff. With only the top file, Prefactory detects 43 functions, so a strict file cutoff loses several valid targets. Increasing the file budget from one to three recovers most of the lost recall, and increasing from three to five files adds two more functions. c) Repair loop: Finally, we evaluate the repair loop used during refactoring validation. When the differential test fails or is inconclusive, Prefactory feeds the failing input and observed outputs back to the LLM and asks it to repair the edit. This ablation asks whether validation feedback improves the final number of validated refactorings. Figure 5 shows that repair feedback steadily converts failing proposals. With only the
Count (out of 49 proposed)
TABLE V E FFECT OF DETECTOR TYPE ON CANDIDATE DETECTION . C OUNTS REPORT FILES AND FUNCTIONS DETECTED OUT OF 100 BENCHMARK INSTANCES .
Pass 49diff test Fail diff test
50 43
40
36 28
30 20
21 13
10
6 0
0 No repair
1 repair
2 repairs
3 repairs (default)
Fig. 5. Effect of repair budget on differential-test outcomes (out of 49 proposed). Green: pass the differential test; red: still fail it.
initial generation, 21 of 49 proposals pass the differential test and 28 still fail. Each repair round shrinks the failing group: after three repairs, all 49 pass the differential test. Of these, 40 are also validated by project tests; the remaining 9 pass the differential test but are rejected by project tests. 5) RQ5: Open-Source Usefulness: The fixed benchmark evaluates whether Prefactory can recover historical developer refactorings, but this setting has two limitations. First, historical commits may appear in model training data, so benchmark recovery alone cannot rule out memorization effects. Second, a benchmark success does not show whether the generated refactorings are useful to maintainers of current projects. We therefore run Prefactory on the latest versions of five benchmark projects and submit selected refactorings to the corresponding open-source maintainers. We select projects whose latest versions still install and test successfully under our harness, and run Prefactory with the corresponding target libraries from the benchmark. For each project, Prefactory reports at most ten candidate functions. We manually inspect the final candidates and find that 67% correspond to real library-adoption opportunities. To avoid overwhelming maintainers with refactoring pull requests, we submit one refactoring per repository, choosing a small validated edit that is easy to review. Two of the five submitted pull requests have already been accepted by maintainers. Figure 6 shows one such accepted refactoring: a recursive Cartesian-product implementation replaced by a single itertools.product call, reducing 17 lines to 3. These results provide initial evidence that Prefactory can surface actionable refactorings on current project versions, not only on mined historical commits. IV. L IMITATIONS AND T HREATS TO VALIDITY Prefactory handles function-level, behavior-preserving replacements. It does not address refactorings spanning files, requiring architectural changes, or modifying public interfaces. It rejects code that extends a library API with project-specific logic, since direct replacement would drop that behavior. Without a generated detector match on the target file or function, an opportunity cannot reach ranking or generation. This is most visible for large, heterogeneous libraries such as numpy and pandas, with API families that use different vocabulary and shapes. Specialized API-family or submodule detectors
BEFORE from typing import List, Any ... def create_combinations(combinations: List[Any]) -> List[ Any]: def __create_combinations(combinations_inner: List[ Any]) -> List[Any]: ... ⟨+14 removed lines⟩ return [combo for combo in arr if len(combo) == count ] AFTER (accepted pull request) from itertools import product ... def create_combinations(combinations: List[Any]) -> List[ Any]: if not combinations: return [] return [list(combo) for combo in product(* combinations)]
Fig. 6. Accepted pull request: recursive Cartesian-product enumeration replaced by itertools.product (17 lines to 3 lines).
could improve coverage. We use the original implementation as the oracle and combine LLM-generated differential tests with project tests when available. However, tests cannot prove semantic equivalence. Validated refactorings may differ on untested inputs, while correct refactorings may be rejected because differential tests allow no divergence, even when benign. PrefactoryBench is mined from real developer refactorings, but reflects only committed opportunities and may omit other valid refactorings. Our implementation and evaluation focus on Python; the detector representations and validation workflow would need adaptation for other languages. Baseline results also depend on prompts, tool configurations, model versions, and search budgets; we mitigate this with comparable targetlibrary hints and, for Codex CLI, the same model as Prefactory. V. R ELATED W ORK a) Refactoring detection and mining: Much work detects refactorings by mining repository histories. RefactoringMiner and its successors [13], [14] compare consecutive program versions and classify edits into Fowler-style categories [15]. Downstream studies use these tools to study refactoring frequency and motivation [16], [17]. These approaches mine existing history; Prefactory instead finds and applies new library-adoption opportunities. b) Custom-to-API replacement and API migration: The closest prior work studies replacing custom implementations with API calls. RETIWA [4] and AKIRA [5] detect or recommend custom Java logic replaceable by an API, but do not generate or validate the resulting edits. Prefactory differs in three ways: it targets library-adoption refactoring in Python projects, generates candidate edits, and accepts edits only after relevance filtering and execution-based validation. LibraryMigration [18], A3 [19], and API-usage recommendation systems such as MAPO and DeepAPI [20], [21] rely on existing API usages, migration examples, or usage patterns. Library adoption is harder because the replaced code may never reference the target library. Prefactory therefore generates detectors for hand-written implementations and combines lexical and structural evidence with project context before invoking an LLM to rewrite selected candidates. Since
API documentation improves LLM code generation for lesscommon libraries [22], Prefactory includes the target library and relevant API hints in the refactoring-generation prompt. c) Static modernizers and linters: Static modernizers and linters encode refactoring rules. Python tools such as pyupgrade [23], refurb [2], Ruff [1], Pylint [3], and flake8 plugins [24] detect modernization and simplification opportunities with manually written rules; Zhang et al. [25] extend this to automatically rewrite non-idiomatic Python using AST transformations for predefined Pythonic idioms. The broader tradition of semantic patching, exemplified by Coccinelle [26] for Linux C code, showed that executable AST-level patterns can be applied systematically across large codebases. Largescale experience with such analyses confirms that they deliver reliable, actionable results for covered patterns, but extending coverage to new domains requires substantial ongoing manual effort [27], [28]. Prefactory targets this gap by generating detectors automatically from library metadata. d) LLM agents for software engineering: LLM agents support end-to-end software engineering tasks, including issue resolution in SWE-Agent and SWE-bench [29], [30], project setup and test execution in ExecutionAgent [31], and interactive coding with Aider and Claude Code [32], [33]. These systems show that LLMs can navigate repositories and produce edits, but open-ended agentic search can be expensive and hard to control when scanning many projects or target libraries. Prefactory uses LLMs in bounded roles: one generates reusable detectors offline, deterministic scanning applies them, and another is invoked only for top-ranked candidates. e) Validating generated edits: Generated code can be evaluated by static checks, LLM judges, or execution. LLMas-judge methods [34] scale assessment, but judgment alone cannot establish behavior preservation. Execution-grounded benchmarks such as SWE-bench [30] and systems such as ExecutionAgent [31] instead emphasize running project tests. Prefactory follows this view: LLM-generated differential tests pre-filter and guide repair, but final acceptance depends on relevance filtering and project tests. VI. C ONCLUSION This paper introduced Prefactory, an automated approach for library-adoption refactoring in Python. On PrefactoryBench, Prefactory detects the correct file in 75 of 100 instances, surfaces the correct function in 56, and produces 40 validated refactorings, outperforming all baselines at roughly one-third the cost of the strongest agentic competitor. The failure analysis points to what remains difficult: large, heterogeneous libraries need finer-grained detector targeting, and I/O-heavy or distributed code resists differential validation. These results show that capturing LLM knowledge in reusable, executable detectors is more effective and cheaper than applying LLM reasoning through repeated project-wide queries. R EFERENCES [1] C. Marsh and Astral Software Inc., “Ruff: An extremely fast python linter and code formatter, written in Rust,” https://github.com/astral-sh/ ruff, 2022, accessed 2026-06-27.
[2] dosisod, “refurb: A tool for refurbishing and modernizing python codebases,” https://github.com/dosisod/refurb, 2022, accessed 2026-06-27. [3] Pylint contributors, “Pylint: A static code analyser for Python,” https: //github.com/pylint-dev/pylint, 2003, accessed 2026-06-27. [4] R. Tufano, E. Aghajani, and G. Bavota, “Don’t reinvent the wheel: Towards automatic replacement of custom implementations with APIs,” in 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2022, pp. 394–398. [5] B. Nyirongo, Y. Jiang, Y. Zhang, and H. Liu, “From custom logic to APIs: Understanding and recommending API replacement refactorings,” 2026. [6] W. M. McKeeman, “Differential testing for software,” Digital Technical Journal, vol. 10, no. 1, pp. 100–107, 1998. [7] E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,” IEEE Transactions on Software Engineering, vol. 41, no. 5, pp. 507–525, 2015. [8] C. S. Xia and L. Zhang, “Conversational automated program repair,” arXiv preprint arXiv:2301.13246, 2023. [9] Z. Zhang, Z. Xing, X. Xia, X. Xu, and L. Zhu, “Making python code idiomatic by automatic refactoring non-idiomatic python code with pythonic idioms,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2022. ACM, 2022, pp. 696–708. [10] Y. Xu, J. Yang, and T.-H. Chen, “Swe-refactor: A repository-level benchmark for real-world llm-based code refactoring,” arXiv preprint arXiv:2602.03712, 2026. [11] D. Spadini, M. F. Aniche, and A. Bacchelli, “PyDriller: Python framework for mining software repositories,” in Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2018, pp. 908–911. [12] Semgrep, Inc., “Semgrep: Lightweight static analysis for many languages,” https://github.com/semgrep/semgrep, 2020. [13] N. Tsantalis, M. Mansouri, L. M. Eshkevari, D. Mazinanian, and D. Dig, “Accurate and efficient refactoring detection in commit history,” in Proceedings of the 40th International Conference on Software Engineering (ICSE), 2018, pp. 483–494. [14] N. Tsantalis, A. Ketkar, and D. Dig, “RefactoringMiner 2.0,” IEEE Transactions on Software Engineering, vol. 48, no. 3, pp. 930–950, 2022. [15] M. Fowler, Refactoring: Improving the Design of Existing Code, 2nd ed. Boston, MA: Addison-Wesley Professional, 2018. [16] D. Silva, N. Tsantalis, and M. T. Valente, “Why we refactor? confessions of GitHub contributors,” in Proceedings of the 24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE), 2016, pp. 858–870. [17] E. R. Murphy-Hill, C. Parnin, and A. P. Black, “How we refactor, and how we know it,” IEEE Transactions on Software Engineering, vol. 38, no. 1, pp. 5–18, 2012. [18] C. Teyton, J.-R. Falleri, and X. Blanc, “Mining library migration graphs,” in 19th Working Conference on Reverse Engineering (WCRE). IEEE Computer Society, 2012, pp. 289–298. [19] M. Lamothe, W. Shang, and T.-H. P. Chen, “A3: Assisting Android API migrations using code examples,” IEEE Transactions on Software Engineering, vol. 48, no. 2, pp. 417–431, 2022. [20] H. Zhong, T. Xie, L. Zhang, J. Pei, and H. Mei, “MAPO: Mining and recommending API usage patterns,” in European Conference on ObjectOriented Programming (ECOOP), ser. Lecture Notes in Computer Science, vol. 5653. Springer, 2009, pp. 318–343. [21] X. Gu, H. Zhang, D. Zhang, and S. Kim, “Deep API learning,” in Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE). ACM, 2016, pp. 631– 642. [22] J. Chen, S. Chen, J. Cao, J. Shen, and S.-C. Cheung, “When LLMs meet API documentation: Can retrieval augmentation aid code generation just as it helps developers?” 2025. [23] A. Sottile, “pyupgrade: A tool to automatically upgrade syntax for newer versions of the language,” https://github.com/asottile/pyupgrade, 2017, accessed 2026-06-27. [24] PyCQA, “flake8: Your tool for style guide enforcement,” https://github. com/PyCQA/flake8, 2010, accessed 2026-06-27. [25] Z. Zhang, Z. Xing, X. Xia, X. Xu, and L. Zhu, “Making Python code idiomatic by automatic refactoring non-idiomatic Python code with Pythonic idioms,” 2022.
[26] Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller, “Documenting and automating collateral evolutions in Linux device drivers,” in Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems (EuroSys). ACM, 2008, pp. 247–260. [27] C. Sadowski, J. van Gogh, C. Jaspan, E. Söderberg, and C. Winter, “Tricorder: Building a program analysis ecosystem,” in 37th IEEE/ACM International Conference on Software Engineering (ICSE), Volume 1. IEEE Computer Society, 2015, pp. 598–608. [28] C. Sadowski, E. Aftandilian, A. Eagle, L. Miller-Cushon, and C. Jaspan, “Lessons from building static analysis tools at Google,” Communications of the ACM, vol. 61, no. 4, pp. 58–66, 2018. [29] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “SWE-agent: Agent-computer interfaces enable automated software engineering,” in Advances in Neural Information Processing Systems (NeurIPS), 2024. [Online]. Available: https://openreview.net/ forum?id=mXpq6ut8J3 [30] C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “SWE-bench: Can language models resolve real-world GitHub issues?” in The Twelfth International Conference on Learning Representations (ICLR), 2024. [Online]. Available: https://openreview.net/forum?id=VTF8yNQM66 [31] I. Bouzenia and M. Pradel, “You name it, i run it: An llm agent to execute tests of arbitrary projects,” Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 1054–1076, 2025. [32] P. Gauthier, “Aider: Ai pair programming in your terminal,” https: //github.com/Aider-AI/aider, 2023, accessed 2026-06-27. [33] Anthropic, “Claude code: Anthropic’s agentic coding system,” https:// www.anthropic.com/product/claude-code, 2025, accessed 2026-06-27. [34] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” in Advances in Neural Information Processing Systems 36 (NeurIPS), Datasets and Benchmarks Track, 2023.