Understanding the Energy Impact of Software Refactoring: A Workload-Aware Study of Controlled Examples and Real-World Commits

Understanding the Energy Impact of Software Refactoring: A Workload-Aware Study of Controlled Examples and Real-World Commits HAIBO WANG, Concordia University, Canada HENG LI, Polytechnique Montréal, Canada SHIN HWEI TAN∗ , Concordia University, Canada Refactoring improves software maintainability while preserving functional behavior, yet behavior preservation does not imply energy neutrality. Existing studies primarily examine isolated refactorings under fixed or simple workloads, leaving the effects of workload variation, real-world refactoring practices, explanatory factors, and energy regression identification insufficiently understood. We present the first large-scale empirical study of the energy impact of refactoring across two complementary Java benchmarks: a Micro-benchmark, comprising 68 refactoring types evaluated under diverse workloads, and a Practical-benchmark, containing 481 real-world refactoring commits from 430 GitHub projects. Using repeated paired energy measurements, we analyze workload sensitivity, refactoring patterns, explanatory factors, and the effectiveness of metric- and LLM-based regression identification. In

arXiv:2608.06620v1 [cs.SE] 6 Aug 2026

the Micro-benchmark, 199 of 384 refactoring-workload pairs (51.8%) exhibit statistically significant energy differences, and 45.3% of refactoring instances change energy-impact classification across workloads. In the Practical-benchmark, only 36 commits (7.5%) show significant energy changes, although two-thirds differ by at least 10%. Refactoring type alone is insufficient to predict energy outcomes, while certain recurring refactoring combinations are associated with energy reductions. Changes in execution time consistently explain energy variation in the controlled benchmark but correlate weakly with energy changes in real-world commits. Our findings highlight the need for workload-diverse evaluation of the energy impact of refactoring; neither existing metric-based approaches nor LLM-based predictors can reliably identify refactoring-induced energy regressions, motivating the development of more accurate techniques for predicting the energy impact of refactoring. CCS Concepts: • Software and its engineering → Software evolution; Maintaining software. Additional Key Words and Phrases: Refactoring, Energy Consumption, Green Coding, Sustainable Software Engineering ACM Reference Format: Haibo Wang, Heng Li, and Shin Hwei Tan. 2026. Understanding the Energy Impact of Software Refactoring: A Workload-Aware Study of Controlled Examples and Real-World Commits. 1, 1 (August 2026), 47 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

1

Introduction

Refactoring is one of the most prevalent forms of software evolution [41, 47]. Developers refactor code to improve its internal structure, reduce duplication, simplify comprehension, and lower future maintenance costs, while preserving the externally observable behavior of the program [20, 57, 64]. Unlike functional changes, refactoring aims to modify how software is implemented rather than what it does, making behavioural preservation a central requirement [65, 66]. ∗ Corresponding author.

Authors’ Contact Information: Haibo Wang, Concordia University, Montréal, Quebec, Canada, haibo.wang@mail.concordia.ca; Heng Li, Polytechnique Montréal, Montréal, Quebec, Canada, heng.li@polymtl.ca; Shin Hwei Tan, Concordia University, Montréal, Quebec, Canada, shinhwei.tan@concordia.ca. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. Manuscript submitted to ACM

Manuscript submitted to ACM

1

2

Haibo Wang, Heng Li, and Shin Hwei Tan

However, preserving functional behaviour does not guarantee that other software properties remain unchanged. Refactoring can modify execution paths, object creation patterns, library interactions, exception handling, synchronization mechanisms, and compiler optimization opportunities, thereby affecting runtime characteristics. Such changes may influence not only execution time but also memory allocation, garbage collection, CPU utilisation, and ultimately energy consumption [54, 59]. Thus, behavioural preservation does not imply energy neutrality. Energy consumption has emerged as an important quality concern for modern software systems. While traditionally viewed as a hardware or deployment-level issue, energy efficiency is also influenced by software design and evolution decisions. Software now operates across a wide range of environments, from battery-constrained mobile and embedded devices, where energy consumption directly affects usability and service lifetime, to cloud platforms, data centers, and continuous integration infrastructures, where large-scale computation contributes to operational costs and environmental impacts [4, 10, 28, 49, 50, 62]. Empirical studies have demonstrated that software-level changes can introduce measurable energy consequences. For example, analyses of Android applications have linked energy consumption to application-level implementation choices and API usage patterns, while large-scale studies of mobile software evolution have shown that feature additions, bug fixes, and other code changes can alter energy behavior [35, 52]. These findings indicate that energy efficiency should be considered throughout the software lifecycle, rather than only during hardware selection or system deployment. Among software evolution activities, refactoring is particularly important because it is routinely applied to improve internal structure, maintainability, and long-term sustainability while preserving externally observable behavior. However, behavioral equivalence does not necessarily guarantee energy neutrality: changes intended solely to improve code quality may alter execution characteristics and energy consumption. Despite the widespread adoption of refactoring, its potential energy implications remain insufficiently understood. Understanding when and why refactoring affects energy consumption is therefore essential for enabling developers to perform energy-aware software maintenance and evolution. Several studies have investigated whether code refactoring can change software energy consumption. Sahin et al. [54] conducted an empirical study of six commonly used refactoring types on nine Java applications. They applied each refactoring type in Java applications and compared the energy consumption of the original and refactored versions, and showed that refactoring can statistically significantly affect energy usage in both directions: some refactoring reduce energy consumption, while others increase it. They also found that easily collectible signals such as execution time and dynamic execution counts are insufficient to fully predict the energy impact of refactoring. Park et al. [48] investigated the energy consumption of 63 refactoring types in the refactoring book [20] by constructing original and refactored examples and estimating their energy consumption by executing simple tests. Their results classified refactoring techniques into positive, neutral, and negative groups from the perspective of energy efficiency. More recently, Şanlıalp et al. [56] studied the energy efficiency of triple combinations of five types of refactoring. Together, these studies provide important evidence that refactoring can influence energy consumption. However, existing studies on the energy impact of refactoring share the following limitations: Limitation 1: The energy measurement is performed under fixed or simple workloads. Energy measurement results can depend heavily on the workload used during execution. Prior performance studies have shown that workload variation can substantially change measured performance behavior [42]. This concern also applies to refactoring-energy analysis: the same refactoring may show different energy effects when the refactored code is executed with different workloads. Therefore, workload design is not merely an implementation detail, but an important factor in measuring the energy impact of refactoring.

Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

3

Existing refactoring-energy studies usually rely on simple workloads. For example, Park et al. [48] constructed 63 small examples for Fowler refactoring types [20] and customized them to run without external input, so that input variation would not affect power estimation. While this design improves controllability, it measures each refactoring under a fixed workload. Similarly, when functional tests are used as workloads, they are primarily designed to validate program behavior, not to expose energy-sensitive runtime behavior. Such tests may execute the refactored code only with small or typical inputs, and may not stress execution paths where the refactoring changes allocation, garbage collection, or CPU utilization. To address this limitation, we construct a workload-aware Micro-benchmark in which each refactoring item is exercised under five-level workloads (XS–XL), with repeated executions for energy measurement. As we later show in Section 5.1.1, 29 of the 64 eligible refactoring instances (45.3%) receive different increase, decrease, or unchanged classifications across XS–XL. These results demonstrate that relying on a single simple or fixed workload may miss or even reverse the inferred energy impact of a refactoring. Limitation 2: Lack of energy measurement on real-world refactoring commits. Existing refactoring-energy studies mainly rely on manually constructed refactoring examples or simple example code. For example, Sahin et al. [54] selected six commonly used Eclipse refactoring types, applied one refactoring type at a selected program location, and then measured the energy consumption of the original and refactored versions. Similarly, Park et al. [48] constructed small original and refactored examples for Fowler refactoring types. These designs are useful for isolating the effect of a refactoring type, but they do not directly represent how refactoring appears in software evolution history. In real-world development, refactoring is often performed at commit level rather than as a single isolated transformation. A refactoring commit may modify multiple files and contain multiple refactoring instances. Its energy impact may therefore differ from that observed in a manually constructed single-refactoring example. This limitation motivates our construction of the Practical-benchmark, where we study the energy impact of real-world refactoring commits. Limitation 3: Limited understanding of the causes of refactoring-induced energy differences. Measuring whether refactoring changes energy consumption is important, but measurement alone does not explain why the change occurs. To make refactoring-energy results actionable, it is necessary to associate observed energy differences with concrete source-level and runtime-level factors. Such explanations can help developers understand which types of refactoring are more energy-sensitive, guide the design of energy-aware refactoring recommendations, and inspire future techniques for detecting or optimizing energy-related regressions. Prior studies on software performance and energy optimization have shown that identifying recurring code patterns and runtime behaviors is useful for building practical detection and optimization techniques [29, 37, 67]. Existing refactoring-energy studies mainly report whether energy consumption changes after refactoring, but provide limited explanation of the causes behind such changes. For example, Sahin et al. [54] found that refactoring can increase or decrease energy consumption, and further showed that execution time and dynamic execution counts are insufficient to accurately predict the energy impact of applying a refactoring. This result indicates that refactoring-induced energy differences cannot be fully explained by a single simple metric. However, the underlying factors, such as changes in allocation behavior and garbage collection, remain underexplored. This limitation motivates our investigation of the factors that help explain why refactoring leads to energy consumption differences (RQ3). Limitation 4: Lack of evaluation of existing techniques for detecting refactoring-induced energy regressions. Energy measurement is expensive and difficult to apply to every refactoring change during development. If energyregression refactoring can be identified before full energy measurement, developers could prioritize risky refactoring commits for further validation. This is important because energy regressions may not be visible from functional behavior: a refactored program may preserve the same outputs while consuming more energy during execution. Therefore, Manuscript submitted to ACM

4

Haibo Wang, Heng Li, and Shin Hwei Tan

beyond measuring and explaining energy differences, it is also important to study whether existing regression-detection techniques can help identify refactoring-induced energy regression. Existing refactoring-energy studies provide limited evidence on this problem. For example, Sahin et al. [54] examined whether easily collectible information, execution time and dynamic execution counts, can characterize the energy impact of refactoring, but found that these two signals are insufficient to predict the effect of applying a refactoring. This result suggests that detecting refactoring-induced energy regressions may require richer information than simple runtime or execution-count metrics. However, it remains unclear how well existing regression-prediction techniques can identify refactoring commits that significantly increase energy consumption. This limitation motivates our evaluation of metric-based techniques adapted from existing regression-prediction studies [67] and our further exploration of whether LLMs can help identify refactoring-induced energy regressions (RQ4). To address these limitations, we conduct an empirical study of the energy impact of software refactoring from two complementary perspectives. First, we construct Micro-benchmark, a workload-aware benchmark of executable Java refactoring examples. It covers 68 refactoring types and exercises the refactored code under diverse inputs and repeated executions, allowing us to study how individual refactoring types affect energy consumption under controlled but non-trivial workloads. Second, we construct Practical-benchmark, a benchmark of real-world GitHub refactoring commits. This benchmark allows us to measure the energy impact of refactoring as it appears in software evolution history, where a refactoring commit may modify multiple files and contain multiple refactoring instances. Across both benchmarks, we measure the energy consumption before and after refactoring, apply paired statistical analysis, and further examine factors that help explain energy differences and techniques that identify energy regression refactoring commits. Specifically, we investigate the following research questions: RQ1: What is the impact of refactoring on energy consumption? We measure and compare the energy consumption of original and refactored programs in both Micro-benchmark and Practical-benchmark, and analyze whether refactoring leads to statistically significant energy differences. RQ2: Which refactoring types are more likely to influence energy consumption? We further analyze energy changes across refactoring types in Micro-benchmark and examine refactoring co-occurrence patterns in Practicalbenchmark to understand which refactoring types or combinations are more associated with energy differences. RQ3: What factors explain refactoring-induced energy differences? We investigate source-level and runtimelevel factors, such as execution time, allocation behavior, garbage collection, and compiler optimization behavior, to understand why refactoring can lead to energy consumption differences. RQ4: How effective are existing techniques in identifying refactoring-induced energy regressions? We evaluate metric-based techniques adapted from existing regression-prediction studies [67] and further explore whether LLMs can identify refactoring commits that significantly increase energy consumption. Key Findings. Our empirical study shows that refactoring is not inherently energy-neutral. Across controlled benchmark (Micro-benchmark), 51.8% of refactoring–workload comparisons exhibit statistically significant energy differences, and over one-quarter (27.9%) change energy consumption by at least 10%. However, the observed impact is highly workloaddependent: nearly half (45.3%) of refactorings change their inferred energy outcome across workloads, demonstrating that conclusions drawn from a single workload can be misleading. In contrast, statistically significant energy changes occur less frequently in real-world refactoring commits (7.5%), although those changes are often substantial, with twothirds (66.7%) exceeding 10%. We further show that while many refactoring types and recurring refactoring combinations are associated with significant energy differences, their effects are not deterministic and depend on execution context. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

5

Finally, execution time explains only part of the observed energy variation, and neither metric-based approaches nor LLM-based predictors reliably identify energy regressions. Overall, these findings demonstrate that the energy impact of refactoring is complex, context-dependent, and difficult to predict without direct empirical measurement. This paper makes the following contributions: • We construct Micro-benchmark, a workload-aware benchmark of executable Java refactoring examples covering 68 refactoring types. This benchmark enables controlled energy analysis of individual refactoring types under diverse workloads. • We construct Practical-benchmark, a benchmark of real-world GitHub refactoring commits. This benchmark enables the study of refactoring energy impact as it appears in software evolution history, where commits may involve multiple files and multiple refactoring operations. • We conduct a systematic empirical study of refactoring energy impact across both controlled examples and real-world commits. Our study measures before- and after-refactoring energy consumption under repeated executions and applies statistical analysis to quantify refactoring-induced energy differences. To support open science, the code and data of our study are publicly available at [63]. • We evaluate whether refactoring-induced energy regressions can be identified using metric-based techniques adapted from regression-prediction studies, and further explore the capability of LLM for this task. 2

Motivation Example 1 2 3 4 5 6 7 8 9 10 11

public static final class Names { public List < String > namesOfProgrammers ( List < Person > input ) { List < String > names = new ArrayList < >() ; for ( Person p : input ) { if (" programmer ". equals (p . getJob () ) ) { names . add ( p. getName () ) ; } } return names ; } }

Listing 1. Code before refactoring. 1 2 3 4 5 6 7 8

+ + + +

public static final class Names { public List < String > namesOfProgrammers ( List < Person > input ) { return input . stream () . filter (p -> " programmer ". equals (p . getJob () ) ) . map ( After . Person :: getName ) . collect ( Collectors . toList () ) ; } }

Listing 2. Code after refactoring.

Fig. 1. Example of Replace Loop with Pipeline refactoring which increases energy consumption across different workload groups.

Figure 1 shows the Replace Loop with Pipeline refactoring used as the motivating example from our Micro-benchmark. The method takes a list of Person objects and returns the names of people whose job is "programmer". As shown in Listing 1, before refactoring, the method creates an ArrayList, traverses the input list with a for loop, checks whether each person is a programmer, and appends the selected names to the result list. After refactoring, the same computation is expressed with a stream pipeline1 that applies filter, map, and collect as shown in Listing 2. The output of the 1 https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html

Manuscript submitted to ACM

6

Haibo Wang, Heng Li, and Shin Hwei Tan

method remains the same, but the execution is no longer the same direct loop-and-append procedure. The refactored version delegates traversal, filtering, mapping, and result construction to the stream() API. This change captures the case where a refactoring improves code style or readability, but may also change the runtime behavior of the program. Table 1. Workload input scale and coverage summary for the motivating Replace Loop with Pipeline example. Aspect

Value

Input type Input-size levels Modified-line coverage

List XS=100, S=1,000, M=10,000, L=100,000, XL=1,000,000 100% for both versions

To make the energy comparison meaningful, the workload must execute the changed code and run long enough to produce observable energy measurements. Table 1 summarizes the workload setup for this example. The input is a List, and the workload suite covers five input-size levels, from XS with 100 elements to XL with 1,000,000 elements [15, 16]. In addition to reporting each size level separately, we also report an Aggregate workload group. This group executes the full workload suite across all input-size levels, and therefore represents the overall behavior of the example when the diverse workloads are considered together. The workloads achieve 100% modified-line coverage for both versions, ensuring that the refactored logic is exercised during measurement. For each workload group, we first estimate how many repetitions are needed to make the execution last around 10 seconds, so that the energy consumption is large enough to be measured more reliably [15, 16]. As the before and after versions may have different execution speeds, we use a common repetition count for the two versions. Specifically, we choose the larger repetition count between the before and after versions, and then run both versions with that same count. This avoids comparing two versions under different amounts of logical work. We repeat the energy measurement 30 times for the before version and 30 times for the after version, and use these repeated measurements for the distributions and statistical comparisons reported below [17, 40].

Fig. 2. Energy consumption distributions before and after the Replace Loop with Pipeline refactoring across workload groups.

Figure 2 compares the before- and after-refactoring energy distributions across the Aggregate workload group and the five input-size groups. The after-refactoring distributions are consistently shifted upward, indicating that the stream-based version consumes more energy than the loop-based version under every workload group. The figure Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

7

also shows that absolute energy consumption generally increases as the workload becomes larger, with XL having the highest energy values for both versions. The Aggregate group summarizes the combined workload suite across XS to XL, so it provides an overall view of the refactoring effect across diverse input scales rather than representing another individual input size. Together, these patterns show that the refactoring increases energy consumption consistently, while the absolute energy cost and distribution shape vary across workload groups.

Fig. 3. Distributions of energy consumption change ratios across workload groups for the motivating Replace Loop with Pipeline example. The change ratio is computed as (𝐸 after − 𝐸 before )/𝐸 before × 100.

The change ratios in Figure 3 show that the increase is not uniform. The Aggregate workload has a mean increase of 12.99%, while the individual input-size groups range from 11.93% for S to 26.99% for L. The XS group is close to L, with a 26.00% increase. XL has the largest raw energy values, but its relative increase is 14.99%, smaller than those of L and XS. Thus, a single aggregate value would correctly indicate that the refactoring increases energy, but it would hide how much the measured effect changes with input size. The same refactoring does not correspond to one fixed energy effect across workloads. This motivates the need for workload-aware measurement, since conclusions drawn from one input scale may understate or overstate the impact of a refactoring. Table 2. Energy comparison results for the motivating Replace Loop with Pipeline example. Workload group Aggregate XS S M L XL

Before (J)

After (J)

Δ (J)

Δ (%)

95% CI of Δ (J)

WSR test

102.52 38.81 40.06 51.76 63.07 130.84

115.84 48.90 44.84 65.12 80.09 150.45

+13.32 +10.09 +4.78 +13.36 +17.02 +19.61

+12.99% +26.00% +11.93% +25.81% +26.99% +14.99%

[10.07, 16.58] [7.90, 12.28] [2.88, 6.68] [9.10, 17.62] [12.77, 21.26] [14.77, 24.44]

𝑊 = 15, 𝑝 = 2.55 × 10 −7 𝑊 = 3, 𝑝 = 2.35 × 10 −6 𝑊 = 41, 𝑝 = 1.82 × 10 −5 𝑊 = 5, 𝑝 = 1.86 × 10 −8 𝑊 = 12, 𝑝 = 1.30 × 10 −7 𝑊 = 12, 𝑝 = 1.30 × 10 −7

Cliff’s 𝛿

0.60 0.66 0.40 0.51 0.91 0.64

Before and after values are means over repeated measurements. Δ is computed as 𝐸 after − 𝐸 before , and the percentage change is computed as (𝐸 after − 𝐸 before )/𝐸 before × 100. WSR test denotes the Wilcoxon signed-rank test.

Table 2 provides the statistical comparison behind these distributions. The after-refactoring version has a higher mean energy consumption for every workload group. The mean difference is +13.32 J for Aggregate and ranges from +4.78 J to +19.61 J across the individual input-size groups. The 95% confidence intervals of the mean differences are all positive and the Wilcoxon signed-rank tests are statistically significant for all workload groups. Cliff’s 𝛿 ranges from 0.40 to 0.91, indicating non-negligible effects. These results support two conclusions. First, the refactoring is Manuscript submitted to ACM

8

Haibo Wang, Heng Li, and Shin Hwei Tan

not energy-neutral in this example. Second, both absolute and relative views are needed. XL has the largest absolute mean difference, while L and XS have the largest relative increases. This means that a single workload may not fully characterize the energy impact of the refactoring. To understand refactoring-induced energy changes, measurement needs to consider how the refactored code behaves under different workload scales.

Table 3. Selected runtime-profile metrics for the motivating Replace Loop with Pipeline example. Workload group

Δ𝐸 (%)

Δ Time (%)

Δ Alloc. samples (%)

Δ Young GC (%)

Δ GC pause (%)

Δ CPU-load records (%)

Aggregate XS S M L XL

+12.99 +26.00 +11.93 +25.81 +26.99 +14.99

+16.06 +93.57 +47.06 +38.45 +23.55 +26.78

+13.29 +86.12 +47.88 +33.71 +19.05 +28.87

+4.88 +50.00 +21.88 +3.90 +57.58 +8.70

+7.87 +77.14 +15.47 +4.61 +11.42 +36.37

+20.00 +114.29 +66.67 +44.44 +30.00 +25.00

All values are computed as (𝑋 after − 𝑋 before )/𝑋 before × 100. Δ𝐸 compares mean energy consumption across repeated measurements. Δ Time is the change ratio of execution time. Δ Alloc. samples compares sampled heap-object allocation events during execution, which reflect objects created by the program. Δ Young GC compares young-generation garbage-collection events, which mainly reclaim short-lived heap objects. Δ GC pause compares the total time spent in garbage-collection pauses. Δ CPU-load records compares the number of recorded CPU-load observations during execution.

The energy results show that the refactoring changes energy consumption, but they do not explain the source of the change. Table 3 reports selected runtime-profile metrics for the same example. After refactoring, execution time, allocation samples, young-generation GC events, GC pause time, and CPU-load records all increase. These changes are consistent with the implementation difference in Figure 1: the refactored version replaces a direct loop and explicit list update with stream operations, lambda-based processing, and collector-based result construction. Such changes can introduce additional runtime work even when the method output is preserved. The runtime-profile metrics also show that the energy increase cannot be explained by a single factor. Under XS, energy increases by 26.00%, while execution time and allocation samples increase by 93.57% and 86.12%, respectively. Under M, the energy increase is 25.81%, but the young-GC increase is only 3.90%. Under L, energy increases by a similar amount, 26.99%, while young GC increases by 57.58%. These patterns suggest that execution time, allocation behavior, garbage collection, and CPU activity provide complementary signals. Energy measurement reveals that a difference exists, but runtime-profile metrics are needed to help interpret why the difference occurs. This motivating example highlights several challenges in understanding the energy implications of software refactoring. First, although the refactoring preserves externally observable behavior, it introduces a statistically significant energy change, demonstrating that the energy impact cannot be inferred from functional equivalence alone and must be empirically measured (RQ1). Second, the magnitude and direction of the energy change might vary across workload scales, highlighting the importance of workload-aware evaluation rather than relying on a single execution scenario. Third, this is only one refactoring instance, therefore, a broader analysis is needed to characterize the relationship between refactoring types, refactoring patterns, and energy changes (RQ2). Finally, the observed energy variation is correlated with changes in execution time, memory allocation, garbage collection, and CPU activity, suggesting that runtime characteristics can provide valuable evidence to explain energy differences (RQ3). These observations motivate our empirical study, which systematically investigates refactoring-induced energy changes across controlled benchmarks and real-world commits, identifies source-level and runtime factors associated with these changes, and evaluates whether available signals can support the detection of refactoring-induced energy regressions (RQ4). Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring 3

9

Methodology

3.1

Study Overview

Fig. 4. Overview of the study workflow.

Figure 4 summarizes the overall workflow of our study. The methodology consists of four main steps: benchmark construction, energy consumption measurement, metric extraction, and research-question-oriented analysis. We design the workflow around two complementary benchmarks. The first one, Micro-benchmark, provides controlled Java refactoring examples for studying individual refactoring types under workload-aware executions. The second one, Practical-benchmark, contains real-world GitHub refactoring commits for studying refactoring energy impact as it appears in software evolution history. In the benchmark construction step, we prepare executable before/after program pairs for both benchmarks. For Micro-benchmark, we start from Java refactoring examples, use an LLM-assisted pipeline to generate workloads, and validate the generated benchmark items through compilation and coverage checks. If a generated workload fails validation, the pipeline regenerates it until a valid executable benchmark item is obtained or the retry limit is reached. For Practical-benchmark, we collect candidate GitHub refactoring commits and apply filtering and validation steps to Manuscript submitted to ACM

10

Haibo Wang, Heng Li, and Shin Hwei Tan

obtain executable commit pairs whose before and after versions can be measured under comparable test workloads. The detailed construction procedures are described in Section 3.2. After benchmark construction, we measure energy consumption by repeatedly executing the before and after versions of each benchmark subject. This measurement step records per-run energy and timing information, which is later used to compare the energy behavior before and after refactoring. For Micro-benchmark, the measurement uses generated workloads and a common repetition setting for the before and after versions so that the two versions execute the same amount of logical work. For Practical-benchmark, the measurement uses project test suites as workloads. The detailed energy measurement strategy is described in Section 3.3. We also collect metrics to support explanation and regression-identification analyses. Runtime-profile metrics characterize how the program executes under the benchmark workload, such as execution time, memory allocation activity, garbage collection, and CPU-related observations. In addition, source-level, code-change, refactoring, and semantic metrics are extracted from the program source code and commit diffs to describe the structure and content of the refactoring changes. These metrics provide complementary views for analyzing why refactoring can lead to energy differences and whether such differences can be identified from measurable signals. Section 3.4 describes the metrics and their extraction procedures in detail. Finally, the collected energy measurements and metrics are used to answer the four research questions. RQ1 compares before/after energy consumption to quantify the energy impact of refactoring. RQ2 analyzes which refactoring types and refactoring patterns are more strongly associated with energy differences. RQ3 examines runtime-level and source-level factors that help interpret observed energy changes. RQ4 evaluates whether metric-based and LLM-based techniques can identify refactoring-induced energy regressions. Section 4 presents the experimental design for answering these research questions.

3.2

Benchmark Construction

3.2.1 Micro-benchmark. The goal of Micro-benchmark is to provide a controlled benchmark for measuring the energy impact of individual refactoring types. In real-world commits, multiple refactoring operations may be applied together, making it difficult to isolate the effect of one refactoring type. Micro-benchmark complements such commit-level analysis by providing small executable Java programs in which the before and after versions implement one intended refactoring type and can be exercised under the same generated workloads. We construct Micro-benchmark based on Fowler’s refactoring catalog [21]. We use the second edition catalog as the reference list of refactoring types because it provides a comprehensive and updated organization of refactoring operations [20]. However, the online examples in the second edition are mainly written in JavaScript and many of them are illustrative fragments rather than directly executable Java benchmark programs. Since our measurement pipeline targets Java programs, we use the first edition of Fowler’s refactoring book as the primary source for Java-based examples [19]. For each refactoring type that appears in both editions, we match the refactoring by name and manually extract the corresponding Java before- and after-refactoring example from the first edition. For refactoring types that do not have Java examples in the first edition, we manually extract the corresponding JavaScript example from the second-edition online catalog and translate it into Java. After collecting these examples, we inspect and validate all resulting program pairs to ensure that the before and after versions are compilable and implement the intended refactoring. In this way, each retained benchmark item contains a Java before/after program pair associated with one refactoring type. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

11

Workload generation. The Java before/after program pairs extracted from Fowler’s refactoring examples are designed to illustrate refactoring mechanics; they do not provide workloads for repeated energy measurement. However, energy and performance measurements can be sensitive to the workload input-size levels used during execution [15, 16, 23]. As discussed in Limitation 1, the same refactoring may show different energy effects when the refactored code is executed with different workload input-size levels, as shown in our motivation example in Section 2. To address this issue, we generate diverse workloads for each benchmark item by varying input-size levels.

Fig. 5. Prompt structure for LLM-assisted workload generation in Micro-benchmark.

We use an LLM with a one-shot example to generate workloads for each refactoring item. Figure 5 summarizes the prompt structure. The prompt is organized into seven main components. (1) Description of the task. This component states that the goal is to generate workloads for one refactoring item. It defines the generation objective: the workloads should exercise the refactored computation and support repeated execution for energy measurement. (2) Workload specification. This component gives the main workload-generation requirements. It requires deterministic workloads, a mandatory input-size scale axis with the XS/S/M/L/XL levels, and coverage of the modified logic in both the before and after versions. It also allows additional logic axes only when they directly affect the refactored logic. (3) Required files and roles. This component describes the expected output organization and the role of each generated output. Its purpose is to make the generated workload executable by the downstream measurement pipeline, rather than producing only isolated test inputs. (4) Input source files. This component provides the before- and after-refactoring Java programs for the current item. The LLM uses these source files to identify the method under measurement, identify relevant input types, and generate workloads that exercise the refactored code. Manuscript submitted to ACM

12

Haibo Wang, Heng Li, and Shin Hwei Tan (5) Output format requirements. This component specifies the format of the LLM response so that the pipeline can automatically extract and compile the generated output. This reduces manual intervention and enables the same generation procedure to be applied across benchmark items. (6) One-shot example reference. This component provides one complete example to illustrate the expected result. The example is used as a reference for workload generation, deterministic input construction, and compatibility with the measurement pipeline, but the generated workloads are adapted to the current refactoring item. (7) Feedback from previous attempt. When a generation attempt fails file extraction or compilation, the pipeline includes the corresponding diagnostic information in the next prompt. This feedback allows the LLM to repair the generated output in subsequent attempts. This prompt design makes the workload generation systematic across refactoring items while still allowing each

workload to be adapted to the input types and execution behavior of the corresponding program. Table 4 summarizes the input-size levels used during workload generation. Each workload suite includes one mandatory scale axis with five levels: XS, S, M, L, and XL. We choose these levels as an order-of-magnitude scale rather than as item-specific constants. Prior Java performance and energy studies commonly vary input size to construct performance or energy profiles [15, 16, 23]. Following this practice, we use five progressively larger levels from 102 to 106 so that each refactoring item can be exercised from small to large inputs under a uniform scale. Depending on the input type of the refactored code, the same scale values may correspond to the number of collection elements, string characters, loop iterations, or map/set entries. Table 4. Input-size levels for workload generation in Micro-benchmark.

Scale level

XS

S

M

L

XL

Size value

100

1,000

10,000

100,000

1,000,000

Besides input size, some refactored code may also be affected by the distribution of input values. Therefore, the prompt allows additional workload characteristics to be varied only when they are relevant to the refactored logic. For example, a workload may vary the proportion of values in a collection that satisfy a condition or the proportion of null values in the input. These additional characteristics are not added by default. We do not predefine these additional characteristics for all benchmark items; instead, we instruct the LLM to identify them from the current before/after program pair when constructing the workloads. The generated workloads follow several design requirements. They must exercise the modified logic in both the before and after versions, be deterministic and reproducible. They also support both aggregate execution over all size levels and separate execution for individual size levels. The aggregate execution provides an overall view of the refactoring item’s energy behavior, while the per-size execution allows us to examine whether the energy effect changes across input-size levels as shown in the motivation example in Section 2. Validation of generated workloads. After each LLM generation attempt, the pipeline applies several validation checks to the generated output. It first checks whether the LLM response contains all required files. It then compiles the generated Java files to check that the before- and after-refactoring programs and the generated workloads are syntactically valid and can be built together. If file extraction or compilation fails, the pipeline feeds the diagnostic information back to the LLM and asks it to regenerate the output, up to a predefined retry limit. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

13

For workloads that compile successfully, we further check whether they execute the refactored code. We use modifiedline coverage as the validation criterion: the workload must cover the executable lines changed by the refactoring in both the before and after versions. This check helps prevent us from using workloads that compile and run but do not exercise the modified logic. After these automated checks, we manually inspect the generated workloads as a final quality-assurance step. The inspection focuses on whether the workload inputs are meaningful for the program under measurement, whether the workloads exercise the computation affected by the refactoring, and whether the input-size levels and any additional workload characteristics are consistent with the program’s input type and modified logic. If the inspection identifies an obvious mismatch, we ask the LLM to regenerate the workloads with diagnostic information and then rerun the automatic validation checks. Items that cannot pass both the automatic validation checks and the final manual inspection are excluded. Only the remaining items are included as executable Micro-benchmark items. The final Micro-benchmark contains executable Java benchmark items covering 68 refactoring types. Each item consists of a before-refactoring program, an after-refactoring program, and generated workloads that support both aggregate execution over all input-size levels and separate execution for individual input-size levels. This benchmark enables controlled before/after energy comparison for individual refactoring types and supports the input-size-aware analysis used in RQ1 and RQ2. 3.2.2 Practical-benchmark. The refactoring examples in Micro-benchmark are intentionally small and controlled. This design helps isolate individual refactoring types, but it does not capture the complexity of refactoring in real software evolution. In practice, developers may apply multiple refactoring operations in the same commit, and the surrounding context, dependencies, and test workload may all influence the observed energy behavior. To complement the controlled setting of Micro-benchmark, we construct Practical-benchmark, a benchmark of real-world Java refactoring commits collected from GitHub repositories. Candidate commit collection. We collect candidate refactoring commits from existing refactoring datasets and empirical studies [36, 44, 61, 64]. For each candidate commit, we retrieve its parent commit and construct a before/after pair, where the parent commit is treated as the before-refactoring version and the candidate commit is treated as the after-refactoring version. We exclude merge commits because they have multiple parents and do not provide a clean one-to-one comparison between the before and after versions. Precondition checking. We apply several precondition checks to ensure that each commit pair can support controlled energy measurement. First, we restrict the benchmark to Maven-based Java projects and require the project to contain a root-level pom.xml file. Second, we require both the before and after versions to compile successfully. Third, we exclude commits that modify test files, because changes to tests may change the test oracle used to compare the two versions. We then check whether the project tests exercise the modified code. We run the project test suite with coverage collection and compute the intersection between covered lines and the executable lines changed by the commit. If the tests do not cover the modified code, the commit is excluded because its energy impact would not be observable under the test workload. Finally, we require the before and after versions to pass the project tests under the same test suite. These checks ensure that the two versions can be built, tested, and measured under comparable conditions. Manual refactoring purity inspection. After the automated precondition checking, we manually inspect the remaining candidate commits to further reduce the risk of including tangled or behavior-changing commits [5, 57]. This step follows prior refactoring studies, which use manual inspection of commit messages and code diffs to identify pure Manuscript submitted to ACM

14

Haibo Wang, Heng Li, and Shin Hwei Tan

refactoring changes and remove commits containing non-refactoring edits [44, 59]. In our study, two annotators inspected 543 candidate commits that passed the automated checks. For each candidate commit, the annotators inspected the commit message and the code diff between the before and after versions. Following prior studies [44], the goal was to check whether the code changes consisted only of behavior-preserving restructuring rather than functional edits. A commit was classified as a pure refactoring commit when its changes only involved refactoring operations, such as extracting, moving, or inlining. A commit was excluded when it introduced new functionality, fixed a bug, or changed program logic. To improve consistency, the two annotators first inspected the commits independently and then discussed disagreements until reaching an agreement. The initial annotations differed for 23 commits, accounting for 4.2% of the inspected candidates. The two annotators then held a discussion meeting to review these inconsistent cases and reached an agreement on all of them. After the manual inspection, we excluded 62 commits that were not pure refactoring commits. The remaining 481 commits form the final Practical-benchmark. The manual inspection took approximately 120 person-hours. After applying the collection, precondition-checking, and manual-inspection steps, Practical-benchmark contains real-world refactoring commits whose before and after versions are compilable, whose tests are unchanged, whose modified code is exercised by the test workload, and whose code changes are manually checked as pure refactoring changes. Unlike Micro-benchmark, an item in Practical-benchmark may contain multiple refactoring operations and may modify multiple files. This is intentional: the benchmark is designed to study the energy impact of refactoring as it appears in real software evolution. The final Practical-benchmark contains 481 refactoring commits from 430 GitHub projects and supports the commit-level analyses used in RQ1–RQ4.

3.3

Energy Measurement Procedure

After constructing Micro-benchmark and Practical-benchmark, we measure the energy consumption of each benchmark item before and after refactoring. We use JoularJX [43] to measure the energy consumption. JoularJX is a Java-based software power-monitoring tool that reports energy consumption. It supports multiple operating systems and builds on platform-specific power interfaces, such as RAPL on Linux and powermetrics on macOS. Following established energy-measurement guidelines [17, 40], we adopt several strategies to make the measurements reliable. We run the experiments on a dedicated machine, avoid unrelated workloads, and keep the runtime configuration fixed across measurements. Before collecting measurements, we perform a warm-up run. Between consecutive measurements, we insert a cool-down interval. The before- and after-refactoring versions of each benchmark item are executed under identical workloads and the same number of repetitions, ensuring that observed energy differences reflect refactoringinduced changes rather than variations in executed workloads. Each version is measured repeatedly to mitigate runtime variability and obtain sufficient observations for statistical analysis. In particular, we collect 30 independent energy measurements for each version, following recommendations that repeated measurements are necessary for reliable energy comparison [17]. We also randomize the measurement order of the before and after versions within each pair, so that one version is not systematically measured earlier or later than the other. The measurement process is automated to ensure that repeated executions follow the same procedure.

3.3.1 Micro-benchmark Energy Measurement. For Micro-benchmark, each benchmark item contains a before-refactoring program, an after-refactoring program, and generated workloads. The generated workloads support two execution modes. The first mode is aggregate execution, where the full workload suite across all input-size levels is executed Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

15

together. The second mode is per-size execution, where each input-size level is executed separately. This design allows us to measure both the overall energy behavior of a refactoring item and its behavior under individual input-size levels. For each workload group, the measurement pipeline first determines the repetition count required to ensure that a single measurement run is sufficiently long for reliable energy estimation [15, 16]. Since refactoring may alter execution speed, independently selecting repetition counts for the before and after versions could result in different amounts of executed work and introduce measurement bias. To ensure a fair comparison, we determine the repetition count for each version separately and use the maximum value as the common repetition count for both versions. Consequently, the pre- and post-refactoring versions execute identical generated workloads with the same number of repetitions, allowing measured energy differences to be attributed to refactoring-related changes rather than differences in execution volume. After the repetition count is determined, we repeatedly measure the energy consumption of the before and after versions. For each workload group, we collect 30 measurements for the before version and 30 measurements for the after version. These repeated measurements form the paired before/after samples used later for statistical comparison. 3.3.2 Practical-benchmark Energy Measurement. For Practical-benchmark, each benchmark item is a real-world refactoring commit. The parent commit is measured as the before-refactoring version, and the refactoring commit is measured as the after-refactoring version. Unlike Micro-benchmark, where workloads are generated, Practical-benchmark uses the project’s existing test suite as the measurement workload, following prior empirical studies that use tests as practical workloads for measuring software runtime or energy behavior [25, 54, 59]. For each refactoring commit, the measurement pipeline first prepares the before and after versions, including checking out and building each version. These preparation steps are completed before energy measurement starts. During measurement, we measure only the energy consumption of test-suite execution. We use the same test command for the before and after versions so that the measured energy difference is not caused by different test workloads. The resulting measurements are used in the later experimental analysis. For Practical-benchmark, these measurements allow us to study refactoring energy impact at commit level, where a commit may contain multiple refactoring operations and modify multiple files. 3.4

Metric Extraction

Energy measurements indicate whether the after-refactoring version consumes more or less energy than the beforerefactoring version, but they do not by themselves explain why the difference occurs. To support the explanatory analysis in RQ3 and the regression-identification analysis in RQ4, we extract two groups of complementary metrics. The first group captures runtime behavior, such as execution time, memory allocation, and garbage collection. The second group includes source-level, code-change, coverage, refactoring, and semantic metrics. Together, these metrics describe the runtime behavior, program structure, the amount and distribution of code modifications, how the modified code is exercised, which refactoring operations are applied, and the semantic content of the change. We extract different metrics according to the roles of the two benchmarks. For Micro-benchmark, we collect runtime-profile metrics for the controlled analysis in RQ3. For Practical-benchmark, we collect execution time together with the other types of metrics for RQ3 and RQ4. Table 5 summarizes the metric families used in our study. 3.4.1 Runtime-profile Metrics. Runtime-profile metrics describe how the before and after versions behave at runtime [8, 9]. We extract these metrics because refactoring may preserve program output while changing execution behavior, such as running time, memory allocation, garbage collection, or other JVM runtime behavior. These runtime changes can help explain why a refactoring leads to an energy increase or decrease. Manuscript submitted to ACM

16

Haibo Wang, Heng Li, and Shin Hwei Tan Table 5. Overview of extracted metric families. Category

Runtime-profile metrics [8, 9]

Source-level metrics [13, 18, 67]

Code change metrics [18, 25]

Coverage metrics [11, 59] Refactoring metrics [54, 55, 59] Semantic metrics [6]

Metrics

Description

Execution-time Allocation and memory metrics Garbage-collection metrics Other runtime metrics CK metrics Complexity metrics Performance-related static metrics Code churn Entropy Lines added Lines deleted Changed hunks Changed dirs Changed files Changed subsystems Line coverage Applied refactoring types and counts Refactoring operations total Distinct refactoring types Code diff

Time spent executing the workload. Object-allocation activity and memory-usage observations. Garbage-collection events and pause time. Other runtime-profile observations. Class-level code metrics obtained by static analysis. Cyclomatic complexity and cognitive complexity. Static code metrics related to performance-sensitive constructs. Sum of lines added and deleted. Distribution of modified code across files. Total number of lines added in a commit. Total number of lines deleted in a commit. Number of changed code hunks. Number of modified directories. Number of modified files. Number of modified subsystems. Line coverage ratio for modified files and lines. Set of detected refactoring types and the count of each type. Total number of detected refactoring operations. Number of distinct refactoring types. Code diff for a commit.

We extract runtime-profile metrics in separate profiling runs rather than during energy measurement. This separation is important because profiling can introduce additional overhead and may interfere with energy measurement. Therefore, energy measurement and runtime profiling are conducted as separate steps. The profiling step uses the same versions and workload settings as the energy measurement step, but its purpose is to collect runtime-behavior evidence rather than energy values. We focus on representative runtime-profile metrics, including execution-time, heap-object allocation event, young-generation garbage-collection event, total time spent in garbage-collection pauses, and the number of recorded CPU-load observations. The concrete profiling tools are described later in the experimental setup. For each runtime-profile metric, we compare the after-refactoring value with the before-refactoring value. These runtime-profile metrics are not used as replacements for energy measurement. Instead, they provide complementary evidence for interpreting the measured energy differences. For example, if a refactoring increases both energy consumption and allocation activity, this may suggest that additional object creation contributes to the observed energy increase. Similarly, changes in garbage collection or other runtime behavior can help explain energy differences that are not fully reflected by execution time alone. 3.4.2 Source-level, Change, Coverage, Refactoring, and Semantic Metrics. In addition to runtime-profile metrics, we extract metrics that describe the code structure, code change, test coverage, applied refactorings, and semantic content of the change. These metrics are especially important for Practical-benchmark, where each item is a real-world commit and may contain multiple refactoring operations. Previous work on commit-level performance regression prediction and performance-aware testing has shown that combining signals from multiple perspectives can improve risk identification compared with using a single metric family alone [13, 18, 67]. Source-level metrics. Source-level metrics characterize the structure of the code before and after refactoring. We extract CK metrics, code complexity, and performance-related static metrics [13, 18, 67]. CK metrics capture class-level structure, such as coupling and cohesion. Complexity metrics approximate control-flow complexity. Performance-related static metrics capture static constructs that are commonly associated with performance sensitivity [67]. These metrics help describe whether a refactoring changes structural properties that may affect runtime behavior. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

17

Code change metrics. Code change metrics describe how much code is changed and how the change is distributed across the project. We extract metrics such as code churn, change entropy, lines added, lines deleted, changed hunks, and changed files [18, 25]. These metrics are widely used in defect and regression prediction because larger changes and changes scattered across multiple files or modules can be harder to reason about and may be more likely to introduce unintended effects. In our study, these metrics help characterize whether refactoring-related energy differences are associated with the amount and distribution of code modifications. Coverage metrics. Coverage metrics describe whether the changed code is exercised by the measurement workload. We compute line coverage for modified files and modified lines using JaCoCo [26]. These metrics are useful because energy differences can only be observed when the workload executes the relevant changed code. For Practical-benchmark, coverage metrics indicate what fraction of the modified code is executed by the project test suite, which helps interpret whether the measured energy behavior is likely to reflect the refactoring change. Refactoring metrics. Refactoring metrics explicitly describe which refactoring operations appear in a commit. For each commit in Practical-benchmark, we extract the set of applied refactoring types together with the count of each type [54, 55, 59]. From these counts, we derive the total number of detected refactoring operations and the number of distinct refactoring types. These metrics provide refactoring-specific information that is not captured by generic sourcelevel or change metrics. They are used to analyze whether certain refactoring types or combinations of refactoring types are more associated with energy increases or decreases. Semantic metrics. Count-based metrics cannot fully capture what a code change does. To capture the semantic content of the refactoring edits, we extract the code diff for each refactoring commit [6]. This provides a summary of the transformation content beyond surface statistics, helping distinguish commits that have similar churn or refactoring counts but perform different kinds of changes. Overall, the extracted metrics provide complementary views of each refactoring change. Runtime-profile metrics describe how the program behaves at runtime, source-level metrics describe the structure of the code, code change metrics describe the amount and distribution of code modifications, coverage metrics describe whether the changed code is executed, refactoring metrics describe the applied refactoring operations, and semantic metrics describe the content of the transformation. This multi-view design supports the explanatory analysis in RQ3 and the regression-identification analysis in RQ4. 4

Experiments

4.1

Experimental Setup

All experiments are conducted on a dedicated Mac mini server running macOS with an Apple M4 Pro processor and 24 GB of memory. We use JDK 25 for all benchmark execution. The same machine, operating system, Java runtime, workload, repetition setting, and measurement configuration are used for the before and after versions of each benchmark item. For Practical-benchmark, repository checkout and project build are completed before energy measurement starts; only workload execution is measured. Runtime-profile metrics are collected separately from energy measurements. This separation avoids introducing profiling overhead into the energy measurements. For runtime profiling, we use JMH together with Java Flight Recorder. The profiling runs use the same machine, JDK version, before/after versions, and workload settings as the corresponding energy measurements, but their outputs are used only to extract runtime-profile metrics for RQ3. Manuscript submitted to ACM

18

Haibo Wang, Heng Li, and Shin Hwei Tan

In addition to runtime-profile metrics, we extract the source-level, code change, coverage, refactoring, and semantic metrics described in Section 3.4.2. For source-level metrics, we use the CK tool [3] to extract CK metrics and complexity metrics, and we use the tools from prior work to extract performance-related static metrics [13, 18, 67]. Code change and semantic metrics are extracted from the GitHub REST API for each refactoring commit [22]. Coverage metrics are extracted from JaCoCo [26] coverage reports. Refactoring metrics are extracted using RefactoringMiner [60], a widely used tool for detecting refactoring operations. For each benchmark item, the pipeline stores the raw energy-measurement outputs, runtime-profiling outputs, and extracted metric data. The energy-measurement outputs are used to compute before/after energy differences and statistical results. The runtime-profiling outputs are parsed to extract runtime-profile metrics. The source-level, code change, coverage, refactoring, and semantic metric outputs are used in the RQ3 analysis and in the RQ4 regressionidentification experiments. The RQ-specific statistical tests, association analyses, and prediction settings are described in the following subsections.

4.2

RQ1: Overall Energy Impact of Refactoring

RQ1 investigates whether refactoring changes energy consumption. We answer this question by comparing the repeated energy measurements of the before-refactoring and after-refactoring versions in both Micro-benchmark and Practicalbenchmark. For each comparison, we use the paired before/after samples collected under the same workload and repetition setting, as described in Section 3.3. 𝑏𝑒 𝑓 𝑜𝑟𝑒

Let 𝐸𝑖

𝑎𝑓 𝑡𝑒𝑟

and 𝐸𝑖

denote the energy consumption of the before and after versions in the 𝑖-th paired measurement.

We compute the paired energy difference as: Δ𝐸𝑖 = 𝐸𝑖

𝑎𝑓 𝑡𝑒𝑟

𝑏𝑒 𝑓 𝑜𝑟𝑒

− 𝐸𝑖

.

(1)

× 100,

(2)

We also compute the relative energy change ratio as: 𝑎𝑓 𝑡𝑒𝑟

Δ𝐸 (%) =

𝐸

−𝐸

𝑏𝑒 𝑓 𝑜𝑟𝑒

𝑏𝑒 𝑓 𝑜𝑟𝑒

𝐸 𝑏𝑒 𝑓 𝑜𝑟𝑒

where 𝐸

𝑎𝑓 𝑡𝑒𝑟

and 𝐸

are the mean energy consumption values across repeated measurements.

To determine whether the before/after energy difference is statistically significant, we apply the paired Wilcoxon signed-rank test [51]. We use this non-parametric test because the measurements are paired by construction and energy measurements may not follow a normal distribution. We use 𝛼 = 0.05 as the significance threshold. In addition to 𝑝-values, we report Cliff’s delta [38] as an effect-size measure. Reporting effect size helps distinguish statistically significant differences from changes with limited practical magnitude. For each comparison, we classify the measured energy impact into three categories. If the Wilcoxon signed-rank test is statistically significant and the mean paired difference is positive, the comparison is classified as an energy increase. If the test is statistically significant and the mean paired difference is negative, the comparison is classified as an energy decrease. Otherwise, the comparison is classified as unchanged. This classification summarizes the direction of the measured effect, while the absolute energy difference, relative change ratio, and effect size report its magnitude. The unit of comparison differs between the two benchmarks. For Micro-benchmark, we perform the analysis for the aggregate workload group and for each individual input-size group, which allows us to examine both overall and input-size-specific energy changes. For Practical-benchmark, we perform the analysis at commit level using the project Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

19

test suite as the workload. The resulting energy-impact categories, effect sizes, and change ratios are used in Section 5.1 to report the overall energy impact of refactoring across the two benchmarks. 4.3

RQ2: Refactoring Types and Refactoring Patterns

RQ2 investigates whether the energy impact of refactoring is associated with particular refactoring types or refactoring patterns. While RQ1 focuses on whether refactoring changes energy consumption overall, RQ2 further examines how these changes are distributed across refactoring types in Micro-benchmark and across refactoring co-occurrence patterns in Practical-benchmark. For Micro-benchmark, each benchmark item is associated with one intended refactoring type. We therefore group the before/after comparisons by refactoring type. For each type, we count how many workload groups are classified as increase, decrease, or unchanged according to the RQ1 procedure. We also summarize the distribution of relative energy change ratios for each refactoring type. This analysis allows us to examine whether some refactoring types tend to be more energy-sensitive than others, and whether their effects are consistent or vary across workload groups. For Practical-benchmark, the analysis is different because a real-world refactoring commit may contain multiple refactoring operations. We use the refactoring metrics extracted by RefactoringMiner to characterize the refactoring operations in each commit, including the detected refactoring types and their counts. We then analyze whether specific refactoring co-occurrence patterns are associated with statistically significant energy increases or decreases. A cooccurrence pattern is defined as a set of refactoring types that appear together in the same commit. We consider refactoring patterns of size two and three, corresponding to pairs and triples of refactoring types. For each pattern 𝑆, we compute its support as the number of commits containing all refactoring types in 𝑆. For each target outcome 𝑦, where 𝑦 denotes either a statistically significant energy increase or decrease, we compute the base rate, confidence, and lift of each pattern following standard association-rule analysis [1, 27]. Let 𝑌 denote the energy-impact classification of a commit. The base rate 𝑃 (𝑌 = 𝑦) is the proportion of analyzed commits exhibiting outcome 𝑦. The confidence of a pattern 𝑆 for outcome 𝑦 is the probability that a commit exhibits 𝑦, given that it contains all refactoring types in 𝑆: conf (𝑆, 𝑦) = 𝑃 (𝑌 = 𝑦 | 𝑆).

(3)

The lift compares this confidence with the base rate of the target outcome: lift(𝑆, 𝑦) =

conf (𝑆, 𝑦) . 𝑃 (𝑌 = 𝑦)

(4)

A lift greater than 1 indicates that commits containing 𝑆 are more likely to exhibit outcome 𝑦 than an average commit. In addition to support, confidence, and lift, we report the odds ratio and the 𝜙 coefficient as effect-size measures for the association between the occurrence of 𝑆 and the outcome 𝑌 = 𝑦 [2]. We assess each association using Fisher’s exact test [2] and apply the Benjamini–Hochberg false-discovery-rate correction for multiple testing [7]. The correction is applied separately for each pattern size and target outcome. Because statistically significant energy increases and decreases are limited in number, we interpret the results as associative rather than causal evidence. 4.4

RQ3: Factors Associated with Energy Differences

RQ3 investigates which runtime-level and code-level factors are associated with the energy differences observed after refactoring. While RQ1 measures whether energy consumption changes and RQ2 analyzes how the changes are distributed across refactoring types and patterns, RQ3 further examines factors that help interpret these differences. We Manuscript submitted to ACM

20

Haibo Wang, Heng Li, and Shin Hwei Tan

use the energy change ratios computed in RQ1 together with the runtime-profile, source-level, code change, coverage, refactoring, and semantic metrics described in Section 3.4. These metric families follow prior studies showing that runtime behavior, source-level properties, change characteristics, coverage, and semantic representations can provide useful signals for performance or regression analysis [8, 9, 13, 18, 67]. For runtime-profile metrics, we compare the before-refactoring and after-refactoring values of each metric under the same workload setting. Let 𝑋 𝑏𝑒 𝑓 𝑜𝑟𝑒 and 𝑋 𝑎𝑓 𝑡𝑒𝑟 denote the values of a runtime-profile metric before and after refactoring. We compute the relative change ratio of the metric as: 𝑎𝑓 𝑡𝑒𝑟

Δ𝑋 (%) =

𝑋

−𝑋

𝑏𝑒 𝑓 𝑜𝑟𝑒

× 100,

(5)

𝑏𝑒 𝑓 𝑜𝑟𝑒

𝑋 𝑏𝑒 𝑓 𝑜𝑟𝑒

where 𝑋

𝑎𝑓 𝑡𝑒𝑟

and 𝑋

are the mean metric values across profiling runs. When the before-refactoring value is zero,

we use the absolute difference instead of the relative change ratio to avoid undefined ratios. These metric changes allow us to examine whether energy changes are accompanied by changes in execution time, allocation behavior, garbage collection, or other runtime observations. Runtime-profile analysis is particularly relevant for Java programs because JVM execution behavior, memory allocation, garbage collection, and runtime optimization can affect measured performance and energy behavior [8, 9]. We first analyze the association between energy change and runtime-profile changes in Micro-benchmark. For each benchmark item and workload setting, we pair the energy change ratio with the corresponding runtime-profile change ratios. We then compute the Spearman correlation coefficient between the energy change and each runtime-profile metric change [24]. We also analyze the association between energy changes and non-runtime metrics in Practical-benchmark. For source-level metrics, we compare the before- and after-refactoring metric values and compute their changes. For code change, coverage, refactoring, and semantic metrics, we use the extracted commit-level values described in Section 3.4.2. We examine how these metrics differ across the energy-increase, energy-decrease, and unchanged groups defined in RQ1. This analysis follows prior work showing that static code metrics, change metrics, test-related metrics, and learned representations can help characterize performance regressions and related software-quality risks [13, 18, 67]. In addition, we conduct a JVM execution-mode sensitivity analysis for Micro-benchmark. Since Java programs can be affected by just-in-time compilation, comparing results under different execution modes can help assess whether the observed refactoring energy impact is sensitive to JVM optimization behavior [8, 9]. We compare the default JVM execution mode with a JIT-disabled execution mode using -Xint. This analysis is not used as the primary energy result; instead, it examines whether the measured refactoring energy impact changes when JIT compilation is disabled. For each refactoring item, we compare the before energy, after energy, and refactoring energy change ratio under the two execution modes. We also rank refactoring types by the difference in energy change ratio between the two modes to identify refactoring types whose estimated energy impact is more sensitive to JIT behavior.

4.5

RQ4: Refactoring-induced Energy Regression Identification

RQ4 investigates whether refactoring-induced energy regressions can be identified using commit-level metrics. While RQ1–RQ3 measure and interpret energy differences, RQ4 treats energy regression identification as a binary classification problem on Practical-benchmark. We focus on Practical-benchmark because it represents real-world refactoring commits, where developers may need to decide whether a refactoring commit should be prioritized for energy validation. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

21

4.5.1 Ground-truth Labels. We derive the ground-truth labels from the RQ1 statistical analysis. A refactoring commit is labeled as Reg if its after-refactoring version shows a statistically significant energy increase compared with its before-refactoring version. Specifically, a commit is labeled as Reg when the paired Wilcoxon signed-rank test [51] is significant at 𝛼 = 0.05 and the mean paired energy difference is positive. Commits that do not satisfy this condition are labeled as Not Reg. Therefore, the Not Reg class includes commits with no statistically significant energy change and commits with statistically significant energy decreases. This labeling follows the goal of RQ4: identifying refactoring commits that may introduce energy regressions. We treat Reg as the positive class and report precision, recall, and F1 for this target class. 4.5.2 Metric-based Prediction Models. We first evaluate metric-based prediction models from prior performance regression studies [18, 67]: Logistic Regression, Random Forest, and XGBoost. Logistic Regression provides a linear baseline [32], Random Forest captures non-linear feature interactions through an ensemble of decision trees [12], and XGBoost provides a gradient-boosted tree model that is widely used for tabular prediction tasks [14]. Each refactoring commit is represented using the extracted metric families described in Section 3.4, including source-level, code change, coverage, refactoring, and semantic metrics, where semantic metric is represented by a 1,024-dimensional code-diff embedding generated by calling the OpenAI text-embedding-3-large model through the OpenAI API. These metrics provide different views of the refactoring commit, such as code structure, change size and distribution, test coverage, detected refactoring operations, and the semantic content of the diff. To reduce the influence of different feature scales, numeric features are standardized before training. We use ten-fold cross-validation and model-specific class weighting to address class imbalance [18, 67]. All metric-based models are trained and evaluated using the same data split, so their results are comparable. The models are trained on the extracted metrics and evaluated on their ability to identify Reg commits. We use the metric-based models as baselines to assess whether existing commit-level signals are sufficient for identifying refactoring-induced energy regressions 4.5.3 LLM-based Predictor. In addition to metric-based models, we evaluate an LLM-based predictor. The motivation is that LLMs may capture relationships among heterogeneous signals that are difficult to model with standard tabular classifiers. For each refactoring commit, we construct a structured prompt containing the available metric families and ask the model to classify the commit as Reg or Not Reg. The prompt does not include the measured energy values or the ground-truth label. We use gpt-5-mini through the OpenAI API [45] as the LLM-based classifier. We further conduct an ablation study to examine the contribution of different metric families. We organize the prompt information into five groups: static code metrics, code change metrics, test-related metrics, refactoring metrics, and semantic metrics. We first evaluate the model with all metric groups included. Then, we remove one metric group at a time and re-run the classifier. This leave-one-group-out design allows us to observe whether removing a particular metric family reduces the model’s ability to identify energy regressions. The LLM-based predictor is not fine-tuned on our dataset. Instead, it is evaluated as a prompt-based classifier. This design reflects a lightweight use case where developers or researchers may query an LLM with structured information about a refactoring commit without training a project-specific prediction model. 4.5.4 Evaluation Metrics. We evaluate the classifiers using precision, recall, and F1 score, which are standard metrics for binary classification [58]. Since the goal is to identify energy-regression commits, we treat Reg as the positive class. Precision measures how many predicted regressions are true regressions, while recall measures how many true regressions are successfully identified. F1 summarizes the trade-off between precision and recall. For both approaches, Manuscript submitted to ACM

22

Haibo Wang, Heng Li, and Shin Hwei Tan

Modified-line coverage (%)

100 80 60 40 20 0

Mean Before

Program version

After

Fig. 6. Distributions of modified-line coverage before and after refactoring.

we report precision, recall, and F1 for the Reg class because the primary purpose is to identify energy regressions rather than recognize the majority Not Reg class. 5

Results

This section presents the empirical results organized by the four research questions. We first report the overall energy impact of refactoring (RQ1), then examine which refactoring types and patterns are associated with energy differences (RQ2). We next analyze runtime-level and code-level factors that help explain the observed differences (RQ3), and finally evaluate whether existing techniques can identify refactoring-induced energy regressions (RQ4). 5.1

RQ1: What is the impact of refactoring on energy consumption?

5.1.1

Energy Impact across Workload Groups in Micro-benchmark.

Modified-line coverage. To ensure that the measured energy differences are attributable to the refactored code rather than unexecuted changes, we first examine modified-line coverage, defined as the proportion of lines modified by a refactoring that are executed by the benchmark workload. High modified-line coverage increases confidence that the measured energy consumption reflects the behavior of the refactored code instead of unaffected portions of the program. The analysis begins with 71 candidate items. An item is included in RQ1 only if it has complete and valid energy measurements for the Aggregate workload and all five input-size workloads. Under this criterion, 64 items (90.1%) are retained, while seven are excluded because the energy-measurement stage could not be completed. The resulting analysis comprises 384 item–workload comparisons (64 items × six workload groups) and 11,520 matched before/after measurement pairs. Figure 6 summarizes the modified-line coverage of the eligible items. Coverage is highly concentrated at 100% for both the pre- and post-refactoring versions. Consequently, the median and interquartile range coincide at the upper bound, causing the overlaid boxplots to collapse into nearly indistinguishable horizontal lines. Mean coverage is 95.97% before refactoring and 93.59% after refactoring, while the median remains 100% for both versions. These results indicate that the benchmark workloads exercise nearly all refactored code, reducing the likelihood that the observed energy differences are attributable to insufficient coverage. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

23

Table 6. Energy-change classifications by workload group.

Workload

Unchanged, 𝑛 (%)

Significant, 𝑛 (%)

Increase, 𝑛 (%)

Decrease, 𝑛 (%)

Aggregate XS S M L XL

34 (53.1) 32 (50.0) 29 (45.3) 31 (48.4) 29 (45.3) 30 (46.9)

30 (46.9) 32 (50.0) 35 (54.7) 33 (51.6) 35 (54.7) 34 (53.1)

16 (25.0) 18 (28.1) 18 (28.1) 22 (34.4) 22 (34.4) 17 (26.6)

14 (21.9) 14 (21.9) 17 (26.6) 11 (17.2) 13 (20.3) 17 (26.6)

Frequency and direction of energy changes. Table 6 summarizes how frequently each energy-impact classification occurs under each workload. A comparison is statistically significant when its paired Wilcoxon signed-rank test has 𝑝 < 0.05. Significant comparisons are classified as increases or decreases using the sign of the mean paired difference; therefore, the Significant column is the sum of Increase and Decrease. Unchanged denotes the absence of a statistically significant difference rather than an observed difference of exactly zero. Each workload group contains 64 items, and all percentages use this number as the denominator. Across the 384 item–workload comparisons, 199 (51.8%) exhibit a significant energy change, while 185 (48.2%) are unchanged. The significant comparisons comprise 113 increases (29.4%) and 86 decreases (22.4%). The proportion of significant comparisons ranges from 46.9% for Aggregate to 54.7% for S and L, and every workload contains both significant increases and significant decreases. Among the significant comparisons, no workload contains significantly more increases than decreases, or vice versa, according to two-sided exact binomial tests with Holm correction (𝑝 Holm ≥ 0.481). Finding 1 The benchmark workloads achieve consistently high modified-line coverage, indicating that the observed energy differences are attributable to the refactored code rather than unexecuted modifications. Across the covered refactoring instances, 51.8% of refactoring instance-workload comparisons exhibit statistically significant energy changes. Both energy increases and decreases occur under every workload, with neither direction predominating. Thus, although refactoring preserves functional behaviour, it often impact energy consumption, and its impact cannot be assumed to be energy-neutral or consistently beneficial.

Magnitude and effect sizes of energy changes. Figure 7 presents boxplots overlaid with item-level relative energy changes for each workload group. We use absolute relative changes of 10%, 50%, and 100% as descriptive magnitude thresholds. Of the 384 item–workload comparisons, 107 (27.9%) change by at least 10%. All 107 are statistically significant and span 26 of the 64 eligible items. Their directional distributions differ across workloads: Aggregate is balanced between increases and decreases (9/9), and XL is nearly balanced (9/8), whereas M (13/5) and L (13/6) are weighted toward increases. This directional imbalance strengthens at larger magnitudes: increases account for 34 of the 42 comparisons that change by at least 50%. Moreover, 21 comparisons spanning five refactoring items and all six workload groups increase mean energy consumption by at least 100%. Thus, substantial changes occur under every workload, but energy increases dominate the higher-magnitude portion of the distributions. The effect-size results provide complementary evidence (Table 7). All 199 statistically significant comparisons have non-negligible Cliff’s-𝛿 values: 17 (8.5%) are small, 20 (10.1%) are medium, and 162 (81.4%) are large. In contrast, 129 of the 185 non-significant comparisons (69.7%) have negligible effects, and only three (1.6%) have medium or large effects. Manuscript submitted to ACM

24

Haibo Wang, Heng Li, and Shin Hwei Tan

Relative energy change (%)

102 10 0 -10 102

Aggregate

XS

S M Workload group

L

XL

Fig. 7. Item-level relative energy changes across workload groups.

Table 7. Statistical significance and Cliff’s-𝛿 magnitude across the 384 item–workload comparisons.

Statistical result

Negligible

Small

Medium

Large

Significant (𝑛 = 199) Not significant (𝑛 = 185)

0 (0.0%) 129 (69.7%)

17 (8.5%) 53 (28.6%)

20 (10.1%) 2 (1.1%)

162 (81.4%) 1 (0.5%)

Therefore, the statistically significant energy differences are generally accompanied by substantial separation between the before and after measurements. Finding 2 Energy changes induced by refactoring are often substantial rather than marginal. More than one-quarter (27.9%) of item–workload comparisons change energy consumption by at least 10%, and these changes are consistently statistically significant with predominantly large effect sizes. Although both energy increases and decreases occur, larger-magnitude changes are increasingly dominated by energy increases, with all observed changes of at least 100% corresponding to increased energy consumption. These results indicate that refactoring can produce practically meaningful energy effects and that substantial energy regressions, while not universal, represent a non-negligible risk.

Within-item workload sensitivity. To directly examine whether workload choice changes the inferred energy impact of the same refactoring item, we compare each eligible item’s increase, decrease, or unchanged classification across the five input-size workloads (XS–XL). An item is considered workload-sensitive when it does not receive the same classification under all five workloads. Under this definition, 29 of the 64 eligible items (45.3%) are workload-sensitive. Nineteen items vary only between a statistically significant change and an unchanged classification, without switching between increase and decrease. More importantly, ten items (15.6%) are classified as significant increases under at least one input size and significant decreases under another. We also calculate, for each item, the range between its maximum and minimum relative energy changes across XS–XL. Twenty-one items (32.8%) have a range of at least 10 percentage points. These within-item differences demonstrate that Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

25

Table 8. Population and commit-composition statistics for Practical-benchmark.

Characteristic Energy-evaluable commits GitHub projects Measurement pairs per commit Commits with RefactoringMiner output Commits with at least one detected refactoring Detected refactoring per analyzed commit Distinct detected refactoring per analyzed commit Commits modifying multiple production Java files

Value 481 430 30 479 345 Mean 10; median 3 Mean 3; median 2 278 (57.8%)

measuring a refactoring under only one workload may fail to detect an effect observed under another workload or may produce the opposite directional conclusion. Finding 3 Workload choice affects the inferred energy impact of individual refactorings. Across XS–XL, 29 of the 64 eligible items (45.3%) change their increase, decrease, or unchanged classification. This includes ten items (15.6%) that switch between significant energy increases and decreases. Moreover, 21 items (32.8%) exhibit a relative-energy-change range of at least 10 percentage points. Thus, a single workload may miss or even reverse the observed energy effect of a refactoring.

Overall, the Micro-benchmark results show that refactoring can produce statistically reliable energy increases and decreases across all workload groups. Although changes occur in both directions, energy increases become increasingly prominent among the largest observed differences. The within-item comparisons further show that workload selection can change whether the same refactoring item is classified as an increase, decrease, or unchanged. These controlled results demonstrate both the potential energy impact of individual refactorings and the importance of measuring them under diverse workloads; we next examine whether energy differences also arise in practical refactoring commits. 5.1.2

Energy Impact in Practical-benchmark.

Analysis population and commit composition. The RQ1 analysis covers 481 unique refactoring commits from 430 GitHub projects. For each commit, we evaluate the energy impact using 30 matched before/after energy measurements, yielding 14,430 matched measurement pairs (481 commits × 30 measurements). Table 8 summarizes the analysis population and the composition of the commits. The composition of these commits differs substantially from the isolated transformations in Micro-benchmark. Among the 479 commits with RefactoringMiner output, 56 (11.7%) contain exactly one detected refactoring operation, whereas 289 (60.3%) contain multiple operations. Similarly, 89 (18.6%) contain exactly one distinct refactoring type, whereas 256 (53.4%) contain multiple distinct types. RefactoringMiner records no supported operation for the remaining 134 commits (28.0%). Among the 345 commits with at least one detected refactoring operation, 83.8% contain multiple operations and 74.2% contain multiple distinct types. Together with the 278 commits that modify multiple production Java files, these results show that commit-level measurements frequently capture bundles of transformations rather than a single isolated refactoring. Manuscript submitted to ACM

26

Haibo Wang, Heng Li, and Shin Hwei Tan Table 9. Commit-level energy-change classifications in Practical-benchmark.

Classification

𝑛

%

Significant increase Unchanged Significant decrease

15 445 21

3.1 92.5 4.4

Total

481

100.0

Mean relative energy change (%)

300 100 50 20 10

0

10 20 50 100

Significant decrease

No significant difference

Significant increase

Fig. 8. Distributions of commit-level mean relative energy changes by statistical classification. Dots represent individual commits, and boxes summarize medians and quartiles. The vertical axis uses a symmetric logarithmic scale with a linear region from −10% to 10%.

Frequency and direction of energy changes. As in the Micro-benchmark analysis, a commit is statistically significant when its paired Wilcoxon signed-rank test has 𝑝 < 0.05. A significant commit is classified as an increase or decrease using the sign of its mean paired difference. Unchanged denotes the absence of a statistically significant difference under the executed project test-suite workload rather than equality between the before and after measurements. Table 9 summarizes the commit-level classifications. Thirty-six of the 481 commits (7.5%) exhibit a statistically significant energy difference. Fifteen commits (3.1%) increase energy consumption, 21 (4.4%) decrease it, and 445 (92.5%) are unchanged. The 36 significant commits come from 36 different projects, so the result is not driven by multiple significant commits from one project. Among these commits, increases account for 41.7% and decreases for 58.3%; a two-sided exact binomial test finds no directional imbalance (𝑝 = 0.405). Finding 4 Real-world refactoring commits typically comprise multiple co-occurring refactoring operations and often modify multiple source files, reflecting the complexity of practical software evolution. Under these realistic conditions, statistically significant energy changes are relatively uncommon, occurring in only 7.5% of commits. When they do occur, both energy increases and decreases are observed, with neither direction predominating. These results suggest that most real-world refactoring commits are effectively energy-neutral under the evaluated workloads, but a non-negligible subset can still produce measurable energy changes.

Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

27

Table 10. Statistical significance and Cliff’s-𝛿 magnitude across the 481 refactoring commits.

Statistical result

Negligible

Small

Medium

Large

Significant (𝑛 = 36) Unchanged (𝑛 = 445)

6 (16.7%) 361 (81.1%)

16 (44.4%) 82 (18.4%)

10 (27.8%) 2 (0.4%)

4 (11.1%) 0 (0.0%)

Magnitude and effect sizes of energy changes. Figure 8 shows the distribution of mean relative energy changes. Across all 481 commits, the median change is −0.25% (first quartile: −6.35%; third quartile: 9.73%), with values ranging from −95.21% to 301.89%. The wide range among unchanged commits also shows why the difference between sample means should be considered together with repeated-measurement statistics. Among the 36 statistically significant commits, the median increase is 16.61%, while the median decrease is −17.52%. Using the same descriptive magnitude thresholds as in the Micro-benchmark analysis, 24 of the 36 commits (66.7%) change by at least 10% (11 increases and 13 decreases), five (13.9%) change by at least 50% (three increases and two decreases), and one (2.8%) increases by at least 100%. Table 10 summarizes the conventional Cliff’s-𝛿 effect sizes. Thirty of the 36 significant commits (83.3%) have nonnegligible effects: 16 are small, ten are medium, and four are large. The remaining six have negligible effects. Medium or large effects are strongly concentrated among statistically significant commits: 14 of 36 significant commits (38.9%), compared with two of 445 unchanged commits (0.4%). These results show that many of the detected energy differences are supported by both substantial relative changes. Finding 5 The statistically significant commit-level energy differences vary in magnitude, with most commits exhibiting non-negligible effect sizes. Among the 36 significant commits, 24 (66.7%) change mean energy consumption by at least 10%, including five (13.9%) with changes of at least 50% and one (2.8%) with an increase of more than 100%. Regarding effect size, 30 commits (83.3%) have non-negligible Cliff’s-𝛿 values, including 14 (38.9%) with medium or large effects.

Overall, the Practical-benchmark results extend the controlled evidence from Micro-benchmark to complete refactoring commits drawn from real projects. Although most commits are classified as unchanged under their test-suite workloads, statistically significant increases and decreases occur and can be substantial. Because these commits frequently combine multiple operations, distinct refactoring types, and production files, their measured energy impact reflects the complete change rather than the isolated effect of any single operation. We next examine which refactoring types and co-occurrence patterns are associated with these energy differences. 5.2

RQ2: Which refactoring types are more likely to influence energy consumption?

RQ1 shows that refactoring can produce statistically significant increases and decreases in energy consumption. RQ2 examines whether these differences are concentrated among particular refactoring types and, when they occur, how large they are. We compare refactoring types along two complementary dimensions. The first is the observed frequency of statistically significant before/after differences under the paired Wilcoxon signed-rank criterion (𝑝 < 0.05) used in RQ1. The second is the magnitude of these differences: for evaluations that exhibit a statistically significant difference, we further analyze the direction and mean relative change in energy consumption, together with the Cliff’s-𝛿 effect size. The frequency dimension shows how recurrently an energy difference is observed for a type, whereas the relative Manuscript submitted to ACM

28

Haibo Wang, Heng Li, and Shin Hwei Tan

change and effect size characterize the extent of the detected difference. We therefore interpret these dimensions together when determining which refactoring types are more likely to influence energy consumption. In Micro-benchmark, each benchmark item isolates a target refactoring type, allowing its energy behavior to be examined in a controlled setting. For each type, we analyze the frequency and direction of statistically significant differences across the six workload groups—Aggregate, XS, S, M, L, and XL—and characterize their relative changes and Cliff’s-𝛿 effect sizes. Aggregate represents the combined workload, whereas XS–XL represent individual input-size workloads. We retain this distinction when interpreting the results, but examine all six groups together to provide a unified view of each refactoring type’s energy behavior across the evaluated workloads. In Practical-benchmark, the measured energy difference belongs to the complete commit. We therefore distinguish commits containing exactly one distinct detected refactoring type from those containing multiple distinct types. For single-type commits, we examine the frequency, direction, and relative-change magnitude of the commit-level differences according to the sole detected type. Although such a commit may contain multiple operations of that type, it provides more direct type-specific evidence because no other detected refactoring type co-occurs in the commit. For multi-type commits, we examine whether individual types and sufficiently supported co-occurrence patterns are more prevalent among commits with statistically significant increases or decreases than among unchanged commits. We also characterize the relative-change magnitudes of the corresponding significant commits. These multi-type results are interpreted as associations because the contribution of one type cannot be isolated from those of the other co-occurring types. Accordingly, Micro-benchmark provides controlled evidence for isolated types, single-type practical commits provide type-focused evidence in real-world contexts, and multi-type practical commits provide evidence about individual-type and co-occurrence associations. 5.2.1

Refactoring-Type Effects in Micro-benchmark.

Analysis population and type-level measures. We reuse the 64 Micro-benchmark items included in the RQ1 analysis, each of which has valid before/after energy measurements for Aggregate, XS, S, M, L, and XL. Four items target Extract Function/Method, while each of the remaining 60 items represents a different refactoring type; the analysis therefore covers 61 refactoring types. For each item, we conduct one before/after statistical comparison in each of the six workload groups, yielding 64 × 6 = 384 item–workload comparisons. Each comparison is based on 30 paired before/after measurements; these measurements support the statistical test but are not counted as separate item– workload comparisons. For each refactoring type, we calculate the observed frequency of statistically significant energy differences as the proportion of its item–workload comparisons that yield a significant result, reporting increases and decreases separately. The denominator is therefore six comparisons for each single-item type and 24 comparisons for Extract Function/Method. We use mean relative energy change and Cliff’s-𝛿 to characterize the magnitude of the observed differences. Aggregate is a combined workload and XS–XL are individual input-size workloads, but all six are examined together as the tested workload groups. Frequency and direction across workload groups. Figure 9 ranks the 45 refactoring types that exhibit at least one statistically significant item–workload comparison by the proportion of their comparisons classified as significant increases or decreases. These 45 types constitute 73.8% of the 61 types; the other 16 types (26.2%) exhibit no significant difference and are omitted from the figure. Among the 45 types shown, 18 (40.0%) exhibit only increases, 16 (35.6%) exhibit only decreases, and 11 (24.4%) exhibit both directions. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring Significant decrease

29 Significant increase

(a) Ranks 1-23

(b) Ranks 24-45

Replace Error Code with Exception

Introduce Parameter Object

Replace Type Code with Subclasses

Replace Constructor with Factory Function

Substitute Algorithm

Extract Superclass

Separate Query from Modifier

Slide Statements

Split Loop

Move Field

Collapse Hierarchy

Move Function

Add Parameter

Encapsulate Collection

Replace Loop with Pipeline

Replace Command with Function

Introduce Special Case

Replace Superclass with Delegate

Extract Class

Move Statements into Function

Move Statements to Callers Extract Function/Method (4 items) Parameterize Function Preserve Whole Object Change Value to Reference Consolidate Conditional Expression

Change Reference to Value

Replace Derived Variable with Query

Encapsulate Variable

Remove Parameter

Replace Subclass with Delegate

Replace Query with Parameter

Replace Inline Code with Function Call

Split Phase

Remove Subclass

Encapsulate Record

Remove Flag Argument Combine Functions into Transform

Inline Variable

Inline Class

Extract Variable

Replace Conditional with Polymorphism

Replace Nested Conditional with Guard Clauses

Return Modified Value

Pull Up Constructor Body

100

75

50

25

0

25

50

75

Significant comparisons within type (%)

100

100

75

50

25

0

25

50

75

100

Significant comparisons within type (%)

Fig. 9. Observed frequency and direction of statistically significant energy differences by refactoring type in Micro-benchmark. Only the 45 types with at least one significant item–workload comparison are shown. For each type, the bars show the percentages of its comparisons classified as significant increases or decreases across Aggregate, XS, S, M, L, and XL. Types are ordered by the sum of these percentages and split into two panels for readability.

However, only 22 of the 61 types (36.1%) exhibit a statistically significant energy difference across all six workload groups. Among these 22 types, 12 (54.5%) consistently increase energy consumption, including Replace Error Code with Exception, Replace Type Code with Subclasses, Substitute Algorithm, and Replace Loop with Pipeline. Six (27.3%) consistently decrease energy consumption, including Replace Inline Code with Function Call, Remove Subclass, and Remove Flag Argument. The remaining four (18.2%)—Change Value to Reference, Change Reference to Value, Encapsulate Variable, and Replace Subclass with Delegate—exhibit significant differences in all six workload groups but include both increases and decreases. Finding 6 Statistically significant energy differences occur for most refactoring types, but only a smaller subset exhibits such differences across all six workload groups. Of the 61 types, 45 (73.8%) exhibit at least one significant item–workload comparison, whereas 22 (36.1%) exhibit a significant difference across all six workload groups. Among these 22 types, 12 (54.5%) consistently increase energy consumption, six (27.3%) consistently decrease it, and four (18.2%) exhibit both directions. The other 16 types (26.2%) exhibit no significant difference.

Magnitude and workload dependence of refactoring-type effects. Figure 10 complements the frequency view by showing the mean relative energy change and Cliff’s-𝛿 magnitude for all 45 refactoring types with at least one statistically Manuscript submitted to ACM

30

Haibo Wang, Heng Li, and Shin Hwei Tan

Significant comparisons (p < 0.05); circle size: Cliff's |δ|: Aggregate

XS

Small

Medium

S

M

Large

L

XL

Replace Error Code with Exception Replace Type Code with Subclasses Substitute Algorithm Separate Query from Modifier Split Loop Collapse Hierarchy Add Parameter Replace Loop with Pipeline Introduce Special Case Extract Class Move Statements to Callers Parameterize Function Change Value to Reference Change Reference to Value Encapsulate Variable Replace Subclass with Delegate Replace Inline Code with Function Call Remove Subclass Remove Flag Argument Combine Functions into Transform Inline Class Replace Conditional with Polymorphism Return Modified Value Introduce Parameter Object Replace Constructor with Factory Function Extract Superclass Slide Statements Move Field Move Function Encapsulate Collection Replace Command with Function Replace Superclass with Delegate Move Statements into Function Extract Function/Method (function item) Extract Function/Method (no locals) Extract Function/Method (reassign local) Extract Function/Method (use locals) Preserve Whole Object Consolidate Conditional Expression Replace Derived Variable with Query Remove Parameter Replace Query with Parameter Split Phase Encapsulate Record Inline Variable Extract Variable Replace Nested Conditional with Guard Clauses Pull Up Constructor Body

-500

-50

-5

0

+5

+50

≥ + 500

Mean relative energy change (%)

Fig. 10. Mean relative energy change across Aggregate, XS, S, M, L, and XL for all 45 refactoring types with at least one statistically significant comparison in Micro-benchmark. The four Extract Function/Method items are displayed as separate rows, resulting in 48 benchmark-item rows. Red denotes higher energy after refactoring and blue denotes lower energy. Circles mark statistically significant comparisons, with their sizes representing small, medium, and large Cliff’s-𝛿 effects.

significant comparison. The four Extract Function/Method items are shown separately so that their within-type variation is not hidden by aggregation. Of the 384 item–workload comparisons defined above, 199 (51.8%) yield a statistically significant difference: 113 increases and 86 decreases. Among these 199 comparisons, 162 (81.4%) have a large Cliff’s-𝛿 effect, 20 (10.1%) have a medium effect, and 17 (8.5%) have a small effect; none has a negligible effect. The 22 types that differ across all six workload groups account for 132 of these comparisons. Within this subset, 124 (93.9%) have a large effect and the remaining eight (6.1%) have a medium effect. Table 11 reports the direction, median relative change, range, and number of large effects for these 22 types. The magnitude of all-workload differences varies substantially even within the directionally consistent groups. Across the 12 consistently increasing types, the median relative change ranges from +3.07% for Parameterize Function Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

31

Table 11. The 22 refactoring types with statistically significant differences across all six workload groups. Inc./dec. reports the numbers of increases and decreases among the six comparisons. Median and range summarize the six mean relative energy changes. Large |𝛿 | reports how many comparisons have a large Cliff’s-𝛿 effect.

Inc./dec.

Median Δ𝐸

Range of Δ𝐸

Consistent increases (12 types) Replace Error Code with Exception Replace Type Code with Subclasses Substitute Algorithm Separate Query from Modifier Split Loop Collapse Hierarchy Add Parameter Replace Loop with Pipeline Introduce Special Case Extract Class Move Statements to Callers Parameterize Function

6/0 6/0 6/0 6/0 6/0 6/0 6/0 6/0 6/0 6/0 6/0 6/0

+13,022.16% +293.60% +147.80% +95.57% +51.21% +37.59% +21.62% +13.38% +9.25% +8.92% +4.04% +3.07%

[+12,216.65%, +13,778.64%] [+258.84%, +406.74%] [+122.37%, +193.92%] [+84.76%, +105.04%] [+18.10%, +54.35%] [+26.11%, +47.88%] [+18.31%, +25.82%] [+2.83%, +27.00%] [+1.52%, +56.49%] [+6.06%, +9.97%] [+2.64%, +6.01%] [+1.35%, +7.55%]

6/6 6/6 6/6 6/6 6/6 6/6 6/6 6/6 5/6 6/6 6/6 5/6

Consistent decreases (6 types) Replace Inline Code with Function Call Remove Subclass Remove Flag Argument Combine Functions into Transform Inline Class Replace Conditional with Polymorphism

0/6 0/6 0/6 0/6 0/6 0/6

−55.64% −39.33% −29.40% −6.44% −5.16% −4.82%

[−61.26%, −49.03%] [ −43.14%, −33.90%] [−32.85%, −28.38%] [−12.54%, −5.36%] [−10.50%, −1.09%] [−8.78%, −3.33%]

6/6 6/6 6/6 6/6 5/6 2/6

Mixed directions (4 types) Change Value to Reference Change Reference to Value Encapsulate Variable Replace Subclass with Delegate

4/2 3/3 3/3 3/3

+36.80% −9.70% +0.73% −0.38%

[ −41.12%, +169.92%] [ −52.31%, +15.78%] [−16.72%, +20.58%] [−23.97%, +10.98%]

6/6 6/6 6/6 5/6

Refactoring type

Large |𝛿 |

to +13,022.16% for Replace Error Code with Exception. The next-largest medians are +293.60% for Replace Type Code with Subclasses. Across the six consistently decreasing types, the medians range from −4.82% for Replace Conditional with Polymorphism to −55.64% for Replace Inline Code with Function Call. The four mixed-direction types further show that significance across all six workload groups does not imply a stable direction. For example, Change Value to Reference has four increases and two decreases, ranging from −41.12% to +169.92%. The other 23 types in the heatmap exhibit significant differences only for a subset of their comparisons, revealing additional workload dependence. Return Modified Value is unchanged under Aggregate but increases energy under all five input-size workloads, with changes from +13.12% to +97.12%. Introduce Parameter Object decreases energy under XS and S, increases it under Aggregate, L, and XL, and is unchanged under M. The four Extract Function/Method items provide within-type evidence: 11 of their 24 comparisons are significant, comprising two increases and nine decreases, with significant changes ranging from −70.46% to +30.20%. Therefore, both the executed workload and the concrete benchmark item can alter whether an energy difference is detected, its direction, and its magnitude. Finding 7 Refactoring-type effects are often large but remain workload-dependent: 162 of the 199 statistically significant item–workload comparisons (81.4%) have large Cliff’s-𝛿 effects, yet effect magnitudes vary across workload groups. Refactoring type therefore indicates the potential for energy impact but does not determine a fixed energy outcome.

Manuscript submitted to ACM

32

Haibo Wang, Heng Li, and Shin Hwei Tan Overall refactoring-type patterns. Taken together, the controlled results reveal two complementary type-level patterns.

First, 22 of the 61 types (36.1%) exhibit statistically significant energy differences across all six workload groups, and 18 of these 22 (81.8%) maintain the same direction. Three consistently increasing types—Replace Error Code with Exception, Replace Type Code with Subclasses, and Substitute Algorithm—have the three largest median increases, whereas three consistently decreasing types—Replace Inline Code with Function Call, Remove Subclass, and Remove Flag Argument—have the three largest median decreases. All six comparisons for each of these six types have large Cliff’s-𝛿 effects. However, these six refactoring types are not supported by RefactoringMiner, preventing their automatic identification in practical commits. This limitation does not imply that the refactorings are absent from real-world development; rather, they are often realized through complex sequences of coordinated edits that are difficult for existing refactoring detectors to recognize as instances of a single high-level refactoring. As a result, we cannot isolate their occurrences in Practical-benchmark to verify whether they exhibit the same directional energy behavior observed in Micro-benchmark. Accordingly, we interpret these findings as evidence for the controlled transformations evaluated in our study, rather than as universal properties of these refactoring types. Second, the remaining evidence is more workload-specific. Among the 45 types with at least one significant comparison, 23 differ only under selected workload groups, and 11 exhibit both increases and decreases across their significant comparisons. Refactoring type can therefore help prioritize which transformations to evaluate for energy impact, but it cannot by itself predict a fixed direction or magnitude across workloads. We next examine whether similar type-level patterns appear among the refactoring types detected in practical commits, separating single-type commits from those containing multiple refactoring types. 5.2.2

Refactoring Patterns Associated with Energy Differences in Practical-benchmark.

Analysis population and commit-level refactoring composition. We begin with the 481 Practical-benchmark commits analyzed in RQ1, drawn from 430 projects and each containing 30 paired before/after energy measurements. RefactoringMiner successfully analyzes 479 of these commits (99.6%). Since RefactoringMiner detects only a predefined set of supported refactoring types,2 it records no operation in 134 of the 479 commits (28.0%). These commits may contain refactorings outside the supported set or instances that the tool does not detect. We report these commits for transparency but exclude them from the type- and pattern-level analyses because they provide no detected refactoring type. For the remaining two commits, RefactoringMiner exceeds its default 90-second time limit and returns no analysis result. These two timeout commits are therefore excluded. The resulting RQ2 analysis population therefore contains 345 commits from 318 projects, each with at least one detected refactoring operation. We distinguish the number of refactoring operations from the number of distinct refactoring types in a commit. Each detected occurrence counts as one operation, whereas a type is counted once regardless of how many times it occurs. Accordingly, a single-type commit contains exactly one distinct detected type but may contain multiple operations of that type, while a multi-type commit contains at least two distinct detected types. Of the 345 commits, 89 (25.8%) are single-type commits and 256 (74.2%) are multi-type commits. In particular, 33 of the 89 single-type commits (37.1%) apply their sole type more than once, confirming that “single-type” does not mean “single-operation”. Because the subsequent individual-type analysis relies on the single-type commits, we also assess how broadly they represent the detected refactoring types. The 89 single-type commits span 26 types, whereas the multi-type commits and the complete RQ2 population span all 90 detected types.

2 RefactoringMiner refactoring support history.

Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

33

Table 12. Commit-level refactoring composition and energy classifications in Practical-benchmark. “Total distinct types” reports the number of distinct refactoring types detected across all commits in each group. Operations and distinct types are reported per commit as median [𝑄 1 –𝑄 3 ], where 𝑄 1 and 𝑄 3 are the 25th and 75th percentiles. Outcome percentages are calculated within each commit group. “Unchanged” denotes no statistically significant energy difference.

Commit group

Operations Distinct types Commits Projects median [𝑄 1 –𝑄 3 ] median [𝑄 1 –𝑄 3 ]

Total distinct types

Increase

Decrease

Unchanged

No detected type Single-type Multi-type

134 89 256

121 86 238

0 [0–0] 1 [1–2] 9 [4–20]

0 [0–0] 1 [1–1] 4 [3–8]

0 26 90

3 (2.2%) 6 (6.7%) 6 (2.3%)

7 (5.2%) 4 (4.5%) 10 (3.9%)

124 (92.5%) 79 (88.8%) 240 (93.8%)

RQ2 analysis population

345

318

5 [2–14]

3 [1–6]

90 12 (3.5%)

14 (4.1%)

319 (92.5%)

Table 12 relates this composition to the commit-level energy classifications established in RQ1. Among the 345 included commits, 12 (3.5%) exhibit a statistically significant energy increase, 14 (4.1%) exhibit a statistically significant decrease, and 319 (92.5%) are unchanged (i.e., no statistically significant difference is detected). A significant difference is observed in 10 of the 89 single-type commits (11.2%) and 16 of the 256 multi-type commits (6.3%). This difference is not statistically significant (Fisher’s exact test: OR = 1.89, exact 95% CI: [0.74, 4.65], 𝑝 = 0.160), providing no statistically significant evidence that the number of distinct detected types is associated with a greater likelihood of observing an energy difference. Nevertheless, the predominance of multi-type commits means that most practical outcomes cannot be attributed to one isolated refactoring type. We therefore next examine individual refactoring types and recurring combinations of types. Individual types and co-occurring patterns associated with energy differences. We first analyze single-type commits to determine whether any individual refactoring type exhibits a repeatable energy-increase or energy-decrease tendency in real-world commits. Each such commit contains only one distinct refactoring type detected by RefactoringMiner, although it may contain multiple instances of that type. This reduces the ambiguity introduced by co-occurring types and provides the clearest practical evidence for individual-type effects. Before examining their energy outcomes, we assess the diversity of the single-type group and the number of commits available for each type. These statistics indicate whether the practical benchmark provides sufficiently broad and repeated evidence to support general conclusions about particular refactoring types. The 89 single-type commits cover 26 of the 90 refactoring types detected in the full Practical-benchmark population. However, the evidence is unevenly distributed: 13 of the 26 types occur in only one single-type commit, whereas only Inline Variable (20 commits) and Extract Method (14 commits) occur in at least ten commits. Thus, the single-type group covers a range of refactoring types, but the limited number of observations for most types restricts our ability to establish repeatable type-level energy tendencies. We then examine whether any of these 26 types is repeatedly associated with statistically significant energy increases or decreases. Only 10 of the 89 single-type commits exhibit a significant energy difference: six increases and four decreases distributed across nine types. Inline Variable is the only type represented by more than one significant commit, and both differences have negligible Cliff’s-𝛿 magnitudes. Moreover, no individual type remains associated with an energy increase or decrease after correcting the type-level tests for multiple comparisons using the Benjamini–Hochberg procedure. Therefore, the current single-type evidence does not support assigning a fixed energy direction to any practical refactoring type. Manuscript submitted to ACM

34

Haibo Wang, Heng Li, and Shin Hwei Tan

Table 13. Strongest recurring co-occurrence patterns for significant energy increases and decreases. For each pattern-size–outcome group, the three patterns with the smallest Fisher exact-test 𝑝-values are shown. All patterns occur in at least five multi-type commits. sup is pattern support, out is the number of commits with the target outcome, and conf is out/sup. Lift is calculated relative to the overall outcome rate among the 256 multi-type commits: 6/256 (2.3%) for significant increases and 10/256 (3.9%) for significant decreases. 𝑞 is the BH-adjusted value; bold values survive correction at 0.05.

Pattern Energy increase: pairs {Change Attribute Type, Remove Thrown Exception Type} {Inline Method, Rename Parameter} {Add Parameter, Remove Thrown Exception Type} Energy increase: triples {Change Attribute Type, Change Return Type, Rename Method} {Add Parameter, Change Return Type, Remove Thrown Exception Type} {Add Parameter, Inline Method, Rename Method} Energy decrease: pairs {Inline Variable, Remove Method Annotation} {Change Parameter Type, Inline Variable} {Change Parameter Type, Remove Method Annotation} Energy decrease: triples {Change Parameter Type, Inline Variable, Remove Method Annotation} {Change Parameter Type, Change Variable Type, Move Attribute} {Change Parameter Type, Change Variable Type, Remove Method Annotation}

sup

out

conf

lift

𝑞

5 5 6

2 40.0% 17.07 2 40.0% 17.07 2 33.3% 14.22

0.394 0.394 0.394

15 5 5

3 20.0% 8.53 2 40.0% 17.07 2 40.0% 17.07

0.391 0.391 0.391

7 11 11

4 4 4

14.63 9.31 9.31

0.0168 0.0489 0.0489

6 5 5

4 66.7% 17.07 3 60.0% 15.36 3 60.0% 15.36

0.0198 0.159 0.159

57.1% 36.4% 36.4%

We therefore focus the co-occurrence analysis on pairs and triples of types in the 256 multi-type commits, without repeating a size-one presence analysis. As Table 8 shows, the number of distinct detected refactoring types per analyzed practical commit has a mean of three and a median of two. Pairs and triples consequently represent the most typical co-occurrence sizes, whereas larger combinations would be increasingly sparse and project-specific. We treat each pattern as an unordered set of distinct types and count it at most once per commit, regardless of how many operations of its constituent types occur. This binary definition matches the commit-level energy outcome and prevents commits with many repeated operations from receiving disproportionate weight. We retain patterns occurring in at least five of the 256 multi-type commits. For each pair or triple, we compare commits containing the pattern with the other multi-type commits using a two-sided Fisher’s exact test. We apply BH correction separately for pairs and triples and for increases and decreases. This procedure evaluates 424 pairs and 1,142 triples for each direction. Table 13 reports the three strongest patterns in each pattern-size–outcome group. No pair or triple associated with an energy increase survives correction; the smallest adjusted value is 𝑞 = 0.391. In contrast, three pairs and one triple remain associated with energy decreases. The triple {Change Parameter Type, Inline Variable, Remove Method Annotation} occurs in six commits from six projects, four of which exhibit a significant decrease (66.7%), compared with 10 of the 256 multi-type commits overall (3.9%). Its decrease rate is therefore 17.07 times the multi-type baseline. The three surviving pairs are {Inline Variable, Remove Method Annotation} (4/7 decreases; 𝑞 = 0.0168), {Change Parameter Type, Inline Variable}, and {Change Parameter Type, Remove Method Annotation} (both 4/11; 𝑞 = 0.0489). The same four decrease commits support all four corrected patterns. These rows therefore represent an overlapping co-occurrence signal, not four independent effects. The next-ranked decrease triples also combine parameter- or variable-type changes with annotation removal or code movement, but they do not survive correction (𝑞 = 0.159). Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

35

Finding 8 Real-world commits containing certain specific recurring combinations of refactoring types are substantially more likely to exhibit significant energy changes than multi-type commits overall. In our dataset, the strongest evidence concerns energy reductions: commits containing the combination of Change Parameter Type, Inline Variable, and Remove Method Annotation show a 66.7% decrease rate, representing a 17.1× increase over the 3.9% baseline for all multi-type commits, whereas no recurring combination is significantly associated with energy increases after multiple-comparison correction. These results suggest that energy impact analysis should consider recurring refactoring combinations alongside individual refactoring types, motivating the identification of energy-aware refactoring patterns rather than just reasoning about isolated refactoring types.

Magnitude and robustness of practical refactoring-pattern associations. We next examine whether the four corrected associations correspond to broader energy reductions across all commits containing each pattern and whether they remain stable across projects. Including commits classified as unchanged, all four patterns have negative median relative energy changes, ranging from −2.72% to −10.71%. However, the project-bootstrap 95% confidence intervals for the median remain entirely below zero only for {Change Parameter Type, Inline Variable} and the complete triple; the intervals for the other two pairs include zero. Association with a significant decrease therefore does not imply consistent energy savings in every commit containing a pattern. All four associations retain BH-corrected significance in 94.5%–98.3% of the 238 leave-one-project-out analyses. The triple also remains significant when its comparison group is restricted to the 193 multi-type commits containing at least three distinct refactoring types. Sensitivity to the minimum-support threshold is less uniform: corrected decrease associations remain at thresholds from 3 to 11 commits, but the surviving patterns change, and none remains at thresholds from 12 to 15. Overall, the robustness analyses show that the decrease-associated signal is not driven by any single project and remains detectable across multiple analysis settings. Overall refactoring patterns in practical commits. Taken together, the practical results show that neither individual refactoring types nor the number of types in a commit alone adequately characterizes its energy behavior. Although 256 of the 345 analyzed commits (74.2%) contain multiple refactoring types, multi-type commits are not significantly more likely than single-type commits to exhibit an energy difference, and no individual type shows a repeatable association after BH correction. By contrast, the co-occurrence analysis identifies recurring pairs and triples associated with significant energy decreases, and these associations remain detectable across multiple robustness settings. This indicates that refactoring composition provides useful information beyond refactoring multiplicity. These findings complement the controlled results from Micro-benchmark, which isolate an intended refactoring type and reveal its possible energy behavior under specific workloads. Practical commits combine multiple types and operations across files and execute them under project-specific workloads; consequently, isolated type-level effects cannot simply be transferred or added together to predict the energy impact of a complete commit. Together, the two benchmarks show that the energy impact of refactoring should be interpreted through both refactoring composition and execution context. RQ3 therefore examines the source-level and runtime factors that may explain why particular refactoring changes exhibit energy differences. Consistency of refactoring-type energy effects across benchmarks. To determine whether the workload-consistent effects observed in Micro-benchmark also appear in real-world commits, we compare the 18 Micro-benchmark types Manuscript submitted to ACM

36

Haibo Wang, Heng Li, and Shin Hwei Tan

Table 14. Cross-benchmark comparison of the consistently directional Micro-benchmark types that also occur among the single-type commits. Practical outcomes are followed by their mean energy changes in parentheses.

Refactoring type Extract Class Replace Loop with Pipeline

Micro-benchmark direction

Median Δ𝐸

Single-type commits

Practical outcome

Increase (6/6) Increase (6/6)

+8.92% +13.38%

1 1

Unchanged (+5.64%) Unchanged (+15.17%)

that exhibit statistically significant differences in the same direction across all six workload groups with the refactoring types in the 89 single-type commits. Focusing on single-type commits reduces the ambiguity caused by co-occurring refactoring types and provides the clearest available practical evidence for comparison. Of these 18 Micro-benchmark types, only Extract Class and Replace Loop with Pipeline also occur among the 26 types in the single-type commits. As Table 14 shows, both types significantly increase energy consumption across all six workload groups in Microbenchmark. Their single-type practical commits also have positive mean energy changes, but neither difference is statistically significant. Moreover, each type is represented by only one single-type commit, and none of the consistently decreasing Micro-benchmark types occurs among the single-type commits. The practical evidence is therefore insufficient to confirm that either type consistently increases energy across the two benchmarks. Overall, our results do not identify any refactoring type that can currently be classified as universally energy-increasing or energy-decreasing; establishing such a classification would require repeated single-type observations across diverse real-world projects. 5.3

RQ3: What factors explain refactoring-induced energy differences?

RQ1 established the frequency, direction, and magnitude of energy differences after refactoring, while RQ2 examined how these differences vary across refactoring types and combinations. RQ3 investigates the factors that help explain these differences. We first use Micro-benchmark to relate refactoring-induced energy differences to changes in runtime behavior and examine their sensitivity to workload and JVM optimization. We then use Practical-benchmark to examine which measured characteristics of real-world refactoring commits are associated with energy differences. 5.3.1

Runtime Mechanisms in Micro-benchmark.

Analysis population and runtime measures. We analyze all Micro-benchmark items for which both the energy measurements and the corresponding runtime profiles are complete. Of the 71 candidate items, 64 have valid energy measurements for Aggregate and all five input-size workloads, as reported in RQ1. Among these items, one does not have a complete JMH/JFR profile, leaving 63 items for the runtime analysis. This population contains 63 × 6 = 378 item–workload observations and 11,340 paired energy measurements. For every observation, we relate the mean relative energy change to changes in execution time, normalized allocation per operation, allocation samples, young-generation garbage-collection events, total garbage-collection pause time, and CPU-load records. The runtime profiles are collected separately from the energy measurements using the same before/after versions and workload settings, thereby avoiding profiler overhead in the energy measurements. Each runtime profile contains one JMH/JFR run per version; we therefore interpret the runtime results as explanatory evidence rather than as repeated performance estimates. Execution time and normalized allocation are expressed as relative changes. For event and pause metrics that can equal zero, we use the signed difference between their log-transformed values, log(1 + 𝑋 after ) − log(1 + 𝑋 before ), so that zero-to-nonzero transitions remain defined. Because the change distributions are strongly skewed and include a verified extreme item, Figure 11(a) reports Spearman correlations rather than correlations that can be dominated Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

37

by a single extreme value. We control the false-discovery rate across the 36 metric–workload associations using the Benjamini–Hochberg procedure. Integrated relationships between runtime behavior and energy. Figure 11(a) shows that execution-time change has a positive association with energy change in every workload group. The correlations range from 𝜌 = 0.368 for XL to 𝜌 = 0.551 for XS, and all six remain significant after correction (𝑞 ≤ 0.009). CPU-load records show a similar workloadwide pattern, with correlations from 𝜌 = 0.394 to 𝜌 = 0.673 (𝑞 ≤ 0.005). Since these values count recorded CPU-load events rather than directly measuring processor utilization, we treat them as supporting rather than independent evidence. Among the 194 item–workload observations with a statistically significant energy difference in this 63-item population, execution time changes in the same direction as energy in 146 cases (75.3%); the workload-specific agreement ranges from 66.7% for XL to 87.5% for M. As a consistency check using the energy-run records, mean batch duration agrees in direction with energy in 182 of the 194 cases (93.8%), compared with 114 cases (58.8%) for mean power. Together, these results identify execution time as the most consistent measured pathway through which the controlled transformations are associated with energy differences. Allocation samples count sampled heap-object allocation events during execution and therefore reflect objects created by the program. Young-generation garbage-collection events count collections that mainly reclaim short-lived heap objects, while garbage-collection pause time measures the total time spent in garbage-collection pauses. Allocation per operation reports the average heap memory allocated for the workload. CPU-load records count the JFR CPUload observations captured during execution. As shown in Figure 11(a), changes in allocation samples are positively associated with energy change under S (𝜌 = 0.361, 𝑞 = 0.010) and L (𝜌 = 0.341, 𝑞 = 0.016), but not under the other four workloads. Normalized allocation per operation, young-generation garbage-collection events, and garbage-collection pause time show no association that remains significant after correction under any workload. The absence of a benchmark-wide monotonic relationship does not imply that memory allocation and garbage collection are irrelevant to individual transformations. For example, as shown in Table 3, Replace Loop with Pipeline increases energy under all six workload groups, while the accompanying changes in execution time, allocation samples, young-generation garbage-collection events, and garbage-collection pause time vary substantially across workloads. Thus, memory-allocation and garbage-collection activity can provide supporting explanations for particular transformations and workloads, whereas execution time provides the most consistent explanation across items. The remaining cases in which execution time and energy change in different directions further show that execution time alone cannot account for every observed energy difference. Workload and JVM-optimization sensitivity. The runtime associations are workload-dependent. Execution-time change remains associated with energy change across Aggregate and XS–XL, although its strength varies, while allocation-sample change is associated only under S and L. Item-level profiles further show that the magnitudes of execution-time, allocation, and garbage-collection changes vary with input size. Thus, workload affects both whether an energy difference is detected and which runtime changes accompany it. We assess JVM-optimization sensitivity by comparing Aggregate relative before/after effects under default JIT and JIT-disabled execution with -Xint. Of the 64 RQ1-eligible items, 30 (46.9%) yield 30 complete before/after energy pairs under both modes. The remaining 34 do not yield complete valid -Xint results within the measurement setup, which uses a 7,200-second timeout, and are excluded. Because the stored states do not distinguish timeouts from other failures and the complete-case subset may favor faster items, these results apply only to the 30 matched items. We compare within-item relative effects because batch repetition counts can differ between JVM modes. Manuscript submitted to ACM

38

Haibo Wang, Heng Li, and Shin Hwei Tan (b) JVM-mode sensitivity

0.48*

0.55*

0.45*

0.51*

0.40*

0.37*

Allocation per op

0.23

-0.11

-0.12

-0.05

0.19

0.26

Allocation samples

-0.09

0.13

0.36*

0.15

0.34*

0.12

Young-GC events

0.01

0.06

0.24

0.27

0.20

0.16

GC pause time

0.05

-0.07

0.22

0.12

-0.04

0.04

CPU-load records

0.40*

0.56*

0.67*

0.48*

0.39*

0.41*

Aggregate

XS

S

M

L

XL

Workload group

Default-JIT outcome

(a) Runtime associations Execution time

Increase

5 (71%)

1 (14%)

1 (14%)

Unchanged

5 (31%)

7 (44%)

4 (25%)

Decrease

4 (57%)

0 (0%)

3 (43%)

se

d

a re

Inc

ge an ch

Un

e

as

cre

De

JIT-disabled outcome

−0.6

−0.4

−0.2

0.0

0.2

0.4

Spearman correlation with energy change

0.6

0%

50%

100%

Share within default-JIT outcome

Fig. 11. Runtime relationships and JVM-optimization sensitivity in Micro-benchmark. (a) Spearman correlations between relative energy change and runtime-metric change for the 63 items with complete energy and JMH/JFR results. Asterisks denote associations that remain significant after Benjamini–Hochberg correction (𝑞 < 0.05). Event-count and pause-time changes use signed logtransformed differences. (b) Transitions among significant increase, unchanged, and significant decrease outcomes from default-JIT execution (rows) to JIT-disabled execution using -Xint (columns) for the 30 items with complete Aggregate results under both modes. Cell labels report counts and row percentages.

Figure 11(b) shows substantial item-level sensitivity. The default-JIT outcomes comprise seven increases, 16 unchanged cases, and seven decreases, compared with 14 increases, eight unchanged cases, and eight decreases under -Xint. Only 15 items (50.0%) retain the same outcome category (𝜅 = 0.272), and the signs of the raw change ratios agree for only 16 (53.3%). Among the 14 items significant under default JIT, eight remain significant in the same direction, five reverse direction, and one becomes unchanged; nine of the 16 previously unchanged items become significant under -Xint. The relative effects show no systematic shift between modes (Wilcoxon signed-rank test, 𝑝 = 0.221) and no monotonic association in item rankings (𝜌 = −0.001, 𝑝 = 0.997). The latter result remains nonsignificant after excluding Replace Error Code with Exception (𝜌 = −0.108, 𝑝 = 0.578). Disabling JIT therefore does not shift all refactorings in a common direction; instead, it changes the direction, significance, or magnitude of many individual effects. Finding 9 Execution time and CPU-load records changes provide the most consistent runtime-level explanations for refactoring-induced energy differences in Micro-benchmark: both are positively associated with energy change under every workload. Execution time change also agrees in direction with 75.3% of statistically significant item– workload results. Allocation and garbage-collection relationships are more workload-specific. JVM optimization further changes the item-level outcome: only half of the 30 matched items retain the same increase, unchanged, or decrease classification when JIT compilation is disabled.

5.3.2

Explanatory Factors in Practical-benchmark.

Analysis population and factor families. We analyze the same 481 Practical-benchmark commits from 430 projects used in RQ1. For each commit, Δ𝐸 and Δ𝑇 denote the before/after log ratios of energy and execution time. We define Δ𝑅 = Δ𝐸 − Δ𝑇 , where 𝑅 = 𝐸/𝑇 is the derived energy consumed per unit of execution time rather than directly measured power. All commits have valid energy summaries and 30 paired before/after execution-time observations. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

39

Static-code metrics (A) are available for 461 commits, code-change metrics (B) for 481, coverage metrics (C) for 418, RefactoringMiner metrics (D) for 479, semantic replacement distance (E) for 454, and covered-diff alignment for 255. Each association uses the largest valid population; factor-family comparisons use the 384 commits from 350 projects with complete execution-time and A–E values. We consolidate the 59 static metrics into six standardized, signed log-transformed groups: size and structure, control flow, computation, data and object use, coupling, and I/O or concurrency proxies. The remaining factors capture change size and dispersion, changed-line coverage and its change, refactoring count and diversity, and semantic replacement distance and covered-diff alignment. Undefined static aggregates caused by absent constructs are set to zero; extraction failures remain missing. We calculate Spearman correlations with Δ𝐸, |Δ𝐸|, Δ𝑇 , and Δ𝑅, apply BH correction across the 59 associations in Figure 12(a), and obtain confidence intervals by resampling projects. Integrated associations across execution and A–E factors. Figure 12(a) shows a weak positive association between execution-time and energy changes (𝜌 = 0.167, project-bootstrap 95% CI [0.084, 0.247], 𝑞 = 0.013), the only displayed association that remains significant after BH correction. Among the 32 commits with significant energy differences and with complete execution-time and A–E values, Δ𝐸 and Δ𝑇 have the same sign in 25 cases (78.1%). However, execution time changes significantly in the same direction in only six cases; it is unchanged in 25 and changes significantly in the opposite direction in one. Sign agreement therefore overstates the explanatory role of execution time. Execution-time changes are also small: the median |Δ𝑇 |/|Δ𝐸| is 0.008, and |Δ𝑇 | reaches half of |Δ𝐸| in only two cases. In contrast, Δ𝑅 has the same direction as Δ𝐸 and reaches at least half its magnitude in 31 cases, with a median |Δ𝑅|/|Δ𝐸| of 0.996. Thus, most significant energy differences are reflected in the derived energy-rate change rather than execution-time change. No individual A–E association remains significant after correction, and all coefficients satisfy |𝜌 | ≤ 0.172. Hence, none of the measured source, change, coverage, refactoring, or semantic factors provides a consistent monotonic explanation across commits, although commit-specific effects may still exist. Relative explanatory contributions. Figure 12(b) shows that execution time contributes 2.5 𝑅 2 percentage points to Δ𝐸, or 44.6% of the complete model’s explained variation. Static-code and refactoring factors contribute 1.9 and 0.7 points, respectively; each remaining family contributes at most 0.3 points. Nevertheless, the complete model explains only 5.7% of the rank variation in Δ𝐸. The A–E factors explain 6.4% of the variation in Δ𝑇 and 3.1% in Δ𝑅, with static-code factors providing most of these limited contributions. Most between-commit variation therefore remains unexplained. Finding 10 In Practical-benchmark, execution-time change is the only measured factor that remains associated with energy change after correction, but the relationship is weak, and a significant timing change agrees with the energy outcome in only 6 of the 32 commits with a significant energy difference. No individual static-code, code-change, test-exposure, refactoring, or semantic factor remains significant, and all measured factors together explain only 5.7% of the rank variation in energy change. Thus, execution time and the available commit-level characteristics do not provide a sufficient general explanation for refactoring-induced energy differences in real-world commits.

Overall explanation across the two benchmarks. Across both benchmarks, behavior preservation does not guarantee identical runtime behavior or energy consumption. In Micro-benchmark, execution time provides the most consistent explanation, while allocation and garbage-collection relationships depend on workload and many outcomes change Manuscript submitted to ACM

40

Haibo Wang, Heng Li, and Shin Hwei Tan (a) Integrated factor associations +0.17*

-0.06

Static: size/structure

-0.09

+0.02

(b) Relative explanatory contributions +0.13

-0.02

Execution time

-0.10

2.5

0.20

-0.04

-0.02

+0.04

-0.02

Static: computation

-0.00

-0.00

+0.06

+0.00

Static: data/object use

-0.04

+0.07

-0.07

-0.04

Static: coupling

-0.06

+0.01

+0.10

-0.06

Static: I/O/concurrency proxies

+0.01

+0.08

-0.06

+0.02

Change: size

-0.06

-0.09

-0.00

-0.05

Change: dispersion

-0.05

-0.10

+0.01

-0.04

Test: changed-line exposure

+0.05

+0.07

-0.04

+0.06

Test: coverage change

-0.02

+0.04

-0.04

-0.02

Refactoring: operation count

-0.06

-0.08

+0.01

-0.05

0.15

Refactoring: type diversity

-0.04

-0.09

+0.01

-0.03

+0.04

+0.06

+0.05

+0.02

Semantic: covered-diff alignment

+0.13

+0.10

+0.17

+0.13

ΔE

|ΔE|

ΔT

ΔR

A: Static code

1.9

5.1

1.8 4

0.10

0.05

0.00

−0.05

Semantic: replacement distance

5

Spearman correlation

Static: control flow

B: Code change

0.2

0.4

0.2 3

C: Test exposure

0.3

0.4

0.3

D: Refactoring

0.7

0.2

0.7

E: Semantic context

<0.1

0.3

<0.1

ΔE R 2=0.06

ΔT R 2=0.06

ΔR R 2=0.03

2

Shapley R 2 contribution (points)

Execution-time change

−0.10 1 −0.15

−0.20

0

Fig. 12. Explanatory-factor analysis for Practical-benchmark. (a) Spearman correlations between execution and A–E factors and energy, energy-change magnitude, execution time, and derived energy-rate change. Each factor uses its largest valid population (𝑁 = 255–481); the execution-time self-correlation is omitted. The asterisk marks the only association significant after BH correction across the 59 tests (𝑞 < 0.05). (b) Shapley contributions to rank-based ordinary-least-squares 𝑅 2 for the common population of 384 commits from 350 projects. Cells report percentage-point contributions, and column labels report total 𝑅 2 . The Δ𝐸 model includes execution time and A–E; the Δ𝑇 and Δ𝑅 models include A–E.

when JIT compilation is disabled. In Practical-benchmark, execution time remains the strongest measured factor, but its association is weak and the available commit-level factors explain little variation. This difference reflects the heterogeneous transformations and execution contexts of practical commits, whose static characteristics only indirectly represent runtime behavior. Overall, refactoring-induced energy differences arise from interactions among the transformation, workload, and JVM behavior, rather than from fixed refactoring-type effects or coarse code characteristics. The limited explanatory power of these factors motivates RQ4, which examines whether they can nevertheless identify energy regressions. 5.4

RQ4: How effective are existing techniques in identifying refactoring-induced energy regressions?

To answer RQ4, we evaluate two approaches on Practical-benchmark: metric-based classifiers adapted from prior performance-bug prediction work [67] and a prompt-based LLM classifier. Following the ground-truth definition in RQ1, a commit is labeled Reg when the after-refactoring version exhibits a statistically significant energy increase under the paired Wilcoxon signed-rank test (𝑝 < 0.05) and has a positive mean paired energy difference. All other commits, including those with no significant difference and those with a significant decrease, are labeled Not Reg. We use precision, recall, and F1 for Reg as the primary measures of identification effectiveness. Origin and adaptation of the existing techniques. Our metric-based baselines are adapted from the performance-bug prediction approach of Zhao et al. [67]. Their study predicts performance regression using code metrics and evaluates seven machine-learning algorithms. We select the three best-performing algorithms reported in their study: Logistic Regression, Random Forest, and XGBoost. For our task, each metric-based classifier uses all five information families: Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

41

Table 15. Performance on the Reg class across identification techniques.

Tool gpt-5-mini Logistic Regression* Random Forest* XGBoost*

Precision

Recall

F1

0.056 0.031 0.000 0.000

0.357 0.200 0.000 0.000

0.096 0.053 0.000 0.000

* Existing metric-based technique adapted from Zhao et al. [67]. These are the three best-performing algorithms in their study.

Table 16. Performance of the LLM-based technique and its leave-one-information-family-out ablations.

Reg Tool

gpt-5-mini

Omitted information None Static-code metrics (A)† Code-change metrics (B)† Test/coverage metrics (C) Refactoring metrics (D) Code diff (E)

Precision

Recall

F1

0.056 0.040 0.031 0.011 0.040 0.024

0.357 0.286 0.214 0.071 0.286 0.143

0.096 0.070 0.055 0.019 0.070 0.041

Each row after the first omits the indicated information family from the complete prompt. † Information family adapted from the existing metric-based technique of Zhao et al. [67]; C–E are additional information used in this study. For E, the LLM receives the textual code diff. Bold values are the highest in each reported column.

static-code metrics (A), code-change metrics (B), test/coverage metrics (C), refactoring metrics (D), and code-diff information (E). We additionally evaluate gpt-5-mini using the same five information families (A–E). Metric-based identification. Table 15 marks the three classifiers adapted from Zhao et al. [67] and reports their performance using A–E. Logistic Regression is the only metric-based classifier that identifies any regression cases, but its precision of 0.031 and recall of 0.200 yield an F1 of only 0.053. Random Forest and XGBoost obtain zero precision, recall, and F1 for Reg, meaning that neither identifies a true regression under the evaluated setting. LLM-based identification. Table 16 reports the performance of gpt-5-mini when its structured prompt contains all five information families and when one family is omitted at a time. With all families included, the model achieves a Reg precision of 0.056, recall of 0.357, and F1 of 0.096. Thus, it identifies 35.7% of the actual regressions but misses the remaining 64.3%. Moreover, only 5.6% of its regression predictions are correct, equivalent to approximately one true regression for every 18 commits flagged for validation. Contribution of the information families. The complete A–E prompt obtains the highest Reg precision, recall, and F1, indicating that no individual family can be removed without reducing target-class performance. Test/coverage metrics (C) have the largest observed ablation effect: omitting them reduces Reg recall from 0.357 to 0.071 and Reg F1 from 0.096 to 0.019. Code-diff information (E) provides the next clearest minority-class signal; its removal reduces Reg recall to 0.143 and F1 to 0.041. Removing code-change metrics (B) also lowers recall to 0.214, whereas omitting static-code (A) or refactoring (D) metrics lowers it to 0.286. These leave-one-family-out results suggest that test exposure and the textual content of the code change provide complementary evidence to the LLM. Cross-approach comparison. Table 15 compares the complete LLM configuration with the three metric-based classifiers on the Reg measures. The LLM has the highest reported values on all three measures: compared with Logistic Regression, its recall is 0.357 rather than 0.200 and its F1 is 0.096 rather than 0.053. More importantly, the LLM’s numerical advantage Manuscript submitted to ACM

42

Haibo Wang, Heng Li, and Shin Hwei Tan

does not make its absolute performance adequate. It still misses almost two thirds of the regressions and produces predominantly false regression alerts. The uniformly low absolute results show that neither the adapted existing techniques nor the LLM-based technique provides a dependable decision boundary for this rare outcome.

Finding 11 Metric-based techniques adapted from prior performance-bug prediction transfer poorly to refactoring-induced energy-regression identification: Logistic Regression achieves only 0.053 F1, while Random Forest and XGBoost identify no true regressions. Using all five information families, gpt-5-mini performs best among the evaluated configurations, but its 0.357 recall and 0.056 precision mean that it misses 64.3% of regressions and yields only about one true regression per 18 alerts. Consequently, none of the evaluated techniques is reliable as a standalone replacement for direct energy validation.

6

Implications

Refactoring preserves program behaviour but does not guarantee preservation of energy consumption. Across both controlled and real-world benchmarks, refactoring can increase, decrease, or leave energy consumption unchanged, depending on the transformation, workload, and execution context. These findings show that the energy impact of refactoring is context-dependent rather than deterministic, leading to several implications. Developers should validate energy-sensitive refactorings under representative workloads. Knuth argues that programmers should avoid optimizing noncritical parts of a program [31]. Our results lead to the same practical lesson for energy: developers should first identify the code and workloads that materially contribute to energy consumption, and then measure the refactorings that affect them. They should compare the original and refactored versions through repeated paired measurements under representative workloads. Workload selection is crucial: in Micro-benchmark, 45.3% of items change classification across input sizes, and 15.6% switch between increases and decreases. Energy-aware refactoring tools should verify predictions with targeted measurement. Execution time change is the clearest measured mechanism in Micro-benchmark, but it is only weakly associated with energy change in Practicalbenchmark. The best evaluated LLM also misses 64.3% of regressions, and only about one in 18 of its regression alerts is correct. No evaluated proxy or predictor is therefore reliable enough to replace energy measurement. Moreover, tools should not directly extrapolate isolated type-level effects to real-world commits, particularly when multiple refactorings may interact or offset one another. Tools can instead use refactoring, code change, coverage, and runtime evidence to select candidates and representative workloads, then confirm consequential cases through paired measurements. This keeps measurement in the decision loop while using prediction to reduce its cost. Researchers should study refactoring as an execution-conditioned change. Our results suggest that a refactoring type is too coarse to serve as a universal energy label. Future benchmarks should treat the combination of transformation pattern, workload, and runtime environment as the unit of analysis, with repeated measurements and diverse workloads. They should also analyze single-type and mixed-type commits separately. This would reveal when a strong isolated effect matters in practice and when it becomes small or is offset by other refactorings. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring 7

43

Threats to Validity

Internal. Energy measurements can be affected by JVM warm-up, background activity, and execution workloads. We mitigate this threat by conducting all experiments on a dedicated machine without concurrent workloads, warming up the JVM before measurement, inserting cooling intervals between runs, and randomizing the execution order of the before- and after-refactoring versions. Within each pair, we keep the workload and execution configuration fixed. In Micro-benchmark, we additionally use a common repetition count and collect 30 paired measurements. We use the paired Wilcoxon signed-rank test and report Cliff’s delta to reduce sensitivity to measurement noise and distributional assumptions. In Practical-benchmark, non-refactoring changes within a commit could confound the observed energy difference. We reduce this threat by requiring both versions to compile and pass the same unchanged tests, excluding commits that modify test files, requiring positive modified-line coverage measured by JaCoCo, and manually inspecting the retained commits for non-refactoring changes. These controls reduce the influence of broken builds, changed test behavior, and unexecuted or unrelated code on the measured differences. External. The measured energy impact of refactoring depends on the workload used to exercise the program. We address this threat in Micro-benchmark by evaluating each eligible item under an aggregate workload and five input sizes. In Practical-benchmark, we repeatedly execute the test suite of each real project. Requiring modified-line coverage further ensures that the changed code is exercised during measurement. We also use two complementary benchmarks: Micro-benchmark isolates refactoring effects under controlled workloads, whereas Practical-benchmark captures refactoring in real repositories. This combination provides evidence from both controlled and practical settings. Our analysis focuses on Java programs. We chose Java because it is one of the most widely used programming languages [46], has a mature refactoring and execution ecosystem, and provides continuity with closely related research. Fowler’s original refactoring catalog uses Java examples [19]; Sahin et al. study the energy impact of refactorings on Java applications [54]; and prior performance-bug and performance-regression studies also examine Java systems [13, 18, 67]. This ecosystem enables Micro-benchmark to cover 68 refactoring types and allows real Maven projects in Practicalbenchmark to be compiled, tested, and measured consistently.

8 8.1

Related Work Energy Effects of Program Transformations

Software energy consumption can change even when functionality is preserved. Sahin et al. [53] show that the energy effects of design patterns vary across individual patterns and cannot be inferred from broad pattern categories. Manotas et al. [39] present SEEDS, which searches among behavior-preserving implementation alternatives and measures them under a target workload to identify lower-energy choices. Other studies report that code obfuscation can alter energy consumption [55] and propose source-level transformations for removing energy-inefficient constructs in embedded software [30]. Refactoring-specific studies provide the closest basis for our work. Sahin et al. [54] manually apply six refactoring types to nine Java applications and measure energy through repeated JUnit-test executions. They find both increases and decreases, and show that execution time and dynamic execution counts do not fully predict the resulting energy effect. Park et al. [48] estimate the energy consumption of small examples covering 63 Fowler refactoring types, whereas Şanlıalp et al. [56] examine triple combinations of five refactoring types. Complementing these controlled energy studies, Traini et al. [59] mine refactoring-focused commits and show that refactoring can also produce execution-time improvements and regressions in real Java systems. However, prior energy studies primarily Manuscript submitted to ACM

44

Haibo Wang, Heng Li, and Shin Hwei Tan

examine isolated examples under fixed or simple workloads. They therefore leave open how refactoring energy effects vary across workloads, manifest in real-world commits, and relate to source-level and runtime factors. 8.2

Performance-Regression Identification

Prior work has used tests and software metrics to identify performance regressions. Ding et al. [18] investigate whether tests already available in release pipelines can serve as performance tests, and Chen et al. [13] propose test-level just-in-time prediction of performance-regression-inducing commits. Liao et al. [33, 34] leverage black-box machine learning models that learn performance from execution events to detect performance regressions. More recently, Zhao et al. [67] predict performance regression using code metrics with multiple machine-learning algorithms. These techniques target execution-performance bugs or regressions, not energy increases introduced by behavior-preserving refactoring. Our study connects these two research lines. We combine a controlled, workload-aware Micro-benchmark spanning 68 refactoring types with a Practical-benchmark of real-world GitHub refactoring commits, where multiple operations may co-occur. Beyond measuring whether energy changes, we analyze workload sensitivity, refactoring types and co-occurrence patterns, and source-level and runtime factors associated with the changes. We further evaluate whether metric-based techniques adapted from performance-bug prediction and a prompt-based LLM can identify refactoring-induced energy regressions. Thus, our study extends prior evidence from whether refactoring can affect energy to when such effects arise, how they can be explained, and how effectively regressions can be identified. 9

Conclusion

This paper examined the energy impact of behavior-preserving refactoring using a workload-aware Micro-benchmark covering 68 refactoring types and a Practical-benchmark of 481 real-world refactoring commits. In Micro-benchmark, 51.8% of the 384 refactoring-instance–workload comparisons exhibit statistically significant energy differences, and 45.3% of the 64 eligible refactoring instances yield different increase, decrease, or unchanged outcomes across input sizes. In Practical-benchmark, significant differences occur in only 7.5% of commits, although two thirds of these statistically significant changes have an absolute magnitude of at least 10%. Both benchmarks contain increases and decreases. Energy effects cannot be assigned as fixed labels to refactoring types. In the controlled benchmark, the effects of individual refactoring types are often workload-dependent; in Practical-benchmark, no individual refactoring type has a consistent association with energy change, although several recurring combinations are associated with decreases. Across the Micro-benchmark workloads, energy change is associated with changes in execution time and CPU-load-record count. In Practical-benchmark, however, the execution-time relationship is weak, and all measured factors together explain only 5.7% of the variation in energy change. Overall, behavior-preserving refactoring is not inherently energy-neutral: its observed energy impact varies across refactoring types and workloads and differs between isolated refactoring instances and real-world refactoring commits. Neither the adapted metric-based classifiers nor the evaluated LLM can reliably identify refactoring-induced energy regressions. Acknowledgment We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), [funding reference number ALLRP 597968 - 24]. Cette recherche a été financée par le Conseil de recherches en sciences naturelles et en génie du Canada (CRSNG), [numéro de référence ALLRP 597968 - 24]. Ce projet de recherche # 361973 est rendu possible grâce au financement du Fonds de recherche du Québec. We acknowledge the support of the Government of Canada’s New Frontiers in Research Fund (NFRF) under Grant No. NFRFE-2024-00612. Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

45

References [1] Rakesh Agrawal, Tomasz Imieliński, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD international conference on Management of data. 207–216. [2] Alan Agresti. 2013. Categorical data analysis. John Wiley & Sons. [3] Maurício Aniche. 2026. Code metrics for Java code by means of static analysis. https://github.com/mauricioaniche/ck [4] Robert Arntzenius, Xutong Liu, and Andy Zaidman. 2026. On the Energy Consumption of Continuous Integration in Open-Source Java Projects. In 2026 IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C). 181–188. doi:10.1109/SANERC67878.2026.00030 [5] Amirreza Bagheri and Péter Hegedüs. 2022. Is refactoring always a good egg? exploring the interconnection between bugs and refactorings. In Proceedings of the 19th International Conference on Mining Software Repositories (Pittsburgh, Pennsylvania) (MSR ’22). Association for Computing Machinery, New York, NY, USA, 117–121. doi:10.1145/3524842.3528034 [6] Moritz Beller, Hongyu Li, Vivek Nair, Vijayaraghavan Murali, Imad Ahmad, Jürgen Cito, Drew Carlson, Ari Aye, and Wes Dyer. 2023. Learning to Learn to Predict Performance Regressions in Production at Meta. In 2023 IEEE/ACM International Conference on Automation of Software Test (AST). 56–67. doi:10.1109/AST58925.2023.00010 [7] Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1 (1995), 289–300. [8] Stephen M. Blackburn, Zixian Cai, Rui Chen, Xi Yang, John Zhang, and John Zigman. 2025. Rethinking Java Performance Analysis. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 (Rotterdam, Netherlands) (ASPLOS ’25). Association for Computing Machinery, New York, NY, USA, 940–954. doi:10.1145/3669940.3707217 [9] Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanović, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. 2006. The DaCapo benchmarks: java benchmarking development and analysis. In Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (Portland, Oregon, USA) (OOPSLA ’06). Association for Computing Machinery, New York, NY, USA, 169–190. doi:10.1145/1167473.1167488 [10] Verónica Bolón-Canedo, Laura Morán-Fernández, Brais Cancela, and Amparo Alonso-Betanzos. 2024. A review of green artificial intelligence: Towards a more sustainable future. Neurocomputing 599 (2024), 128096. [11] Déaglán Connolly Bree and Mel Ó Cinnéide. 2021. Automated refactoring for energy-aware software. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 689–694. [12] Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (2001), 5–32. doi:10.1023/A:1010933404324 [13] Jinfu Chen, Weiyi Shang, and Emad Shihab. 2022. PerfJIT: Test-Level Just-in-Time Prediction for Performance Regression Introducing Commits. IEEE Transactions on Software Engineering 48, 5 (2022), 1529–1544. doi:10.1109/TSE.2020.3023955 [14] Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 785–794. doi:10.1145/2939672.2939785 [15] Diego Costa and Artur Andrzejak. 2018. CollectionSwitch: a framework for efficient and dynamic collection selection. In Proceedings of the 2018 International Symposium on Code Generation and Optimization (Vienna, Austria) (CGO ’18). Association for Computing Machinery, New York, NY, USA, 16–26. doi:10.1145/3168825 [16] Diego Costa, Artur Andrzejak, Janos Seboek, and David Lo. 2017. Empirical Study of Usage and Performance of Java Collections. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (L’Aquila, Italy) (ICPE ’17). Association for Computing Machinery, New York, NY, USA, 389–400. doi:10.1145/3030207.3030221 [17] Luís Cruz. 2021. Green Software Engineering Done Right: a Scientific Guide to Set Up Energy Efficiency Experiments. https://luiscruz.github.io/2021/10/ 10/scientific-guide.html [18] Zishuo Ding, Jinfu Chen, and Weiyi Shang. 2020. Towards the use of the readily available tests from the release pipeline as performance tests: are we there yet?. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 1435–1446. doi:10.1145/3377811.3380351 [19] Martin Fowler. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley. [20] Martin Fowler. 2018. Refactoring: improving the design of existing code. Addison-Wesley Professional. [21] Martin Fowler. 2018. This is the online catalog of refactorings to support the refactoring book 2nd edition. https://refactoring.com/catalog/ [22] GitHub. 2026. GitHub REST API documentation. https://docs.github.com/en/rest?apiVersion=2026-03-10 [23] Samir Hasan, Zachary King, Munawar Hafiz, Mohammed Sayagh, Bram Adams, and Abram Hindle. 2016. Energy profiles of Java collections classes. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE ’16). Association for Computing Machinery, New York, NY, USA, 225–236. doi:10.1145/2884781.2884869 [24] Jan Hauke and Tomasz Kossowski. 2011. Comparison of values of Pearson’s and Spearman’s correlation coefficients on the same sets of data. Quaestiones geographicae 30, 2 (2011), 87–93. [25] Abram Hindle. 2015. Green mining: a methodology of relating software change and configuration to power consumption. Empirical Software Engineering 20, 2 (2015), 374–409. Manuscript submitted to ACM

46

Haibo Wang, Heng Li, and Shin Hwei Tan

[26] Marc R. Hoffmann. 2026. JaCoCo is a free Java code coverage library distributed under the Eclipse Public License. https://github.com/jacoco/jacoco [27] Liusheng Huang, Huaping Chen, Xun Wang, and Guoliang Chen. 2000. A fast algorithm for mining association rules. Journal of Computer Science and Technology 15, 6 (2000), 619–624. [28] Asif Imran, Tevfik Kosar, Jaroslaw Zola, and M. Fatih Bulut. 2024. Towards Sustainable Cloud Software Systems through Energy-Aware Code Smell Refactoring. In 2024 IEEE 17th International Conference on Cloud Computing (CLOUD). 223–234. doi:10.1109/CLOUD62652.2024.00034 [29] Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. 2012. Understanding and detecting real-world performance bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (Beijing, China) (PLDI ’12). Association for Computing Machinery, New York, NY, USA, 77–88. doi:10.1145/2254064.2254075 [30] Doohwan Kim, Jang-Eui Hong, Ilchul Yoon, and Sang-Ho Lee. 2018. Code refactoring techniques for reducing energy consumption in embedded computing environment. Cluster computing 21, 1 (2018), 1079–1095. [31] Donald E. Knuth. 1974. Structured Programming with go to Statements. ACM Comput. Surv. 6, 4 (Dec. 1974), 261–301. doi:10.1145/356635.356640 [32] Michael P LaValley. 2008. Logistic regression. Circulation 117, 18 (2008), 2395–2399. [33] Lizhi Liao, Jinfu Chen, Heng Li, Yi Zeng, Weiyi Shang, Catalin Sporea, Andrei Toma, and Sarah Sajedi. 2021. Locating performance regression root causes in the field operations of web-based systems: An experience report. IEEE Transactions on Software Engineering 48, 12 (2021), 4986–5006. [34] Lizhi Liao, Jinfu Chen, Heng Li, Yi Zeng, Shang Weiyi, Guo Jianmei, Sporea Catalin, Andrei Toma, and Sajedi Sarah. 2020. Using black-box performance models to detect performance regressions under varying workloads: an empirical study. Empirical Software Engineering 25, 5 (2020), 4130–4160. [35] Mario Linares-Vásquez, Gabriele Bavota, Carlos Bernal-Cárdenas, Rocco Oliveto, Massimiliano Di Penta, and Denys Poshyvanyk. 2014. Mining energy-greedy API usage patterns in Android apps: an empirical study. In Proceedings of the 11th Working Conference on Mining Software Repositories (Hyderabad, India) (MSR 2014). Association for Computing Machinery, New York, NY, USA, 2–11. doi:10.1145/2597073.2597085 [36] Bo Liu, Hui Liu, Nan Niu, Yuxia Zhang, Guangjie Li, He Jiang, and Yanjie Jiang. 2025. An automated approach to discovering software refactorings by comparing successive versions. IEEE Transactions on Software Engineering 51, 5 (2025), 1358–1380. [37] Yepang Liu, Chang Xu, and Shing-Chi Cheung. 2014. Characterizing and detecting performance bugs for smartphone applications. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 1013–1024. doi:10.1145/2568225.2568229 [38] Guillermo Macbeth, Eugenia Razumiejczyk, and Rubén Daniel Ledesma. 2011. Cliff’s Delta Calculator: A non-parametric effect size program for two groups of observations. Universitas Psychologica 10, 2 (2011), 545–555. [39] Irene Manotas, Lori Pollock, and James Clause. 2014. SEEDS: a software engineer’s energy-optimization decision support framework. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 503–514. doi:10.1145/2568225.2568297 [40] Jérôme Maquoi, Maxime Cauz, Benoit Vanderose, and Xavier Devroey. 2025. Energy Codesumption, Leveraging Test Execution for Source Code Energy Consumption Analysis. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (Clarion Hotel Trondheim, Trondheim, Norway) (FSE Companion ’25). Association for Computing Machinery, New York, NY, USA, 1432–1436. doi:10.1145/3696630.3728707 [41] T. Mens and T. Tourwe. 2004. A survey of software refactoring. IEEE Transactions on Software Engineering 30, 2 (2004), 126–139. doi:10.1109/TSE. 2004.1265817 [42] Stefan Mühlbauer, Florian Sattler, Christian Kaltenecker, Johannes Dorn, Sven Apel, and Norbert Siegmund. 2023. Analysing the Impact of Workloads on Modeling the Performance of Configurable Software Systems. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 2085–2097. doi:10.1109/ICSE48619.2023.00176 [43] Adel Noureddine. 2022. PowerJoular and JoularJX: Multi-Platform Software Power Monitoring Tools. In 18th International Conference on Intelligent Environments (IE2022). Biarritz, France. [44] Bridget Nyirongo, Yanjie Jiang, Nan Niu, and Hui Liu. 2025. An Empirical Study of Software Refactorings in Real-World Open-Source Java Projects. IEEE Transactions on Software Engineering (2025). [45] OpenAI. 2026. OpenAI API Platform. https://developers.openai.com/api/docs [46] Aleksandra Orlowska, Christos Chrysoulas, Zakwan Jaroucheh, and Xiaodong Liu. 2021. Programming Languages: A Usage-based Statistical Analysis and Visualization. In Proceedings of the 4th International Conference on Information Science and Systems (Edinburgh, United Kingdom) (ICISS ’21). Association for Computing Machinery, New York, NY, USA, 143–148. doi:10.1145/3459955.3460614 [47] Jevgenija Pantiuchina, Fiorella Zampetti, Simone Scalabrino, Valentina Piantadosi, Rocco Oliveto, Gabriele Bavota, and Massimiliano Di Penta. 2020. Why Developers Refactor Source Code: A Mining-based Study. ACM Trans. Softw. Eng. Methodol. 29, 4, Article 29 (Sept. 2020), 30 pages. doi:10.1145/3408302 [48] Jae Jin Park, Jang-Eui Hong, and Sang-Ho Lee. 2014. Investigation for software power consumption of code refactoring techniques.. In SEKE. 717–722. [49] Abhinav Pathak, Y. Charlie Hu, and Ming Zhang. 2012. Where is the energy spent inside my app? fine grained energy accounting on smartphones with Eprof. In Proceedings of the 7th ACM European Conference on Computer Systems (Bern, Switzerland) (EuroSys ’12). Association for Computing Machinery, New York, NY, USA, 29–42. doi:10.1145/2168836.2168841 [50] Abhinav Pathak, Abhilash Jindal, Y. Charlie Hu, and Samuel P. Midkiff. 2012. What is keeping my phone awake? characterizing and detecting no-sleep energy bugs in smartphone apps. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services (Low Manuscript submitted to ACM

Understanding the Energy Impact of Software Refactoring

47

Wood Bay, Lake District, UK) (MobiSys ’12). Association for Computing Machinery, New York, NY, USA, 267–280. doi:10.1145/2307636.2307661 [51] Bernard Rosner, Robert J Glynn, and Mei-Ling T Lee. 2006. The Wilcoxon signed rank test for paired comparisons of clustered data. Biometrics 62, 1 (2006), 185–192. [52] Rui Rua and João Saraiva. 2024. A large-scale empirical study on mobile performance: energy, run-time and memory. Empirical Software Engineering 29, 1 (2024), 31. [53] Cagri Sahin, Furkan Cayci, Irene Lizeth Manotas Gutiérrez, James Clause, Fouad Kiamilev, Lori Pollock, and Kristina Winbladh. 2012. Initial explorations on design pattern energy usage. In 2012 First International Workshop on Green and Sustainable Software (GREENS). 55–61. doi:10.1109/ GREENS.2012.6224257 [54] Cagri Sahin, Lori Pollock, and James Clause. 2014. How do code refactorings affect energy usage?. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (Torino, Italy) (ESEM ’14). Association for Computing Machinery, New York, NY, USA, Article 36, 10 pages. doi:10.1145/2652524.2652538 [55] Cagri Sahin, Philip Tornquist, Ryan Mckenna, Zachary Pearson, and James Clause. 2014. How Does Code Obfuscation Impact Energy Usage?. In 2014 IEEE International Conference on Software Maintenance and Evolution. 131–140. doi:10.1109/ICSME.2014.35 [56] İbrahim Şanlıalp, Muhammed Maruf Öztürk, and Tuncay Yiğit. 2022. Energy efficiency analysis of code refactoring techniques for green and sustainable software in portable devices. Electronics 11, 3 (2022), 442. [57] Danilo Silva, Nikolaos Tsantalis, and Marco Tulio Valente. 2016. Why we refactor? confessions of GitHub contributors. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (Seattle, WA, USA) (FSE 2016). Association for Computing Machinery, New York, NY, USA, 858–870. doi:10.1145/2950290.2950305 [58] Marina Sokolova and Guy Lapalme. 2009. A Systematic Analysis of Performance Measures for Classification Tasks. Information Processing & Management 45, 4 (2009), 427–437. doi:10.1016/j.ipm.2009.03.002 [59] Luca Traini, Daniele Di Pompeo, Michele Tucci, Bin Lin, Simone Scalabrino, Gabriele Bavota, Michele Lanza, Rocco Oliveto, and Vittorio Cortellessa. 2021. How Software Refactoring Impacts Execution Time. ACM Trans. Softw. Eng. Methodol. 31, 2, Article 25 (Dec. 2021), 23 pages. doi:10.1145/3485136 [60] Nikolaos Tsantalis. 2026. Open source projects using RefactoringMiner diff tool for code reviews. https://github.com/tsantalis/RefactoringMiner [61] Nikolaos Tsantalis, Ameya Ketkar, and Danny Dig. 2022. RefactoringMiner 2.0. IEEE Transactions on Software Engineering 48, 3 (2022), 930–950. doi:10.1109/TSE.2020.3007722 [62] Roberto Verdecchia, June Sallou, and Luís Cruz. 2023. A systematic review of Green AI. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 13, 4 (2023), e1507. [63] Haibo Wang. 2026. Open-source Repository. Concordia University. https://github.com/HaiboWang1992/EARefactor [64] Haibo Wang, Zezhong Xing, Chengnian Sun, Zheng Wang, and Shin Hwei Tan. 2025. Towards Diverse Program Transformations for Program Simplification. Proc. ACM Softw. Eng. 2, FSE, Article FSE015 (June 2025), 23 pages. doi:10.1145/3715730 [65] Haibo Wang, Zhuolin Xu, and Shin Hwei Tan. 2025. Testing Refactoring Engine via Historical Bug Report driven LLM. In 2025 IEEE/ACM Second International Conference on AI Foundation Models and Software Engineering (Forge). 113–124. doi:10.1109/Forge66646.2025.00020 [66] Haibo Wang, Zhuolin Xu, Huaien Zhang, Nikolaos Tsantalis, and Shin Hwei Tan. 2026. Towards Understanding Refactoring Engine Bugs. ACM Trans. Softw. Eng. Methodol. 35, 5, Article 138 (April 2026), 55 pages. doi:10.1145/3747289 [67] Guoliang Zhao, Stefanos Georgiou, Safwat Hassan, Ying Zou, Derek Truong, and Toby Corbin. 2024. Enhancing Performance Bug Prediction Using Performance Code Metrics. In Proceedings of the 21st International Conference on Mining Software Repositories (Lisbon, Portugal) (MSR ’24). Association for Computing Machinery, New York, NY, USA, 50–62. doi:10.1145/3643991.3644920

Manuscript submitted to ACM

Related documents

• From Tool Invocation to Source-Mechanism Exploration: Protected White-Box DSE for Open-Source EDA#630075
• From Tool Invocation to Source-Mechanism Exploration: Protected White-Box DSE for Open-Source EDA#628213
• CG-Diff: Organizing Code Changes Around Call Graphs#1116013
• Investigating and Archiving the Scholarly Git Experience#939610
• An Empirical Study of Java Code Improvements Based on Stack Overflow Answer Edits#665574