MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads

Nanda Velugoti

Joseph Manzano

Andres Marquez

Oregon State University velugotn@oregonstate.edu

Pacific Northwest National Laboratory joseph.manzano@pnnl.gov

Pacific Northwest National Laboratory andres.marquez@pnnl.gov

Nathan Tallent

Kyle C. Hale

Pacific Northwest National Laboratory nathan.tallent@pnnl.gov

Oregon State University kyle.hale@oregonstate.edu

Abstract—Managing the energy consumption and power efficiency of parallel applications is a significant issue in both HPC environments and in the cloud. As emerging applications continue to push against the memory wall of modern machines, the growing imbalance between compute and data movement creates new opportunities to intelligently tune CPU power consumption. Unfortunately, existing frequency and voltage scaling techniques do not adequately capture fine-grained changes in memory access behavior, rendering the compute/data access imbalance invisible to the components of the system that could capitalize on it, thus leaving potential power savings on the table. In this paper, we propose M EM P OWER, a flexible, model-based approach to exposing compute/data movement imbalance that characterizes the fine-grained memory behavior of parallel workloads. This characterization then informs our automated software framework which can statically instrument the application binary with model-determined voltage/frequency transitions that balance fine-grained changes in memory access behavior with the costs of hardware transitions. Using M EM P OWER, we demonstrate a reduction in EDP of 6% to 42% on a range of HPC benchmarks with minimal impact on execution time when compared to the standard OS/hardware-managed power control mechanism. Index Terms—memory trace analysis, energy savings, power management, cost model, binary instrumentation, Linux kernel

I. I NTRODUCTION The impact of CPU power management on overall energy efficiency in the context of HPC applications is well known [1], [2], and modern systems can effectively reduce power using hardware- and OS-managed schemes. These systems all rely on some flavor of dynamic voltage and frequency scaling (DVFS), where the CPU clock rate and voltage are adjusted to modulate performance per Watt [3]. Early DVFS schemes such This research is supported by the U.S. Department of Energy (DOE) through the Office of Advanced Scientific Computing Research’s “Advanced Memory to Support Artificial Intelligence for Science” (76125) and “Orchestration for Distributed & Data-Intensive Scientific Exploration” (77765). Pacific Northwest National Laboratory is operated by Battelle for the DOE under Contract DEAC05-76RL01830. Additional support is provided by the U.S. National Science Foundation (NSF) via awards CNS-2533773 and CNS-2310422.

1.6 Normalized EDP

arXiv:2608.20734v1 [cs.DC] 21 Aug 2026

M EM P OWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads

1.4

Baseline (schedutil) EDP Best manual selection MemPower EDP

1.2 1.0 0.8 0.6 8 10 11 13 15 17 18 20 22 23 25 27 29 30 32 P-state

Fig. 1. Relative energy-delay product (EDP) of miniVite when executed at all CPU Power State configurations. M EM P OWER outperforms both the manual power selection scheme and the OS baseline (schedutil). Lower is better.

as Intel SpeedStep [4] allowed the OS to manage voltage and frequency using a hardware-exposed interface, for example with Linux CPUFreq [5]. However, more recent Intel chips (since Skylake) have shifted DVFS control back to the hardware via SpeedShift, with the stated goal of mitigating software latency and allowing more responsiveness to instantaneous microarchitectural events [6]. In this regime, software can provide hints, and the hardware can respond to them appropriately in conjunction with its own collected metrics. With increasing hardware control, and hints that machine learning models will be tightly integrated in the hardware power control loop for coming chips [7], it is paramount to ensure that the hardware has sufficient information to effectively modulate energy efficiency. The state-of-the-art automatic power management scheme, schedutil [8], performs well on memory intensive applications that are bandwidth bound, as it analyzes the workload’s instruction mix on demand to select an effective CPU Power State. However, it does not adequately exploit the opportunities

© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

in memory intensive applications that are latency-bound [9], better capitalize on memory-bound code regions compared to [10], as it lacks information about the dynamic access trace. the hardware’s available heuristics. When a core is already stalled waiting on memory, reducing its M EM P OWER builds on MemGaze [12], a low-level memory frequency costs little performance. This means that the more trace analysis tool, to characterize the memory access patterns latency-bound an application is, the greater the opportunity of code regions of a target binary. To perform frequency for transitioning into a lower CPU Power State to save energy management, we use Intel’s DVFS mechanism [25] by imwith minimal performance cost. plementing a userspace library and a custom OS kernel driver For example, consider the parallel graph workload, (see §IV) for Linux. miniVite [11], which is known to have memory intensive code We evaluate M EM P OWER on a range of micro- and applicaregions [12]. We modified this application so that memory tion benchmarks and demonstrate a reduction in overall EDP intensive code regions can be executed at manually controlled of 6% to 42%. Our contributions are as follows: frequencies/voltages using software-directed DVFS (ACPI P• We investigate the idea of selecting a lower CPU Power states). Figure 1 shows the energy delay product (EDP) for this State for memory intensive portions of an HPC application workload, capturing both energy consumption and execution and empirically show that executing these portions at a time on a modern Alder Lake machine. We compare the modern single lower CPU Power State (i.e., manual selection) Linux approach for power control (schedutil, discussed can result in better overall EDP compared to the baseline further in §II) as baseline with a simple approach that sets schedutil. a static P-state for memory intensive functions for the entire • We propose M EM P OWER , a model-based power manrun of the workload and M EM P OWER (described in §III-C). agement framework that profiles the dynamic memory Surprisingly, there are several fixed DVFS states that do better behavior of a target binary and uses high-level memory than the default dynamic OS/hardware coordination mechanism metrics to extract memory intensive code regions. An as seen in Figure 1, with P-state 30 (second highest CPU Power analytical model then chooses efficient CPU Power State State) being the best setting (red hatched bar). transitions for those regions, which M EM P OWER injects The reason for this effect is that the baseline scheme treats into the binary to produce a more energy-efficient one. all memory accesses as equivalent and fails to distinguish • We evaluate M EM P OWER on memory intensive HPC between those with short and long latencies. However, idenbenchmarks to show that our model-based solution reduces tifying code regions with latency-bound accesses for a given the overall EDP by up to 42% when compared to the workload would either require access latencies to be exposed baseline approach. by hardware performance counters or would require detailed microarchitectural and cache simulation. Building on either II. BACKGROUND approach runs the risk of sacrificing generality. Here we provide background on hardware-managed dynamic Motivated by this observation, we propose a new modelvoltage and frequency scaling, OS control of DVFS states, and based, memory-centric approach to coordinating CPU Power the tools we build on. States that focuses on capturing fine-grained memory-bound DVFS: DVFS [3] aims to manage a CPU’s power behavior in a way that generalizes across workloads. Our consumption by taking advantage of the superlinear savings approach does not require complex microarchitectural modeling one can achieve by changing both voltage and frequency1 at or simulation to effectively characterize the workload in terms of memory latencies. Software-directed DVFS is certainly the same time. This is achieved using circuits that modulate the not new [13]–[16], even at the compiler level [17]. Others CPU’s supply voltage and core clock frequencies. In the ACPI have attempted to bridge the semantic gap between hardware, standard, each (V, f ) voltage–frequency pair for a processor system software, and the application [18], even with learned defines an “operational state,” with a fixed number of such models [19]. This includes leveraging phase behavior, e.g., states determined by the vendor. Modern ACPI exposes these of distributed message passing [20], [21] and the decoupling states (and more) via Collaborative Processor Performance of data access and execution [22]–[24] for energy efficiency. Control (CPPC) [26], which higher software layers use to However, prior work is limited in terms of capturing dynamic coordinate with hardware when selecting processor power memory access patterns to make efficient and granular power states [27]. Policies for choosing operational states may be implemented in hardware, firmware (BIOS), or the OS, with management decisions. (e.g., Intel’s We propose augmenting the hardware/OS with M EM P OWER, recent microarchitectures favoring hardware control 2 Hardware-Controlled Performance States, HWP ). In all cases, an analytical model paired with a framework that transforms vendors expose DVFS control via an extended control register the target application binary into one that is more energyset (MSRs on Intel and AMD). Under the older SpeedStep efficient. The goal of the model is to analyze the memory access patterns of a given application at fine granularity (i.e., individual interface, software sets P-states directly [4], while with HWP and architecturally visible loads made by the application) and (introduced in Skylake) the OS provides “hints” and the modulate the CPU clock frequency based on memory behavior 1 This is due to the switching equation determining the dynamic power of a at runtime. Our key insight is that a model-based framework CPU, P = αCV 2 f . 2 See Section 16.4 of the Intel system programming guide [25]. is uniquely suited to extracting high-level information that can

Density

Normalized EDP

hardware ultimately selects the operational state. Throughout 1.2 this paper we identify a CPU Power State by the frequency 1.0 ratio (aka. P-state ranging from 8 till 32) written to the 0.8 IA32_PERF_CTL MSR rather than by its ACPI index, so distUpdateLocalCinfo Baseline (schedutil) larger numbers denote higher frequency; note that this is the 0.6 distSumVertexDegree All functions (incl. P-state overhead) fillRemoteCommunities All functions (excl. P-state overhead) reverse of the ACPI convention, in which P0 is the highest0.4 32 30 29 27 25 23 22 20 performance state. P-state Linux software control: In the Linux kernel, intel-pstate (for Intel CPUs) and acpi-cpufreq Fig. 2. Illustrating the complexity of capturing the CPU Power State behavior (generic) are two kernel drivers that abstract away the and its effects on the EDP of an application (miniVite). This shows that even platform-level hardware details associated with DVFS control. when ideal P-states are selected for code regions, the overall EDP can be Each driver comes with a set of frequency governors which worse due to P-state transition overhead. Lower is better. integrate deeply with the kernel’s scheduler and provide finer-granularity information at lower latency to direct Schedutil 0.4 the hardware [8]. The intel-pstate active mode has Manual powersave and performance governors, whereas both 0.2 passive mode and the acpi-cpufreq driver have access to 0.0 extra governors: conservative, ondemand, userspace, 5 10 15 20 25 30 35 P-state and schedutil. We use the acpi-cpufreq driver with the userspace governor for manual instrumentation. Fig. 3. Manual vs. schedutil CPU Power State distribution for miniVite. Software-driven DVFS: We aim to use application information derived from our modeling framework to drive direct DVFS transitions from software. For example, to switch from CPU Power State Pi to Pj manually, we can write a order of microseconds. To illustrate this effect, we take value Pj to a model-specific register (MSR) of the target core the miniVite workload, which clearly has opportunity for (namely, the IA32_PERF_CTL MSR). After some delay (at better CPU Power State selection as shown in Figure 1, and least 6046 ns [18]), the hardware will complete the transition instrument three different code regions (functions) individually: to a new frequency and/or voltage. Because this MSR is only distUpdateLocalCinfo, distSumVertexDegree, accessible by privileged software, a user program can only and fillRemoteCommunities. We run the entire initiate these transitions by delegating via the kernel interface. workload with one function instrumented at a time, at On Linux, this can be achieved with mechanisms like system CPU Power States ranging from 20 to 32. We also calls, ioctl, sysfs interface, and netlink sockets. In our run the same application once more, only this time we case, the final MSR manipulation is performed in a custom select the ideal CPU Power State for each function kernel driver that exposes an interface to userspace. §IV-B that resulted in the best EDP from previous runs, i.e., CPU Power States 25, 30, and 32 are chosen for details both our userspace and kernel-resident components. MemGaze: MemGaze [12] is a low-level memory analysis distUpdateLocalCinfo, distSumVertexDegree, tool that dynamically captures memory access traces of an and fillRemoteCommunities functions respectively. application binary. Instead of capturing these at the software Figure 2 shows the normalized EDP (Energy Delay Product) level, MemGaze uses Intel’s modern hardware tracing capability of miniVite with all of these configurations, and we observe (enabled with the PT_WRITE instruction) [25] to capture that, even though we empirically choose the ideal CPU Power memory access information during a program run. MemGaze States for each individual function, the overall EDP (dashed instruments the target binary using DynInst [28] to inject line) is worse than the baseline (schedutil). The reason for this behavior is that the simple approach of PT_WRITE instruction before every memory access. It then runs the instrumented binary to collect detailed memory traces, picking the best (empirical) CPU Power State manually based which inform various sophisticated analyses that measure on traces does not consider the overhead of placing a CPU footprint growth rate (i.e., average footprint per memory access; Power State transition call within the code, especially when see Table I), reuse distance, spatio-temporal locality, and more. the transition overhead can quickly dominate the workload’s runtime. For example, placing a transition inside a loop or A. Identifying Profitable Power State Changes inside a recursive function will incur the transition cost more Designing a model that chooses an optimal CPU Power than necessary. We also counted the transitions that occurred in State setting for a given code region using an aggregate order to estimate their performance overhead; Figure 2 shows performance metric is challenging as the relationship that if this overhead were avoided, the overall EDP (dotted between the CPU Power State setting and any type of line) would be lower than the baseline. Moreover, we consider memory behavior is complex. Furthermore, any model must this to be a conservative estimate of the EDP because it does account for the fact that state changes incur a switching not incorporate the energy savings obtained due to the reduced cost that counteracts any benefit; these can be on the execution time of the program.

M EM P OWER is a power management framework that finds efficient CPU Power State transitions fine-tuned for memory intensive code regions by profiling the target application’s memory behavior and automatically injecting these transitions at the binary level to achieve good performance per Watt.

Footprint Growth (FPG)

0.6 0.4 0.2 0.0

A. Design Goals The primary objective of M EM P OWER is to capitalize on memory-bound code regions to uncover power control opportunities not available to the hardware/OS-coordinated default mechanism. We lay out the following goals in designing our framework: Goal G1: Improved EDP. M EM P OWER should improve energy-delay product compared to the baseline. Goal G2: Identify latency-bound code regions. M EM P OWER should capture irregular memory intensive access behavior. Goal G3: Balance. M EM P OWER should balance energy utility and fidelity with cost amortization. Goal G4: Minimal burden. To reduce the user’s programming burden, M EM P OWER must automatically instrument and manage the target application’s power settings. Assumptions: We assume that applications execute on Intel x86 CPUs (with PT_WRITE support), running Linux, as M EM P OWER’s CPU Power State management mechanism is implemented as a Linux kernel driver along with acpi-cpufreq driver. Even though M EM P OWER is application agnostic, we assume HPC workloads, focusing on applications at the node-level that leverage OpenMP for intranode parallelism. We first discuss how to achieve Goal G2. B. Characterizing Memory and Power Behavior Critical to M EM P OWER is the ability to identify opportune regions of code to modulate CPU Power State based on memory-bound behavior. The underlying memory trace is provided by MemGaze. In order to choose an effective CPU

Access Class total irregular constant strided

0.8

32 30 29 27 26 25 23 22 20 17 15 13 11 10 9 806 807 12341 12342 12367 12570 12571 12899 12903 12904 12907 12910 12927 13022 13023 13024 13025 13026 13027 13029 13226 13227 13228 13229 13230 13235 13236 14015 14016

III. M EM P OWER D ESIGN

1.0

CPU Power State

The effect motivates a more principled model that can statically predict advantageous (not necessarily optimal) power transitions and injection locations for a given workload based on its memory behavior such that the overall CPU Power State transition overhead is amortized over time. Moreover, Figure 3 compares the CPU Power States that are chosen by schedutil (baseline) and manual (i.e., static/fixed) CPU Power State selection approaches (interpolated using KDE) for miniVite workload and shows that schedutil chooses suboptimal CPU Power States compared to manual selections which are the highest (32) and the ideal (30) CPU Power States. This explains why static CPU Power State setting outperforms schedutil as described in Figure 1.

Basic Block

0

20

40

60

Power State count

80

Area of opportunity for picking a lower Power State

Fig. 4. Areas of opportunity (red hatched rectangles) to pick an efficient (lower) CPU Power State for memory intensive code regions of miniVite workload.

Power State based on memory behavior, we first need metrics that capture both the temporal reuse and operational intensity of a given code region within the target application. We achieve this using footprint growth rate (i.e., average footprint per access) and access class (i.e., memory accesses that are of type irregular or constant or strided) where the former metric captures memory intensity and the latter captures locality. The intuition is that if a particular code region displays irregular memory accesses that exhibit high footprint growth, then that code region is likely to be latency-bound due to stalls waiting on the memory system, and therefore a prospect for executing at a lower CPU Power State. In other words, if the core pipeline cannot extract sufficient instruction-level parallelism to hide DRAM accesses caused by cache pressure, the workload will be in a memory latency-bound rather than bandwidth-bound regime. Choosing a lower CPU Power State for code regions with poor locality is worthwhile because the existing power management scheme (schedutil), which we primarily compare against, does not have such information about fine-grained memory accesses made by an application. To identify areas of opportunity to improve energy consumption when compared to the baseline approach (schedutil), we analyze the memory trace of miniVite collected by running it through MemGaze and extract footprint growth rate categorized based on the access class. We also use Linux’s perf tool to collect the application power trace to see the frequency distribution of CPU Power State transitions that occurred over

app binary energyoptimized binary

MemGaze

mem. trace

FP analysis & pruning

Cost model

MemPower Modeling & Analysis Framework

DynInst-based injector

1000

A

20

N

B 400

target code regions + target P-states

5

MemPower instrumentation engine and runtime system

Fig. 5. High-level design of M EM P OWER.

C

6000

Memory access from basic block N

I

D

G 3000

Count

J

Access by block I followed by J

50

7000

time, directed by schedutil. For both traces, we attribute the data to basic blocks in the application binary, which lets us correlate memory metrics with power transitions. In Figure 4, the top half shows the footprint growth rate distribution per basic block based on the access class for miniVite. This is a distribution because we combine footprint growth rates of all code regions that contain the corresponding basic block. The bottom half of Figure 4 shows the CPU Power State transition distribution per basic block of miniVite represented as a heatmap, where the hue represents the number of transitions that have occurred for each available CPU Power State. Furthermore, we can also observe that there are clear areas of opportunity (highlighted with hatched rectangles on the heatmap) for picking a lower CPU Power State to save energy, as the code regions in those areas have high footprint growth rate distribution of type irregular access compared to schedutil. In those regions, schedutil used only two CPU Power States, allowing us to choose a lower CPU Power State to reduce energy.

H

F

200

E

10000

Inner Outer Clusters (Code regions)

Fig. 6. Clustering basic blocks to form code regions based on the memory access trace.

Given these inputs we describe our CPU Power State instrumentation cost metric (C): C=

P ×L T × Firr

(1)

D. Selecting Memory Intensive Sequences

To identify memory intensive regions, we parse the application’s memory trace obtained from MemGaze, which gives us fine-grained access information. A memory trace from MemGaze is organized as time-series data where each trace element has an instruction pointer of the LOAD instruction, the basic block that contains the IP, the memory address that was accessed, and time. We use this trace to construct a C. Model Design and Description weighted, directed graph that compresses the trace (which At a high level, to choose effective CPU Power State is total-order time series data) into a representation of the most transitions, we propose a model where we analyze the target frequent dynamic basic-block transitions. In the graph, each program’s memory access pattern to identify code regions with node represents a basic block, edges indicate a “followed both irregular memory accesses and high footprint growth by” relationship (i.e., access from source block occurred rate. The model then uses this information to calculate the before access from destination block in the trace) and edge overall cost (Equation 1) and uses that cost to choose a CPU weight represents access intensity which is a count of all such Power State such that the cost would be minimum. Figure occurrences in the trace. Constructing the graph at basic block 5 depicts all the steps performed by M EM P OWER at a high granularity captures sequencing while avoiding unnecessarily level. M EM P OWER takes a binary (target application) as input large graphs. and generates memory traces by profiling the application using We cluster the graph (using the Louvain Method [29]) MemGaze. M EM P OWER then uses these traces (along with the to identify all sub-graphs that are tightly connected to each original binary) to perform footprint growth analysis (§III-B) other, resulting in clusters of basic-block sequences we call and candidate selection (§III-D) to generate code regions along code regions. Each code region is attributed with its footprint with target CPU Power States. Finally, M EM P OWER uses this growth rate along with its access type (Goal G2). To explain information to inject target CPU Power State calls for each this visually, Figure 6 depicts a graph where edges from one specified code region in the original binary. node (basic block) to another represent access intensity and The model is broken into two pieces: (1) selecting the clustered sub-graphs represent code regions. Moreover, since memory intensive code regions of the application, and (2) the Louvain method produces clusters with proper hierarchical trade-off decision, i.e., placing the CPU Power State transition relationships, we can identify inner vs. outer sequences, which calls in appropriate locations in the program such that the lets us select code-region boundaries that amortize CPU Power transition overhead is amortized over time. State transition cost. Available parameters to the model are described in Table I. Given these clusters (code regions), we need a way to filter

TABLE I T ERMINOLOGY AND D ESCRIPTION

Calling Context Tree Transition Cost Level Cycles

Symbol Description F Average footprint (unique memory accesses) per memory access of a program, computed per code region. Classified into three subcategories: Firr Rate of accesses made in an irregular pattern, e.g., pointer chasing. Fstr Rate of accesses separated by a distance k (where k ≥ 1). Fcon Rate of accesses to scalar data within a stack frame or to global data. CR A subsection of the program represented as a set of basic blocks, annotated with footprint growth rate. Constructed from last branch record (LBR) events captured during memory trace collection CCT by MemGaze. It is incomplete as it lacks invocation counts, which cannot be inferred from sampled LBR data alone. P Time required to change from the current to the desired P-state. L Call depth of the target function (or code region) in a CCT. Total time (in cycles) spent executing a function, excluding time in child functions (exclusive T metric).

out all those code regions that are not beneficial to execute at a lower CPU Power State, i.e., code regions with relatively low irregular memory access footprint. Code regions with regular access patterns (constant or strided) have better locality than irregular ones and are therefore poor candidates for execution at a lower CPU Power State.. To do this we construct a new metric, Relative Footprint Growth in Equation 2, that can be used to identify the code regions with low irregular footprint growth rate and filter them out of the candidate code regions for further instrumentation. The relative footprint growth metric for the Firr can be calculated as follows: Firr Relative Firr = (2) Firr + Fstr + Fcon

user mode

Term Footprint Growth Rate Irregular Strided Constant Code Region

Instrumented binary libmempower

Freq. change API calls ioctl interface

MemPower kernel driver (Linux) MEMPOWER RUNTIME

Intel CPU

MSR rd/wr

Fig. 7. M EM P OWER Architecture.

E. Cost-benefit Analysis and a binary instrumentation tool to inject CPU Power State As part of the model we need to compute the cost of choosing transition calls into the target binary (§IV-C). an effective CPU Power State for a code region. Since every node in the CCT is attributed with a code region, we compute this cost for each node using Equation 1. To map a cost to an effective CPU Power State, we define cost thresholds such that A. Model Implementation a CPU Power State is chosen if the computed cost falls within We implement M EM P OWER’s model (§III) as a Python a particular threshold range for a node in CCT . We construct framework that takes memory traces and LBR data (which are these threshold ranges empirically based on the benchmarks we generated by MemGaze) as input and outputs the code regions evaluated. For example, if the computed cost is within a range with corresponding CPU Power State transitions. (900 to 1000), then we choose CPU Power State 30. If the Initially, M EM P OWER invokes a graph clustering tool cost is higher than all available cost thresholds at a given node, we skip that node for instrumentation, and move to its parent. (implemented in C) to analyze the input memory trace and This is repeated until we reach the root node. We essentially generate candidate code regions as described in §III-D. Then, end up with code regions with corresponding efficient CPU a Python script takes the LBR data and the generated code regions as input to construct a CCT where each node represents Power State transition calls (Goal G3). a candidate code region. Finally, M EM P OWER’s cost benefit IV. I MPLEMENTATION model (§III-E), implemented in Python, iterates over this CCT In this section, we describe the implementation details to select efficient CPU Power States for each code region. of M EM P OWER and its subcomponents: the cost model Thus, the final output of M EM P OWER’s model is a set of code implementation (§IV-A), a userspace API along with a kernel regions (i.e., start and end instruction pointers) along with the driver to enable manual CPU Power State transition (§IV-B), predicted CPU Power State value for each region.

TABLE II M EM P OWER USERSPACE API. API call

Description

mempow_init() mp_set_pstate_min() mp_set_pstate_max() mp_set_pstate_eff(int p) mempow_deinit()

Initialize M EM P OWER Transition to the lowest CPU Power State Transition to the highest CPU Power State Transition to the given CPU Power State Tear down M EM P OWER

1 2 3

4 5

6

// in main mempow_init(); mp_set_pstate_eff(p); // manual PState ctrl // ... mp_set_pstate_max(); // return to max freq mempow_deinit(); Fig. 8. Code snippet demonstrating our C userspace API.

B. Userspace Library and Kernel Driver We implement a light-weight userspace library, specified in Table II, that interfaces with our custom Linux driver to perform on-demand CPU Power State transition calls while running a program. The mempow_init() and mempow_deinit() functions perform setup and cleanup, respectively, coordinating with the custom kernel driver to perform required CPU Power State transitions. The mp_set_pstate_min(), mp_set_pstate_max() and mp_set_pstate_eff() functions implement the minimum, maximum, and effective CPU Power State transition functionality respectively. Here, minimum means the lowest available CPU Power State, maximum means the highest available CPU Power State, and effective means the predicted CPU Power State (output of the model described in §III). We take inspiration from TURBO Diaries [18] work to implement a light-weight Linux kernel module to perform CPU Power State transitions on demand. A CPU Power State transition can be triggered by writing the desired CPU Power State value to IA32_PERF_CTL MSR (Model Specific Register) on the desired CPU core. We implement this in a kernel driver that exposes an ioctl() interface to userspace; the driver takes the desired CPU Power State as input and performs the transition by writing that value to the executing core’s IA32_PERF_CTL MSR.

b) Call the mp_set_pstate_max() function after (or at the very end of) the candidate region to perform maximum CPU Power State transition. 3) Call the mempow_deinit() function after (or at the very end of) the main() function to perform library cleanup. In Step 2.(b), the CPU Power State is reset from effective frequency to maximum frequency such that the portions of the program that are not part of candidate functions can be executed at maximum frequency as they are filtered out by the selection process in §III-D. Figure 8 illustrates how application code is modified after injecting userspace library function calls. Lines 2, 3, 5, and 6 represent the appropriate userspace library calls that are injected into a target application after performing the aforementioned injection process. Line 3 is where the effective CPU Power State transition is performed. In the case of M EM P OWER, we implement a DynInst based binary instrumentation tool (written in C++) to inject appropriate CPU Power State transition calls directly into the target binary based on the output of M EM P OWER’s model component (§IV-A). Availability: Source code for M EM P OWER is available at https://github.com/nandavelugoti/MEMPOWER. V. E VALUATION

C. Binary Instrumentation We instrument appropriate CPU Power State transition calls defined in §IV-B. This can be done in two ways: at the source level (if source is available) or at the binary level. We use static binary instrumentation to evaluate all the benchmarks for our work to demonstrate that using M EM P OWER does not require programmers to recompile their code (Goal G4). However, if a developer for any reason prefers to insert P-state transition calls directly into the code, they can do so using our userspace library, which is also documented in Table II. For both cases, the injection process is as follows: 1) Call the mempow_init() function before (or at the very beginning of) the main() function to initialize the library. 2) For each candidate code region: a) Call the mp_set_pstate_eff() function before (or at the very beginning of) the candidate region to perform effective CPU Power State transition.

In evaluating M EM P OWER, we seek to answer the following questions: • How effective is attributing footprint metrics to a code region to make better choices for specific CPU Power States relative to the HW/firmware? (§V-A) • To what degree does M EM P OWER capitalize on memory intensive workloads, and how does it handle compute intensive workloads? (§V-B) • How does M EM P OWER compare against the default mechanisms across all workloads? (§V-C) • How does each component of M EM P OWER contribute to overall application performance? (§V-D) Experimental setup: We evaluate our experiments on the system specified in Table III. We compile our OpenMPbased benchmarks with GCC (v11.5). We also use MemGaze (v1.0) for profiling memory behavior of the benchmarks. We run the benchmarks with 8 threads and pin the OpenMP threads to performance cores using the OMP_NUM_THREADS

when compared to the baseline (schedutil). The workload size is constant across all tests, which is Class B (average execution time is around 60 to 70 seconds). Component Description CG, MG, and SP show the best results with 40% to 42% CPU 12th Gen Intel® Core™ Processors reduction of overall EDP when compared to the baseline. This Number of Cores 16 (8 P-cores and 8 E-cores) is because all the applications have large memory intensive DRAM 128 GB sections: conj_grad in CG; norm2u3 and mg3p in MG; OS Linux 6.0 and x_solve, y_solve and z_solve in SP. Thus, when Frequency driver acpi-cpufreq the CPU Power States are lowered for these regions, the cost Default policy schedutil of transition overhead amortizes over time and M EM P OWER M EM P OWER policy userspace + M EM P OWER driver capitalized on this opportunity to reduce the EDP. In the case of FT, even though it has a significant number of irregular memory intensive sections, it also has regions TABLE IV B REAKDOWN OF MINI V ITE ’ S CODE REGIONS ( ATTRIBUTED TO FUNCTIONS ) with intense floating point operations which can be run at ALONG WITH THE CONSTANT, STRIDED AND IRREGULAR MEMORY higher CPU Power States to reduce the execution time. Even ACCESSES . so, M EM P OWER was able to reduce relative EDP by 20%. This is because for code regions such as these, where operational Code Region Fcon Fstr Firr P State intensity is high, M EM P OWER’s candidate selection process distUpdateLocalCinfo 0.00 0.60 0.60 30 will not tag these regions as candidates for effective CPU Power fillRemoteCommunities 0.00 0.40 0.05 32 distSumVertexDegree 0.00 0.41 0.62 30 State instrumentation. Therefore, M EM P OWER sets the CPU distBuildLocalMapCounter 0.90 0.39 0.78 29 Power State to the maximum value for these code regions, thus reducing the overall execution time (i.e., reduction in EDP). Similarly, IS also has irregular memory access regions resulting and OMP_PROC_BIND environment variables. Finally, we run in a 10% reduction in EDP. Both BT and LU are memory each benchmark 10 times and use the arithmetic mean of our intensive with strided memory access patterns resulting in measurements to present data. better locality; using M EM P OWER reduced the relative EDP by 10-15% compared to the baseline. A. Memory Behavior vs. CPU Power State EP has only 6% reduction in EDP compared to the baseline. In order to understand the effectiveness of using memory This is because EP is a highly parallel and compute intensive metrics (footprint growth and access class) to pick an workload. Since M EM P OWER recognizes that EP does not efficient CPU Power State for the memory intensive code have memory intensive code regions with irregular accesses region of a workload, Table IV shows the CPU Power State (§III-D), it sets the CPU Power State to the maximum available picked by M EM P OWER (process detailed in §III-E) for each frequency by default. This means running the workload at the memory intensive code region (distUpdateLocalCinfo, max available CPU Power State is the best option. fillRemoteCommunities, distSumVertexDegree, In addition to NAS Parallel benchmarks, we also evaluate and distBuildLocalMapCounter) of HPCG, in Figure 10, to showcase M EM P OWER’s ability miniVite along with the corresponding memory metrics to capitalize on sparse and irregular memory behavior to (Fcon , Firr , and Fstr ). For the region with highest Firr , pick an efficient CPU Power State resulting in an 18% distBuildLocalMapCounter, M EM P OWER chose CPU reduction in EDP relative to the baseline. This is because Power State 29 (third highest CPU Power State), and for HPCG’s workload consists of four major subroutines: sparse the region with lowest Firr , fillRemoteCommunities, matrix-vector multiplication (SpMV), symmetric Gauss-Seidel CPU Power State 32 (maximum) is chosen. Figure 1 shows (SymGS), vector updates (WAXPBY) and dot product (DDOT), the effectiveness of this policy by comparing M EM P OWER’s where SpMV has irregular memory access pattern, SymGS relative EDP with the baseline (schedutil) setting and the has a recursive access pattern (i.e., current iteration compute manual selection of all possible CPU Power States. depends on data from the previous iteration) and the rest have B. Effectiveness of M EM P OWER streaming/strided access patterns. Thus M EM P OWER chose To understand the degree to which M EM P OWER capitalizes lower CPU Power State (29) for SpMV, SymGS and maximum on the opportunity for picking an efficient CPU Power State value (32) for WAXPBY and DDOT. The inputs to HPCG are: based on the memory characteristics of a given workload, we nx = 104, ny = 104, nz = 104, and rt = 60 (seconds). evaluate NAS Parallel Benchmark [30] and HPCG [31] which In summary, M EM P OWER was able to capture the memory contain workloads with varying degrees of memory/compute characteristics for a given workload and choose an efficient intensity and access patterns, and we discuss the impact on CPU Power State scheme, resulting in reduction in overall overall EDP for each workload over baseline and M EM P OWER. application EDP, of up to 42%, compared to the baseline. FurFigure 9 shows the normalized EDP of the NAS parallel thermore, in all the NAS workloads, M EM P OWER performed benchmark suite scaling across all available CPU Power States as well as or better than the best manual selection. TABLE III E XPERIMENT SETUP.

Normalized EDP

app = CG

app = EP

app = FT

app = IS

app = LU

app = MG

app = SP

2.0 1.5 1.0 0.5

Normalized EDP

app = BT

2.0 1.5 1.0 0.5

8 1011131517182022232527293032 8 1011131517182022232527293032 8 1011131517182022232527293032 8 1011131517182022232527293032

P-state

P-state

Baseline (schedutil) EDP

P-state

Best manual selection

P-state

MemPower EDP

Fig. 9. Relative EDP of NAS Parallel Benchmarks when executed at all CPU Power State configurations. Highlighted bars show where manually picked frequencies do better than the baseline. Lower is better.

0.9

Normalized EDP

Normalized EDP

3.5

0.8 0.7 0.6

Baseline (schedutil) EDP Best manual selection MemPower EDP

8 10 11 13 15 17 18 20 22 23 25 27 29 30 32 P-state

Fig. 10. Normalized EDP of HPCG when executed at all CPU Power State configurations. Lower is better.

C. M EM P OWER vs. Other Policies We evaluated M EM P OWER on the following benchmarks: NAS Parallel Benchmarks, HPCG (§V-B), and miniVite (§V-A). These workloads contain different data structures: graphs in miniVite, arrays in NAS parallel benchmark’s IS, and matrices in rest of the workloads. Furthermore, these workloads also contain different memory access patterns, i.e., irregular accesses in miniVite, IS, and CG; strided accesses in FT, LU, MG, and SP; constant accesses in EP. Figure 11 summarizes the best cases of energy utilization and time costs resulting from our model when compared to the OS (schedutil) baseline, with 6% to 42% reduction in overall EDP (bars with left Y-axis). The geometric mean of all the benchmarks (last set of bars in Figure 11) is a 20% reduction in EDP. We observe that in all cases, M EM P OWER meets or exceeds the

3.0 2.5 2.0 1.5 1.0

1.4 Normalized Execution Time

4.0

1.0

1.2 1.0 0.8 0.6 0.4 0.2

0.5

mV BT CG EP FT IS LU MG SP HPCG Geo 0.0 Norm EDP (Bars & Left Y-axis) MemPower Norm Exec Time (Circles & Right Y-axis) Best manual selection Baseline (schedutil) EDP Intel Active (Powersave) Baseline (schedutil) Exec Time Intel Active (Performance)

Fig. 11. Comparing normalized EDP and execution times of M EM P OWER vs. Intel active mode vs. schedutil vs. best manual CPU Power State selection policies across all benchmarks. The last set of bars and circles are the geometric means (EDP and execution time) of all benchmarks. Lower is better.

best manual selection (Goal G1). In addition to schedutil and manual CPU Power State selection policies, we also compare M EM P OWER with Intel active mode policy which, unlike schedutil, has the capability to use boost/turbo frequency ranges (3.3GHz to 6GHz) but lacks the capability of choosing a static CPU Power State (one can only set upper and lower bounds). Intel active mode has two governors: performance and powersave. In both cases, M EM P OWER outperforms them by 2× in terms of EDP, because active mode’s use of turbo frequencies increases overall energy consumption significantly even though it lowers execution time relative to the other policies. Since EDP is a combined metric (product of energy and time),

Normalized EDP

Improving M EM P OWER: Intel’s hardware analyzes the instruction mix of a running program to choose a P-state. However, based on the observations from M EM P OWER, we speculate that incorporating auxiliary information such as cache misses collected at runtime (using the existing performance counters such as LONGEST_LAT_CACHE.MISS and LLC_MISSES) in addition to instruction mix would significantly improve hardware’s ability to choose an efficient P-state as it would allow MemPower MemPower w/o CandidateSelection the hardware to recognize latency-bound memory behavior. MemPower w/o CandiateSelection or CostModel Furthermore, since M EM P OWER is a framework as well as a Baseline (schedutil) EDP CG EP FT IS LU MG SP HPCG tool, it can be extended to incorporate other memory metrics that can capture spatial locality, memory contention and any other memory related metric to achieve desired performanceFig. 12. Evaluating the EDP of M EM P OWER along with its components enabled/disabled with respect to the baseline. Lower is better. energy trade-off. Another aspect of improving M EM P OWER is to enhance the cost model itself by either incorporating other memory metrics to the cost formula (Equation 1) or replacing it can mask the performance degradation of an application while it entirely. In addition, a programmer can configure a custom selecting a lower CPU Power State. To address this concern, cost threshold policy to the cost model depending on their we have included the normalized execution times (circles with needs. right Y-axis) for all policies across all benchmarks. We can Developer impact: Even though M EM P OWER does not see that the performance degradation for M EM P OWER policy require code recompilation and is application agnostic, it across all benchmarks is less than 3% (geometric mean). requires a programmer to profile and instrument the target application binary. For the benchmarks used in this paper, M EM P OWER typically takes three to five minutes of developer D. M EM P OWER Component Analysis time for each individual application. Once done, future runs do To explore the effectiveness of M EM P OWER’s components, not require any instrumentation for the same input. Furthermore, we evaluate M EM P OWER on all benchmarks with various the system administrator needs to install the M EM P OWER’s configurations: 1) M EM P OWER without the candidate selection; kernel driver. Making M EM P OWER completely transparent to 2) M EM P OWER without the candidate selection and the cost the user and integrating it at the system level (OS or hardware) model; and 3) M EM P OWER without any change. Figure 12 would encourage wide-spread adoption. shows the normalized EDP of all these configurations relative to Limitations: A key limitation of M EM P OWER is that the baseline, and we can observe that turning any of the M EM - it currently works only with specific hardware, namely Intel P OWER’s subcomponents off would result in poor performance x86 CPUs with PT_WRITE support, as it is built on top of compared to M EM P OWER as a whole. For configuration 1, the MemGaze. In terms of other architectures like ARM, MemGaze overall EDP is worse than M EM P OWER’s because the code can be modified to support MMIO-based STM (System Trace regions that are supposed to be filtered out by the candidate Macrocell) or the ITM (Instrumentation Trace Macrocell) selection process instead qualify for instrumentation, increasing interface to collect the memory traces. the overall overhead due to CPU Power State transitions. OpenMP workloads: All the evaluations (§V) of M EM Similarly, for configuration 2, where both candidate selection P OWER are done on OpenMP enabled HPC workloads as we are and the cost model are turned off (i.e., CPU Power States trying to optimize EDP at the node level. Though out of scope are chosen manually), the EDP is much worse than that for our paper, it would be interesting to study M EM P OWER of M EM P OWER because of the increased CPU Power State on workloads at scale using MPI and we consider this to be transition overhead along with inefficient placement of those future work. transition calls. One other possible configuration, M EM P OWER VII. R ELATED WORK with candidate selection but without the cost model, is an invalid configuration, because the cost model is what assigns At the hardware level, DVFS [3] enables dynamic scaling a CPU Power State to each selected region. of voltage and frequency in CPU cores. Kumar et al. [32] introduced single-ISA heterogeneous multicore architectures— the basis for today’s P-core/E-core designs—and later showed VI. D ISCUSSION that performance can be improved [33]. At the OS level, there Key takeaways: 1) hardware is not aware of high-level are workload scheduling policies such as scheduling on “wimpy” memory characteristics, like footprint growth, which opens up cores [34], PIE [35], POW [36], and PShifter [37]. At the opportunities for efficient power management at the software compiler level, many works have proposed algorithms [38], and level and 2) amortizing the cost of a CPU Power State transition models/frameworks such as DAE [23], PULSE [39] that focused is key in order to achieve efficient EDP. on reducing application power utilization. At the library/runtime

1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 mV BT

level, runtime systems such as PART [40], Adagio [41], [42], COUNTDOWN [20], UPSCavenger [43], and online predictors [44] were proposed to balance performance and energy usage. At the application level, similar to M EM P OWER, Magklis et al. [13] proposed a profile-driven approach where an application is profiled to identify code regions for which efficient power transitions are applied. However, there are a few key differences when compared to M EM P OWER: 1) their approach profiles an application over multiple “training” runs whereas M EM P OWER requires the application to be run once to obtain the memory trace; 2) their approach characterizes code regions as L+F (Loops and Functions) which are coarse grained whereas M EM P OWER code regions can be more fine-grained, i.e., code blocks with arbitrary start and end IP addresses; and 3) their approach is evaluated on a simulator whereas M EM P OWER is evaluated on real hardware. Similarly, PowerDial [16] augments a target application and manages power using a heartbeat based power control system. Another work used customized JIT runtime to improve energy efficiency of an application [14]. Unlike M EM P OWER, these works require an online component (apart from an instrumented binary) to manage power. Apart from the works mentioned above, there are others that have used memory characterization but not in the context of power/energy optimization. For example, using miss ratio curves (MRCs) to characterize storage workloads [45], custom page replacement policies for memory intensive applications [46], using memory footprint to improve LLM fine-tuning [47], [48], and characterizing memory behavior of Google’s TCMalloc memory allocator at warehouse scale [49]. In contrast to all the studies referenced in this section, M EM P OWER is a novel, profile-driven, model-based power management framework that improves applications’ performance per Watt. VIII. C ONCLUSION We presented M EM P OWER, a novel, memory-centric modeling framework that performs profile-based analysis to identify and actuate effective DVFS transitions for HPC workloads. We demonstrated that M EM P OWER can save up to 42% in total EDP compared to the default OS/hardware coordinated mechanism. Our approach imposes minimal burden on users, and in principle can be applied to a broader set of workloads. In future work, we plan to extend our model further by incorporating other memory metrics such as reuse distance or MPKI (misses per kilo-instruction) and further investigate the EDP trade-off in MPI enable applications. We also envision the development of more sophisticated governors for the kernel. We plan to extend our memory characterization model to improve aspects such as deciding when to offload memory intensive tasks, e.g., in architectures that employ near-data processing. R EFERENCES [1] T. Ilsche, S. Schrader, and R. Schöne, “Optimizing idle power of hpc systems: Practical insights and methods,” in 2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops), 2024, pp. 19–25.

[2] H. Zhang and H. Hoffmann, “Maximizing performance under a power cap: A comparison of hardware, software, and hybrid techniques,” in Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS ’16. New York, NY, USA: Association for Computing Machinery, Mar. 2016, p. 545–559. [Online]. Available: https://doi.org/10.1145/2872362.2872375 [3] M. Weiser, B. Welch, A. Demers, and S. Shenker, “Scheduling for reduced CPU energy,” in Proceedings of the 1st USENIX Conference on Operating Systems Design and Implementation, ser. OSDI ’94. USENIX Association, Nov. 1994. [Online]. Available: https://doi.org/10.5555/1267638.1267640 [4] I. Corporation, “Enhanced intel® speedstep® technology for the intel® pentium® m processor,” Mar. 2004. [Online]. Available: https://download.intel.com/design/network/papers/30117401.pdf [5] R. J. Wysocki, “CPU performance scaling,” 2017. [Online]. Available: https://docs.kernel.org/admin-guide/pm/cpufreq.html [6] J. Doweck, W.-F. Kao, A. K.-y. Lu, J. Mandelblat, A. Rahatekar, L. Rappoport, E. Rotem, A. Yasin, and A. Yoaz, “Inside 6th-generation Intel core: New microarchitecture code-named Skylake,” IEEE Micro, vol. 37, no. 2, pp. 52–62, 2017. [7] R. Piersma, T. Bhuiyan, T. A. Khan, and S. Sethumadhavan, “Reverse engineering DVFS mechanisms,” in Proceedings of the IEEE International Symposium on Hardware Oriented Security and Trust, ser. HOST ’25, May 2025, pp. 111–122. [Online]. Available: https://dx.doi.org/10.1109/HOST64725.2025.11050037 [8] The Linux Kernel Development Community, “Schedutil — The Linux Kernel documentation,” https://docs.kernel.org/scheduler/schedutil.html, 2026, accessed: 2026-02-05. [9] W. D. Gropp, D. K. Kaushik, D. E. Keyes, and B. F. Smith, “Understanding the parallel scalability of an implicit unstructured mesh CFD code,” Argonne National Lab., IL (US), Tech. Rep., 2000. [10] A. Kashi, N. Koukpaizan, H. Lu, M. Matheson, S. Oral, and F. Wang, “Scaling the memory wall using mixed-precision - HPGMxP on an exascale machine,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC ’25. New York, NY, USA: Association for Computing Machinery, Nov. 2025, p. 281–297. [Online]. Available: https://doi.org/10.1145/3712285.3759877 [11] S. Ghosh, M. Halappanavar, A. Tumeo, A. Kalyanaraman, and A. H. Gebremedhin, “Minivite: A graph analytics benchmarking tool for massively parallel systems,” in 2018 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS), ser. PMBS ’18, Nov. 2018, pp. 51–56. [Online]. Available: https://doi.org/10.1109/PMBS.2018.8641631 [12] O. O. Kilic, N. R. Tallent, Y. Suriyakumar, C. Xie, A. Marquez, and S. Eranian, “MemGaze: Rapid and effective load-level memory trace analysis,” in Proceedings of the 24th IEEE International Conference on Cluster Computing, ser. CLUSTER ’22. IEEE, Sep. 2022, pp. 484–495. [13] G. Magklis, M. Scott, G. Semeraro, D. Albonesi, and S. Dropsho, “Profile-based dynamic voltage and frequency scaling for a multiple clock domain microprocessor,” in Proceedings of the 30th Annual International Symposium on Computer Architecture, ser. ISCA ’03. IEEE, Jun. 2003, pp. 14–25. [Online]. Available: https: //dx.doi.org/10.1109/ISCA.2003.1206985 [14] Q. Wu, V. Reddi, Y. Wu, J. Lee, D. Connors, D. Brooks, M. Martonosi, and D. Clark, “A dynamic compilation framework for controlling microprocessor energy and performance,” in Proceedings of the 38th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’05. IEEE, Nov. 2005. [Online]. Available: https://dx.doi.org/10.1109/MICRO.2005.7 [15] P. Cicotti, A. Tiwari, and L. Carrington, “Efficient speed (ES): Adaptive DVFS and clock modulation for energy efficiency,” in Proceedings of the 2014 IEEE International Conference on Cluster Computing. IEEE, Sep. 2014, pp. 158–166. [Online]. Available: https://dx.doi.org/10.1109/CLUSTER.2014.6968750 [16] H. Hoffmann, S. Sidiroglou, M. Carbin, S. Misailovic, A. Agarwal, and M. Rinard, “Dynamic knobs for responsive power-aware computing,” in Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, ser. ASPLOS XVI. New York, NY, USA: Association for Computing Machinery, Mar. 2011, p. 199–212. [Online]. Available: https://doi.org/10.1145/1950365.1950390

[17] F. Xie, M. Martonosi, and S. Malik, “Compile-time dynamic voltage scaling settings: opportunities and limits,” in Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation, ser. PLDI ’03. New York, NY, USA: Association for Computing Machinery, May 2003, p. 49–62. [Online]. Available: https://doi.org/10.1145/781131.781138 [18] J.-T. Wamhoff, S. Diestelhorst, C. Fetzer, P. Marlier, P. Felber, and D. Dice, “The TURBO diaries: Application-controlled frequency scaling explained,” in Proceedings of the 2014 USENIX Annual Technical Conference. Philadelphia, PA: USENIX Association, Jun. 2014, pp. 193–204. [Online]. Available: https://www.usenix.org/conference/atc14/ technical-sessions/presentation/wamhoff [19] D. Mukherjee, S. Hachem, J. Bao, C. Madsen, T. Ma, S. Ghose, and G. Agha, “CRAVE: Analyzing cross-resource interaction to improve energy efficiency in systems-on-chip,” in Proceedings of the 20th European Conference on Computer Systems, ser. EuroSys ’25. New York, NY, USA: Association for Computing Machinery, Mar. 2025, p. 59–75. [Online]. Available: https://doi.org/10.1145/3689031.3717498 [20] D. Cesarini, A. Bartolini, P. Bonfà, C. Cavazzoni, and L. Benini, “COUNTDOWN: a run-time library for application-agnostic energy saving in MPI communication primitives,” in Proceedings of the 2nd Workshop on AutotuniNg and ADaptivity AppRoaches for Energy Efficient HPC Systems, ser. ANDARE ’18. New York, NY, USA: Association for Computing Machinery, 2018. [Online]. Available: https://doi.org/10.1145/3295816.3295818 [21] M. Y. Lim, V. W. Freeh, and D. K. Lowenthal, “Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs,” in Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, ser. SC ’06. New York, NY, USA: Association for Computing Machinery, Nov. 2006. [Online]. Available: https://doi.org/10.1145/1188455.1188567 [22] A. Jimborean, K. Koukos, V. Spiliopoulos, D. Black-Schaffer, and S. Kaxiras, “Fix the code. don’t tweak the hardware: A new compiler approach to voltage-frequency scaling,” in Proceedings of the Annual IEEE/ACM International Symposium on Code Generation and Optimization, ser. CGO ’14. New York, NY, USA: Association for Computing Machinery, Feb. 2014, p. 262–272. [Online]. Available: https://doi.org/10.1145/2581122.2544161 [23] K. Koukos, D. Black-Schaffer, V. Spiliopoulos, and S. Kaxiras, “Towards more efficient execution: a decoupled access-execute approach,” in Proceedings of the 27th International ACM Conference on Supercomputing, ser. ICS ’13. New York, NY, USA: Association for Computing Machinery, Jun. 2013, p. 253–262. [Online]. Available: https://doi.org/10.1145/2464996.2465012 [24] K. Koukos, P. Ekemark, G. Zacharopoulos, V. Spiliopoulos, S. Kaxiras, and A. Jimborean, “Multiversioned decoupled access-execute: the key to energy-efficient compilation of general-purpose programs,” in Proceedings of the 25th International Conference on Compiler Construction, ser. CC ’16. New York, NY, USA: Association for Computing Machinery, Mar. 2016, p. 121–131. [Online]. Available: https://doi.org/10.1145/2892208.2892209 [25] Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3 (3A, 3B & 3C): System Programming Guide, Intel Corporation, Oct. 2025. [26] UEFI Forum, Inc., “Advanced configuration and power interface (acpi) specification, version 6.6,” Unified Extensible Firmware Interface (UEFI) Forum, Inc., May 2025, accessed: 2025-12-16. [Online]. Available: https://uefi.org/sites/default/files/resources/ACPI_Spec_6.6.pdf [27] L. K. D. Project, “Acpi cppc sysfs interface,” Linux Kernel Organization, accessed: 2025-12-16. [Online]. Available: https://docs.kernel.org/adminguide/acpi/cppc_sysfs.html [28] A. R. Bernat and B. P. Miller, “Anywhere, any-time binary instrumentation,” in Proceedings of the 10th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools, ser. PASTE ’11. New York, NY, USA: Association for Computing Machinery, Sep. 2011, p. 9–16. [Online]. Available: https://doi.org/10.1145/2024569.2024572 [29] S. Ghosh, M. Halappanavar, A. Tumeo, A. Kalyanaraman, H. Lu, D. Chavarrià-Miranda, A. Khan, and A. Gebremedhin, “Distributed louvain algorithm for graph community detection,” in Proceedings of the 32nd IEEE International Parallel and Distributed Processing Symposium, ser. IPDPS ’18, May 2018, pp. 885–895. [Online]. Available: https://doi.org/10.1109/IPDPS.2018.00098 [30] D. Bailey, E. Barszcz, B. J.T, B. D.S, C. R.L, D. D, F. R.A, P. Frederickson, L. T.A, R. Schreiber, H. Simon, V. Venkatakrishnan, and W. K, “The

nas parallel benchmarks,” International Journal of High Performance Computing Applications, vol. 5, pp. 63–73, 09 1991. [31] J. Dongarra and M. A. Heroux, “Toward a new metric for ranking high performance computing systems,” University of Tennessee and Sandia National Laboratories, Tech. Rep. SAND2013-4744, Jun. 2013. [32] R. Kumar, K. I. Farkas, N. P. Jouppi, P. Ranganathan, and D. M. Tullsen, “Single-isa heterogeneous multi-core architectures: The potential for processor power reduction,” in Proceedings of the 36th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO 36. USA: IEEE Computer Society, Dec. 2003, p. 81. [33] R. Kumar, D. M. Tullsen, P. Ranganathan, N. P. Jouppi, and K. I. Farkas, “Single-isa heterogeneous multi-core architectures for multithreaded workload performance,” in Proceedings of the 31st Annual International Symposium on Computer Architecture, ser. ISCA ’04. USA: IEEE Computer Society, Mar. 2004, p. 64. [Online]. Available: https://doi.org/10.1145/1028176.1006707 [34] T. Hruby, H. Bos, and A. S. Tanenbaum, “When slower is faster: On heterogeneous multicores for reliable systems,” in Proceedings of the 2013 USENIX Annual Technical Conference, ser. USENIX ATC ’13. San Jose, CA: USENIX Association, Jun. 2013, pp. 255–266. [Online]. Available: https://www.usenix.org/conference/atc13/technicalsessions/presentation/hruby [35] K. Van Craeynest, A. Jaleel, L. Eeckhout, P. Narvaez, and J. Emer, “Scheduling heterogeneous multi-cores through performance impact estimation (pie),” in Proceedings of the 39th Annual International Symposium on Computer Architecture, ser. ISCA ’12. USA: IEEE Computer Society, Jun. 2012, p. 213–224. [Online]. Available: https://doi.org/10.1145/2366231.2337184 [36] D. A. Ellsworth, A. D. Malony, B. Rountree, and M. Schulz, “POW: System-wide dynamic reallocation of limited power in HPC,” in Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, ser. HPDC ’15. New York, NY, USA: Association for Computing Machinery, Jun. 2015, p. 145–148. [Online]. Available: https://doi.org/10.1145/2749246.2749277 [37] N. Gholkar, F. Mueller, B. Rountree, and A. Marathe, “PShifter: feedbackbased dynamic power shifting within HPC jobs for performance,” in Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing, ser. HPDC ’18. New York, NY, USA: Association for Computing Machinery, Jun. 2018, p. 106–117. [Online]. Available: https://doi.org/10.1145/3208040.3208047 [38] C.-H. Hsu and U. Kremer, “The design, implementation, and evaluation of a compiler algorithm for cpu energy reduction,” in Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation, ser. PLDI ’03. New York, NY, USA: Association for Computing Machinery, May 2003, p. 38–48. [Online]. Available: https://doi.org/10.1145/781131.781137 [39] Y. Tang, S.-s. Lee, A. Bhattacharjee, and A. Khandelwal, “PULSE: Accelerating distributed pointer-traversals on disaggregated memory,” in Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ser. ASPLOS ’25. New York, NY, USA: Association for Computing Machinery, Mar. 2025, p. 858–875. [Online]. Available: https://doi.org/10.1145/3669940.3707253 [40] C.-h. Hsu and W.-c. Feng, “A power-aware run-time system for high-performance computing,” in Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, ser. SC ’05. IEEE Computer Society, Nov. 2005. [Online]. Available: https://doi.org/10.1109/SC.2005.3 [41] B. Rountree, D. K. Lowenthal, S. Funk, V. W. Freeh, B. R. de Supinski, and M. Schulz, “Bounding energy consumption in large-scale MPI programs,” in Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, ser. SC ’07. New York, NY, USA: Association for Computing Machinery, Nov. 2007. [Online]. Available: https://doi.org/10.1145/1362622.1362688 [42] B. Rountree, D. K. Lownenthal, B. R. de Supinski, M. Schulz, V. W. Freeh, and T. Bletsch, “Adagio: making DVS practical for complex HPC applications,” in Proceedings of the 23rd International Conference on Supercomputing, ser. ICS ’09. New York, NY, USA: Association for Computing Machinery, Jun. 2009, p. 460–469. [Online]. Available: https://doi.org/10.1145/1542275.1542340 [43] N. Gholkar, F. Mueller, and B. Rountree, “Uncore power scavenger: a runtime for uncore power conservation on HPC systems,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC ’19. New York, NY, USA:

Association for Computing Machinery, Nov. 2019. [Online]. Available: https://doi.org/10.1145/3295500.3356150 [44] M. Curtis-Maury, J. Dzierwa, C. D. Antonopoulos, and D. S. Nikolopoulos, “Online power-performance adaptation of multithreaded programs using hardware event-based prediction,” in Proceedings of the 20th Annual International Conference on Supercomputing, ser. ICS ’06. New York, NY, USA: Association for Computing Machinery, Jun. 2006, p. 157–166. [Online]. Available: https://doi.org/10.1145/1183401.1183426 [45] J. Wires, S. Ingram, Z. Drudi, N. J. A. Harvey, and A. Warfield, “Characterizing storage workloads with counter stacks,” ser. OSDI’14. USA: USENIX Association, Oct. 2014, p. 335–349. [46] M. Wu, S. Isaacman, and A. Bhattacharjee, “Characterizing emerging page replacement policies for memory-intensive applications,” in Proceedings of the 2024 IEEE International Symposium on Workload Characterization, ser. IISWC ’24, Sep. 2024, pp. 284–294. [Online]. Available: https://doi.org/10.1109/IISWC63097.2024.00033 [47] T. Wang, X. Chen, K. Li, T. Cao, J. Ren, and Y. Zhang, “JENGA: enhancing LLM long-context fine-tuning with contextual token sparsity,” in Proceedings of the 2025 USENIX Annual Technical Conference, ser. USENIX ATC ’25. USA: USENIX Association, Jul. 2025. [Online]. Available: https://doi.org/10.5555/3768039.3768047 [48] K. Huang, S. Zhu, M. Zhai, L. Zheng, K. Lei, J. He, Y. Jin, and J. Zhai, “mtuner: accelerating parameter-efficient fine-tuning with elastic tensor,” in Proceedings of the 2025 USENIX Conference on Usenix Annual Technical Conference, ser. USENIX ATC ’25. USA: USENIX Association, Jul. 2025. [Online]. Available: https://doi.org/10.5555/3768039.3768046 [49] Z. Zhou, V. Gogte, N. Vaish, C. Kennelly, P. Xia, S. Kanev, T. Moseley, C. Delimitrou, and P. Ranganathan, “Characterizing a memory allocator at warehouse scale,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, ser. ASPLOS ’24. New York, NY, USA: Association for Computing Machinery, Apr. 2024, p. 192–206. [Online]. Available: https://doi.org/10.1145/3620666.3651350

Related documents

• MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads#659289
• MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads#657153
• FMER: An Energy-Efficient Error Recovery Methodology for SRAM-Based FPGA Designs#190632
• VARA: A Voltage-Aware ReRAM-Based Accelerator for Energy-Efficient Computing#819869
• VARA: A Voltage-Aware ReRAM-Based Accelerator for Energy-Efficient Computing#809873