Hybrid Adaptive Tuning for Tiered Memory Systems Xi Wang University of California, Merced
arXiv:2604.12165v1 [cs.OS] 14 Apr 2026
Shuangyan Yang University of California, Merced
Jie Liu University of California, Merced
Jongryool Kim SK hynix
Pengfei Su University of California, Merced
Dong Li University of California, Merced Abstract
[42, 50, 56, 58, 77]. Such a system software (called the memory tiering solution) relies on dynamic memory profiling to collect statistics on page access frequency and recency, and detect hot pages. Based on the profiling and detection results, the memory tiering solution allocates or migrates frequently accessed memory pages (hot pages) into fast memory for high performance (called page promotion), while placing less frequently accessed pages (cold pages) into slow memory to save fast memory space (called page demotion). Problems. The memory profiling, hotness detection, and page migration in memory tiering often come with system parameters. Those parameters control memory profiling overhead and quality, and impact page migration frequency and effectiveness. The parameter configuration impacts the performance significantly. For example, Colloid [69] (a state-of-the art memory tiering solution) uses a parameter, watermark_scale_factor, to control page demotion aggressiveness and maintain available fast-memory space for page promotion. Using an appropriate configuration for this parameter improves Graph500 [52] performance by 17%, compared to using the default configuration (see Sec. 2.3). However, tuning system parameters for memory tiering is challenging, because different workloads, either coming from different execution phases of the same application or different applications, can exhibit different memory access patterns, hence demanding different parameter configurations. Existing memory tiering solutions [19, 30, 36, 39, 42, 47, 49, 50, 56–58, 62, 68–71, 75–77] use invariant configurations, which leads to suboptimal performance. Today, parameter tuning for memory tiering remains largely manual, as no systematic approach exists to automate this process across different tiering solutions. We make three key insights driving our work. First, tuning system parameters for memory tiering is a inherently “stateful” problem. The effect of a system parameter setting depends not only on workload characteristics (e.g., cache hit rates and arithmetic intensity) but also on memory tiering states. The memory tiering state refers to how pages have been allocated and migrated by a memory tiering solution. A workload un-
Memory tiering provides a cost-effective solution to increase memory capacity, utilization, and even bandwidth. Memory tiering relies on system software for memory profiling, detection of frequently accessed pages, and page migration. Such a system software often comes with system parameters. The configurations of those parameters impact application performance. We comprehensively classify system parameters, and characterize the sensitivity of application performance to them using representative memory tiering solutions. Furthermore, we introduce a lightweight and user-friendly framework PTMT, which automates tuning of parameters at runtime for various memory tiering solutions. We identify major challenges for online tuning of memory tiering. PTMT uses a hybrid “offline + online” tuning method: while the offline phase builds a performance database for online queries and reduces runtime overhead, the online phase uses reinforcement learning (customized to memory tiering) to tune. PTMT improves performance by 30%, 26%, 21%, and 14%, on four memory tiering solutions (TPP, UPM, Colloid, and AutoNUMA), compared to using the default configurations. PTMT outperforms the state-of-the-art [21] by 32% on average.
1
Introduction
Memory tiering provides a cost-effective solution to increase memory capacity, utilization, and even bandwidth. By integrating multiple memory components with various properties (e.g., capacity, latency, bandwidth, and monetary cost), memory tiering addresses the scaling challenges faced by the traditional single-tiered memory. A typical tiered-memory system comprises fast memory and slow memory where fast memory offers shorter latency or higher bandwidth but limited memory capacity, and slow memory provides larger capacity but relatively worse performance. To manage tiered memory, a system software can enable a unified address space across memory components and model each memory component as a NUMA node 1
der different memory tiering states can respond differently to the parameter tuning, even if the workload has the same characteristics. Second, the combination of many parameter configurations and the essentially unbounded space of workload states creates a rather large search space to decide the optimal configuration. Building a general model aiming to navigate the space for all applications can be ineffective to decide the configurations. Third, considering that characteristics and memory-tiering states of a workload evolve over time, the problem naturally lends itself to reinforcement learning (RL), which is designed to adapt to changing environments and learn effective decisions online. In this paper, we introduce Parameter Tuning for Memory Tiering (PTMT), a framework that enables automatic tuning of system parameters across diverse memory tiering solutions. PTMT faces multiple challenges discussed as follows. Challenges. First, the process of deciding parameter configurations must be lightweight to make runtime-tuning feasible. In memory tiering, the time intervals for periodical memory profiling and page migration are typically every few seconds [21, 25, 27, 28, 35] to accommodate changing memory access patterns in applications. Consequently, the time overhead of deciding parameter configurations must be on the same scale or even smaller. Exhaustive search or frequent sampling of the search space is therefore not feasible. Second, although RL is promising for adapting to changing environments and deciding parameter configurations, RL convergence can be slow due to the large state and action space inherent in memory tiering. How to accelerate RL convergence to timely respond to changing memory access patterns is challenging. Third, parameter tuning must account for how different application inputs shape memory access behavior. Inputs often influence an application’s control-flow paths, which in turn affect memory-level parallelism and ultimately the frequency and recency of memory accesses. Efficiently capturing these input-dependent effects at runtime is challenging. Solutions. To address the above challenges, PTMT uses a hybrid “offline + online” tuning approach. The offline phase, in essence, builds a “performance database” where there are a number of workload states associated with parameter configurations and aftermath performance. During the online phase, the performance database is queried using the application’s current workload state (WS) to find a parameter configuration that leads to the best performance on a similar WS. To handle a diverse set of application inputs that lead to WS outliers, PTMT uses an RL model that automatically learns a policy to decide the parameter configurations. Using the hybrid approach, the time overhead of deciding the parameter configuration is small. The offline phase clusters WSs by similarity, such that finding a group of similar workloads at runtime is just a matter of finding a nearest cen-
troid among those clusters. Within the cluster, WSs are further organized and indexed to enable efficient lookup of the most similar WSs. In addition, using the performance database, we build a pre-trained model as the RL model such that the model convergence can be faster. We adopt an application-specific solution to design PTMT, meaning both the performance database and the RL model are tailored to each application. This specialization enables PTMT to identify higher-quality parameter configurations, yielding a 20% performance gain while also reducing modelconstruction cost by 11× on average, compared to a general approach (see Sec. 6.3). Contributions. We summarize the main contributions as follows. • We comprehensively classify system parameters and characterize the sensitivity of application performance to them using four representative memory tiering solutions (AutoNUMA [36], Colloid [69], TPP [50], and UPM [14]). • We identify key technical challenges in enabling automatic tuning of system parameters for memory tiering. We introduce PTMT, which is the first lightweight and user-friendly parameter-tuning framework for memory tiering. PTMT requires no changes to applications or the operating system (OS) and can be applied across diverse memory-tiering solutions. The design of PTMT answers multiple questions to tune memory tiering and addresses unique challenges unseen in any other tuning frameworks, e.g., how to avoid OS modification for memory tiering? Should we use an application-specific solution or a general solution for memory tiering? • By adaptively tuning system parameters, PTMT achieves 14%, 21%, 30%, and 26% performance improvements on AutoNUMA, Colloid, TPP, and UPM respectively, compared to using the default parameter configurations. Compared with the state-of-the-art parameter-tuning approach [21], PTMT delivers a 32% performance improvement on average.
2
Understanding Parameter Tuning in Memory Tiering 2.1 Memory Profiling There are three common memory profiling methods. PTE scanning-based profiling tracks page accesses by manipulating ACCESS bits in page table entries (PTEs). By repeatedly scanning PTE to detect the change of ACCESS bit, we can detect memory accesses. This profiling method is used in Linux’s memory swapping mechanism to maintain active and inactive lists for tracking page hotness. This method is commonly used in memory tiering [24, 30, 34, 38, 49, 55, 58]. NUMA hint faults-based profiling employs NUMA scanning that partitions the whole virtual address space into memory regions, and profile them in order by manipulating 2
Table 1: Memory tiering solutions and their system parameters. Solutions
Memory Profiling Mechanism Parameters
AutoNUMA
NUMA Scanning + Hint Fault
Colloid
NUMA Scanning + Hint Fault
TPP
NUMA Scanning + Hint Fault
UPM
HW Event Sampling
Hotness Detection Mechanism
Mechanism
Page Migration Parameters
scan_size_mb scan_period_max_ms scan_period_min_ms scan_delay_ms scan_size_mb scan_period_max_ms scan_period_min_ms scan_delay_ms scan_size_mb scan_period_max_ms scan_period_min_ms scan_delay_ms
Hint Fault Latency
hot_threshold
NUMA Hint Fault kswapd
watermark_scale_factor promote_rate_limit
Hint Fault Latency Fast/Slow Mem Latency Balancing
hot_threshold
NUMA Hint Fault kswapd
watermark_scale_factor promote_rate_limit
PTE Scanning LRU-based Active List
None
NUMA Hint Fault kswapd
watermark_scale_factor demote_scale_factor
sample_period
Sampled Access Count
hot_threshold
move_pages() syscall
page_migration_interval
PRESENT bit. With the bits set in a memory region, any page access in the region triggers a hint page-fault, which allows the kernel to identify access recency. After a time interval, NUMA scanning moves on to another memory region to profile. This profiling method is used in recent efforts on memory tiering [19, 36, 39, 50, 68, 69, 75]. Performance counter-based profiling tracks memory accesses by sampling hardware performance counters available in modern CPUs, which allows the user to track memory accesses at various granularities (e.g., cache line level and page level). Compared to the other two profiling methods, this method can be more lightweight. It is commonly used in the existing efforts [23, 30, 42, 54, 56].
2.2
Parameters
to promote or demote — by comparing with expected access latencies. TPP [50] relies on NUMA scanning to trigger page hint faults. However, unlike AutoNUMA and Colloid, TPP identifies hint-faulted pages as hot by verifying if these pages are present in the OS’s active list. Furthermore, TPP decouples allocation from demotion by performing demotion asynchronously in a background process. UPM [14] is a user-space page management solution developed by us. UPM uses performance counters to track memory accesses at the page level. While there are other performance counter-based solutions (e.g., HeMem [56] and MEMTIS [42]), we cannot use them due to stability issues in our evaluation. UPM shares a similar design with HeMem in memory profiling and hotness detection.
Memory Tiering
A memory tiering solution typically includes three components [19, 30, 36, 39, 42, 47, 49, 50, 56, 58, 62, 68, 69, 75, 77] (i.e., memory profiling, hotness detection, and page migration), each of which includes one or more system parameters. Table 1 summarizes the parameters in multiple representative memory-tiering solutions. We discuss them as follows. AutoNUMA [36] is the default memory tiering solution in Linux. It utilizes NUMA hint faults to decide page promotion. AutoNUMA identifies page hotness based on the hint fault latency, which is the time interval between the recent hint fault time and the recent scanning time of a page. When the hint fault latency is short enough, the page is promoted. AutoNUMA relies on Linux swapping daemon (kswapd) for page demotion. When fast memory is in shortage, kswapd uses PTE scanning to determine cold pages to demote. Colloid [69] is a state-of-the-art memory tiering solution. Integrated with AutoNUMA (Linux v6.3), Colloid prevents overloading of fast memory. Like AutoNUMA, Colloid uses NUMA hint faults and fault latency to identify page hotness and promotion. Different from AutoNUMA, Colloid launches a background kernel thread to monitor memory access latencies for both fast and slow memories. Upon NUMA hint faults, Colloid determines the migration direction — whether
2.3
System Parameters and Performance Sensitivity
We classify system parameters in the context of the three components in memory tiering. System parameters in memory profiling are often used to set a balance between profiling overhead and quality. Take NUMA scanning as an example. Frequent NUMA scanning over memory regions leads to large profiling overhead because of frequent hint faults and manipulation of PTE bits. However, frequent NUMA scanning can capture page access in a timely manner. Hence, Linux provides a system parameter, scan_size_mb, allowing users to strike the above balance. This parameter controls the size of the memory region to scan. Given a scanning time interval, a smaller region to scan leads to smaller overhead but suffers from lower profiling quality, and vice versa. System parameters in hotness detection are used to control the definition of page hotness. For example, UPM uses a system parameter hot_threshold. The number of accesses to a page is compared against it to determine page hotness. A smaller hot_threshold increases the number of candidate pages to promote but also raises migration overhead, and vice 3
versa. UPM does not change hot_threshold throughout application execution. AutoNUMA and Colloid have a similar system parameter, but dynamically adjust it according to a promotion rate limit. PTMT does not tune such a dynamic parameter, and focuses on static ones. System parameters in page migration can be used to balance fast-memory utilization and migration overhead. For page promotion, AutoNUMA and Colloid use promote_ rate_limit to control promotion traffic. For page demotion, AutoNUMA, Colloid, and TPP employ kswapd to demote pages in the inactive list. These three solutions periodically inspect free space in fast memory and wake up kswapd when the free memory falls below a threshold watermark_scale_ factor. TPP further introduces a threshold, demote_scale_ factor, to determine when the background demotion process should terminate. A higher watermark_scale_factor or demote_scale_factor enables more prompt page demotion, creating more space for page promotion but at the cost of higher page migration overhead, and vice versa. Unlike the other three memory-tiering solutions, UPM uses page_migration_interval to control page migration frequency. Specifically, UPM performs page demotion and promotion at each page_migration_interval. Instead of using kswapd for demotion, UPM uses its own thread to demote pages whose access count falls below hot_ threshold. This page demotion continues until there is enough fast memory available for hot page promotion. A smaller page_migration_interval improves responsiveness to the change of hot pages but incurs higher overhead, and vice versa. Performance sensitivity study. We study performance sensitivity to parameter configurations using representative memory tiering solutions discussed in Sec. 2.2. We study the parameters listed in Table 1, covering memory profiling, hotness detection, and page migration. We run two benchmarks, CG from the NAS Parallel Benchmarks (NPB) suite [3] and Graph500 [52], using 24 threads on an Intel Optane machine (depicted in Sec. 6.1). We use static tuning, i.e., the parameters are invariant during each execution. The results are presented in Figure 1, leading to three key observations. 1. Tuning parameters can lead to much better performance, compared with using the default parameter configurations. For example, tuning scan_size_mb improves CG performance by 15%, and tuning watermark_scale_factor improves Graph500 performance by 17%. 2. Different workloads exhibit different preferences for parameter configurations. For example, CG achieves optimal performance with scan_size_mb set to 8192MB, while Graph500 performs best with scan_size_mb set to 256MB. This disparity demands an adaptive strategy to adjust parameters based on specific workload requirements. 3. Tuning certain parameters cannot bring performance benefit. For example, tuning promote_rate_limit and NUMA scanning-related parameters (except
better
Default Configuration
Figure 1: Performance with various parameter configurations, normalized by that with the default configurations. scan_size_mb) leads to small performance variance (less than 5%). Hence, dynamic tuning of these parameters should not bring large runtime overhead to avoid performance loss. We do not employ static tuning in our design. With static tuning, a parameter configuration is pre-determined before the workload is launched and does not change throughout workload execution. Static tuning is not practical, because of three reasons. (1) There is no universal static configuration leading to the best performance for all workloads (discussed in Observation 2) and all workload inputs. (2) Static tuning cannot correctly respond to the changing memory access patterns for better performance. (3) Finding a static configuration is challenging, and exhaustive search is time-consuming.
3
Related Work and Motivation
Existing approaches for parameter tuning cannot work well for memory tiering. We classify them into three categories, each with fundamental limitations. Rule-based methods [18, 35, 40], although simple, face difficulty in designing rules that make multiple decisions and account for complex WSs. For example, AMP [35] uses a rule-based algorithm to select the best page migration policy. This policy can only be one of three options (LRU, LFU, and random) and is decided based on a single WS (page hit ratio). Heuristic-based methods [28, 29, 31, 53, 61, 67, 72, 74] rely on handcrafted optimizations, which are difficult to be scaled to tune multi-parameters. For example, Cori [28] tunes the page migration frequency based on page reuse distance. TMO [72] relies on a fixed control policy to decide how much memory to offload from each application to SSD, but the mapping from application profiling to the offload decisions is embedded in handcrafted control logic, and only works for a specific kernel implementation. Machine learning methods [21, 22, 25–28, 30, 35, 41, 48, 59] are promising but suffer from prohibitive resource require4
(1) user-space parameter-tuning that requires no changes to existing memory tiering implementations or the OS, (2) automatic decision-making, and (3) flexibility in customizing tuning objects. PTMT defines the workload state (WS), in terms of performance characterization of the workload, and memory tiering state. The memory tiering state is determined based on access traffic to fast and slow memory, such that we can capture the impact of page migration. By formulating the parametertuning problem as a stateful problem, PTMT aims to capture the relationship between the future performance, current WS, and the choice of parameter configurations. To effectively identify parameter configurations to optimize performance, PTMT relies on offline WS clustering and online RL, as depicted in Figure 2. The offline WS clustering is based on our unique observation that WSs under memory tiering with various parameter configurations can be aggregated into a set of clusters. Given an application, such clusters are built offline, using various execution phases and parameter configurations. Data points within each cluster are organized and indexed by a k-nearest neighbor (K-NN) algorithm for fast search. The above offline phase is called the clustering modeling. During the online phase, PTMT periodically collects workload information and builds WSs. Given a WS, PTMT determines whether it falls within any cluster. If a match is found, PTMT identifies the most similar data points within that cluster and then uses the parameter configuration from the data point that has the best performance. If the given WS deviates from all clusters, PTMT uses an RL model to decide the parameter configuration. To accelerate RL convergence, PTMT pre-trains the RL model offline. In general, the clustering model and RL work synergistically to guide the selection of parameter configurations. Figure 2 overviews PTMT.
User Space
Workload Collect Workload States Online
PTMT
Collect Workload States
Offline
Collector Tuner K-NN RL
Tune Parameters Kernel Space
Trainer
Load Models
K-NN
Collector Random Explorer
RL
Randomly Adjust Parameters
Memory Tiering Solutions
Figure 2: The overview of PTMT. ments and narrow applicability. For example, Kleio [25] trains an RNN-based model (i.e., LSTM) per page for memory access prediction, resulting in consumption of tens of GBs of memory and 2 hours of training per 100 models. Google’s warehouse-scale computing (WSC) [41] uses a Gaussian Process Bandit model to find the best parameter configuration, requiring one week of WSC’s memory traces to run GP bandit. IDT [21] employs RL to adjust the criteria for page demotion, while Cronus [27] uses computer vision methods for page access prediction. However, those studies focus on a single aspect of memory management, e.g., access pattern analysis [25, 26, 48], page demotion criteria [21], page movement frequency [28], or page migration policy [35], rather than comprehensive parameter tuning. Existing solutions also face challenges on system integration. Some solutions (e.g., [21, 28, 30, 35]) operate in kernel space, necessitating kernel modifications and recompilation, which is cumbersome in production environments. Some solutions (e.g., [21, 25–27, 35, 48]) focus on optimizing a specific memory management strategy rather than providing a framework applicable to memory tiering solutions with arbitrary configurations. In addition, parameter tuning has been explored in various domains [20,31,43–46,63,66,78,79], such as using GP regression and RL for database systems [43, 63, 66, 78, 79], K-NN for cloud-server energy optimization [46], and Bayesian optimization for network protocols [20]. However, the approaches in those domains do not address the unique challenges faced by memory tiering: the stateful nature of page placement, the need for lightweight runtime decisions, and the requirement to work across various memory tiering solutions without OS modifications.
4
5 5.1
Design Workload State
The WS consists of (1) workload characteristics and (2) memory tiering state. During application execution, PTMT samples WSs and decides the configurations of system parameters accordingly. The workload characteristics are represented using performance events. We select two performance events (i.e., LLC hit rate and the second-last-level cache hit) that are most correlated with performance (instruction per cycle or IPC) using Pearson correlation analysis. In our experiments, we observed that using more than two performance events does not yield significant performance improvement. The memory tiering state is designed to represent how effectively fast memory has been utilized by a memory tiering solution. We use slow (or fast) memory read and write traffic normalized by total memory traffic to represent the memory tiering state. The traffic refers to the total number of bytes
Overview
We introduce PTMT, an adaptive parameter-tuning framework for memory tiering. The goals of PTMT are multifold: 5
corresponding clusters to the execution phases. For example, the marker “×” corresponds to a data copy phase; the marker “+” corresponds to the computation of the regular-sparse, block lower triangular solution. The above results indicate that given an input to the application, WSs collected from the same execution phase, even if they are collected using different parameter configurations and have diverse memory-tiering states, tend to be clustered together. WSs collected from different execution phases have difference in both workload characteristics and memory tiering state, tend to go to different clusters. Also, using more application inputs to collect WSs, we add more clusters. In general, WSs are not uniformly distributed in the space. The reason for the above clustering results is that memory tiering is a fine-tuning process to distribute memory access traffic between fast/slow memories and avoid frequent page migration [50]. As a result, the memory tiering state for an execution phase does not change significantly across different parameter configurations, compared to the change in workload characteristics across execution phases. Application-specific clustering. For each application, we build a clustering model. We chose application-specific clustering rather than general clustering across all applications to improve the effectiveness of parameter tuning. Our evaluation shows that using application-specific clustering outperforms the general clustering by 20% across four memory tiering solutions (see Sec. 6.3). For a single application, building application-specific clustering requires 11× less time than building a general clustering model (see Sec. 6.3). The application-specific clustering is feasible because it aligns with the common deployment practices of major publiccloud providers for HPC and AI. Those vendors focus on a narrow set of applications in a pre-defined domain. This practice is especially common in the business of HPC- and AI-centric SaaS [5, 10]. For example, Amazon Web Services (AWS) provides HPC clouds [7, 9], focusing on individual domains (e.g., healthcare and life science). In this type of customized environment, a specific modeling and simulation workload (e.g., computational fluid dynamics (CFD), pharmacokinetics, clinical trial simulation, or systems biology) is frequently launched. Google Cloud Platform (GCP) provides similar services. For example, GCP provides HPC blueprints [4] for computeraided engineering (CAE) that target standard solvers such as Ansys Fluent, Siemens Simcenter STAR-CCM+, and OpenFOAM. We have similar observations in AI-centric services where cloud vendors offer catalogs of ready-to-use AI models and pre-built training and inference pipelines [2, 6]. Improving similarity match within the cluster. To narrow down the search space in a cluster, given a WS, we employ a feature-weighted K-nearest neighbors algorithm (K-NN) [51] to find data points most similar to the given WS. The similarity is defined in terms of workload characteristics and memory tiering state.
Table 2: Metrics to build WS, and the quantification of their correlation to IPC using absolute correlation coefficient (CC) based on the Pearson correlation analysis. Metric
Definition
CC
RdRs W rRs Tot_RdR L2h L3h
Slow-mem read traffic / Total read traffic Slow-mem write traffic / Total write traffic Read traffic / Total memory L2 cache hit rate Last level cache hit rate
0.82 0.76 0.98 0.94 0.96
loaded/stored within a profiling interval to build the WS. Such a memory tiering state is irrelevant to the memory footprint on fast and slow memories but instead captures the effectiveness of page migration. Ideally, most memory traffic (excluding that related to page migration) should happen on fast memory. In addition, we differentiate between read and write traffic to account for their distinct latency and bandwidth characteristics. In conclusion, at a given time t, the WS is defined as a vector shown below. Table 2 summarizes WS along with the correlation to IPC.
W S(t) = [ L2h, L3h, RdRs , W rRs , Tot_RdR ]
5.2
(1)
Clustering of Workload States
Definition of the search space. The search space is a collection of data points where each data point is a vector consisting of W S(t) at a time t, the parameter configurations to be applied Params(t), and the aftermath performance IPC(t ′ ) after applying the configurations. The search space is large. Given an application, we can use various application inputs and measure W S throughout application execution to sample the search space. Clustering WSs. We observe that WSs in data points in the search space can be clustered. Figure 3.a shows the distribution of WSs for the benchmark NPB-LU [3] (a Lower-Upper Gauss-Seidel solver) as an example. We execute LU 100 times with AutoNUMA and various parameter configurations, using the same input (Class E), and collect 6K WSs. We run the k-means [11] to cluster those WSs. To determine the appropriate number of clusters (i.e., k), we employ the elbow method [1], a standard technique to identify the point where adding more clusters provides diminishing returns in terms of the reduction of a metric, Within-Cluster Sum of Squares (WCSS). Using the elbow, we ensure that our clustering model captures sufficient granularity to distinguish between different WSs without over-fitting. Figure 3.b indicates that “4” is the optimal number of clusters for NPB-LU. We find that the WS clusters correspond to execution phases, even though WSs are collected with various parameter configurations. See Figure 4. The figure maps WSs and the 6
optimal outcome. We employ Proximal Policy Optimization (PPO) [60], a widely used and robust RL algorithm, to find the optimal policy π∗ . Why RL? RL can benefit parameter tuning for memory tiering for three reasons. First, the RL workflow aligns naturally with the stateful parameter tuning process. In memory tiering, the selection of parameter configurations affects the way WS transits from one state to another, ultimately affecting system performance. RL excels in such a context by making decisions that influence state transitions, thus optimizing the outcome over time. Second, RL’s ability to continuously learn and adapt to changing environments is crucial to respond to dynamics in WS. RL can update its decision-making processes in real time, allowing it to make better choices as WS evolves. Third, RL naturally handles scenarios with application coruns. Co-running applications create a combinatorially large search space, making it infeasible to collect sufficient data points within a reasonable time budget. By treating a set of co-running applications as a single aggregate workload, RL can learn the evolving mixed WS and continuously refine its policy at runtime. The RL model in PTMT is a lightweight three-layer MultiLayer Perceptron (see Sec. 6.7 for overhead analysis). Hence, RL effectively addresses the limitation discussed in Sec. 3.
Figure 3: k-means clustering of WSs in the benchmark NPBLU. Each dot represents a WS. Red dots are the cluster centroids. Different colors indicate different clusters. Initialization
Iterations
Figure 4: Map WSs and clusters in Figure 3 to execution phases. Each marker in Figure 4 represents about 100 points. The feature-weighted K-NN assigns different weights to different features (i.e., the metrics listed in Table 2), and calculates the distance between two data points based on the weighted Euclidean distance. To determine appropriate weights for features, we construct a decision tree using data points in the cluster, with IPC specified as the decision target. In this decision tree, the internal nodes represent features, and the leaves represent IPC predictions. After the construction, a feature importance value is calculated based on the feature’s contribution to IPC prediction. The importance value is used as the feature weight for K-NN. We use the above approach, because it differentiates features according to their importance to performance, hence allowing us to find the most similar data points.
5.3
5.3.2
We define the key elements of RL. Optimization goal is to maximize application performance by finding a policy π∗ over application’s lifetime T . π∗ = arg max π
1 T ∑ IPC(t) T t=1
(2)
State is the current environment information and RL performs an action upon it. Equation 1 defines the state. Action represents the parameter configurations that PTMT applies to different WSs to optimize performance. The action space consists of possible parameter configurations. At any time t, the action is a specific set of parameter configurations chosen from the action space, described as follows.
Reinforcement Learning
We employ RL to tune parameter configurations at runtime. 5.3.1
RL Formulation
Background
Action(t) = {(P1 (t), P2 (t), . . . , Pn (t))}
RL is a type of ML that makes decisions to achieve the best outcome by interacting with the environment and learning through an agent. Specifically, given an environment State(t) at a time step t, the agent takes Action(t), which causes a transition to the next State(t + 1). The agent then receives Reward(t + 1) for moving from State(t) to State(t + 1) based on Action(t). An episode in RL is a complete cycle of interactions that starts in an initial state and ends when a terminal state is reached. The policy π guides the action the agent performs in a specific state. RL aims to find the optimal policy π∗ that yields the highest cumulative reward to achieve an
(3)
where P1 (t), P2 (t), . . . , Pn (t) represent individual parameter tuning at t, and n is the number of parameters to tune. Reward reflects the effect of the previous action performed on the previous state. As we aim to improve performance, we directly measure the reward based on IPC. We normalize IPC such that it falls in [-1, 1]. Otherwise, the reward received by the agent is always a positive value, failing to penalize the suboptimal action. IPC(t) − IPCmin Reward(t) = 2 −1 (4) IPCmax − IPCmin 7
where IPCmin and IPCmax represent the lower and upper bounds of IPC collected offline. 5.3.3
(σ) of the distance of all data points from the centroid. If the distance between W S and the centroid is larger than a threshold (µ + 3σ), WS is considered to be outside the cluster. The threshold is statistically determined based on the assumption that the distance of the data points from the centroid follows a normal distribution, and hence 99.7% data points have a distance less than µ + 3σ [12]. WS with a distance greater than this threshold is considered an outlier.
RL Model Structure and Hyper-Parameters
The RL model consists of three layers. The first layer comprises five neurons, corresponding to the five metrics listed in Table 2. The second layer is a fully connected hidden layer with 64 neurons. The third layer varies in size, depending on the number of parameters to tune. For hyper-parameters in RL, according to the sensitivity study in Sec. 6.6, we set the discount factor to 0.9 and the learning rate to 0.01. In PPO, as Stable-Baselines3 (SB3) [13], actions are explored based on probability distributions predicted for each action by RL, without the use of an explicit exploration rate. We set the rollout buffer size (the number of steps per policy update) to 4. 5.3.4
5.4
We implement PTMT as a user-level framework without requiring OS or application modifications, making it applicable to production environments. We verified the compatibility of PTMT with Linux kernel 5.13, 6.1, and 6.3. Workload states collection. PTMT employs Intel Performance Counter Monitor (PCM) [37] and AMD Instruction Based Sampling (IBS) [15] to collect WSs. To build the performance database, for each workload, we repeatedly run it with multiple application inputs, collecting WS and IPC, and randomly setting parameter configurations at each tuning interval until 3K data points are gathered. Models. For the clustering model, PTMT uses k-means algorithm to build clusters and builds K-NN within each cluster, supported by Python’s scikit-learn. For RL model, PTMT uses PPO from Stable-Baselines3 (SB3) [13] as the policy network in RL. Before online tuning, PTMT uses BC to pre-train PPO as the initial policy in RL. Parameter adjustment. For kernel-based memory tiering solutions (AutoNUMA, Colloid, and TPP), PTMT tunes parameters by writing values to files under /sys/kernel/ debug and /proc/sys. For the user-space UPM, PTMT uses its client-server interface via command-line instructions. Overhead control. To minimize CPU resource consumption, PTMT uses a single helper thread for WS collection and parameter tuning. Both the RL training and inference processes also operate on this thread. We analyze the overhead in Sec. 6.7.
RL Convergence Acceleration
The RL model convergence in our scenario can be slow because of the large exploration space. Assuming that S is the number of possible WSs and P is the number of parameter configurations in the action space, the capacity of the exploration space is SP , which is large. Directly applying RL for online tuning degrades application performance because the agent may require numerous episodes to perform extensive explorations. To accelerate model convergence and provide a solid baseline for online tuning, we pre-train the RL model using Behavioral Cloning (BC) [64]. BC is an ML technique. Using BC, an agent learns to perform tasks by mimicking the behavior of an expert. To generate the expert dataset required for BC, we reuse our clustering model. Specifically, for each W S collected offline, we identify the cluster to which it belongs and then employ K-NN within that cluster to find the k most similar data points. Among these data points, we search for the one with the best IPC and label the corresponding Params for the given WS. After processing all WSs, we obtain optimal WS-Params pairs. Those WS-Params pairs serve as the expert dataset in BC. Using this expert dataset, the RL agent can quickly learn a policy by directly mapping optimal Params to WS without extensive exploration.
6 6.1
5.3.5
More Implementation Details
Hybrid Method
Evaluation Evaluation Methodology
Evaluation platform. We evaluate PTMT on a dual-socket machine equipped with Intel Xeon Gold 6252 @2.10 GHz processors (24 cores per socket). Each socket has 6×16GB DDR4 DRAM as fast memory, and 6×128GB Intel Optane DCPMM as slow memory. To study the impact of page migration with memory tiering and avoid cross-socket NUMA effects, we conduct all experiments on a single socket, similar to prior works [42, 50, 77]. We use GRUB mmap to limit fast memory capacity to 34GB, such that the workload memory is allocated on both fast memory and slow memory.
By default PTMT uses the clustering model, which is effective when WS measured at runtime is close to one of the centroids and falls into one of the clusters. However, when WS deviates substantially from all centroids, the clustering model cannot lead to optimal parameter configurations. In such a case, the RL model becomes more applicable. The switch between the clustering model and RL happens automatically in PTMT. To decide whether WS is outside all clusters, for each cluster, PTMT calculates the mean (µ) and standard deviation 8
Memory tiering settings. For AutoNUMA, Colloid, and TPP, we enable page promotion by setting /proc/sys /kernel/numa_balancing to 2, and enable page demotion by setting /sys/kernel/mm/numa/demotion_enabled to 1. Benchmarks. We use seven memory-intensive benchmarks summarized in Table 3. The benchmarks consist of a graph processing benchmark (Graph500 [52]), an in-memory database engine (Silo [65]), a machine learning benchmark (Liblinear [33]), and four high performance computing benchmarks (LU, SP, BT, and FT) from NPB [17]. During the offline phase, we collect WSs to generate the clustering model and RL pre-trained model using some input problems (named WSS (a)). During the online phase, we assess the effectiveness of PTMT by measuring the performance of each benchmark under two scenarios: (1) using similar input problems as in the offline phase (WSS (a)), such that WS collected online falls into the clusters built offline; (2) using largely different input problems, such that WS may randomly fall out of the clusters (named WSS (b)). Baselines. We compare PTMT performance against that of using the default parameter configurations. Each benchmark is executed using 24 threads, and execution time is measured to assess performance. Besides PTMT and the four memory tiering solutions, we evaluate NoBalance in Linux by disabling NUMA page migration, which represents static page placement. We also evaluate IDT [21], a state-of-the-art solution that uses an RL-based policy to efficiently demote cold pages to slow memory. IDT uses RL to tune page demotion criterion age_thres, which is a threshold determining whether a page is “aged” enough for demotion. Specifically, IDT uses a function of each memory region’s age as RL’s state and uses the performance (inversely proportional to the slow memory access frequency) as RL’s optimization goal to determine the appropriate age_thres. IDT is not application-specific, but pre-trained using a general micro-benchmark GUPS [8] emulating three common access patterns: uniform random, static hot set (where 90% of accesses happened to a static hot-memory region), and dynamic hot set (where the hot set changes every 150 seconds). Parameter selection for tuning. Parameters are selected based on performance sensitivity study (some results are shown in Sec. 2.3). A parameter is selected if at least one benchmark’s performance is sensitive to its configuration. Table 4 summarizes the parameters for tuning.
6.2
Table 3: Benchmarks for evaluation Benchmark
Description
WSS (a)
WSS (b)
LU SP BT FT Graph500 Liblinear Silo
Lower-Upper Gauss-Seidel solver Scalar Penta-diagonal solver Block Tri-diagonal solver Discrete 3D fast Fourier Transform Graph generation and traversal Large-scale linear classification In-memory database engine
134GB 174GB 166GB 80GB 64GB 69GB 115GB
58GB 74GB 71GB 40GB 46GB 59GB 145GB
Table 4: Parameters to tune in memory tiering Memory Tiering Solution AutoNUMA Colloid
Parameters scan_size_mb watermark_scale_factor scan_size_mb watermark_scale_factor
TPP
demote_scale_factor
UPM
hot_threshold sample_period page_migration_interval
configurations. Specifically, while the default outperforms the NoBalance baseline by an average of 47%, 31%, 21%, and 36% on AutoNUMA, Colloid, TPP, and UPM, respectively, the clustering-based tuning outperforms NoBalance by 62%, 40%, 44%, and 57% on the same memory tiering solutions. This demonstrates the effectiveness of the clustering model. We observe that some workloads are very sensitive to parameter tuning in specific memory tiering solutions. For example, compared to the default, the clustering-based tuning outperforms by 10% on SP, 43% on Graph500, and 10% on Silo in AutoNUMA and by 14% on Graph500, 10% on Liblinear, and 14% on Silo in Colloid. However, the clustering-based tuning achieves little performance improvement for BT and FT in AutoNUMA and Colloid. The best performance improvements are achieved on Graph-500 when using PTMT with AutoNUMA and Colloid. Further analysis reveals that both AutoNUMA and Colloid employ NUMA scanning and hint fault latency for memory profiling and hotness detection, and incorporate flexible page migration mechanisms. The memory profiling speed is controlled via scan_size_mb, and the page demotion aggressiveness is controlled via watermark_scale_factor. This flexibility allows PTMT to fine-tune memory profiling for quicker detection of changes in hot page set and to adjust the proactiveness of page demotion in response. This adaptability is suited to Graph500’s frequent changes in the hot page set. In all cases (except for Liblinear with UPM), PTMT does not cause performance loss compared to the default configurations. Liblinear with UPM and PTMT is the only exception. Further analysis reveals that, unlike other solutions, UPM’s page demotion appears on the critical path of page promotion, slowing down the promotion of hot pages. PTMT amplifies this issue, resulting in reduced performance.
Overall Performance
We evaluate two scenarios: (1) assessing the effectiveness of the clustering model using WSS (a), and (2) assessing PTMT’s adaptiveness using WSS (b). WSS (a). The clustering-based tuning without RL takes effect, as shown in Figure 5. PTMT improves the performance of AutoNUMA, Colloid, TPP, and UPM by 10%, 7%, 20%, and 17% on average, respectively, compared to their default 9
better
better
Figure 5: WSS (a): Performance speedup over NoBalance performance.
better
Figure 6: WSS (b): Performance speedup over NoBalance performance.
Figure 7: WSS (b): Evaluation of application-specific RL. The performance speedup is measured over the default configuration. WSS (b). Figure 6 shows that with RL, PTMT outperforms the default configurations by 14%, 21%, 30%, and 26% on average for AutoNUMA, Colloid, TPP, and UPM, respectively. PTMT automatically enables RL model to tune parameters when WSS changed. These results demonstrate PTMT effectiveness in tuning parameters across a diverse set of WSs. Compared to the default configurations, PTMT achieves performance gain by 103% for LU with TPP, while other memory tiering solutions do not have a large gain. That substantial gain is due to TPP’s more aggressive demotion mechanism, which employs an asynchronous background process. LU’s hot regions are distributed across fast and slow memories, hence calling for an effective demotion mechanism to accommodate emerging hot pages. By tuning demote_scale_ factor in TPP, PTMT optimizes demotion aggressiveness, improving page placement and overall performance. PTMT’s clustering-based tuning degrades Liblinear’s performance across all memory tiering solutions. This occurs because WSS (b) has approximately 80% of its WSs falling outside the clusters (formed using WSS (a)), indicating that
the clustering-based tuning lacks the flexibility needed to adapt to the diverse WSs in a workload.
6.3
Effectiveness of RL
Adaptiveness of RL. We compare the clustering-based, RLbased (PTMT-RL), and RL (without pre-training)-based tuning using WSS (b). Figure 6 shows that the RL-based tuning outperforms the clustering-based tuning by 9%, 21%, 19%, and 14% on average for AutoNUMA, Colloid, TPP, and UPM, respectively, indicating that RL is more effective in handling diverse WSs. RL with pre-training outperforms without pre-training by 34%, 31%, 24%, and 14% on average for AutoNUMA, Colloid, TPP, and UPM, respectively. This improvement is because the pre-training phase provides RL with prior knowledge, reducing the number of iterations needed to explore parameter configurations and develop an optimal policy. Effectiveness of application-specific RL. We compare with IDT. Since IDT is only used to tune a specific parame10
micro-benchmark. In contrast, the application-specific RL significantly improves performance: PTMT-RL outperforms IDT by 29%, 26%, 56%, and 17% for AutoNUMA, Colloid, TPP, and UPM, respectively, and outperforms RL (general pre-training) by 12%, 10%, 46%, and 12% respectively. Figure 8: (a) Memory access heatmap of Graph500. (b) Tuning demote_scale_factor using PTMT. The markers in the figure represent parameter configurations. Demotion 2 1.5 1
Generation Construction
BFS
0.5 0
0
100
200
300
Execution Time(s) (a) PTMT
400
Number of Migrated Pages
Number of Migrated Pages
7
#10
2.5
#10
2 1.5 Generation Construction 1
BFS
0.5 0
0
100
200
300
400
Analysis of Dynamic Parameter Tuning
We look deep into how the parameters are tuned at runtime by PTMT. We use Graph500 (WSS (b)) with TPP as an example, shown in Figure 8.b. We then provide an in-depth comparison of parameter tuning and page migration between PTMT and the default configuration (i.e., demote_scale_ factor = 200), shown in Figure 9. Figure 8.a shows memory access heatmap with execution time. Graph500 has three execution phases, as shown in Figure 8.b. In the generation phase where edge lists are generated, PTMT uses the default configuration and there is minor page migration. This phase is characterized with limited page reuse, and hence lack hot pages. PTMT recognizes this and keeps the parameter configuration. In the graph construction phase where the graph structure is built from the edge lists, PTMT uses a smaller value (“100”) for the parameter. This phase has a rather large hot-page set, inevitably spilled to slow memory. Hence, the memory tiering should avoid page thrashing between memory tiers [50]. Using the smaller value, TPP is able to reduce unnecessary page demotion by 15% and page promotion by 20%, compared with using the default configuration. Thanks to PTMT, this phase improves performance by 32% (see Figure 9). In the BFS phase, the hot region becomes smaller and stable (see Figure 8.a) because Graph500 uses a stochastic Kronecker graph generator to create a power-law graph where a few vertices hold most edges. PTMT uses a larger value (400 or 800), enabling more aggressive page migration. Figure 9 shows 2.1× more page demotion and 2.0× more page promotion, compared to using the default configuration. In conclusion, PTMT improves the performance of Graph500 by 20%, and PTMT responds correctly to WSs throughout application execution.
Promotion
7
2.5
6.4
500
Execution Time(s) (b) Default Config
Figure 9: Page migration when tuning Graph500. ter for a specific memory tiering solution (see Sec. 6.1), we improve IDT in three ways (see our repo [16]): (1) enabling IDT to tune multiple system parameters rather than a single parameter, (2) incorporating WSs as the RL state representation, and (3) making the decision epoch configurable as a tuning period. We pre-train four separate models, each tailored to one of the four memory tiering solutions. To train each pre-trained model, we repeatedly run GUPS with three distinct memory access patterns and five different hot-region distributions until adding more iterations no longer significantly reduces the loss of RL’s value function, indicating that the model’s estimates of state values are stable. This general pre-training process leads to 11× more execution iterations than those using application-specific models in PTMT. Following IDT’s approach to build a general pre-trained model, we pre-train RL (named “RL (general pre-training)”) and compare it with RL that uses the application-specific pretraining (named “PTMT-RL”). To build the general model offline, we run GUPS with the same memory access patterns and iteration counts as IDT to construct a performance database for RL (general pre-training). After generating the expert dataset from this performance database, we apply Behavioral Cloning to pre-train PPO as the initial policy in RL. To optimize the performance of this RL (general pre-training) for online tuning, we use sensitivity study to decide its hyperparameters. Figure 7 shows the results. Compared to the default configuration, IDT improves performance by -10%, -1%, -8%, and 9% for AutoNUMA, Colloid, TPP, and UPM, respectively. RL with general pre-trained improves performance by 2%, 11%, -10%, and 13%. These results suggest that a general RL model has difficulty capturing application-specific optimization, even if it collects various WSs from a general
6.5
PTMT with Application Co-run
We launch FT, LU, and SP simultaneously and compare NoBalance, the default configuration, and PTMT -RL (using general pre-training on GUPS) under AutoNUMA. Because PTMT uses system-wide performance metrics to represent WSs and tune system-wide parameters accordingly, it can treat co-running applications as a single aggregate workload by collecting global WSs for the entire system. In the case of application co-run, pre-training RL for each combination of co-running applications is not scalable. Therefore, we use the general pre-training method (see Sec. 6.3). In particular, we use the general pre-trained RL model and tune the model 11
better
Figure 10: Sensitivity study to hyper-parameters. Performance speedup is over that of the default configuration.
better
better
Table 5: Execution time for PTMT components
Figure 11: Overall ANTT-STP and per-application throughput (Mop/s) for application co-run.
Time (ms)
Component
Time (ms)
WS Collection RL Inference Config Setting
21 1 9 - 18
Clustering Query Model Switch RL Policy Update
24 1 84
Figure 10.c shows that a learning rate of 0.01 outperforms the default setting in SB3 [13] (0.0003) and a higher rate (0.1). Because a lower rate reduces the model’s adaptability to dynamic environments, while a higher rate increases computational overhead of online training due to frequent weight updates. We choose 0.01 for PTMT. The discount factor determines the weighting of past and immediate rewards. Figure 10.d reveals that the discount factor of 0.9 performs better than 0 or 0.99 (the default setting in SB3). This suggests that focusing solely on immediate rewards or placing excessive emphasis on past rewards leads to suboptimal performance. “0.9” balances the tradeoff.
parameters when co-running FT, LU, and SP (WSS(b)) based on the collection of global WSs. We use two metrics, System Throughput (STP) and Average Normalized Turnaround Time (ANTT) [32, 73]. STP reflects system-wide throughput (the larger is better), while ANTT captures the average slow-down experienced when multiple applications share server resources (the smaller is better). We also report the per-application throughput (Mop/s, millions of operations per second), provided by NPB. Figure 11 shows the results: compared with NoBalance and the default, PTMT-RL (using general pre-training) reduces ANTT by 18% and 8%, increases STP by 18% and 8%, and boosts application throughput by 21% and 8%, respectively. This indicates that, with general pre-training, the PTMT solution is applicable to application co-run cases.
6.6
Component
6.7
Overhead Analysis
For storage overhead, the offline-generated clustering model (built from ∼ 3, 000 data points) requires about 3 MB, and the pre-trained RL model requires 70 KB. During online tuning, the transient W S(t) consumes only 96 bytes (12 FP numbers). For computation overhead, we measure the time for various components in PTMT, as summarized in Table 5. The overall performance overhead of PTMT is less than 1.0% when the tuning period is set to 10 seconds. Collecting 3, 000 data points for an application takes about 8.3 hours with a tuning period of 10 seconds.
Sensitivity Study
We use Graph500 (WSS (b)) as an example. We see similar results from the sensitivity study for other benchmarks. K value in K-NN. We vary K using the cluster modeling. Figure 10.a shows that when K < 25, the performance degrades due to a limited number of nearest WSs to find the best performance. But when K > 25, the performance variance is less than 10%. We choose K= 25 for PTMT. Tuning period. We vary the tuning period from 2 to 30 seconds. Figure 10.b shows that short (2-5s) and long (30s) tuning periods bring trivial performance improvement. A short period fails to give the workload to respond to parameter tuning, while a long period misses opportunities to tune parameters. We choose 10 seconds for PTMT. RL hyperparameters. We study two key RL hyperparameters: the learning rate and discount factor. The learning rate dictates how quickly the RL updates its network weights.
7
Conclusions
Although there is much progress in building system software for memory tiering, tuning system parameters for memory tiering is largely missing. Tuning system parameters for memory tiering can bring large performance benefits but is challenging due to the concerns on runtime overhead, tuning convergence, and input-awareness. In this paper, we look deeply into representative memory-tiering solutions and introduce a novel tuning framework. We hope that our work can make memory tiering more easily deployable. 12
References
[18] Javier Baliosian, Jorge Visca, Eduardo Grampin, Leonardo Vidal, and Martin Giachino. A rule-based distributed system for self-optimization of constrained devices. In 2009 IFIP/IEEE International Symposium on Integrated Network Management, 2009.
[1] Elbow Method, 2022. https://www.scikit-yb.org/ en/latest/api/cluster/elbow.html. [2] Model Garden on Vertex AI, 2024. https://cloud. google.com/model-garden. [3] NAS Parallel Benchmarks, 2024. https://www.nas. nasa.gov/software/npb.html. [4] Running Computer-Aided Engineering Workloads on Google Cloud, 2024. https://cloud.google.com/solutions/ running-computer-aided-engineering-workloads. [5] AI-Native Engineering Simulation in the Cloud, 2025. https://www.simscale.com/. AI Foundry Model Catalog, 2025. [6] Azure https://azure.microsoft.com/en-us/products/ ai-foundry/models.
[19] Shai Bergman, Priyank Faldu, Boris Grot, Lluís Vilanova, and Mark Silberstein. Reconsidering OS Memory Optimizations in the Presence of Disaggregated Memory. In Proceedings of the 2022 ACM SIGPLAN International Symposium on Memory Management, 2022. [20] Manaf Bin-Yahya, Yifei Zhao, Hossein Shafieirad, Anthony Ho, Shijun Yin, Fanzhao Wang, and Geng Li. {Config-Snob}: Tuning for the best configurations of networking protocol stack. In 2024 USENIX Annual Technical Conference (USENIX ATC 24), 2024. [21] Juneseo Chang, Wanju Doh, Yaebin Moon, Eojin Lee, and Jung Ho Ahn. IDT: Intelligent Data Placement for Multi-tiered Main Memory with Reinforcement Learning. In International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2024.
[7] Computational Fluid Dynamics, 2025. https://aws. amazon.com/hpc/cfd.
[22] Peng Cheng, Yutong Lu, Yunfei Du, Zhiguang Chen, and Yang Liu. Optimizing data placement on hierarchical storage architecture via machine learning. In Network and Parallel Computing: 16th IFIP WG 10.3 International Conference, 2019.
[8] GUPS (Giga Updates Per Second), 2025. https://icl. utk.edu/projectsfiles/hpcc/RandomAccess/. [9] High Performance Computing for Healthcare & Life Sciences, 2025. https://aws.amazon.com/hpc/hcls. [10] HPC as a Service, 2025. https://rescale.com/ platform/hpc-as-a-service/. [11] k-means clustering, 2025. https://en.wikipedia. org/wiki/K-means_clustering.
[23] Jinyoung Choi, Sergey Blagodurov, and Hung-Wei Tseng. Dancing in the dark: Profiling for tiered memory. In 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2021.
[12] Normal distribution, 2025. https://en.wikipedia. org/wiki/Normal_distribution.
[24] Jonathan Corbet. LRU-list manipulation with DAMON. 2022. https://lwn.net/Articles/905370/.
[13] Stable Baselines3, 2025. https://github.com/ DLR-RM/stable-baselines3.
[25] Thaleia Dimitra Doudali, Sergey Blagodurov, Abhinav Vishnu, Sudhanva Gurumurthi, and Ada Gavrilovska. Kleio: A hybrid memory page scheduler with machine intelligence. In Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, 2019.
[14] User-Space Page Management, 2025. anonymous.4open.science/r/UPM.
https://
[15] AMD. Analysis with Instruction Based Sampling. https://docs.amd.com/r/en-US/57368-uProf-userguide, 2025. [16] Anonymous. PTMT, 2025. 4open.science/r/PTMT.
[26] Thaleia Dimitra Doudali and Ada Gavrilovska. Coeus: Clustering (a) like patterns for practical machine intelligent hybrid memory management. In 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), 2022.
https://anonymous.
[17] D. H. Bailey, L. Dagum, E. Barszcz, and H. D. Simon. Nas parallel benchmark results. In Supercomputing ’92: Proceedings of the 1992 ACM/IEEE conference on Supercomputing, pages 386–393, Los Alamitos, CA, USA, 1992. IEEE Computer Society Press.
[27] Thaleia Dimitra Doudali and Ada Gavrilovska. Cronus: Computer vision-based machine intelligent hybrid memory management. In Proceedings of the 2022 International Symposium on Memory Systems, 2022. 13
[28] Thaleia Dimitra Doudali, Daniel Zahka, and Ada Gavrilovska. Cori: Dancing to the right beat of periodic data movements over hybrid memory systems. In 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2021.
the 44th Annual International Symposium on Computer Architecture, 2017. [39] Jonghyeon Kim, Wonkyo Choe, and Jeongseob Ahn. Exploring the design space of page management for {Multi-Tiered} memory systems. In 2021 USENIX Annual Technical Conference (USENIX ATC 21), 2021.
[29] Subramanya R Dulloor, Amitabha Roy, Zheguang Zhao, Narayanan Sundaram, Nadathur Satish, Rajesh Sankaran, Jeff Jackson, and Karsten Schwan. Data tiering in heterogeneous memory systems. In Proceedings of the Eleventh European Conference on Computer Systems, 2016.
[40] KR Krish, Ali Anwar, and Ali R Butt. hats: A heterogeneity-aware tiered storage for hadoop. In 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2014.
[30] Padmapriya Duraisamy, Wei Xu, Scott Hare, Ravi Rajwar, David Culler, Zhiyi Xu, Jianing Fan, Christopher Kennelly, Bill McCloskey, Danijela Mijailovic, et al. Towards an adaptable systems architecture for memory tiering at warehouse-scale. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, 2023.
[41] Andres Lagar-Cavilla, Junwhan Ahn, Suleiman Souhlal, Neha Agarwal, Radoslaw Burny, Shakeel Butt, Jichuan Chang, Ashwin Chaugule, Nan Deng, Junaid Shahid, et al. Software-defined far memory in warehouse-scale computers. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, 2019.
[31] Gil Einziger, Ohad Eytan, Roy Friedman, and Ben Manes. Adaptive software cache management. In Proceedings of the 19th International Middleware Conference, 2018.
[42] Taehyung Lee, Sumit Kumar Monga, Changwoo Min, and Young Ik Eom. Memtis: Efficient memory tiering with dynamic page classification and page size determination. In Proceedings of the 29th Symposium on Operating Systems Principles, 2023.
[32] Stijn Eyerman and Lieven Eeckhout. System-level performance metrics for multiprogram workloads. IEEE micro, 28(3):42–53, 2008.
[43] Guoliang Li, Xuanhe Zhou, Shifu Li, and Bo Gao. Qtune: A query-aware database tuning system with deep reinforcement learning. Proceedings of the VLDB Endowment, 12(12):2118–2130, 2019.
[33] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, XiangRui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. Journal of machine Learning research, 9:1871–1874, 2008.
[44] Yan Li, Kenneth Chang, Oceane Bel, Ethan L Miller, and Darrell DE Long. Capes: Unsupervised storage performance tuning using neural network-based deep reinforcement learning. In Proceedings of the international conference for high performance computing, networking, storage and analysis, 2017.
[34] Vishal Gupta, Min Lee, and Karsten Schwan. Heterovisor: Exploiting resource heterogeneity to enhance the elasticity of cloud platforms. ACM SIGPLAN Notices, 50(7):79–92, 2015.
[45] Chieh-Jan Mike Liang, Hui Xue, Mao Yang, Lidong Zhou, Lifei Zhu, Zhao Lucis Li, Zibo Wang, Qi Chen, Quanlu Zhang, Chuanjie Liu, et al. {AutoSys}: The design and operation of {Learning-Augmented} systems. In 2020 USENIX Annual Technical Conference (USENIX ATC 20), 2020.
[35] Taekyung Heo, Yang Wang, Wei Cui, Jaehyuk Huh, and Lintao Zhang. Adaptive page migration policy with huge pages in tiered memory systems. IEEE Transactions on Computers, 71(1):53–68, 2020. [36] Ying Huang. autonuma: Optimize page placement for memory tiering system, 2020. https: //patchwork.kernel.org/project/linux-mm/ patch/20201027063217.211096-2-ying.huang@ intel.com/.
[46] Weiwei Lin, Xiaoxuan Luo, ChunKi Li, Jiechao Liang, Guokai Wu, and Keqin Li. An energy-efficient tuning method for cloud servers combining dvfs and parameter optimization. IEEE Transactions on Cloud Computing, 2023.
[37] Intel. Intel® Performance Counter Monitor (Intel® PCM). https://github.com/intel/pcm.
[47] Jinshu Liu, Hamid Hadian, Hanchen Xu, and Huaicheng Li. Tiered memory management beyond hotness. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25), 2025.
[38] Sudarsun Kannan, Ada Gavrilovska, Vishal Gupta, and Karsten Schwan. Heteroos: Os design for heterogeneous memory management in datacenter. In Proceedings of 14
[48] Martin Maas, David G Andersen, Michael Isard, Mohammad Mahdi Javanmard, Kathryn S McKinley, and Colin Raffel. Learning-based memory allocation for c++ server workloads. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020.
[57] Jie Ren, Bin Ma, Shuangyan Yang, Benjamin Francis, Ehsan K Ardestani, Min Si, and Dong Li. Machine learning-guided memory optimization for dlrm inference on tiered memory. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 1631–1647. IEEE, 2025.
[49] Adnan Maruf, Ashikee Ghosh, Janki Bhimani, Daniel Campello, Andy Rudoff, and Raju Rangaswami. Multiclock: Dynamic tiering for hybrid memory systems. In 2022 IEEE International Symposium on HighPerformance Computer Architecture (HPCA), 2022.
[58] Jie Ren, Dong Xu, Junhee Ryu, Kwangsik Shin, Daewoo Kim, and Dong Li. MTM: Rethinking Memory Profiling and Migration for Multi-Tiered Large Memory Systems. In European Conference on Computer Systems, 2024. [59] Jinting Ren, Xianzhang Chen, Yujuan Tan, Duo Liu, Moming Duan, Liang Liang, and Lei Qiao. Archivist: A machine learning assisted data placement mechanism for hybrid storage systems. In 2019 IEEE 37th International Conference on Computer Design (ICCD), 2019.
[50] Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Johannes Weiner, Niket Agarwal, Pallab Bhattacharya, Chris Petersen, Mosharaf Chowdhury, Shobhit Kanaujia, and Prakash Chauhan. TPP: Transparent page placement for CXL-enabled tiered-memory. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2023.
[60] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [61] Harald Servat, Antonio J Peña, Germán Llort, Estanislao Mercadal, Hans-Christian Hoppe, and Jesús Labarta. Automating the application data placement in hybrid memory systems. In 2017 IEEE International Conference on Cluster Computing (CLUSTER), 2017.
[51] Tsvetelina Mladenova. A feature-weighted rule for the k-nearest neighbor. In International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2021. [52] Richard C Murphy, Kyle B Wheeler, Brian W Barrett, and James A Ang. Introducing the graph 500. Cray Users Group (CUG), 19(45-74):22, 2010.
[62] Kevin Song, Jiacheng Yang, Zixuan Wang, Jishen Zhao, Sihang Liu, and Gennady Pekhimenko. Hybridtier: an adaptive and lightweight cxl-memory tiering system. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2025.
[53] Yuanjiang Ni, Pankaj Mehra, Ethan Miller, and Heiner Litz. Tmc: Near-optimal resource allocation for tieredmemory systems. In Proceedings of the 2023 ACM Symposium on Cloud Computing, 2023.
[63] Jun Sun, Feng Ye, Nadia Nedjah, Ming Zhang, and Dong Xu. Workload-aware performance tuning for multimodel databases based on deep reinforcement learning. International Journal of Intelligent Systems, 2023(1):8835111, 2023.
[54] Deok-Jae Oh, Yaebin Moon, Eojin Lee, Tae Jun Ham, Yongjun Park, Jae W Lee, and Jung Ho Ahn. Maphea: A lightweight memory hierarchy-aware profile-guided heap allocation framework. In Proceedings of the 22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, 2021.
[64] Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observation. arXiv preprint arXiv:1805.01954, 2018.
[55] SeongJae Park, Madhuparna Bhowmik, and Alexandru Uta. Daos: Data access-aware operating system. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, 2022.
[65] Stephen Tu, Wenting Zheng, Eddie Kohler, Barbara Liskov, and Samuel Madden. Speedy transactions in multicore in-memory databases. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, 2013.
[56] Amanda Raybuck, Tim Stamler, Wei Zhang, Mattan Erez, and Simon Peter. Hemem: Scalable tiered memory management for big data applications and real nvm. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, 2021.
[66] Dana Van Aken, Andrew Pavlo, Geoffrey J Gordon, and Bohan Zhang. Automatic database management system tuning through large-scale machine learning. In Proceedings of the 2017 ACM international conference on management of data, 2017. 15
[67] Evangelos Vasilakis, Vassilis Papaefstathiou, Pedro Trancoso, and Ioannis Sourdis. Hybrid2: Combining caching and migration in hybrid memory systems. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2020.
cxl memory pooling. arXiv preprint arXiv:2602.22457, 2026. [77] Dong Xu, Junhee Ryu, Jinho Baek, Kwangsik Shin, Pengfei Su, and Dong Li. FlexMem: Adaptive Page Profiling and Migration for Tiered Memory. In 30th USENIX Annual Technical Conference (ATC), 2024.
[68] Vishal Verma. Tiering-0.8. 2022. https: //git.kernel.org/pub/scm/linux/kernel/ git/vishal/tiering.git/log/?h=tiering-0.8.
[78] Feng Ye, Yang Li, Xiwen Wang, Nadia Nedjah, Peng Zhang, and Hong Shi. Parameters tuning of multi-model database based on deep reinforcement learning. Journal of Intelligent Information Systems, 61(1):167–190, 2023.
[69] Midhul Vuppalapati and Rachit Agarwal. Tiered memory management: Access latency is the key! In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, 2024.
[79] Ji Zhang, Yu Liu, Ke Zhou, Guoliang Li, Zhili Xiao, Bin Cheng, Jiashu Xing, Yangtao Wang, Tianheng Cheng, Li Liu, et al. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proceedings of the 2019 international conference on management of data, 2019.
[70] Xi Wang, Jie Liu, Jianbo Wu, Shuangyan Yang, Jie Ren, Bhanu Shankar, and Dong Li. Performance characterization of cxl memory and its use cases. In 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 1048–1061. IEEE, 2025. [71] Xi Wang, Bin Ma, Jongryool Kim, Byungil Koh, Hoshik Kim, and Dong Li. cmpi: Using cxl memory sharing for mpi one-sided and two-sided inter-node communications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2025. [72] Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, et al. Tmo: Transparent memory offloading in datacenters. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, 2022. [73] Bo Wu, Guoyang Chen, Dong Li, Xipeng Shen, and Jeffrey Vetter. Enabling and exploiting flexible task assignment on gpu through sm-centric program transformations. In Proceedings of the 29th ACM on International Conference on Supercomputing, 2015. [74] Kai Wu, Yingchao Huang, and Dong Li. Unimem: Runtime data managementon non-volatile memory-based heterogeneous main memory. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2017. [75] Lingfeng Xiang, Zhen Lin, Weishu Deng, Hui Lu, Jia Rao, Yifan Yuan, and Ren Wang. Nomad: NonExclusive Memory Tiering via Transactional Page Migration. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2024. [76] Dong Xu, Han Meng, Xinyu Chen, Dengcheng Zhu, Wei Tang, Fei Liu, Liguang Xie, Wu Xiang, Rui Shi, Yue Li, et al. Cccl: Node-spanning gpu collectives with 16