CW-Ghost: Search-Free Granularity Selection for Helper-Thread Prefetching via Capacity Windows Ya Zhang1,2[0009−0005−6625−6214] , Tong Lei1,2[0009−0007−9748−5997] Yao Chen1,2[0000−0002−8748−4508] , Yonggang Che⋆1,2[0000−0001−6906−4940] , Chuanfu Xu1,2[0000−0002−4876−2368] , Haozhong Qiu1,2[0009−0009−0434−8075] , and Yusong Tan1,2[0000−0003−1233−5679]
arXiv:2607.25363v1 [cs.DC] 28 Jul 2026
1
Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology, Changsha 410073, China 2 National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China [email protected]
Abstract. Helper-thread prefetching hides the latency of irregular memory accesses by executing address dependency chains ahead of the main thread. However, its effectiveness depends on the range of future iterations covered by the helper thread. A fixed coverage range cannot consistently accommodate different workloads and processors, whereas exhaustively evaluating candidate configurations incurs substantial configuration cost. This paper presents CW-Ghost, which uses a single offline profiling run to estimate the average demand cache line fill volume generated per target iteration in a target region. CW-Ghost combines this estimate with a cache capacity budget to derive a Capacity Window, which determines the iteration granularity of each prefetch chunk. In addition, bounded chunk-level synchronization limits the number of chunks by which the helper thread may run ahead of the main thread. Across 14 workload instances evaluated on Intel and AMD CPU platforms, CWGhost achieves geometric mean speedups of 1.54× and 1.33×, respectively, over the original programs. Compared with Ghost Threading, it improves geometric mean performance by 15.8% and 10.8%, respectively, while achieving more than 99% of the empirically optimal performance within the candidate set on both platforms. These results demonstrate that cache capacity constraints can effectively guide the selection of granularity for helper-thread prefetching. Keywords: Irregular memory accesses · Helper-thread prefetching · Profileguided optimization · Prefetch granularity · Capacity Window
1
Introduction
Applications such as graph analysis, databases, and high performance computing commonly involve irregular memory accesses, where the target address typ⋆
Corresponding author
2
Y. Zhang et al.
ically depends on preceding index accesses or data-dependent address calculation chains [2, 4, 5, 11]. Such dependences delay the generation of future memory requests, making it difficult for conventional hardware prefetchers to consistently hide memory access latency [2, 8, 24]. Existing research has addressed such memory accesses through dedicated hardware prefetching or data access acceleration mechanisms [8, 24], software and compiler prefetching [2, 7, 9, 14, 20, 21, 25], and helper-thread prefetching or runahead execution [12, 15–17, 19, 22]. Ghost Threading [11] targets existing SMT processors by extracting an addresscomputation p-slice for a target load and running a helper thread on a sibling SMT context to prefetch data for the main thread’s future accesses. On real Intel systems, it achieved a geometric mean speedup of 1.33× compared to the single-threaded baseline. However, synchronization parameters, including the inter-thread distance, still require manual tuning through profiling [11]. Even after offloading address computation to a helper thread, it remains necessary to determine how many future target iterations that thread should cover at a time. We refer to a consecutive group of future target iterations processed by the helper thread at a time as a prefetch chunk, and define its granularity, denoted by B, as the number of target iterations in the chunk. When B is too small, prefetches may not be issued sufficiently far in advance, while chunk boundary checks and synchronization account for a larger fraction of the execution overhead. When B is too large, the expanded prefetch footprint can cause premature prefetching, data being evicted before it is used, and increased contention for caches, memory bandwidth, and SMT-shared resources [2, 11]. Consequently, neither a single fixed granularity nor a configuration selected on a source platform is consistently effective. Evaluating all candidate granularities for every workload–platform pair, however, incurs substantial configuration cost. This paper proposes CW-Ghost, which replaces candidate-by-candidate performance search for prefetch-chunk granularity with a capacity-guided window computation. CW-Ghost uses a single offline profiling run to estimate the demand cache line fill volume generated per target iteration within the target region. It combines this estimate with a capacity budget for the target cache to compute a Capacity Window, denoted by WC . A prefetch chunk determined by WC is defined as a Capacity-Bounded Chunk (CBC), whose granularity is set to BCBC = WC . The optimized program partitions the target iteration space according to this granularity and uses an independent CBC-level synchronization bound, K, to limit the chunk-level runahead of the helper thread. Thus, WC determines the amount of work covered by each prefetch chunk, whereas K controls the progress of the helper thread relative to the main thread. We evaluated CW-Ghost using 14 workload instances on two x86 SMT hardware platforms equipped with an Intel Xeon Gold 6258R and an AMD EPYC 7H12. Compared with the original single-threaded programs, CW-Ghost achieves geometric mean speedups of 1.54× and 1.33× on the two platforms, respectively. Compared with Ghost Threading, it improves geometric mean performance by 15.8% and 10.8%, respectively. Oracle-Chunk exhaustively evaluates a predefined candidate set and selects the best observed granularity for each workload–
CW-Ghost: Search-Free Granularity Selection
3
platform pair. CW-Ghost achieves more than 99% of Oracle-Chunk’s geometric mean performance on both platforms, while also reducing the process-level retired instruction count and the main thread’s L1D-miss and LLC-miss MPKI. The main contributions of this paper are as follows: 1. We characterize the performance impact of helper thread prefetch-chunk granularity. Granularity sensitivity and cross-processor configuration migration experiments show that near-optimal granularity is both workload- and processordependent and therefore cannot be reliably replaced by a single fixed value or a configuration selected on another processor. 2. We propose the Capacity Window and CW-Ghost. CW-Ghost computes the CBC granularity from the demand cache line fill volume per target iteration obtained through offline profiling and a cache capacity budget. It further uses bounded chunk-level synchronization to control the progress of the helper thread. 3. We evaluate CW-Ghost on two x86 SMT hardware platforms based on Intel Cascade Lake and AMD Zen 2, which differ in microarchitecture, cache organization, and PMU event semantics. The results show that CW-Ghost improves the end-to-end performance of Ghost Threading and achieves overall performance close to the best observed configuration in the candidate set.
2
Background and Related Work
2.1
Challenges in Prefetching Indirect Memory Accesses
Data-intensive applications, such as graph analytics and databases, commonly contain a large number of indirect memory accesses. A representative access pattern is A[B[i]], in which the processor must first obtain the index B[i] before it can generate the address of the corresponding access to A [2,5,9–11,18]. Because the target address depends on runtime data, such accesses delay the generation of subsequent memory requests, making it difficult for conventional hardware prefetchers to request the target data sufficiently early before the corresponding demand accesses [2, 8, 24]. Prefetching effectiveness depends on request accuracy and timeliness, as well as the overhead of address generation and request injection [2,11,14]. Prefetches issued too late cannot sufficiently hide memory access latency, whereas those issued too early may cause the prefetched data to be evicted before use and increase cache and memory bandwidth pressure. Existing hardware mechanisms extend address prediction capability by exploiting local delta [23], temporal correlations [3], and online learning [6]. Other designs employ specialized structures to capture indirect-access relationships or accelerate address generation [8, 24]. Software prefetching [2, 7, 21] and execution-based prefetching [12, 19], on the other hand, explicitly generate future memory requests by executing address computation chains ahead of the demand accesses. 2.2
Software Prefetching
Software prefetching relies on programmers or compilers to insert explicit prefetch instructions into a program and compute the addresses of data that will be ac-
4
Y. Zhang et al.
cessed in the future [2, 7, 21]. Early studies established fundamental compiler techniques for automatically inserting and scheduling prefetches [7,21], and subsequent work extended these techniques to recursive and pointer-based data structures [20]. For indirect memory accesses, Ainsworth and Jones extract the address computation chain of a target load and replicate the relevant instructions to generate prefetch requests for subsequent loop iterations [2]. Magellan further exploits loop structures and inter-loop dependences to identify and schedule both intra-loop and cross-loop prefetches for indirect memory accesses, thereby expanding the range of address dependences that can be handled by software prefetching [9]. The effectiveness of software prefetching is generally sensitive to the insertion point and lead distance of prefetch instructions, and optimal configurations may vary with the program’s dynamic behavior and the target microarchitecture. APT-GET uses dynamic runtime profiling to select prefetch injection sites and distances, thereby improving prefetch timeliness [14]. RPG2 profiles a running program, injects prefetches, and dynamically adjusts the prefetch distance to reduce the performance loss caused by unfavorable configurations [25]. Because these approaches still execute address computation and prefetch instructions on the main thread, their performance benefits must be balanced against the additional overhead of dynamic instructions and execution resources.
2.3
Helper-Thread Prefetching
Helper-thread prefetching executes a compact pre-execution slice, or p-slice, in a separate execution context. The p-slice consists of a target long-latency load and the instructions required to compute its address, allowing the helper thread to generate future memory requests for the main thread in advance [19]. Since a p-slice is typically shorter than the corresponding execution path in the original program, the helper thread can run ahead of the main thread and increase memory-level parallelism. However, insufficient runahead cannot effectively hide memory-access latency, whereas excessive runahead may cause eviction before use, cache pollution, and contention for shared execution resources. Prior studies have investigated helper-thread prefetching on real SMT systems [17], compilerassisted slice extraction [15], and helper threads that migrate across processor cores [16]. Related runahead execution techniques speculatively expose subsequent memory accesses within the processor rather than using a separate software thread context [12, 22]. Ghost Threading [11] implements software helper-thread prefetching on existing SMT processors. It uses profiling to select a target load, extracts the corresponding p-slice, and executes a ghost thread on a sibling SMT context. However, its synchronization and inter-thread distance parameters still require manual profile-guided tuning [11]. Building on this helper execution framework, CW-Ghost focuses on the low-cost selection of prefetch-chunk granularity.
CW-Ghost: Search-Free Granularity Selection
3
5
CW-Ghost: Capacity-Guided Granularity Selection and Bounded Execution
This section first characterizes the workload and processor dependence of prefetchchunk granularity, and then presents the Capacity Window computation, CBCbased program transformation, and bounded chunk-level execution mechanisms of CW-Ghost. 3.1
Problem Characterization and Design Goals
We refer to a consecutive group of future target iterations processed by the helper thread as a prefetch chunk. A prefetch chunk is a general helper execution unit with no predefined cache-capacity constraint; its granularity, denoted by B, is the number of target iterations it contains. To determine the feasibility of uniform granularity or cross-processor reuse configurations, this section examines the workload dependency, processor dependency, and configuration cost of B, while keeping other helper execution mechanisms unchanged. Here, B represents a general candidate granularity; the CBC granularity determined by the Capacity Window will be defined in Section 3.2. Workload Dependence To examine the effect of chunk granularity on performance, we fix the target region, target load, address computation slice, chunklevel synchronization mechanism, maximum runahead bound, and compilation configuration on the Intel platform. We vary only B over the following set of 12 power of two candidates, i.e., B = {2k | 0 ≤ k ≤ 11} = {1, 2, . . . , 2048}. All other experimental settings are identical to those used in the main experiments described in Section 4.1. Figure 1 shows that the performance of camel, hj2, and cc-kron varies non-monotonically with B. Moreover, neither the peak granularity nor the near-optimal range whose performance is at least 95% of the corresponding peak is consistent across the three workloads. Therefore, a single fixed granularity cannot be reliably applied to different workloads. Processor Dependence To determine whether a chunk-granularity configuration can be reused across processors, we evaluate the same candidate set on the Intel and AMD platforms while holding the workload, input, target load, address computation slice, synchronization mechanism, and compiler optimizations constant. Let Sp (B) denote the speedup achieved on platform p using granularity B, relative to the original single-threaded program on the same platform. The locally empirically optimal granularity for platform p is defined as: Bp∗ = arg max Sp (B).
(1)
B∈B
The normalized transfer performance from a source platform p to a target platform q is defined as: Sq Bp∗ . (2) Tp→q = Sq Bq∗
6
Y. Zhang et al. (b) Peak and near-optimal granularities 512 camel hj2 cc-kron
3.0 2.5
Chunk granularity, B
Speedup over Baseline (x)
(a) Sensitivity to chunk granularity 3.5
2.0 1.5 1.0
256
Whisker: contiguous range with Speedup ≥ 95% of peak 1.54×
128 64 32
3.31× 1.41×
16 8
1
2
4
8
16
32
64
128
256
512
Chunk granularity, B (target iterations/chunk)
4
102
8
204
el
cam
hj2
n kro
cc-
Fig. 1. Workload dependence of prefetch-chunk granularity. (a) Speedup of camel, hj2, and cc-kron over Baseline on the Intel platform under different B ∈ B. Stars mark the peak configurations in the candidate set, and the horizontal dashed line denotes Baseline. (b) Peak granularities and near-optimal ranges. Circles mark the peak granularities, the annotations report the corresponding peak speedups, and each whisker indicates the largest contiguous candidate interval that contains the peak and achieves at least 95% of the peak performance.
Here, Tp→q = 1 indicates that the granularity selected on the source platform achieves the target platform’s local peak performance, whereas Tp→q < 1 indicates a transfer loss. Both the numerator and denominator are measured on the target platform relative to the same Baseline. The metric therefore reflects the quality of the transferred granularity rather than differences in absolute execution time between the two platforms. As shown in Figure 2, the peak granularity of the same workload can vary across processors. For hj2, the platform-local empirically optimal granularities are 16 and 32 on the Intel and AMD platforms, respectively; for cc-kron, they are 128 and 64, respectively. When the Intel-local optimum is transferred to AMD, hj2 and cc-kron achieve 0.86 and 0.85 of the corresponding AMD-local optimal performance. In the reverse direction, both workloads retain approximately 0.99 of the Intel-local optimal performance. Transfer quality depends not only on the locations of the peaks on the two platforms, but also on whether the source-platform granularity falls within the near-optimal range of the target platform. Consequently, a granularity that is locally optimal on one processor is not necessarily near-optimal on another. Configuration Cost Oracle-Chunk evaluates every granularity in B and selects the configuration with the highest measured performance for each workload– platform pair. Although this procedure provides an empirical performance upper bound within the candidate set, it requires all candidates to be executed for every target region and platform. Its cost further increases with the numbers of processors, inputs, and target regions, making it unsuitable as a practical configuration procedure. These observations lead to three design goals. First, granularity selection should reflect the dynamic behavior of the current workload–platform pair rather
Speedup over Baseline (x)
(a) hj2 1.8
(b) cc-kron Intel AMD
1.6 1.4 1.2 1.0 0.8
1
2
4
8
16 32 64 128 256 512 024 048 1 2
Chunk granularity, B
1
2
4
8
16 32 64 128 256 512 024 048 1 2
Chunk granularity, B
Normalized transfer performance
CW-Ghost: Search-Free Granularity Selection
7
(c) Cross-processor transfer 1.0
0.99 0.86
0.99 0.85
0.8 0.6 0.4 0.2 0.0
hj2
TI→A
cc-
n kro
TA→I
Fig. 2. Cross-processor transferability of prefetch-chunk granularity. (a)–(b) Granularity sensitivity of hj2 and cc-kron on the Intel and AMD platforms. Stars indicate the platform-local empirically optimal granularities within the predefined candidate set, and the horizontal dashed line denotes Baseline. (c) Bidirectional normalized transfer performance. TI→A and TA→I denote the normalized performance obtained by applying the Intel- and AMD-local optimal granularities, respectively, to the other platform.
than rely on a single fixed value or a configuration transferred from another platform. Second, the configuration procedure should avoid evaluating candidate granularities one by one. Third, profiling logic should not be included in the final optimized program or affect its performance measurement. 3.2
CW-Ghost Overview
Figure 3 illustrates the three stages of CW-Ghost: offline profiling, Capacity Window computation, and CBC-based execution. The method takes as input a predetermined target region, target load, and corresponding address computation slice. Target region identification, target load selection, and generic slice extraction are beyond the scope of the automation provided by this work. During the offline profiling stage, CW-Ghost combines software iteration counting with platform-specific PMU sampling to obtain the dynamic number of target iterations, the number of cache event samples, and the sampling period for the target region. These statistics are organized into a unified normalized profiling record. Platform-specific differences are confined to the event collection and report parsing front ends. All subsequent stages use the same window computation logic, without evaluating candidate chunk granularities one by one. During the Capacity Window computation stage, CW-Ghost combines the normalized profiling record with the cache configuration and capacity budget of the target processor to compute the Capacity Window, WC . We define a prefetch chunk whose granularity is determined by WC as a Capacity-Bounded Chunk (CBC), and set BCBC = WC . Here, B denotes the general candidate prefetch-chunk granularity introduced in Section 3.1, whereas BCBC denotes the CBC granularity computed by CW-Ghost. During the CBC-based execution stage, the final optimized program partitions the target iteration space according to BCBC and runs the prefetch helper
8
Y. Zhang et al.
Fig. 3. End-to-end workflow of CW-Ghost. Offline profiling produces a normalized profiling record, from which the window computation stage derives WC and sets BCBC = WC . The final program partitions the target iteration space into CBCs using this granularity and limits the helper thread’s chunk-level runahead through the synchronization bound K.
thread on a sibling SMT context of the same physical core. The value BCBC determines the number of target iterations covered by each CBC, while an independent CBC-level synchronization bound K limits the chunk-level runahead of the helper thread relative to the main thread. 3.3
Profiling and Capacity Window Computation
The offline profiling stage of CW-Ghost collects only the statistics required to compute the Capacity Window. For a target region Rt , profiling obtains the number of hardware event samples attributed to the region St , the sampling period I, and the dynamic number of target iterations Nt . These statistics form the following normalized profiling record: Πt = ⟨St , I, Nt ⟩ .
(3)
Here, Nt denotes the dynamic number of target iterations covered by the final CBC-based transformation and does not necessarily equal the static trip count of an enclosing loop. Normalized Profile Hardware event sampling captures the distribution of cache events across the program address space, but does not directly provide the dynamic target iteration count Nt . The profiling version therefore uses conditionally compiled software counters to record Nt . For a region whose loop extent is directly available, the counter is incremented in bulk by the actual loop extent. For a region whose iteration count depends on the input or control flow, the actual number of target iterations executed is recorded. These software counters are excluded from the final optimized program. Hardware samples are attributed to the target region Rt using instruction addresses and debug information.
CW-Ghost: Search-Free Granularity Selection
9
Platform-Specific Collection The Intel and AMD platforms use different front ends for hardware event collection and report parsing. On Intel, CW-Ghost uses PEBS to collect samples of retired demand loads that miss in the L1D cache. On AMD, it uses EBS to collect demand data cache refill samples. The detailed event definitions follow the corresponding vendor documentation [1, 13], while the counting scopes are described in Section 4.1. Although these events have different semantics, both are used in the window model to approximate the L1D refill behavior caused by demand accesses in the target region. The model runs independently on each platform, and no absolute cross-processor comparisons are made between the number of raw samples and the number of reconstructed events.
Capacity Window Model For sampling at fixed intervals, St · I estimates the total number of monitored events in the target region. Let L denote the cache line size. The average demand cache line fill volume per target iteration is then estimated as: St × I × L . (4) Fbt = Nt The unit of Fbt is bytes per target iteration. CW-Ghost uses the physical L1D capacity, CL1D , as the modeling capacity because the sampled events approximate L1D misses/refills and the main and helper threads running on the same physical core share this cache level. Let α ∈ (0, 1] be the global capacity budget ratio. Under the condition that Fbt approximates the refill volume for different target iterations within a chunk and is accumulated over iterations, the number of non-discretized target iterations that the capacity budget can cover is: Wraw =
α × CL1D × Nt α × CL1D = . b St × I × L Ft
(5)
We use the same value of α for all workloads and processors, and its sensitivity is evaluated in Section 4.3.
Validity and Discretization The model proceeds only when St > 0, I > 0, Nt > 0, CL1D > 0, L > 0. If Fbt ≤ 0 or Wraw < 1, CW-Ghost does not generate a CBC configuration for the target region. Otherwise, the valid result is rounded down to the largest power of two no greater than Wraw : WC = 2⌊log2 Wraw ⌋ .
(6)
This rounding ensures that the resulting WC does not exceed the upper bound given by the capacity model, while keeping chunk boundaries and granularity representation regular. The final CBC granularity is BCBC = WC .
10
Y. Zhang et al.
Model Scope The quantity Fbt is a cumulative fill volume proxy derived from sampled events. It is not equivalent to the set of unique cache lines, a strict working set size, or the actual cache residency at any given moment. Repeated fills, inter-iteration reuse, and cache replacement may all cause it to differ from the actual cache occupancy. The Capacity Window is therefore a granularity selection rule based on capacity budgets, rather than an exact predictor of cache state or of the granularity that maximizes end-to-end performance. 3.4
CBC-Based Program Transformation and Bounded Execution
CW-Ghost sets BCBC = WC , partitions the target iteration space accordingly, and uses CBC-level synchronization to control the relative progress of the main thread and the prefetch helper thread. CBC Partitioning For an execution containing Nt target iterations, CWGhost partitions the target iterations into consecutive Capacity-Bounded Chunks and uses a consistent CBC index space for the main and helper threads. The total number of CBCs is: Nt Nt = . (7) NCBC = BCBC WC For 0 ≤ c < NCBC , the cth CBC covers the following target-iteration interval: CBCc = [cBCBC , min((c + 1)BCBC , Nt )) .
(8)
Except for the final CBC, each CBC contains exactly BCBC target iterations. The main thread executes CBCs in increasing index order while preserving the original iteration order within each CBC. The transformation introduces only chunk boundary computation and progress publication, without changing the original computation semantics of the main thread. Helper Execution The helper thread uses the same CBC index space and executes the target address computation slice At ahead of the main thread for future CBCs. The slice preserves the control flow, index computations, and data dependences required to generate the target addresses, replaces the target load with a prefetch hint, and does not execute its downstream consumer semantics. Bounded Progress Control While BCBC determines the number of target iterations covered by each CBC, an independent CBC-level synchronization bound K restricts the range of future CBCs that the helper thread may select. The two threads share an atomic variable Cmain , which denotes the CBC currently being executed by the main thread. The main thread is the sole writer of Cmain , while the helper thread only reads it and locally maintains Cpf , the index of the next CBC it intends to process. Valid CBC indices lie in [0, NCBC − 1]. Before execution begins, the main thread initializes Cmain to 0. After completing the target
CW-Ghost: Search-Free Granularity Selection
11
Fig. 4. CBC-level bounded runahead control. The main thread publishes the index of its current CBC through Cmain , while the helper thread maintains the next CBC to be processed in Cpf . Based on their relative positions, the helper performs Continue, Skip, or Wait.
region, it updates Cmain to NCBC , which serves as the completion sentinel. The main thread publishes its progress only when entering a new CBC. The helper thread reads the shared variable when selecting the next CBC or waiting for the main thread to advance. As shown in Figure 4, the helper thread skips CBCs where Cpf ≤ Cmain , continues prefetching when 0 < Cpf − Cmain ≤ K, and waits for the main thread to advance when Cpf > Cmain +K. Algorithm 1 summarizes the complete process of CBC progress publication by the main thread and bounded prefetching by the helper thread. The two threads join only after the target region has completed. The synchronization bound K limits the maximum chunk-level lead when the helper selects new work relative to the main thread progress most recently observed by the helper. Specifically, a newly selected CBC satisfies 0 < Cpf − Cmain ≤ K. This relation need not remain valid throughout the helper’s execution of the selected CBC. The main thread may continue to advance while the helper executes its slice. Before selecting the next CBC, the helper reloads the main thread progress and uses the Skip action to bypass CBCs that the main thread has already entered. Consequently, K is neither a strict iterationlevel prefetch distance nor an upper bound on the aggregate cache residency of prefetched but unconsumed data. It merely constrains the relative CBC-level progress when new helper work is selected. Correctness and Scope The CBC-based transformation requires At to commit no state modifications or external side effects visible to the application and to introduce neither invalid memory accesses nor undefined data races. The main and helper threads must also establish a consistent and monotonic index mapping for the target iterations. Shared progress is communicated through an
12
Y. Zhang et al.
atomic variable using release semantics for main thread stores. However, this communication does not replace the data safety conditions required by At itself. For target regions that do not satisfy the above conditions, CW-Ghost does not apply this transformation.
Algorithm 1 Bounded CBC-Based Main/Helper Execution Input: Nt BCBC At K
// number of target iterations in the current execution // CBC granularity, set to WC // pre-extracted target-address computation slice // CBC-level synchronization bound, K ≥ 1
procedure MainThread(Nt , BCBC , At , K) 1: NCBC ← ⌈Nt /BCBC ⌉ 2: if NCBC = 0 then 3: return 4: end if 5: Cmain .store(0, release) 6: launch PrefetchHelper (NCBC , BCBC , At , K) 7: ExecuteOriginal(CBC0 ) 8: for c ← 1 to NCBC − 1 do 9: Cmain .store(c, release) 10: ExecuteOriginal(CBCc ) 11: end for 12: Cmain .store(NCBC , release) // completion sentinel 13: join PrefetchHelper procedure PrefetchHelper (NCBC , BCBC , At , K) 14: Cpf ← 0 15: while Cpf < NCBC do 16: c ← Cmain .load(acquire) 17: if c ≥ NCBC then 18: break 19: end if 20: if Cpf ≤ c then 21: Cpf ← c + 1 // Skip 22: end if 23: if Cpf ≥ NCBC then 24: break 25: end if 26: if Cpf > c + K then 27: Pause() // Wait 28: continue 29: end if 30: PrefetchCBC (CBCCpf , At ) // Continue 31: Cpf ← Cpf + 1 32: end while
CW-Ghost: Search-Free Granularity Selection
4
Experimental Evaluation
4.1
Experimental Setup
13
Platforms and Workloads Table 1 summarizes the two hardware platforms: an Intel Xeon Gold 6258R based on the Cascade Lake microarchitecture and an AMD EPYC 7H12 based on the Zen 2 microarchitecture. SMT is enabled on both platforms. Each experiment uses one main thread and one prefetch helper thread, which are pinned to sibling SMT contexts on the same physical core. CW-Ghost performs offline profiling and computes the Capacity Window WC independently for each workload–platform pair. Table 2 lists the evaluated programs, inputs, and target regions. The evaluation suite includes graph workloads, an HPC kernel, synthetic indirect memory access programs, and database hash joins, comprising 14 workload instances in total. We refer to the combination of a kernel and a specific input as a workload. For each workload, hardware sample attribution, dynamic target iteration counting, and the CBC-based transformation refer to the same target region. Unless otherwise stated, offline profiling and performance evaluation use the same input. Configurations We evaluate the following five configurations: – Baseline. The single-threaded program without helper thread prefetching. – Ghost Threading. Our port of Ghost Threading [11] to the evaluated hardware platforms. It adopts the helper thread execution framework and synchronization mechanism of the original design, with the adaptations required for each target platform. The relevant parameters follow the recommendations of the original work and the settings used in our port. – CW-Ghost. CW-Ghost computes WC through a single offline profiling run, sets BCBC = WC , and applies the CBC-based program transformation and bounded-execution mechanism described in Section 3.4. – Fixed-Chunk. A fixed granularity shared across all workloads on the same platform, used to evaluate the effect of the Capacity Window relative to that fixed granularity. – Oracle-Chunk. Selects the granularity with the lowest mean execution time from the predefined candidate set B as the empirical performance upper bound within the candidate set. This configuration is used only to evaluate granularity-selection quality; it is not part of the practical configuration procedure of CW-Ghost. Fixed-Chunk and Oracle-Chunk reuse the target regions, address computation slices, program transformation, and CBC-level synchronization mechanism of CW-Ghost, and differ only in granularity selection. Except for the differences in the mechanisms described above, all configurations on the same platform use identical inputs, main thread computation, compiler options, thread and NUMA placement, hardware prefetcher configuration, and timing scope.
14
Y. Zhang et al. Table 1. Platform configuration.
Component Processor
Intel platform AMD platform Intel Xeon Gold 6258R (Cascade AMD EPYC 7H12 (Zen 2, Rome) Lake) CPU topology 2 sockets × 28 cores/socket × 2 2 sockets × 64 cores/socket × 2 threads/core threads/core L1D cache 32 KiB/core 32 KiB/core L2 cache 1 MiB/core 512 KiB/core LLC organization 38.5 MiB/socket, shared 16 MiB/4-core CCX, shared Main memory 376 GB DDR4 503 GB DDR4 Table 2. Workload configuration. Kernel cc sssp camel kangaroo is hj2 hj8
Input(s) kron, twitter, urand, road, web kron, twitter, urand, web 225 elements 225 keys, three indirect arrays Class B, 225 keys |R| = |S| = 12.8 million tuples |R| = |S| = 12.8 million tuples
Target region Afforest sampled-neighbor linking ∆-stepping edge relaxation Main pointer-indirect traversal rank two-level hash-indirect update rank histogram update NPO hash-table probe NPO hash-table probe
Parameters and Measurement Unless otherwise stated, the capacity budget ratio is set to α = 0.5, and the CBC-level synchronization bound is set to K = 2. Both parameters remain fixed across all workloads and both platforms, without per-workload or per-platform tuning. Their sensitivity is evaluated in Section 4.3. We use kernel-level end-to-end execution time as the primary performance metric. The timed region excludes input loading, test data generation, and one-time initialization, but includes the complete computation kernel and all overhead associated with helper thread creation, address computation, prefetching, CBC-level synchronization, waiting, thread termination, and thread joining. Each configuration is executed three times, with each run launched as a separate process. Speedup over Baseline is computed using the mean execution times, and speedups across workloads are summarized using the geometric mean.
PMU Metrics Performance timing and PMU metrics are collected in separate runs to prevent hardware event collection from affecting the timing results. Dynamic instruction counts are collected at process scope and include all retired instructions from both the main and helper threads. For cache MPKI, both the cache event counts and the retired instruction denominators are collected at main thread scope. Because the Intel and AMD platforms use PMU events with different semantics, these metrics are normalized to Baseline only within the same platform; absolute event counts are not compared across processors.
CW-Ghost: Search-Free Granularity Selection