Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing Sven Hepkema

Bowen Wu

Christos Kozyrakis

Systems Group, ETH Zurich Switzerland sven.hepkema@inf.ethz.ch

Systems Group, ETH Zurich Switzerland bowen.wu@inf.ethz.ch

NVIDIA & Stanford University United States ckozyrakis@nvidia.com

Gustavo Alonso Systems Group, ETH Zurich Switzerland alonso@inf.ethz.ch

arXiv:2608.13696v1 [cs.DB] 13 Aug 2026

ABSTRACT Datacenter GPUs have seen an order-of-magnitude increase in memory bandwidth with the adoption of newer generations of HBM. Meanwhile, GPU database systems are gaining traction, many building on cuDF, an open-source library of GPU relational operators. Previously, query performance was bound by memory bandwidth, but the increase in memory bandwidth has not resulted in a proportional speedup of cuDF kernels. To investigate why performance has not kept up, we built Valk, a performance analysis tool that combines data from multiple profilers. We profile cuDF running TPC-H in-memory on two extremes of hardware capability, the L4 and GH200 GPUs. The GH200 has 13.4× the memory bandwidth and 2.5× the instruction throughput of the L4, yet is only 5.2× faster in running TPC-H. Our analysis shows that when memory bandwidth is increased, kernels become compute bound. From our analysis, we make three recommendations to fully utilize the GPUs’ potential for relational workloads when the memory wall is removed: kernels need to 1) make more efficient use of caches, and 2) increase occupancy and/or instruction level parallelism, and 3) execute fewer instructions per memory access. Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/SvenHepkema/Valk/tree/arxiv-artifact.

1

INTRODUCTION

GPUs are increasingly being used for relational analytics [29, 31, 42, 57, 73]. It makes sense to explore running analytics on GPUs given the abundance of GPU capacity, especially as AI workloads move to the latest generation of GPUs, leaving large pools of older GPUs underutilized. GPUs offer orders of magnitude more bandwidth and compute than CPUs, but have been tuned and architected for AI workloads, which demand higher memory bandwidth [21, 30]. Intuitively, one would expect that the higher memory bandwidth of GPUs would also benefit relational workloads. Unfortunately, this is not the case. Although new datacenter GPUs have access to increasingly more memory bandwidth, the CUDA core instruction throughput of GPUs (those relevant for relational workloads) has not kept up. Figure 1 plots the memory bandwidth and instruction 1 Clock frequencies for the B200 and Rubin are unpublished, we assume for those GPUs

the clock frequency of the H200’s.

Relative to P100

Yannis Chronis Systems Group, ETH Zurich Switzerland chronis@inf.ethz.ch

32 16 8 4 2 1

Mem. bandwidth Instr. throughput

30.1 × 5.4 ×

P100 V100 A100 H100 H200 B200 Rubin

Figure 1: GPUs have increasingly more memory bandwidth compared to CUDA core instruction throughput.1

throughput for different generations of GPUs showing the growing gap between the two. As a result of this gap, when running TPC-H, a GPU with 13.4× more bandwidth (GH200) delivers only 5.2× more performance (Table 1) than a smaller GPU (L4). For both GPUs the performance is roughly linear with the TPC-H scale factor, but the performance advantage of having more memory bandwidth is not correlated with the bandwidth ratios (and features as well as price) between the two GPUs. The reason for this is that the relevant instruction throughput is only 2.5× larger in the GH200 than in the L4. Previous work on earlier GPUs has suggested that query execution is memory bound (e.g., [16, 60, 75]). In this paper, we study how the bottleneck has moved from the memory bandwidth to the CUDA core instruction throughput, the instructions used by cuDF kernels. cuDF [56] is an open source library of relational GPU operators and the one most commonly used in GPU-based database systems [1, 8, 10, 31, 39, 73], thus representative of the state of the art in data analytics on GPUs. To explore the behavior of cuDF in detail, we conduct an indepth exploration and performance analysis of running relational workloads (TPC-H) using Maximus [31] and cuDF kernels. We focus on two points of the architectural space, a smaller, more economical GPU with limited memory bandwidth (NVIDIA L4), and a large, high end GPU with high memory bandwidth (NVIDIA GH200). To facilitate the exploration, we have built Valk, an novel analysis tool that combines data from multiple profilers. The experimental exploration shows that the 13.4× increase in memory bandwidth from the L4 to the GH200 is not matched

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

Table 1: Instruction throughput, memory bandwidth, and the sum of kernel execution times, per GPU and SF. L4 GH200 Ratio Instr. throughput (G inst./s) 413 Mem. bandwidth (GB/s) 300 Í TPC-H SF1 kernel (ms) 84 Í TPC-H SF3 kernel (ms) 256 Í TPC-H SF10 kernel (ms) 924 Í TPC-H SF30 kernel (ms) 3,142 Í TPC-H SF100 kernel (ms) -

1,037 2.5× 4,022 13.4× 34 69 204 606 2,020

2.4× 3.7× 4.5× 5.2× -

with an equal increase in TPC-H performance. We use roofline analysis [67] to demonstrate that the kernels used in cuDF to run the key operators in TPC-H queries are mostly instruction bound instead of being memory bound. Furthermore, we show that the instruction throughput that is available is underutilized on both GPUs for compute bound kernels, indicating that much more work needs to be devoted to implementing relational operators on GPUs. We make three recommendations to alleviate this new bottleneck, recommendations that should help in developing a new generation of more efficient operators on GPUs that make better use of the GPU. These insights are also useful to inform the development of future GPUs oriented towards data analytics rather than AI workloads. We also identify which cuDF kernels make up more than 90% of the workload on both GPUs, indicating which kernels are most affected by the new instruction bottleneck and those that should be redesigned first. The main contributions of the paper include: • Valk, a tool and visualization for performance analysis of queries on GPUs. We open source Valk including all experimental data. • A roofline analysis that showing how cuDF kernels moved from being memory bound to compute bound. • A performance analysis that identifies three sources of ineffiencies when the memory wall is lifted. • A cuDF kernel analysis that identifes the most important kernels for executing TPC-H and how their performance shifts across the two GPUs.

2

BACKGROUND

We introduce relevant GPU concepts, the roofline model used to analyse kernel performance, and the cuDF library.

2.1

GPUs

GPUs consist of streaming multiprocessors (SM) divided into four subpartitions (SMSP), each one with a scheduler and multiple warps. A warp consists of 32 threads, and implements a Single Instruction, Multiple Threads execution model (SIMT [52]). A warp issues one instruction for all threads in the warp. Every clock cycle, each scheduler picks one warp that may issue one instruction to the instruction pipelines on the SM. The maximum instructions per cycle (IPC) per SM is therefore equal to the number of subpartitions. The achieved IPC may be lower, if in one clock cycle all warps in the

subpartition are stalled. Stalls occur if the warp needs to wait before issuing the next instruction, e.g., when waiting for an instruction dependency or memory latency. The IPC can be increased by either decreasing dependencies between instructions, i.e., increasing instruction level parallelism (ILP)[28, 65, 66], or by increasing the number of active warps [47], i.e., increasing occupancy. The memory hierarchy of a GPU consists of one L1 cache per SM, and an L2 cache shared by all SMs, backed up by device memory, the RAM of the GPU. The most fine grained access unit of device memory is a sector, which consists of 32 consecutive bytes. GPUs execute programs by launching kernels. Kernels are programs executed over a large amount of threads, where groups of threads are subdivided into thread blocksexecuted on a SM. A kernel might launch more thread blocks than there are SMs. In this case, the thread blocks are executed across multiple waves. When there are multiple waves, later thread blocks need to wait until the previous thread blocks finish executing on the SM. A kernel might not launch enough thread blocks to launch a full wave, or the last wave is not full, and then the thread blocks do not span all SMs and the GPU is underutilized during that wave.

2.2

Roofline Model

The roofline model [67] is a tool for performance analysis that shows whether kernels are bottlenecked by memory bandwidth or instruction throughput. Canonically, the model consists of a graph with instructions/operations per byte of memory traffic on the x-axis, and instructions/operations per second on the y-axis. Two additional lines are drawn on the graph. The first line is a horizontal line that indicates the maximum theoretical instruction throughput. The second line is a diagonal line that crosses the first line and indicates the maximum memory bandwidth. These two lines form the roofline. Additionally, a ridge line can be drawn, which is a vertical line that crosses through the point where the diagonal memory bandwidth line and the horizontal floating point operations throughput line intersect. Every kernel that is executed on the GPU can be plotted in this graph as a dot. If a kernel is located to the left of the ridge line, the kernel is memory bound. Increasing the memory bandwidth will speed up this kernel. If a kernel is located to the right of the ridge line, the kernel is compute bound. Increasing the instruction throughput will speed up the kernel. The kernel must be below either of the rooflines, as no kernel can achieve more than the theoretical limit. The closer a kernel is to this roofline, the more efficiently it utilizes the full potential of the hardware. In this paper, we use the instructions per second (instruction throughput) based roofline from Ding and Williams [18], created for workloads that only rarely execute floating point operations. In cuDF kernels, floating point operations are rare. In section 5 we show that less than one percent of the issued instructions are floating point instructions. See Figure 2 for an example for the L4 and GH200. This instruction throughput we consider in this paper is for warp instructions, the instructions that a warp issues for all threads active in the warp, and can execute an integer, memory, control, or similar operation. Instead of using operational intensity, which is the number of floating point operations per DRAM byte, instruction intensity is used, the number of (warp) instructions per

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

/s

2

4,0

1010

L4 GH200

Ratio

/s

Memory capacity (GiB) L2 cache (MiB) L1 cache / SM (KiB)

22 48 128

95 60 256

4.31× 1.25× 2.00×

ridge 44.0 inst./sector

L4: 413 G inst./s

B 2G

SM count SMSP per SM Threads per SM Base clock (GHz) Boost clock (GHz) Measured SM clock (GHz)

58 4 1,536 0.80 2.04 1.78

132 4 2,048 1.53 1.98 1.96

2.28× 1.00× 1.33× 1.92× 0.97× 1.10×

Instr. throughput (G inst./s) Memory bandwidth (GB/s) Roofline ridge (inst./sector)

413 300 44.0

1,037 2.51× 4,022 13.42× 8.3 0.19×

B 0G

30 ridge 8.3 inst./sector

Performance (inst. / s)

1012

1011

Table 2: Comparison of L4 and GH200.

GH200: 1,037 G inst./s

101 100 102 Instruction intensity (inst. / DRAM sector) 2.3 Figure 2: Roofline comparison of the L4 and GH200 GPUs.

DRAM sector. DRAM sectors are used instead of bytes, as sectors are the actual access unit in which memory is fetched from DRAM. For example, a random memory access during a join will fetch an entire sector from memory, even if only a few bytes are relevant. For the memory bandwidth, we take the maximum memory bandwidth as reported by Nsight Systems [49]. To calculate the (warp) instruction throughput of a GPU, we use the number of streaming multiprocessors (NSM ), the number of subpartitions per streaming multiprocessor (NSMSP per SM ), and the measured clock frequency (CFMeasured ). GPUs dynamically set their clock frequency and lower the clock frequency when needed to match thermal requirements. Therefore we do not use the theoretical max clock frequency, but measure the clock frequency during all 22 TPC-H kernels and all scale factors. We found the clock frequency to be stable, and therefore use the average. The instruction throughput is calculated as follows:

Inst. Throughput = NSM × NSMSP per SM × CFMeasured This formula gives the number of warp instructions per second that a kernel can execute if all SMs on the GPU are active. It is important to note that the number of floating point operations per second that a GPU can execute is different from the number on warp instructions per second due to two reasons. 1) Warp instructions execute an instruction for all active threads within the warp, so a warp instruction can map to 32 operations, or more when the instruction performs multiple operations within an instruction, such as a fused multiply-add instruction. 2) GPUs use CUDA cores for most instructions, but also have access to tensor cores, specialized floating point operations cores that can execute many more floating point operations per cycle per thread than even a fused multiplyadd instruction [50]. Tensor cores are not used by cuDF.

cuDF

cuDF is an open source library of optimized relational database operators for GPUs originally developed for Data Frames. cuDF uses a columnar data layout compatible with Arrow [6]. Data processing workloads on GPUs with cuDF use the operator API. The operator API of cuDF calls one or more kernels, with each kernel completing a separate computation to execute the operator. These kernels can be small, e.g., when a single value needs to be computed and communicated to the CPU to decide the next step in the operator execution; or very large, when e.g., probing a build table as part of a join. The user of the API has no control over how and which kernels are launched. cuDF is used in many GPU database systems, such as SiriusDB [73], Presto/Velox [10], GQE [39], Theseus [8], BlazingSQL [1], and Maximus [31]. In section 7, we list the most important kernels making up more than 90% of the aggregated kernel execution time and explain what they do.

3

EXPERIMENTAL SETUP

In what follows, we discuss the experimental setup and workloads used in the rest of the paper.

3.1

Hardware

We benchmark TPC-H on an L4 GPU and on a single GPU from a GH200. The GH200 system consists of a Grace CPU and one to four Hopper architecture GPUs analogous to the H100 SXM GPU [20]. As the GPU in the GH200 does not have an individual name, we refer to the single GPU as “GH200”. We chose the L4 and GH200 because they differ substantially in price and memory bandwidth while remaining otherwise comparable as their underlying architecture (Ada Lovelace and Hopper, respectively) belongs to the same hardware generation [20]. This makes them ideal for comparing kernel performance with low and high memory bandwidth. Table 2 shows the main hardware properties of both GPUs, and the difference between them as a ratio. The GH200 has both more device memory capacity as well as larger caches for both L1 and L2. The GH200 has 13.4× more memory bandwidth but only 2.51× more cuDA core instruction throughput. Figure 2 shows how this disproportionate increase causes the roofline ridge line to shift from

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

Table 3: Information each profiler provides, and the data Valk extracts from each.

Cost Profiler None NSYS NCU (NC) NCU (D)

Metrics

Role in Valk

Query Kernel Query Kernel NVTX System-wide Per-kernel Data overhead overhead timing timing regions metrics counters · + ++ +++

– · · ++

✓ ✗ ✗ ✗

✗ ✓ ✓ ✗

✗ ✓ ✗ ✗

✗ ✓ ✗ ✗

✗ ✗ ✓ ✓

baseline, validation timeline, system metrics time-sensitive counters time-insensitive counters

✓ provided, ✗ not available or distorted. Overhead: · none, +/++/+++ low/moderate/high; – not measurable. NSYS=Nsight Systems, NCU (NC)=Nsight Compute with no clock control, NCU (D)=Nsight Compute with default settings.

a kernel capable of executing 44 instructions per sector in the L4 before the kernel becomes compute bound, to being able to execute 8.3 instructions per sector in the GH200 before becoming compute bound. A kernel that is located at the ridge line on the GH200 can execute 5.3× fewer instructions per memory access than in the L4 to avoid becoming compute bound. The GH200 has 2.51× more instruction throughput than the L4, mostly due to the 2.28× increase in SMs, as the clock frequency is only slightly higher on the GH200, and the number of SMSPs of both GPUs is equal. The GH200 has more threads per SM, but the same number of SMSPs. On the L4, the scheduler has fewer warps to select instructions from, but because the number of SMSPs is equal, the number of instructions per cycle that can be executed per SM is equal on both GPUs. Because the higher instruction throughput is due to the increase in SMs, kernels find it harder to exploit the full throughput on the GH200 than the L4, as kernels need to launch more thread blocks to create at least one full wave of thread blocks. The question we answer in the rest of the paper is how these hardware characteristics affect cuDF kernels, and thus the performance of database engines relying on cuDF.

3.2

Workloads and Execution

We benchmark TPC-H using Maximus on scale factors 1, 3, 10, and 30. Scale factor 100 goes out of memory on the L4 for most queries, so it is not included in the comparison. However, we do publish scale factor 100 results for the GH200 in our artifacts and in Valk. We aim to measure how kernel performance differs across GPUs, without including any overhead from the database system that uses cuDF. Therefore, we define the query execution time as the sum of kernel execution times. Any overhead in creating query plans, or in executing the query plan from the CPU is ignored, as it can vary greatly per system and highly depends on the implementation. We also assume that all data is resident on the GPU before the query starts, as the interconnect overhead depends on the interconnect used, and how well the system can overlap query data transfer with query execution. The role of the interconnect has already been extensively studied in previous work [27, 32, 40, 74, 75]. We execute each query five times, and take the measurements of the last repetition. We use cuDF version 25.12 and compile and run everything in a Docker image (cuda:13.0.3-cudnn-devel-ubuntu24.04) to ensure that the environments are identical across both GPUs.

4

VALK

As part of our analysis efforts, we have built Valk. Valk is an open source tool for studying analytics on GPUs. It works as a meta profiler, and combines data from multiple profilers and workload information from Maximus into a unified dataset. The dataset is structured as a set of Parquet [7] files. All results reported in this paper are based on this unified dataset. Valk offers a user interface that displays the combined data from the profilers and engine specific information. It shows timelines, roofline graphs, queryable tables, bar charts, and query plans, giving the user a comprehensive overview on how the query is executed on the GPU. Valk is a useful tool for understanding query execution on GPUs but also provides crucial insights on how to do performance analysis on a GPU.

4.1

Dataset

Data sources. Currently the tool supports the following data sources: execution traces from Maximus, system profiling data from NVIDIA Nsight Systems [49], and per kernel hardware counters data from NVIDIA Nsight Compute [48]. Each of these data sources contain different slices of information on how a query was executed on the GPU. Individually, none of these tools provides a complete view of the execution of a query, motivating the need for a tool that combines the information from all profilers. Table 3 summarizes the execution overhead, measured metrics, and information ingested by Valk for each profiler. Besides running queries with a profiler active, Valk also runs the query without any profiler active. This run is used to measure the query execution time without any profiling overhead. Then Valk runs the query once with Nsight Systems, and two times with Nsight Compute with different settings active. Nsight Systems. Nsight Systems is a low-overhead profiler that returns detailed execution information such as timestamps of all kernels, memory copies, and NVTX regions [51]. These timestamps can be used to construct a timeline of all GPU events. NVTX regions are a method to annotate code regions with semantic information, and is used in Valk to detect which operator is active at any point in time. Nsight Compute. Running a query with Nsight Compute imposes a high overhead on the end-to-end query execution time, but returns

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

detailed kernel execution information using hardware counters. The query is executed twice with Nsight Compute, once with default settings, and once with clock control disabled. Nsight Compute will repeat each kernel many times in order to collect a large set of metrics. Each repetition collects a different subset of metrics. In order to combine the data from multiple kernel repetitions, Nsight Compute fixes by default the clock frequency to the base frequency, as this avoids distortions from different repetitions running with different clock frequencies. However, executing kernels with a clock fixed to base clock frequency lowers the instruction throughput, increasing the kernel execution time, and distorts time sensitive measurements such as executed instructions per second. Additionally, the GPU’s memory clock is separate from the GPU’s main clock, and runs at a separate frequency that is constant (and is not changed by Nsight Compute). This distorts the roofline of kernels run with a fixed base clock, as the instruction throughput is lower, but the memory bandwidth is the same. Therefore we run the query with Nsight Compute a second time with clock control disabled, and collect only a limited set of time-sensitive metrics that can be collected within a single repetition. This bypasses the constraint that, for multiple repetitions, the clock frequency must be constant.

Table 4: Ratio of end-to-end query execution times in comparison to no profiler, and ratio of sum of kernel execution times in comparison to Nsight Systems’ measurement. Individual kernel execution times are unknown without a profiler.

L4 Profiler

Query

Kernel

Query

Kernel

None NSYS NCU (NC) NCU (D)

1.00× 1.04× 7.57× 3,057×

– 1.00× 0.96× 1.36×

1.00× 1.14× 126× 4,989×

– 1.00× 1.01× 1.21×

NSYS=Nsight Systems, NCU (NC)=Nsight Compute with no clock control, NCU (D)=Nsight Compute with default settings.

for kernel execution times as the end to end query overhead is smaller, and the timeline depends on Nsight Systems’ timestamps.

4.2 Combining Profiler Data. Valk combines the data by validating that each profiler executed the query identically and then matches all data from different profilers to stitch together a complete view of the query’s performance. We instrumented Maximus with more detailed logging to log query plans and operator statistics, such as row counts, selectivity, and match ratios. These logs form an execution trace that can be compared across query executions by different profilers to ensure each profiler executed the query in the same way. Then we match data from Nsight Systems with Nsight Compute by comparing the profiled kernels from both, and if the count, order, and names match, we can combine the data from both profilers. In our dataset Valk was able to match the profiler data for almost all queries, except for Q2 at SF3 & Q11 at SF 1 and 3 on GH200, and Q11 at SF 1 on the L4. These missing queries appeared at low scale factors and only account for 10 milliseconds of execution time out of a total of 7.9 seconds of execution for all sacle factors, queries, and GPUs, resulting in less than 0.2% of total time not being matched. They are excluded from the dataset. Profiler Overhead. In Table 4 the overhead of each profiler is shown, split in end-to-end query execution overhead and kernel timing overhead. This data shows that using Nsight Systems to construct a timeline is possible, as the end-to-end query overhead is minimal. The overhead is slightly larger for smaller scale factors than larger scale factors, indicating that NSYS has a fixed overhead cost per measurement that is amortized more effectively on larger scale factors. On the GH200 the overhead is larger, as all queries are processed faster, so the fixed overhead cost distorts the timings more. The overhead of Nsight Systems on individual kernels can not be measured, as running without any profiler can not measure individual kernel times. Nsight Compute with default settings can not be used for time-sensitive metrics, as the fixed base clock distorts the kernel execution times. Nsight Compute with clock control disabled matches the kernel times of Nsight Systems closely. Nsight Systems’ measurements are considered the source of truth

GH200

User Interface

The user interface of Valk is based on three pages. The first page is the matrix page, which shows an overview of the execution times and relative speedup of GH200 over L4 of all queries across all scale factors (including SF100 for GH200) to identify interesting queries to analyse. After query selection, the user is directed to the second page. The second page is the core performance analysis page, the inspector page. It contains interactive data visualizations, such as query timeline, roofline graphs, bar graphs, query plan, and queryable tables. The third page, the schema page, gives users an explanation of the metrics shown in the inspector. Inspector overview. The inspector page offers many different data visualizations as shown in Figure 3. The main visualization is the timeline view, which shows the duration and order of operators, kernels, and memory copies. To the right of the timeline is the sidebar, which has four subtabs: roofline, bar graphs, kernels, and details. Below the timeline and sidebar is the metrics table. This table contains all metrics that the visualizations are based on, and enables the user to inspect all data interactively. Timeline. The timeline has three bars: the upper bar is the operator bar, the middle bar is the memory bar, and the lower bar is the compute bar. The operator bar shows when each operator is active, and each operator type has a unique color. The memory and compute bar both show when kernels and memory copies are active. The memory bar is overlaid with a line graph of memory bandwidth usage, showing three lines for: read bandwidth percentage, write bandwidth percentage, and total bandwidth percentage. The compute bar is also overlaid with a line graph, one line for percentage of streaming multiprocessors active, and one line for percentage of instruction throughput. The timeline section also has a sub tab for showing the query plan. Sidebar. The sidebar’s main tab is the roofline graph, which shows each kernel as a circle, where larger circles indicate that that kernel was active for longer, and kernels that executed the same function

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

Figure 3: The inspector page of the Valk UI. share the same color. The sidebar also has tabs for three additional visualizations. 1) The bar graphs tab shows for each kernel the distribution in instructions per pipeline, and the distribution of stall reasons. 2) The kernels tab shows for a selection of multiple kernels, how much time is spent percentage wise in each distinct kernel, and how many times each distinct kernel is executed. 3) The details tab shows summary metrics depending on the current selection. Table. The metrics table in the lower half of the inspector shows the full dataset that all other visualizations are based upon. For every kernel, a wide range of metrics is shown. The table is interactive, clicking on a row in the table will highlight that kernel in the timeline, clicking on a column header will sort the table by that column, and ALT clicking a column header will show summary statistics such as sum, mean and standard deviations, for that metric in the details tab. The table is fully SQL queryable, above the table is a text input in which users can enter SQL queries, and when those queries are executed the table will show the result of that query. The kernels in the query result are highlighted in the timeline as well. This queryability of the tables allows users to perform powerful data analysis, and relate hardware counter data from Nsight Compute to timeline data from Nsight Systems.

5

ROOFLINE ANALYSIS

We perform a roofline analysis on all cuDF kernels used at all scale factors of TPC-H to understand why cuDF’s speedup on the GH200 GPU over the L4 GPU is limited. We show the kernel execution time distribution across memory and compute bounds for both GPUs. To

Table 5: Instruction mix per GPU. Integer arithmetic forms the majority of instructions, floating point is rare. Instruction Class

L4

GH200

Integer Arithmetic Control Memory Bitwise Operations Uniform Inter-thread Comm. Conversion Float Arithmetic

61.80% 13.82% 10.65% 4.72% 3.79% 2.09% 0.44% 0.29%

56.74% 12.38% 13.21% 4.59% 7.61% 1.86% 0.42% 0.27%

Other

2.40%

2.93%

illustrate how the distribution shifts, we show a heatmap of kernel execution time on the rooflines of each GPU. Instruction Types. Table 5 shows the instructions issued by cuDF kernels when executing TPC-H. Less than one percent of the issued instructions are floating point instructions. A majority of the instructions are integer instructions, with control and memory instructions accounting for around a quarter of all instructions. Memory vs Compute Bound. Table 6 shows the distribution of kernel time per GPU across roofline classification: memory versus compute bound. Furthermore, we show how often memory bound kernels utilize less than 75% of the available memory bandwidth,

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

L4

Memory Bound Memory Bandwidth ≥75% Memory Bandwidth <75%

74.1% 42.5% 31.7%

22.5% 9.2% 13.4%

Compute Bound Instruction Throughput ≥75% Instruction Throughput <75%

25.9% 3.5% 22.4%

77.5% 7.4% 70.1%

and how often compute bound kernels utilize less than 75% of the available instruction throughput. We expressly choose 75% as a rather low boundary, since a kernel that cannot reach even this value is certainly not bandwidth or throughput bound. On the L4, most time (74%) is spent in memory bound kernels, of which a majority uses more than 75% of memory bandwidth. In contrast, the GH200 spends the most time (78%) in compute bound kernels, of which a majority utilize less than 75% of the available instruction throughput. Insight 1 | New Bottleneck: On a GPU with low memory bandwidth, kernels are memory bound. On a GPU with high memory bandwidth, the kernels become compute bound and fail to fully exploit the available instruction throughput.

Causes. To analyse the shift in bottleneck, we plot the execution time of all kernels on a combination of a roofline and a logarithmic heatmap (Figure 4). The heatmap shows the time distribution of all kernels in TPC-H up to and including SF 30 on a roofline plot. When we compare the heatmap of the L4 with the heatmap of the GH200, we observe two effects. 1) Due to the shifted ridge line, more kernels become compute bound. 2) Because of the larger increase in memory bandwidth, kernels that were memory bound experience relatively more speedup in comparison to compute bound kernels, so on the GH200 less time is spent on memory bound kernels.

6

235

10 2

232

10 3

229

10 4

226

10 5

Memory bound: 74%

20

24

Compute bound: 26%

GH200

28

212

10 6

238

10 2

235

10 3

232

10 4

229

time per bin

GH200

10 1

238

time per bin

L4

Performance (Inst. / s)

Kernel Time

Performance (Inst. / s)

Table 6: Percentage of per-GPU kernel time spent in memoryand compute-bound kernels when executing TPC-H, split by whether the utilization of the bottleneck reaches 75%.

10 5

226 Memory bound: 23%

Compute bound: 77%

10 6

20 28 212 24 Instruction Intensity (Inst. / DRAM Sector) Figure 4: Logarithmic heatmap of kernel execution time plotted over a roofline graph of all cuDF kernels in TPC-H per GPU. Due to the larger increase in memory bandwidth than in instructions per second, the ridge line shifts, and more kernels become compute bound.

PERFORMANCE ANALYSIS

Due to the larger increase in memory bandwidth compared to the increase in instruction throughput, cuDF has become compute bound and the kernels are not fully utilizing the instruction throughput. Worse, now that cuDF is compute bound, cuDF will not benefit as much from the trend of GPUs being equipped with more memory bandwidth. Therefore it is imperative that cuDF kernels (and data processing kernels in general) account for this shift in hardware balance, make use of the available instruction throughput as much as possible, and optimize for the new compute bottleneck. In this section, we identify three recommendations on how to optimise cuDF kernels for the new compute bottleneck. Beforehand, we note that, at lower scale factors, kernels severely underutilize the GPU which lowers the individual kernel’s performance, but is not problematic for the system as a whole the execution of those

kernels can be overlapped although it is not clear that it makes sense to use a GPU for such small data sizes. We start the analysis by noting that the instructions per cycle (IPC) is low across all kernels, and we breakdown why by examining the stall reasons. We split the stall reason analysis into compute bound kernels and memory bound kernels. For compute bound kernels, we note that occupancy can be increased on both GPUs, but that even with ideal occupancy, peak IPC can not be reached in a large part of the kernels. Instruction-level parallelism (ILP) should also be increased. For memory bound kernels, we show that both the L1 and L2 cache do not achieve peak throughput in the large majority of kernels. Cache-aware algorithms could help reduce the average memory access latency, which would cause the number of stalls due to memory stalls to decrease as well.

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

% of Time

60%

L4 GH200

40%

Table 7: Geometric mean IPC for memory bound and compute bound kernels on both GPUs. Compute bound kernels only achieve marginally more than half of maximum IPC. Kernels

L4

GH200

20%

All Kernels

0.96

1.95

0%

Memory Bound Compute Bound

0.58 2.09

0.90 2.32

1

3

10

TPC-H scale factor

30

Memory (any tier BW 75%) Memory (all tiers BW < 75%) Dependency

Figure 5: The percentage of time spent in a kernel that does not launch at least one full wave of threads.

Finally, we show that even if stalls are perfectly overlapped and all kernels reach peak IPC, the majority of time on the GH200 would be spent in kernels that are instruction throughput bound. Those kernels will still not be exploiting the hardware trend of relatively more available memory bandwidth and relatively smaller increases in instruction throughput. Thus, even if all kernels achieve peak IPC, the speedup of GPU data processing would still largely be bound by increases in instruction throughput. Therefore we conclude that kernels need to decrease their instruction intensity to exploit the larger increases in memory bandwidth.

6.1

Saturation

At smaller scale factors, kernels tend to launch fewer thread blocks, and this results in more kernels that do not manage to launch even a single full wave of thread blocks. Figure 5 shows that at SF 1, 22% and 52% of kernel execution time on the L4 and GH200 respectively is spent in kernels that do not launch at least one full wave. As instruction throughput is the product of the number of SMs and clock frequency, not using all SMs diminishes the maximum instruction throughput kernels can reach. Undersaturation has a larger effect on kernels on the GH200, because the increase in instruction throughput over the L4 is largely due to the higher SM count, as the clock frequency is approximately equal. The effect is reduced at large scale factors, but underutilization might occur when using several GPUs in parallel to process a query over skewed data.

Insight 2 | Undersaturation: Kernels find it harder to exploit the full instruction throughput of GPUs with many SMs, especially when data sizes are small.

We do not classify these undersaturated kernels as problematic, as the database using cuDF can use multiple streams of kernels to run kernels concurrently. An individual kernel might not exploit the full instruction throughput, but the set of concurrent kernels as a whole could. Thus, kernels should be optimized to saturate GPUs with many SMs by exposing more data parallelism across SMs.

Synchronization Throttle

L4 mem.-bound (74% of time)

35

26

L4 comp.-bound (26% of time)

29

33

GH200 mem.-bound (23% of time)

27

GH200 comp.-bound (77% of time)

34

0

37

12

9

Other Issued + ready 11 15

11

8

35

20 40 60 80 Share of warp cycles (%)

15

100

Figure 6: Stall breakdown per GPU and kernel bound. Memory stalls are split up into stalls due to throughput (a kernel achieving more than 75% bandwidth on L1, L2, or main memory) and stalls due to latency (a kernel achieving less than 75% bandwidth on L1, L2, and main memory).

6.2

Instructions Per Cycle

Table 7 shows that the IPC on both L4 and GH200 is 0.96 and 1.95 respectively, less than half of maximum IPC (4.0 = number of SMSP per SM). The IPC of memory bound kernels is lower than that of compute bound kernels, as memory bound kernel instructions are bound by memory bandwidth. However, even when only considering the IPC of compute bound kernels, the IPC is less than 60% of maximum IPC on both GPUs. Insight 3 | Low IPC: The instructions per cycle for compute bound kernels is low on both GPUs.

Stalls. To investigate the reasons for low IPC, we show a breakdown of stall reasons per GPU and split in memory and compute bound kernels in Figure 6. As expected, the number of memory stalls is higher in memory bound kernels, on both GPUs. We split memory stalls up per kernel, based on whether that kernel achieved more than 75% bandwidth on any tier of the memory hierarchy (L1, L2, main memory). This distinction is made because kernels that saturate bandwidth at any tier will have more stalls stalled on memory requests delayed due to throughput constraints. However, a kernel that does not saturate the bandwidth of any tier in the memory hierarchy and encounters a memory stall is stalled due to the latency of the memory request. In memory bound kernels, many kernels are still bound by memory latency related stalls. This is a

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

Kernel time where occupancy x (%)

Achieved occupancy

100 75 50 25 0

L4

Theoretical occupancy

GH200

78% 47%

60%

Table 8: Share of kernel time reaching at least 2, 3 or 4 instructions per cycle (IPC), counting only kernels that stay below 75% of L1, L2 and memory bandwidth. In the scaled column every kernel’s occupancy is scaled to 100%, where IPC grows linearly in the added warps. Even under that ideal occupancy, only 51.1% of GH200 kernel time reaches peak IPC, and none of L4’s.

20%

0 25 50 75 100 0 25 50 75 100 Occupancy x (%) Occupancy x (%)

L4 IPC (% of peak) 2.0 (50%) 3.0 (75%) 4.0 (100%)

GH200

Measured

Scaled

Measured

Scaled

23.3% 6.1% 0.0%

55.4% 25.7% 0.0%

67.1% 8.6% 0.0%

88.1% 76.2% 51.1%

Figure 7: Occupancy is low on the GH200, and can still be increased on the L4.

result of memory bound kernels not always maximizing memory bandwidth, as was shown in Table 6. Compute Bound Stalls. Compute bound kernels encounter low IPC due to 1) memory latencies while not saturating throughput on any memory tier, and 2) instruction dependencies. These two categories cause 60-70% of all stalls for compute bound kernels. Both categories can be mitigated by overlapping latencies by either increasing occupancy or by increasing instruction level parallelism (ILP) [28, 65, 66]. Increasing occupancy hides latencies by having more warps active that can issue instructions, at the cost of fewer memory resources per warp [47]. With instruction level parallelism, instructions have fewer interdependencies, allowing warps to be able to issue a new instruction sooner. GPU kernels might draw inspiration from techniques such as the vectorized query execution model, which unrolls loops and uses branchless code that exposes large amounts of ILP to the compiler, which in turn increases IPC on CPUs [15, 77, 78]. Occupancy. The percentage of warps that are active is called the occupancy, which can be increased by reducing the number of resources, such as registers, that each warp uses. If there are more warps active, latencies can be hidden by overlapping work. The total number of resources on a SM is fixed, and the fewer resources an individual active warp uses, the more warps can be active simultaneously. Theoretical occupancy indicates how many warps can be active at the same time, while achieved occupancy shows how many warps were active averaged over all cycles a kernel was active. The achieved occupancy is smaller than theoretical occupancy, as some warps finish their work earlier than other warps. Figure 7 shows how much kernel time is spent at each level of occupancy. The occupancy is higher on the L4, as the L4 has fewer warps per SM, but more registers per warp. This decreases the register pressure on L4 SMs, and enables more occupancy. However, on both GPUs a majority of kernel time is spent in kernels with 75% occupancy or less, showing that kernels could be optimized to increase occupancy and hide more latency by having more warps active concurrently.

Insight 4 | Low Occupancy: Occupancy on both GPUs has room for improvement, showing that kernels could hide more latency with overlapping more work by increasing occupancy.

Instruction-level parallelism. Table 8 shows the measured IPC and the IPC when all kernels would reach 100% occupancy (disregarding any resource limits), assuming that IPC is increased linearly with the added active warps, and IPC is not bound by any memory tier. Kernels that utilise more than 75% bandwidth of L1, L2 or memory are excluded because IPC is not relevant for those kernels, and because their IPC is lower as they are bound by bandwidth. This would skew the IPC to be lower while not being relevant for those kernels. The table shows that even if kernels would achieve ideal occupancy, they would still not saturate instruction throughput and reach a high IPC. Therefore Table 8 shows that kernels need to increase their ILP, as only increasing occupancy would not achieve peak IPC. Also, kernels could consider increasing their ILP at the cost of occupancy, as this trade off is a performance optimization technique that can result in more performant kernels if the benefit of increasing ILP is higher than the cost of decreasing occupancy. Decreasing occupancy gives individual warps more resources, such as registers, which allows kernels to keep more data in registers, which can decrease memory spilling, enable different algorithm implementations, and increase opportunities for ILP [65, 66]. Insight 5 | Low Instruction-Level Parallelism: Increasing occupancy alone would not result in peak IPC, showing that kernels need to consider other methods such as increasing instruction level parallelism as well.

Recommendation 1 | Overlap Latency: To increase IPC, kernels need to hide latencies by overlapping more work by increasing occupancy and/or instruction-level parallelism.

Kernel time where saturation x (%)

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

100 75 50 25 0

Memory BW

L4

L2 BW

L1 BW

GH200

85%

Instruction throughput

L4

Max of all

GH200

85% 63%

49% 19% measured

latency hidden: 1.36x speedup

measured

14% 13% 9%

53% latency hidden: 1.56x speedup

22% 17% 7%

0 25 50 75 100 0 25 50 75 100 0 25 50 75 100 0 25 50 75 100 Saturation x (%) Saturation x (%) Saturation x (%) Saturation x (%)

Figure 8: The left plots show the measured saturation across kernel time for instruction throughput and the utilized bandwidth of L1, L2 and memory. The right plots show saturation when IPC is increased until at least one of the resources reaches 100% saturation. The max of all line shows the highest saturation on any resource by kernel time.

Kernel time where hit ratio x (%)

L2 hit ratio

100 75 50 25 0

L4

L1 hit ratio

the underutilized cache throughput, memory latencies from main memory accesses can be reduced.

Max of both

GH200 86%

71% 42%

51%

0 25 50 75 100 0 25 50 75 100 Hit ratio x (%) Hit ratio x (%)

Figure 9: Cache hit ratios by kernel time and GPU.

6.3

Resource utilization

Figure 8 shows the resource utilization across L1, L2, & memory bandwidth and instruction throughput by kernel time. The figure shows that the L4 achieves the highest resource utilization on memory bandwidth, while the GH200 utilizes instruction throughput the most, in line with what Figure 4 showed. The ’Max of both’ line shows the maximum resource utilization of across all of the four resources by kernel time. L4 kernels utilize at least 75% of throughput of at least one resource 49% of the time, while kernels on the GH200 only use more than 75% of throughput of any resource 19% of the time. This shows that on the GH200, 81% of the time kernels are not throughput bound on any of the four resources. Caches Stalls due to memory when none of the tiers are not throughput bound can be reduced by decreasing the latency of memory accesses. Figure 8 shows that on both GPUs, very little kernel time utilizes the full throughput of the L1 or L2 cache. This underutilization is not due to low cache hit ratios, Figure 9 shows that the cache hit ratios are reasonable. By using algorithms or kernel implementations that more heavily rely on caches and exploit

Recommendation 2 | Utilize Caches: To decrease stalls due to memory latencies, kernels should increase cache utilization.

6.4

Instruction Intensity

Finally, Figure 8 shows that it is not enough to only increase IPC. Figure 8 shows in the right two plots what happens if kernels would be able to increase IPC up until at least one memory tier reaches 100% utilization, or IPC is maximized. The point at which each resource line crosses the right border of the plot, indicates how much kernel time is bounded by that resource when IPC would be increased and the resourse utilization is linearly scaled by the same amount. Memory bandwidth would be the limiting factor for 63% of kernel time on the L4, while only 9% of kernels would be instruction throughput bound. 27% of kernel time would be bounded by L1 or L2 cache throughput. For the GH200, 53% of kernel time would be bounded by instruction throughput, while only 7% of kernel time would be bounded by memory bandwidth. 39% of kernel time would be bounded by L1 or L2 cache throughput. Insight 6 | Instruction Bottleneck: Even if peak IPC is achieved, kernels on the high bandwidth GPU are bound by instruction throughput. This shows that with the current trend in GPU hardware, the bottleneck for data processing kernels is moving away from being memory bound to compute bound.

Recommendation 3 | Decrease Instruction Intensity: Kernels should aim to execute fewer (warp) instructions per

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

Table 9: Key performance metrics of the 13 kernels that make up more than 90% of the runtime on both GPUs at SF 30. L4 Kernel

Op

transform_kernel retrieve DeviceMergeSortMergeKernel DeviceRadixSortOnesweepKernel DeviceMergeSortBlockSortKernel compute_column_kernel count insert_if_n static_kernel DeviceScanKernel single_pass_shmem_aggs_kernel gather_chars_fn_string_parallel gather_chars_fn_char_parallel Total

GH200

#Var

Speedup

#

Sum

Mem.%

IPC

#

Sum

Mem.%

IPC

JGFPO J GO GO GO FP J J JGO JGFPO G JFO JGFO

26 2 2 2 2 2 2 2 15 9 1 2 4

8.3× 2.8× 6.7× 12.3× 3.6× 3.1× 3.0× 10.5× 8.1× 8.4× 3.4× 4.6× 3.8×

514 49 66 32 17 73 49 58 186 163 10 10 140

20.3% 10.7% 10.0% 8.2% 7.7% 6.9% 6.4% 6.1% 5.4% 3.3% 3.3% 3.0% 2.3%

81±27% 37±19% 54±40% 90±1% 13±4% 71±21% 59±17% 65±4% 85±12% 85±13% 43±2% 75±4% 80±22%

0.59±1.12 1.79±0.47 0.31±0.16 0.20±0.09 1.72±0.40 1.60±0.58 2.10±0.95 0.33±0.16 0.27±0.27 0.19±0.13 1.22±0.02 1.64±0.38 1.61±0.82

515 49 66 32 17 73 49 58 192 163 12 10 140

12.1% 19.0% 7.4% 3.3% 10.5% 10.9% 10.5% 2.9% 3.3% 2.0% 4.8% 3.3% 3.0%

45±36% 7±7% 29±16% 71±9% 4±3% 15±9% 13±10% 42±11% 44±27% 46±20% 9±1% 21±6% 21±13%

1.90±1.57 2.42±0.15 0.94±0.55 1.02±0.43 1.89±0.65 2.28±0.29 2.64±0.25 1.62±0.50 0.94±0.40 0.67±0.33 1.31±0.06 2.82±0.07 2.74±0.62

JGFPO

71

5.0×

1367

93.7%

64±31%

0.96±0.95

1376

92.9%

22±24%

1.99±0.90

Op=Operators that use the kernel: J=Join, G=Group By, F=Filter, P=Project, O=Order By, #Var=Number of templated variations, #=Number of invocations, Sum=Sum of execution times as percentage of total, Mem.%=Percentage of maximum memory bandwidth used. IPC=Weighted arithmetic mean and standard deviation of instructions per SM cycle.

DRAM sector access to be able to exploit the large increases in memory bandwidth in newer GPUs.

7

KERNEL ANALYSIS

In this section, we analyse the GPU kernels that dominate aggregate execution time across the 22 TPC-H queries on L4 and GH200. The 13 kernels listed in Table 9 account for more than 90% of total kernel time on both GPUs and are therefore load-bearing for TPCH performance. Our goal is to determine how their performance and bottlenecks shift between the two GPUs. Overall, memory utilization on GH200 is only one third of that on L4, as kernels become instruction-bound on the larger GPU. We contextualize this trend by examining selected kernels in detail. The most expensive kernel by aggregate time, and the one with the most variants, is transform_kernel, largely because of its high invocation count. It applies a user-defined functor or lambda to the input element-wise and, through indirection, also implements gather logic (out[i]=in[map[i]]) for materialization. It therefore spans a wide range of instruction intensities and access patterns, giving it the most diverse performance profile of any kernel we study. Its backbone is identical across functors: a single unrolled main loop that exposes more in-flight instructions. It does not, however, use vectorized loads and stores, leaving memory bandwidth under-utilized. Its memory utilization is lower on GH200 than on L4, implying that loop unrolling alone does not saturate GH200’s memory bandwidth. Use cases with frequent invocations and sequential accesses should therefore consider specialized kernels with vectorized loads and stores wherever possible. The radix sort kernel (DeviceRadixSortOneSweepKernel) achieves the highest memory utilization on both GPUs. It reads unsorted keys and values sequentially and writes the sorted sequences back sequentially, using shared memory as a write-combining buffer,

and it unrolls aggressively to expose instruction-level parallelism. As a result, GH200 outperforms L4 by a large factor, yet it still reaches only 71% memory utilization compared to 90% on the L4. This mirrors the observation made for transform_kernel: sequential, high-ILP kernels that saturate L4’s bandwidth leave a gap on GH200. The compute_column_kernel evaluates arbitrary expression trees in cuDF through an interpretation mechanism, which costs many additional instructions. Despite its sequential memory accesses, it reaches only around 15% memory utilization on GH200, compared to 71% on L4. This is a clear example of a kernel turning from memory-bound to instruction-bound when moving from L4 to GH200, and it suggests that on GPUs with abundant memory bandwidth, reducing instruction count at the cost of additional sequential memory accesses can pay off. Two other kernels show low memory utilization on both GPUs, and lower still on GH200. These kernels heavily rely on shared memory, which is located in the L1 cache, instead of main memory. single_pass_shmem_aggs_kernel aggregates values per block in shared memory, avoiding a global memory update for every aggregated value, and DeviceMergeSortBlockSortKernel likewise sorts in shared memory. The hash join kernels insert_if_n (builds hash table), count (counts probe-side matches), and retrieve (materializes matches) read their input sequentially but probe the hash table randomly. The insert_if_n kernel is memory bound on both GPUs, but suffers from long memory latencies into uncached rows in main memory. The count and retrieve kernels are memory bound on the L4, but become compute bound on the GH200 due to the ridge line shift limiting their speedup across the two GPUs. For the rest of the listed kernels we only briefly explain what they do. The static_kernel executes a for_each loop over a range of indices using a user-supplied lambda. DeviceScanKernel computes a prefix sum over the results of a fused per-element functor, mainly to turn string sizes into a string column’s offsets. The

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

two gather_chars kernels perform a gather operation on string columns to reorder them after an operator reordered the column. In summary, the most expensive TPC-H kernels use significantly less memory bandwidth on the GH200 than on the L4. For kernels with sequential access patterns, high instruction counts and non-vectorized accesses, both inconsequential on the L4, become limiting on the GH200.

8

RELATED WORK

CPU Performance Analysis. Hardware-level performance analysis for query processing on CPUs is a well-studied subject. Boncz et al. studied how databases should be optimized for a main memory bandwidth bottleneck, as in-memory databases became feasible over disk-based databases [14]. Ailamaki et al. [5] investigated where in-memory databases were leaving performance on the table by breaking down the reasons behind stalls of the CPUs, and proposing optimizations to reduce stalls. Later, Boncz et al. showed how instead of a Volcano-style execution model [24], a vectorized execution model could reduce stalls and increase IPC by increasing instruction-level parallelism [15]. Kersten et al. [35] compared how vectorized execution and compiled query execution [43] differ in performance and usage of the CPU. GPU Performance Analysis. Previously, GPU performance analysis has mostly focused on SSB queries [53] and prototype operator implementations, and do not consider performance across GPUs with large differences in memory bandwidth and instruction throughput balance. Yuan et al. [75] predict SSB query performance as implemented by their own restricted set of operators by creating a model that calculates the amount of data accessed and assumes that all queries are memory bandwidth bound. Shanbhag et al. [60] create a model with the same assumptions, for their own tile-based operator implementations. Cao et al. [16] predict SSB query performance of multiple GPU database systems with a roofline based model. They predict performance when a GPU is split into multiple subunits using MIG [17], where each subunit has a similar balance of memory bandwidth and instruction throughput. Furthermore, there are numerous studies on the role of CPU-GPU and GPU-GPU interconnects for query processing [12, 27, 32, 40, 68, 74, 75] that complement our study of the performance of cuDF kernels. Alternative Implementations. There are also GPU database systems that do not use cuDF and implement operators themselves: TQP [9, 25] and its successor Coddspeed [29] are closed source and build operators on top of PyTorch [55], BOSS [42] builds operators on top of ArrayFire [72], HeavyDB [2] compiles queries into kernels with LLVM [37], and Beischl et al. [12] implement a subset of SSB queries with their own operator implementations for Apple general purpose GPUs. Additionally, there is a significant amount of studies that do not implement a complete GPU database system, but implement, optimise and analyse a specific operator [19, 22, 23, 26, 33, 34, 36, 41, 54, 58, 59, 61–64, 69–71]. Database Performance Analysis Tools. To the best of the authors’ knowledge, Valk is currently the only tool that combines detailed GPU kernel performance data with database specific context data such as query plans and operator statistics. Beischl et

al. [11, 13] created a profiler for Umbra [44] that combines detailed performance data with database specific context, but Umbra is a CPU-based system. Quent [3] is an experimental framework that can combine telemetry data from GPUs and database specific context, but does not offer detailed performance data on kernels. Outside of the database domain, there is a wide variety of performance analysis tools for GPUs that provide different data and insights, but none of them link the performance data with database specific context [4, 38, 48, 49, 76]. Valk can expand it’s data sources by directly using the underlying libaries of these related tools, such as CUPTI [45] or NVML [46].

9

CONCLUSION

In response to the increasing importance of AI workloads, GPUs have increased memory bandwidth in comparison to CUDA core instruction throughput. We show that TPC-H performance does not keep up with memory bandwidth, and use Valk, our performance analysis tool, to investigate why. We show that on a low-bandwidth GPU, cuDF kernels are memory bound, but on a high bandwidth GPU cuDF kernels become compute bound and do not fully exploit the available instruction throughput. We then demonstrate that throughput utilization could be improved by increasing occupancy and/or instruction-level parallelism to overlap work and hide latencies, and that cuDF kernels can utilize caches more efficiently to decrease memory latencies. Finally, we suggest that, even if latencies are completely hidden and peak instruction throughput is reached, cuDF kernels are still instruction throughput bound on the GH200 and will not benefit from the current trends in GPU hardware. Therefore, the number of warp instructions per sector access needs to be decreased as well. These are important insights to improve the implementation of relational operators on GPUs and to inform the architecture of GPUs to better support data analytics.

ACKNOWLEDGMENTS This work has been supported by a grant from the Swiss National Supercomputing Centre (CSCS) under project ID sm94 on Alps, and an unrestricted research donation from NVIDIA.

REFERENCES [1] [n.d.]. BlazingSQL. https://github.com/BlazingDB/blazingsql Accessed: 202607-16. [2] [n.d.]. HeavyDB. https://github.com/heavyai/heavydb Accessed: 2026-07-16. [3] [n.d.]. Quent. https://github.com/rapidsai/quent Accessed: 2026-08-01. [4] L. Adhianto, S. Banerjee, M. Fagan, M. Krentel, G. Marin, J. Mellor-Crummey, and N. R. Tallent. 2010. HPCTOOLKIT: tools for performance analysis of optimized parallel programs http://hpctoolkit.org. Concurr. Comput.: Pract. Exper. 22, 6 (April 2010), 685–701. [5] Anastassia Ailamaki, David J. DeWitt, Mark D. Hill, and David A. Wood. 1999. DBMSs on a Modern Processor: Where Does Time Go?. In Proceedings of the 25th International Conference on Very Large Data Bases (VLDB ’99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 266–277. [6] Apache Software Foundation. 2026. Apache Arrow. https://arrow.apache.org/. [7] Apache Software Foundation. 2026. Apache Parquet. https://parquet.apache.org/. Accessed: 2026-07-19. [8] Felipe Aramburú, William Malpica, Kaouther Abrougui, Amin Aramoon, Romulo Auccapuclla, Claude Brisson, Matthijs Brobbel, Colby Farrell, Pradeep Garigipati, Joost Hoozemans, Supun Kamburugamuve, Akhil Nair, Alexander Ocsa, Johan Peltenburg, Rubén Quesada López, Deepak Sihag, Ahmet Uyar, Dhruv Vats, Michael Wendt, Jignesh M. Patel, and Rodrigo Aramburú. 2025. Theseus: A Distributed and Scalable GPU-Accelerated Query Processing Platform Optimized for Efficient Data Movement. arXiv:2508.05029 [cs.DC] https://arxiv.org/abs/ 2508.05029

Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

[9] Yuki Asada, Victor Fu, Apurva Gandhi, Advitya Gemawat, Lihao Zhang, Dong He, Vivek Gupta, Ehi Nosakhare, Dalitso Banda, Rathijit Sen, and Matteo Interlandi. 2022. Share the tensor tea: how databases can leverage the machine learning ecosystem. Proc. VLDB Endow. 15, 12 (Aug. 2022), 3598–3601. https://doi.org/10. 14778/3554821.3554853 [10] Daniel Bauer, Luis Garces-Erice, Deepak Majeti, Zoltan Arnold Nagy, Sean Rooney, Greg Kimball, Devavret Makkar, Todd Mostak, and Karthikeyan Natarajan. 2026. Accelerating Presto with GPUs. arXiv:2606.24647 [cs.DB] https: //arxiv.org/abs/2606.24647 [11] Alexander Beischl, Timo Kersten, Maximilian Bandle, Jana Giceva, and Thomas Neumann. 2021. Profiling dataflow systems on multiple abstraction levels. In Proceedings of the Sixteenth European Conference on Computer Systems (Online Event, United Kingdom) (EuroSys ’21). Association for Computing Machinery, New York, NY, USA, 474–489. https://doi.org/10.1145/3447786.3456254 [12] Alexander Beischl, Mykola Morozov, Michael Jungmair, Lukas Haussmann, and Thomas Neumann. 2026. Evaluating Apple Silicon for Data Processing. In Proceedings of the 22nd International Workshop on Data Management on New Hardware (DaMoN ’26). Association for Computing Machinery, New York, NY, USA, Article 4, 10 pages. https://doi.org/10.1145/3789237.3809127 [13] Alexander Beischl and Thomas Neumann. 2025. UmbraPerf - Profiling Results Tailored for DBMS Developers. Proc. VLDB Endow. 18, 12 (Aug. 2025), 5291–5294. https://doi.org/10.14778/3750601.3750654 [14] Peter A. Boncz, Stefan Manegold, and Martin L. Kersten. 1999. Database Architecture Optimized for the New Bottleneck: Memory Access. In Proceedings of the 25th International Conference on Very Large Data Bases (VLDB ’99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 54–65. [15] Peter A Boncz, Marcin Zukowski, and Niels Nes. 2005. MonetDB/X100: HyperPipelining Query Execution.. In Cidr, Vol. 5. 225–237. [16] Jiashen Cao, Rathijit Sen, Matteo Interlandi, Joy Arulraj, and Hyesoon Kim. 2023. GPU Database Systems Characterization and Optimization. Proc. VLDB Endow. 17, 3 (Nov. 2023), 441–454. https://doi.org/10.14778/3632093.3632107 [17] NVIDIA Corporation. (last accessed) 2026. NVIDIA Multi-Instance GPU. [Online] Available from: https://www.nvidia.com/en-us/technologies/multiinstance-gpu/. [18] Nan Ding and Samuel Williams. 2019. An Instruction Roofline Model for GPUs. In 2019 IEEE/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS). 7–18. https://doi.org/10.1109/PMBS49563. 2019.00007 [19] Harish Doraiswamy, Vikas Kalagi, Karthik Ramachandra, and Jayant R. Haritsa. 2023. A Case for Graphics-Driven Query Processing. Proc. VLDB Endow. 16, 10 (June 2023), 2499–2511. https://doi.org/10.14778/3603581.3603590 [20] Luigi Fusco, Mikhail Khalilov, Marcin Chrapek, Giridhar Chukkapalli, Thomas Schulthess, and Torsten Hoefler. 2024. Understanding Data Movement in Tightly Coupled Heterogeneous Systems: A Case Study with the Grace Hopper Superchip. arXiv:2408.11556 [cs.DC] https://arxiv.org/abs/2408.11556 [21] Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, and Kurt Keutzer. 2024. AI and Memory Wall. IEEE Micro 44, 3 (2024), 33–39. https://doi.org/10.1109/MM.2024.3373763 [22] Diego Gomes Tomé, Tim Gubner, Mark Raasveldt, Eyal Rozenberg, and Peter Boncz. 2018. Optimizing group-by and aggregation using GPU-CPU coprocessing. In Proceedings of the International Workshop on Accelerating Analytics and Data management Systems Using Modern Processor and Storage Architectures. 1–10. [23] Naga Govindaraju, Jim Gray, Ritesh Kumar, and Dinesh Manocha. 2006. GPUTeraSort: high performance graphics co-processor sorting for large database management. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data (Chicago, IL, USA) (SIGMOD ’06). Association for Computing Machinery, New York, NY, USA, 325–336. https://doi.org/10.1145/1142473.1142511 [24] G. Graefe. 1994. Volcano — An Extensible and Parallel Query Evaluation System. IEEE Trans. on Knowl. and Data Eng. 6, 1 (Feb. 1994), 120–135. https://doi.org/ 10.1109/69.273032 [25] Bingsheng He, Mian Lu, Ke Yang, Rui Fang, Naga K. Govindaraju, Qiong Luo, and Pedro V. Sander. 2009. Relational query coprocessing on graphics processors. ACM Trans. Database Syst. 34, 4, Article 21 (Dec. 2009), 39 pages. https://doi. org/10.1145/1620585.1620588 [26] Bingsheng He, Ke Yang, Rui Fang, Mian Lu, Naga Govindaraju, Qiong Luo, and Pedro Sander. 2008. Relational joins on graphics processors. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (Vancouver, Canada) (SIGMOD ’08). Association for Computing Machinery, New York, NY, USA, 511–524. https://doi.org/10.1145/1376616.1376670 [27] Jiong He, Mian Lu, and Bingsheng He. 2013. Revisiting co-processing for hash joins on the coupled CPU-GPU architecture. Proc. VLDB Endow. 6, 10 (Aug. 2013), 889–900. https://doi.org/10.14778/2536206.2536216 [28] John L. Hennessy and David A. Patterson. 2017. Computer Architecture, Sixth Edition: A Quantitative Approach (6th ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. [29] Matteo Interlandi, Nicolas Bruno, Brandon Haynes, Carlo Curino, Rathijit Sen, Yinan Li, Kaushik Rajan, Bailu Ding, Lukas M. Maas, Wei Cui, Kevin Gaffney,

Mingsheng Hong, Brian Kroth, Sampath Rajenda, Peng Cheng, Surajit Chaudhuri, Johannes Gehrke, Raghu Ramakrishnan, Lidong Zhou, Momin Al-Ghosien, Craig Peeper, Marius Dumitru, Conor Cunningham, Kevin Bocksrocker, Prasanna Sundar, Ron Boskovic, YongChul Kwon, Marko Zivanovic, Runbin Shi, Krystian Sakowski, Denis Lamtsov, Josep Aguilar Saborit, Krish Srinivasan, Adarsh Kapil, Andrew Putnam, Anudeep Kambapu, Aparna Konduri, Aaron Landy, Aaron Moore, Aaron Pitman, Artem Oks, Ashit Gosalia, Babu Kandimalla, Blake Pelton, Bogdan Crivat, Cesar Galindo-Legaria, Chidamber Kulkarni, Jesus Camacho Rodriguez, Evgeny Babin, Hans Lehnert Marino, Igor Konev, Israel Arroyo, Luis Penaranda, Mandar Datar, Morten Borup Petersen, Nicholas Simmons, Parminder Poonian, Pravija Danda, Rob Rydberg, Esteban Calvo, Ronak Bajaj, Sai Sumanth Kammal Shetty, Sharanya Bhat, and Zach Iracheta. 2026. CoddSpeed: Hardware Accelerated Query Processing in Microsoft Fabric. In Companion of the International Conference on Management of Data (India) (SIGMOD Companion ’26). Association for Computing Machinery, New York, NY, USA, 359–372. https://doi.org/10.1145/3788853.3803077 [30] Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun, Shigang Li, and Torsten Hoefler. 2021. Data Movement Is All You Need: A Case Study on Optimizing Transformers. In Proceedings of Machine Learning and Systems 3 (MLSys 2021). [31] Marko Kabić, Shriram Chandran, and Gustavo Alonso. 2025. Maximus: A Modular Accelerated Query Engine for Data Analytics on Heterogeneous Systems. Proc. ACM Manag. Data 3, 3, Article 187 (June 2025), 25 pages. https://doi.org/10.1145/ 3725324 [32] Marko Kabić, Bowen Wu, Jonas Dann, and Gustavo Alonso. 2025. Powerful GPUs or Fast Interconnects: Analyzing Relational Workloads on Modern GPUs. Proc. VLDB Endow. 18, 11 (July 2025), 4350–4363. https://doi.org/10.14778/3749646. 3749698 [33] Tim Kaldewey, Guy Lohman, Rene Mueller, and Peter Volk. 2012. GPU join processing revisited. In Proceedings of the Eighth International Workshop on Data Management on New Hardware (Scottsdale, Arizona) (DaMoN ’12). Association for Computing Machinery, New York, NY, USA, 55–62. https://doi.org/10.1145/ 2236584.2236592 [34] Tomas Karnagel, René Müller, and Guy M. Lohman. 2015. Optimizing GPU-accelerated Group-By and Aggregation. In ADMS@VLDB. https://api. semanticscholar.org/CorpusID:5017248 [35] Timo Kersten, Viktor Leis, Alfons Kemper, Thomas Neumann, Andrew Pavlo, and Peter Boncz. 2018. Everything you always wanted to know about compiled and vectorized queries but were afraid to ask. Proc. VLDB Endow. 11, 13 (Sept. 2018), 2209–2222. https://doi.org/10.14778/3275366.3284966 [36] Artem Kroviakov, Petr Kurapov, Christoph Anneser, and Jana Giceva. 2024. Heterogeneous Intra-Pipeline Device-Parallel Aggregations. In Proceedings of the 20th International Workshop on Data Management on New Hardware (Santiago, AA, Chile) (DaMoN ’24). Association for Computing Machinery, New York, NY, USA, Article 3, 10 pages. https://doi.org/10.1145/3662010.3663441 [37] Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization (Palo Alto, California) (CGO ’04). IEEE Computer Society, USA, 75. [38] Mao Lin, Keren Zhou, and Pengfei Su. 2023. DrGPUM: Guiding Memory Optimization for GPU-Accelerated Applications. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 (Vancouver, BC, Canada) (ASPLOS 2023). Association for Computing Machinery, New York, NY, USA, 164–178. https://doi.org/10.1145/3582016.3582044 [39] Clemens Lutz, Tyler Allen, Miloni Atal, Viktor Rosenfeld, and Eric Schmidt. 2026. Designing GPU-Accelerated Query Engines with NVIDIA GQE. NVIDIA Technical Blog. https://developer.nvidia.com/blog/designing-gpu-acceleratedquery-engines-with-nvidia-gqe/, accessed 2026-07-17. [40] Clemens Lutz, Sebastian Breß, Steffen Zeuch, Tilmann Rabl, and Volker Markl. 2020. Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD ’20). Association for Computing Machinery, New York, NY, USA, 1633–1649. https://doi.org/10.1145/3318464. 3389705 [41] Clemens Lutz, Sebastian Breß, Steffen Zeuch, Tilmann Rabl, and Volker Markl. 2022. Triton Join: Efficiently Scaling to a Large Join State on GPUs with Fast Interconnects. In Proceedings of the 2022 International Conference on Management of Data (Philadelphia, PA, USA) (SIGMOD ’22). Association for Computing Machinery, New York, NY, USA, 1017–1032. https://doi.org/10.1145/3514221.3517911 [42] Hubert Mohr-Daurat, Xuan Sun, and Holger Pirk. 2025. BOSS - An Architecture for Database Kernel Composition. SIGMOD Rec. 54, 1 (April 2025), 37–46. https: //doi.org/10.1145/3733620.3733629 [43] Thomas Neumann. 2011. Efficiently compiling efficient query plans for modern hardware. Proc. VLDB Endow. 4, 9 (June 2011), 539–550. https://doi.org/10.14778/ 2002938.2002940 [44] Thomas Neumann and Michael J. Freitag. 2020. Umbra: A Disk-Based System with In-Memory Performance. In 10th Conference on Innovative Data Systems Research, CIDR 2020, Amsterdam, The Netherlands, January 12-15, 2020, Online

Sven Hepkema, Bowen Wu, Christos Kozyrakis, Yannis Chronis, and Gustavo Alonso

Proceedings. www.cidrdb.org. https://vldb.org/cidrdb/2020/umbra-a-disk-basedsystem-with-in-memory-performance.html [45] NVIDIA Corporation. 2026. NVIDIA Management Library (CUPTI). https: //docs.nvidia.com/cupti/. Accessed: 2026-08-01. [46] NVIDIA Corporation. 2026. NVIDIA Management Library (NVML). https: //developer.nvidia.com/management-library-nvml. Accessed: 2026-08-01. [47] NVIDIA Corporation. 2026. NVIDIA Nsight Compute (NCU). https://docs.nvidia. com/cuda/cuda-c-best-practices-guide. Accessed: 2026-07-19. [48] NVIDIA Corporation. 2026. NVIDIA Nsight Compute (NCU). https://developer. nvidia.com/nsight-compute. Accessed: 2026-07-19. [49] NVIDIA Corporation. 2026. NVIDIA Nsight Systems (NSYS). https://developer. nvidia.com/nsight-systems. Accessed: 2026-07-19. [50] NVIDIA Corporation. 2026. NVIDIA Tensor Cores. https://www.nvidia.com/enus/data-center/tensor-cores/. Accessed: 2026-08-01. [51] NVIDIA Corporation. 2026. NVIDIA Tools Extension (NVTX). https://docs. nvidia.com/tools-extension/index.html. Accessed: 2026-07-19. [52] NVIDIA Corporation. 2026. NVIDIA Writing SIMT Kernels. https://docs.nvidia. com/cuda/cuda-programming-guide/02-basics/writing-cuda-kernels.html. Accessed: 2026-08-01. [53] Patrick O’Neil, Elizabeth O’Neil, Xuedong Chen, and Stephen Revilak. 2009. The Star Schema Benchmark and Augmented Fact Table Indexing. In Performance Evaluation and Benchmarking, Raghunath Nambiar and Meikel Poess (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 237–252. [54] Sungwoo Park, Seyeon Oh, and Min-Soo Kim. 2025. cuMatch: A GPU-based Memory-Efficient Worst-case Optimal Join Processing Method for Subgraph Queries with Complex Patterns. Proc. ACM Manag. Data 3, 3, Article 143 (June 2025), 28 pages. https://doi.org/10.1145/3725398 [55] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: an imperative style, high-performance deep learning library. Curran Associates Inc., Red Hook, NY, USA. [56] RAPIDS. (last accessed) 2026. cuDF: A GPU DataFrame Library. [Online] Available from: https://github.com/rapidsai/cudf. [57] Viktor Rosenfeld, Sebastian Breß, and Volker Markl. 2022. Query Processing on Heterogeneous CPU/GPU Systems. ACM Comput. Surv. 55, 1, Article 11 (Jan. 2022), 38 pages. https://doi.org/10.1145/3485126 [58] Viktor Rosenfeld, Sebastian Breß, Steffen Zeuch, Tilmann Rabl, and Volker Markl. 2019. Performance Analysis and Automatic Tuning of Hash Aggregation on GPUs. In Proceedings of the 15th International Workshop on Data Management on New Hardware (Amsterdam, Netherlands) (DaMoN’19). Association for Computing Machinery, New York, NY, USA, Article 8, 11 pages. https://doi.org/10.1145/ 3329785.3329922 [59] Ran Rui and Yi-Cheng Tu. 2017. Fast Equi-Join Algorithms on GPUs: Design and Implementation. In Proceedings of the 29th International Conference on Scientific and Statistical Database Management (Chicago, IL, USA) (SSDBM ’17). Association for Computing Machinery, New York, NY, USA, Article 17, 12 pages. https: //doi.org/10.1145/3085504.3085521 [60] Anil Shanbhag, Samuel Madden, and Xiangyao Yu. 2020. A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD ’20). Association for Computing Machinery, New York, NY, USA, 1617–1632. https://doi.org/10.1145/3318464.3380595 [61] Panagiotis Sioulas, Periklis Chrysogelos, Manos Karpathiotakis, Raja Appuswamy, and Anastasia Ailamaki. 2019. Hardware-Conscious Hash-Joins on GPUs. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). 698–709. https://doi.org/10.1109/ICDE.2019.00068 [62] Evangelia A. Sitaridi and Kenneth A. Ross. 2013. Optimizing select conditions on GPUs. In Proceedings of the Ninth International Workshop on Data Management on New Hardware (New York, New York) (DaMoN ’13). Association for Computing Machinery, New York, NY, USA, Article 4, 8 pages. https://doi.org/10.1145/ 2485278.2485282 [63] Elias Stehle and Hans-Arno Jacobsen. 2017. A Memory Bandwidth-Efficient Hybrid Radix Sort on GPUs. In Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD ’17). Association for Computing Machinery, New York, NY, USA, 417–432. https: //doi.org/10.1145/3035918.3064043 [64] Wenbo Sun, Asterios Katsifodimos, and Rihan Hai. 2023. An Empirical Performance Comparison between Matrix Multiplication Join and Hash Join on GPUs. In 2023 IEEE 39th International Conference on Data Engineering Workshops (ICDEW). 184–190. https://doi.org/10.1109/ICDEW58674.2023.00034 [65] Vasily Volkov. 2016. Understanding Latency Hiding on GPUs. Ph.D. Dissertation. EECS Department, University of California, Berkeley. http://www2.eecs.berkeley. edu/Pubs/TechRpts/2016/EECS-2016-143.html [66] Vasily Volkov and James W. Demmel. 2008. Benchmarking GPUs to tune dense linear algebra. In Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (Austin, Texas) (SC ’08). IEEE Press, Article 31, 11 pages.

[67] Samuel Williams, Andrew Waterman, and David Patterson. 2009. Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52, 4 (April 2009), 65–76. https://doi.org/10.1145/1498765.1498785 [68] Bowen Wu, Wei Cui, Carlo Curino, Matteo Interlandi, and Rathijit Sen. 2025. Terabyte-Scale Analytics in the Blink of an Eye. Proc. VLDB Endow. 19, 2 (Oct. 2025), 141–155. https://doi.org/10.14778/3773749.3773754 [69] Bowen Wu, Marko Kabić, Sven Hepkema, Vasilis Mageirakos, Christos Kozyrakis, and Gustavo Alonso. 2026. Eiger: An Efficient Library for GPU-based Data Analytics. arXiv:2607.04489 [cs.DB] https://arxiv.org/abs/2607.04489 [70] Bowen Wu, Dimitrios Koutsoukos, and Gustavo Alonso. 2025. Efficiently Processing Joins and Grouped Aggregations on GPUs. Proc. ACM Manag. Data 3, 1, Article 39 (Feb. 2025), 27 pages. https://doi.org/10.1145/3709689 [71] Makoto Yabuta, Anh Nguyen, Shinpei Kato, Masato Edahiro, and Hideyuki Kawashima. 2017. Relational Joins on GPUs: A Closer Look. IEEE Transactions on Parallel and Distributed Systems 28, 9 (2017), 2663–2673. https://doi.org/10. 1109/TPDS.2017.2677451 [72] Pavan Yalamanchili, Umar Arshad, Zakiuddin Mohammed, Pradeep Garigipati, Peter Entschev, Brian Kloppenborg, James Malcolm, and John Melonakos. 2015. ArrayFire - A high performance software library for parallel computing with an easy-to-use API. https://github.com/arrayfire/arrayfire [73] Bobbi Yogatama, Yifei Yang, Kevin Kristensen, Devesh Sarda, Abigale Kim, Adrian Cockcroft, Yu Teng, Joshua Patterson, Gregory Kimball, Wes McKinney, Weiwei Gong, and Xiangyao Yu. 2026. Rethinking Analytical Processing in the GPU Era. [74] Bobbi W. Yogatama, Weiwei Gong, and Xiangyao Yu. 2022. Orchestrating data placement and query execution in heterogeneous CPU-GPU DBMS. Proc. VLDB Endow. 15, 11 (July 2022), 2491–2503. https://doi.org/10.14778/3551793.3551809 [75] Yuan Yuan, Rubao Lee, and Xiaodong Zhang. 2013. The Yin and Yang of processing data warehousing queries on GPU devices. Proc. VLDB Endow. 6, 10 (Aug. 2013), 817–828. https://doi.org/10.14778/2536206.2536210 [76] Keren Zhou, Xiaozhu Meng, Ryuichi Sai, and John Mellor-Crummey. 2021. GPA: a GPU performance advisor based on instruction sampling. In Proceedings of the 2021 IEEE/ACM International Symposium on Code Generation and Optimization (Virtual Event, Republic of Korea) (CGO ’21). IEEE Press, 115–125. https://doi. org/10.1109/CGO51591.2021.9370339 [77] Marcin Zukowski, Sándor Héman, and Peter Boncz. 2006. Architecture-conscious hashing. In Proceedings of the 2nd International Workshop on Data Management on New Hardware (Chicago, Illinois) (DaMoN ’06). Association for Computing Machinery, New York, NY, USA, 6–es. https://doi.org/10.1145/1140402.1140410 [78] Marcin Zukowski, Sandor Heman, Niels Nes, and Peter Boncz. 2006. SuperScalar RAM-CPU Cache Compression. In Proceedings of the 22nd International Conference on Data Engineering (ICDE ’06). IEEE Computer Society, USA, 59. https://doi.org/10.1109/ICDE.2006.150

Related documents

• Rotary GPU: Exploring Local Execution Paths for Large Mixture-of-Experts Models Under Limited GPU Memory#238571
• GPU-Accelerated First-Order Method with Randomized Sampling for Binary Integer Programs#1062806
• Performance Isolation and Semantic Determinism in Efficient GPU Spatial Sharing#2722
• Performance modeling and optimization techniques for heterogeneous computing#957827
• Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives#381713