arXiv CS open access

Performance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study

Gianmarco Accordi et al. · arxiv_cs

arXiv CS · Papers

cloud distributed-computing parallel-computing

Achieving performance portability is challenging for irregular scientific applications built as repeated multi-kernel GPU pipelines, because throughput depends on out-of-kernel decisions as much as on kernel efficiency. Using molecular docking as a representative case study, we propose an out-of-ker

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Performance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study Gianmarco Accordi∗ , Leonardo Beltrame∗ , Theresa Pollinger† , Jens Domke† , Davide Gadioli∗ , Gianluca Palermo∗ ∗ Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy

arXiv:2611.06207v1 [cs.DC] 4 Nov 2026

{gianmarco.accordi, leonardo.beltrame, davide.gadioli, gianluca.palermo}@polimi.it † RIKEN Center for Computational Science, Kobe, Japan {jens.domke, theresa.pollinger}@riken.jp

a representative throughput-oriented workload in which a speed-up directly increases the screening capacity. Structurebased virtual screening uses docking to rank large libraries of compounds, called ligands, against one or more protein targets, allowing researchers to focus downstream experimental validation on the most promising candidates [4]–[6]. Because docking simulations are much cheaper than experimental screening, modern campaigns increasingly target very large chemical libraries, sometimes reaching hundreds of millions of compounds [7]–[9]. This trend makes portable GPU efficiency a first-order concern for docking pipelines. Docking is also a challenging workload for portable performance. Ligands differ in atom count, flexibility, and interaction patterns, so the cost and memory footprint of processing each ligand can differ substantially [10]. This irregularity makes batching, resource usage, and concurrent device use especially important for endto-end throughput. Prior work on out-of-kernel optimizations has already shown that these decisions can be as important as kernel-level tuning [11]. I. I NTRODUCTION In this paper, we study this problem using muDock [12], High-performance computing (HPC) plays a central role [13], a compact molecular docking mini-app designed to in reducing the time-to-solution of large-scale scientific ap- preserve the key computational patterns of larger docking codes plications. As production systems increasingly rely on GPU while enabling systematic experimentation across platforms and acceleration, however, achieving this objective across different backends. Our goal is to study whether a more portable out-ofvendors and software ecosystems becomes more difficult in kernel formulation can improve throughput in a repeated multipractice. Deploying a scientific application across platforms kernel GPU pipeline. Instead of relying on occupancy-driven requires not only functional portability, but also tuning choices batch sizing, which depends on backend-specific information that remain effective despite differences in compilers, runtimes, and assumptions, we study an alternative formulation based on and hardware resource behavior [1]–[3]. This problem is explicit resource bounds. In particular, we focus on batching particularly relevant for applications that are not dominated by policy, worker concurrency, and per-worker memory allocation a single monolithic kernel, but instead execute long sequences as coupled tuning dimensions, and evaluate how these choices of repeated kernel launches. In these applications, throughput transfer across CUDA, HIP, and SYCL implementations. This work makes the following contributions: depends on how inputs are grouped and submitted across the full pipeline, not only on the efficiency of one kernel in isolation. • We study an out-of-kernel tuning approach for repeated As a result, achieving portable performance requires looking multi-kernel GPU pipelines based on explicit resource at the full application pipeline, not only at kernel-level code bounds, treating batching policy, worker concurrency, generation. and per-worker memory allocation as the main tuning As an example, biological applications consume a signifdimensions. • We implement and evaluate this approach in muDock icant fraction of HPC resources, and virtual screening is Abstract—Achieving performance portability is challenging for irregular scientific applications built as repeated multi-kernel GPU pipelines, because throughput depends on out-of-kernel decisions as much as on kernel efficiency. Using molecular docking as a representative case study, we propose an out-ofkernel tuning methodology that treats batching policy, worker concurrency, and per-worker memory allocation as explicit optimization parameters, replacing backend-dependent occupancydriven choices with a more portable execution strategy based on explicit resource bounds. We implement this methodology in the muDock mini-app under a manager/worker execution model where multiple host workers share a GPU through independent streams. We then evaluate the methodology on NVIDIA, AMD, and Intel GPUs across CUDA, HIP, and SYCL implementations. Across all evaluated platform/backend pairs, the memory-bounded formulation exceeds the peak throughput of the occupancy-guided baseline. The fully tuned methodology improves the end-to-end throughput on all tested configurations, achieving an average speedup of 1.53× over the baseline. Index Terms—HPC, GPU, Molecular Docking, Performance Portability

on NVIDIA GH200, AMD MI250, and Intel Max 1550 systems using CUDA, HIP, and SYCL, showing that the tuned configuration improves end-to-end throughput by 1.53× on average over the reference configuration. The rest of the paper is organized as follows. Section II introduces the workload and portability context. Section III presents the batching policies and tuning dimensions studied in this work. Sections IV and V describe the experimental setup and the main results. Section VI discusses limitations and threats to validity. Section VII summarizes the most relevant prior work. Section VIII concludes the paper. II. BACKGROUND Drug discovery is a long and costly process with a high failure rate. One of the challenges is the sheer size of the chemical space that we can explore to find a promising drug candidate [14]. To mitigate this problem, a recent trend introduces an in-silico step that virtual screen a chemical library of ligands, to find the most promising ones to test in-vitro. Recent studies demonstrate how increasing the number of virtual screened ligands, increases the probability of finding good drug candidates as well [9]. In the context of a virtual screening campaign, it is of paramount importance to maximize the number of virtual screened ligands within the computation budget. For this reason, we can capitalize on every optimization opportunity by evaluating more ligands, instead of reducing the time to solution, making virtual screening a relevant example of an irregular throughput-oriented workload. To study this problem in a controlled setting, we use muDock as a compact molecular docking mini-app designed for benchmarking and portability studies. It preserves the main computational structure of larger docking codes while remaining compact enough to support systematic experimentation across platforms and backends. From the application point of view, muDock docks each ligand independently from the others, making the workload embarrassingly parallel. Docking a ligand means estimating the 3D displacement of its atoms when they interact with the target protein. To reach its goal it uses a genetic algorithm approach. Each individual’s genes encode geometric transformations to the ligands atoms. In particular, from the initial ligand pose, we encode in the first six genes the rigid rotations and translations along the x, y, and z axis. The other genes encode the angles of the ligand’s rotatable bonds. To evaluate the fitness of a population individual, muDock uses a scoring function that is functionally equivalent to the AutoDock force field described by Huey et al. [15]. As the population evolves, we expect the best individuals to converge toward the scoring function maximum. Figure 1 shows how the genetic algorithm maps with computation kernels. The idea is to compose a computation pipeline to produce the best 3D atom displacement in output, starting from the initial pose. The genetic stage initializes and updates the candidate poses explored during docking. The geometry stage applies the corresponding pose transformations to the ligand coordinates. The scoring stage evaluates the

transformed poses and returns the fitness values used by the next genetic iteration. After each iteration, the pipeline returns to the genetic stage until the stopping criterion is met. muDock uses a manager/worker execution model. In this context, a worker is a host process pinned to a CPU core and associated with one execution device selected by the user from the command line. The device can be either a GPU or a CPU core. For a GPU, the runtime can spawn multiple workers for the same device. For a CPU execution path, the runtime instead uses one worker for each CPU core. In this paper, it is assumed that the main compute power is provided by the GPUs of the node, and assigning workers on CPU is omitted. We use this model as a straightforward way of tightly coupling the different resources (data, compute, memory). Of course, there is a multitude of other approaches in the context of task-based runtime systems and performance portability frameworks to coordinate different kinds of resources, and even combinations [16]. However, the molecular docking workload allows for a straightforward one-dimensional partitioning of all the work, by grouping into batches. There are no data dependencies that warrant a complex task graph. Thus, by focusing on the manager/worker model, one can avoid unnecessary synchronization and data movement: every worker is responsible for a single GPU stream. Still, the presented insights should be largely transferable to the wider context of task-based runtime systems on GPU workloads. III. M ETHODOLOGY This section describes the out-of-kernel knobs that we investigate in this paper. While they are evaluated in the context of muDock, they are agnostic with respect to the mechanism used to orchestrate the computation. At a high level, our study focuses on two complementary aspects of the out-of-kernel execution strategy. The first aspect concerns how the work is grouped into batches under explicit resource constraints. The second aspect concerns how the workers coordinate and submit these batches to the GPU to overlap data transfer with computation and ensure that the pipeline remains busy. In this work, these two aspects are jointly analyzed rather than in isolation, because the effect of batch sizing depends on how much work can be kept in flight through concurrent submission and memory partitioning across workers. A. Batch sizing Grouping a set of inputs in a batch is an efficient way of using a GPU when we have data parallelism [10]. In this work, we investigate two different methods to automatically size the batch. 1) Occupancy-Guided Batching: The first method considered in this paper is occupancy-guided batching. Its goal is to form batches that keep the GPU fully occupied while grouping together ligands with similar characteristics, so that the ligands inside the same batch have a similar expected computation time. This follows the same rationale used in earlier docking work based on occupancy-guided execution [11].

Input ligands

GENETIC kernel initialization/update

GEOM kernel pose/transform stage

SCORE kernel fitness evaluation

GENETIC kernel selection/mutation

Stop criterion?

yes

Output

no (repeat)

Fig. 1: Overview of the genetic algorithm that muDock uses to dock a bucket of ligands. The rounded boxes represent kernel invocations, the circles represent the input and output of the algorithm, and the rhombus represents the termination condition.

Ligands are first grouped into classes according to their input features. For each class c, the runtime estimates the per-ligand memory footprint mlig (Ac ). Given the memory budget assigned to a worker, the runtime determines the largest safe batch size, Mavail Bmax (c) = , (1) mlig (Ac ) where Mavail is the memory budget assigned to the worker. This value is an estimated upper bound imposed by correctness and memory availability. In addition to Bmax , the runtime derives an occupancyrelated base multiple from backend helpers [11]. For one worker, let asm be the estimated number of active blocks per SM or compute unit returned from the backend, and let nsm be the number of SMs or compute units on the device. The value of asm is kernel dependent, since it reflects how the resource usage of the selected kernel maps onto the hardware limits of the target device. The occupancy-derived base multiple is then

The purpose of this method is different from the occupancyguided one. Instead of sizing the batch for the most constraining kernel in the pipeline, it tries to increase the amount of work in flight by allowing larger batches and concurrent workers to use the device together. This is particularly relevant for irregular multi-kernel workloads, where the different stages do not necessarily share the same preferred batch multiple. Under this method, the number of workers W and the memory budget M assigned to each worker become explicit tuning parameters. Each worker uses its own memory budget to build the next batch and submits work through an independent stream. The method therefore relies on worker concurrency and larger in-flight batches to improve overlap and to help hide latencies across the pipeline. B. Kernel execution policy

Computation efficiency on GPU is usually achieved by overlapping data transfer with computation. It is common to Bbase = max(1, asm ) · max(1, nsm ). (2) use W host workers that enqueue kernels on the same device. In muDock, a manager owns the shared input backlog and the The runtime can then use integer multiples of this quantity to shared output queue. The input backlog contains the ligand enlarge the batch while preserving the same occupancy-related descriptions that still need to be processed, while the output alignment, as long as the resulting batch still fits within the queue collects the completed results produced by the workers. available memory budget. The manager spawns one or more workers according to the In a repeated multi-kernel pipeline, however, the occupancyselected devices and backends. Each worker repeatedly removes related base multiple is not unique. Different kernels may work descriptions from the shared input backlog, prepares the expose different preferred multiples. In the experiments, we corresponding input data, and stores it in a local reordering combine these hints conservatively by selecting the minimum buffer. The role of this local buffer is to group inputs with value across the relevant kernels. This means that the batch is similar features before forming the next batch, reducing the dimensioned for the worst case, i.e., for the kernel with the imbalance inside the same device submission. When the inputs highest resource usage. This is appropriate for the limiting accumulated in the local buffer satisfy the batching rule of the kernel, but it can lower the occupancy reached by the other selected method, the worker forms a batch and submits it to kernels in the pipeline. Given this limitation, we also consider a the device. The worker then executes the target pipeline for simpler method which is agnostic with respect to the occupancy, all inputs in that batch and writes the completed results to the as it considers only the memory occupancy of batches, that shared output queue. This process continues until the input we describe in the following section. backlog has been drained. 2) Memory-Bounded Batching: The second method is This execution model exposes two levels of concurrency. memory-bounded batching. Here, the runtime still groups Inside the device, each submitted batch is processed by a ligands by class and still respects the same memory-safety sequence of GPU kernels. Outside the kernels, multiple host condition, but it resolves the batch size directly from the worker workers may concurrently prepare inputs, reorder them, form memory budget, batches, and submit work to the same device. As a consequence, B = Bmax . (3) end-to-end throughput depends not only on the behavior of In this case, the full memory budget assigned to a worker the computational stages, but also on how work is distributed is used to form the batch, without requiring alignment to an across workers, how inputs are grouped in the local buffers, occupancy-derived multiple. and when each worker decides that the next batch is ready

for execution. However, the two methods that we investigate Linux 9.6 with Linux 5.14 kernels, and GCC 13.2.0 is used in this paper to overlap data transfer with computation are as the host compiler across the campaign. On GH200, the agnostic with respect to the worker implementation. This setup native CUDA implementation is built with CUDA 12.6.2 exposes the main tradeoff studied in the paper. Increasing W and NVIDIA driver 590.48.01, while the HIP-on-NVIDIA can improve overlap and reduce launch-related idle time, but it configuration also resolves to the same CUDA 12.6.2 toolchain also reduces the memory M available to each worker and may underneath. The SYCL builds use an Intel LLVM/oneAPIincrease contention. For this reason, the worker count and the based compiler stack: on GH200 they target NVIDIA GPUs, per-worker memory budget are part of the methodology itself on MI250 they target AMD gfx90a, and on Max1550 they rather than fixed background settings. target intel:pvc. On Max1550, the SYCL runtime uses 1) Explicit synchronization: An established approach uses Intel oneAPI Unified Runtime over Level Zero 1.6.33578+42, explicit synchronization between two workers to implement with OpenCL NEO 25.18.33578 also available on the system. double buffering [10]. The idea is that while one thread is For AMD HIP, we use the ROCm 6.2.1 LLVM/AMDGPU executing a computation kernel on a buffer, another worker toolchain targeting gfx90a, together with the corresponding is transferring data to a second buffer. Once the kernel is rocrand and hiprand libraries. The Intel LLVM SYCL completed, the second worker can issue its own computation compiler used in the study was built from source from the kernel and the first worker can transfer data. This can be easily main branch at commit a123482f3. generalized to W workers. The benefit is that with this approach c) Workloads: The application performance is highly the developer is certain to overlap data transfer and computation input-dependent, as the execution time is strongly related regardless of the backend used to offload computation. The to the number of heavy atoms and rotamers [7]. For this drawback is that a worker needs to wait for another kernel to reason, the study focuses on the heterogeneous mixed_n500k complete before launching its own kernel. For this reason, this workload, built as a subsample of 500,000 compounds from the policy is usually paired with occupancy-guided batching [11] MEDIATE dataset [17]. The selected ligands are representative to maximize the computation efficiency. of different combinations of heavy-atom counts and rotamer 2) Concurrent execution: An alternative way relies on GPU counts, so that the workload preserves the irregular behavior streams and the underlying driver to overlap data transfer with that drives the batching problem. At the same time, the sampled computation. With this approach, each worker enqueues data distribution is intentionally biased toward a higher presence of transfer and computation kernels in a stream independently. For smaller ligands than larger ones, to better reflect a common example, in case of the CUDA backend, a stream is represented screening scenario in which lighter compounds are more by the CUDA stream. However, also the other backends expose frequent than the most complex molecules. All policy sweeps, to the developer a similar runtime to launch kernel concurrently. headline comparisons, and portability results discussed in the The benefit is that the resulting implementation is simpler, and paper are therefore derived from this mixed workload. it is possible to overlap data transfer and computation across the d) Throughput metric: Throughput is the main perforwhole execution. The drawback is that the ability of maximizing the efficiency is up to the backend runtime and the underlying mance metric of interest for the end user, since it directly GPU driver. This policy can be paired with memory-bounded estimates how many ligands can be screened within a fixed time batching to maximize the number of in-flight ligands on the budget. Runs are monitored over time through observer samples. For each experiment, we report the throughput measured device. when the input backlog has been drained, namely when all IV. E XPERIMENTAL S ETUP ligands have either already completed or are in flight inside This section provides the details about the platforms that we the pipeline. This choice lets us compare experiments that used to carry out the experimental campaign, the input dataset, complete naturally with experiments that are stopped after and how we measure the throughput. a fixed timeout, without penalizing runs whose input queue a) Hardware platforms: The campaign spans three pro- empties early. We introduce a timeout of 5 minutes to keep the duction GPU platforms from different vendors. The NVIDIA campaign duration reasonable, after observing that the pipeline GH200 measurements are collected on a node with a 72-core reaches a stable saturation regime well before that limit in the Arm Neoverse-V2 host processor, 477 GB of shared system valid configurations. Each experiment is executed five times; memory, and one NVIDIA GH200 GPU. The AMD MI250 we discard the first run as warm-up and average the remaining measurements are collected on a node with two AMD EPYC four. If one of the five repetitions fails, for example, because 7713 processors, 1,024 GB of system memory, and eight AMD the selected configuration exceeds the available memory budget, MI250 GPUs with 128 GB of memory each. The Max1550 we discard the entire set of repetitions and mark that experiment measurements are collected on a node with two Intel Xeon as not available (N/A). As a consequence, missing points in Platinum 8470 processors, 208 logical CPU cores in total, 2 TB the worker/memory and bucket-multiplier plots correspond to of system memory, and eight Intel Data Center GPU Max 1550 configurations for which at least one repetition could not be accelerators with 128 GB of memory each. completed, typically because the requested batching and worker b) Software stack: All nodes use Linux-based software configuration required more device memory than the device environments. The GH200 and Max1550 systems run Rocky could provide.

e) Performance Portability Metrics: To summarize the tuned results across platforms and backends, we use the application-efficiency view adopted from the muDock study on CPU performance portability [12] and grounded in the discussion of Pennycook and Sewall [1], [2]. For a platform p and a supported backend b, the application efficiency is e(b, p) =

T (b, p) , maxb′ ∈Bp T (b′ , p)

(4)

where T (b, p) is the measured throughput and Bp is the set of backends supported on platform p. This metric reports how close a backend is to the best tuned result available on the same platform. We then summarize these efficiencies across platforms using the arithmetic mean and the harmonic mean. For a backend supported on N platforms, the arithmetic mean is 1 X AM(b) = e(b, p), (5) N p while the harmonic mean is N . p 1/e(b, p)

HM(b) = P

(6)

The arithmetic mean gives a simple average-efficiency view. The harmonic mean is stricter because it penalizes lowefficiency cases more strongly and becomes zero when a backend is unsupported on one of the investigated platforms. For this reason, we use the harmonic mean as the main portability-oriented summary, while reporting the arithmetic mean as a complementary metric.

on its own GPU stream. We do not include Max1550/SYCL in this figure because the occupancy-guided method depends on backend information about the number of active units per kernel, and on Intel GPUs this quantity was returned through SYCL as a default value equivalent to using the whole compute unit regardless of the kernel under analysis. If the monolithic occupancy-guided assumption remained valid across multiple kernels, the two methods would stay close even as the number of workers increases. This is not the case. With one worker, the two configurations remain close, but they diverge once multiple workers are active. When locked GPU access is enabled, the workers cannot keep enough work in flight and the attainable occupancy remains lower. By contrast, under concurrent GPU access, the occupancy-guided approach already benefits from concurrent submissions and reaches a broader high-throughput region. The same trend also appears along the Bucket Multiplier axis: once the GPU is no longer used in exclusive mode, fewer workers are sufficient to sustain high throughput even at lower Bucket Multiplier values. In practice, concurrent GPU access reduces the need to rely on a large occupancy-aligned batch to keep the device busy, because overlap across workers already increases the amount of work in flight. B. Batching Policies Under Worker and Memory Scaling

This experiment compares occupancy-guided batching and memory-bounded batching under explicit worker and memory partitioning. The goal is to measure how the two batching methods behave on an irregular repeated multi-kernel workload. Figure 3 reports throughput in ligands/s on the y-axis and the memory budget assigned to each worker on the x-axis. The V. E XPERIMENTAL R ESULTS curves correspond to different worker counts, and each subplot This section reports four experiments. We first study the reports one platform/backend pair. Within each subplot, the left occupancy-guided approach by varying the Bucket Multiplier, panel reports the occupancy-guided version and the right panel and therefore the batch size, together with the number of active reports the memory-bounded version; the same worker counts workers. We then compare occupancy-guided and memory- are shown in both panels to make the comparison direct. The results show that memory-bounded batching is better bounded batching under different worker counts and per-worker memory budgets. Next, we report the end-to-end throughput suited to this multi-kernel execution model, it provides equal or gain of the Tuned configuration over the Baseline configuration. better peak throughput and a broader useful operating region. Finally, we summarize the tuned results through application- This is the main trend visible in the plots. Across the evaluated platform/backend pairs, memory-bounded batching matches or efficiency and performance-portability metrics. exceeds the peak throughput of occupancy-guided batching and A. Bucket Multiplier and GPU Access Mode avoids dependence on backend-specific occupancy estimates. This experiment studies the occupancy-guided approach. The goal is to evaluate whether this approach still holds in a C. End-to-End Throughput Improvement repeated multi-kernel pipeline. In particular, we study whether This experiment compares the Tuned and Baseline configuexplicit synchronization, implemented through kernel locking, rations on the mixed workload as summarized in Table I. The remains beneficial across different architectures and backends. goal is to measure the end-to-end effect of tuning batching, For each backend-architecture pair, we measured the application worker count, memory allocation, and runtime configuration throughput by varying the memory budget and number of together. In this context, the Baseline is the approach proposed workers. Figure 2 reports throughput in ligands/s as a function in literature [11], which used occupancy-guided batching with of the Bucket Multiplier. The curves correspond to different explicit synchronization. It uses double buffering, a Bucket worker counts, and, for each platform/backend pair, the two Multiplier equal to 3, and two workers on all supported panels compare the two GPU access modes. Locked GPU platform/backend pairs. Tuned denotes the memory-bounded Access means that one worker uses explicit synchronization, configuration with concurrent GPU access, using the worker whereas Concurrent GPU Access means that each worker relies count and per-worker memory budget selected from the

W =1

W =2

6,000

4,000

2,000

1 2 3 4 5 6 7

W =8

Locked GPU Access

Memory Budget [GB]

1 2 3 4 5 6 7

6,000

4,000

2,000

3,000

2,000

1,000

1 2 3 4 5 6 7

Memory Budget [GB]

1 2 3 4 5 6 7

2,000

1,500

1,000

500

1 2 3 4 5 6 7

Memory Budget [GB]

Throughput [ligands/s]

1 2 3 4 5 6 7

(b) GH200/HIP

Throughput [ligands/s]

3,000

1 2 3 4 5 6 7

Memory Budget [GB]

(a) GH200/CUDA

4,000

Locked GPU Access

6,000

Memory Budget [GB]

4,000

W = 16

Concurrent GPU Access Throughput [ligands/s]

Throughput [ligands/s]

Concurrent GPU Access

W =4

1 2 3 4 5 6 7

Memory Budget [GB]

(d) MI250/HIP

2,000

1,500

1,000

500

1 2 3 4 5 6 7

Memory Budget [GB]

1 2 3 4 5 6 7

Memory Budget [GB]

(e) MI250/SYCL

Fig. 2: Application throughput by varying the Bucket Multiplier on the x-axis, leading to larger batches toward right values. The curves correspond to worker counts, and the two panels compare the occupancy-guided method under two different GPU access methods.

worker/memory study for each platform/backend pair. We report the result of the experiment in Figure 4. Baseline denotes the fixed occupancy-guided reference configuration used in the comparison. The results show that the Tuned configuration improve throughput across the tested vendor/framework combinations. This is what Figure 4 shows. The Tuned configuration improves the mixed-workload throughput on all evaluated platform/backend pairs, with a minimum/mean/maximum speedup of 1.38×/1.53×/1.83× over Baseline. The comparison between Baseline and Tuned is intentionally practical rather than fully controlled: the former is a fixed occupancy-guided reference

TABLE I: Tuned runtime configurations selected from the worker/memory experiment and used for the Tuned bars in Figure 4. Platform/Backend GH200/CUDA GH200/HIP GH200/SYCL MI250/HIP MI250/SYCL Max1550/SYCL

Workers 4 8 4 8 4 8

Memory [GB] 8 4 16 4 8 8

point, while the latter uses the best worker and memory settings

W =1

W =2

6,000

4,000

2,000

W =8

Memory-bounded

Memory Budget [GB]

W = 16

Occupancy-guided

Throughput [ligands/s]

Occupancy-guided

W =4

Memory Budget [GB]

6,000

4,000

2,000

1,000

Memory Budget [GB]

1,500

1,000

500

1,000 500

Memory Budget [GB]

(e) MI250/SYCL

Throughput [ligands/s]

1,000

Memory Budget [GB]

(d) MI250/HIP

1,500

Memory Budget [GB]

1,500

Memory Budget [GB]

1,500

500

(b) GH200/HIP

Throughput [ligands/s]

3,000

Memory Budget [GB]

(a) GH200/CUDA

3,000

Memory-bounded

3,000

2,000

1,000

Memory Budget [GB]

(f) Max1550/SYCL

Fig. 3: Application throughput by varying the memory budget assigned to each worker (x-axis), the application throughput in ligands per second, and the number of host workers (curves). Each subplot reports one platform/backend pair; within each subplot, the left panel reports the occupancy-guided version and the right panel reports the memory-bounded version.

found for each platform/backend pair. For this reason, the figure should be read as an end-to-end result for the full methodology rather than as a policy-only comparison. D. Performance Portability This experiment summarizes how the tuned methodology transfers across the supported platforms and backends. Here, the tuned methodology is the same Tuned configuration introduced in the previous subsection. For each supported platform/backend pair, we consider the best configuration selected from the worker/memory study, that is, the worker count and per-worker memory budget used for the Tuned bars in Figure 4. To evaluate portability we use two complementary

views. First, we report application throughput in ligands per second, which is the primary metric for end-to-end screening. Second, we discuss application efficiency, defined as the throughput of one backend on a platform divided by the best backend throughput observed on that same platform. This distinction is important: absolute throughput captures practical usefulness, while efficiency highlights how close each programming model comes to the best available result on the target machine. Figure 5 reports application efficiency, defined as throughput normalized to the best Tuned backend on the same platform. The lower rows report two summaries across platforms: the

Baseline

Throughput (lig/s)

1.49x

Tuned

1.45x

6,000

4,000

1.57x

2,000

1.38x 1.46x

1.83x

HIP

SYCL

0 CUDA

HIP GH200

SYCL

MI250

SYCL Max1550

Fig. 4: Cross-platform mixed-workload throughput. The x-axis reports the tested backends grouped by platform, the y-axis reports throughput in ligands/s, the bars compare the reference configuration and the tuned methodology, and the labels above the tuned bars report speedup over the reference.

CUDA

HIP

SYCL

GH200

0.99

1.00

0.52

MI250

N/A

0.91

1.00

Max1550

N/A

1.00

HM(eff)

0.00

0.76

AM(eff)

0.33

0.64

0.84

Fig. 5: Application efficiency and performance-portability view of the tuned methodology. Each matrix cell reports application efficiency relative to the best backend on the same platform; gray indicates unsupported platform/backend combinations. The lower rows report the harmonic and arithmetic means across the investigated platforms.

harmonic mean and the arithmetic mean of efficiency. We use the harmonic mean as the stricter portability-oriented metric, because it penalizes weak or unsupported combinations more strongly, while the arithmetic mean is reported as a complementary average-efficiency view [1], [2]. The results show that the tuned methodology retains a good platform-local efficiency while exposing the remaining portability limits across backends. This is the trend shown by fig. 5: The matrix reports high local efficiency on the supported backends, while the summary rows show the portability cost of unsupported combinations and backend-dependent performance differences.

VI. R ESULTS D ISCUSSION In this section, we interpret the implications of the results, discuss the portability trade-offs revealed by the campaign, and explain the limitations of occupancy-derived heuristics as a basis for portable tuning. a) Portability tradeoffs revealed by the campaign: The cross-platform results also make clear that portability has more than one dimension. The results measure practical end-to-end throughput, while the application-efficiency view normalizes each backend against the best result on the same platform. Read together, these results show a recurring tradeoff: some backends achieve very strong local efficiency where they are available, but broader platform coverage still matters when the goal is portability across vendors. This is why the paper reports both raw throughput and efficiency-based summaries rather than collapsing the discussion into a single number. b) Limitations of occupancy-derived heuristics: One consistent issue encountered in the data is that occupancyderived quantities do not transfer cleanly across backends. The occupancy-guided policy is reasonably well behaved on some platform/backend pairs, but its underlying heuristic is much less stable on others. On Max1550 is the clearest example in this study, where the inferred occupancy multiple can become inflated relative to the values observed on GH200 and MI250. For this reason, the paper should treat occupancyguided batching as a useful implementation hint rather than as a portable first-principles rule. c) Additional observations: Beyond the main methodological result, the experiments also expose a secondary layer of runtime decisions that affect the throughput. The bucketmultiplier results show that synchronization choices matter when multiple workers are competing for the same device. Removing kernel locking has little visible effect with one

worker, but becomes beneficial as worker concurrency increases. HIP, and SYCL implementations [1]–[3]. The most direct This does not change the main conclusion of the paper, but it methodological precedent is prior out-of-kernel docking work, does show that finer-grained execution choices can widen or which showed that docking throughput can depend strongly on narrow the high-throughput operating region once the broader bucket sizing and other host-side decisions outside the main resource-management strategy is fixed. kernels [11]. Follow-on portability studies of that pipeline d) Main takeaway from the results: The clearest conclu- have shown that these host-side decisions are entangled sion of this study is methodological rather than backend-specific. with backend-dependent effects such as occupancy estimation, Across the investigated vendor/framework combinations, the compiler behavior, and register pressure [3], [25]. This literature memory-bounded formulation consistently matches or exceeds is important for our paper because it shows that performance the peak throughput of the occupancy-guided baseline. These depends not only on kernel code, but also on how work experimental results suggest that other irregular scientific ap- is formed and executed across the full pipeline. Our work plications can benefit from this type of partitioning to increase builds on that point, but treats these choices as part of a more occupancy. The same data also shows that portable throughput general optimization methodology for repeated multi-kernel in repeated multi-kernel applications depends on out-of-kernel GPU pipelines rather than as a docking-specific optimization. This broader framing also connects to the mini-app and decisions such as work-unit formation, worker concurrency, and resource partitioning, not only on the performance of proxy-app tradition in HPC, where compact applications are individual kernels viewed in isolation [11], [18]. On average, used to expose performance-critical behavior without the this methodology delivers a 1.5× speedup across all tested full complexity of a production workflow [26]–[28]. From that perspective, muDock plays the role of a representative platforms. repeated multi-kernel scientific application: it preserves the VII. R ELATED W ORK irregularity and orchestration structure that matter to optimizaMolecular docking and virtual screening have long been tion, while remaining small enough to study systematically driven by the tension between search accuracy and throughput. across platforms and backends. Previous work has already Classical docking systems such as AutoDock4 and AutoDock used muDock to study vectorization-oriented performance Vina established the structure still used in many workflows: portability on CPUs [12]; the present paper extends that line repeated evaluation of large ligand sets under irregular search of work toward GPU-oriented out-of-kernel optimization and dynamics [4], [5], [19], [20]. Recent work has increasingly used backend portability. HPC to speed up virtual screening campaigns and reduce their Finally, our study is embedded in the wider performancetime to solution, while also targeting larger molecular libraries portability literature. Pennycook and coauthors formalized and more heterogeneous systems [7]–[9], [21], [22]. This is performance portability as a joint requirement of efficiency the application context of our work. Our focus, however, is and portability, and follow-on work has emphasized that evalnot docking accuracy or scoring-model design, but the systems uating portability requires more than demonstrating functional problem of sustaining portable throughput across heterogeneous correctness across programming models [1], [2]. Subsequent GPU backends. cross-platform studies have reinforced this point for both full Recent GPU docking work has made that systems dimension applications and benchmarking suites, including recent work increasingly explicit. AutoDock-GPU and related studies show on programming-model comparisons and on SYCL behavior that docking is not only compute intensive but also structurally across NVIDIA, AMD, and Intel platforms [29]–[31]. Our irregular, with performance shaped by ligand-dependent search contribution is deliberately narrower and more applicationbehavior, batching choices, and host-device interaction [23], driven: we isolate one consequential out-of-kernel policy [24]. Other modern GPU docking engines, such as Uni- question in a repeated multi-kernel scientific workflow and Dock, push throughput further through aggressive batching show how a portable methodology can improve throughput and GPU-oriented dataflow design [18], while comparative while also revealing the remaining limits imposed by backendstudies in earlier GPU docking work show that different specific behavior. mapping strategies can shift the balance between latency and VIII. C ONCLUSION throughput [10]. Our paper is closest to this systems-oriented This paper presented a portable out-of-kernel optimization branch of docking research, but differs in focus: rather than proposing a new fastest docking engine, we study how batching methodology for scientific applications organized as repeated and worker-level resource decisions transfer across CUDA, HIP, multi-kernel GPU pipelines, and studied it through the muDock molecular docking mini-app. More generally, the paper shows and SYCL implementations. This work builds on the out-of-kernel perspective introduced that for similar irregular throughput-oriented workloads, appliin prior docking studies [10], [11]. The main idea is that cation throughput can be improved by tuning three coupled performance portability is shaped by policy choices outside the runtime dimensions: the memory allocated to each worker, the kernels themselves, such as bucketing, batching, and stream- batching policy used to form the next execution chunk, and level concurrency. These choices are especially delicate on the number of workers that concurrently feed the device. heterogeneous GPU platforms, where occupancy information, Across the investigated platforms and backends, this methodregister usage, and runtime behavior differ across CUDA, ology improves the mixed-workload throughput by 1.53× on

average over the monolithic reference configuration. At the policy level, the memory-bounded batching method matches or exceeds the peak throughput of the occupancy-guided method on every evaluated platform/backend pair. These results indicate that tuning memory allocation, batching, and worker concurrency together provides a practical way to improve throughput while preserving portability across CUDA, HIP, and SYCL implementations. The broader lesson is that repeated multi-kernel pipelines should be tuned at application level, not only at kernel level. For example, for the use case under consideration, the proposed methodology delivers a significant speedup, meaning that large-scale virtual screening setups can analyze a larger chemical space in the same amount of time budget [8]. In this setting, explicit resource partitioning and concurrent work submission provide a stronger basis for portable performance than occupancy-derived alignment alone, even though backenddependent behavior still limits full portability. ACKNOWLEDGMENT The authors take full responsibility and ownership for the paper content, while ChatGPT and DeepL have been used for language editing. DATA AVAILABILITY The source code, benchmarking scripts, and datasets used in this study will be made publicly available upon acceptance via GitHub and Zenodo. R EFERENCES [1] S. J. Pennycook, J. D. Sewall, and V. W. Lee, “Implications of a metric for performance portability,” Future Generation Computer Systems, 2019. [Online]. Available: https://doi.org/10.1016/j.future.2017.08.007 [2] S. J. Pennycook and J. D. Sewall, “Revisiting a metric for performance portability,” in 2021 International Workshop on Performance, Portability and Productivity in HPC (P3HPC). IEEE, 2021. [Online]. Available: https://doi.org/10.1109/P3HPC54578.2021.00004 [3] L. Crisci, M. Salimi Beni, B. Cosenza, N. Scipione, D. Gadioli, E. Vitali, G. Palermo, and A. Beccari, “Enabling performance portability on the ligen drug discovery pipeline,” Future Generation Computer Systems, 2024. [Online]. Available: https://doi.org/10.1016/j.future.2024.03.045 [4] B. K. Shoichet, “Virtual screening of chemical libraries,” Nature, 2004. [Online]. Available: https://doi.org/10.1038/nature03197 [5] D. B. Kitchen, H. Decornez, J. R. Furr, and J. Bajorath, “Docking and scoring in virtual screening for drug discovery: Methods and applications,” Nature Reviews Drug Discovery, 2004. [Online]. Available: https://doi.org/10.1038/nrd1549 [6] E. Lionta, G. Spyrou, D. K. Vassilatis, and Z. Cournia, “Structure-based virtual screening for drug discovery: Principles, applications and recent advances,” Current Topics in Medicinal Chemistry, 2014. [Online]. Available: https://doi.org/10.2174/1568026614666140929124445 [7] D. Gadioli, E. Vitali, F. Ficarelli, C. Latini, C. Manelfi, C. Talarico, C. Silvano, C. Cavazzoni, G. Palermo, and A. R. Beccari, “Exscalate: An extreme-scale virtual screening platform for drug discovery targeting polypharmacology to fight sars-cov-2,” IEEE Transactions on Emerging Topics in Computing, 2023. [Online]. Available: https://doi.org/10.1109/TETC.2022.3187134 [8] J. Lyu, S. Wang, T. E. Balius, I. Singh, A. Levit, Y. S. Moroz et al., “Ultra-large library docking for discovering new chemotypes,” Nature, 2019. [Online]. Available: https://doi.org/10.1038/s41586-019-0917-9 [9] F. Liu, O. Mailhot, I. S. Glenn, S. F. Vigneron, V. Bassim, X. Xu, K. Fonseca-Valencia, M. S. Smith, D. S. Radchenko, J. S. Fraser, Y. S. Moroz, J. J. Irwin, and B. K. Shoichet, “The impact of library size and scale of testing on virtual screening,” Nature Chemical Biology, 2025. [Online]. Available: https://doi.org/10.1038/s41589-024-01797-w

[10] E. Vitali, F. Ficarelli, M. Bisson, D. Gadioli, G. Accordi, M. Fatica, A. R. Beccari, and G. Palermo, “Gpu-optimized approaches to molecular docking-based virtual screening in drug discovery: a comparative analysis,” Journal of Parallel and Distributed Computing, 2024. [Online]. Available: https://doi.org/10.1016/j.jpdc.2023.104819 [11] G. Accordi, D. Gadioli, E. Vitali, L. Crisci, B. Cosenza, A. R. Beccari, and G. Palermo, “Out of kernel tuning and optimizations for portable large-scale docking experiments on gpus,” The Journal of Supercomputing, 2024. [Online]. Available: https://doi.org/10.1007/s11227-023-05884-y [12] G. Accordi, J. Domke, T. Pollinger, D. Gadioli, and G. Palermo, “Towards high-performance and portable molecular docking on cpus through vectorization,” in Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 2025. [Online]. Available: https://doi.org/10.1109/CLUSTER59342.2025.11186493 [13] elvispolimi, “mudock,” https://github.com/elvispolimi/muDock, 2026, gitHub repository, accessed 2026-04-01. [14] P. G. Polishchuk, T. I. Madzhidov, and A. Varnek, “Estimation of the size of drug-like chemical space based on gdb-17 data,” Journal of Computer-Aided Molecular Design, 2013. [Online]. Available: http://dx.doi.org/10.1007/s10822-013-9672-4 [15] R. Huey, G. M. Morris, A. J. Olson, and D. S. Goodsell, “A semiempirical free energy force field with charge-based desolvation,” Journal of Computational Chemistry, 2007. [Online]. Available: https://doi.org/10.1002/jcc.20634 [16] G. Daiß, M. Simberg, A. Reverdell, J. Biddiscombe, T. Pollinger, H. Kaiser, and D. Pflüger, “Beyond Fork-Join: Integration of Performance Portable Kokkos Kernels with HPX,” in 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2021. [Online]. Available: https://doi.org/10.1109/IPDPSW52791.2021.00066 [17] G. Vistoli, C. Manelfi, C. Talarico, A. Fava, A. Warshel, I. V. Tetko, R. Apostolov, Y. Ye, C. Latini, F. Ficarelli, G. Palermo, D. Gadioli, E. Vitali, G. Varriale, V. Pisapia, M. Scaturro, S. Coletti, D. Gregori, D. Gruffat, E. Leija, S. Hessenauer, A. Delbianco, M. Allegretti, and A. R. Beccari, “Mediate - molecular docking at home: Turning collaborative simulations into therapeutic solutions,” Expert Opinion on Drug Discovery, 2023. [Online]. Available: https://doi.org/10.1080/17460441.2023.2221025 [18] Y. Yu, C. Cai, J. Wang, Z. Bo, Z. Zhu, and H. Zheng, “Uni-dock: Gpu-accelerated docking enables ultralarge virtual screening,” Journal of Chemical Theory and Computation, 2023. [Online]. Available: https://doi.org/10.1021/acs.jctc.2c01145 [19] G. M. Morris, R. Huey, W. Lindstrom, M. F. Sanner, R. K. Belew, D. S. Goodsell, and A. J. Olson, “Autodock4 and autodocktools4: Automated docking with selective receptor flexibility,” Journal of Computational Chemistry, 2009. [Online]. Available: https://doi.org/10.1002/jcc.21256 [20] O. Trott and A. J. Olson, “Autodock vina: Improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading,” Journal of Computational Chemistry, 2010. [Online]. Available: https://doi.org/10.1002/jcc.21334 [21] S. LeGrand, A. Scheinberg, A. F. Tillack, M. Thavappiragasam, J. V. Vermaas, R. Agarwal, J. Larkin, D. Poole, D. Santos-Martins, L. Solis-Vasquez, A. Koch, S. Forli, O. Hernandez, J. C. Smith, and A. Sedova, “Gpu-accelerated drug discovery with docking on the summit supercomputer: Porting, optimization, and application to covid-19 research,” in Proceedings of the 11th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB 2020). ACM, 2020. [Online]. Available: https://doi.org/10.1145/3388440.3412472 [22] D. Gadioli, G. Accordi, J. Krenek, M. Golasowski, L. Foltyn, J. Martinovic, A. R. Beccari, and G. Palermo, “A portable drug discovery platform for urgent computing,” Procedia Computer Science, 2024. [Online]. Available: https://doi.org/10.1016/j.procs.2024.07.007 [23] D. Santos-Martins, L. Solis-Vasquez, A. F. Tillack, M. F. Sanner, A. Koch, and S. Forli, “Accelerating autodock4 with gpus and gradientbased local search,” Journal of Chemical Theory and Computation, 2021. [Online]. Available: https://doi.org/10.1021/acs.jctc.0c01006 [24] L. Solis-Vasquez, A. F. Tillack, D. Santos-Martins, A. Koch, S. LeGrand, and S. Forli, “Benchmarking the performance of irregular computations in autodock-gpu molecular docking,” Parallel Computing, 2022. [Online]. Available: https://doi.org/10.1016/j.parco.2021.102861 [25] G. Accordi, D. Gadioli, G. Palermo, L. Crisci, L. Carpentieri, B. Cosenza, and A. R. Beccari, “Unlocking performance portability on lumi-g supercomputer: A virtual screening case study,” in International Workshop on OpenCL and SYCL (IWOCL ’24). Chicago, IL, USA: ACM, 2024. [Online]. Available: https://doi.org/10.1145/3648115.3648125

[26] P. S. Crozier, H. K. Thornquist, R. W. Numrich, A. B. Williams, H. C. Edwards, E. R. Keiter, M. Rajan, J. M. Willenbring, D. W. Doerfler, and M. A. Heroux, “Improving performance via mini-applications,” Sandia National Laboratories, Tech. Rep., 2009. [Online]. Available: https://doi.org/10.2172/993908 [27] M. Thavappiragasam, A. Scheinberg, W. Elwasif, O. Hernandez, and A. Sedova, “Performance portability of molecular docking miniapp on leadership computing platforms,” in 2020 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC). IEEE, 2020. [Online]. Available: https://doi.org/10.1109/P3HPC51967. 2020.00009 [28] A. Poenaru, W.-C. Lin, and S. McIntosh-Smith, A Performance Analysis of Modern Parallel Programming Models Using a Compute-Bound Application. Springer International Publishing, 2021. [Online]. Available: http://dx.doi.org/10.1007/978-3-030-78713-4 18 [29] T. J. Deakin, A. Poenaru, T. Lin, and S. N. McIntosh-Smith, “Tracking performance portability on the yellow brick road to exascale,” in 2020 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC). IEEE, 2021. [Online]. Available: https://doi.org/10.1109/P3HPC51967.2020.00006 [30] I. Z. Reguly, “Comparative evaluation of bandwidth-bound applications on the intel xeon cpu max series,” ser. SC-W ’23. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3624062.3624195 [31] Z. Jin and J. S. Vetter, “A benchmark suite for improving performance portability of the sycl programming model,” in 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 2023. [Online]. Available: https://doi.org/10.1109/ISPASS57527.2023.00041

Performance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study

Analyze & explore related research in Conceptio

Related documents