A comprehensive evaluation of spatial co-execution on GPUs using MPS and MIG technologies Jorge Villarrubia1*, Luis Costero1 , Francisco D. Igual1 , Katzalin Olcoz1
arXiv:2604.22430v1 [cs.DC] 24 Apr 2026
1
Departamento de Arquitectura de Computadores y Automática Universidad Complutense de Madrid (Spain).
*Corresponding author(s). E-mail(s): [email protected]; Contributing authors: [email protected]; [email protected]; [email protected]; Abstract To mitigate the increasingly common underutilization of computational resources in modern GPUs, spatial sharing methods enable multiple applications to use them simultaneously. This work presents a comprehensive evaluation of NVIDIA’s primary technologies to achieve that goal: Multi-Process Service (MPS) and Multi-Instance GPU (MIG). Our findings reveal a crucial trade-off between MPS’s flexibility and MIG’s isolation, and provide many key insights for improving the co-execution strategy according to job profiles. In the most favorable scenarios, MPS improves performance by up to 30% and reduces energy by about 20%, using its provisioning option to avoid resource monopolization. However, under memory contention, it suffers severe degradation, worsening performance by around 30%. Conversely, MIG’s full hardware isolation resolves memory contention, leading to more consistent improvements, but these gains are tempered by higher overhead, and its rigid scheme can degrade performance in certain cases. Keywords: Resource management; GPU co-execution; GPU spatial sharing; Multi-Process Service (MPS); Multi-Instance GPU (MIG).
1 Introduction Graphics Processing Units (GPUs) have become the primary devices for accelerating massively parallel computations such as those required for Artificial Intelligence (AI) or a plethora of scientific applications [1, 2]. With the rise of increasingly demanding tasks, notably the training of Large Language Models (LLMs), GPU manufacturers 1
20k
90 80 70 60 50 40 30 20 10 0
CUDA cores Memory capacity (GiB) Peak performance FP32 (TFLOPS)
Performance FP32 (TFLOPS)
25k CUDA Cores (FP32)
Memory (GiB)
200 175 150 125 100 75 50 25 0
15k 10k 5k 0
2012 2013 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 Tesla Tesla Tesla Tesla A100 H100 H200 B200 K20X K40 P100 V100 Fig. 1: Evolution of FP32 peak performance and amount of resources of the most powerful GPU each year. Data extracted from NVIDIA’s official datasheets. have significantly improved the performance of these devices, particularly by increasing the amount of computational resources available [3]. For instance, Figure 1 depicts the evolution of NVIDIA’s flagship GPUs since 2012 and highlights the rise in FP32 cores, DRAM capacity and peak single-precision performance. It is striking that these metrics reveal a particularly pronounced growth from 2019 onwards, coincident with the appearance of the first large-scale LLMs [4]. For example, until 2019 the NVIDIA V100 was the flagship device, offering 9.3 TFLOPS (in single-precision), 3,584 FP32 cores grouped across 56 SMs (Streaming Multiprocessors), and 32 GiB of memory with 900 GiB/s bandwidth. By contrast, the current NVIDIA B200 delivers roughly 80 TFLOPS thanks to 20,480 FP32 cores organized into 160 SMs, along with 192 GiB of memory and 8 TiB/s of bandwidth [5]. While this scaling enables effective support for highly demanding applications, many workloads that have traditionally been accelerated on GPUs cannot take full advantage of the vast amount of resources offered by today’s datacenter GPUs. Several empirical studies have documented these inefficiencies across a range of High-Performance Computing (HPC) workloads [6–8], and particularly for Deep Learning (DL) model inference [9, 10]. This underutilization not only prevents the device from reaching its maximum potential performance but also leads to substantial energy waste [11], with significant economic and environmental consequences [12]. A natural solution to mitigate resource underutilization is the concurrent execution of multiple applications on a single GPU, allowing them to share its computational resources simultaneously. This co-execution approach, known as spatial sharing, improves upon the default GPU mechanisms for concurrent applications, which solely rely on temporal sharing of the device (i.e., time-slicing, where a scheduler switches between applications). While time-slicing can hide some memory latencies, it fails to address the aforementioned underutilization issues because multiple applications never use the resources simultaneously.
2
For spatial resource sharing, NVIDIA, the prominent leader in the GPU market, offers the Multi-Process Service (MPS) [13] and Multi-Instance GPU (MIG) [14] technologies, whose main difference lies in their approach towards isolation. Whereas MPS shares resources among applications—with an option to limit the maximum percentage of SMs, but not to restrict memory system usage—MIG is based on the physical partitioning of the device into smaller, independent instances, each with its own fully isolated compute and memory resources. Consequently, MPS provides flexible sharing but may suffer from memory-system contention that can violate QoS (Quality of Service) or SLA (Service Level Agreement), while MIG provides a stronger isolation at the cost of coarser execution patterns and more overhead. However, these trade-offs have been insufficiently evaluated in the prior literature [15–17], which lacks comprehensive comparisons between both technologies, and especially, in-depth analyses and characterization with diverse workload combinations. This work contributes to this area by performing a multi-factorial evaluation of different co-execution schemes on NVIDIA GPUs. Our main contributions are as follows:
• We evaluate the performance of MPS and MIG under different configurations and compare them to baselines such as time-slicing. To this end we run many benchmark combinations (mainly, more computation-oriented or more demanding on memory) that expose the primary strengths and weaknesses of each approach. • We analyze the energy consumption of the co-execution schemes considering the power-performance trade-off, and evaluating their scalability with the number of concurrent applications. Specifically, we observe that spatial sharing methods can greatly reduce energy consumption compared to the baselines, and that MPS slightly outperforms MIG in overall efficiency (although in some cases it exhibits significant degradation due to memory contention that MIG does not). • We accompany the co-execution experiments with auxiliary metrics on resource utilization, to extract practical lessons. Our main findings include: substantial performance improvements (especially with MIG) when using the appropriate scheme with optimal configuration, severe memory contention issues in MPS, or the benefits of SM provisioning to prevent resource monopolization. The rest of this paper is organized as follows. Section 2 provides background on GPU concurrent execution alternatives and supporting technologies. Section 3 details the experimental setup employed in the evaluation. Section 4 provides a comprehensive performance comparison of the evaluated technologies. Section 5 evaluates energy efficiency and consumption, as well as scalability with the degree of concurrency. Section 6 discusses related work. Finally, Section 7 summarizes the main conclusions.
2 Background This section outlines the fundamentals of time-slicing, MPS and MIG, the primary GPU co-execution technologies evaluated in this work. A brief overview of GPU architecture precedes this description to provide the necessary context.
3
GPC1 SM1
SM2
SM3
SM4
32 cores
32 cores
32 cores
32 cores
Warp plan L1 cache
SM5
Warp plan L1 cache
Warp plan L1 cache
L2 cache and DRAM shared GPC2 SM6 SM7
Warp plan L1 cache
1
Warp 1 2 32
Warp 2 33 64
Warp 8 225 256
Ins.
Ins.
Ins.
Ins.
Ins.
Ins.
Ins.
...
...
...
...
Ins. Ins.
Thread block of SM4 Warp plan SM4
SM8
32 cores
32 cores
32 cores
32 cores
Warp plan L1 cache
Warp plan L1 cache
Warp plan L1 cache
Warp plan L1 cache
Memory access
32 Cores SM4
Ins. warp 2 (Hidden latency)
Ins.
warp 1
Ins.
warp 7
Time
Ins.
warp 2
Fig. 2: GPU architecture overview (left) and latency hiding by warp switching (right).
2.1 GPU architecture overview A GPU application is composed of distinct pieces of code called kernels, which are launched from the host with a specific configuration of threads and blocks that determines how the work is mapped onto the hardware. Kernels implement a parallel programming model known as Single Instruction, Multiple Threads (SIMT), in which thousands of threads execute the same instructions concurrently but (potentially) on different data [18]. As illustrated in Figure 2, to enable this massive parallelism, GPUs contain thousands of arithmetic logic units, known as cores, which are grouped into physical blocks called Streaming Multiprocessors (SMs), each equipped with its own L1 cache. In turn, several SMs combine to form a Graphics Processing Cluster (GPC), which, in the context of MIG technology, acts as a partitioning unit (see Section 2.4). Each SM receives an independent thread block and divides it into groups known as warps, performing efficient context switches between these warps to hide memory access latencies. The scheduler not only switches between warps, but several of them are executed simultaneously by mapping their threads to the SM’s cores (NVIDIA GPUs use 32 threads per warp, and the number of cores per SM is typically a multiple of this). For each of these warps, all threads execute the same instruction in parallel on different cores (SIMT model). Thus, parallelism is exploited at three levels: different warps execute concurrently on different SMs; within each SM, the instructions for all threads of many warps are executed simultaneously across cores; and within each SM, context is switched between warps to hide memory access latency.
2.2 Time-slicing The most basic method of concurrent execution on a GPU is device time-sharing among multiple applications, which is, in fact, the default behavior when multiple processes are submitted. Technically, the NVIDIA GPU driver time-multiplexes the 4
OS Process A
OS Process B
Thread ... Thread
Thread ... Thread
Time
A CUDA context A
CUDA context B B
Concurrent scheduler
Concurrent scheduler
...
Time-slicing scheduler SM
SM
SM GPU
OS Process A
OS Process B
Thread ... Thread
Thread ... Thread
Client context A
Client context B
Kernel A1 Kernel A2
A
Time
Context mapping Kernel A3
Kernel Kernel A1 B1
MPS context
Kernel B1
MPS server process
... B
Concurrent scheduler
SM
Concurrent scheduler
...
Kernel A2 Kernel A3
Time sharing
SM
SM
SM GPU
(a) Time-slicing co-execution scheme.
...
SMs
SM
Spatial sharing
(b) MPS co-execution scheme.
Fig. 3: Schematic representation of CUDA context management from multiple processes, together with a timing diagram illustrating kernel concurrency.
device among the different GPU processes, known as CUDA contexts, such that each one receives a time slice to execute before the context is switched to another process (see Figure 3a). Historically, these switches were performed at the kernel level, but modern schedulers can now preempt and switch contexts with relatively low overhead (this involves saving the context and incurs a residual cost, such as some pollution of memory structures). The primary benefit of this time-slicing is fairness among processes, which can particularly reduce the average completion time in multi-application environments. However, it can also yield utilization benefits by eliminating idle periods during operations such as host-to-device (or vice versa) transfers, CPU computations between kernels, or I/O waits. As we will show, our experiments indicate that the aggregate performance of the applications improves slightly with this technique, suggesting that its benefits marginally outweigh the overhead from context switches. While it is possible to modify the scheduling policy and the duration of time slices using tools like Kubernetes with the NVIDIA GPU Operator (e.g., equal-share, fixed-share, long/medium/short slices), we will use the default policy simply as a baseline against which to compare the more advanced methods described below. These methods will be more ambitious than time-slicing, achieving superior improvements in performance and energy consumption, but they require explicit configuration actions that can sometimes be complex (involving scheduling decisions) and incur higher overhead.
2.3 Multi-Process Service (MPS) NVIDIA’s Multi-Process Service (MPS) allows multiple processes to simultaneously utilize the computational resources of a single GPU. Under normal circumstances,
5
each process that uses a GPU creates its own CUDA context, which encapsulates its entire state (e.g., memory allocations, loaded modules, or address space). However, the GPU driver imposes a fundamental limitation: only one context can be active on the GPU at any given time. As a result, when multiple processes access the same GPU, the hardware defaults to the time-slicing mechanism explained in the previous section, which does not deal with the resources underutilization issues. MPS overcomes this limitation by implementing a client-server architecture, as depicted in Figure 3b. An MPS daemon runs as a server, managing a single CUDA context to which multiple client applications connect instead of creating their own contexts. The server funnels the operations from all clients through this unified context, enabling it to dispatch kernels from different processes to the GPU’s hardware work queues simultaneously. Underlying technologies, such as Hyper-Q, which equips the GPU with multiple physical queues and a dynamic scheduler, allow these kernels to execute in parallel across the different functional units (spatial resource sharing). The communication between the MPS client and server is fully encapsulated within the CUDA driver, making MPS transparent to the programmer. The user simply needs to enable the MPS daemon and launch the applications that will share the GPU. However, with this technology, processes compete for shared resources, which can lead to interference. For instance, a process may be delayed because it requires a resource that is being used by another, in addition to other residual conflicts like cache pollution. This can negatively impact performance and may even allow one application to monopolize the device, thereby hindering the ability to provide QoS or SLA. To mitigate this issue, the Volta generation (released in 2017) introduced several enhancements in MPS, most notably a resource provisioning feature. This allows users to set an upper limit on the percentage of SMs that each process can use. This limit is configured via the CUDA MPS ACTIVE THREAD PERCENTAGE environment variable. By using this provisioning, SM sharing can be avoided if the sum of the limits for all concurrent applications does not exceed 100%. This partitioning ensures that warps from different applications do not run on the same SM, eliminating contention for L1 cache, among other benefits (see Figure 2). Nevertheless, the rest of the memory hierarchy (L2 cache, DRAM, and communication buses/crossbars) remains shared and susceptible to conflicts. The official documentation recommends partitioning 100% of the SMs among the concurrent applications according to their respective demands. In this work, we will evaluate the utility and impact of this provisioning strategy following that advice.
2.4 Multi-Instance GPU (MIG) NVIDIA introduced Multi-Instance GPU (MIG) in 2020 to support QoS in the coexecution of GPU applications through isolation. This technology allows a physical GPU to be partitioned into several virtual sub-GPUs, each with its own dedicated and fully isolated hardware resources. Unlike MPS, which only allows for the provisioning of SMs, each sub-GPU has segregated paths throughout the entire memory system, including independent portions of L2 cache and DRAM, as well as guaranteed bandwidth via dedicated buses and crossbar networks. This physical isolation enables
6
A100, H100, B100, B200 partitions
MIG slice
SM
SM
2
4
3
4 4
4
...
5
3
6 7 8
3
9 L2+DRAM
A30 partitions 4
1 2
2 3
2 1
2
4
1
1
5
1
1
1 2
1
1
15 16 17
1
2 2
2 1
1
1
2
1 1
2
Create 2 Create 2 Destroy 4
3
4
1 1
2 1 3 2 1
1
1
Merger of two 2s into one 4
Time 1 1
4 Create 4 Destroy 2
3 1
2 1
1
2 1
1
1 1
1
18
1
1
19
1
1
1
1
1
1
1 1 1 1
1 1
1
2
3
2
2 1 1
1
3 2 2
2
1 1
1
2
2 2
13 14
3 2
3
10 11 12
Partition of 4 into two 2s
Time
7
1 GPC
1
3 1
2 1
1
Fig. 4: MIG valid partitions.
Destroy 2 2
3
2
1
Fig. 5: MIG reconfig. examples.
applications to run concurrently on the virtual GPUs without interference. MIG technology is available on datacenter GPUs with the latest architectures: Ampere, Hopper, and Blackwell. Specifically, it is supported by the NVIDIA A30, A100, H100, H200, and B200 models, and recently extended to enterprise and professional GPUs such as RTX 5000/6000 PRO. MIG division is performed at the GPC level (introduced in Section 2.1), which constitutes the smallest unit that can be isolated. In MIG terminology, these units are called slices, a term we will use henceforth. Numerically consecutive slices (where the numbering relates to their physical position on the GPU) can be grouped into what are known as GPU instances. These instances are independent and fully isolated virtual devices. The size of an instance is determined by the number of slices it comprises, while the set of instances into which the GPU is divided is known as a partition. When partitioning the GPU, not all combinations of slices are valid for creating instances, nor are all instance configurations possible. Figure 4 illustrates the possible configurations for MIG-capable GPUs, which currently follow two partitioning schemes: what we will refer to as the NVIDIA A30 scheme (as it is followed by that GPU, as well as the RTX 5000 PRO and RTX 6000 PRO); and the A100 scheme (used by the NVIDIA A100 and the remaining models). As shown in Figure 4, GPUs following the A30 scheme consist of 4 slices that can be grouped into instances of size 1, 2, and 4, forming 5 possible configurations. Creating instances of size 3 is not permitted, nor is forming an instance with the two central slices ({S2 , S3 }). In this scheme, all slices are identical, featuring the same amount of SMs and memory. For example, each slice on an NVIDIA A30 has 8 SMs and 6 GiB of memory (one-fourth of the total GPU resources). In the case of GPUs with the A100 scheme, there are more partitioning possibilities, corresponding to their greater computational power. As illustrated in Figure 4, these GPUs have 7 slices that can be grouped into instances of size 1, 2, 3, 4, and 7, resulting in 19 possible configurations. As before, instances of
7
size 5 or 6 cannot be formed, nor can slices be arbitrarily grouped to form instances of a valid size. For example, a size-4 instance can only be formed with the first 4 slices: {S1 , . . . , S4 }. Additionally, there is a peculiarity where certain configurations with two size-3 instances are possible, which keeps the S4 slice unavailable. Unlike the GPU partitioning technologies from AMD [19] and Intel [20], MIG allows for the dynamic reconfiguration of any instance that is idle, i.e., not running a task. Reconfiguration involves destroying certain instances and creating new ones in their place. This process only affects the slices of the modified instances. Therefore, the transformation is transparent and non-disruptive to unmodified instances, which can continue to execute other tasks concurrently. For example, as illustrated in Figure 5 (top), the size-4 instance of a 4-3 partition could be destroyed and replaced by two new size-2 instances, transitioning to 2-2-3 without affecting the application running on the size-3 instance. Similarly (bottom of Figure 5), the first instances of a 2-2-3 partition could be merged to transition to 4-3.
3 Experimental setup 3.1 Experimental environment To evaluate the presented co-execution technologies, we primarily used an NVIDIA A30 GPU with 24 GiB of memory, although we also had access to a 40 GiB NVIDIA A100 PCIe to verify the results of some experiments. The host system is equipped with an Intel Xeon Silver 4314 CPU @2.40 GHz and Debian GNU/Linux 12, using NVIDIA driver version 580.82.07 and CUDA 13.0. To ensure fair and reproducible results, all experiments were conducted with persistence mode enabled, locking the memory and SM clock frequencies to 1215 MHz and 1440 MHz, respectively (the maximum supported by the A30). We also verified that no throttling events (clock reduction due to temperature or power limits) occurred by monitoring reports from the Data Center GPU Manager (DCGM) tool [21].
3.2 Co-execution schemes The main evaluation consists of a comparison between four co-execution schemes based on the three technologies described in Section 2: time-slicing, MPS, and MIG. Since MPS offers a provisioning option, we evaluate this technology both with and without this feature, particularly in the initial experiments. The four schemes evaluated in this study are as follows:
• Time-slicing: The default temporal switching performed by the scheduler when it receives multiple applications simultaneously (see Section 2.2). • MPS: Spatial sharing with the basic version of MPS, not using the provisioning option (see Section 2.3). • MPS-best-prov: Spatial sharing with MPS, but provisioning the maximum percentage of resources that each application can use. This provides a degree of isolation when the total allocation is under 100% (see Section 2.3). We select the optimal provisioning (-best-prov ) by sweeping through resource distributions
8
in 10% increments, choosing the split that minimizes the makespan (i.e., the completion time of the last application to finish).
• MIG: Spatial sharing with full resource isolation via MIG hardware partitioning (see Section 2.4). The instances sizes used are detailed in each experiment; however, on the A30, the options are limited, and we will primarily use the 2-2 in Section 4, and also 1-1-1-1 in Section 5.
3.3 Evaluation metrics This study focuses on both performance and energy consumption. Performance is evaluated using metrics derived from application execution times under each scheme. Specifically, we use the following:
• Individual slowdown: The ratio of an application’s co-execution time compared to its standalone execution time. In Section 4, we focus on the individual slowdown of each application to ensure that one application’s performance does not improve significantly at the expense of another, which would indicate an unfair or imbalanced resource distribution. • Overall slowdown: In Section 5, we use the overall slowdown, defined as the total co-execution runtime (i.e., makespan) relative to the sum of the all standalone execution times. This work will analyze energy consumption by examining the fundamental tradeoff between performance and power draw, dissecting the independent contribution of each factor. While higher performance reduces execution time, it often requires a power increase. In this context, we measured the total Joules consumed during the execution with the nvmlDeviceGetTotalEnergyConsumption function from the NVML library [22]. This value is divided by the total execution time to calculate the power (in watts), which will be important in the analysis, and is also presented in relation to the values measured for the applications running alone. To deepen our analysis, we complement these primary metrics with resource utilization data. We employ Nsight Compute (NCU) to obtain fine-grained throughput metrics—specifically, peak bandwidth utilization for memory and peak SM performance per kernel—in select scenarios (Figure 8 and Table 4). However, NCU is generally incompatible with MPS due to the unified context, which prevents the accurate attribution of hardware counters to individual processes. Therefore, following prior studies on MPS [9, 23–25], we primarily rely on the DCGM tool [21] to track memory and Compute (SM) utilization. DCGM performs periodic sampling (every 200 ms in our case), dividing each period into 100 equal-time slots for which it checks resource activity to obtain a utilization percentage. Finally, these percentages are averaged over time. In addition, to quantify fairness by assessing the slowdown disparity between co-executed applications (Section 4.2), we employ the Gini coefficient as a statistical measure of inequality [26, 27].
9
Table 1: Specification of the applications used for evaluation. The first six belong to the Altis suite [28]. Test name
Description
Problem size
Passes
Sort
Update for modern GPUs of the test from SHOC suite [29] that uses the radix sort algorithm on integer key-value pairs.
Level 2
3
GUPS
Giga UPdates per Second measures the GPU’s performance in updating randomly generated memory locations.
Level 3
58
KMeans
Test 11 different GPU implementations of the popular clustering algorithm.
131072 points 32 dimensions 32 centroids
2
SRAD
Solves differential equations for image noise reduction. It is a Rodinia [30] test improved by CUDA Cooperative Groups.
Level 4
1
GeMM
Square matrix multiplication with the GeMM cuBLAS routine, using medium, single, and double precision.
Level 3
4
LavaMD
Calculates the interaction forces between a set of particles and their resulting relocation in 3D space.
Level 3
2
ResNet50 [31]
A 50-layer Convolutional Neural Network (CNN) for classi- CIFAR-10 (test 10K) Batch size 32 fying images of size 224 × 224. It has ∼ 25M parameters.
BERT [32]
Transformer model used for embedding extraction and text classification. It has ∼ 110M parameters.
Enformer [33]
Hybrid CNN-transformer for predicting gene expression from large DNA sequences. It has ∼ 230M parameters.
Data: GLUE SST-2 Batch size 32 Max. sequence 512 DNA seq. size 196k Batch size 1
PosEGNN [34]
Graph Neural Network of IBM for predicting molecular potential energy and inter-atomic forces. ∼ 6M parameters.
Dynamic simulation for 80 atoms
1
13 50 150
3.4 Workloads Regarding the workloads, we selected various tests from the Altis GPU benchmark suite [28], which has been widely used in other evaluations of the literature [35–37], as well as four popular deep learning models used for inference (particularly one applied to genomics and another to materials analysis). Most of the benchmarks are configured by a level parameter from 1 to 5, that adjusts the input configuration to vary the computational demand (level = 1 uses the least demanding configuration and level = 5 the most). In other cases, that configuration can be determined manually. Table 1 lists the chosen applications (with a brief description of each), the level or problem size used, and the number of passes set for each test. Correspondingly, Table 2 shows their standalone execution times in an NVIDIA A30, along with SM and memory utilization metrics (average and maximum), and their energy and power consumption with full GPU access. It should be noted that for the benchmarks and workloads used in this study, the program that launches the test automatically selects the thread configuration (number of blocks and threads per block for deployment on the GPU as explained in Section 2.1) based on the problem size and the GPU’s characteristics. As can be observed, the average SM utilization is quite low (below 40%) in most cases, which indicates sparse resource usage, making them good candidates for coexecution. The exception among the Altis applications is GeMM, with 78% average SM utilization. While this might suggest it is not a good candidate for co-execution, its memory utilization is very low (7.87% on average), leaving memory resources free for other applications. Furthermore, the Enformer model demonstrates high average
10
Table 2: Duration, compute and memory utilization, and energy consumption metrics on an NVIDIA A30 for the applications evaluated for co-execution. SMs utilization (%)
Mem. utilization (%)
Type
Test name
Time (s)
Avg.
Max.
Avg.
Max.
Energy (J)
Power (W)
M M M M
Sort GUPS KMeans SRAD
5.16 4.81 5.31 6.04
36.62 17.31 36.92 38.14
70 54 75 69
24.60 17.72 23.92 24.08
52 67 45 40
321.62 337.18 304.52 367.29
62.33 70.10 57.35 60.81
C C
GeMM LavaMD
5.74 6.21
78.42 34.41
100 100
7.87 0.61
35 6
543.18 531.51
94.63 85.59
RDL RDL RDL RDL
ResNet50 BERT Enformer PosEGNN
5.36 5.30 5.98 5.11
65.42 45.74 81.23 34.98
85 78 100 60
25.10 10.76 44.70 12.47
42 25 67 18
540.11 526.95 599.21 333.42
100.77 99.42 100.20 65.25
SM utilization (81.23%), combined with moderate memory utilization (44.70%). This is a common workload with a substantially different profile from the others, which will provide revealing conclusions. To ensure the results were illustrative, we sought to balance the application runtimes without altering their inherent nature. This is because if one application finishes much later than another, the latter spends a significant amount of time running alone, which dilutes the co-execution results. This effect is particularly pronounced with approaches like MPS, where the remaining application can expand to use all GPU resources once the other has completed. While one can imagine applications in a continuous co-execution stream, for practical purposes, we must evaluate them over a finite horizon. Thus, the number of execution passes was adjusted so that the test durations were similar; Table 1 specifies the passes for each, while Table 2 shows that the execution times are all within a narrow range of 5 to 6 seconds. For our analysis, it is important to discriminate how each application uses memory, as this is the resource where MIG and MPS primarily differ (the first isolates it at all levels, while the latter does not). Accordingly, the selection was guided by the goal of having applications that were sufficiently differentiated in this respect, which we classify into two types (Table 2 indicates the type for each):
• Type M (Memory): These applications show moderate memory utilization in Table 2 (average values around 20%), which aligns with other profiling metrics from the original Altis suite paper [28]. • Type C (Compute): The memory utilization of these applications is residual (values of just 8% and almost nil for LavaMD), meaning their resource demand is predominantly computational. These results also align with profiling metrics from the suite’s presentation [28]. Although the Altis tests correspond to real applications, they are designed to test the GPU in a particular way, and limiting our study to them could restrict its validity. To broaden the workload spectrum, we incorporated four prominent deep learning
11
applications, all executed within the PyTorch framework. For standard inference tasks, we utilized ResNet50 and BERT models via the TorchVision library. Additionally, to represent scientific domain-specific workloads, we included the Enformer and Pos-EGNN neural networks—targeting genomic prediction and materials analysis, respectively—sourced from the Hugging Face repository. Their evaluation datasets and batch sizes are specified in Table 1. We classify these models into a new category, Type RDL (Real Deep Learning), which we will be used for additional validation. As shown in Table 2, the selected models exhibit diverse resource usage profiles: BERT demonstrates moderate SM utilization with limited memory pressure; Enformer is resource-intensive in both dimensions; and ResNet50 or Pos-EGNN impose significantly lighter loads. These varied profiles help corroborate the findings from the Altis benchmark tests. Finally, it is worth noting that the power consumption values in Table 2 are well below the 165W Thermal Design Power (TDP) of the NVIDIA A30. The applications that come closest are the RDL workloads and GeMM, likely due to their higher intensity, yet they barely reach 100W, while other cases consume only around 60W. These data will serve as baselines for analyzing relative changes in consumption, which, as indicated, never approached the TDP causing throttling effects.
4 Performance analysis This section presents the results of pairwise co-execution experiments. While the next section examines concurrency beyond two applications, restricting the scope here to pairs enables an exhaustive exploration of all combinations and makes it easier to attribute performance interactions to specific patterns. For each pair, we compute the individual slowdown for both applications to detect and prevent imbalances: a performance gain for one job can hide a severe loss for its partner, and such asymmetries are not well captured by overall slowdown or by the slowdown of the first finishing job. As previously noted, runtimes were adjusted so standalone durations are comparable to reduce bias from imbalanced executions (see Table 2).
4.1 Impact of memory resource contention Figure 6 compares the individual slowdown of the co-execution schemes for each pair of M-type applications. For reference, the black horizontal line marks a slowdown of 2×, which can be considered a satisfactory reference, since it means that the application runs concurrently with another without either exceeding twice its solo execution time. It is particularly striking that MPS-based schemes perform so poorly for these combinations: they systematically degrade both applications’ performance, reaching slowdowns of up to 2.5 (especially for pairs including Sort). The underlying cause is revealed in Figure 7, which illustrates a considerable surge in memory utilization with MPS compared to time-slicing or MIG, evident in both average and peak values. In cases of greater slowdown (Sort test), it can be observed that MPS exhibits an average memory utilization of roughly 60%, rising to nearly 80% for the Sort –Sort pair (the highest slowdown), with peaks of 100% for applications such as GUPS. By contrast, time-slicing barely exceeds 20%—values that closely match the solo-execution 12
MPS
MPS-best-prov
MIG (2-2)
2.0 1.5
SR AD -SR AD
KM ea
ns
-SR AD
s ea n
KM ea
ns
-KM
-SR AD PS
GU
-KM ea
ns
S PS
UP
GU
-G
GU
rtSo
So
PS
SR AD
an rt-
So
rt-
KM e
Gu
So rtSo
s
ps
1.0
rt
Individual slowdown
Time-slicing
2.5
ea
ns
MPS
So rtKM
GU So rt-
So rtSo
PS
Time-slicing
MPS-best-prov
MIG (2-2)
So rtSR AD GU PS -G UP S GU PS -KM ea ns GU PS -SR AD KM ea ns -KM ea ns KM ea ns -SR AD SR AD -SR AD
100 80 60 40 20 0
rt
Memory utilization (%)
Fig. 6: Co-execution slowdown for M-type applications pairs (one bar per application).
Fig. 7: Average memory utilization % (and vertical projection to maximum) for Mtype application pairs. metrics in Table 2—while executions under MIG show only a slight increase over those solo values. This reflects prolonged DRAM engagement, which points to contention in the GPU memory subsystem shared under MPS. The likely causes are an increased rate of misses in lower-level caches and simultaneous access attempts saturating the L2 cache and DRAM bandwidth1 . In the rest of the executions, the differences in memory usage are not as exaggerated, but still very noticeable, which justifies the performance degradation observed in the slowdown. While some of these effects were foreseeable given MPS’s shared-resource design, their impact on performance is severe and it is concluded that MPS should be avoided for these kinds of workloads. Note that even the default time-slicing mechanism, which only switches between kernels and can just marginally hide latencies at the cost of a small preemption overhead, achieves slowdowns slightly below 2× in Figure 6. In contrast, those of MPS are well above this, as we have highlighted. Furthermore, the performance difference between MPS and MIG reaches almost a full slowdown point in many cases (i.e., approximately 100% of the solo execution time). MIG, with a partition of 2 slices for each application (2-2), yields quite good performances with 1 As previously mentioned, finer-grained profiling, such as that offered by NCU tool, is unfortunately incompatible with MPS, preventing analysis of lower-level metrics like cache misses.
13
Compute (SM)
100 75 50 25
0
L1
KMeans Sort SRAD
Table 3: Duration (in seconds) and scaling efficiency according to the number of slices s of the MIG instance (E = t(1)/(t(s) · s)).
DRAM
Slices
L2
Fig. 8: Throughput percentage (%) profile in NVIDIA A1002 .
Time (s)
6 4
KMeans Sort SRAD
51%
4.13
4.23
6.37
75%
KMeans
Sort
t
E
t
E
t
E
1 2 3 4 7
16.27 5.56 4.58 4.13 3.68
1.00 1.46 1.18 0.98 0.63
8.70 4.23 4.02 3.95 3.75
1.00 1.03 0.72 0.55 0.45
4.65 2.46 1.48 1.29 0.96
1.00 0.95 1.05 0.90 0.69
6.38
6.76
4.65
% memory utilization 4.04
44%
4.12 4.15
2 0
SRAD
s
40%
6.68 0.89 2.37
3.42
MIG (4-2-1)
MPS (57%-29%-14%)
Ideal (57%-29%-14%)
No co-execution
Fig. 9: Duration and memory utilization on A100 for three benchmarks under MIG, MPS, non-interference ideal and no concurrency (bar width for the available resources). slowdowns below 2× in all cases and approaching 1.5× in some, indicating that these applications are highly suitable for spatial co-execution, as they are far from saturating the GPU, but require the strong resource isolation that MIG provides and MPS lacks. Although the co-execution performance under MPS-best-prov in Figure 6 remains poor, it represents a slight improvement over basic MPS. Recall that if the sum of the SM percentages does not exceed 100% (as in this case in MPS-best-prov as explained in Section 3.2), isolation is achieved at the SM level. This means no single SM hosts threads from both applications, which prevents contention from sharing resources like the L1 cache. While this could potentially be counterproductive for warp interleaving, the thread and block configurations for each application are generally sufficient to achieve 100% theoretical occupancy (as verified with NCU tool for numerous kernels). Figure 7 shows a slight decrease in memory utilization that hints at this improvement, yet this metric is still much higher than that of time-slicing or MIG, since SM provisioning does not isolate the rest of the memory subsystem as these workloads demand. To confirm that this memory contention is a fundamental issue and not limited to two-application tests on an NVIDIA A30, we evaluated additional co-execution patterns on an NVIDIA A100. The results were similar, demonstrating the significance 2 NCU reports metrics at the kernel level, and each application consists of several kernels. To reduce the results to the application level, the metrics of its kernels have been weighted according to their duration.
14
of the previous results. For instance, we selected the M-type workloads KMeans, Sort and SRAD, which individually make little use of the throughput offered by the resources of an NVIDIA A100 (see Figure 8), but mainly use memory (see Section 3.3 for detail about NCU’s throughput percentages). Based on their durations and efficiencies (Table 3), an effective co-execution would assign 4 slices to KMeans, 2 to Sort, and 1 to SRAD. In this sense, Figure 9 compares four scenarios based on this resource ratio: (1) co-execution with MIG using a 4-2-1 partition; (2) MPS with equivalent SM provisioning percentages (≃57%, ≃29%, ≃14%); (3) an ideal case simulating no inter-application interference (execute each alone with the corresponding SM provisioning and assume that they would co-execute equally); and (4) sequential execution. The results of MPS are significantly worse than of MIG, with the percentage of time spent on memory operations increasing from 51% to 75%, which is consistent with our contention hypothesis. While applications under MIG finish considerably earlier than with MPS, they do not match the performance of the ideal interference-free case (MPS without real co-execution), which demonstrates that the problem is not MPS itself, but rather the simultaneous use of shared memory resources. In fact, MPS performs even worse than sequential execution in this experiment, despite the limited use of GPU throughput by applications. Furthermore, the MIG results are close to the ideal case, highlighting the suitability of this partitioning scheme for these tasks and indicating a low intrinsic overhead for the technology.
4.2 Resource monopolization and fair sharing Figure 10 illustrates the slowdown for C-C and M-C workload combinations. In contrast to the MPS contention issues previously observed for memory-intensive (M-M) pairs, these new combinations show that MPS often achieves slightly better results than MIG and significantly better ones than time-slicing. This holds true even for M-C pairs, where one application has moderate memory usage but does not co-execute with another that heavily competes for those resources. These results reaffirm that the root cause of the problem is the shared memory contention within the MPS service. This also suggests that when such contention problems are absent, MPS can outperform MIG—since each application can access the full memory capacity. However, the performance differences observed in these cases are small and not very significant. A striking observation from Figure 10 is the asymmetric performance degradation on pairs involving the GeMM benchmark. With MPS, the application co-executing with GeMM suffers significant slowdown, often exceeding 2× (see GeMM paired with GUPS or SRAD). Conversely, with MIG, it is GeMM itself that is penalized, while the co-running application remains largely unaffected. Quantifying this inequality, the average Gini coefficient for pairs involving GeMM reaches 0.35 for MPS and 0.12 for MIG, compared to just 0.03 for non-GeMM pairs, and dropping to 0.01 for these same GeMM pairs under Time-slicing and MPS-prov-best. We attribute this behavior to the distinct resource requirements of GeMM, which, as detailed in Section 3.4, is significantly more compute-intensive than the other workloads. Consequently, under MPS, GeMM monopolizes the SMs, effectively starving the co-running application even if the memory system remains underutilized. In contrast, under MIG, GeMM is strictly confined to a single instance (half the GPU resources in a 2–2 partition). 15
2.0
MPS
MPS-best-prov
MIG (2-2)
50-50% 70-30% 50-50% 70-30% 70-30% 70-30% 70-30% 50-50% 50-50% 50-50% 50-50%
1.5
mm -so rt ge mm -gu ps ge mm -km ea ns ge mm -sr ad lav am d-s ort lav am d-g up lav s am d-k me an s lav am d-s rad
d
ge
am av
am d-l
mm -la
ge
mm -ge ge
va md
1.0
mm
Individual slowdown
Time-slicing
2.5
lav
C-C C-M Fig. 10: Co-execution slowdowns for pairs between C and M type workloads. Above the bars for each pair, the best percentages selected for MPS provisioning are indicated.
This isolation prevents it from accessing the necessary compute resources, causing its performance to degrade by more than 2×, whereas the less demanding co-running application operates efficiently within its isolated slice. To address this inefficiency, MPS provisioning, which allows limiting the percentage of SMs each process can use, proves effective. As shown in Figure 10, provisioning significantly improves the performance of the application running alongside GeMM without excessively penalizing GeMM itself (its slowdown remains well below 2×). In our experiments, a 70-30% SM split yielded the best results, which was determined through the parameter sweep mentioned in Section 3.2. However, the optimal configuration can vary, and should be guided by QoS requirements and a brief application profiling. The key takeaway is that the MPS daemon alone does not manage resource allocation efficiently—or at least, not fairly for the less demanding processes—but the problem can be effectively mitigated with the right choice of provisioning. Furthermore, MIG may struggle to provide sufficient resources due to its rigid partitioning limitations: an instance of 2 slices is insufficient for GeMM, whereas a larger, 3-slice instance likely would be (as evidenced by its good performance with 70% of SMs under provisioned MPS), but a 3-1 partitioning scheme is unfortunately not available in an NVIDIA A30. As mentioned, to enhance the comprehensiveness of the analysis, we included four Real Deep Learning (RDL) applications that are representative of current trends: ResNet50, BERT, Enformer, and PosEGNN. Figure 11 depicts the slowdown incurred when co-executing pairs of RDL applications, as well as combinations of an RDL workload with Sort (selected as a representative of the M-type class). For ResNet50 and PosEGNN, which are significantly more compute- and memoryintensive (see Table 2), the same phenomenon as in M-type scenarios is observed: MPS leads to slowdowns reaching 2.5×. This degradation is particularly pronounced when these two models are paired together or combined with Sort —configurations characterized by simultaneous high memory pressure, where contention issues arise. Furthermore, these models perform poorly on a 2-slice MIG instance due to resource insufficiency (slowdown ≃ 2×). Under default MPS, they tend to monopolize the GPU, negatively impacting co-runners; however, this interference is largely mitigated
16
60-40%
30-70%
50-50%
Time-slicing MPS 70-30% 70-30%
MPS-best-prov 50-50% 70-30%
rt
2.5
50-50%
N
60-40%
MIG (2-2) 50-50%
2.0 1.5
GN
N-s
ort
rt Po sE
r-s o En for me
RTso BE
ort 0-s et5
r-P o
sE
GN
GN sE RTPo
BE
En for me
RDL-RDL
Re sN
r for me
N
RTEn BE
et5
0-P osE
GN
r rm e
Re sN
et5
0-E
nfo
0-B Re sN
et5 Re sN
N
1.0
ER T
Individual slowdown
3.0
RDL-M
Fig. 11: Co-execution slowdowns for pairs of RDL and M-type workloads (the best MPS provisioning is indicated).
by optimal provisioning (70-30% or 60-40%). In contrast, BERT and PosEGNN impose lighter loads, and co-execute efficiently with each other and with Sort (slowdown well below 2), mirroring the behavior of C-type applications. All in all, these results reinforce all the previous observations from Altis benchmarks.
4.3 Impact of partitioning granularity and flexibility As observed in Figures 10 and 11, GeMM, ResNet50 and Enformer suffer from significant performance degradation when co-executing on 2-slice instances (much more than double the solo runtime), that presumably would not occur with 3 slices (if such an instance were possible). In these cases, it could be argued that co-locating applications is not as critical, as the problem arises from their already high resource utilization (see Table 2). This means MIG partitioning lacks the granularity and flexibility to accommodate such fine-grained adjustments. However, the problem not only lies in the coarse-grained nature of the partitioning and the limited valid configurations, but also that every instance maintains the same proportion of different resource types (as explained in Section 2.4, instances group identical slices, so the ratio of SMs, memory capacity, or bandwidth, is always the same). This characteristic can be unsuitable for applications with asymmetric demand patterns (e.g., when the ratio between compute demand on the SMs and memory subsystem usage differs significantly from the fixed ratio of the instances). The least detrimental solution with MIG might be to use an instance that satisfies the maximum of these demands, leading to an implicit underutilization of other resources. However, sometimes even this allocation is not possible due to partitioning granularity and restrictions, whereas MPS memory sharing can provide the necessary flexibility to distribute this resource (provided there is no contention for it, which, as seen earlier, has an even worse impact). To illustrate this, we used tests with a different profile: in particular, HotSpot (50 passes of the Rodinia suite benchmark [30] on a 1024 square grid) and Raytracing (15 passes of the level 3 from Altis [28]). Table 4 reports their execution times and the percentage of peak throughput for the SMs and the memory hierarchy on an A30 GPU (see Section 3.3) under the following three schemes: sequential execution (the entire GPU for each benchmark), a 2-2 MIG partition, and the MPS 17
Table 4: Example of MIG inflexibility issues, using two applications and reporting throughput usage and durations in an NVIDIA A30. No co-execution Bench.
HotSpot Raytracing End time:
Throughput (%) SMs
L1
L2
RAM
38 39
42 46
2 32
0 5
MPS (50%–50%)(a)
MIG (2–2)
Time (s)
42.75 44.93
Throughput (%) SMs
L1
L2
RAM
38 36
45 19
2 46
0 46
87.68
Time (s)
Time (s)
65.32 92.24
60.32 61.45
92.24
61.45
(a)
SM and memory performance was profiled using the NCU tool, which is not compatible with MPS, and there is no tool that provides similar data.
equivalent (50-50% of SMs). When run sequentially, both applications utilize nearly 40% of the SMs’ peak performance, suggesting that a 2-2 partition is reasonable from a compute perspective. However, their memory profiles are very different: HotSpot makes little use of memory (almost exclusively local to the L1 cache), whereas Raytracing intensively uses the shared L2 cache (32% of throughput). This suggests that the applications should not conflict with each other, exhibiting a rather asymmetric device usage pattern between compute and memory. When co-executed with MIG, Raytracing appears to saturate its instance’s memory resources, shifting a large portion of accesses to DRAM (46% of throughput) but with low SM utilization (36%). In contrast, with MPS, the memory is shared between the two applications with little interference, achieving better co-execution than MIG (61.45s vs. 92.24s) and completing both jobs faster than the sequential execution. It is worth noting several nuances regarding this issue. First, we have illustrated this on an A30 GPU because its partitioning space is more limited: it only has 4 slices, 3 instance sizes (1, 2, and 4), and 5 possible partitions, making it unfeasible, for example, to assign 3 or 4 slices to the application with higher memory demand and 2 to the other. On bigger GPUs such as the A100, H100, or B200, which feature higher throughput and greater MIG granularity (7 slices, 5 possible instance sizes, and up to 19 partitions), this issue becomes less relevant because it is easier for applications to adapt well to one of the available device partition proportions. Second, MIG includes Compute Instances (see Section 2.4), which allow memory to be shared among concurrent applications within the same instance without sharing the SMs. With these instances, it is possible to obtain results similar to those observed with MPS, although in general cases, this implies predicting and controlling interference to avoid contention problems, which is not always easy, as some scheduling proposals have noted [16, 38]. Finally, the ideal solution would be a MIG-like, but asymmetric, isolation: for example, being able to form an instance with 2 compute slices and 4 memory slices, all of them fully isolated. However, that seems infeasible or technically very difficult, at least in the short term; currently, MIG uses isolation at the GPC level, but it would likely require the ability to isolate resources at a lower, decoupled level.
18
1.3 Overall slowdown Trel
1.2
Time-slicing MPS MPS-best-prov MIG (2-2)
Enformer-Enformer Sort-Sort Enformer-Enformer former Enformeer-reEln= 1.4 GeMM-GeMM
Sort-Sort
1.1 1.0
erel = 1.2
Sort-GUPS
0.9
erel = 1.0
0.8 LavaMD-LavaMD
0.7 0.95
1.00
SRAD-LavaMD SRAD-LavaMD 1.05 1.10 1.15 Power ratio Prel
erel = 0.8
1.20
1.25
Fig. 12: Scatter plot of application pairs according to their power increase ratio (xaxis Prel = Pco-exec / Pseq ) and their overall slowdown (y-axis Trel = Tco-exec / Tseq ). The isocurves for constant values of the energy ratio (erel = x · y ) are highlighted.
5 Efficiency and scalability analysis 5.1 Energy consumption Another dimension of great relevance in the comparison between co-execution technologies is energy consumption. However, it is closely tied to the performance analysis just carried out: cases of memory contention or GPU monopolization with MPS will inevitably make energy consumption worse than other options or even worse than sequential execution without sharing the device (this was already the case in terms of performance). Similarly, co-executions of applications that proved very efficient (slowdown of about 1.5× for both) will very likely be very efficient in their energy consumption as well. Nevertheless, the energy analysis is not exactly the same as the performance analysis, since it is also influenced by the power consumed by the device (energy per time unit, i.e., watts), which in principle should increase with co-execution as device resources are used more intensively (in addition to control logic or other overheads introduced by the technologies). To visualize these aspects together, Figure 12 presents a scatter plot for all the application pairs executed on an NVIDIA A30 under each co-execution scheme (color legend). The x-axis represents the relative power draw, which is the ratio of the average power during co-execution to that of sequential execution: Prel = Pco-exec / Pseq . The y-axis shows the overall slowdown (see Section 3.3), i.e., the ratio of the co-execution
19
duration to the sequential execution time: Trel = Tco-exec / Tseq .3 Notably, the product of these two axes corresponds directly to the relative energy consumption: x · y = (Pco-exec · Tco-exec ) / (Pseq · Tseq ) = Eco-exec / Eseq . This relationship allows for plotting energy isocurves for key values, where a constant energy ratio is defined by erel = x · y . The plot includes a primary isocurve for a ratio of 1.0, which serves as a baseline: points below this line represent a net reduction in energy consumption, while points above it indicate an increase. In addition, consumption isocurves at 0.8 (20% reduction), 1.2 (20% increase), and 1.4 (40% increase) are also highlighted. Finally, in the points we will refer to, the name of the co-executed pair is highlighted. As observed in Figure 12, the time-slicing scheme maintains a power draw very close to the sequential baseline (points are clustered around Prel = 1). This is expected, as this method does not allow for the simultaneous use of functional units (only alternation). Under time-slicing, the total energy consumption improves for some application pairs and degrades for others (falling below or above the isocurve, respectively), primarily depending on the slight performance variations discussed previously (the y-axis value). However, these changes are modest, with no application pair approaching the 0.8 or 1.2 isocurves. In contrast, spatial sharing inevitably increases instantaneous power consumption due to higher resource concurrency. However, this overhead is often cost-effective, as the reduction in execution time frequently offsets the power penalty, leading to overall energy savings (up to 20% in some cases). As shown in Figure 12, MPS exhibits a modest power increase (≃ 7%) for non-contentious pairs such as SRAD-LavaMD (Prel = 1.07, Trel = 0.68). Here, the trade-off is clearly advantageous, yielding a net energy reduction of over 20% (situated below the 0.8 isocurve). Conversely, for pairs suffering from contention or saturation, such as Sort-Sort or Enformer-Enformer (Prel = 1.15, Trel ≃ 1.3), the power penalty rises to 15%. In these scenarios, the combination of higher power and longer execution time results in an energy degradation exceeding 40% (above the 1.4 isocurve)—a penalty even more severe than the performance loss itself. In conclusion, MPS has the potential to produce highly polarized results, and using it exclusively without resource contention is even more important in terms of energy consumption. Using MPS with SM provisioning results in a consistently higher power draw versus basic MPS, ranging from 10% to 15% extra. This is potentially attributable to the additional logic required for SM-level isolation. However, there appears to be no significant difference in the overall power-performance trade-off, as the data points for both basic MPS and MPS-best-prov show a similar distribution relative to the isocurves (see particularly SRAD-LavaMD, Sort-Sort and Enformer-Enformer, which are highlighted in both). This suggests that the higher power draw of provisioning is generally offset by corresponding performance gains. Finally, MIG exhibits the highest power levels in Figure 12: typically around a 15% increase, reaching 25% when handling applications more demanding than the assigned instance such as Enformer and GeMM. This is natural, because full resource isolation must necessarily incur an energy overhead, especially when the instance size is 3 Note that Tseq and Pseq are relative to the sequential execution of both applications, resulting in ratios close to 1. This differs from the slowdowns of Section 4, where individual slowdowns were calculated to verify that the performance of one application did not degrade the other, so the values were around 2.
20
insufficient for the application’s demand as in those cases, causing issues like increased stress on high memory levels (as in the example from Table 4). Although MIG delivered the best average performance (with y-values more frequently below 1.0), its peak performance does not match the best cases of MPS, and conversely, it also avoids severe performance degradation. Consequently, the energy trade-off is many times unfavorable, as approximately half of the MIG test cases fall above the 1.0 isocurve, even though most are below Trel = 1 (they improve the original performance). For example, Sort-GUPS, despite reducing execution time by 10% (Trel = 0.9), is not energy-efficient because its power draw increases by 19% (Prel = 1.19). Furthermore, even when MIG generates energy savings, these savings do not reach the levels seen with MPS (no MIG pair falls below the 0.8 isocurve, although LavaMD-LavaMD is close). The conclusion is that MIG hardware isolation carries an important inherent energy cost, and to be energy efficient, the performance gains must be substantial enough to offset this overhead (which occurs in approximately half of our test cases).
5.2 Performance concurrency scaling The detailed analysis of each combination has so far allowed us to characterize key aspects of the technologies, but to do so, the degree of concurrency has been kept fixed at two applications (pairs of workloads, in order to analyze each case). In this regard, we extend the study in this dimension by also evaluating performance scaling to four concurrent applications (the MIG instance not evaluated so far on an A30 GPU is the 1-slice instance, which allows up to four applications to run on a 1-1-1-1 partition). Since the possible combinations grow exponentially with the number of applications (the reason why an exhaustive analysis was only done with 2 applications), we limit the co-execution to copies of the same application (in particular, these combinations should reflect the MPS contention issues that occurred between M-type workloads). Naturally, the scaling will be conditioned by the applications’ resource utilization; while co-executing pairs of these workloads was reasonable based on the profiles of Table 2, running four of them simultaneously would be unrealistic. Consequently, we divide this study into two scenarios:
• Medium workloads: With the same configuration as in the previous tests, stated in Table 1. ResNet50 and BERT will be the RDL models reported because the others have very similar results and these have lightweight versions (ResNet18 and DistilBERT) to evaluate with a higher degree of co-execution. • Small workloads: Compared to the specification in Table 1, the workload level was reduced by one for Altis benchmarks, KMeans was run on only 16,384 points with 16 dimensions and 16 centroids, and the RDL models were replaced with smaller equivalent versions: ResNet18 [31] and DistilBERT [39]. Table 5 shows the metrics of new durations, utilization, and energy for these tests, which are now very undemanding (except for GeMM, whose utilization is moderate).4 To streamline the analysis, we exclude time-slicing from the results, as it offers nothing new: as previously noted, it behaves like sequential execution in all cases. 4 In this case, it is not as important for the durations to be similar because we will not be co-executing different tests, but rather multiple copies of the same application.
21
Table 5: Small workloads metrics when running solo on an NVIDIA A30. SMs utilization (%) Time (s) Avg.
Mem. utilization (%)
Type
Test name
Max.
Avg.
Max.
M M M M
Sort GUPS KMeans SRAD
2.84 1.11 0.53 2.12
12.20 7.18 6.07 8.87
20 15 12 20
9.42 12.11 8.22 6.44
22 58 20 32
140.36 52.28 24.00 92.12
49.42 47.10 45.28 43.45
C C
GeMM LavaMD
2.54 0.24
35.24 5.09
100 49
1.20 0.51
28 5
187.88 10.20
73.97 42.50
2.16 1.82
10.11 12.52
42 30
2.21 3.39
22 45
124.21 83.56
57.50 45.91
RDL ResNet18 RDL DistilBERT
Energy (J) Power (W)
MPS-best-prov MIG 2 copies 4 copies
KMeans LavaMD
0.6
BERT
0.8
Sort GUPS SRAD GEMM
ResNet50
1.0 1.2 1.4 Overall slowdown Trel = Tco exec /Tseq
1.6
1.8
(a) Medium workloads.
Sort GUPS KMeans SRAD LavaMD 0.6
0.8
ResNet18 DistilBERT
MPS-best-prov MIG 2 copies 4 copies
GeMM
1.0 1.2 1.4 Overall slowdown Trel = Tco exec /Tseq
1.6
1.8
(b) Small workloads.
Fig. 13: Dumbbell plots comparing the overall slowdown when co-executing 2 versus 4 copies of each application under different co-execution schemes.
We also stick with the MPS-best-prov scheme, which in all cases outperforms MPS, and it is very easy to determine the optimal provisioning beforehand as copies of the same application are co-executed, so SMs must be distributed uniformly: 50-50% for 2 applications and 25-25-25-25% for 4. Figure 13 shows the changes in overall slowdown when moving from 2 to 4 concurrent copies of each application, comparing both schemes and workload sizes (Medium and Small). For the Medium workloads (Figure 13a), the poor scaling is striking: performance degrades significantly compared to sequential execution, showing a wide and unfavorable shift in overall slowdown. This is logical, as the utilization profiles of these tasks 22
suggested that limiting their resources to a quarter of what is available on this GPU would be insufficient. For M-type applications (Sort, GUPS, KMeans, and SRAD), this effect is much more exaggerated with MPS, likely due to the memory contention issues being greatly exacerbated by having more applications sharing resources in parallel. With MIG, this degradation is more moderate for those applications due to its resource isolation. In any case, it can be concluded that the degree of application coexecution must be closely linked to their level of resource underutilization, otherwise the performance losses can be huge (over 80% slower than sequential execution in some cases). In contrast, the Small workloads also show a worse overall slowdown when moving from 2 to 4 concurrent copies (Figure 13b), but in a much more moderate fashion. They start from more notable improvements over sequential execution (overall slowdowns around 0.6 in many cases) and are still preferable to no co-execution even with 4 applications (a small increase in slowdown that remains well below 1.0). The exception is GeMM, which remains relatively demanding even in its Small version, making it inefficient to co-execute 4 copies in performance terms (much like many Medium workloads). It is also apparent for the M-type applications (the first four) that scaling to 4 copies is considerably better with MIG than with MPS-aprov (this is not the case for the other workloads). Again, this appears to be a consequence of MIG’s total isolation versus MPS’s shared memory system. In conclusion, if resource underutilization is higher, opting for a finer-grained MIG partition to co-execute more applications seems to be a good option, especially if their memory system usage is not entirely residual.
5.3 Energy efficiency scaling To analyze energy consumption scaling, we focused on the Small applications. As shown in Figure 13a, running four concurrent instances of the Medium workloads significantly worsens performance and, consequently, energy consumption (given this clear inefficiency, a deeper analysis of this scenario was not pursued). For the Small applications, we analyzed the variation in the relative energy increase ratios Erel when moving from two to four concurrent applications (Figure 14). To better evaluate the trade-off with performance, we also examined the relative increase ratios of the EnergyDelay Product (EDP) in Figure 15, defined as the energy consumed multiplied by the execution time: EDPrel = Erel · Trel = (Eco-exec · Tco-exec ) / (Eseq · Tseq ). The energy consumption results in Figure 14 accentuate the inefficiencies of both schemes when scaling to four applications. While running two concurrent copies generally consumes less energy than sequential execution, scaling to four copies often proves detrimental (all of them except LavaMD and GeMM). In fact, this is purely due to energy overhead because these applications were faster than the reference with four copies (Trel < 1 in Figure 13b). In the case of the M-type applications, the effects of memory contention that occur with MPS but not MIG are accentuated to the point that the consumption of four copies of Sort, GUPS, KMeans, or SRAD is greater with MPS than with MIG, but it was the opposite with two copies. In contrast, for applications without contention issues like ResNet18 and DistilBERT, MIG’s overhead leads to significantly higher energy use, even surpassing the sequential baseline (Erel > 1), while MPS remains more efficient (Erel < 1). One clear conclusion is that 23
Sort
MPS-best-prov MIG 2 copies 4 copies
GUPS KMeans SRAD
GEMM LavaMD
0.6
0.8
1.0
ResNet18 DistilBERT 1.2
1.4
Erel = Eco exec /Eseq
1.6
1.8
2.0
Fig. 14: Energy consumption ratio from 2 to 4 concurrent copies of Small workloads.
Sort GUPS KMeans SRAD
MPS-best-prov MIG 2 copies 4 copies GEMM
LavaMD
0.5
ResNet18 DistilBERT 1.0
1.5
EDPrel = EDPco exec /EDPseq
2.0
2.5
Fig. 15: Energy-Delay Product ratio from 2 to 4 concurrent copies of Small workloads.
the effect of both phenomena intensifies significantly as the degree of concurrency increases. Along the same lines, for GeMM, a significantly more demanding application than the others, co-executing four copies is extremely detrimental, increasing energy consumption by approximately 80% compared to its sequential baseline. This underscores the importance of profiling application demands to determine an appropriate degree of concurrency. Conversely, an application like LavaMD, which saw substantial performance gains (Figure 13b), continued to improve its energy efficiency at four copies with both schemes. Regarding the Energy-Delay Product (EDP), the results in Figure 15 show that performance improvement successfully compensates for the increased energy consumption with four concurrent copies. As shown in Table 5, the reduction in execution time was approximately 20% compared to the sequential baseline, while the corresponding energy increase was only around 10% (see in Figure 14 Sort, GUPS, KMeans and SRAD, especially with MPS, and ResNet18 and DistilBERT with MIG). Consequently, the overall EDP, which gives equal weight to both factors, is favorable (EDPrel < 1), indicating that the gains in performance outweigh the energy penalty (for all applications except GeMM). The application-by-application analysis reveals a combination of the phenomena already discussed, and it is striking how GeMM’s EDP
24
Table 6: MIG reconfiguration latency and power consumption overhead (relative to 32W of idle GPU) by instance size and operation type (average of 10 runs per test). treconfig. (s)
△ Power (W)
△ Energy (J)
Slices
Create
Destroy
Create
Destroy
Create
Destroy
1 2 4
0.11 0.12 0.13
0.10 0.10 0.10
21.12 19.64 18.95
17.54 17.71 20.82
2.32 2.36 2.46
1.75 1.77 2.08
is extremely poor with 4 copies (not so with 2, with an enormous difference between them) because its demand is excessive for that degree of concurrency, which impairs performance but also increases consumption (unlike other tests where compensatory trade-offs were observed).
5.4 Dynamic reconfiguration Until now, MPS and MIG have been exhaustively analyzed from both performance and energy perspectives under various static configurations (initially with co-executed pairs, and subsequently with sets of four applications). However, it remains to evaluate the change of configurations, which can be done “on the fly”, i.e., while some applications are running (dynamic reconfiguration). In the case of MPS, the inherent cost of such reconfigurations is practically negligible. Whether introducing a new application into an existing MPS context or varying the SM limit percentage (in the provisioned version), no overhead is observed in application launch relative to the baseline cost (i.e., launch latency or power consumption prior to the reconfiguration request). A separate issue is whether the new configuration yields better or worse performance and consumption, a factor already analyzed in our previous study of static configurations, specifically regarding the contention problems and the scaling from 2 to 4 concurrent applications. In the case of MIG, reconfiguration entails a more significant cost because it involves not only a logical modification of resource distribution but also a hardware alteration of the grouping of fully isolated instances (i.e., their physical destruction and creation). Table 6 summarizes the duration of MIG instance destruction and creation according to its size on an NVIDIA A30 (treconfig. ), as well as the increase in power (△ Power) and energy consumption (△ Energy) relative to baseline values (the idle GPU consumes a stable 32W at fixed operating frequencies). As observed, the overhead is quite low, with latencies of merely a few tenths of a second per instance—affecting only the slices of that specific instance and not those of others running applications— and power increases of ∼ 20W, which translates to ∼ 2J of extra energy per operation. This makes the use of reconfigurations to better adapt resource distribution to workload characteristics highly feasible for performance optimization, as demonstrated in our previous MIG scheduling proposals [8, 37]. Furthermore, the overhead is shown to be practically independent of instance size, as creating or destroying instances of size 1, 2, or 4 incurs nearly identical costs.
25
Finally, it has been empirically verified that the performance of multiple reconfigurations is equivalent to sequential execution, even when attempted in parallel (where one might a priori expect parallelization over disjoint resources). This means that even if distinct CPU threads or processes attempt to simultaneously create or destroy disjoint instances on the GPU, the total duration is approximately the sum of the individual operations. This behavior has been verified for all possible partition transitions on A30 and A100 GPUs using NVIDIA driver version 580.82.07 (subsequent versions show no changes in this regard). This might be improved in the future, unless there are inherent technical limitations unknown to us.
6 Related work While the growing underutilization of GPU resources has led to significant interest in co-execution technologies like MPS and MIG [6, 7], prior work has predominantly centered on developing job schedulers to optimize a target metric [8, 15, 16, 37]. In these studies, the evaluation of the technologies themselves is often secondary and limited, serving merely as a preliminary motivation for their proposed techniques rather than the focus of the research. Consequently, these analyses lack the depth required for a thorough understanding. For instance, works like MISO [15] and MIGER [16] illustrate the memory contention issues in MPS versus the isolation in MIG, but their evidence is typically limited to a few DL models in a single provisioning configuration. They do not provide broader insights into the frequency of these problems or their correlation with different workload types. Similarly, while MIGER [16] explores technology scaling with concurrent applications, it does so with a few examples aimed only at motivating its optimization algorithms. In contrast, our study provides a comprehensive characterization that can serve as a source of lessons for future research. Although considerably scarcer, some studies do focus purely on evaluating these technologies. However, they often have different scopes or limitations, such as focusing on a single technology rather than a direct comparison. For example, Wende et al. [40] evaluated the underlying hardware support for MPS (Hyper-Q) at the level of OpenMP parallel regions and MPI processes, concluding that the process-level parallelism is much more effective (the approach used by MPS). Meanwhile, the work of Weaver et al. [24] is more analogous to ours, evaluating MPS with ten workload combinations characterized by their performance and energy efficiency. Nonetheless, their study does not utilize MPS resource provisioning nor does it compare MPS against MIG, which are two fundamental axes of our work. Other studies like MIGPerf [41] focus primarily on MIG, and concentrate on DL inference using batch size as a primary scaling dimension. Finally, Robroek et al. [23] provide a direct comparison between MPS and MIG with a larger scaling test than in our study (up to 7 concurrent instances on an A100), but they do not use the MPS provisioning option and evaluate three pairs of DL workloads that do not incur memory contention. Our study focuses on MPS and MIG as they are the standard GPU sharing technologies provided by the NVIDIA driver (although MIG is only supported by some GPU models). Unlike experimental frameworks, MPS and MIG are integrated into the
26
hardware and driver ecosystem, making their performance a critical baseline for realworld deployments. Understanding their overhead and limitations is therefore more relevant for practitioners than analyzing research prototypes. However, other lowerlevel works have presented custom concurrency tools that explicitly manage CUDA streams and application switching, usually oriented to more specific goals and workloads. For example, Tally [42] is a virtualization layer that provides performance isolation, reporting a 7.2% average overhead on inference latency, compared to 195.5% for MPS and 188.9% for the prior state-of-the-art [43].
7 Conclusions This paper has presented a comprehensive empirical evaluation of spatial co-execution on modern GPUs, focusing on NVIDIA’s MPS and MIG technologies. We have compared these against a standard time-slicing baseline, analyzing their performance, energy consumption, and scalability across a diverse set of workloads with varying computational and memory demands. Our findings highlight the significant potential of spatial sharing to improve GPU utilization and efficiency, while also revealing the critical trade-offs between the flexibility of MPS and the strong isolation of MIG. GPUs and technologies from other manufacturers such as AMD and Intel work differently and would require their own evaluation, but NVIDIA is more relevant (it is the clear market leader) and the basic principles regarding the trade-offs between sharingbased and isolation-based co-execution are, a priori, extrapolable. The key insights obtained are summarized below.
7.1 Key insights A primary finding is that, when correctly matched with workloads, both spatial sharing technologies can yield substantial improvements over the standard time-slicing baseline. In some of our test cases, this resulted in overall performance improvements of up to 30% (Trel ≃ 0.7 in Figure 12) and reductions in energy consumption approaching 20%. However, the path to achieve these gains critically depends on understanding the trade-offs of each technology.
➢ Co-execution through MIG hardware resource isolation MIG provides robust performance isolation by partitioning hardware resources, which effectively prevents interference between co-running applications, at the cost of a minor overhead during dynamic reconfiguration (Section 5.4). Consequently, most application pairs achieved a notable performance improvement compared to sequential execution, with many showing a 20% reduction in overall slowdown and some reaching up to 30% (see Figure 12). This benefit was particularly stark for combinations of two memory-intensive (M-type) workloads, which completed their execution almost twice as fast as with MPS (see Figure 6). As a drawback, Section 4.3 highlighted that rigid, discrete partitioning of GPU resources into predefined instance sizes can lead to underutilization if an application’s resource requirements do not align well with the available instance configurations (we saw in Table 4 much slower co-execution with higher L2 and DRAM memory usage). 27
The issue is especially pronounced on GPUs with fewer slicing options, such as the A30, where the lack of granularity can yield suboptimal performance. Furthermore, while MIG consistently improves energy efficiency over sequential execution, it introduces slightly more overhead than MPS and does not reach the same peak efficiency as the most favorable cases of MPS (Erel < 0.8 in Figure 12). This efficiency gap became more noticeable when scaling to four concurrent applications (Figures 14 and 15).
➢ MPS flexibility and memory sharing issues MPS offers a more flexible approach to resource sharing, allowing for fine-grained allocation of computational resources. This flexibility, however, comes at the cost of potential contention for shared resources, particularly the memory subsystem. The ability to provision SMs provides a mechanism to mitigate some SM-level sharing issues and, crucially, to prevent resource monopolization, as was observed with the dominant GeMM application in Figure 10 and the Enformer model in Figure 11. Nevertheless, this provisioning does not solve the underlying memory contention problems inherent in memory-intensive (M-type) applications. As a result, the performance and energy consumption outcomes with MPS are highly polarized (see Figure 12). It achieves better results than MIG in favorable scenarios but performs significantly worse when memory conflicts arise. This polarization was further accentuated when scaling to four concurrent applications (Figures 5 and 14). These findings underscore the critical importance of careful workload characterization to select the appropriate technology.
➢ Potential future enhancements The evolution of MIG would benefit from more granular and flexible partitioning schemes to reduce resource underutilization. Specifically, the ability to create instances with asymmetric resources (e.g., more memory bandwidth relative to compute resources) would better match diverse application profiles. While technically challenging in the short term, this would help bridge the gap between MIG’s strong isolation and MPS’s flexibility. Similarly, dynamic scheduling would benefit if reconfiguration operations on disjoint slices could be effectively parallelized, given that their current performance is equivalent to sequential reconfiguration (Section 5.4). On the other hand, to mitigate its primary weakness, MPS could incorporate lightweight memory management mechanisms. While full isolation remains the domain of MIG, adding features that throttle memory bandwidth at the process level (analogous to SM provisioning) could prevent worst-case contention scenarios and make MPS a more robust solution across a wider range of applications.
7.2 Future work Our findings pave the road for a dynamic, workload-aware scheduler that automatically selects the optimal co-execution strategy. While prior work has already explored this approach—particularly for MIG—our analysis provides a solid empirical foundation for developing more robust, and potentially hybrid, schedulers that combine MPS and MIG according to task characteristics.
28
Declarations Ethical Approval Not applicable.
Competing interests There are no competing interests.
Authors’ contributions J.V. conducted the design, implementation and evaluation of the experiments, and wrote the main manuscript text. L.C., F.I. and K.O. collaborated on the definition and supervision of research tasks, the design and critical analysis of the experiments, and the review and writing of the manuscript.
Acknowledgements This work is funded by Grants PID2021-126576NB-I00 and PID2024-158311NB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”. We thank the HPC&A group at Universitat Jaume I de Castellón for granting us access to A100 and H100 GPUs for profiling and evaluation purposes.
Availability of data and materials Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
References [1] Pandey M, Fernandez M, Gentile F, Isayev O, Tropsha A, Stern AC, et al. The transformational role of GPU computing and deep learning in drug discovery. Nature Machine Intelligence. 2022;4(3):211–221. https://doi.org/10.1038/ s42256-022-00463-x. [2] Silvano C, Ielmini D, Ferrandi F, Fiorin L, Curzel S, Benini L, et al. A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms. ACM Comput Surv. 2025 Jun;57(11). https://doi.org/10.1145/3729215. [3] Atluri A. The Evolution of NVIDIA GPUs for Deep Learning: From Gaming to AI Powerhouse. International Journal of Advanced Research in Engineering & Technology. 2025 02;16:540–551. https://doi.org/10.34218/IJARET 16 01 038. [4] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I.: Language Models are Unsupervised Multitask Learners. OpenAI technical report. 2019. [5] NVIDIA.: Technical documentation of the Blackwell architecture. URL https: //resources.nvidia.com/en-us-blackwell-architecture. 29
[6] Adufu T, Ha J, Kim Y. Exploring the Diversity of Multiple Job Deployments over GPUs for Efficient Resource Sharing. In: International Conference on Information Networking. IEEE Computer Society; 2024. p. 777–782. https://doi.org/10.1109/ ICOIN59985.2024.10572198. [7] Durvasula S, Zhao A, Kiguru R, Guan Y, Chen Z, Vijaykumar N.: ACS: Concurrent Kernel Execution on Irregular, Input-Dependent Computational Graphs. Available from: https://arxiv.org/abs/2401.12377. [8] Villarrubia J, Costero L, Igual FD, Olcoz K. Leveraging Multi-Instance GPUs through moldable task scheduling. Journal of Parallel and Distributed Computing. 2025;204:105128. https://doi.org/10.1016/j.jpdc.2025.105128. [9] Elvinger P, Strati F, Jerger NE, Klimovic A.: Measuring GPU utilization one level deeper. Available from: https://arxiv.org/abs/2501.16909. [10] Gao Y, He Y, Li X, Zhao B, Lin H, Liang Y, et al. An Empirical Study on Low GPU Utilization of Deep Learning Jobs. In: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. ICSE ’24. New York, NY, USA: Association for Computing Machinery; 2024. https://doi.org/10.1145/ 3597503.3639232. [11] You J, Chung JW, Chowdhury M. Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training. In: 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). Boston, MA: USENIX Association; 2023. p. 119–139. Available from: https://www.usenix.org/ conference/nsdi23/presentation/you. [12] Morand C, Ligozat AL, Névéol A.: How Green Can AI Be? A Study of Trends in Machine Learning Environmental Impacts. Available from: https://arxiv.org/ abs/2412.17376. [13] NVIDIA.: Multi-Process Service. URL https://docs.nvidia.com/deploy/mps/. [14] NVIDIA.: Multi-Instance GPU User Guide. datacenter/tesla/mig-user-guide/.
URL https://docs.nvidia.com/
[15] Li B, Patel T, Samsi S, Gadepally V, Tiwari D. MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant GPU Clusters. In: 13th Symposium on Cloud Computing; 2022. https://doi.org/10.1145/3542929.3563510. [16] Zhang B, Li S, Li Z. MIGER: Integrating Multi-Instance GPU and Multi-Process Service for Deep Learning Clusters. In: 53rd International Conference on Parallel Processing; 2024. p. 504–513. https://doi.org/10.1145/3673038.3673089. [17] Zhao C, Gao W, Nie F, Zhou H. A Survey of GPU Multitasking Methods Supported by Hardware Architecture. IEEE Transactions on Parallel and Distributed
30
Systems. 2022;33(6):1451–1463. https://doi.org/10.1109/TPDS.2021.3115630. [18] NVIDIA Corporation.: CUDA C++ Programming Guide. Available from: https: //docs.nvidia.com/cuda/pdf/CUDA C Programming Guide.pdf. [19] AMD.: MxGPU deployment guide. URL https://drivers.amd.com/relnotes/amd mxgpu deploymentguide vmware.pdf. [20] Intel.: iGPU SR-IOV documentation. URL https://www.intel.sg/content/dam/ www/central-libraries/us/en/documents/2022-09/intel-whitepaper2022-dfi-v11. pdf. [21] NVIDIA Corporation.: NVIDIA Data Center GPU Manager (DCGM) Documentation. https://developer.nvidia.com/dcgm. [22] NVIDIA Corporation.: API Reference Guide of NVIDIA Management Library (NVML). Version vR580. Available from: https://docs.nvidia.com/deploy/ nvml-api/index.html. [23] Robroek T, Yousefzadeh-Asl-Miandoab E, Tözün P. An Analysis of Collocation on GPUs for Deep Learning Training. In: Proceedings of the 4th Workshop on Machine Learning and Systems. EuroMLSys ’24. New York, NY, USA: Association for Computing Machinery; 2024. p. 81–90. https://doi.org/10.1145/3642970. 3655827. [24] Weaver A, Kavi K, Milojicic D, Enriquez RPH, Hogade N, Mishra A, et al. Granularity- and Interference-Aware GPU Sharing with MPS. In: SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis; 2024. p. 1630–1637. https://doi.org/10.1109/ SCW63240.2024.00203. [25] Zhao Y, Liu X, Liu S, Li X, Zhu Y, Huang G, et al.: MuxFlow: Efficient and Safe GPU Sharing in Large-Scale Production Deep Learning Clusters. Available from: https://arxiv.org/abs/2303.13803. [26] Giudici P, Raffinetti E. RGA: a unified measure of predictive accuracy. Advances in Data Analysis and Classification. 2025;19(1):67–93. https://doi.org/10.1007/ s11634-023-00574-2. [27] Giudici P, Raffinetti E, Toscani G. Measuring multidimensional inequality: a new proposal based on the Fourier transform. Statistics. 2025;59(2):330–353. https://doi.org/10.1080/02331888.2024.2434904. [28] Hu B, Rossbach CJ. Altis: Modernizing GPGPU Benchmarks. In: 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS); 2020. p. 1–11. https://doi.org/10.1109/ISPASS48437.2020.00011.
31
[29] Danalis A, Marin G, McCurdy C, Meredith JS, Roth PC, Spafford K, et al. The Scalable Heterogeneous Computing (SHOC) benchmark suite. In: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units. GPGPU-3. New York, NY, USA: Association for Computing Machinery; 2010. p. 63–74. https://doi.org/10.1145/1735688.1735702. [30] Che S, Boyer M, Meng J, Tarjan D, Sheaffer JW, Lee SH, et al. Rodinia: A benchmark suite for heterogeneous computing. In: 2009 IEEE International Symposium on Workload Characterization (IISWC); 2009. p. 44–54. https: //doi.org/10.1109/IISWC.2009.5306797. [31] He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 770–778. [32] Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 4171–4186. https://doi.org/10.18653/v1/N19-1423. [33] Avsec Ž, Agarwal V, Visentin D, Ledsam JR, Grabska-Barwinska A, Taylor KR, et al. Effective gene expression prediction from sequence by integrating longrange interactions. Nature methods. 2021;18(10):1196–1203. https://doi.org/10. 1038/s41592-021-01252-x. [34] Research I.: IBM Materials: AI for Materials Science. GitHub. Model: posegnn.v1-6M. https://github.com/IBM/materials. [35] de la Calle E, Garcı́a C. Evaluation of Juliana Tool: A translator for Julia’s CUDA.jl code into KernelAbstraction.jl. Future Generation Computer Systems. 2025;171:107813. https://doi.org/10.1016/j.future.2025.107813. [36] Saiz A, Prieto P, Abad P, Gregorio JA, Puente V. Top-Down Performance Profiling on NVIDIA’s GPUs. In: 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS); 2022. p. 179–189. https://doi.org/10. 1109/IPDPS53621.2022.00026. [37] Villarrubia J, Costero L, Igual FD, Olcoz K. Solving the task scheduling and GPU reconfiguration problem on MIG devices via deep reinforcement learning. Future Generation Computer Systems. 2026;176:108145. https://doi.org/10. 1016/j.future.2025.108145. [38] Xiao W, Bhardwaj R, Ramjee R, Sivathanu M, Kwatra N, Han Z, et al. Gandiva: Introspective Cluster Scheduling for Deep Learning. In: 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). Carlsbad, CA: USENIX Association; 2018. p. 595–610. Available from: https://www.usenix.org/
32
conference/osdi18/presentation/xiao. [39] Sanh V, Debut L, Chaumond J, Wolf T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. Available from: https://arxiv.org/abs/1910. 01108. [40] Wende F, Steinke T, Cordes F. Multi-threaded Kernel Offloading to GPGPU Using Hyper-Q on Kepler Architecture. Takustr. 7, 14195 Berlin: ZIB; 2014. 14-19. [41] Zhang H, Li Y, Xiao W, Huang Y, Di X, Yin J, et al.: MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs. Available from: https://arxiv.org/abs/2301.00407. [42] Zhao W, Jayarajan A, Pekhimenko G. Tally: Non-Intrusive Performance Isolation for Concurrent Deep Learning Workloads. In: Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. ASPLOS ’25; 2025. p. 1052–1068. https: //doi.org/10.1145/3669940.3707282. [43] Wu B, Zhang Z, Bai Z, Liu X, Jin X. Transparent GPU Sharing in Container Clouds for Deep Learning Workloads. In: 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). Boston, MA: USENIX Association; 2023. p. 69–85. Available from: https://www.usenix.org/conference/ nsdi23/presentation/wu.
33