Exploiting Task-Based Parallelism for the Red-Black Gauss-Seidel Method on 2D Grids

Exploiting Task-Based Parallelism for the Red-Black Gauss-Seidel Method on 2D Grids Shiting Long1,∗ , Gustavo Ramirez-Hidalgo2, , Andreas Frommer3, , and Dirk Pleiter1,4, 1

KTH Royal Institute of Technology Forschungszentrum Jülich GmbH 3 Bergische Universität Wuppertal 4 University of Groningen

arXiv:2607.01735v1 [cs.DC] 2 Jul 2026

2

Gauss–Seidel is a well-established iterative method for the solution of linear systems, and multicoloring has been widely used to increase parallelism in iterative solution techniques. Implementing multi-color Gauss–Seidel with conventional divideand-conquer parallelization strategies, however, may be inefficient due to global synchronization requirements and load imbalances. Task-based programming models can mitigate these issues by enabling fine-grained parallelism, removing global barriers and allowing updates of different colors to partially overlap in time. In this work, we implement the red–black Gauss–Seidel method using two task-based programming models and compare them with a classical divide-and-conquer parallel implementation to evaluate the impact of fine-grained parallelism on execution efficiency. The red–black scheme serves as a representative example, as task-based approaches naturally extend to more general multi-color schemes arising from unstructured grids and wider stencils. Using the solve of the 2D Poisson equation as benchmark, our results show that task-based implementations can achieve performance comparable to conventional divide-and-conquer parallelization while providing greater resilience to hardware-level asynchronicity. Copyright line will be provided by the publisher

1

Introduction

Contemporary performance improvements are increasingly driven by parallel computing rather than higher CPU clock frequencies. As a result, achieving better performance depends on decomposing workloads into concurrent tasks, enabling efficient utilization of hardware resources and translating computational capacity into measurable speedups. Programmers are therefore required to exploit the performance potential of modern many-core processors through nodelevel and/or thread-level parallelism. Since parallelization cannot be handled entirely by compilers, applications must be adapted to these architectures by explicitly defining task granularity for distributing computation across cores, as well as synchronization mechanisms to ensure data consistency among parallel tasks. The de facto parallelization approach for numerical applications is to decompose or transform the problem into smaller partitions, where solving each partition constitutes a task that can be executed in parallel with the others. This divide-andconquer strategy maps naturally onto the fork-join multithreading programming model, enabling efficient utilization of manycore systems and achieving parallel speedup. The divide-and-conquer strategy is not suitable for all algorithms. One example is block LU factorization, where the number of concurrently executable block updates varies throughout execution. When using a fork-join model, threads may spend significant time waiting at synchronization barriers, resulting in hardware underutilization. In such cases, a programming model that supports asynchronous execution and dynamic work scheduling becomes essential. Furthermore, modern computing systems are increasingly heterogeneous, typically combining CPUs with accelerators such as GPUs. A programming model that enables workloads to be efficiently distributed across different hardware architectures is therefore important. Task-based programming models address these challenges by embedding dynamic schedulers within the runtime system, allowing tasks, i.e., units of work that can be executed in parallel, to be automatically mapped onto the underlying hardware resources. Such models have proven effective in addressing both asynchronous scheduling [3, 14] and heterogeneous scheduling [1, 2] challenges in the optimization of numerical applications. In addition to the asynchronicity inherent in certain numerical algorithms, hardware resource contention and operating system management introduce non-deterministic latencies in many-core systems [16, 17]. Consequently, even in a simple parallel program, threads may require different amounts of time to complete the same workload. Such variability can lead to excessive waiting in the fork-join model, whereas task-based programming models can better tolerate and mitigate these effects. In this paper, we investigate the effectiveness of task-based programming models in coping with the asynchronicity introduced by many-core systems. As a benchmark application, we consider the solve of the 2D Poisson equation using the red-black Gauss–Seidel method. This benchmark is simple yet representative of a broad class of stencil applications commonly found in scientific computing. Although it is highly homogeneous from an algorithmic perspective, its execution on ∗

Corresponding author: e-mail [email protected]

2

S. Long et al.: Task-Based Parallelism for Red-Black Gauss-Seidel on 2D Grids

hardware is not expected to be perfectly regular, as variability is introduced by the machine. We evaluate two task-based implementations using two programming models, namely OpenMP implemented in the GCC toolchain (hereafter referred to simply as OpenMP) and OmpSs-2 [22], and compare them against a conventional fork-join implementation based on the OpenMP parallel for construct. The evaluation is conducted on two systems: JUWELS (x86) and HAICGU (Arm). Previous studies on task-based stencil applications [10, 15] have generally concluded that task-based approaches provide limited benefits for regular workloads and are therefore less attractive than conventional fork-join parallelization strategies. In contrast, our results show that task-based models can remain competitive for regular stencil applications, particularly on modern many-core systems where architectural asynchronicity becomes increasingly important. In particular, the OmpSs-2 implementation, benefiting from its NUMA-aware support [20], outperforms the OpenMP tasking implementation (hereafter referred to as OpenMP task) and achieves performance comparable to the OpenMP parallel for implementation. Moreover, on HAICGU, which features a large number of cores and NUMA (non-uniform memory access) domains, OmpSs-2 delivers both higher and more stable performance than OpenMP parallel for. The remainder of this paper is organized as follows. Section 2 introduces the benchmark implementations, Section 3 describes the experimental setup, Section 4 presents the performance results and analysis, and Section 5 concludes the paper with a summary and directions for future work.

2

Benchmark Design and Parallelization

The 2D Poisson equation is an elliptic partial differential equation (PDE) widely used in physics and engineering (e.g., electrostatics, fluid dynamics, and heat conduction). It is also a well-studied kernel in the context of high-performance computing (HPC), as it can exemplify behaviors of a wide range of stencil applications. The equation with Dirichlet boundary conditions can be expressed as ∇2 u =

∂2u ∂2u + 2 = f (x, y) for (x, y) ∈ Ω, u(x, y) = g(x, y) for (x, y) ∈ ∂Ω, ∂x2 ∂y

where Ω is a suitable domain. To solve this equation numerically, we discretize Ω into a uniform grid with spacing ∆x and ∆y. For simplicity, we assume a square grid where ∆x = ∆y = h = N1+1 . The continuous function u(x, y) is approximated at discrete grid points as ui,j = u(i∆x, j∆y). Applying the central difference approximation to the second derivatives yields the standard 5-point stencil: ui,j =

1 ui+1,j + ui−1,j + ui,j+1 + ui,j−1 − h2 fi,j , i, j = 1, . . . , N, 4

(1)

where ui±1,j , ui,j±1 take the boundary values g((i ± 1)∆x, j∆y), g(i∆x, (j ± 1)∆y) if i ± 1, j ± 1 ∈ {0, N + 1}. The resulting linear system can be solved iteratively using the Gauss–Seidel method, where each grid point is updated using the most recently available values of its neighboring points. To expose parallelism, we employ the red-black Gauss–Seidel variant, which partitions the grid into two sets of points, labeled red and black, following a checkerboard pattern. Since points of the same color do not directly depend on one another, all red or all black points can be updated concurrently without data conflicts. In this paper, grid points satisfying i + j even are labeled red, while the remaining points are labeled black. In the following, we first present a simple OpenMP parallel for implementation to introduce the iterative method. We then transition to an overview of task-based programming, exploring the historical evolution, core motivations, and various models that define this paradigm. Finally, we present our task-based implementations of the benchmark. 2.1

OpenMP Parallel For Implementation

We show our parallel for implementation of the red-black Gauss–Seidel method for solving the 2D Poisson equation in Algorithm 1. We consider a square grid of size n × n with Dirichlet boundary conditions. We store the values associated with red and black sites separately. Specifically, the value of ui,j is stored in grid_red when i + j is even, and in grid_black otherwise. The same storage scheme is used for the source function f . Since this implementation follows the fork-join model, it relies on bulk synchronization through barriers. The two for constructs in Lines 3 and 8 each introduce an implicit barrier at the end of their execution regions. These barriers ensure that all red-site updates are completed before the computation proceeds to the black-site updates, and repeating across all iterations. 2.2

Task-Based Programming

An early work on task-based programming was introduced in Cilk [9], where a program was considered a collection of procedures. The procedure, later termed task, is a sequence of instructions that can be executed in parallel and managed by a scheduler to optimize performance on multiprocessor systems. This approach to parallelism can handle irregular control Copyright line will be provided by the publisher

Preprint

3

Algorithm 1: Parallel For Red-Black Gauss-Seidel Require: grid_red, grid_black, f_red, f_black, h, n, num_iterations Ensure : grid_red, grid_black (Converged numerical solution) #pragma omp parallel 2 for it = 0; it < num_iterations − 1; it + + do /* STEP 1: Update red sites */ 3 #pragma omp for 4 for x = 1; x < n − 1; x + + do 5 y_start ← (x%2 == 0)?2 : 1; 6 for y = y_start; y < n − 1; y+ = 2 do 7 grid_redx,y ← 14 (grid_blackx+1,y + grid_blackx−1,y + grid_blackx,y+1 + grid_blackx,y−1 − h2 f_redx,y ); 1

8 9 10 11 12

/* STEP 2: Update black sites */ #pragma omp for for x = 1; x < n − 1; x + + do y_start ← (x%2 == 0)?1 : 2; for y = y_start; y < n − 1; y+ = 2 do grid_blackx,y ← 41 (grid_redx+1,y + grid_redx−1,y + grid_redx,y+1 + grid_redx,y−1 − h2 f_blackx,y );

Algorithm 2: OpenMP Task Red-Black Gauss-Seidel Require: grid_red, grid_black, bs (block size), nb (number of blocks), num_iterations Ensure : grid_red, grid_black #pragma omp parallel; 2 #pragma omp single; 3 for it = 0; it < num_iterations; it + + do 4 update_color(grid_red, grid_black, bs, nb, . . . ) ; 5 update_color(grid_black, grid_red, bs, nb, . . . ) ; 1

6

7

8

9 10

11

12

13

// update red // update black

Function update_color(grid_self, grid_neighbor, bs, nb , . . . ) is /* STEP 1: Update boundary block 0 #pragma omp task depend(inout: grid_self[0]) depend(in: grid_neighbor[bs]); update_block(grid_self[0], grid_neighbor[0], NULL, grid_neighbor[bs], . . . );

*/ grid_neighbor[0],

/* STEP 2: Update inner blocks */ for b = 1; b < nb − 1; b + + do #pragma omp task depend(inout: grid_self[b*bs]) depend(in: grid_neighbor[b*bs], grid_neighbor[(b-1)*bs], grid_neighbor[(b+1)*bs]); update_block(grid_self[b∗bs], grid_neighbor[b∗bs], grid_neighbor[(b−1)∗bs], grid_neighbor[(b+1)∗bs], . . . ); /* STEP 3: Update boundary block nb-1 */ #pragma omp task depend(inout: grid_self[(nb-1)*bs]) depend(in: grid_neighbor[(nb-1)*bs], grid_neighbor[(nb-2)*bs]); update_block(grid_self[(nb − 1) ∗ bs], grid_neighbor[(nb − 1) ∗ bs], grid_neighbor[(nb − 2) ∗ bs], NULL, . . . ); /* update_block parameters mapping: /* 1st: Target block (current color) /* 2nd: Target block (opposite color) /* 3rd: Top neighbor block (opposite color) /* 4th: Bottom neighbor block (opposite color)

*/ */ */ */ */

Copyright line will be provided by the publisher

4

S. Long et al.: Task-Based Parallelism for Red-Black Gauss-Seidel on 2D Grids

Fig. 1: Illustration of the 2D grid data decomposition.

structures such as while-loops and recursive functions, as opposed to standardized for-loop parallelism supported by OpenMP in the 1990s. Several prominent task-based programming models emerged in the 2000s, including Intel Threading Building Blocks (TBB) [23] and OpenMP with its tasking extensions [5]. Later, the HPX C++ library was introduced [18], further expanding the ecosystem of task-based programming. Task-based programming models were subsequently extended beyond shared-memory systems. StarPU was among the pioneering task-based runtime systems that provided an efficient interface to execute parallel tasks across heterogeneous hardware architectures [4]. OmpSs introduced a task-based programming model supporting both homogeneous and heterogeneous systems [12], adopting a design philosophy akin to OpenMP’s approach to shared-memory parallelism. Subsequent enhancements extended OmpSs to incorporate GPU acceleration [13]. This model has recently been developed into OmpSs-2. In this paper, we evaluate the impact of the task-based execution on the red-black Gauss–Seidel method using two taskbased programming models: OpenMP and OmpSs-2. These models are chosen because they follow a directive-based programming approach, where parallelism is expressed through compiler directives (pragmas) embedded in the source code rather than through explicit thread management. Although this work focuses on native task-based programming models, similar asynchronous execution strategies can also be implemented using distributed-memory approaches such as MPI. In MPI-based implementations, workloads are distributed across ranks while synchronization and communication between neighboring ranks are managed explicitly by the programmer. Compared with task-based programming models considered here, this approach generally requires substantially more programming effort, including manual management of synchronization and memory buffers.

2.3

OpenMP Task and OmpSs-2 Implementations

A task-based implementation of the 2D Poisson solver using red-black Gauss-Seidel requires defining tasks as units of work to be executed in parallel. To achieve better control over task granularity, a common approach is to decompose the data. For simplicity, we partition the 2D grid along one dimension, resulting in horizontal stripes, as illustrated in Fig. 1. In our task-based implementations, updating all red or black sites within a block constitutes a task, and we treat the number of blocks (nb) as an input. Thus, a total of 2 · nb tasks are issued per iteration. Consider Fig. 1 as an example, where the grid size is 6 × 6 and we set nb = 3; hence, 6 tasks are issued in each iteration of the iterative method. Algorithm 2 illustrates our OpenMP task implementation. The core function, update_color(), updates sites of a given color within a block. Data dependencies are specified to constrain task execution. In particular, an update requires read and write access to the block of its own color, as well as read access to the same block and the neighboring blocks of the opposite color. These dependencies are enforced through the depend clauses in Lines 7, 10, and 12. The implementation details of the update_block() function are omitted from the algorithm for brevity, as it simply applies (1) to all red or black sites within the block. Task-based programming models with data dependency support, such as OpenMP, enforces execution order through data tracking. More specifically, tasks that exhibit conflicting dependencies, such as read-after-write (RAW), write-after-read (WAR), or write-after-write (WAW) on the same data region, are serialized by the runtime system to preserve correctness. Hence, the correctness of Algorithm 2 is ensured by issuing all red tasks first, followed by all black tasks (Lines 1-5). It is also worth noting that a single construct is used so that task generation is performed by only one thread. The OmpSs-2 implementation is similar to that of OpenMP. Changes are required for each pragma, i.e., from #pragma omp to #pragma oss. OmpSs-2 initiates a thread pool at the beginning of the program; therefore it does not need a parallel construct. By default, one thread executes the main function for task creation, and all other runtime threads can be used for task execution. A taskwait construct is needed at the end of the iterations (i.e., after Line 5) to enforce a barrier so that all tasks are finished before exiting the region of interest (ROI). Copyright line will be provided by the publisher

Preprint

5

Table 1: Node-level hardware and software for benchmarking platforms.

Architecture Processor Number of Cores Number of NUMA domains CPU Frequency L1 Data Cache L2 Cache L3 Cache SIMD Width Memory Theor. Mem. Bandwidth Meas. Mem. Bandwidth Roofline Ridge Point Compiler Used

3

JUWELS

HAICGU

x86 2×Xeon Platinum 8168 2 × 24 2 2.7 GHz 32 KiB per core 1 MiB per core 33 MiB per CPU 512 bit 96 GiB 256 GB s−1 155 GB s−1 16.2 FLOP B−1 GCC v13.3.0

Arm 2×Kunpeng 920 2 × 64 4 2.6 GHz 64 KiB per core 512 KiB per core 32 MiB per CPU 128 bit 128 GiB 341 GB s−1 218 GB s−1 1.96 FLOP B−1 GCC v14.1.0

Experimental Setup

Before presenting the evaluation of the performance of our implementations on hardware, we outline the test setup on JUWELS (x86) and HAICGU (Arm). The details of the hardware and software used on the two platforms are listed in Table 1. We evaluate two task-based programming models: OmpSs-2 with the Nanos6 [7] v4.3 runtime and GCC OpenMP (GCC versions listed in Table 1). In addition to the theoretical memory bandwidth, we use the STREAM benchmark [21] v5.10 to measure the peak attainable memory bandwidth. The values reported in Table 1 are obtained using the maximum available cores. Specifically, we use the STREAM Triad rates, which typically reflect the highest sustainable bandwidth on modern architectures. To assess the attainable performance of the benchmark, we employ the roofline model [24]. The roofline model characterizes application performance in terms of its arithmetic intensity, Ifp /Imem , where Ifp and Imem denote the floating-point operation count and memory traffic, respectively. For the benchmark considered in this work, the arithmetic intensity is approximately 0.29. Applications with low arithmetic intensity are memory-bound, meaning their performance is limited by memory bandwidth, whereas applications with high arithmetic intensity are compute-bound and limited by floating-point throughput. The transition between these two regimes occurs at the ridge point of the roofline model, an arithmetic-intensity threshold defined as the ratio of peak floating-point performance to peak memory bandwidth. Since the arithmetic intensity of the benchmark lies well below the ridge points of both platforms listed in Table 1, the benchmark is firmly memory-bound. Consequently, its attainable performance can be estimated from the measured memory bandwidth as Att. Perf. = Meas. Mem. Bandwidth ×

Ifp Imem

.

(2)

We consider single-node execution, as OpenMP does not natively support distributed-memory parallelism. For distributed computing, MPI remains the de facto standard, and both programming models support a hybrid MPI+tasking approach. We also focus on CPU architectures. All performance results reported in this paper are obtained by executing the ROI ten times and recording the median value to reduce the impact of statistical noise and system interference. For simplicity, we use a 1:1 mapping between physical cores and runtime threads, and all available cores are used for all experiments. For OpenMP, thread affinity is configured using OMP_PROC_BIND=spread and OMP_PLACES=cores. For OmpSs-2, thread pinning is performed using taskset, and NUMA tracking is enabled for the Nanos6 runtime.

4

Evaluation

We compare the implementations introduced in Section 2, namely the OpenMP parallel for baseline and the two taskbased implementations: OpenMP task and OmpSs-2. Their performance results, together with the peak attainable performance derived from (2), are presented in Fig. 2. We consider three problem sizes, each sufficiently large to ensure memory-bound execution, and fix the number of red-black Gauss-Seidel iterations to 100. Copyright line will be provided by the publisher

6

S. Long et al.: Task-Based Parallelism for Red-Black Gauss-Seidel on 2D Grids

Fig. 2: Performance (median) of red-black Gauss-Seidel with varying matrix size, 100 iterations are taken.

Fig. 3: Box-and-whisker plots (five-number summary: min, first quartile, median, third quartile, and max) of performance.

Compared with the OpenMP parallel for implementation, the task-based implementations introduce an additional tuning parameter, namely nb in Algorithm 2, which controls task granularity. A small number of tasks may lead to hardware underutilization, while an excessively large number of tasks can incur significant runtime overhead. Through empirical analysis, we determine the task counts that achieve optimal performance, and report them in Fig. 2. The superior performance of OmpSs-2 on both architectures is attributable to its NUMA-aware support [20]. NUMA is a widely used memory architecture in HPC systems that enables larger core counts and memory capacities by partitioning memory into multiple NUMA domains. However, this design introduces differences in memory access latency and bandwidth between local and remote NUMA domains. Modern operating systems are aware of the NUMA topology and typically follow a first-touch policy, allocating memory in the NUMA domain of the thread that first accesses it. Consequently, if threads executing tasks are not carefully scheduled on the NUMA domains where the corresponding data reside, remote memory accesses may occur, resulting in higher memory transfer costs and degraded performance. Working with operating systems’ first-touch policy is relatively straightforward with OpenMP parallel for: the data can simply be initialized using another parallel for with the same access pattern as the subsequent computation. OmpSs2 provides low-level memory allocation APIs that allow data to be distributed across NUMA domains in an interleaved manner, effectively mitigating the first-touch policy. Its task scheduler can also place tasks on the NUMA domains where their data reside through data tracking mechanisms. OpenMP task, however, does not provide such NUMA-aware support. Without enabling the NUMA-aware features of OmpSs-2, we observe performance degradations of approximately 40% on JUWELS and 43% on HAICGU. The NUMA effect is expected to be more pronounced on HAICGU than on JUWELS. The Kunpeng 920 processor used in HAICGU consists of four NUMA domains, within each of which the cores are organized in a ring topology. This design likely incurs higher synchronization and communication costs between cores compared with the Intel Xeon Platinum 8168 used in JUWELS, which connects its cores via a two-dimensional mesh on-chip network and has two NUMA domains. When data placement does not match the data access pattern (here achieved by using numactl [19] to interleave memory allocation), we observe that the STREAM benchmark reports bandwidth reductions of 47% and 55% relative to the values listed in Table 1 for JUWELS and HAICGU, respectively. OmpSs-2 achieves performance comparable to OpenMP parallel for on JUWELS, and outperforms it on HAICGU. Furthermore, for large problem sizes on HAICGU, parallel for exhibits greater execution variability than OmpSs-2, as shown in Fig. 3. Although some runs of OpenMP parallel for can achieve performance similar to OmpSs-2 on HAICGU, its overall performance is less stable. Copyright line will be provided by the publisher

Preprint

7

(a) First 10 iterations

(b) Entire 100 iterations

Fig. 4: Mapping of OpenMP execution to CPU cores on HAICGU (n = 30 000).

(a) JUWELS (nb = 300)

(b) HAICGU (nb = 400)

Fig. 5: Mapping of OmpSs-2 execution of the first 10 iterations to CPU cores on HAICGU (n = 30 000).

Such instability may arise from varying execution latencies that force threads to wait at barriers. These latencies can be caused by operating system noise or resource contention, where threads compete for shared resources. They are more likely to occur on the Kunpeng 920 processor of HAICGU as it features substantially more cores and NUMA domains. To further investigate this behavior, we use Extrae [6] to trace the OpenMP parallel for execution and Ovni [11] to trace the OmpSs-2 execution. We further use Paraver [8] to extract the execution traces. We plot the first 10 iterations of an OpenMP parallel for execution in Fig. 4(a). A clear separation between phases corresponding to different color updates can be observed, as all threads begin executing the same amount of work simultaneously, and the next phase starts only after all threads have completed their work. Execution variability is also evident, with some threads taking noticeably longer to finish than others, leaving the remaining threads idle and making the gaps between phases more pronounced. These gaps persist throughout the execution and are shown in blue in Fig. 4(b). Such clear separation between phases is not required in task-based implementations, as they do not rely on rigid synchronization between all red and all black updates. Overlaps between red and black phases, as well as between successive iterations, can be observed in the OmpSs-2 execution traces shown in Fig. 5. The threads remain continuously busy throughout execution. Note that the gaps in Fig. 5(b) for cores 96–128 do not indicate idle cores; rather, they correspond to tasks from later iterations being executed. This ability to hide execution variability through dynamic scheduling explains the higher and more stable performance of OmpSs-2 over OpenMP parallel for on HAICGU. Another potential source of performance differences is cache reuse. While a parallel for implementation follows a regular work distribution, tasks in a task-based implementation are scheduled dynamically by the runtime system. Consequently, the resulting data access pattern depends on task granularity, scheduling decisions, and hardware characteristics, leading to different cache locality properties. However, with the problem size and task granularity used in this work, each task accesses a sufficiently large amount of data that the relevant cache lines are unlikely to remain in cache until a core finishes its current task and is assigned another task by the runtime. We therefore do not expect the cache reuse of the task-based implementation to differ significantly from that of OpenMP parallel for. This is confirmed by hardware performance counter measurements on JUWELS1, which show nearly identical cache refill rates for OmpSs-2 and OpenMP parallel for. This is also consistent with the small performance difference observed between the two implementations on JUWELS. 1 L3 cache data-read misses and total data-read requests are obtained using the OFFCORE_RESPONSE counter with request=ALL_DATA_RD and the filters L3_MISS and ANY_RESPONSE. L2 cache data-read misses and total data-read requests are obtained using the L2_RQSTS counter, which includes information on both on-demand and hardware prefetcher data reads. Copyright line will be provided by the publisher

8

S. Long et al.: Task-Based Parallelism for Red-Black Gauss-Seidel on 2D Grids

5

Conclusion

Although task-based paradigms have been widely introduced in numerical computing, previous studies have primarily focused on their ability to parallelize irregular workloads for performance optimization. In this work, we analyzed task-based implementations of the 2D Poisson solver using the red-black Gauss-Seidel method, and compared them against a conventional OpenMP parallel for implementation. Our results show that the additional implementation complexity of the task-based approach is modest and remains comparable to that of parallel for. Transitioning from a parallel for implementation to either OpenMP task or OmpSs-2 requires relatively little programming effort, particularly when compared with achieving a similar fine-grained parallelism using MPI. Through experimental evaluation, we found that the best-performing task-based variant, OmpSs-2, achieves performance comparable to or better than parallel for. On HAICGU, OmpSs-2 delivers both higher and more stable performance. Its dynamic scheduling mechanism mitigates the impact of execution-time variability by keeping hardware resources occupied and reducing the synchronization overhead associated with the fork-join model. These results demonstrate that task-based approaches can effectively tolerate architectural asynchronicity, making them a competitive option even for regular workloads. Future work includes extending this study from the simple 2D Poisson problem to more complex systems involving higher dimensions and more sophisticated computations. In particular, multi-color variants of Gauss-Seidel introduce more synchronization requirements, potentially increasing the benefits of task-based execution. It would also be interesting to investigate other iterative methods with behavior similar to red-black Gauss-Seidel, such as the odd-even techniques commonly used in scientific computing for various fields of study. Acknowledgements The authors gratefully acknowledge the Gauss Centre for Supercomputing e. V. for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS at Jülich Supercomputing Centre (JSC), under the project with id MUL-TRA. Furthermore, we want to thank the Open Edge and HPC Initiative for access to an Arm-based development environment through the HAICGU cluster at the Goethe University of Frankfurt. Funding for parts of this work has been received from the European Union’s HORIZON MSCA Doctoral Networks programme AQTIVATE, under grant agreement No. 101072344. G.R-H. acknowledges financial support from the EoCoE-III project, which has received funding from the European High Performance Computing Joint Undertaking under grant agreement No. 101144014.

References [1] Emmanuel Agullo, Cédric Augonnet, Jack Dongarra, Hatem Ltaief, Raymond Namyst, Samuel Thibault, and Stanimire Tomov. A hybridization methodology for high-performance linear algebra software for GPUs. In GPU Computing Gems Jade Edition, pages 473–484. Elsevier, 2012. [2] Emmanuel Agullo, Olivier Aumage, Mathieu Faverge, Nathalie Furmento, Florent Pruvost, Marc Sergent, and Samuel Paul Thibault. Achieving high performance on supercomputers with a sequential task-based programming model. IEEE Transactions on Parallel and Distributed Systems, 2017. [3] Emmanuel Agullo, Bérenger Bramas, Olivier Coulaud, Eric Darve, Matthias Messner, and Toru Takahashi. Task-based FMM for multicore architectures. SIAM Journal on Scientific Computing, 36(1):C66–C93, 2014. [4] Cédric Augonnet, Samuel Thibault, Raymond Namyst, and Pierre-André Wacrenier. StarPU: a unified platform for task scheduling on heterogeneous multicore architectures. In Euro-Par 2009 Parallel Processing: 15th International Euro-Par Conference, Delft, the Netherlands, August 25-28, 2009. Proceedings 15, pages 863–874. Springer, 2009. [5] Eduard Ayguadé, Nawal Copty, Alejandro Duran, Jay Hoeflinger, Yuan Lin, Federico Massaioli, Xavier Teruel, Priya Unnikrishnan, and Guansong Zhang. The design of OpenMP tasks. IEEE Transactions on Parallel and Distributed systems, 20(3):404–418, 2008. [6] Barcelona Supercomputing Center. Extrae: Instrumentation framework for performance analysis. Software available from tools.bsc.es, 2026. Available at https://tools.bsc.es/extrae. [7] Barcelona Supercomputing Center. Nanos6 Runtime System for OmpSs-2. Software available from github.com/bsc-pm/nanos6, 2026. Available at https://github.com/bsc-pm/nanos6. [8] Barcelona Supercomputing Center (BSC). Paraver: A Trace-Based Performance Analysis Tool. Software and Documentation Website, 2026. [9] Robert D Blumofe, Christopher F Joerg, Bradley C Kuszmaul, Charles E Leiserson, Keith H Randall, and Yuli Zhou. Cilk: An efficient multithreaded runtime system. ACM SigPlan Notices, 30(8):207–216, 1995. [10] Lionel Boillot, George Bosilca, Emmanuel Agullo, and Henri Calandra. Task-based programming for seismic imaging: Preliminary results. In 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC, CSS, ICESS), pages 1259–1266. IEEE, 2014. [11] BSC-PM (Barcelona Supercomputing Center - Programming Models). OVNI: Obtuse but Versatile Nanoscale Instrumentation. GitHub Repository, 2026. [12] Javier Bueno, Luis Martinell, Alejandro Duran, Montse Farreras, Xavier Martorell, Rosa M Badia, Eduard Ayguade, and Jesús Labarta. Productive cluster programming with OmpSs. In Euro-Par 2011 Parallel Processing: 17th International Conference, Euro-Par 2011, Bordeaux, France, August 29-September 2, 2011, Proceedings, Part I 17, pages 555–566. Springer, 2011. Copyright line will be provided by the publisher

Preprint

9

[13] Javier Bueno, Judit Planas, Alejandro Duran, Rosa M Badia, Xavier Martorell, Eduard Ayguade, and Jesús Labarta. Productive programming of GPU clusters with OmpSs. In 2012 IEEE 26th International Parallel and Distributed Processing Symposium, pages 557–568. IEEE, 2012. [14] Alfredo Buttari, Julien Langou, Jakub Kurzak, and Jack Dongarra. A class of parallel tiled linear algebra algorithms for multicore architectures. Parallel Computing, 35(1):38–53, 2009. [15] Thomas Heller, Hartmut Kaiser, and Klaus Iglberger. Application of the parallex execution model to stencil-based problems. Computer Science-Research and Development, 28(2):253–261, 2013. [16] Torsten Hoefler, Timo Schneider, and Andrew Lumsdaine. Characterizing the influence of system noise on large-scale applications by simulation. In SC’10: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–11. IEEE, 2010. [17] Robert Hood, Haoqiang Jin, Piyush Mehrotra, Johnny Chang, Jahed Djomehri, Sharad Gavali, Dennis Jespersen, Kenichi Taylor, and Rupak Biswas. Performance impact of resource contention in multicore systems. In 2010 IEEE international symposium on parallel & distributed processing (IPDPS), pages 1–12. IEEE, 2010. [18] Hartmut Kaiser, Thomas Heller, Bryce Adelstein-Lelbach, Adrian Serio, and Dietmar Fey. Hpx: A task based programming model in a global address space. In Proceedings of the 8th international conference on partitioned global address space programming models, pages 1–11, 2014. [19] Andi Kleen. numactl(8) - Control NUMA policy for processes or shared memory. Linux Man Pages, 2005. Accessed: 2024-05-07. [20] Marcos Maroñas, Antoni Navarro, Eduard Ayguadé, and Vicenç Beltran. Mitigating the NUMA effect on task-based runtime systems. The Journal of Supercomputing, 79(13):14287–14312, 2023. [21] John D McCalpin et al. Memory bandwidth and machine balance in current high performance computers. IEEE computer society technical committee on computer architecture (TCCA) newsletter, 2(19-25), 1995. [22] Josep M Perez, Vicenç Beltran, Jesus Labarta, and Eduard Ayguadé. Improving the integration of task nesting and dependencies in OpenMP. In 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 809–818. IEEE, 2017. [23] Thomas Willhalm and Nicolae Popovici. Putting intel® threading building blocks to work. In Proceedings of the 1st international workshop on Multicore software engineering, pages 3–4, 2008. [24] Samuel Williams, Andrew Waterman, and David Patterson. Roofline: an insightful visual performance model for multicore architectures. Communications of the ACM, 52(4):65–76, 2009.

Copyright line will be provided by the publisher