NVIDIA Vera: A Microarchitecture-Level Comparison with Grace, Turin, and Granite Rapids

arXiv:2611.15926v1 [cs.DC] 11 Nov 2026

NVIDIA Vera: A Microarchitecture-Level Comparison with Grace, Turin, and Granite Rapids Joseph Moore

Thomas Green

Simon McIntosh-Smith

University of Bristol Bristol, United Kingdom joseph.moore@bristol.ac.uk

University of Bristol Bristol, United Kingdom thomas.green@bristol.ac.uk

University of Bristol Bristol, United Kingdom s.mcintosh-smith@bristol.ac.uk

Abstract—The majority of new HPC systems are centered around NVIDIA, AMD, or Intel CPUs. Vera is NVIDIA’s latest data-center CPU, including its first custom data-center core design, Olympus. This paper investigates Vera in comparison to AMD Turin and Intel Granite Rapids, using NVIDIA Grace as a baseline. We navigate up the performance ladder, from microbenchmarks that isolate performance metrics, through NAS kernels and miniapps, to single-socket performance of a range of scientific codes. We find that Vera achieves the best performance of the architectures evaluated in most codes tested, and that this gain is primarily attributed to its high memory bandwidth, effective Spatial Multithreading implementation, and low-latency memory subsystem aided by the architecture’s predictive structures. While Vera represents a major leap over its predecessor, we show that its comparatively modest floating-point performance trails Intel and AMD in a small number of compute-bound workloads reflecting NVIDIA’s GPU focus for such workloads. Index Terms—Computer architecture, high performance computing, performance analysis, parallel processing

I. I NTRODUCTION AND RELATED WORK CPUs compatible with the Arm ISA are no longer a novelty within HPC – as of June 2026, four of the top ten supercomputers support the Arm ISA, two of which have the NVIDIA Grace-Hopper superchip [1]. Instruction set architecture does not determine performance in isolation; careful design of the core architecture, memory subsystem, and die topology is required to advance performance of scientific applications on CPUs. Isolating the contribution of each requires a multi-level evaluation of microbenchmarks, synthetic kernels, and realworld applications. NVIDIA Vera is the latest data-center CPU released by NVIDIA following on from their first data-center CPU release, Grace. Vera has a monolithic compute die of 88 “Olympus” cores compatible with the Armv9.2-A ISA, the first data-center CPU core designed by NVIDIA. This work analyzes the performance of a single socket of the next-generation Vera platform in comparison to currentgeneration CPU architectures: NVIDIA Grace, AMD Turin, and Intel Granite Rapids. We find that Vera achieves a geometric mean of 1.89 × improved performance compared to Grace across a range of HPC workloads, and outperforms the 96-core EPYC 9R45 (Turin) and Xeon 6975P-C (Granite Rapids) in the majority of codes tested. We attribute much of this speedup to advancements in the following microarchitecture structures:

The highest sustained memory bandwidth of the platforms evaluated at 1088.2 GB/s. • Lowest average latencies in both cache access and coreto-core communication. • Spatial Multithreading that accelerates select workloads by maximizing core utilization. • Prefetchers with the ability to target different patterns of irregular memory access. •

CPU architecture comparison studies are often motivated by integration into a large HPC system [2], [3], [4], which primarily target scientific workloads or multi-node scaling performance as their main figure of merit. We take inspiration from these but extend the depth of synthetic benchmarking used, tying observed microarchitectural advancements to the performance of scientific workloads. Other works instead analyze a CPU’s single-node performance more narrowly, either its microarchitecture with performance models [5], its power and energy efficiency [6], or a single performance factor of a specific application [7]. We instead gather a holistic view of Vera, comparing performance against Grace, Turin, and Granite Rapids on a range of scientific codes, microbenchmarks, the NAS Parallel Benchmarks, and miniapps, connecting our results to understand why Vera performs as it does. The primary contributions of our work are as follows: The first independent microarchitectural characterization of Vera for HPC workloads to the best of our knowledge. • A methodology for isolating the performance of microarchitectural features to support a holistic comparison of CPUs from different vendors. • Validation of synthetic performance benchmarks against a representative suite of real-world scientific applications. •

The remainder of this paper is structured as follows: We introduce the Olympus core microarchitecture in Section II and our system setup in Section III; Section IV evaluates results of microbenchmarks that highlight the influence of the microarchitectural design; Section V analyzes performance and scaling behavior in suites of synthetic kernels or miniapps that serve as a proxy to a wide range of HPC codes. In Section V-C we compare the achieved performance on a representative set of real-world scientific applications. Finally, we close the paper in Section VI with our conclusions and direction for future work.

© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Accepted for publication in the SC26 Workshops (PMBS26), November 15–20, 2026, Chicago, Illinois, USA.

II. A RCHITECTURE OVERVIEW Olympus core (88x per socket) Instruction TLB

Branch Predictor

64 entry, fully associative

2 taken br / cycle

L1 Instruction Cache 64 KB, 4-way

Fetch - 16 inst / cycle Decode queue - 48 entries Decode - 10-way

Front-end

Rename / allocate / commit

Prefetchers

10 μops per cycle

Stream Indirect Correlated-miss Spatial Graph

Memory Value renaming prediction Rename / Dispatch

Move elimination

Unified Second-Level TLB

Out-of-order issue Integer 8x simple ALU 2x complex ALU 2x CRC / shift

3k entries

Branch 4x branch units

L2 Unified Cache 2 MB, 8-way

Data TLB

Vector

Address

6x 128-bit units FP64 / FP32 / FP8

4x load pipelines 2x store pipelines

Execution

112 entry, fully associative

L1 Data Cache 96 KB, 6-way On-core memory

L3 Unified Cache

LPDDR5X - 9600 MT/s

164 MB

32 channels, 32-bit, 1.2 TB/s per socket

Off-core

Fig. 1. Abstracted block diagram of the Olympus core microarchitecture, including additional information of the off-core memory subsystem.

Vera is a CPU with 88 custom Olympus cores on a monolithic compute die. Figure 1 provides an abstract architecture block diagram of the core pipeline. Olympus provides a wide front end relative to other architectures, with a fetch width of 16 instructions per cycle and 10way decode which can include fused instructions, backed by a branch predictor capable of predicting two taken branches per cycle. Grace’s Neoverse-V2 cores sustain six micro-ops per cycle [8], Intel’s Redwood Cove can fetch 32 bytes per cycle and also sustains six micro-ops per cycle [9], while AMD’s Zen 5 can provide 64 bytes per cycle aggregated bandwidth via two ports and can sustain eight micro-ops per cycle [10]. Olympus supports Arm’s Scalable Vector Extension (SVE) and NEON vector instructions and has six 128-bit SVE units. Each core is equipped with a 96 KiB L1d, 64 KiB L1i, and 2 MiB unified L2 cache. All cores share a single 164 MiB unified L3 cache. Vera’s main memory is provided by LPDDR5X memory modules. NVIDIA discloses five classes of prefetcher on Olympus: Stream, Indirect, Correlated-Miss, Spatial, and Graph. Vera improves on the high main-memory bandwidth of Grace, extending Grace’s 32 16-bit memory channels running at a measured 8532 MT/s to Vera’s 32 32-bit channels running

at 9600 MT/s. This leads to a peak main-memory bandwidth bit of 9600 MT/s × 8 32 bit/byte × 32 channels = 1228.8 GB/s. This represents a 2.25 × increase over the Grace-Grace superchip’s theoretical per-socket peak bandwidth of 546.0 GB/s. Vera implements Simultaneous Multithreading (SMT) via Spatial Multithreading with two threads per core for a total of 176 threads. Spatial Multithreading partitions a core’s pipeline resources between threads physically. Intel’s HyperThreading [11] duplicates the architectural state and splits some buffers, such as the reorder buffer and the load/store buffers, between the two threads, but the threads share the execution units dynamically. Instead, Olympus splits nearly all pipeline resources between both threads, meaning that they mainly share the caches. This removes contention between threads for execute units, but also limits the pipeline width of each thread to half of the core’s width when both threads are active. Grace has no SMT implementation, while Turin does but it is not exposed as an option on the node tested in this work, hence limiting the generalization of our SMT analysis for Turin. For the remainder of this paper, “SMT on” is used to indicate that two software threads are assigned per physical core, while “SMT off” is used to indicate only one thread per core. Olympus dynamically detects whether the core is in singlethreaded or multi-threaded mode depending on whether the second thread is idle. In single-threaded mode, the single thread has access to all core resources. Upon a switch, nearly all pipeline resources are halved. In order to avoid idle vector units where unnecessary, the option to have vector execution units only split when both threads are executing either SVE or NEON instructions is available in the BIOS. The branch prediction unit also splits between threads, with a dual ported unit easily separable into a single port per thread. III. E XPERIMENTAL S ETUP This paper benchmarks a single socket of the Vera CPU, comparing against the previous generation NVIDIA Grace CPU, as well as an AMD 5th-Generation “Turin” CPU and an Intel 6th-generation Xeon “Granite Rapids” CPU. We test an early access single-socket engineering sample of Vera that represents the Vera CPU dual-socket server product, so dualsocket or CPU-GPU performance analysis was not available at this time. The air-cooled engineering board was artificially power capped to achieve a 450 W TDP equivalent. We compare against NVIDIA Grace as part of the Isambard 3 supercomputer at the Bristol Centre for Supercomputing. Each of the 384 nodes is equipped with the NVIDIA Grace Superchip containing two Grace CPUs per node, connected with the NVIDIA NVLink-C2C interconnect. Each CPU has access to 120 GB of local LPDDR5X memory for a total of 240 GB of memory per node. As Isambard 3 has dual-socket Grace CPUs we take care throughout our benchmarking to pin all cores and memory allocations to a single socket. Our comparisons to Turin and Granite Rapids CPUs are performed on compute-optimized bare-metal AWS instances for both architectures. Both instances have cloud-specific SKUs

TABLE I S YSTEM CONFIGURATIONS OF CPU S EVALUATED . Cores / System Threads

OS / Kernel

NVIDIA Vera

88 / 176 72 / 72

Engineering Sample Isambard 3

96 / 96

AWS c8a. metal-24xl AWS c8i. metal-48xl

Ubuntu 24.04.3, 6.17.0-1017-nvidia-64k SLES 15 SP7, 6.4.0-150700. 53.66-64kb Ubuntu 24.04.4, 6.17.0-1017-aws Ubuntu 24.04.4, 6.17.0-1017-aws

NVIDIA Grace AMD EPYC 9R45 (Turin) Intel Xeon 6975P-C (Granite Rapids)

96 / 192

Triad bandwidth (GB/s)

CPU

1400

Grace (72c) Vera (88c)

1200 1000 800 600 400 200 0 0

with 96 physical cores per socket each. While both the Turin c8a.metal-24xl and Granite Rapids c8i.metal-48xl are singlesocket instances, the Intel instance has additional vCPUs due to two Hyper-Threads per core, while the Turin SKU has no multithreading. We once again take care to only assign one thread per physical core when we are not testing multithreaded behavior. These SKUs were chosen due to availability of hardware and similar core counts to Vera (88 versus 96). SKUs with greater core counts exist in both families, with more memory channels available in different Granite Rapids CPUs. Our results therefore only compare performance to the 9R45 and 6975P-C. Table I reports the system configurations used in testing. We use GCC 15.3.0 for all tests, and align OS and kernel versions where possible. IV. M ICROBENCHMARKS A. Sustained Memory Bandwidth We measure sustained memory bandwidth using the STREAM microbenchmark [12]. STREAM measures achieved memory bandwidth using four kernels - Copy, Scale, Add, and Triad, each performing a mix of loads, stores, and arithmetic. We utilize triad as our primary figure of merit as it best reflects the achievable bandwidth in codes performing arithmetic on loaded data as with many HPC codes. All arrays are sized according to the cache sizes of the platform STREAM is tested on, ensuring each array requires at least 4× the size of the last-level cache. For Vera, we use an array size of 183,000,000 which equates to 1.36 GiB per array. When running across all cores, we use OMP_PROC_BIND=spread and OMP_PLACES=cores with one thread per core. We do not observe that distributing thread placement leads to more performance when measuring memory bandwidth with all cores active, but we specify it for a deterministic, reproducible thread-to-core mapping. As we only want to compare to a single Vera socket while our Grace implementation is a dual-socket machine, on Grace we run with numactl --cpunodebind=0 --membind=0 to pin all threads to a single socket and allocate to the memory closest to that socket. Neither Vera

EPYC 9R45 (96c) Xeon 6975P-C (96c)

16

32 48 64 80 Cores (1 thread per core)

96

Fig. 2. Sustained STREAM triad memory bandwidth of different CPUs, scaling across the total number of physical cores. Core placement is spread in a stride of 8 with a round-robin pattern.

nor Grace have multiple NUMA domains per socket, and our Turin node is configured with a single NUMA domain per socket. Our Granite Rapids instance has three NUMA domains per socket and thus relies on first-touch allocation within STREAM’s initialization step. For scaling the number of cores, we increase the available pool of cores via taskset with a stride of 8 between logical CPU IDs. While distant logical cores do not guarantee distant physical cores, we find that it aids the scaling performance by spreading out cores faster. We choose 8 as this is the size of a single CCD in our Turin SKU, thus spreading cores evenly across CCDs. This also benefits Intel as it more quickly fills cores across each of its three compute chiplets. We find that compared to sequentially incrementing core ID that this does not have a performance impact on Vera or Grace, but has a significant scaling impact on Turin due to the CCDbased topology and on Intel with their coarser chiplet-based architecture. Where Turin and Granite Rapids require careful pinning across applications to maximize memory bandwidth, Vera provides this regardless of core placement. Figure 2 demonstrates that Vera achieves the highest sustained memory bandwidth at 1088.2 GB/s in an all-core run, reaching 88.6 % of the theoretical peak bandwidth. In comparison, Grace achieved 449.4 GB/s which is 82.3 % of its theoretical peak. We cannot calculate a fixed theoretical peak memory bandwidth for our Granite Rapids and Turin CPUs as it can depend on both the number of DIMMs as well as the MT/s rate of those DIMMs. In this case, the Intel Xeon 6975P-C achieved 552.6 GB/s while the AMD EPYC 9R45 achieved 332.1 GB/s. We measure that Vera had 1.97 × greater main-memory bandwidth than the next best CPU which was Granite Rapids. Notably, Vera’s per-core contribution of memory bandwidth at few cores is the largest out of the comparison architectures. At 12 cores, Vera achieves 725.5 GB/s – 2.54 × that of Grace, 2.24 × that of Turin, and 3.41 × that of Granite Rapids.

Access latency (ns)

L1d L1d L1d L1d

L2 L2 L2 L2

L3 L3 L3 L3

200 100 50 20 10 5 2 1 0.5 4K

64K

1M 16M 256M 4G Region size (bytes) Grace (72c) EPYC 9R45 (96c) Vera (88c) Xeon 6975P-C (96c)

Fig. 3. Measured time per dependent load (ns) across different region sizes, measured in a random-order pointer chasing sequence. 2 MiB huge pages enabled to reduce additional TLB latency. Dashed lines mark the L1d, L2 and L3 sizes of each CPU; where two CPUs have the same size, the line alternates color.

B. Cache latencies We measure the latency of memory accesses that correspond to the capacity of each level of cache through the testtlb benchmark [13]. This microbenchmark performs pointer chasing across a fixed size array, allowing the timing of a chain of indirect loads within a working set sized to fit into different levels of cache. One random permutation of memory accesses is repeated for at least 0.2 s. This setup is repeated for five trials with new permutations each time, with the minimum of these reported. These results are further averaged over multiple trials for most array sizes, with ten repeats for the shortest arrays and scaling down to one for the largest to balance experiment runtime. The random-access pattern is designed to circumvent stride prefetchers, while utilizing huge pages reduces TLB latency that would otherwise obscure our results. This benchmark is intended to be used to measure the latency of cache accesses, but many factors including prefetching, page sizes, and huge page allocation, can change the result from the true cache latency to one specific to this memory access pattern. We use a stride of 64 bytes, though as the array is randomly permuted, this instead sets the spacing between array elements. On Vera, we run this experiment with the recommended 64 KiB page size in addition to 2 MiB huge pages. We do not use the recommended 512 MiB huge pages on Vera to make a fair comparison with Grace, where we could not modify huge page sizing. Grace was also fixed to a 64 KiB page size and 2 MiB huge pages enabled, while the remaining CPUs had 4 KiB pages and 2 MiB huge pages.

Figure 3 compares the measured access latencies at different region sizes for each of our CPUs. We find that Vera achieves the lowest time per load at region sizes lower than 128 MiB, though we observe unexpected results for both Grace and Vera. The step function presented for Turin and Granite Rapids is what we would expect to see for a random-access pattern, with the latency of each access spiking once our region is larger than the previous layer of cache, before maintaining roughly the same latency. Instead, once the region size is greater than the size of L1d cache, the latency is lower than expected, increases gradually within each cache level, and does not spike sharply upon a cache boundary. Frumusanu observed a similar pattern in [14] for the CortexX1 and Cortex-A78 core designs between the L2 and L3 caches. They showed that this is only observable in repeated pointer-chasing paths and was the effect of the temporal prefetcher which is documented to exist in other Arm cores such as the Neoverse-N2 [15]. Grace’s Neoverse-V2 cores show the same behavior as these cores, which are confirmed to have a temporal prefetcher. The correlated-miss prefetcher that is confirmed to be present in Vera is an implementation of a temporal prefetcher [16]. This class of prefetcher differs from a stride prefetcher, which relies on fixed deltas between memory addresses, by exploiting recurrences in access patterns by recording the memory addresses in a chain of misses, allowing them to be replayed in the event of a recurrence. A temporal prefetcher can therefore achieve lower latency on a random-access memory pattern if the same memory address path is repeated across iterations. In test-tlb, a correlatedmiss prefetcher would be effective as the same permutation of memory accesses is repeated after one untimed warm-up run. On Vera, the time per load stays close to the L1d value beyond the L1d size, then rises gradually up to 2 MiB. The timing at the size of the L2 appears to expand beyond Grace’s lower latency region of 16 MiB to 32 MiB. A load that misses L2 cannot return in L2 hit time, so the cache line must have been prefetched. Note that unlike Grace, the effect of the correlated-miss prefetcher is observed beyond L1d sizes instead of only beyond L2. We could not disable individual prefetchers on Vera, so Vera’s true L1d and L2 hit latencies are not measured. Figure 3 therefore shows that up to a 128 MiB region, Vera has the lowest time per load of the four CPUs. Codes that have repeated irregular accesses, such as sparse iterative solvers or unstructured-mesh codes, may see lower effective latency compared to Turin or Granite Rapids which demonstrate no equivalent prefetching in this benchmark. While NVIDIA lists several prefetchers for different memory access patterns, if a pattern is not prefetchable or exceeds the size of the L3, this benefit is lost. Above 512 MiB, Vera’s main-memory latency is the largest out of the CPUs tested. C. Core-to-Core latency Core-to-core latency is measured through timing the cachecoherency protocol when one core writes to a shared cache

Figure 4 presents the measured latencies for all pairs of cores on each CPU, while Figure 5 provides further statistics on the latency distribution. While logical core IDs do not necessarily equate to fabric distance between pairs of cores, 32 the patterns of the results often imply fabric topology. We see 32 similar checkerboard patterns on NVIDIA Grace and Vera, implying that the latency is distance-based in a 2D mesh-like 64 fabric. Vera has a reduction of 18 % in 5th percentile, 17 % in 64 mean, 17 % in median, and a 13 % reduction in 95th percentile core-to-core latencies compared to Grace despite having 22 % 0 32 64 0 32 64 more cores, implying a larger compute die and thus greater EPYC 9R45 (96c) Xeon 6975P-C (96c) distance between some core pairs. 0 0 Vera also has lower core-to-core latencies than Granite Rapids, with a 31 % reduction in mean latencies, 18 % in 32 32 the 5th percentile, and 39 % in the 95th percentile. Granite Rapids’ three compute dies are visible in the heatmap, with latencies between 45 ns and 61 ns within a chiplet, increasing 64 64 by approximately 14 ns for each hop to other chiplets. As two hops are required to the farthest chiplet, we observe an extra 28 ns of latency for a single core in the farthest chiplet, or doubled to 56 ns if both cores are resident in the farthest 0 32 64 0 32 64 Core ID Core ID compute chiplet from the cache line. Within a single compute chiplet, there is a range of approximately 20 ns which is similar to that of Vera across its single die. 30 40 60 100 150 200 Turin has a unique chiplet architecture with 8-core CCDs, Core-to-core latency (ns), lower is better leading to the lowest latencies out of any of the CPUs at approximately 27 ns within a CCD. Latency is significantly Fig. 4. Core-to-core latencies (ns) of each pair of cores on different CPUs. higher than that of the other CPUs once communicating Results measure the timing of the cache-coherency protocol for a shared cache line between two cores. A logarithmic latency color-scale is used. between cores in different CCDs, ranging between 130 ns and 244 ns with no clear pattern. In the best case, this is 1.69× Vera’s worst latency of 77 ns, and thus we expect that any Grace (72c) EPYC 9R45 (96c) latency-bound multi-core codes will exhibit poorer scaling on Vera (88c) Xeon 6975P-C (96c) Turin when scaling beyond a single CCD compared to Vera 250 which has more consistent and lower average latencies. 200 V. S YNTHETIC CODES 150 A. NAS Parallel Benchmarks Grace (72c)

Vera (88c)

0

215

109

77 66

Mean

83

5th percentile

69 57

83

150

155

50

68 57

100

57 47 27 58

Latency (ns), lower is better

Core ID

Core ID

0

0 Median

95th percentile

Fig. 5. Distribution of core-to-core latencies (ns) across all core pairs per CPU. Results derived from the same measurements shown in Figure 4.

line and another core immediately attempts to read. This is implemented in a microbenchmark written in Rust, with commit 2592303 compiled on all CPUs with a rustc version newer than 1.90.0 (October 2025) [17]. Each measurement is half the mean round-trip time over 300 samples of 1000 round trips after one discarded warm-up. The test is symmetric, so each pair of core IDs is measured once. Core IDs are pinned to one hardware thread per core.

The NAS Parallel Benchmarks are a suite of kernels and pseudo-applications derived from CFD codes that present mixed performance profiles [18]. They are written to have no external dependencies, and the main release that we use contains no architecture-specific optimizations. We run all original eight benchmarks (split into five kernels and three pseudo-applications), as well as UA as a benchmark for irregular memory accesses. Figure 6 presents the speedup in throughput, measured in Mop/s, that each CPU achieves compared to a 72-thread NVIDIA Grace baseline. We observe that Vera achieves the greatest speedup in all kernels except CG, a conjugate gradient kernel made up of unstructured matrix-vector multiplications. Surprisingly, AMD’s 9R45 and Intel’s 6975P-C performed poorly on EP in comparison to both NVIDIA Grace and Vera despite this kernel intending to estimate the floating point capabilities of the CPU, something which should demonstrate the larger vector lengths present on AMD and Intel’s offerings. Vera’s single-thread-per-core speedup compared to Grace here

Vera (88c)

Vera (88c), SMT on

EPYC 9R45 (96c)

Xeon 6975P-C (96c) 2.13

2 1.18 1.15

Speedup vs Grace (×)

4

1

0.5 BT

CG

EP

FT

IS

LU

MG

NAS kernel (class D)

SP

UA

Geo. mean (best of SMT off/on)

Fig. 6. Speedup of each CPU across NAS Parallel Benchmark codes for class D input size when compared to an NVIDIA Grace full-chip baseline. Geometric mean over all kernels presents the faster of SMT off and SMT on.

is expected, due to 6×128 bit SVE execute units on Vera compared to 4×128 bit SVE on Grace, leading to a 1.5× increased theoretical FLOPs per cycle. Perf counter results reveal that Granite Rapids and Turin had low CPU utilization at 53 % and 61 % respectively, compared to 99 % on Vera. We were unable to determine the cause for the low utilization as measured clock frequencies to be within their expected ranges. We find that FT, MG, and IS codes yield the largest speedup for Vera. Characterization of each kernel by Brown and Jamieson finds that MG spends 88 % of its runtime under high main-memory bandwidth utilization on Intel Skylake, and 18 % for FT which also has frequent all-to-all collective inter core communication [19]. We therefore attribute much of the 2.52× speedup of MG compared to Grace to the 2.42× all core achieved main-memory bandwidth. FT’s 2.49× speedup is due to 22 % additional cores, 2.42× greater main-memory bandwidth, and 17 % reduced mean core-to-core latencies. IS makes frequent indirect memory accesses, so its 2.39× speedup comes from Vera’s lower cache latencies and its 22 % more cores. Figure 7 shows the strong scaling behavior for each NAS code compared to the other CPUs. Threads were incremented by sequential core IDs. We observe that Vera achieves the highest performance of CPUs tested at every core count in seven of the nine kernels, indicating both strong singlethreaded performance and strong scaling that means that many of the kernels such as BT, SP, and CG, scale more linearly across the total core count compared to the x86 offerings which plateau more significantly as the number of cores increases. We note that while Turin achieved 1.22× more Mop/s than Vera on CG for class D, Vera outperforms Turin on class B. Perf results demonstrate that Vera has 94.5× more dTLB misses per operation than Turin at 0.194 against 0.0021 for class D. This indicates that at class D the kernel is limited by address translation rather than bandwidth. The effects of using all SMT threads are demonstrated in

Figure 8 where we see a speedup in four of the NAS codes when compared to a single-thread-per-core baseline. EP leads to the greatest speedup at 1.39×. We infer that this is because the kernel has inter-thread independence allowing for strong scaling of threads, but each thread has a dependent chain of instructions leading to low ILP. An additional spatial multithread with half of the core’s resources allows the exploitation of more ILP by saturating more of the core’s resources as two independent instruction chains can run per core. Perf counter data supports our claim: SMT achieves 1.43× increased IPC while maintaining similar clock frequency, cache miss rates, and branch misprediction rate. SMT led to poor performance on FT with class D with 0.61× the Mop/s compared to running with a single thread per core. Interestingly, our class B results show roughly equal performance regardless of SMT use. This demonstrates two threads with independent data contending for the same cache, effectively halving the available cache per thread. We validate this with perf, finding that with SMT there are 3.4× increased LLC-loads (L2 misses) while having 5.5× LLC-misses, with the LLC miss rate increasing from 20.4 % to 32.8 %. The L1d miss-rate remains consistent at 19 %. We infer that much of the data that was resident in L2 cache with a single thread per core misses both L2 and L3 cache in the two threads per-core case, increasing latency significantly. The default block factor of 32 leads to poor SMT performance for class D over 176 threads, while a factor of 8 leads to a 1.13× improvement with two SMT threads per core. B. Miniapps Miniapps are reduced proxy applications based on common algorithms or derived from larger HPC codes. These proxy applications mimic the implementation of long-running portions of complex production codes and are used to deduce performance trends through careful experimentation. We utilize miniapps in this study to compare how each CPU’s

BT

Throughput (Mop/s), class B

1M

CG

200k

750k

150k

500k

100k

250k

50k

0

0

EP

30k

FT

600k

IS

60k

20k

400k

40k

10k

200k

20k

0

0

0 0

LU

400k

MG

600k

300k

SP

800k

1.5k

600k

400k

200k

UA

100k

1k

0

0 0

32 64 Cores

96

500

200k 0 0

32 64 Cores

96

64

96

Grace (72c) Vera (88c) Vera, SMT on EPYC 9R45 (96c) Xeon 6975P-C (96c) SMT on faster SMT on slower

400k 200k

32

0 0

32 64 Cores

96

0

32 64 Cores

96

Fig. 7. Scaling graph of Mop/s achieved by each CPU on NAS Parallel Benchmarks with class B input size. The shaded band presents the difference between one versus two threads per core on Vera.

Vera, 2 threads per core (176 threads) Speedup vs 1 thread per core (×)

2

1.39 1.14 1

1.13 1.00

0.98

0.91

1.22

0.98

0.61 0.5 BT CG EP FT IS LU MG SP UA NAS kernel (class D)

Fig. 8. Speedup achieved across NAS Parallel Benchmark codes by NVIDIA Vera with two threads per core compared to a single thread per core.

performance scales across multiple cores in a range of different compute patterns. We use four codes with different characteristics – CloverLeaf [20], TeaLeaf [21], miniBUDE [22], and Neutral [23]. All miniapps were compiled and run with their OpenMP parallel implementation only. CloverLeaf is a hydrodynamics miniapp that solves the compressible Euler equations in a 2D stencil grid. TeaLeaf solves the heat diffusion equations via a Conjugate Gradient linear solver on a 2D grid. Both are main memory-bandwidth bound on a single node; CloverLeaf’s scaling is tightly coupled

with the increase in aggregated memory bandwidth while TeaLeaf becomes communication-bound at scale due to the per-iteration dot-product reductions. Both are included in the SPEChpc 2021 suite of HPC benchmarks [24] and, more recently, SPEC CPU 2026 [25]. MiniBUDE is a proxy application for BUDE, the Bristol University Docking Engine, a molecular docking code that predicts the structure formed between two molecules. This code is compute bound, with the L1d cache miss rate below 0.1 % on each architecture. The performance of the OpenMP implementation is impacted by the poses per workitem (PPWI) parameter which aids compiler unrolling of the inner loops, leading to improved vectorization. We fix PPWI to 32 across all architectures as larger values would not decompose the fixed problem-size properly over large core counts, leading to work imbalance and thus CPU idle time. Neutral is a Monte Carlo neutron transport miniapp. This code has data dependencies on the mesh that lead to random memory access patterns and poor data locality, which therefore leads the code to be memory latency bound. The impact of this memory latency varies depending on the input data, changing which particles interact and thus the memory access pattern. Figure 9 represents the speedup by each CPU compared to Grace on each miniapp. Vera achieves the highest performance on three of the four miniapps with a geometric mean of 2.41× speedup. This is greatest in CloverLeaf which is 2.76× faster than Grace. We can attribute most of this speedup to the 2.42× increase in main memory bandwidth. Turin achieves a 2.38× speedup over Vera on miniBUDE. Vera issued 3.91× more instructions per core compared to Turin, consistent with 128 bit vector instructions on Vera compared to 512 bit vector instructions on Turin. While Vera

CloverLeaf (bm32 short) 60 400

EPYC 9R45 (96c) Xeon 6975P-C (96c)

1 0.5

l f f tra DE ea ea u L L U r e a B N ve ni Te mi Clo

Geo. mean (best of SMT off/on)

Fig. 9. Speedup achieved by each CPU across HPC miniapp codes when compared to an NVIDIA Grace full-chip baseline. Geometric mean over all miniapps presents the faster of SMT off and SMT on.

achieved 1.81× higher IPC at 4.32, Turin executed 1.98× fewer cycles while also having more cores. Granite Rapids underperforms in miniBUDE which we uncover as a compiler performance issue due to -march=native leading to preferring a vector width of 256 bit, thus missing significant floating-point throughput in compute bound applications such as miniBUDE. Figure 10 measures the strong scaling performance of the miniapps. Threads were again incremented by sequential core IDs. In CloverLeaf, TeaLeaf, and Neutral, Vera achieves the highest performance at all but one core count. In TeaLeaf and CloverLeaf, Vera reaches close to its highest performance more quickly than other architectures before plateauing as its memory bandwidth has been saturated. SMT causes a significant speedup to Neutral for Vera. Neutral has the lowest IPC on Vera of the four miniapps at one thread per core at 1.17, reflecting its latency-bound profile. Introducing two threads per core increased the core’s IPC to 2.31, a 1.97× increase, while the instruction count only increased by 1.01×. SMT helps as the additional thread does not add significant memory pressure, with the miss rate for L1d and LLC remaining similar. This IPC gain is reflected in the 1.91× speedup we see through running with two threads per core. C. Applications Application performance provides a “real-world” assessment of the types of work that are performed on high-end CPUs. Since Vera is not yet widely available, library dependencies are yet to be optimized for this architecture so results should not be considered a final reflection of performance. While the choice of performance libraries such as BLAS and FFT can have a noticeable impact on performance, we use one open-source software stack on all four CPUs such that a

Speedup vs 1 Grace core (×)

2

2.41

4

miniBUDE (bm1)

300

40

1.56 1.24

Speedup vs Grace (×)

Vera (88c) Vera (88c), SMT on 8

200 20

100

0

0 0

60

32

64

96

TeaLeaf (bm 5e 2)

0

100

20

50

0

64

96

Neutral (stream)

150

40

32

0 0

32 64 96 Cores Grace (72c) Vera (88c) Vera (88c), SMT on EPYC 9R45 (96c)

0

32 64 96 Cores Xeon 6975P-C (96c) Linear scaling SMT on faster SMT on slower

Fig. 10. Scaling performance of each CPU across HPC miniapp codes when compared to an NVIDIA Grace single-core baseline. The shaded band presents the difference between one versus two threads per core on Vera.

speedup reflects the hardware performance, not the maturity of the relevant vendor library. We therefore use GCC 15.3.0, FFTW [26], and OpenBLAS [27] across all applications and architectures. Our Vera results therefore rely on optimizations targeting generic Armv8 or SVE targets and are likely to be conservative for BLAS and FFT-heavy codes such as CASTEP. We select six applications due to their frequency of use in UK science; CASTEP [28], CP2K [29], GROMACS [30], NAMD [31], OpenFOAM [32] and WRF [33]. Each will show a particular performance characteristic of the code and are well studied and familiar to the communities. Each application was built using Spack [34] version 1.1 and archspec was updated to automatically set the recommended compiler flags for Vera along with applying upstream fixes to Turin. Granite Rapids did not have support in archspec, and no upstream addition was available and was identified as Sapphire Rapids for compiler options. Spack was chosen for its flexibility and provides a consistency to the approach. Spack does not provide an out-of-the-box optimal build configuration and would require further work to tune the installation for each CPU outside the scope of this study. Each benchmark case fully populated the CPU with the most appropriate method for a single socket using Slurm (e.g. to stop distributing tasks across sockets). For most codes this resulted in MPI tasks across all CPU cores and when

Vera (88c)

Vera, SMT on

EPYC 9R45 (96c)

Xeon 6975P-C (96c)

Xeon, SMT on

2

1.39 1.72

1.89

Speedup vs Grace (×)

4

1

0.5 CASTEP (al3x3)

CP2K (fayalite)

GROMACS (TestCaseB)

WRF NAMD OpenFOAM∗ (stmv) (hpc motorbike) (conus12k)

Geo. mean (best of SMT off/on)

Fig. 11. Speedup achieved by each CPU across scientific applications when compared to an NVIDIA Grace full-chip baseline. Geometric mean over all applications presents the faster of SMT off and SMT on. ∗ Native threading is unsupported in OpenFOAM, so only one rank per core is presented.

testing multithreading, OpenMP was used to populate any extra hardware threads (SMT) on a CPU core. The summary of results for each CPU and application is shown in Figure 11. CASTEP calculates material properties using plane-waves density-functional theory. It is closed-source, but free for academic use across the world. The al3x3 case is small enough to fit into main-memory across each system. The results suggest the choice of FFT and BLAS libraries may be driving the lower than expected performance for Vera, which achieves 1.25× over Grace with one thread per core and 1.43× with two, and for Turin at 1.63×, against 2.34× for Granite Rapids. We used perf to find that zgemm kernels were dominant and their implementation would drive the performance characteristic. Intel, AMD and NVIDIA all provide optimized BLAS libraries that should benefit this code. CP2K is a package for atomistic simulation. It supports ab initio electronic structure and classical force-field molecular dynamics. The Fayalite-FIST case runs classical molecular dynamics of 28 000 atoms with the FIST force-field module for 1000 steps. This input writes coordinates and velocities every step and a restart every 20 steps on a single rank, leading to 62 % of the runtime being dominated by IO. Wall time therefore measures file output more than the CPU architecture. We instead report the force-evaluation time (fist_calc_energy_force maximized over all ranks, as reported in the code’s output file) which is the primary kernel for this input case. Vera evaluates forces 1.54× faster than Grace with one thread per core or 1.68× with two. Turin and Granite Rapids report a 1.03× and 1.28× speedup respectively despite their wider vector units, which suggests this routine is not limited by vector floating-point throughput. GROMACS is an extensively used molecular dynamics package that solves Newton’s equations of motion. Systems of interest, such as proteins, can contain millions of particles. At low node counts it is considered to be FLOP/s-bound. We used

perf to identify the SIMD kernels provided by GROMACS as the dominant factor. The performance of GROMACS is seen to benefit the Intel and AMD CPUs due to the well optimized FLOP/s performance whilst other CPUs compatible with the Arm ISA have known issues with maximizing FLOP/s. We found that NEON provided improved performance over SVE instructions for Arm and is the recommendation in GROMACS documentation. NAMD is a molecular dynamics simulation program designed to scale up to millions of atoms. The multicore version of NAMD was built to simplify the runtime configuration on a single socket. Similar to CP2K, many factors define performance. Vera’s 2.42× higher memory bandwidth and overall microarchitecture improvements drive its 1.88× speedup compared to Grace. OpenFOAM is a modular C++ framework that aims to simplify writing custom computational fluid dynamics (CFD) solvers. It is considered memory-bound on a single node. As there is no shared-memory threading option in the main distribution, we did not experiment with oversubscribing MPI processes and thus left SMT investigation for future work. The code takes advantage of the available memory bandwidth in Vera and achieves a speedup of 2.20×, similar to the increase in bandwidth. Intel and AMD have closer memory bandwidth to Grace and therefore have reduced gains in performance. WRF is a mesoscale numerical weather prediction system. We report the mean wall time per model step, excluding the first step which includes initialization. We use the CONUS 12 km case for 600 time steps. Of these, 57 also compute radiation, where these steps take approximately 9× the time of other steps, accounting for roughly half of the total step time. Vera is 2.05× faster than Grace on non-radiation steps and 1.78× on the radiation steps, leading to 1.91× overall and 2.07× with two threads per core. WRF is partially memory-bound on a single node, while radiation steps depend

more on floating point throughput. Granite Rapids achieves a 1.77× speedup on non-radiation steps with only 1.23× Grace’s STREAM bandwidth, suggesting that per-core compute throughput also contributes to performance. Overall, Vera achieves a significant speedup over our selection of applications at 1.89× of Grace when considering the best time out of SMT on/off. While this result is higher than the 1.72× speedup Granite Rapids achieves, Granite Rapids’ results are skewed upwards due to its very strong performance on CASTEP and GROMACS, while Vera presents large performance improvements when compared to Grace throughout. Many FLOP/s dominated codes, including GROMACS and CASTEP, have GPU implementations that provide significantly higher floating-point throughput. Further work on utilizing optimized vendor libraries with each application would also likely improve performance for each application. VI. C ONCLUSION This work has provided a holistic analysis of NVIDIA Vera through the use of microbenchmarks, miniapps, and scientific applications. We base our claims in comparison to NVIDIA Grace, while providing an insight into x86 offerings through the inclusion of AMD Turin and Intel Granite Rapids. Overall, we find that Vera is the strongest CPU out of our selection of CPUs. Vera provides a significant generational performance improvement compared to Grace; when comparing geometric means, we observe a 2.13× speedup in NAS Parallel Benchmarks, 2.41× speedup across our miniapps, and 1.89× speedup in our scientific applications. While Turin and Granite Rapids have competing results in select applications, such as a 1.65× for Granite Rapids against Vera for GROMACS, and 2.38× for Turin against Vera in miniBUDE, Vera remains the highest performing option for the majority of codes tested. Our methodology helps to uncover why we observe such speedups from Vera. Vera’s 1088.2 GB/s sustained memory bandwidth, 2.42× what was achieved on Grace, contributes significantly to the performance uplift of memory bound applications. Spatial Multithreading benefits the majority of the scientific workloads tested yielding a 1.11× geometric mean speedup. We find four of nine NAS codes have a performance improvement, all of which are between 1.13× and 1.39×. In the most extreme case of a highly latency-bound miniapp, Neutral, we find a 1.91× speedup through using SMT. While Intel’s Hyper-Threading model provides improvement over many of the real-world applications, the magnitude of this improvement is often weaker, achieving a 1.05× speedup in the scientific codes tested where it provided a performance gain. Vera’s lowest average core-to-core latencies across its monolithic compute die and low cache latencies, aided by prefetching select irregular data layouts, further enable Vera to scale performance across its full core count effectively on most workloads, particularly communication-heavy or latency bound codes. Vera primarily struggles in two cases: First, its modest 6×128 bit SVE vector units lead to lower floating point

performance per cycle while having fewer cores which is visible in compute-bound applications such as miniBUDE and GROMACS. Second, while adoption of the Arm ISA has grown in recent years, performance tuning for long-standing applications is still catching up. For example, GROMACS performance suffers compared to x86 and likely requires further algorithmic changes or more architectural optimizations to close the gap for some Arm-compatible platforms. We expect that as Arm becomes more mainstream in HPC, this will broaden optimization efforts across the software ecosystem, further enhancing performance on platforms such as Vera. Overall, we believe Vera to be an impressive data-center CPU that is not limited to iterative improvements over its predecessor Grace, but instead makes significant leaps in multiple areas of the core and chip design that provide strong performance across the majority of tested workloads when compared to Grace, Turin, and Granite Rapids. Future work should explore how Vera performs against next-generation AMD and Intel offerings, Venice and Diamond Rapids respectively. Alternatively, a deeper dive into Vera’s core architecture including specifics of the Spatial Multithreading implementation would help understand further where scientific codes can exploit this microarchitecture fully. AI D ISCLAIMER Large Language Models, specifically Claude Opus 5, Opus 5.5, and Sonnet 5, were used to aid the creation of code to generate run scripts, build scripts, scripts to help query and interpret results, and plotting scripts. These are included within the Artifact Description (AD). Within the paper text, they were only used during the review stage to help strengthen and condense text, spot inconsistencies in writing, as well as for LaTeX formatting. These tools were not used for conceptual ideas, result analysis, or citations. The authors edited all text output carefully and take full accountability for the text provided. ACKNOWLEDGEMENTS The authors would like to thank NVIDIA for providing prerelease access to engineering samples of Vera, as well as for accommodating a visit to NVIDIA HQ to learn more about the Olympus core architecture. In particular, we highlight the support from Giridhar Chukkapalli, Filippo Spiga, Dan Ernst, and Ananya Chakraborty. The authors acknowledge the use of resources provided by the Isambard 3 Tier-2 HPC Facility. Isambard 3 is hosted by the University of Bristol and operated by the GW4 Alliance (https://gw4.ac.uk) and is funded by UK Research and Innovation; and the Engineering and Physical Sciences Research Council [EP/X039137/1]. R EFERENCES [1] Top500 [Online]. Available: https://top500.org/lists/top500/list/2026/06/ [2] Green, T., Alam, S., McIntosh-Smith, S., Gilham, R. and Wishart, W., 2025. Evaluation of the NVIDIA Grace Superchip in the HPE/Cray XD Isambard 3 supercomputer. In Proceedings of the Cray User Group (pp. 93-102).

[3] Banchelli, F., Vinyals-Ylla-Catala, J., Pocurull, J., Clascà, M., Peiro, K., Spiga, F., Garcia-Gasulla, M. and Mantovani, F., 2024, January. Nvidia grace superchip early evaluation for hpc applications. In Proceedings of the International Conference on High Performance Computing in AsiaPacific Region Workshops (pp. 45-54). [4] McIntosh-Smith, S., Price, J., Deakin, T. and Poenaru, A., 2019. A performance analysis of the first generation of HPC-optimized Arm processors. Concurrency and Computation: Practice and Experience, 31(16), p.e5110. [5] Laukemann, J., Hager, G. and Wellein, G., 2026. Microarchitectural comparison, in-core modeling, and memory hierarchy analysis of stateof-the-art CPUs: Grace, Sapphire Rapids, and Genoa. Parallel Computing, p.103183. [6] Afzal, A., Hager, G. and Wellein, G., 2023, November. SPEChpc 2021 benchmarks on Ice Lake and Sapphire Rapids Infiniband clusters: A performance and energy case study. In Proceedings of the SC’23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis (pp. 1245-1254). [7] Laukemann, J., Gruber, T., Hager, G., Oryspayev, D. and Wellein, G., 2024, May. CloverLeaf on Intel multi-core CPUs: A case study in writeallocate evasion. In 2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 350-360). IEEE. [8] Lam, C. (2024) Arm’s Neoverse V2, in AWS’s Graviton 4. Chips and Cheese.[Online]. Available: https://chipsandcheese.com/p/armsneoverse-v2-in-awss-graviton-4 [9] Lam, C. (2024) Intel’s Redwood Cove: baby steps are still steps. Chips and Cheese. [Online]. Available: https://chipsandcheese.com/p/intelsredwood-cove-baby-steps-are-still-steps [10] Cohen, B., Subramony, M. and Clark, M. (2024) Next generation ”Zen 5” core. Hot Chips 36, Stanford University, CA, 25–27 August. Available: https://hc2024.hotchips.org/assets/program/conference/ day2/24 HC2024.AMD.Cohen.Subramony.final.pdf [11] Marr, D.T., Binns, F., Hill, D.L., Hinton, G., Koufaty, D.A., Miller, J.A. and Upton, M., 2002. Hyper-Threading Technology Architecture and Microarchitecture. Intel technology journal, 6(1). [12] McCalpin, J.D., 1995. Memory bandwidth and machine balance in current high performance computers. IEEE computer society technical committee on computer architecture (TCCA) newsletter, 2(19-25). [13] Torvalds, L., Test-Tlb. https://github.com/torvalds/test-tlb. [14] Frumusanu, A., (2021) The Snapdragon 888 vs The Exynos 2100: Cortex-X1 & 5nm - Who Does It Better? [Online]. Available: https://www.anandtech.com/show/16463/ snapdragon-888-vs-exynos-2100-galaxy-s21-ultra/3 [15] Pellegrini, A. (2021) ‘Arm Neoverse N2: Arm’s 2nd generation high performance infrastructure CPUs and system IPs’, Hot Chips 33: A Symposium on High Performance Chips, virtual, August 2021. Available at: https://hc33.hotchips.org/assets/program/conference/day1/20210818 Hotchips NeoverseN2.pdf. [16] NVIDIA (2026) NVIDIA Vera CPU Architecture. Whitepaper. Available at: https://nvdam.widen.net/s/nmw5vblpqd/nvidia vera cpu architecture whitepaper [17] Viennot, N., Core-to-Core Latency. https://github.com/nviennot/ core-to-core-latency. [18] Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., Fatoohi, R., Fineberg, S., Frederickson, P., Lasinski, T., Schreiber, R., Simon, H., Venkatakrishnan, V. and Weeratunga, S., 1994. The NAS Parallel Benchmarks. RNR-94-007 [19] Brown, N. and Jamieson, M., 2023, May. Performance characterisation of the 64-core SG2042 RISC-V CPU for HPC. In International Conference on High Performance Computing (pp. 354-367). Cham: Springer Nature Switzerland. [20] Herdman, J.A., Gaudin, W.P., McIntosh-Smith, S., Boulton, M., Beckingsale, D.A., Mallinson, A.C. and Jarvis, S.A., 2012, November. Accelerating hydrocodes with OpenACC, OpenCL and CUDA. In 2012 SC Companion: High Performance Computing, Networking Storage and Analysis (pp. 465-471). IEEE. [21] McIntosh-Smith, S., Martineau, M., Deakin, T., Pawelczak, G., Gaudin, W., Garrett, P., Liu, W., Smedley-Stevenson, R. and Beckingsale, D., 2017, September. Tealeaf: A mini-application to enable design-space explorations for iterative sparse linear solvers. In 2017 IEEE International Conference on Cluster Computing (CLUSTER) (pp. 842-849). IEEE. [22] Poenaru, A., Lin, W.C. and McIntosh-Smith, S., 2021, June. A performance analysis of modern parallel programming models using a compute-bound application. In International Conference on High Per-

formance Computing (pp. 332-350). Cham: Springer International Publishing. [23] Martineau, M. and McIntosh-Smith, S., 2017, September. Exploring onnode parallelism with neutral, a Monte Carlo neutral particle transport mini-app. In 2017 IEEE International Conference on Cluster Computing (CLUSTER) (pp. 498-508). IEEE. [24] SPEChpc 2021 [Online]. Available: https://www.spec.org/hpc2021/ [25] SPEC CPU 2026 [Online]. Available: https://www.spec.org/cpu2026/ [26] Frigo, M., Johnson, S.G., 2005. The Design and Implementation of FFTW3. Proceedings of the IEEE 93, 216–231. [27] OpenBLAS [Online]. Available: https://www.openmathlib.org/OpenBLAS/ [28] Clark, S.J., Segall, M.D., Pickard, C.J., Hasnip, P.J., Probert, M.I.J., Refson, K., Payne, M.C., 2005. First principles methods using CASTEP. Z Kristallogr Cryst Mater 220, 567–570. [29] Kühne, T.D., Iannuzzi, M., del Ben, M., Rybkin, V. v., Seewald, P., Stein, F., Laino, T., Khaliullin, R.Z., Schütt, O., Schiffmann, F., Golze, D., Wilhelm, J., Chulkov, S., Bani-Hashemian, M.H., Weber, V., Borštnik, U., Taillefumier, M., Jakobovits, A.S., Lazzaro, A., Pabst, H., Müller, T., Schade, R., Guidon, M., Andermatt, S., Holmberg, N., Schenter, G.K., Hehn, A., Bussy, A., Belleflamme, F., Tabacchi, G., Glöß, A., Lass, M., Bethune, I., Mundy, C.J., Plessl, C., Watkins, M., VandeVondele, J., Krack, M., Hutter, J., 2020. CP2K: An electronic structure and molecular dynamics software package -Quickstep: Efficient and accurate electronic structure calculations. Journal of Chemical Physics. [30] Abraham, M.J., Murtola, T., Schulz, R., Páll, S., Smith, J.C., Hess, B. and Lindahl, E., 2015. GROMACS: High performance molecular simulations through multi-level parallelism from laptops to supercomputers. SoftwareX, 1, pp.19-25. [31] Phillips, J.C., Hardy, D.J., Maia, J.D.C., Stone, J.E., Ribeiro, J. v., Bernardi, R.C., Buch, R., Fiorin, G., Hénin, J., Jiang, W., McGreevy, R., Melo, M.C.R., Radak, B.K., Skeel, R.D., Singharoy, A., Wang, Y., Roux, B., Aksimentiev, A., Luthey-Schulten, Z., Kalé, L. v., Schulten, K., Chipot, C., Tajkhorshid, E., 2020. Scalable molecular dynamics on CPU and GPU architectures with NAMD. Journal of Chemical Physics 153. [32] Weller, H.G., Tabor, G., Jasak, H., Fureby, C., 1998. A tensorial approach to computational continuum mechanics using object-oriented techniques. Computers in Physics 12. [33] Skamarock, W.C., Klemp, J., Dudhia, J., Gill, D.O., Liu, Z., Berner, J., Wang, W., Powers, J.G., Duda, M.G., Barker, D., Huang, X.-Y., 2019. A Description of the Advanced Research WRF Model Version 4.1. NCAR Tech. Note NCAR/TN-556+STR. [34] Gamblin, T., Legendre, M., Collette, M.R., Lee, G.L., Moody, A., de Supinski, B.R., Futral, S., 2015. The Spack package manager: Bringing order to HPC software chaos, in: International Conference for High Performance Computing, Networking, Storage and Analysis, SC.

Appendix: Artifact Description Artifact Description (AD) I. OVERVIEW OF C ONTRIBUTIONS AND A RTIFACTS A. Paper’s Main Contributions C1

C2

C3

The first independent microarchitectural characterization of NVIDIA Vera for HPC workloads, comparing to NVIDIA Grace, AMD EPYC 9R45 (Turin), and Intel Xeon 6975P-C (Granite Rapids). A methodology that isolates the performance of microarchitectural features (memory bandwidth, cache and core-to-core latency, SMT) with microbenchmarks, NAS kernels, and miniapps, allowing a holistic comparison with CPUs from other vendors. Validation of synthetic performance results against a representative suite of real-world scientific applications.

B. Computational Artifacts A1 A2 A3

https://doi.org/10.5281/zenodo.22969070 https://doi.org/10.5281/zenodo.22969070 https://doi.org/10.5281/zenodo.22969070 II. A RTIFACT I DENTIFICATION

A. Computational Artifact A1 Relation To Contributions A1 measures the memory subsystem of each CPU. More precisely, it measures sustained main-memory bandwidth with STREAM, memory access latency at different region sizes with test-tlb, and core-to-core latency. These results characterize Vera (C1 ) and are included in the methodology of C2 . Expected Results Vera achieves the highest sustained STREAM Triad bandwidth of the four CPUs at all core counts. It should measure the lowest time-per-access for region sizes below 128 MiB, and the lowest mean core-to-core latency. Expected Reproduction Time (in Minutes) Per platform, the expected times are the following: 15 minutes for setup and compilation, 60 minutes for execution (30 minutes for stream, 10 minutes for test-tlb, 20 minutes for core-to-core), and 30 minutes for graph generation and analysis. Artifact Setup (incl. Inputs) Hardware: One socket of each CPU is required: Nvidia Vera (88 cores, pre-release engineering sample, air-cooled, power-capped to a 450W TDP equivalent); NVIDIA Grace (72 cores, one socket of a Grace-Grace Superchip node of Isambard 3); AMD EPYC 9R45 (96 cores, SMT off, AWS c8a.metal-24xl); Intel Xeon 6975P-C (96 cores, AWS c8i.metal-48xl). Pre-release engineering samples of Vera are not publicly available at the time of publication and access cannot be granted.

Software: GCC 15.3.0 on all platforms; STREAM version 5.10; test-tlb (https://github.com/torvalds/test-tlb) commit 14321e3; core-to-core-latency (https://github.com/nviennot/ core-to-core-latency) commit 2592303. rustc version 1.95; Python 3 with pandas and NumPy; gnuplot and LATEX for figures. A1 provides the scripts required to compile and run each code. Datasets / Inputs: No datasets are needed. All inputs are generated by the benchmarks. Installation and Deployment: STREAM: perplatform Makefile-gcc15 with -O3 -fopenmp -mcmodel=large -DNTIMES=100 plus -mcpu=olympus (Vera), -mcpu=neoverse-v2 (Grace) or -march=native (x86). test-tlb: included Makefile, with PAGE_SIZE set to the base page size (64 KiB on Vera and Grace, 4 KiB on x86). Transparent huge pages of 2 MiB must be enabled (always or madvise in /sys/kernel/mm/transparent_hugepage/enabled). core-to-core-latency: cargo install core-to-core-latency. Source paths are set at the top of each script. Artifact Execution T1 (build) → T2 (run) → T3 (parse) → T4 (plot). STREAM: arrays of 183M (Vera), 240M (Grace), 202M (EPYC) and 252M (Xeon) 8-byte elements, each ≥4× the last-level cache; best of 100 iterations; OMP_PROC_BIND=spread, OMP_PLACES=cores; core IDs between 0..n−1 are provided via taskset, either in order or round-robin with stride 8 (Fig. 2 uses round-robin). test-tlb: numactl --cpunodebind=0 --membind=0 test-tlb -Hr <size> 64 (2 MiB huge pages, random pointer chase, 64 B spacing), 22 sizes at powers of two from 4 KiB to 4 GiB, mean of 10/5/3/1 trials depending on size. Core-to-core: all pairs of cores of socket 0, one thread per core, average of 300 iterations with 1000 samples per iteration. All runs are bound to socket 0. Artifact Analysis (incl. Outputs) The parsers convert raw output files to CSV files containing the relevant figure of merit. For STREAM, this is GB s−1 , while for test-tlb and core-to-core-latencies this is ns. The core-to-core-latencies results do not require parsing. Plotting scripts then use this data to generate the figures in the paper. B. Computational Artifact A2 Relation To Contributions A2 runs NAS Parallel Benchmark kernels for class B and D, and four miniapps to contribute to C2 . These are run scaling from one core to a full socket, testing with SMT off and on with Vera. perf stat counters of the same codes support the explanations in Section V (C1 ).

Expected Results Vera leads on a full socket in most NAS kernels and three of four miniapps and scales to all cores. SMT helps EP and the latency-bound Neutral and hurts FT class D. Turin leads in miniBUDE and NAS CG class D. Expected Reproduction Time (in Minutes) Per platform, the expected times are the following: 30 minutes for setup and compilation, 480 minutes for execution (60 minutes for NAS class D, 240 minutes for NAS class B sweeps, 180 minutes for miniapp sweeps), and 30 minutes for graph generation and analysis. Artifact Setup (incl. Inputs) Hardware: This artifact has the same hardware requirements as A1 . Software: GCC 15.3.0; NPB 3.4.4 OpenMP; CloverLeaf OpenMP (https://github.com/UK-MAC/CloverLeaf OpenMP) commit 022eb96; TeaLeaf (https://github.com/UoB-HPC/ TeaLeaf) commit e70261c; Neutral (https://github.com/ UoB-HPC/arch) commit d983598; miniBUDE (https:// github.com/UoB-HPC/miniBUDE commit 570f66c); Linux perf; Slurm scheduler on Grace. Datasets / Inputs: Each required input is provided with the source code of each benchmark: NAS classes B and D; CloverLeaf clover_bm32_short.in; TeaLeaf tea_bm_5e_2.in; Neutral stream.params; miniBUDE bm1. Installation and Deployment: For NPB, -O3 -fopenmp -mcmodel=large is used as default on all architectures. x86 platforms add -march=native, while Vera uses -mcpu=olympus and Grace uses -mcpu=neoverse-v2. miniBUDE and TeaLeaf are built using CMake with the following flags specified: -O3 plus the same march or mcpu flags as above. CloverLeaf and Neutral had the same flags added within their respective Makefile. Artifact Execution T1 (build) → T2 (run) → T3 (parse) → T4 (plot). Whole-socket runs use one thread per core (72/88/96 threads). Sweeps add cores incrementally between 0..n−1 with OMP_PROC_BIND=close, OMP_PLACES=cores and taskset; SMT runs add two threads at a time, core ID i and i+n. Grace runs are Slurm jobs with numactl --cpunodebind=0 --membind=0; Grace class D also sets OMP_STACKSIZE=512M. Class B input size is used for sweeps, class D is used for whole-socket runs. miniBUDE uses 32 poses per work-item (-p 32), so the 65,536 poses give at least 21 work-items per thread on every chip, and 100 iterations (-i 100). NAS, CloverLeaf, TeaLeaf and Neutral were run once per configuration. Artifact Analysis (incl. Outputs) The runs will output the following figures of merit: Mop s−1 for NAS; wall time for all miniapps except miniBUDE which uses the sum of the 100 iteration times as reported in the

output file. Speedups are relative to Grace single-socket results (Figures 6, 9) or single-core (Figure 10). perf counters are reported as ratios (miss rates, IPC), since event names map to different hardware events on each CPU. C. Computational Artifact A3 Relation To Contributions A3 runs six widely used scientific applications on the same four CPUs to test whether the conclusions drawn from A1 and A2 hold for production codes (C3 ). Expected Results Vera achieves the highest geometric-mean speedup over Grace of the four CPUs. Its largest gains are in the memorybound codes (OpenFOAM, WRF, NAMD). Granite Rapids leads in the compute-bound GROMACS and CASTEP. Expected Reproduction Time (in Minutes) Per platform, the expected times are the following: 240 minutes for building the applications and their dependencies with Spack, 180 minutes for execution (six applications, with SMT off and on where supported), and 30 minutes for graph generation and analysis. Artifact Setup (incl. Inputs) Hardware: This artifact has the same hardware requirements as A1 . Software: Spack releases/v1.1 (https://github.com/ spack/spack) with included archspec a patch that adds Vera (-mcpu=olympus) and the correct AMD Zen5 detection; Spack package repository and packages.yaml from isambard-sc/buildit tag v2.1 (https://github.com/isambard-sc/ buildit); ReFrame 4.9.3; GCC 15.3.0; MPICH 4.3.0. Applications: CASTEP 25.1.1, CP2K 2024.3, GROMACS 2025.3 (built using NEON instead of SVE on the Arm systems for improved performance), NAMD 3.0.1 (multicore Charm++ build), OpenFOAM v2312 and WRF 4.6.1 (dm+sm). CASTEP is closed source under a free academic licence (https://www. castep.org); its source cannot be redistributed and is not included (N/A). Datasets / Inputs: CASTEP al3x3; CP2K Fayalite-FIST (from the CP2K repository); GROMACS TestCaseB (PRACE UEABS 2.2); NAMD stmv; OpenFOAM HPC motorbike, Large case (OpenFOAM HPC committee repository); WRF CONUS 12 km (WRF v4.4 benchmark). The download and case-setup commands are included in each ReFrame test. Installation and Deployment: Each application has one ReFrame test per system. The test builds the application in a Spack environment with GCC 15.3.0 and then runs it. Compiler flags come from archspec: -mcpu=olympus (Vera), -mcpu=neoverse-v2 (Grace) and the matching x86 target. The system configuration files set the Spack package repository, packages.yaml and the MPI and nonMPI partitions for each platform.

Artifact Execution T1 (Spack build) → T2 (run) → T3 (collect figures of merit) → T4 (plot). All runs fill one socket. MPI codes (CASTEP, CP2K, GROMACS, OpenFOAM, WRF) run one rank per core with mpirun --bind-to core --map-by numa and OMP_NUM_THREADS=1. SMT runs keep one rank per core and add a second OpenMP thread (OMP_NUM_THREADS=2, OMP_PLACES=threads, OMP_PROC_BIND=true); WRF also sets OMP_STACKSIZE=64M. NAMD runs one multicore process with one worker per hardware thread (+p<N> +setcpuaffinity). OpenFOAM is pure MPI and has no SMT runs; its case runs 100 iterations. GROMACS runs 50,000 steps. Each configuration was run once. Artifact Analysis (incl. Outputs) The figures of merit are: wall run time for CASTEP and OpenFOAM; force evaluation time (fist_calc_energy_force) for CP2K; ns/day for GROMACS and NAMD; and the mean time per step for WRF, excluding the first (initialization) step. Speedups are relative to the Grace single-socket run. “Best of SMT off/on” uses the faster of the two runs for each application.

Related documents

• Layered Reproducibility for High-Performance Computing Applications: The Feel++ and Ktirio Urban Buildings Case Study#1535181
• Layered Reproducibility for High-Performance Computing Applications: The Feel++ and Ktirio Urban Buildings Case Study#1514884
• Performance Characterization of SPEC CPU 2026 on AMD EPYC 9755 Processor#821560
• VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU#155230
• The Energy Blind Spot: NVIDIA's Flagship Edge AI Hardware Cannot Support Process-Level Energy Attribution#609949