A SCEND TO S CIENCE : E XPLORATION OF AI C HIPS FOR S CIENTIFIC C OMPUTING
arXiv:2607.20120v1 [cs.DC] 22 Jul 2026
Weicheng Xue* , Kai Yang* , Yongxiang Liu, Baisong Xu, Dengdong Fan, Xianglin Liu† , Pengxiang Xu† , Yonghong Tian† Pengcheng Laboratory Shenzhen, China {xuewch, yangk, liuyx, xubs, fandd, liuxl, xupx, tianyh}@pcl.ac.cn * Equal contribution. † Corresponding authors.
A BSTRACT The rapid rise of AI-oriented accelerators has reshaped compute systems around low-precision tensor engines, raising a practical question for the HPC community: under what conditions can such hardware support scientific workloads that demand numerical robustness, irregular memory access, and scalability? Using the Ascend 910 NPU series as a representative tensor-centric platform, we characterize precision, execution, and memory-hierarchy bottlenecks that hinder the direct deployment of scientific codes. We then develop and evaluate workload-specific mappings across five application studies—HPL-MxP, LRSVD, SGEMM-cube, PQSim, and SMC-X—combining heterogeneous execution, mixed-precision numerical formulations, precision emulation, hierarchical memory orchestration, and communication–computation overlap. These studies show that AI-native NPUs can achieve numerical robustness, competitive performance, and satisfactory scalability when numerical formulation, execution placement, and data movement are addressed in a coordinated manner. Our results provide a state-of-the-practice case study of how scientific workloads can be adapted to tensor-centric architectures, while distinguishing transferable optimization principles from Ascend-specific implementation details. Keywords scientific computing, Neural Processing Unit (NPU), mixed-precision computing, heterogeneous execution
1
Introduction
Scientific discovery is increasingly driven by large-scale simulation, numerical modeling, and data-intensive analysis, pushing the demand for computational capability beyond what traditional high-performance computing (HPC) platforms can easily sustain. For decades, CPUs and GPUs have formed the backbone of scientific computing by combining massive parallelism with relatively general programmability. Today, however, the rapid rise of modern AI workloads is reshaping the hardware landscape: emerging compute systems are increasingly built around AI-oriented accelerators that prioritize extreme throughput for low-precision tensor operations [1]. This trend raises a fundamental question for the HPC community: can architectures originally optimized for AI also serve as effective platforms for general scientific computing? Hardware vendors are now prioritizing AI acceleration across both domain-specific and general-purpose processors [2]. Dedicated AI chips such as Google TPU [3], Huawei Ascend NPU [4, 5], and AWS Trainium [6] have emerged alongside modern CPUs [7] and GPUs [8] that increasingly incorporate tensor engines, wider vector units, highbandwidth memory (HBM), and bandwidth-oriented cache hierarchies. While these trends are highly effective for AI training and inference, they do not directly align with the algorithmic requirements of many scientific applications, which often demand stronger numerical robustness, more irregular memory access, finer-grained synchronization, and different communication patterns [2]. As flagship computing systems become progressively more expensive and increasingly AI-centric, the scientific computing community will need to adapt important workloads to hardware that was not originally designed for them [1].
Bridging this mismatch requires addressing three fundamental gaps. The first is an execution mismatch. AI accelerators excel on dense, regular, high-arithmetic-intensity kernels, but many scientific workloads are dominated by memorybound, reduction-heavy, or irregular operations whose performance does not scale with tensor throughput alone. The second is a precision gap. Scientific computing often relies on FP64 or numerically stable FP32 formulations, whereas modern AI processors derive efficiency from low-precision execution formats such as FP16 and BF16. The third is a data-movement gap. Scientific applications frequently depend on multi-level data reuse, irregular control flow, and communication-intensive execution, while AI-oriented architectures typically expose performance through explicitly managed memory hierarchies and tile-centric execution models. Together, these gaps make direct deployment of scientific codes on AI accelerators both nontrivial and, in many cases, ineffective. In this work, we investigate these challenges on the Ascend 910 series NPUs, a representative family of tensor-centric AI processors built around the DaVinci core architecture. Ascend combines Cube Units for dense tensor execution with Vector Units for more general vector processing, making it a useful platform for examining both the opportunities and the limitations of running scientific workloads on AI-oriented hardware. Rather than proposing a new programming model, we use the existing Bisheng C++ programming interface and runtime stack to study how scientific kernels and applications map onto this architecture in practice. This work is intended as a state-of-the-practice study rather than a claim that individual optimization primitives such as mixed precision, blocking, gate fusion, or communication overlap are themselves new. Our goal is to port a few workloads, identify required transformations, and extract empirically grounded lessons for scientific computing on tensor-centric processors. Our study proceeds from kernels to applications. We first characterize representative scientific kernels to expose the asymmetry between Cube-dominated, compute-bound kernels and Vector-/memory-bound kernels, revealing a fundamental kernel–architecture mismatch for scientific workloads. We then use five representative application studies— HPL-MxP, low-rank SVD (LRSVD), SGEMM-cube, quantum circuit simulation (PQSim), and scalable Monte Carlo (SMC-X)—to show how these gaps can be bridged across scientific workload classes. The five applications cover dominant, non-exclusive NPU mapping regimes: HPL-MxP combines Cube-friendly dense updates with mixed-precision control and communication overlap; LRSVD uses heterogeneous stage placement for precision-sensitive low-rank linear algebra; SGEMM-cube emulates FP32 dense matrix multiplication through low-precision Cube execution and cache-aware pipelining; PQSim represents bandwidth-dominated state-vector simulation; and SMC-X represents irregular Monte Carlo execution. Their implementations are workload-specific, but share precision-aware numerical reformulation, stage-aware CPU/NPU placement, execution restructuring, and explicit memory/communication orchestration. Across these five applications, we show that scientific performance depends on precision reformulation, execution restructuring, and explicit data-movement orchestration. The contributions of this work are threefold: • We present a kernel-centric characterization of Ascend 910A/910B/910C for scientific computing, showing how representative kernels interact with Cube Units, Vector Units, and the multi-tier memory hierarchy, and identifying the conditions under which scientific workloads can or cannot benefit from tensor-centric execution. • We develop heterogeneous execution strategies for precision-sensitive scientific applications on AI-oriented architectures, combining stage-specialized CPU–NPU partitioning, numerical stabilization under precision constraints, and system-level mixed-precision mapping for HPL-MxP. • We derive a set of systematic optimization principles for scientific computing on NPUs, including hierarchical memory orchestration, workload restructuring, heterogeneous CPU–NPU mapping, precision emulation, and communication–computation overlap, and validate them across five representative application studies spanning HPL-MxP, LRSVD, SGEMM-cube, PQSim, and SMC-X. The description of artifacts used in this work, including code, benchmarks, and experimental workflows, is provided in Appendix 7. Limitations and Scope. While our study demonstrates the feasibility of executing representative scientific workloads on AI-oriented Ascend NPUs, several limitations should be acknowledged. First, due to resource and access constraints, most implementations and evaluations in this work are conducted exclusively on the Ascend 910A/910B/910C platforms. Although many of the architectural trends we analyze (e.g., tensor-centric compute units, mixed-precision pipelines, and bandwidth-oriented memory hierarchies) are common across modern AI accelerators, our quantitative conclusions may not directly generalize to other NPUs or GPUs without further validation. Second, our methodology relies on carefully designed mixed-precision strategies and kernel-level optimizations to bridge the precision gap between scientific computing and AI hardware. The numerical robustness we observe therefore depends on the suitability of these techniques for each workload. Applications with stronger sensitivity to round-off error or stricter FP64 requirements may require additional algorithmic reformulation beyond what is explored in this work. Third, our programming and 2
performance results are contingent on the current maturity of the Ascend software stack, including the Bisheng C++ toolchain and runtime. Some of the observed performance and programmability trade-offs may evolve as the ecosystem improves, which means that parts of our conclusions are sensitive to the present state of compiler, library, and runtime support. Finally, although we evaluate a diverse benchmark suite spanning micro-kernels, BLAS kernels, and full applications, the selected workloads do not cover the entire spectrum of scientific computing. In particular, highly irregular, communication-dominated, or control-flow-heavy applications are underrepresented. Extending this study to such workloads and to multi-vendor AI accelerators remains an important direction for future work.
2
Related Work
Prior efforts on scientific computing over AI-oriented architectures have made substantial progress, but largely along separate routes. One line of work addresses the precision gap between low-precision tensor hardware and the numerical requirements of scientific algorithms. A second line restructures scientific workloads to better match AI-oriented execution. A third line focuses on memory and communication bottlenecks in large-scale accelerator-based simulations. We build on these established routes and do not claim that the underlying primitives are new in isolation. Rather than advancing only one route for one workload class, we connect them into a unified view of scientific computing on AI-oriented NPUs. 2.1
Precision-Oriented Approaches
A major challenge in adapting AI accelerators to scientific workloads is preserving numerical robustness on hardware optimized for low-precision execution. Mixed-precision iterative refinement has become a standard solution path in this space. HPL-MxP is a representative example, performing the dominant LU factorization in low precision while recovering FP64-accurate solutions through GMRES-based refinement [9, 10]. This strategy has been successfully deployed on multiple large-scale systems [11, 12, 13], demonstrating that low-precision accelerators can support numerically reliable scientific solvers when paired with suitable refinement schemes. Another complementary direction is precision emulation. The Ozaki scheme decomposes high-precision operands into lower-precision segments to emulate more rigorous arithmetic [14], while recent tensor-unit implementations of highaccuracy GEMM and related kernels achieve substantial speedups over native high-precision pipelines [15, 16, 17, 18]. These studies provide important numerical techniques, but primarily address precision itself. In contrast, our work treats precision as only one part of a broader architectural mismatch: numerical robustness must be coordinated with heterogeneous execution, memory behavior, and workload structure. 2.2
Workload Restructuring for AI-Oriented Execution
A second route focuses on reformulating scientific algorithms to better align with the dense, regular, high-throughput operators favored by AI accelerators. Ising model simulations have been reorganized into massively parallel tensor operations [19]; density functional theory has been expressed using large-scale tensor contractions [20]; and TPU-based linear algebra frameworks have enabled distributed matrix factorizations and FFTs at scale [21]. Stencil-based models have likewise been cast as matrix multiplications to exploit Tensor Core throughput [22]. JAX-Fluids and JAX-Fluids 2.0 have shown how fluid dynamics can be expressed and differentiated efficiently on AI-oriented systems [23, 24]. These works show that algorithmic restructuring can unlock AI hardware for specific scientific kernels and applications. However, most focus on one domain or one application family. Our work complements this route by first characterizing representative regular scientific kernels, and then using diverse applications to show how precision-sensitive, bandwidthdominated, and irregular workloads must be further restructured for efficient execution on AI-oriented NPUs. The closest delta of our study is therefore not a new blocking or fusion primitive, but the end-to-end mapping of multiple scientific workload classes onto an NPU architecture with explicit Cube/Vector separation, software-managed UB staging, and hierarchical communication. 2.3
Memory- and Communication-Centric Approaches
A third route emphasizes that many scientific workloads on accelerators are constrained less by arithmetic throughput than by memory traffic, data locality, and communication overhead, especially for bandwidth-dominated and irregular workloads. The Cerebras Wafer-Scale Engine, for example, demonstrates extreme-performance molecular dynamics partly by reducing communication bottlenecks through wafer-scale integration [25]. Large-scale mixed-precision solver deployments similarly show that end-to-end efficiency depends on careful coordination among on-chip memory, host-device interaction, and distributed communication [11, 12, 13]. 3
These studies highlight the importance of data movement as a first-class systems concern, but they typically arise within one benchmark, one architecture, or one application path. More broadly, prior work has largely advanced scientific computing on AI hardware along separate routes: numerical reformulation for low-precision execution, workload restructuring for accelerator-friendly execution, and system-level optimization for memory and communication efficiency. In contrast, our work connects these routes within a unified view of scientific computing on AI-oriented NPUs, combining kernel-level characterization with application-level evidence across HPL-MxP, LRSVD, SGEMMcube, PQSim, and SMC-X to identify common requirements in numerical reformulation, heterogeneous execution, hierarchical data orchestration, and communication–computation overlap. We treat this as a practice-oriented synthesis supported by implementation evidence, not as a claim of universal architecture-independent optimality.
3
Background and Motivation
The central question is under what architectural and algorithmic conditions scientific codes can run efficiently and robustly on AI-oriented NPUs. For scientific computing, the answer is governed by three recurring challenges: an execution mismatch, because AI processors are optimized for dense, regular tensor operations whereas scientific workloads often contain memory-bound, reduction-heavy, or control-irregular kernels; a precision gap, because many scientific applications require numerically stable FP32 or FP64 formulations while NPUs derive efficiency primarily from lower-precision execution; and a data-movement gap, because performance is frequently determined not by raw arithmetic throughput alone, but by how effectively computation is orchestrated across on-chip buffers, global memory, and interconnects. The Ascend 910 series provides a representative platform for studying these issues. Unlike traditional GPGPUs, Ascend was developed as a domain-specific AI processor centered on tensor throughput, software-managed memory movement, and heterogeneous execution engines. These features make it a useful platform for studying the opportunities and limitations of scientific computing on AI-oriented hardware. Our goal is not to argue that NPUs should universally replace general-purpose HPC accelerators, but to determine under what workload and architectural conditions they can serve as effective scientific computing substrates in increasingly AI-centric systems. 3.1
DaVinci Architecture
The Ascend 910 family is built around the DaVinci architecture, a heterogeneous AI-core design that integrates multiple execution engines with a software-managed memory hierarchy, as illustrated in Fig. 1. Each AI Core contains both a Cube Core and two Vector Cores, which share access to on-chip storage through explicit data-movement mechanisms. The Cube Core is optimized for dense block-structured computation and operates on fixed-size tensor tiles with dedicated L0/L1 buffers to maximize reuse. The Vector Core, by contrast, provides vector and scalar pipelines with direct access to the Unified Buffer (UB), enabling more flexible element-wise computation, reductions, and control-intensive execution. The UB is a software-managed on-chip buffer used by Vector/Scalar execution and by MTE-staged data movement. It is distinct from Cube-local L0A/L0B/L0C and L1 buffers, which feed matrix tiles to the Cube pipeline, and from the larger shared L2 buffer that connects AI Cores to off-chip HBM. Cube execution offers very high throughput, but only when computation is organized into regular tiles with sufficient arithmetic intensity. Vector execution is more flexible, but provides lower peak throughput and is therefore more exposed to memory-system limitations. At the chip level, AI Cores are connected through a shared L2 buffer and high-bandwidth paths to global memory and HCCS. Data movement among GM, L2, UB, and L0/L1 is explicitly orchestrated by the Memory Transfer Engine (MTE) and associated control logic, making Ascend a compute–memory co-designed architecture whose performance depends on jointly managing execution and data placement. 3.2
Execution and Memory Implications
For scientific workloads, the DaVinci core exposes a clear execution asymmetry. Cube-oriented execution is well suited to dense matrix updates, batched contractions, and other kernels with regular access patterns and high arithmetic intensity. Vector-oriented execution, in contrast, is the primary path for kernels dominated by reductions, streaming access, or fine-grained element-wise updates. This distinction implies that scientific kernels on Ascend will not benefit uniformly: some scale with Cube throughput, whereas others remain limited by Vector throughput and memory bandwidth. This asymmetry is reinforced by Ascend’s multi-tier memory hierarchy (Fig. 1 and Table 1), which spans Cube-local L0/L1 buffers, a shared UB for Vector and Scalar execution, a die-level L2 buffer, and off-chip HBM. Kernels whose working sets can be tiled and staged through on-chip storage can achieve high reuse and approach compute-dominated execution. Dense linear algebra kernels such as GEMM and SYRK fit this model. By contrast, matrix–vector products, 4
L0A Buffer
L0B Buffer L1 Buffer
AI Cube Cube unit
L0C Buffer
BT Buffer
L2 Cache
Global Memory
FP Buffer
DCache
ICache
Scalar unit
Instruction Dispatch
Cube queue MTE queue
AI Vector Unified Buffer
DCache
ICache
Scalar unit
Vector unit
Instruction Dispatch
Vector queue MTE queue
Figure 1: A schematic of the DaVinci architecture. Adapted from [26, 5].
reduction-heavy routines, and other weak-locality kernels more readily overflow the on-chip hierarchy and become dominated by traffic among L2, UB, and HBM. For these workloads, explicit orchestration of data placement, buffering, and transfer overlap becomes as important as tensor throughput. These observations motivate our study in this work. 3.3
Cross-Generation Implications
As summarized in Table 1, the Ascend 910 family shows a clear cross-generation shift toward a more balanced architecture for numerical workloads. Relative to 910A, later generations improve not only Cube throughput, but also FP32 capability, HBM capacity and bandwidth, and inter-chip communication. These changes matter because they help distinguish gains that come from stronger tensor throughput alone from those enabled by a better balance across compute, memory hierarchy, and communication. Table 1: Comparison of Ascend 910 Generations. The 910C peak values are reported per die. 910C Category Metric 910A 910B (per die) Nominal AI Cores (count) 32 20 25 Nominal Vector Cores (count) 32 40 50 FP16 MMA Cube FP16 (TFLOPS) 256 294.9 378.9 HF32 MMA Cube HF32 (TFLOPS) – 147.5 189.4 FP32 MMA Cube FP32 (TFLOPS) – 73.7 94.7 AXPY Vector FP32 (TFLOPS) 2 9.2 11.8 Nominal L2 Capacity (MB) – 192 192 L2/UB L2 BW (TB/s) – 7.86 7.86 Nominal UB Size (MB) 8 3.75 4.68 Nominal HBM Capacity (GB) 32 64 64 HBM/UB HBM BW (TB/s) 1.2 1.6 1.6
3.4
Software Stack
Hardware capability alone does not determine whether AI-oriented processors are useful for scientific computing. Equally important is whether the software stack exposes sufficient control over execution placement, memory movement, 5
Table 2: Programming-model mapping between CUDA Tensor Core and Bisheng Cube execution. CUDA Tensor Core Thread block Shared memory WMMA tile loading Tensor Core MMA Result write-back
Bisheng Cube Execution AI Core UB/L0 buffer GM→L0A/L0B data movement Cube MMA (Mmad) L0C→GM transfer
Scientific Applications SGEMM-cube / Low-Rank SVD / HPL-MxP / PQSim (Quantum) / SMC-X (Monte Carlo) etc.
Bisheng C++ Programming Interface
Kernel API (AIV/AIC)
Memory & DMI Model
Queue & Runtime
CANN Software Stack
AscendCL
Bisheng Compiler
Runtime & Driver
NPU Hardware Layer (Ascend 910) Da Vinci Cores (Cube & Vector Units) HBM L2/L1/L0 Buffers HCCS Interconnected PCIe
Figure 2: A layered view of the Ascend software stack. and communication to make architectural optimizations accessible in practice. Ascend provides this support through the CANN stack and the Bisheng C++ interface. CANN (Compute Architecture for Neural Networks) provides the compilation, runtime, and communication support for Ascend NPUs [27]. Functionally, it plays a role similar to the CUDA toolkit in the GPU ecosystem, integrating device compilation, kernel launch, memory management, and topology-aware collective communication through HCCL. Built on top of CANN, Bisheng C++ provides the low-level heterogeneous programming interface used throughout this work, exposing kernel APIs for AIC/AIV execution, memory and data-movement abstractions, and a queue-based runtime model for asynchronous execution. Together, these layers provide the control needed to express dense Cube-oriented kernels, vector-heavy stages, host–device data movement, and asynchronous execution without introducing a new programming model. Table 2 summarizes the conceptual mapping between CUDA Tensor Core programming and Bisheng Cube execution. While both programming models expose accelerator-oriented matrix operations, Bisheng requires more explicit control over execution placement and data movement across the software-managed memory hierarchy. This difference explains why efficient scientific workloads on Ascend often require architecture-aware kernel restructuring rather than direct source-level translation. 3.4.1
Additional Software-Stack Detail
3.4.2
CANN
CANN (Compute Architecture for Neural Networks) provides the foundational compilation, runtime, and communication support for Ascend NPUs [27]. It plays a role analogous to the CUDA Toolkit in the GPU ecosystem, integrating 6
device compilation, kernel launch mechanisms, memory management, and performance-optimized libraries for the DaVinci architecture. Within CANN, the runtime and driver layers are responsible for managing device contexts, allocating and synchronizing memory, and orchestrating kernel execution on the NPU. The Bisheng compiler translates device code into binaries targeting both Cube and Vector execution units of the DaVinci cores. For multi-chip configurations, CANN includes the Huawei Collective Communication Library (HCCL), which provides topology-aware collective primitives. On 910C systems, HCCL leverages the HCCS interconnect to enable low-latency, high-bandwidth communication without host CPU involvement, which is critical for scalable scientific workloads. 3.4.3
Bisheng C++ Programming Interface
Built on top of CANN, Bisheng C++ provides a C++-based heterogeneous programming interface that allows scientific developers to express device kernels and offload computation to Ascend NPUs with direct control on low-level hardware. As shown in Fig. 2, this interface exposes three core components: a kernel API targeting Ascend Vector and Cube units (AIV/AIC), a memory and data-movement model, and a queue-based execution and runtime abstraction. At the kernel level, Bisheng C++ offers APIs for writing device-side code that maps onto the DaVinci core’s Cube and Vector units. This enables performance-critical kernels, such as dense linear algebra or reduction primitives, to be expressed in a structured C++ form while still exploiting the underlying accelerators. At the memory level, the interface provides abstractions for managing host-device data movement and device-side buffers, allowing scientific codes to control data placement and reuse across kernel invocations. At the execution level, a queue and runtime model is used to schedule kernels and coordinate asynchronous execution on the NPU.
4
Experimental Setup
4.1
System Configuration and Interconnect
For distributed workloads, performance also depends on host–device coordination, communication, and synchronization. The system hierarchy is therefore directly relevant to the scalability results reported later, especially for communicationsensitive workloads such as HPL-MxP. Each Ascend 910C contains two symmetric dies connected by high-speed SIO links, increasing effective on-package compute density and HBM bandwidth. Within a node, multiple chips are connected through HCCS, which provides a scale-up fabric with substantially higher bandwidth and lower latency than PCIe for collective communication. In our platform, a standard node contains four physical chips, or eight NPU dies, connected through this hierarchy. Beyond a single node, communication extends through HCCS-based scale-up and an RDMA-based scale-out plane (RoH), as illustrated in Fig. 3. In this work, the system hierarchy matters because it determines whether communication-intensive stages can be overlapped effectively with computation and whether multi-chip workloads can scale without excessive synchronization overhead. We therefore treat interconnect behavior as part of the broader data-movement cost of the evaluated system rather than as a secondary implementation detail. 4.2
Experimental Environment
To support reproducibility and to clarify platform dependencies, Table 3 summarizes the hardware and software environments used in this work, including Ascend 910A/910B/910C systems and NVIDIA GPU baselines. Because several results depend on both processor generation and runtime/software-stack support, we report the host CPU and the common software environment for each platform here, and leave application-specific execution settings to the corresponding evaluation subsections. 4.3
Correctness, Variability, and Reproducibility
All performance results reported in this work were obtained only after validating correctness against applicationappropriate reference criteria. For HPL-MxP, correctness follows the standard benchmark requirement that iterative refinement recover an FP64-accurate solution. For LRSVD, correctness was assessed through stable convergence and numerically consistent low-rank decomposition behavior under single-precision constraints. For SGEMM-cube, correctness was assessed through relative error against the FP32 OpenBLAS reference and through stable error behavior across increasing matrix sizes. For PQSim and SMC-X, correctness was verified against the corresponding reference implementations while preserving the intended circuit evolution and Monte Carlo update semantics, respectively. 7
Supernode
Scale Up：HCCS（level 2）
Node 0
Node 15
Scale Up：HCCS（level 1）
Chip 0 Ascend 910 Die
Chip 1
Ascend 910 Die
Ascend 910 Die
SIO
Ascend 910 Die SIO
Scale Up
Chip 2 Ascend 910 Die
Chip 3
Ascend 910 Die SIO
Ascend 910 Die
Ascend 910 Die
…
… …
SIO
Scale Out：RDMA Plane (RoH)
Figure 3: Hierarchical Interconnect and System Topology of the 910C Cluster. A standard 910C node contains four physical chips, or eight NPU dies. Adapted from [5]. Table 3: Hardware and software environments used in this work. GPU baselines use optimized vendor libraries where available; application-specific baseline details and timing scopes are described in the corresponding sections. Platform Ascend 910A Ascend 910B Ascend 910C NVIDIA A800 NVIDIA H800 Local Workstation
Host CPU 4 × Kunpeng-920 @ 2.6 GHz 4 × Kunpeng-920 @ 2.6 GHz 2 × Kunpeng-920 @ 2.8 GHz 2 × Intel Sapphire Rapids 8462Y+ @ 2.8 GHz 2 × Intel Sapphire Rapids 8462Y+ @ 2.8 GHz 2 × Intel Xeon Platinum 8168 CPU @ 2.7 GHz
Software Environment CANN 6.3.RC2; OpenBLAS 0.3.18; UCX 1.11.2; OpenMPI 5.0.0rc2 CANN 8.0.RC2.10; OpenBLAS 0.3.18; UCX 1.11.2; OpenMPI 5.0.0rc2 CANN 8.0.RC2.10; OpenBLAS 0.3.18; UCX 1.11.2; OpenMPI 5.0.0rc2 CUDA 12.8; cuQuantum 26.01.0 CUDA 13.0; cuQuantum 26.06.0 Intel oneAPI HPC Toolkit 2024.2.0
For timing, all kernel-level results were measured after warm-up and obtained from at least three repeated runs; we report arithmetic means. Most application-level experiments were repeated three to five times under fixed hardware and software configurations, with additional repetitions when communication jitter was visible. Unless otherwise stated, application timings include the device computation and the communication/data movement required by the evaluated workflow, but exclude one-time compilation and setup costs. The main remaining sources of variability are system-level effects such as communication jitter, runtime scheduling, and host–device synchronization overhead, which are more pronounced in distributed experiments than in single-kernel measurements. To facilitate reproducibility, we report the hardware and software environments in Table 3 and describe applicationspecific mappings and execution strategies in the corresponding sections. Although some optimizations rely on architecture-specific mechanisms such as Cube/Vector execution and software-managed memory, we explicitly describe these mappings to clarify their implementation scope.
5
Scientific Kernels
The role of this section is not merely to report kernel performance, but to establish a kernel-level basis for the application studies that follow. To understand how scientific workloads interact with Ascend NPUs, we benchmark a representative set of scientific kernels spanning the main regular computational patterns relevant to later applications. We evaluated a broader set of 23 scientific kernels and selected a representative subset for detailed discussion. Specifically, we choose dense matrix–matrix multiplication (hgemm, sgemm-hf32) to exercise Cube units under compute-bound conditions, symmetric rank-k update (ssyrk) to represent block linear-algebra updates in factorization algorithms, matrix–vector operations (sgemv, ssymv) to expose Vector execution and memory-bandwidth limits, and complexvalued matrix multiplication (cgemm) to examine a mixed case relevant to quantum simulation. Together, these kernels 8
Table 4: Operational Intensity (OI) formulations for key BLAS operators on Ascend. Formulas consider theoretical FLOPs and total memory traffic (Read inputs + Read/Write output) including lower-order terms. Key: M, N, K are matrix dimensions. S represents element size in bytes (S = 2 for FP16, S = 4 for FP32, S = 8 for Complex FP32). Operator
Precision
Byte Size (S)
FLOPs (W )
Memory Access (Q) [Bytes]
hgemm
FP16
2
2M N K
2(M K + KN + 2M N )
sgemm
FP32
4
2M N K
4(M K + KN + 2M N )
cgemm
Complex FP32
8
8M N K
8(M K + KN + 2M N )
ssyrk
FP32
4
N 2K
4 (N K + N (N + 1))
sgemv
FP32
4
2M N
ssymv
FP32
4
2N 2
4(M N + N + 2M ) 4 N (N2+1) + 3N
ssyr2k
FP32
4
2N 2 K
4(2N K + N 2 + N )
hgemv
FP16
2
2M N
2(M N + N + 2M )
cgemv
Complex FP32
8
8M N
strmv
FP32
4
N2
ctrmv
Complex FP32
8
4N 2
8(M N + N + 2M ) 4 N (N2+1) + 2N 8 N (N2+1) + 2N
FP32
4
2BM N K
4B(M K + KN + 2M N )
batched sgemm
Operational Intensity (I = W/Q) MNK M K + KN + 2M N MNK 2(M K + KN + 2M N ) MNK M K + KN + 2M N N 2K 4(N K + N 2 + N ) MN 2(M N + N + 2M ) N2 N (N + 1) + 6N N 2K 2(2N K + N 2 + N ) MN M N + N + 2M MN M N + N + 2M N2 2N 2 + 10N N2 2 N + 5N MNK 2(M K + KN + 2M N )
span different arithmetic intensities, data types, and access patterns. All results are reported as arithmetic means over at least three runs; run-to-run variation is small in these steady-state experiments. To make the roofline comparison reproducible, Table 4 defines the operational-intensity model used in this section. For each operator, we compute I = W/Q, where W is the algorithmic floating-point operation count and Q is the global-memory traffic required to read the inputs and read/write the output; lower-order terms are retained in the formulas. The resulting expressions capture the key separation observed later: the intensity of GEMM-like operators grows with problem dimensions and can become compute-bound, whereas matrix–vector operators approach low, nearly dimension-independent intensity and remain bandwidth-sensitive. Using the operational intensities defined in Table 4, Fig. 4 and Fig. 5 compare the performance of these kernels on Ascend 910B/C and GPU and reveal a clear architectural separation between Cube-dominant and Vector-/memorydominant execution. For compute-bound kernels such as hgemm, sgemm-hf32, and ssyrk, both processors achieve high fractions of peak tensor throughput, with 910C consistently outperforming 910B. On 910C, hgemm reaches approximately 324 TFLOPS and sgemm-hf32 reaches approximately 132 TFLOPS under the measured device-level configuration. These throughputs correspond to 85.5% of matching FP16 peak and 69.7% of matching effective HF32 peak, respectively. In particular, hgemm reaches approximately 324 TFLOPS on 910C versus 259 TFLOPS on 910B, while hgemm and ssyrk show similar generational gains. These results confirm that dense linear-algebra kernels with regular tiling and high arithmetic intensity map efficiently to the Cube and benefit directly from increased throughput. In contrast, vector- and memory-bound kernels such as sgemv and ssymv achieve much lower absolute throughput and much smaller generational gains. Their performance remains in the range of 0.7∼1.5 TFLOPS, indicating that these kernels are limited less by available compute peak than by memory bandwidth and Vector-unit efficiency. The modest improvement from 910B to 910C shows that stronger Cube or Vector throughput alone does not substantially accelerate bandwidth-sensitive scientific kernels. Although sgemv appears above the slanted roofline, this does not contradict the roofline model used here, which assumes a single-level global-memory bandwidth. In practice, sgemv benefits from on-chip reuse and Vector–UB pipeline overlap, yielding an effective bandwidth above the simple GM→L1 assumption. The complex-valued cgemm kernel occupies an intermediate position. It still benefits from Cube execution, but complex arithmetic and additional data movement reduce utilization relative to real-valued GEMM. Unlike sgemm’s direct GM→L1→L0A/L0B path, interleaved cgemm inputs are first staged in UB for Vector-side real/imaginary unpacking, then moved through L2/L1 to the Cube. This extra unpacking and staging, together with cross-term computation and 9
Achieved Throughput (TFLOPS)
1000 800
A800 H800 910B 910C (per die)
4 2
600
1 400
A800 H800 910B 910C (per die) 2.07
3
679.6
324.2
0
366.5
0.84
1.19 1.05
1.46 0.78
sgemv
0.70 0.79
ssymv
256.6 259.2
200 0
126.9
132.0 107.5
hgemm sgemm-hf32
76.0 51.658.5 18.7
ssyrk
Cube-dominated
0.8 1.2 1.0 1.5
sgemv
0.8 2.1 0.7 0.8
ssymv
81.4 48.665.1 18.5
cgemm
Vector / memory-bound Mixed / complex
Figure 4: Achieved throughput of representative scientific kernels on Ascend 910B, 910C, and NVIDIA GPUs.
recombination, lowers utilization. Native complex tile formats or fused real/imaginary unpacking support could reduce this overhead. As a result, cgemm reaches about 65 TFLOPS on 910B and 81 TFLOPS on 910C: substantially below real-valued GEMM, yet still clearly responsive to improvements in Cube performance across generations. Taken together, these results establish the conditions under which scientific kernels benefit from Ascend NPUs. Highintensity kernels with regular data access can exploit Cube units effectively and scale with hardware generation, whereas vector-heavy, reduction-dominated, and bandwidth-sensitive kernels remain governed by the Vector engine and memory hierarchy. The application studies that follow can therefore be viewed as different ways of bridging the broader architectural gaps identified earlier: reformulating dense computation into Cube-efficient forms, assigning numerically sensitive or control-intensive stages to more suitable execution paths, and restructuring data movement and communication to reduce the cost of non-Cube-dominant execution.
6
Application Studies: Bridging Precision, Execution, and Data-Movement Gaps
The kernel study in Section 5 shows that scientific workloads do not benefit uniformly from Ascend NPUs: some map efficiently to the Cube engine, whereas others remain constrained by Vector execution, memory hierarchy, or communication overhead. The five application studies in this section show how the broader precision, execution, and data-movement gaps can be bridged through heterogeneous execution, numerical reformulation, workload restructuring, and hierarchical data orchestration. HPL-MxP spans the precision and communication gaps through mixed-precision refinement and CPU–NPU–communication overlap; LRSVD addresses precision through heterogeneous stage placement and numerical stabilization; and SGEMM-cube addresses precision through FP32 emulation and cache-aware Cube execution. PQSim and SMC-X, by contrast, show how bandwidth-dominated and irregular workloads must be restructured to match the execution and memory model of AI-oriented NPUs. 10
Performance (TFLOPS)
103
cgemm hgemm sgemm sgemv ssymv ssyrk
102
101
756.0 TFLOPS 312.0 TFLOPS
378.9 TFLOPS 294.9 TFLOPS
+25.1% +22.8% +30.0% +25.0%
Architectures: 910B (Hollow) 910C (Filled) A800 (Black)
H800 HBM Slope = 2.0 TB/s A800 HBM Slope = 2.04 TB/s HBM Slope = 1.6 TB/s
100
1 10 10 1
+39.1%
L2 Slope = 7.86 TB/s +12.1%
100
910B Roofline 910C Roofline A800 Roofline H800 Roofline
101
102
Operational Intensity (Flop/Byte)
103
104
Figure 5: Roofline analysis of representative scientific kernels on Ascend 910B, 910C and GPU. The plot shows achieved performance versus operational intensity (OI) under a single-level global-memory roofline model.
6.1 6.1.1
Bridging the Precision Gap Mixed-Precision System-Level Mapping (HPL-MxP)
Gap and application characteristics HPL-MxP has emerged as a representative benchmark for modern supercomputers in the AI era [9, 28]. Unlike traditional FP64-only HPL [29], HPL-MxP performs the dominant O(N 3 ) LU factorization in low precision and restores FP64-accurate solutions through O(N 2 ) iterative refinement. The benchmark thus directly exposes the precision gap between low-precision accelerator throughput and scientific numerical requirements. On Ascend NPUs, HPL-MxP is challenging not only because of mixed precision, but because it simultaneously stresses dense tensor computation, memory hierarchy, host–device coordination, and distributed communication. The trailing matrix updates are naturally suited to Cube units, but triangular solves and refinement are numerically sensitive and poorly matched to low-precision NPU execution. In distributed runs, panel broadcasts and CPU–NPU coordination introduce additional communication overhead. HPL-MxP thus provides a system-level test of whether mixed-precision scientific computing can be realized efficiently on AI-oriented NPUs. Solution: CPU–NPU–communication mapping Our implementation maps HPL-MxP to Ascend through coordinated CPU–NPU–communication mapping (Fig. 6). The computationally dominant trailing matrix update is executed on Ascend Cube units using FP16 inputs with FP32 accumulation, striking a balance between tensor throughput and numerical robustness. To reduce memory pressure and improve reuse, we adopt a multi-iteration fusion strategy that combines multiple GEMM updates into a single execution window, thereby increasing on-chip reuse and overlapping data movement with computation. Numerically sensitive and control-intensive stages remain on the CPU. In particular, the triangular matrix inversion required by TRSM is carried out in FP64 on the host, and the iterative refinement stage is likewise executed on the CPU because it requires strict FP64 residual computation. This CPU–NPU partition preserves correctness where low-precision NPU execution would be fragile or inefficient. 11
LU Factorization A (NZ FP16) at Ascend
A (ND FP32) at Kunpeng
TRSM Data
Panel Data
Next TRSM pre (On Device)
LU Factorization (FP32 on Host)
Panel Factorization (On Device)
Next Panel pre (On Device)
TRSM (On Device)
Next LU Data
Next TRSM Data
Next Panel Data
LU Data
Next LU Data Convert NZ->ND FP16->FP32
C(FP16)
GEMM GEMM_fusion (FP16 on Ascend)
Figure 6: Fused hierarchical pipeline for HPL-MxP LU factorization. The workflow shows orchestration of LU kernels with fused stages and MPI/HCCL communication–computation overlap to maximize NPU utilization. Adapted from [13].
Communication is handled jointly through HCCL and MPI. Factorized panels are broadcast among NPUs using HCCL, while CPU-side coordination of triangular inverses is handled through MPI. Within TRSM, the CPU computes the inverse of the triangular block, and the resulting matrix is then multiplied with trailing blocks on the NPU Cube units. To integrate these components efficiently, we construct a multi-level pipeline that overlaps computation, memory movement, and communication. Kernel-level pipelining keeps the Cube units saturated, while HCCL and MPI operations are overlapped with NPU execution to reduce idle time at system scale.
System-level mapping contribution The contribution here is not the mixed-precision formulation itself, which is prescribed by HPL-MxP, but its system-level implementation on Ascend: dense updates are assigned to Cube units, numerically sensitive stages remain on the CPU, and communication is pipelined with CPU and NPU execution.
Validation We evaluate the implementation on three generations of Ascend NPUs. Fig. 7 reports sustained HPL-MxP performance as a function of node count, with each node containing 8 NPU dies. The benchmark exhibits approximately linear scaling over the evaluated configurations, showing that the combined CPU–NPU mapping and multi-level pipeline effectively mitigate both memory and communication bottlenecks. Among the three platforms, Ascend 910C consistently delivers the highest throughput, reflecting improved balance among compute, memory hierarchy, and interconnect. Communication is nevertheless a dominant scaling pressure. At 128 dies on the 910A cluster, HCCL collectives account for more than 90% of the exposed communication time in our breakdown. The multi-level pipeline in Fig. 6 is therefore essential: it does not eliminate communication, but hides a large fraction of it behind panel preparation, TRSM-related CPU work, and NPU GEMM updates. These results demonstrate that peak FLOPS alone do not determine scientific performance on AI accelerators. By combining mixed-precision numerical control, fused dense-kernel execution, and aggressive communication–computation overlap, the LU factorization sustains high efficiency without the scalability collapse typical of communication-dominated solvers. This confirms that, for mixed-precision scientific solvers, bridging the precision gap is a system-level problem rather than a kernel-level one: numerical formulation, execution placement, and communication overlap must be coordinated for scalable performance on AI-oriented chips. 12
910A 910B 910C 910A Linear Scaling 910C Linear Scaling
TFLOPS
104
3759
1786
103
25937 15316 10701 7601
7905
3860
1823
1069 799
8
16
32
NPU Dies
64
128
Figure 7: Strong-scaling performance of HPL-MxP on three generations of Ascend NPUs (910A, 910B, and 910C). The 910B platform is a standalone machine. The results show sustained TFLOPS across increasing NPU counts, enabled by heterogeneous CPU–NPU mapping, kernel fusion, and communication–computation overlap.
6.1.2
Heterogeneous Low-Rank SVD (LRSVD)
Gap and application characteristics Singular Value Decomposition (SVD) is a fundamental primitive in highperformance computing and data analytics, but its cubic complexity makes classical formulations impractical for large-scale problems. Randomized low-rank methods substantially reduce both computational and memory cost while retaining strong accuracy guarantees, making them a natural target for accelerator-based scientific computing [30, 31, 32, 33]. For Ascend NPUs, the difficulty is not simply that SVD is expensive, but that its phases exhibit sharply different numerical and execution characteristics. Ascend does not provide a native SVD kernel, and its efficient execution path is centered on single-precision tensor and vector operations. As a result, direct deployment of a standard low-rank SVD pipeline leads to both poor hardware utilization and numerical fragility, especially in orthogonalization and subspace construction. LRSVD therefore serves as a representative test case for the precision gap: it contains dense stages that are well matched to Cube/Vector execution, but also numerically sensitive and control-heavy stages that require stabilization beyond straightforward NPU execution. Solution: heterogeneous decomposition and refinement To bridge this gap, we design a hybrid CPU–NPU low-rank SVD framework that partitions the randomized SVD pipeline by both kernel structure and numerical sensitivity (Fig. 8). Modern randomized SVD consists of three main phases: (i) QB decomposition, (ii) Lanczos bidiagonalization, and (iii) symmetric QR iteration on a small tridiagonal matrix [33]. These stages map naturally to two execution streams. The NPU stream targets data-parallel stages dominated by matrix multiplications, projections, and vector norms. QB decomposition and Lanczos iterations are therefore offloaded to the Ascend Cube and Vector units, allowing the computationally intensive sketching and projection phases to exploit high-throughput tensor and vector execution. The CPU stream handles the shifted QR iteration on the small tridiagonal matrix, whose frequent branching and scalar updates make it better suited to the host processor. 13
Workflow on NPU Workflow on CPU
Vector Unit
Cube Unit
QB Decomposition
The Original Matrix Reduction 𝑪 = 𝑸𝑻 𝑨, 𝑬 = 𝑪𝑪𝑻
Lanczos Algorithm Obtain Eigen Decomposition 𝑬 = 𝑼𝟏 𝜮𝟏 𝑼𝑻𝟏
Symmetric QR Iterative Algorithm with Displacement for Tridiagonal Matrices
The Resulting Matrices 𝟏/𝟐
𝑼 = 𝑸𝑼𝟏 , 𝜮 = 𝜮𝟏 −𝟏/𝟐
𝑽𝑻 = 𝜮𝟏
𝑼𝑻𝟏 𝑪
Figure 8: Heterogeneous execution workflow and hardware mapping for the LRSVD solver. The algorithmic pipeline is partitioned by computational structure and numerical sensitivity.
Beyond execution mapping, a second challenge arises from maintaining numerical stability during QB decomposition. On Ascend, Gram–Schmidt orthogonalization is numerically fragile in single precision: rounding errors accumulate, the orthogonality of basis vectors degrades, and convergence may fail. To restore robustness, we introduce an iterative orthogonalization refinement: ! k X (i) (i−1) qk+1 = I − ql qlH qk+1 , i = 1, . . . , nloop , (1) l=1
which reprojects each new basis vector onto the orthogonal complement of the current subspace. In practice, nloop = 2 or 3 provides the best balance between stability and overhead. Stage-specialized implementation The contribution is not randomized low-rank SVD itself, but a stage-specialized heterogeneous workflow for executing LRSVD on AI-oriented NPUs. Data-parallel QB decomposition and Lanczos stages are mapped to Cube/Vector units, while the branch-heavy shifted QR stage remains on the CPU. In addition, single-precision iterative orthogonalization refinement is introduced to recover numerical stability under NPU-oriented execution. Algorithmic Optimization for Single Precision Directly porting the standard QB decomposition to Ascend’s single-precision environment results in significant accuracy loss. We introduce two key modifications to ensure convergence. Iterative Orthogonalization Refinement The standard update rule for the orthogonal basis vector qk+1 is: qk+1 =
I−
k X
! qi qiH
ak+j+1
(2)
i=1
In double precision, a single pass of this projection is sufficient. In single precision, residual errors remain in the directions of existing basis vectors q1 , . . . , qk . We replace the single projection with an iterative loop to progressively suppress error accumulation. Analysis of nloop : The reported experiments use nloop = 2, matching the configuration in Fig. 9. This repeated projection suppresses the dominant single-precision orthogonality error without adding unnecessary passes. 14
Algorithm 1 Refined Basis Calculation with Iterative Orthogonalization 1: Input: Current vector ak+j+1 , existing basis {q1 , . . . , qk }, loop count nloop (0)
2: qk+1 ← ak+j+1 3: for i = 1 tonloop do
Pk (i−1) (i) qk+1 ← I − l=1 ql qlH qk+1 5: end for (nloop ) 6: qk+1 ← qk+1 7: Output: Refined orthogonal vector qk+1 4:
Adaptive Convergence Thresholds Theoretical algorithms check if the norm ∥qk+1 ∥ = 0 to determine subspace exhaustion. In floating-point arithmetic, strictly zero is rarely achieved. We introduce a relative error threshold ϵ tailored for the Ascend single-precision environment: ϵ = 0.005 (3) This is the QR convergence tolerance used for the plotted results, together with r/N = 0.1 and two QR refinement loops. Validation To evaluate the resulting solver, we compare the Ascend implementation against three baselines: (i) full-rank SVD on an Intel Xeon Platinum 8168 CPU with MKL, (ii) an optimized CPU-based LRSVD, and (iii) a GPU-accelerated randomized SVD on NVIDIA H800/A800 using cuSOLVER. Matrix sizes range from N = 1,000 to 10,000, covering representative single-node scientific problem sizes. Fig. 9(a) shows that the Ascend 910B and 910C implementations consistently outperform all baselines and achieve especially strong gains at large matrix sizes. For N = 10,000, the NPU solver delivers roughly an order-of-magnitude speedup over both the optimized CPU solver and the GPU cuSOLVER baseline. This improvement comes from offloading the dominant sketching and projection phases to the NPU while avoiding performance loss in the controlheavy stages. Fig. 9(b) further shows that the hybrid design is advantageous in the low-rank regime (r/N < 0.2), the regime most relevant to model reduction and data compression. These results demonstrate that low-rank linear algebra workloads can be efficiently mapped to AI-oriented NPUs when execution stages are partitioned according to numerical sensitivity and hardware characteristics.