arXiv:2605.03566v1 [cs.DC] 5 May 2026
Lifting to tensors when compiling scientific computing workloads for AI Engines Nick Brown
Gabriel Rodriguez-Canal
EPCC at the University of Edinburgh Edinburgh, UK [email protected]
EPCC at the University of Edinburgh Edinburgh, UK
Since then AMD have released a range of NPUs, based on the XDNA v1 and v2 architectures. The close coupling between the NPU and CPU cores offers a range of potential opportunities for leveraging this specialised compute, intended primarily for AI/ML, to accelerate scientific computing. However, a major challenge is in the programming of this architecture. Whilst AMD have made progress via IRON [3], one must still rewrite codes and recast algorithms into a form that is suitable for the AIE. This is not only time consuming, but also requires extensive expertise. Furthermore, existing AIE programming approaches are Python and C++ based which, in a world where around 60% of scientific computing codes are written in Fortran [4], is a challenge in and of itself. In this paper we describe an approach which enables general purpose loops to be seamlessly offloaded to the NPU. Focussing on Fortran and OpenMP, two very popular programming technologies in the scientific computing community, by lifting the representation of a loop to tensors our approach is able to exploit this high-level information when making decisions around how to efficiently target the hardware. The contributions of this paper are as follows: • We demonstrate much of the MLIR tensor representation is applicable beyond AI/ML, and transformations are able to lift OpenMP loops to this representation. I. I NTRODUCTION • We highlight how, based upon the rich information present High Performance Computing, HPC, is heavily used for in the tensor representation, the compiler can make scientific computing workloads. Whilst many people might effective decisions when mapping general purpose, loop naturally assume leadership class supercomputers, more conbased, computing workloads to the NPU. strained local clusters are also popular for scientific computing • We demonstrate that the tight coupling on Ryzen-AI workloads and the edge is also worthwhile exploring. HPC between CPU and NPU delivers performance and energy relies on mainstream CPUs and GPUs, but with an emphasis on benefits when co-executing loop iterations. energy efficiency and the continuing importance of delivering II. BACKGROUND AND RELATED WORK increasing performance to meet the ever growing demand from users, there is interest in leveraging specialised hardware AMD Embedded, formerly Xilinx, introduced the AI Engine technologies typically designed for AI/ML. AMD Xilinx’s (AIE) to provide hardened support for common arithmetic AI engines are one example, initially released as part of the operations on their FPGAs. Each AIE employs a very-longVersal Adaptive SoC these are vector arithmetic accelerators. instruction-word (VLIW) architecture capable of issuing up to AI Engines, or AIEs, adopt a Very Long Instruction Word seven instructions per cycle, and supports both scalar and vector (VLIW) design and contain a dedicated 512 bit vector unit. execution via a 512-bit vector unit. In 2023 AMD Launched It has been demonstrated that there is potential for these in their XDNA Phoenix Neural Processing Unit (NPU) which HPC [1] [2], and in 2023 AMD released the Ryzen AI series integrates the AIEs directly into the Ryzen AI CPU. Figure 1 of CPUs which combines AIEs, termed the Neural Processing sketches the XDNA v1 architecture, which we focus on in this Unit (NPU), with traditional x86 cores. paper, where NPU tiles are organised as a two-dimensional Abstract—It has been demonstrated that specialised architectures, such as FPGAs and AMD’s AI Engines (AIEs), have the potential to deliver energy and performance advantages for scientific computing. Given the integration of AIEs into AMD’s CPUs, this is an interesting potential avenue especially when executing on the edge or making better use of local compute constrained resources. However, a major challenge is in enabling existing codes to run on this architecture without extensive modification. Put simply, it requires significant expertise and time to port codes to the AIE’s execution model. In this paper we explore a compilation pipeline for efficiently mapping loops in general purpose, scientific codes to AIEs. Lifting the semantics of an application into tensors, we demonstrate that this is able to capture the intention of general purpose loops annotated with OpenMP and such high-level tensor information provides a richness that is effective when mapping to the AIEs. Requiring only an OpenMP decorated loop, our approach significantly reduces code complexity when targeting the architecture. For six kernel benchmarks, representing AI and scientific computing, using our approach the NPU performs comparatively to the multicore CPU for float32, in all cases at reduced energy to solution. For two scientific computing kernels running across both the CPU and NPU together delivers up to a 40% improvement in performance and 15% reduction in energy usage compared to the CPU alone. Index Terms—MLIR, AMD AI Engines, HPC, tensors
mesh with nearest-neighbour connectivity in both dimensions. An AIE can directly access the local memories of its north, south, and west neighbours. Each engine also incorporates four data movers comprising two 32-bit input streams and two 32-bit output streams.
[7], providing up to a 139 times speed up compared to the CPU. Other efforts have looked to address programmer productivity on the AIEs, for instance [8] proposed an end to end AIE programming model that leverages a Python Domain Specific Language (DSL) and the HPX programming framework was enhanced to support AIEs in [9]. However, both these require rewriting codes into their respective frameworks. A subset of Fortran intrinsic subroutines were offloaded to AIEs in [10] and whilst that work only supports a very restrictive set of workloads that heavily use Fortran intrinsics, it demonstrates the composability benefits of MLIR. A. LLVM and MLIR
LLVM is a modular, reusable compiler and toolchain infrastructure that enables the construction of compilers for diverse programming languages and hardware targets. It provides language front-ends and a broad set of hardware back-ends, connected via the LLVM intermediate representation (LLVM IR). A front-end, such as Flang, emits LLVM IR and can, in principle, target any supported back-end, including CPUs, GPUs, and FPGAs. However, LLVM IR is low level and substantial effort is required, and duplication common, in each Fig. 1. Illustration of AMD’s Hawk Point NPU, comprising five columns front-end when generating LLVM IR. of four rows of AIEs (each with compute core and 64KB of memory). Each To this end, Google developed MLIR and released it open column has a 512KB memory tile and four columns have an interface tile. source in 2019. MLIR provides a multi-dialect intermediate As sketched in Figure 1, the Hawk Point NPU we focus on in representation with standard transformations between these this paper comprises 20 AIEs arranged as five columns by four dialects. Instead of lowering directly to LLVM IR, front-ends rows. Each column additionally incorporates a memory tile, and translate into one or more higher-level dialect representations four of the columns include an interface (shim) tile connecting and then rely on existing MLIR passes to progressively the array to the CPU and main memory. The NPU uses the lower to LLVM IR. MLIR uses the standard Static Single AIE-ML generation of AI engine which has been optimised Assignment (SSA) form for IR, and a key advantage is that for AI workloads and provides a per-engine DRAM memory dialects can be composed and manipulated independently, of 64 KB. Each dedicated memory tile contains 512 KB of enabling staged lowering that incrementally moves towards SRAM and 12 data movers, in contrast to the four provided the target architecture. MLIR promotes extensive sharing of by compute and interface tiles, delivering up to 30 GB/s of compiler infrastructure by reusing established dialects and aggregate bandwidth [5]. XDNA vector units do not natively transformations, substantially reducing development effort. support int32 or float32 datatypes which must be emulated. Core dialects include memref for memory, func for functions This is a limitation but AI engines are evolving rapidly and and arith for arithmetic, and furthermore MLIR provides a framework for defining custom dialects and transformations. coupling the x86 CPU cores with NPU is promising. In our opinion, AIEs integrated with an existing x86 CPU Indeed, AMD have added several MLIR dialects to support are an attractive proposition for general purpose programmers compilation for the AIE such as aie which describes streaming as they can still leverage the CPU for their code as normal, connections between AIE compute tiles and Direct Memory and the tight integration with the NPU delivers the possibility Access (DMA) and adf to express AMD’s Adaptive Data of accelerating key parts of the code. Software written for Flow (ADF) graph connecting tiles. Moreover, a range of the AIEs comprises two parts, the compute kernels which are transformations and optimisations have also been developed mapped to the AI Engines and a graph description that connects by AMD, enabling lowering from these dialects to instructions interfaces, kernels and memories together via streams. Kernels that will execute on the NPU. Flang is the LLVM project’s Fortran front-end and built from follow a producer–consumer model, consuming input from up to two streams and emitting results on up to two output the ground up using MLIR. Aiming to provide comprehensive streams. Streams can connect directly to the CPU through an support for the Fortran language, Flang generates IR based upon the hlfir (High Level Fortran Intermediate Representation) interface tile, to a memory tile, or to another compute tile. There have been a variety of successes in using AMD’s AIEs and fir (Fortran Intermediate Representation) dialects. It then for HPC workloads, for instance [6] demonstrated significant undertakes a series of transformation and optimisation passes performance benefits when running option pricing on AIEs on these dialects before generating LLVM IR. Flang sits outside compared to CPUs. Furthermore, a fundamental operator of of the core MLIR ecosystem and integrates with only a subset CNNs was accelerated using the AIEs on the Versal ACAP in of the core dialects, where hlfir & fir are not part of MLIR
Fig. 2. Illustration of our MLIR-based OpenMP loop compilation flow for the AI Engines.
itself, instead with Flang providing its own path to LLVM IR. Consequently, [11] developed a lowering from hlfir & fir into core MLIR dialects. It was found that integrating with the entire MLIR ecosystem provides some performance benefits, but also crucially flexibility. For example a much wider range of dialects are available, including those provided by vendors for architectures such as the NPU.
III. A LOOP BASED O PEN MP AIE COMPILER FLOW
Figure 2 illustrates our compiler flow where Fortran code, annotated with OpenMP target offload, is first processed by Flang which generates IR that comprises mainly the hlfir, fir and omp dialects. As described in Section II, hlfir and fir are then transformed by [11] to core MLIR dialects including omp. Whilst we use Fortran as a driver, our work is not tied to that B. OpenMP front-end and-so this provides future flexibility. The contribution of our work starts at omp offload to device Since standardisation in 1997, OpenMP has become the de-facto model for shared-memory parallelism in scientific in Figure 2, where operations in the omp dialect are transformed computing. Originally designed for threaded CPU workloads, into our device dialect which simplifies the mapping to host 2017 saw the introduction of the target offload directive as part side XRT calls. Data transfers between the host and device are of OpenMP 4.0 to suppirt accelerators, primarily GPUs. Driven transformed into device.alloc which returns memrefs that by Fortran and C, Listing 1 sketches an example Fortran loop are tagged with an explicit memory space on the device. Host and device IR is then separated into separate modules, and decorated with OpenMP target pragma for offload. MLIR provides support for OpenMP via the omp dialect. host-side device dialect operations are then lowered into the There have been several efforts to target FPGAs with OpenMP, XRT Wrapper MLIR dialect, xrtw, from [15]. Ultimately, this for instance Nymble [12] and [13]. By comparison, instead results in LLVM IR on the host comprising the non-accelerated of FPGAs we target the AI engines that are already present Fortran code and calls to XRT for interacting with the NPU, in Ryzen AI CPUs. The cornerstone of our approach is to which is then compiled by Clang to generate the host binary. The module comprising code to run on the AIEs lift the abstraction level from OpenMP to the MLIR tensor dialect, and our hypothesis is that this will then provide a contains OpenMP operations that describe the structure rich source of information for the compiler. Tensorize [14] is a of the loops, for example omp.parallel for a paralcompiler approach lifting legacy code to a tensor representation, lel loop and omp.loop_nest is a nested loop. Modiin their case the linear algebra linalg MLIR dialect. From this fiers, such as omp.private for thread private data and it generates NumPy or StableHLO. Our approach differs in two omp.declare_reduction are also present in the IR. This main ways, firstly [14] only targets CPUs and GPUs whereas information is now leveraged by the transformation lift to our focus is on other architectures, such as the NPU, which are tensors in Figure 2 that lifts the IR into the tensor and tosa very different and whose compiler support is less mature for (Tensor Operator Set Architecture) dialects. The tensor dialect general purpose programming. Secondly, Tensorize comprises handles generic creation and manipulation of tensors, whereas a significant amount of complexity in mapping a range of the tosa dialect implements the TOSA specification [16] which loops in Python or C. Instead, by requiring the programmer provides a set of common machine learning operations. to have decorated their loops with OpenMP then our lifting Listing 1 sketches a simple Fortran loop using OpenMP to tensors is significantly simplified because we are able to target offload which adds each element in the a and b arrays leverage guarantees, such as the independence of loop iterations, before multiplying the result with a constant stored in c. Listing provided by the corresponding OpenMP pragmas. 2 portrays the IR in the tensor and tosa dialects that has been
lifted from this Fortran code. Tensors provide value semantics, where the focus is on the values rather than the concrete implementation and the device.tensor_compute operation wraps all tensor operations in the IR. This provides a bridge between abstraction levels from reference semantics elsewhere to the high-level value semantics of tensors in this lifted form. Listing 1 Example Fortran loop offloaded with OpenMP 1 2 3 4 5 6
!$omp target parallel do private(t) map(from:c) do i=1, 128 t=a(i)+b(i) c(i)=t*100 end do !$omp end target parallel do
The tosa.add and tosa.mul operations in Listing 2 perform element wise addition and multiplication respectively, with tensor.splat broadcasting a scalar value into each element of a tensor. The resulting tensor from this computation, c, is then yielded as the result of the device.tensor_compute operation. Tensors and TOSA provides a rich representation around the intention of compute without complications at this stage of lower details such as to how it will be achieved. Listing 2 Sketch of IR based on Fortran loop in Listing 1 lifted to the tensor and tosa dialects. %res_t = device.tensor_compute(%0, %1) ({ ^0(%a : tensor<128xf32>, %b : tensor<128xf32>): %t = tosa.add %a, %b : tensor<128xf32> %scalar = arith.constant 100.0 : f32 %const = tensor.splat %scalar : tensor<128xf32> %c = tosa.mul %t, %const : tensor<128xf32> device.tensor_yield(%c) }) : (memref<128xf32>, memref<128xf32>) -> tensor<128xf32> Tensors are capable of capturing a wide range of computation and data access constructs. For example, Listing 3 sketches how stencil calculations such as c[i] = a[i-1] + b[i+1] are represented in tensors, where a_e and b_e are extracted from the a and b tensors respectively using tensor.extract_slice. The indexes [0] [128] [1], for instance in the first tensor.extract_slice operation, denote the offset, number of elements and stride. A tosa.add operation then performs an element wise addition of these slices, with a_e containing 128 elements from index 0, and b_e 128 elements starting from index 2. The resulting slice res is then inserted into c by the tensor.insert_slice operation which produces the final resulting c_res tensor. OpenMP imposes certain restrictions on loops, such as loop iteration independence, and this makes lifting to tensors more straight forwards than in Tensorize [14] which lifted legacy
Listing 3 Sketch of tensor based IR for calculating c[i] = a[i-1] + b[i+1] %a_e= tensor.extract_slice %a[0] [128] [1] : tensor<130xf32> to tensor<128xf32> %b_e= tensor.extract_slice %b[2] [128] [1] : tensor<130xf32> to tensor<128xf32> %res= tosa.add %a_e, %b_e : tensor<128xf32> %c_res= tensor.insert_slice %res into %c[1] [128] [1] : tensor<128xf32> into tensor<130xf32>
code to a tensor representation in the linalg dialect. At the implementation level, our transformation pass identifies the outputs of the loop and, for each of these, walks the IR backwards to build up a dependency graph of operations connecting loop inputs to outputs. A conversion is then undertaken for each constituent operation within each graph to generate its tensor counterpart. Whilst this handles a wide range of loop structures, we do not currently support atomic OpenMP pragams and the presence of these will cause the loop to fallback to the CPU. Listings 2 and 3 illustrate how the tensor representation provides a rich description of the compute within loops. This can then be used to drive decisions when targetting the AIEs. For instance, the offsets in Listing 3 influence how FIFOs are generated and tensors enable dependencies between operations to be discovered. The next transformation in the pipeline of Figure 2, decomposition, uses this dependency information to determine placement of compute across the NPU. We provide two strategies; decomposing operations and/or decomposing loop iterations across the NPU. Mixing of these strategies is supported, for instance in Listing 2, the tosa.mul operation might be placed on one AIE and tosa.add on another, and these groups of two AIEs replicated across four, each acting on a unique chunk of iterations. Limitations imposed by the architecture restrict and influence these decisions, most importantly that compute tiles have a maximum of two inputs and two outputs. Crucially in our approach the compiler handles this rather than it being the programmer’s responsibility. At this stage tensor operations and/or loop iterations have been distributed across the NPU and this is represented by our high-level dialect, hlaie. The hlaie dialect is a step down in abstraction from tensors, and encodes the decomposition across the NPU and AIE interactions, but not how these are achieved. The dialect comprises the following operations: 1) hlaie.kernel defines a compute kernel, taking up to two hlaie.streams as input and up to two as results. 2) hlaie.memory represents a memory tile. 3) hlaie.external host and device connection. 4) hlaie.streams values between tiles. 5) hlaie.stream_read reads value(s) from a stream. 6) hlaie.stream_write writes value(s) to a stream. At this point in the compilation pipeline the IR contains hlaie.kernel, hlaie.memory and hlaie.external operations. Compute has been distributed across the NPU and
each of these contains specific tensor operations, with tile level inputs and outputs connected via hlaie.stream. The materialisation pass of Figure 2 lowers from value semantics of tensors into reference semantics of affine loops that read specific values from stream(s) and the arith dialect then performs arithmetic upon these, with results then written via hlaie.stream_write to output stream(s). The next transformation pass, lower streams & placement in Figure 2, materialises the kernel, memory and external operations to actual AIE tiles. This involves mapping to physical compute, memory and shim tiles and making decisions around placement. We aim to place components that communicate on tiles near each other, for instance mapping hlaie.kernels that stream data to neighbouring aie.cores. After exploiting the dependency information held by the hlaie dialect these operations are then lowered into corresponding FIFO operations within AMD’s aie dialect. Our approach then executes the chunking for vectorisation transformation which manipulates the inner structure of each kernel following [17] to vectorise arithmetic operations. It inserts an inner affine.for loop of iteration count vector width, and an outer loop stepping from one chunk to the next. Lastly, DMA transfer operations are generated which determines DMA between the host and device, both driving the streams and copying data into memory tiles. A complication was that, as part of this work, we discovered AMD’s MLIR flow itself does not support vectorisation and this impacts performance on the NPU as AIEs run scalar only. Consequently, after chunking for vectorisation, transformed IR is extracted and provided to AMD’s aie-translate tool which generates vectorised C++ code using AIE intrinsics from the IR. The rest of the IR is provided to AMD’s aie-opt tool which performs a series of further passes to lower from the AMD specific aie, aievec and aiex dialects to LLVM IR. The generated C++ and remaining LLVM IR is then compiled with either AMD’s open source Peano compiler or Chess, our approach providing automatic vectorisation of user code. The key point of this section is that lifting to tensors is the key enabler here and the tensor abstraction is able to capture the compute pattern of loops, especially when driven by OpenMP due to the guarantees provided by the semantics of those operations. Other alternatives, such as mapping loops to the affine dialect, lack the Destination Passing Style (DPS) of tensor and tosa dialects which provide a richer description of data and dependencies when mapping to the AIEs. Ultimately, this rich information can then be exploited by the compiler to make sensible decisions around how to effectively target the NPU. By lowering the abstraction levels from tensors, through our hlaie dialect, ultimately to AMD’s AIE dialects we are able to progressively materialise the key aspects in the IR to suit the architecture. IV. R ESULTS AND EVALUATION Our experiments run on an eight core Ryzen AI 8945HS CPU equipped with 64GB of DRAM and containing the Hawk Point, XDNA v1, NPU. We use GCC version 14, XRT release
version 2025.1, Flang, LLVM and MLIR versions 20.1.7, and the release version 1.1.0 of AIE-MLIR. All results are averaged over ten runs and AIE execution times include the overhead of transferring data between the host and NPU. All CPU code is compiled at optimisation level three. Experiments conducted in this paper leverage Chess, and all NPU runs are over 16 AIEs (the four columns with a shim tile). Kernel softmax relu saxpy dot product l2norm gemm
Problem size 4m 67m 67m 67m 67m 512
NPU hand written Runtime Lines (ms) of code 11.82 215 5.42 179 6.12 156 9.30 203 8.77 187 1.51 1540 TABLE I
NPU our approach Runtime Lines (ms) of code 11.19 24 5.87 5 5.22 9 9.13 9 8.48 11 10.56 14
C OMPARISON BETWEEN HAND WRITTEN KERNELS FROM [18] USING IRON AND C++, COMPARED TO F ORTRAN AND O PEN MP USING OUR APPROACH . A LL USING FLOAT 32 DATATYPE ( APART FROM GEMM WHICH USES BF 16 AS INPUT AND FLOAT 32 AS OUTPUT ).
We compared foundational kernels important for HPC and AI/ML, and Table I reports a comparison of kernels using OpenMP following our flow against hand written AIE implementations. The softmax, relu, saxpy and gemm hand-written implementations were developed by AMD and from [18], whilst dot product and l2norm were developed by the authors. It can be seen that performance of our approach is generally comparable to that of hand written AIE implementations, with gemm being the outlier because AMD have heavily optimised that kernel but at the cost of code complexity. CPU NPU our approach Runtime Energy Runtime Energy (ms) usage (J) (ms) usage (J) softmax 10.48 0.44 11.19 0.26 relu 28.27 1.04 5.87 0.17 saxpy 13.79 0.39 5.22 0.15 dot product 9.19 0.54 9.13 0.21 l2norm 5.01 0.28 8.48 0.20 gemm 28.96 1.82 10.56 0.27 TABLE II C OMPARISON BETWEEN ALL 8 CORES OF THE CPU AGAINST USING OUR APPROACH ON THE NPU. A LL CODE O PEN MP AND FLOAT 32. Kernel
Problem size 4m 67m 67m 67m 67m 512
Lines of code are also reported in Table I, and it can be seen that the hand written versions require significantly more code to be written than our approach. In calculating this metric we ignore code comments, but includes the IRON graph, C++ compute kernel and C++ host code. This demonstrates one of the major current challenges with writing code for the architecture where to gain performance the programmer must effectively writing three separate pieces of code; the host C++ code using the XRT API, the C++ compute kernel(s) using the AI Engine API, and Python code using IRON for the AIE graph. This is not only time consuming but also requires expertise in all these APIs. By contrast, when using our OpenMP loop based approach the programmer requires just one single codebase and an existing loop that is decorated with OpenMP with the compiler then handling the rest. Consequently, many of the lines of code that are counted under our approach will already exist in the CPU version of the code.
Table II reports a performance and energy usage comparison for these six kernels written in Fortran with OpenMP running multi-threaded on all eight CPU cores and 16 AIEs. It can be observed that performance provided by the NPU is generally competitive against the CPU and regardless energy usage is less when the kernels are run on the NPU. Whilst gemm is the outlier as it performs is around 2.8 times faster on the NPU than the eight-core CPU, it should be highlighted that the CPU implementation using Fortran and OpenMP is fairly naive as one would naturally write it and for instance does not leverage known optimisation techniques such as tiling. A. Evaluation for scientific computing kernels We compared two Fortran HPC codes on the CPU against the NPU using our compilation pipeline. Firstly a 2D PW advection scheme from the Met Office’s MONC atmospheric model [19] used in production runs and optimised for the CPU. The second code is a compute intensive loop from the Shallow Water Equation (SWE) mini-app developed by NCAR and used as a proxy for production workloads. Both kernels perform stencil based computations, an extremely common pattern in scientific computing where calculations are performed for each grid cell and these involve quantities from neighbouring grid cells, a simple example was illustrated in Listing 3. Table III reports performance (million grid points processed per second, where higher is better) and energy to solution on the AMD Ryzen 8945HS. We leverage a hybrid co-execution strategy where separate chunks of iterations run across the CPU (67%) and NPU (33%) concurrently. Float32 is used for all experimental runs, with the addition of the OpenMP target pragma being the only code change required for the NPU. This hybrid approach improves performance as the CPU and NPU are processing loop iterations concurrently. Whilst energy to solution is increased compared to running on the NPU alone in Table II, this is still less than the multicore CPU alone. Kernel PW advection SWE
CPU eight cores Throughput Energy (MPts/s) usage (J) 381.84 43.11 791.99 363.90 TABLE III
Hybrid NPU+CPU Throughput Energy (MPts/s) usage (J) 534.06 41.43 1008.32 315.61
T HROUGHOUT ( MILLION GRID POINTS PER SECOND , HIGHER IS BETTER ) AND ENERGY TO SOLUTION ON CPU AND HYBRID NPU+CPU USING OUR APPROACH . SWE FOR 1 MILLION GRID POINTS AND 4000 ITERATIONS , PW ADVECTION 268 MILLION GRID POINTS . F LOAT 32.
V. C ONCLUSIONS In this paper we have explored the seamless offloading of general purpose loops, driven by Fortran, to the NPU by decorating them with the OpenMP target pragma. By lifting to a tensor representation, we demonstrated that this high-level view of compute provides a rich source of information when targeting the NPU. Driven by guarantees provided by OpenMP, a variety of computing loops can be transformed into a tensor representation, and one is able to then effectively lower through intermediate dialects to AMD’s AIE dialects. In the main our approach achieves competitive performance to that of hand written codes but at significantly reduced number
of lines. The energy to solution is also significantly lower on the NPU compared to either configurations on the CPU. Moreover a hybrid approach, where loops are decomposed across both the CPU and NPU, delivers improved performance whilst still maintaining some energy efficiency benefits. ACKNOWLEDGMENTS This research was supported by an RSE personal research fellowship award number 3271. For the purposes of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. R EFERENCES [1] N. Brown, “Exploring the versal ai engines for accelerating stencilbased atmospheric advection simulation,” in Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2023, pp. 91–97. [2] M. Klaisoongnoen et al., “Evaluating versal ai engines for option price discovery in market risk analysis,” in Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2024, pp. 176–182. [3] E. Hunhoff et al., “Efficiency, expressivity, and extensibility in a closeto-metal npu programming interface,” in 2025 IEEE 33rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, 2025, pp. 85–94. [4] Rodriguez-Canal et al., “Fortran high-level synthesis: Reducing the barriers to accelerating hpc codes on fpgas,” in 2023 33rd International Conference on Field-Programmable Logic and Applications (FPL). IEEE, 2023, pp. 10–18. [5] (2024) Versal adaptive soc aie-ml architecture manual. [Online]. Available: https://docs.amd.com/r/en-US/am020-versal-aie-ml/Overview [6] M. Bouaziz et al., “A dataflow overlay for monte carlo multi-asset option pricing on amd versal ai engines,” in ISC High Performance 2025 Research Paper Proceedings (40th International Conference). Prometeus GmbH, 2025, pp. 1–12. [7] W. Zhang et al., “New filter2d accelerator on the versal platform powered by the ai engine,” in International Symposium on Advanced Parallel Processing Technologies. Springer, 2023, pp. 437–449. [8] M. Levental et al., “An end-to-end programming model for ai engine architectures,” in Proceedings of the 14th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies, 2024, pp. 135–136. [9] T. Kalkhof et al., “Enabling fpga and ai engine tasks in the hpx programming framework for heterogeneous high-performance computing,” in International Symposium on Applied Reconfigurable Computing. Springer, 2024, pp. 75–89. [10] N. Brown et al., “Seamless acceleration of fortran intrinsics via amd ai engines,” in Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2025, pp. 185–185. [11] N. Brown, “Fully integrating the flang fortran compiler with standard mlir,” in SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2024, pp. 939–949. [12] J. Huthmann et al., “OpenMP device offloading to FPGAs using the Nymble infrastructure,” in International Workshop on OpenMP. Springer, 2020, pp. 265–279. [13] G. Rodriguez-Canal et al., “An mlir pipeline for offloading fortran to fpgas via openmp,” 2025. [14] A. Brauckmann et al., “Tensorize: Fast synthesis of tensor programs from legacy code using symbolic tracing, sketching and solving,” in Proceedings of the 23rd ACM/IEEE International Symposium on Code Generation and Optimization, 2025, pp. 15–30. [15] N. Brown and G. Rodriguez-Canal, “Programmer productivity and performance on amd’s ai engines: Offloading fortran intrinsics via mlir a case-study,” 2025. [16] (2025) Tensor operator set architecture (tosa). [Online]. Available: https://www.mlplatform.org/tosa/tosa_spec.html [17] (2025) Aie automatic vectorization. [Online]. Available: https: //github.com/Xilinx/mlir-aie/blob/main/docs/AIEVectorization.md
[18] (2025) Iron api and mlir-based ai engine toolchain. [Online]. Available: https://github.com/Xilinx/mlir-aie [19] N. Brown et al., “A highly scalable met office nerc cloud model,” arXiv preprint arXiv:2009.12849, 2020.