1
T EMPUS: A Temporally Scalable Resource-Invariant GEMM Streaming Framework for Versal AI Edge
arXiv:2605.00536v1 [cs.DC] 1 May 2026
Mahdieh Grailoo∗ José Núñez-Yáñez† ∗ Linköping University, Sweden † Universidad Politecnica de Madrid, Spain ∗ [email protected], † [email protected]
Abstract—Scaling laws for Large Language Models (LLMs) establish that model quality improves with computational scale, yet edge deployment imposes strict constraints on compute, memory, and power. Since General Matrix Multiplication (GEMM) accounts for up to 90% of inference time, efficient GEMM acceleration is critical for edge AI. The Adaptive Intelligent Engines available in the AMD Versal adaptive SoCs are well suited for this task, but existing state-of-the-art (SOTA) frameworks maximize performance through spatial scaling, distributing workloads across hundreds of cores — an approach that fails on resource-limited edge SoCs due to physical implementation failures, bandwidth saturation, and excessive resource consumption. We propose T EMPUS, a Resource-Invariant Temporal GEMM framework for the AMD Versal AI Edge SoC. Rather than expanding hardware resources with matrix size, T EMPUS employs a fixed compute block of 16 AIE-ML cores, achieving scalability through iterative graph execution and algorithmic data tiling and replication in the Programmable Logic. High-speed cascade streaming ensures low-latency partial sum reduction at Initiation Interval (II) of 1, while a deadlock-free DATAFLOW protocol maximizes transfer-compute overlap and PLIO reuse. Evaluated on GEMM workloads, T EMPUS achieves 607 GOPS at 10.677 W total on-chip power. By characterizing systemlevel efficiency through the Platform-Aware Utility (PAU) metric, we prove that T EMPUS achieves a 211.2× higher prominence factor than the leading spatial SOTA (ARIES). Furthermore, the framework maintains a 0.00% utilization of URAM/DSP, yielding 22.0× core frugality, 7.1× power frugality, and a 6.3× reduction in I/O demand, establishing a sustainable, scalable foundation for edge LLM inference. Index Terms—Hardware-Software Co-design, Versal ACAP System-on-Chip, Intelligent Engine, Large Language Models, Temporal Scaling, Resource-Constrained Edge Devices, Parallelization, Sustainable Inference.
I. INTRODUCTION The scaling laws of Large Language Models (LLMs) have demonstrated that even with access to a large resource pool, temporal scaling is the only viable path for complete model deployments [1]–[3]. Therefore the unprecedented scale and computational demands of modern LLMs, require specialized hardware acceleration for deployments [4]–[9], particularly in the constrained edge devices [10]–[14]. In these models, the efficiency of General Matrix Multiplication (GEMM) is the central performance bottleneck of workloads, typically consuming over 90% of the total execution time during inference.
T EMPUS focuses on sustainable acceleration of rectangular GEMM on AMD Versal Adaptive Compute Acceleration (ACAP) Edge platforms which are heterogeneous System-onchips consisting of a processing system (PS), programmable logic (PL) and adaptable intelligent engines (AIE-ML) [15]– [17]. Prior state-of-the-art (SOTA) optimization frameworks designed for Versal ACAPs competed on achieving peak throughput by relying on massive spatial scaling, distributing the workload across hundreds of intelligent engines, typically found on larger devices like VC Versal Core and VE Versal, which host 300 to 400 cores [10]–[14], [18]–[20]. This approach fundamentally fails when ported to resource-limited edge devices. These designs require high core and resource utilization, leading to excessive power consumption and saturation of scarce PL components. This saturation confines the use of the PL fabric for integrating essential non-MM kernels (like Softmax or Layer normalization), needed for complete model inference [21]. Furthermore, pushing spatial limits often leads to physical implementation failures. Therefore, the inherent assumption that performance scales linearly with core count breaks down in this constrained context. To overcome the performance/resource mismatch at the edge, we introduce Temporal rectangular GEMM Scaling, a novel framework that achieves high performance and scalability by limiting and fixing hardware resource allocation, prioritizing efficiency over spatial parallelism. Our major contributions are as follows. 1) Resource-Invariant Frugality Framework: T EMPUS decouples resource utilization from matrix size by considering a fixed spatial compute block. The scaling for large workloads is achieved via iterative AIE-ML graph execution and algorithmic data replication. In addition, the 3D MatMul structure maps onto a fixed 2D array (e.g., Split×Cascade) using data reduction and multi-casting. Versus SOTA, T EMPUS achieves core, power, and I/O frugality. Also, by restricting programmable logic to lightweight streaming FIFOs and fixed-size tiling buffers, the architecture uses 0.00% of URAM/DSP, preserving fabric for non-GEMM kernels (e.g., Softmax, Layer-normalization) required by foundation models. 2) Platform-Aware Utility and Architectural Efficiency:
2
T EMPUS prioritizes architectural proficiency by normalizing performance against physical potential via the Platform-Aware Utility (PAU) metric, achieving a 211.2× higher prominence than the leading spatial SOTA. Evaluation demonstrates near-ideal scaling, where a 32,768× workload increase results in only a 6.8× latency growth, effectively amortizing fixed system initialization costs. We further show that efficiency is modulated by the micro-kernel dimension (DIM), with optimized tile sizes yielding a 10.5× latency reduction; a figure that can be further improved with additional local memory. 3) Compute-Transfer Overlapping Efficiency: High-speed streaming via a cascade interface enables low-latency partial sum reduction, avoiding ∼50% slower buffersharing methods while guaranteeing a pipeline initiation interval (II) of 1. To circumvent the edge “Bandwidth Wall,” we maximize PLIO reuse through hybrid packet/broadcast switching and a deadlockfree DATAFLOW protocol. PL streaming additionally overlaps computation with data transfer between programmable logic and AIE arrays, effectively hiding communication latency. 4) Analytical Modeling of Performance-Critical Parameters: Our work introduces analytical models that derive the parameters, governing system-level efficiency. These models determine scheduling parameters, such as GRAPH_ITER_CNT for temporal scaling, Kernel size for tiling, and the Replication Factor for data reuse. The source code for this framework is openly available at https://github.com/mgrailoo/Versal AI ML Engines GEMM. II. RELATED WORK: SPATIAL VS. TEMPORAL SCALING Prior GEMM acceleration on Versal ACAP evolved across two AI Engine generations, targeting maximized throughput via spatial scaling on large devices (300-400 cores). This philosophy fails on resource-limited edge platforms. Our work proposes temporal GEMM scaling as an alternative, resourceinvariant paradigm. A. Spatial Scaling Frameworks and Utilization Challenges (Gen 1: AIE) These frameworks focused on achieving maximal theoretical performance on large AIE arrays, prioritizing throughput over resource frugality. • CHARM & 2.0 (Heterogeneity-Aware Partitioning): CHARM pioneered GEMM on the AIE array using the Cascade Stream interface. The monolithic CHARM design suffered severe inefficiency with diverse layer sizes, suffering performance drops up to 5760×. CHARM 2.0 addressed this by partitioning the array into heterogeneous accelerators, improving BERT throughput by up to 5.29×. Using 288 AIE cores (72% of VCK1902), 91.52% BRAM and 82.94% URAM Utilization, it achieved 10.03 TOPS on a 10243 INT16 GEMM [10], [13]. CHARM also faced significant resource issues, leading to certain
INT8 designs utilizing only 48% of AIE cores due to congestion problems [13]. • M AX EVA (Throughput-Centric Optimization): M AX EVA addressed the small matrix bottleneck, encountered by prior solutions like CHARM, and achieved high AIE-only throughput in simulation. However, it suffered limitations of using inefficient buffer-sharing, and dedicated AIE cores to reduction kernels, capping real-world efficiency. The pursuit of maximum spatial utilization led to physical implementation failure. For example, M AX EVA’s initial highest-throughput design, which required 100% utilization of all 400 AIE cores, failed during Place-and-Route (PnR) due to routing congestion [20]. This simulation-focused approach provided theoretical performance, but lacked system implementation [20]. • AMA (Algorithmic Efficiency): AMA is an advanced successor to M AX EVA, that eliminated dedicated reduction kernels by augmenting MAC kernels to perform accumulation directly. This innovation yielded performance and energy efficiency gain. However, AMA maintained the same fundamental limitation: it relied on the slower buffer-sharing interface for reduction, and its AIE-only simulation approach isolated it from real-world constraints despite using up to 342 cores [22]. • AUTO MM (Resource-Conscious DSE): AUTO MM introduced a resource-conservative design space exploration (DSE) for INT8/INT16 precision optimization, built on CHARM’s methodology. Utilizing 288 AIE cores (72% of VCK1902), it achieved 7.51 TOPS with 56.8 W total power and lower BRAM utilization (49.33%) versus spatial alternatives. However, its conservative resource approach capped performance scalability, with ARIES later demonstrating 1.57× superior energy efficiency for INT16 [12]. B. Advanced Frameworks (Gen 2: AIE-ML) and CompilerAided Scaling As the architectural optimization space grew, compilation flows provided automated solutions to manage complex resource utilization patterns. • GAMA (AIE-ML Optimization): GAMA is the first study on second-generation of intelligent engines (AIEML) architecture [19], i.e. VE2802. Its innovation was a custom buffer placement algorithm that achieved up to 100% memory utilization, reducing stalls by 12% versus standard compilers. Using staggered kernel placement to mitigate congestion, it achieved array utilization and performance in simulation. Critically, GAMA employed the faster Cascade interface, achieving higher throughput efficiency than M AX EVA and ARIES. • ARIES (MLIR Compilation Flow): Introduced an agile MLIR-based flow for multi-level parallelism across Versal platforms [11]. Its core innovation was a unified MLIR representation spanning AIE and PL, enabling optimization and portability across AIE devices. Unlike simulation-based approaches, ARIES provided real onboard evaluation results. It achieved high throughput
3
PS
Programmable Logic
AXI-MM
AIE-ML Engine
Memory
AIE-ML Engine
AIE-ML Engine
Memory
AIE-ML Engine
Switch
Memory Switch
AXI-S
Async CDC Switch
PL Function
DMA
Versal ACAP offers: Intelligent Engines (AIE-ML) Adabtable Engines (PL) Scalar Engines (PS) AIE array communicates with: PL via AXI4-Streams PS via NoC DRAM via NoC Data communication: Cascade interface Memory Interface AXI4 Interface
Memory
Switch
Switch
NOC
AI-ML Engine Interface Tiles
Ext. DRAM
Figure 1: Versal ACAP Architecture: Heterogeneous System Integration and Execution Flow for our framework
through massive spatial scaling, utilizing 88% (352 cores) of AIEs with high PL resource usage (76% URAM), making it unsuitable for resource-constrained edge devices. • AUTO SA (Polyhedral Compilation): A polyhedral compiler generating monolithic systolic arrays with hardware optimizations (SIMD, II=1, double buffering) [14]. While achieving high performance on 16nm AMD U250 FPGA, CHARM outperformed it with 2.9× higher throughput for the same precision. In contrast to SOTA, our resource-invariant temporal scaling delivers performance through iterative execution, dimension reduction, and data replication. It is within a small, fixed core block using high-speed cascade and DATAFLOW streaming, ensuring resource conservation and edge compatibility. III. VERSAL ACAP ARCHITECTURE: T HE AI E DGE VE2302 The AMD Versal AI Edge VE2302 SoC integrates three distinct processing engines into a single heterogeneous architecture, as illustrated in Figure 1. The Intelligent Engines form a 34-core array of VLIW/SIMD processors (AIE-ML), each with local memory [23] (green-gray boxes), optimized for the deep learning compute kernels at the core of this architecture. The Adaptive Engines (Programmable Logic) (red box) provide the reconfigurable hardware (328K system logic cells, 464 DSPs), utilized for flexible logic and data movement, such as data streaming control (FIFOs), data tiling, de-tiling and replications. The Scalar Engines (Processing System) (blue box) incorporate dual-core Arm® Cortex-A72 and Cortex-R5F processors for system orchestration and general-purpose tasks [15]–[17]. The AIE-ML array interfaces with the broader system through two key paths. It connects directly to the PL via highspeed AXI4-Streams (PLIO), which is the primary conduit for feeding data into the array. Communication with the
Processing System (PS) and access to external DRAM are both facilitated through the high-bandwidth Network-on-Chip (NoC). Within the AIE-ML array itself, three specialized data communication mechanisms enable efficient computation, which are fundamental to our scaling methodology. The Cascade Interface provides direct, low-latency connections (512bit wide in AIE-ML) between adjacent cores for rapid partial sum reduction, facilitating our temporal scaling approach. The Memory Interface enables buffer sharing between neighboring cores, while the AXI4 Switch connects non-adjacent cores and is configured for efficient packet-switching and broadcasting. For the sake of simplicity, all subsequent explanation, and diagrams will consider a 2x2 AIE-ML core array. IV. METHODOLOGY: RESOURCE-INVARIANT TEMPORAL GEMM SCALING Our framework transforms large matrix multiplication into a predictable, iterative streaming process. By mapping the 3D MatMul (GEMM SIZE A × GEMM SIZE AB × GEMM SIZE B) onto a fixed 2D AIE-ML array (Split×Cascade, e.g., 2×8) and employing a constant set of PL resources exclusively for dataflow, we achieve Resource-Invariant performance. A. System Orchestration and Control Flow (PS Side) The coordination of the heterogeneous Versal ACAP and the dataflow for our Temporal Scaling framework is managed by the Processing System/Host CPU, which acts as the central orchestrator as shown in Figure 1. The process begins when the scalar engines initiate execution. Input matrices are stored in external DRAM. A dedicated DMA HLS kernel then manages data transfer from external DRAM to the AIE-ML array using a deadlock-free DATAFLOW design, ensuring a continuous, high-speed data stream. Data is transferred from external DRAM over the high-bandwidth NoC (blue arrows) using the
4
AXI4-MM protocol. From there, data is streamed into the AIE-ML array via the AXI4-Stream (AXIS) network (red arrows). Afterwards, data is streamed into the AIE-ML array via the AXI4-Stream (AXIS) network (red path). In this step, to maximize the reuse of scarce PLIO resources, specialized routing is employed (i.e., Broadcast circuit-switching and Packet Switching). While the fixed-core AIE-ML graph performs the matrix multiplication, internal AIE-to-AIE communication for partial sum reduction is handled by the high-speed, 512-bit Cascade Stream (dark red arrows in Figure 1) [16]. And the Memory Interface (green arrows) is used for local data access within each core’s memory. Finally, the resulting Matrix C streams back from the AIE-ML array through the PL, where the DMA kernel collects it and writes it back to External DRAM via the NoC (red path), completing the execution cycle. The detailed execution of the system is governed by a 7-phase timed control flow, orchestrated by the PS, in Algorithm 1. In Phase 0 (INIT), the host calculates the critical GRAPH_ITER_CNT parameter for temporal scaling. It allocates memory buffers for matrices A, B, and C using XRT’s aligned allocator, ensuring 4096-byte boundary alignment for optimal DMA performance. Phase 12 (DATA PREP, DEVICE INIT), load the generated PLIO streams and the hardware binary (.xclbin) onto the Versal device, initializing the AIE-ML array and PL kernel. Phase 3 (BUFFER CREATE) maps the allocated buffers to External DRAM, establishing host-device data channels. Phase 4 (DATA XFER HOST2DEV) transfers input matrices from host memory to device DRAM, instantiating the PL kernel and AIE graph. The core computation begins with Phase 5 (KERNEL LAUNCH), launching the kernel, followed by Phase 6 (CORE COMPUTATION) where temporal scaling is enacted through iterative AIE graph execution (gemm_aie_gr.run(GRAPH_ITER_CNT)), concurrent with PL kernel operation (dma krnl.wait())). Finally, the host synchronizes completion, and transfers results back to host memory. B. Algorithmic Data Preparation (Tiling, Data Decomposition, and 3D-to-2D Mapping) This phase outlines how the framework overcomes hardware limitations, through dimension reduction, precise tiling, and specialized data repetition, as shown in Figure 2. The dimensional reduction maps the 3D MatMul (GEMM SIZE A × GEMM SIZE AB × GEMM SIZE B) computation onto a fixed 2D core array (SPLIT × CASC LN cores), where CASC LN chains cores for GEMM SIZE AB-dimension reduction via cascade streams, and SPLIT defines parallel groups. The GEMM SIZE AB dimension is processed through temporal iteration, while the GEMM SIZE A and GEMM SIZE B dimensions are distributed spatially across the array. Figure 2 illustrates the hierarchical decomposition strategy. The split boundaries (dotted lines) represent horizontal division into parallel processing groups for temporal scaling, while the cascade paths denote vertical AIE-to-AIE communication
Algorithm 1 Host Application Execution Flow (7 Phases) 1: function MAIN(argc, argv) 2: PHASE 0: Configuration and Memory Setup 3: Calculate GRAPH_ITER_CNT for temporal scaling.
Load matrix data or generate test patterns. Allocate host memory using aligned allocator (4096byte aligned). 6: PHASE 1–4: Initialization and Setup 7: Initialize XRT device and load XCLBIN. 8: Create buffer objects and map them. 9: Transfer A and B to device. 10: Instantiate FPGA HLS kernel and AIE graph. 11: PHASE 5: Kernel Launch ▷ Start timer 12: Launch dma hls rhdl. 13: PHASE 6: CORE COMPUTATION 14: Run AIE graph for GRAPH_ITER_CNT times. 15: Wait for kernel. 16: Record compute total. ▷ Stop timer 17: PHASE 7: Output/Validation 18: Sync output. 19: Write output and validate. 20: Print summary. 21: end function 4: 5:
(governed by CASC LN) for partial sum reduction. The diagram shows large input matrices (A and B) converted into low-latency streams (a0 casc0, a0 casc1, b0 casc0, b0 casc1, b1 casc0, b1 casc1) organized hierarchically into Blocks (temporal units), Tiles (memory-defined by DIM), and Subtiles (vector units). At the block level, matrices decompose into sequential Blocks (e.g., ’block1’, ’block2’). Within blocks, data organize into Tiles (’tile1’ through ’tile4’) or micro-kernel corresponding to the DIM parameter. The maximum DIM is constrained by the AIE-ML core’s local memory capacity, partitioned between matrices. The smallest units are Subtiles (’subtile1’ through ’subtile8’), representing minimal data segments optimized for AIE-ML vector execution. For the sake of simplicity, all subsequent diagrams, and explanation will consider a minimal 2 × 2 AIE-ML core array, rectangular GEMM size of 32 × 16 × 32, DIM of 8, sub tile size of 4, and block size of 16, 8, and 16 for A, B and C, respectively. During PLIO Cascade Stream Generation, Matrix A tiles follow row-major ordering while Matrix B uses column-major ordering, aligning with the core cluster’s communication pattern. Matrix A replication occurs between tiles after each block processing cycle and Matrix B replication after each column within blocks. Subtile dimensions optimize for AIEML instruction efficiency, with physical dimensions adapting to DATA TYPE (e.g., 4×4×4 for int16 and int32), while maintaining row-major element serialization within subtiles. Algorithm 2 (PLIO Stream Generation), transforms large input matrices into sequential data streams for the fixed-core AIE-ML graph. Line 1 calculates the number of elements per 128-bit PLIO chunk, WRD LN. The number of temporal iterations required to process the full workload, defined by the graph iteration count:
5
cascade
split
Matrix A
Matrix B
tile1 subtile1
subtile2
subtile3
subtile4 3
tile3 subtile5
subtile6
subtile7
subtile8
split
b1_casc0
tile1 subtile1
subtile2
tile2 subtile5
subtile6
subtile3
subtile4
subtile7
subtile8
4
a0_casc0
a0_casc1
block1
block3
block1
block3
block2
block4
block2
block4
2
cascade b1_casc 4
b0_casc1 block3
block2
data Replication pattern Matrix A
b0_casc0 block1
a0_casc1
a0_casc0 block1
data Replication pattern Matrix B
b0_casc0
b0_casc1
b1_casc0
b1_casc1
tile1
tile3
tile5
tile7
tile1
tile3
tile5
tile7
tile2
tile4
tile6
tile8
tile2
tile4
tile6
tile8
c0
c1
split Matrix C c0 block1
c1 tile3
tile1 subtile1
subtile2
subtile9
subtile10
subtile3
subtile4
subtile11
subtile12 2
tile2 subtile5 subtile7
subtile6
tile4 subtile13
subtile14
subtile8
subtile15
subtile16
tile1
tile9
tile2
tile10
tile3
tile11
tile4
tile12
tile5
tile13
tile6
tile14
tile7
tile15
tile8
tile16
split 4
block3
GEMM_SIZE = 32 DIM = 8 SPLIT = 2 CASC_LN = 2 sub_tile Sizes = 4 Block Sizes = 16
GRAPH_ITER_CNT = 8 Replication Factor = 2
Figure 2: Hierarchical Data Decomposition and Stream Generation. Table I: Data Ordering Summary for Stream Generation Data Level A B C Elements within sub-tiles Row-major Row-major Row-major Sub-tiles within tiles Row-major Row-major Row-major Tiles within blocks Row-major Column-major Column-major
processing cycle, while Matrix B is replicated after processing each column within blocks, as illustrated in Figure 2. The subsequent loop (lines 5-9) processes the matrices using rowmajor and column-major ordering within this data repetition framework to maximize computational density. C. Hardware Pipelining and AIE-ML Graph Execution
GEMM SIZE A × GEMM SIZE B GRAPH ITER CNT = . (1) DIM A × DIM B × SPLIT
REPLICATION FACTOR A/B =
GEMM SIZE B/A . DIM B/A × SPLIT
(2)
Here, Matrix A is replicated between tiles after each block
This subsection describes the execution of the prepared data streams by the fixed-core AIE-ML graph, detailing the highspeed pipeline protocols that enable overlapping of efficient computation and data transferring. 1) AIE-ML Engine Graph (Array of AIE-ML Cores): Figure 3 illustrates the execution of the AIE-ML graph that
6
1
3
1
2
1
2
2
4
3
4
3
4
A 3
B
AXI4 stream interface: Broadcast Circuit Switching (Data Multicasting) Packet Switching
C
Cascade Interface (Data Reduction)
1
2
1
4
3
1
3 1
2 4
AIE
2
AIE
3
1
3
2 4
2 2
1
3 4
4
AIE
AIE
4
Figure 3: AIE-ML Cores Data Flow for our framework: Fixed AIE-ML Compute Block with Optimized I/O Architecture Algorithm 2 PLIO Stream Generation and Tiling Require: MatA, MatB; Config parameters (GEMM SIZE, DIM, SPLIT, CASC LN, DATA TYPE) Ensure: Cascade input streams (a0 casc*), (b* casc*) ▷ Elements per 1: WRD LN ← 128/DATA TYPE bits 128-bit PLIO chunk 2: GRAPH_ITER_CNT ← calculated via Equ. 1 3: Replication Factor ← calculated via Equ. 2 4: for each temporal block do 5: Matrix A Ordering: Process MatA tiles in row-major order. 6: Matrix B Ordering: Process MatB tiles in columnmajor order. 7: Apply algorithmic data repetition (replication factor) patterns. 8: PLIO formatting: ensure WRD LN elements per line. 9: end for
processes the prepared data streams. The diagram shows the graph’s parameterization by CASC LN (cascade levels) and SPLIT (parallel splits), with the AIE-ML array connected to the FPGA/PL kernel through PLIO interfaces. The figure demonstrates how high-speed streaming protocols enable efficient data movement while overlapping computations. The cascade stream chains—visible as horizontal connections between cores—provide 512-bit wide pathways (red arrows) for AIE-to-AIE partial sum reduction, generating c1 and c2 output streams. It directly implements the dimension reduction from 3D MatMul to 2D array by handling the GEMM SIZE ABdimension accumulation through chaining. This minimizes synchronization overhead to achieve Initiation Interval (II) of 1, avoiding the approximately 50% slower buffer sharing interface. Input distribution follows specialized routing patterns visible in the figure: Matrix A utilizes broadcast circuit-switching (shown as single source branching to multiple destinations (solid blue arrows)) to simultaneously route
Algorithm 3 AI Engine Graph Construction 1: Input: CASC_LN, SPLIT 2: function G E MM C ONSTRUCTOR 3: Instantiate mmult[SPLIT] 4: Create PLIO matA inp[CASC LN]
for i = 0 to SP LIT − 1 do Create PLIO matB inp[CASC LN] Create PLIO matC out[i] for k = 0 to CASC LN − 1 do runtime(mmult[i].kernels[k]) ← 1.0 Connect matA inp[k] to mmult[i].inA[k] 11: Connect matB inp[idx] to mmult[i].inB[k] 12: end for 13: Connect mmult[i].out to matC out[i] 14: end for 15: end function 5: 6: 7: 8: 9: 10:
a0 casc* input streams to all SPLIT groups, while Matrix B employs packet switching (depicted as time-multiplexed streams (dashed blue arrows)) to dynamically route different b* casc* input streams to different splits, maximizing reuse of the constrained VE2302’s PLIO resources. The AIE-ML graph construction and streaming connections are formalized in Algorithm 3. Lines 2-3 instantiate the core computation graph mmult[SPLIT]. Line 4 creates Matrix A PLIO interfaces, implementing the broadcast circuit-switched distribution. The nested loops (lines 5-16) establish all connections. Matrix A, broadcast to all splits (line 10), enables the data multi-casting. Matrix B split-specific connections in line 11 implement the packet-switched routing. In line 14, output collection gathers results through the cascade streams. Runtime ratios are set in line 8 for performance optimization, completing the graph construction for efficient MatMul execution. 2) FPGA/PL Kernel (dma hls): The dma_hls kernel orchestrates high-speed data transfers between Extrenal DRAM
7
Algorithm 4 Top-Level FPGA/PL Execution and Data Flow (DMA HLS) Require: Memory pointers: matA, matB, matC; Streams: strmInp Ensure: Streams: strmOut 1: function DMA HLS(matA, matB, matC, streams) 2: Constants: NUM A FILES = 8 3: Constants: NUM B FILES = 16 4: #pragma HLS DATAFLOW 5: for each memory read transaction do 6: #pragma HLS PIPELINE II=1 ▷ Enforce throughput 7: ReadData ← matX[i] ▷ 128-bit burst ▷ 8: stream idx ← imod NUM A FILES Sequential distribution 9: Write ReadData to strmOut[stream idx] 10: end for 11: out C(strmInp C, matC) ▷ Deadlock-free Matrix C collection (pairwise writes) 12: end function
memory and the fixed-core AIE ML array via PLIO streams. The kernel is implemented with Vitis HLS, and employs a deadlock-free dataflow design that adheres to PL Resource Conservation by using only lightweight streaming FIFOs, avoiding monolithic buffering, unlike SOTA [10], [13]. Algorithm 4 details the kernel’s high-speed execution. In the algorithm, memory pointers are mapped to NoC DDR4 interfaces through AXI4 Memory-Mapped streams. The toplevel DATAFLOW pragma, in Line 4, enables concurrent execution of input/output functions. Sequential distribution via modulo addressing in Line 8 enables efficient 128-bit burst data transfers to the AIE-ML array. The design preserves II=1 pipeline efficiency throughout the data path in Line 6.
Table II: Performance and Resource Utilization for 10243 INT16 GEMM in T EMPUS Metric
Value
Context
I. Performance and Timing (ms) AIE Cores Used Core Computation (tactual ) Achieved Throughput Device/XCLBIN Init Buffer Creation/Mapping Kernel/Graph Create Kernel Launch PL Tiling Graph Run/DMA Wait Output Sync PL performance
16 (47%) 3.537 607 GOPS 226.928 20.362 55.882 0.218 13.276 3.319 0.013 312.5
Fixed, Resource-Invariant Measured execution Derived from latency One-time setup Memory allocation Compilation overhead Launch kernel PL Tiling/Replication overhead Runtime scheduling Result collection PL performance
2.381 W 3.173 W 10.677 W 255 GOPS/W 56.87 GOPS/W
16 cores active B/XRAM + NoC-DDRMC Frugal consumption Core efficiency System efficiency
6.16% 62.58% 0.00% 0.00% 7.65%
PL capacity preserved Streaming FIFOs Resource conservation Resource conservation Resource conservation
II. Power and Energy AIE Engine Power Memory Power Total On-Chip Power Energy Eff. (AIE) Energy Eff. (Total) III. Resource Utilization LUT BRAM URAM DSP CLB Registers
includes the PL and DDR memory controller. This holistic implementation enables a direct comparison with SOTA frameworks that provide on-board results, while excluding simulation-only studies. Finally, power consumption was estimated using the AMD AIE-specific Xilinx Power Estimator (XPE) tool. VI. SIMULATION RESULTS AND SUSTAINABILITY ANALYSIS This section evaluates the performance and sustainability of the Resource-Invariant Temporal Scaling rectangular GEMM framework.
V. E NVIRONMENTAL S ETUP The framework is implemented on the AMD Versal AI Edge VE2302 ACAP (XCVE2302-1LSESFVA784-E), employing AIE-MLv1 architecture with PL components at 312.5 MHz, while limited PLIO resources constrain split (SPLIT) and cascade paths (CASC LN). A fixed spatial compute block of 16 AIE-ML cores for matrices’ workloads of (32 − 1024)3 with INT16/INT32 precisions are utilized. The 16 cores configuration is used because the area group’s 24 registered 128-bit PLIO channels can support it. The VE2302 ACAP architecture, identified in the compiler by __AIE_ARCH__ == 20, features native hardware support for INT4, INT8, and BFLOAT16 data types, in addition to INT16 and INT32. However, our evaluation was constrained by the available software support in the AMD Vitis™ 2024.1 toolchain. Specifically, our design relies on the templated matrix multiplication kernel from the AMD Xilinx DSP Library, which limited the scope of the numerical formats we could report for both simulation and hardware results. All designs were compiled with AMD Vitis 2024.1. We report full system throughput and resource utilization, which
A. System-Level Characterization: Performance, Power, and Resource Usage The operational metrics are detailed in Table II, representing a system implementation for the 10243 INT16 workload rather than AIE-only simulation. The framework achieves 607 GOPS with a core computation latency of 3.537 ms, and total on-chip power of 10.677 W. A detailed breakdown of the execution timeline reveals that the core computation is highly efficient, underscoring the design’s streaming and computation efficiency. Power analysis shows the AIE engines consume only 2.381 W, while memory subsystems (including B/XRAM and NoC-DDRMC) consume 3.173 W, representing a significant portion of the total 10.677 W on-chip power and confirming the I/O-bounded nature of the workload. B. Validation of Temporal Scaling and Workload Analysis The performance analysis validates the principle that scalability can be achieved through temporal iteration rather than physical core expansion, demonstrating the efficacy of the resource-invariant approach.
8
Table III: Tile Dimension (DIM) Scaling in T EMPUS for Fixed Workload (5123 ) in different data types Type
DIM
Latency (ms)
Throughput (GOPS)
INT16
4 8 16 32 64 128
6.194 3.230 1.811 1.123 0.792 0.586
43.338 83.107 148.225 239.034 338.934 458.081
INT32
4 8 16 32 64
11.848 6.171 3.225 1.779 1.150
22.657 43.500 83.236 150.891 233.422
Table IV: Workload scaling in T EMPUS with Maximum Available DIM in different data types Type
Size
DIM
Latency (ms)
Throughput (GOPS)
INT16
323 643 1283 2563 5123 7683 10243
16 32 64 128 128 64 64
0.396 0.389 0.395 0.407 0.586 1.637 3.537
0.165 1.348 10.618 82.443 458.081 553.433 607.148
323 643 1283 2563 5123 7683 10243
16 32 64 64 64 32 32
0.397 0.403 0.396 0.483 1.150 5.412 14.757
0.165 1.301 10.592 69.471 233.422 167.400 145.523
INT32
1) Tile Dimension Scaling: Table III reveals the critical relationship between the micro-kernel tile size (DIM) and computational efficiency for a 5123 workload. Increasing DIM from 4 to 128 improves throughput by 10.5×. This shows that providing more local memory per core would directly reduce latency by enabling larger micro-kernels. Theoretically, DIM could be increased to 256 to further improvement, however the local memory constraint per AIE-ML tile fundamentally caps the practical limit at DIM=128 for INT16. Precision scaling remains predictable: INT32 achieves 233.422 GOPS at its DIM=64 limit, roughly half the throughput of INT16, reflecting the hardware’s 2× data width penalty while confirming T EMPUS’s robust architectural proficiency within a fixed computational fabric. 2) Workload Scaling Analysis and Architectural Efficiency: Table IV characterizes T EMPUS’s temporal scaling across an increase in operations 32 768× (from 323 to 10243 ). T EMPUS amortizes fixed overheads, transitioning from suboptimal efficiency at 323 to a sustained 607 GOPS at 10243 (INT16). Key insights is that the 5123 workload achieves near-ideal scaling at DIM=128, but 10243 is confined to DIM=64, increasing iteration counts and causing a non-linear latency jump to 3.537ms. Notably, precision scaling remains predictable such that INT32 at 10243 is limited to DIM=32, delivering roughly one-quarter of INT16 throughput (145.5 GOPS) due to the 2× data width penalty. This adaptive scaling proves T EMPUS automatically respects hardware boundaries, delivering predictable, sustainable performance for real-time edge AI.
Table V: PL Resource Utilization and Power Consistency of T EMPUS for INT16, URAM/DSP utilization is 0.00% across all workloads Workload
Total On-Chip Power(W) LUT(%) BRAM(%) CLB Regs(%)
3
32 643 1283 2563 5123 7683 10243
10.698 10.639 10.315 10.692 10.661 10.631 10.677
6.09 6.11 6.13 6.11 6.18 6.20 6.16
62.58 62.58 62.58 62.58 62.58 62.58 62.58
7.63 7.64 7.65 7.64 7.65 7.67 7.65
8 × 32 × 8 128 × 768 × 64 512 × 64 × 512 512 × 1024 × 512 128 × 768 × 3072 768 × 3072 × 768 8 × 1024 × 1024 8 × 2048 × 2048 8 × 4096 × 4096
10.701 10.236 10.281 10.680 10.721 10.788 10.282 10.703 10.715
6.11 6.14 6.11 6.17 6.17 6.18 6.15 6.15 6.19
62.58 62.58 62.58 62.58 62.58 62.58 62.58 62.58 62.58
7.64 7.66 7.64 7.65 7.66 7.68 7.65 7.65 7.65
C. Resource and Power Invariance Table V demonstrates that T EMPUS maintains strict resource invariance across exponential workload growth. Total on-chip power stays frugal at ∼10.6 W, and critical PL resources (DSP, URAM) remain at 0.00% utilization. This contrasts sharply with SOTA spatial designs, which saturate resources (e.g., CHARM 2.0 uses 82.94% of URAM). Low LUT and BRAM usage—due only to lightweight streaming FIFOs (FIFO depth=16, outstanding=32, BURST=32), preserves PL capacity for heterogeneous orchestration of essential kernels like Softmax and LayerNorm in complete model pipelines. Further reductions in FIFO depth, outstanding transactions, and burst size could yield even greater BRAM savings. VII. C OMPARATIVE S USTAINABILITY AND R ESOURCE F RUGALITY We evaluate the resource-invariant T EMPUS framework against spatial SOTA frameworks (ARIES, CHARM 2.0, AUTO MM, AUTO SA) in Table VI. These baselines prioritize peak throughput on high-end Versal devices (VCK190/VE2802, ∼300−400 cores), which differ from T EMPUS’s VE2302 in compute efficiency, and memory capacity (see Table VII). However, they fail in edge-class SoCs because reducing core counts violates their assumptions about spatial parallelism, leading to compilation failure. To enable fair comparison despite hardware asymmetry, we introduce platform-aware metrics that decouple algorithmic efficiency from absolute resource budgets. A. Platform-Aware Utility (PAU(n)) Architectural proficiency is evaluated by measuring the extracted computational work relative to the total physical potential and resource footprint of the deployment platform (cores, power, I/O, and peak throughput). It rewards designs that perform well on resource-constrained boards and penalizes brute-force spatial arrays. PAU is defined as: P AU =
TOPS Cores × Power (W) × PLIO × Theoretical Peak (Pk)
9
Table VI: Comprehensive Comparative Analysis of Throughput, Power, and Platform-Aware Utility for 10243 INT16 GEMM [10]–[14] Framework
Cores
Lat.(ms)
TOPS
Pwr(W)
U%(1)
PLIO
T/C(2)
T/P(3)
C-Fru(4)
P-Fru(5)
I-Fru(6)
PAU(n)(7)
T EMPUS (Temporal) ARIES (Spatial) CHARM 2.0 (Spatial) AUTO MM (Spatial) AUTO SA (Spatial)
16 352 288 288 –
3.537 0.1354 0.2141 0.2859 0.6298
0.607 15.86 10.03 7.51 3.41
10.677 76.30 64.80 56.80 84.90
0.00 76.03 82.94 82.94 –
26 164 120 120 –
0.038 0.045 0.035 0.026 –
0.057 0.208 0.155 0.132 0.0401
22.0× 1.0× 1.2× 1.2× –
7.1× 1.0× 1.1× 1.3× –
6.3× 1.0× 1.4× 1.4× –
211.2× 1.0 1.2× 1.1× –
Notes: (1) URAM% Utilization. (2) TOPS/Core density. (3) TOPS/Power efficiency (AI Efficiency). (4) C-Fru: Core Frugality. (5) P-Fru: Power Frugality, (6) I-Fru: I/O Frugality (PLIO). (7) Platform-Aware Utility Factor.
Table VII: Compute, Resource & Power Strengths Comparison Feature
VCK190 (VC1902)
VE2802
VE2302
1st Gen AI Engine [16] 400 64 TOPS 0.71–1.28
AIE-ML v2 [15] 304 101 TOPS 2.69
AIE-ML [17] 34 11.5 TOPS 1.15–1.53
100–180 W
Up to 75 W
15–20 W
1,968K 1,968
1,139K 1,312
328K 464
8 GB @ 3200 Mb/s 8 GB @ 3900 Mb/s
Up to 16 GB 12 GB @ 3733 Mb/s (192-bit)
4 GB (64-bit, upgradable to 8 GB) 4 GB (64-bit)
AI Compute & Efficiency AI Engine Type Cores Peak AIE INT16 Performance AI Efficiency (TOPS/W) Power & Thermal Total Chip Power (TCP)
Programmable Logic & Resources System Logic Cells DSP Engines External Memory Support DDR4 Support LPDDR4 Support
General Notes: INT16 performance inferred from INT8 specifications ( 12 × INT8). AI Efficiency = Peak INT8 TOPS ÷ Max TCP.
To highlight T EMPUS’s architectural advantage, we define the Platform-Aware Utility Factor n = P AUother /P AUARIES , where n > 1 indicates higher utility than the SOTA ARIES baseline. Table VI shows that although T EMPUS has higher absolute latency (3.537 ms on VE2302 vs. 0.135 ms on VCK190 for ARIES), it achieves a 211.2× higher utility factor, avoiding the utilization collapse inherent in rigid spatial architectures. B. Resource-Invariant Frugality and Heterogeneous Orchestration The sustainable execution of T EMPUS is characterized through its multi-dimensional frugality across core, power, and I/O domains. These metrics are defined as: C-Fru =
Coresother , CoresT EMPUS
P-Fru =
I-Fru =
PLIOother PLIOT EMPUS
Powerother PowerT EMPUS
As detailed in Table VI, T EMPUS achieves 22.0× Core Frugality, 7.1× Power Frugality, and 6.3× I/O Frugality relative to the ARIES baseline. While spatial scaling SOTA frameworks reach an architectural dead end on compact devices by saturating 76%–83% of on-chip URAM for a single GEMM kernel, T EMPUS maintains 0.00% URAM/DSP utilization components. The I/O advantage is equally significant and invariant. Our hybrid streaming approach—combining packet switching for dynamic time-multiplexing of data streams with cascade streaming that eliminates redundant I/O for intermediate results—ensures that I/O constraints never limit scalability. This
resource invariance is essential for heterogeneous orchestration, as it preserves the programmable logic fabric and memory resources for concurrent integration of other critical model kernels (e.g., Softmax, Layer Normalization). C. Normalized Computational Efficiencies (T /C, T /P): Throughput per core (T /C) and performance per watt (T /P) are reported to characterize computational density and AI efficiency within strict thermal boundaries: T /C =
TOPS , Cores
T /P =
TOPS Power (W)
Despite operating with a hardware budget nearly 6× lower than the spatial reference (11.5 vs. 64.0 Peak TOPS), T EMPUS achieves competitive T /P and maintains high computational density. This efficiency ensures sustainable, energy-sensitive execution for foundation models where traditional massive parallelism would lead to thermal failure on edge devices. D. Shape-Agnostic Efficiency Across LLM Architectures To demonstrate generality, we evaluate T EMPUS on representative rectangular GEMM shapes from LLM components: decoding projection layers, multi-head attention, and feed-forward networks (FFNs). Table VIII compares each rectangular workload against a cubic shape whose latency matches the rectangular one. The results show that T EMPUS maintains predictable, high efficiency across unbalanced dimensions where spatial frameworks suffer “utilization collapse” and performance drops up to 5760×.
10
Table VIII: Shape-Agnostic TEMPUS Performance: Rectangular GEMMs and Timing-Equivalent Cubic Workloads (INT16) Architectural Role
Rectangular GEMM
Rectangular Latency (ms)
Cubic Equivalent
Cubic Latency (ms)
1923 (DIM = 96) 7683 (DIM = 64) 10243 (DIM = 64)
0.403 1.637 3.537
323 (DIM = 4) 1923 (DIM = 96) 2563 (DIM = 128) 1283 (DIM = 64)
0.396 0.403 0.407 0.395
7683 (DIM = 64) 5123 (DIM = 128) 12163 (DIM = 32)
1.637 0.586 9.907
Decoding Phase (Attention Projection Layers) (narrow shapes) Small/Mobile LLM (e.g., Pythia/MobileLLM) Small/Mobile LLM (TinyLlama/Gemma) Production LLM (LLaMA-2 7B)
8 × 1024 × 1024 (DIM = 128) 8 × 2048 × 2048 (DIM = 64) 8 × 4096 × 4096 (DIM = 32)
Tiny head (experimental) BERT-base single head [24] Attention score matrix (seq=512) [25] Vision Transformer (ViT) head
8 × 32 × 8 (DIM = 4) 128 × 768 × 64 (DIM = 64) 512 × 64 × 512 (DIM = 128) 128 × 128 × 128 (DIM = 64)
BERT-base FFN Up-projection [24] Production-scale mid-size [26] BERT-base FFN expansion [24]
128 × 768 × 3072 (DIM = 96) 512 × 1024 × 512 (DIM = 64) 768 × 3072 × 768 (DIM = 16)
0.604 1.527 5.241
Attention Head Logic (fragmented shapes) 0.394 0.394 0.446 0.395
Feed-forward networks (FFN) (wide shapes)
a) Decoding phase (narrow shapes): In LLMs, while training utilizes massive batches of 2 million tokens [27] to saturate hardware, mobile deployment requires efficiency at GEM M SIZE A ≤ 8. Therefore, the decoding phase models a single-token inference step where the input is a single vector [28]. For Mobile LLMs like TinyLlama-1.1B [27], and Pythia-410M, and Gemma-2B [29], the hidden dimension is 1024, and 2048. Similarly, the larger LLaMA-2 7B has a (8×4096×4096) projection. As tabulated, the small difference between rectangular and cubic comes from a reduction in micro-kernel tile size (DIM) from the optimal DIM=128 to DIM=64, forced by the local memory limit, not a fundamental inefficiency. b) Attention heads (fragmented shapes): Multi-head attention splits embeddings into smaller, fragmented heads [25]. This creates rectangular GEMMs ranging from tiny experimental shapes (8×32×8) to standard BERT-base single heads (128 × 768 × 64). Fragmented heads nearly identical to the cubic equivalent. This proves T EMPUS is resilient to narrow dimensions that cause massive spatial arrays to underutilize resources. c) Feed-forward networks (wide shapes): The feedforward network (FFN) creates ”wide” or ”tall” rectangular GEMMs during up-projections and expansions. For BERTBase, the up-projection of 128 × 768 × 3072 and the full expansion of 768 × 3072 × 768 represent production-scale workloads [24], [26]. These results confirm that T EMPUS is shape-agnostic: it extracts high utility from rectangular LLM workloads without the architectural mismatches of spatial accelerators. VIII. CONCLUSION This work introduces T EMPUS, the first resource-invariant GEMM streaming framework for the Versal AI Edge VE2302. Mapping 3D MatMul onto a fixed 2D array, it achieves 607 GOPS using only 16 AIE-ML cores via algorithmic iteration. Versus spatial SOTA (ARIES), T EMPUS delivers a 211.2× higher PAU prominence factor, formally justifying its architectural superiority for edge deployment. The framework ensures sustainable performance through multi-dimensional frugality (22.0× core, 7.1× power, 6.3× I/O). Critically,
1.258 1.147 19.674
resource utilization remains invariant across workloads at 10.677 W. This conservation preserves programmable logic for heterogeneous orchestration of non-GEMM kernels (Softmax, Layer-normalization) required for foundation model inference. R EFERENCES [1] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. Casas, L. A. Hendricks, J. Welbl, A. Clark et al., “Training compute-optimal large language models,” arXiv preprint arXiv:2203.15556, vol. 10, 2022. [2] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361, 2020. [3] T. Pearce and J. Song, “Reconciling kaplan and chinchilla scaling laws,” arXiv preprint arXiv:2406.12907, 2024. [4] W. Xu, H. Choi, P.-k. Hsu, S. Yu, and T. Simunic, “Slim: A heterogeneous accelerator for edge inference of sparse large language model via adaptive thresholding,” ACM Transactions on Embedded Computing Systems, 2025. [5] F. Jiang, C. Pan, L. Dong, K. Wang, M. Debbah, D. Niyato, and Z. Han, “A comprehensive survey of large ai models for future communications: Foundations, applications and challenges,” arXiv preprint arXiv:2505.03556, 2025. [6] C. Guo, F. Cheng, Z. Du, J. Kiessling, J. Ku, S. Li, Z. Li, M. Ma, T. Molom-Ochir, B. Morris et al., “A survey: Collaborative hardware and software design in the era of large language models,” IEEE Circuits and Systems Magazine, vol. 25, no. 1, pp. 35–57, 2025. [7] Y. Li, S. Zhang, Y. Zeng, H. Zhang, X. Xiong, J. Liu, P. Hu, and S. Banerjee, “Tiny but mighty: A software-hardware co-design approach for efficient multimodal inference on battery-powered small devices,” arXiv preprint arXiv:2510.05109, 2025. [8] J. Nunez-Yanez and H. M. Jeddi, “Sgrace: scalable architecture for on-device inference and training of graph attention and convolutional networks,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2025. [9] M. Grailoo and J. Nunez-Yanez, “Heterogeneous edge computing for molecular property prediction with graph convolutional networks,” Electronics, vol. 14, no. 1, p. 101, 2024. [10] J. Zhuang, J. Lau, H. Ye, Z. Yang, S. Ji, J. Lo, K. Denolf, S. Neuendorffer, A. Jones, J. Hu, Y. Shi, D. Chen, J. Cong, and P. Zhou, “Charm 2.0: Composing heterogeneous accelerators for deep learning on versal acap architecture,” ACM Transactions on Reconfigurable Technology and Systems, vol. 17, Sep 2024. [11] J. Zhuang, S. Xiang, H. Chen, N. Zhang, Z. Yang, T. Mao, Z. Zhang, and P. Zhou, “Aries: An agile mlir-based compilation flow for reconfigurable devices with ai engines,” in Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA ’25), ser. FPGA ’25. Association for Computing Machinery, 2025, pp. 92–102. [12] D. Pal, Y.-H. Lai, S. Xiang, N. Zhang, H. Chen, J. Casas, P. Cocchini, Z. Yang, J. Yang, L.-N. Pouchet et al., “Accelerator design with decoupled hardware customizations: benefits and challenges,” in Proceedings
11
of the 59th ACM/IEEE Design Automation Conference (DAC). ACM, 2022, pp. 1351–1354. [13] J. Zhuang, J. Lau, H. Ye, Z. Yang, Y. Du, J. Lo, K. Denolf, S. Neuendorffer, A. Jones, J. Hu, D. Chen, J. Cong, and P. Zhou, “Charm: Composing heterogeneous accelerators for matrix multiply on versal acap architecture,” in Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2023. [14] J. Wang, L. Guo, and J. Cong, “Autosa: A polyhedral compiler for high-performance systolic arrays on fpga,” in Proceedings of the 2021 ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA ’21). ACM, 2021, pp. 93–104. [15] AMD, “Versal ai edge series gen 2 product selection guide,” Advanced Micro Devices, Inc., Tech. Rep., 2024, product Selection Guide. [Online]. Available: https://www.eetasia.com/wp-content/uploads/sites/ 2/2024/07/16 versal-ai-edge-gen2-psg.pdf [16] ——, AI Engine Kernel and Graph Programming Guide (UG1079), Advanced Micro Devices, Inc., document ID: UG1079. [Online]. Available: https://docs.amd.com/r/en-US/ug1079-ai-engine-kernel-coding [17] Xilinx, “Acap at the edge with the versal ai edge series,” Xilinx / AMD, Tech. Rep. WP518, v1.0, Jun. 2021, white Paper. [Online]. Available: https://docs.amd.com/api/khub/ documents/Xz0szg2HiN1YFYfaJVXcrQ/content?Ft-Calling-App= ft%2Fturnkey-portal&Ft-Calling-App-Version=4.1.3&filename= wp518-ai-edge-intro.pdf [18] D. Danopoulos, E. Lupi, C. Sun, S. Dittmeier, M. Kagan, V. Loncar, and M. Pierini, “Aie4ml: An end-to-end framework for compiling neural networks for the next generation of amd ai engines,” arXiv preprint arXiv:2512.15946, 2025. [19] K. M. Mhatre, E. Taka, and A. Arora, “Gama: High-performance gemm acceleration on amd versal ml-optimized ai engines,” ArXiv preprint: 2504.09688v3, 2025. [20] E. Taka, A. Arora, K.-C. Wu, and D. Marculescu, “Maxeva: Maximizing the efficiency of matrix multiplication on versal ai engine,” arXiv preprint arXiv:2311.04980 [cs], Nov 2023. [21] M. Grailoo, T. Nikoubin, O. Gustafsson, and J. Nunez-Yanez, “Activation function integration for accelerating multi-layer graph convolutional neural networks,” in 2024 IEEE 17th Dallas Circuits and Systems Conference (DCAS). IEEE, 2024, pp. 1–6. [22] X. Deng, S. Wang, T. Gao, J. Liu, L. Liu, and N. Zheng, “Ama: An analytical approach to maximizing the efficiency of deep learning on versal ai engine,” in 2024 34th International Conference on FieldProgrammable Logic and Applications (FPL), 2024. [23] J. Lei and E. S. Quintana-Ortı́, “Mapping parallel matrix multiplication in gotoblas2 to the amd versal acap for deep learning,” in Proceedings of the 4th Workshop on Performance and Energy Efficiency in Concurrent and Distributed Systems, 2024, pp. 1–8. [24] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), 2019. [25] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [26] M. V. Koroteev, “Bert: a review of applications in natural language processing and understanding,” arXiv preprint arXiv:2103.11943, 2021. [27] P. Zhang, G. Zeng, T. Wang, and W. Lu, “TinyLlama: An open-source small language model,” arXiv preprint arXiv:2401.02385, 2024. [28] T. Dao, “Flashattention-2: Faster attention with better parallelism and work partitioning,” arXiv preprint arXiv:2307.08691, 2023. [29] Gemma Team, “Gemma: Open models based on gemini technology,” arXiv preprint arXiv:2403.08295, 2024.