LP-GEMM: Integrating Layout Propagation into GEMM Operations César Guedes Carneiro, Lucas Alvarenga, Guido Araujo, Sandro Rigo Instituto de Computação, Universidade Estadual de Campinas, Campinas – SP, Brazil
arXiv:2604.04599v1 [cs.DC] 6 Apr 2026
Emails: [email protected], [email protected], {guido, srigo}@unicamp.br
Abstract—In Scientific Computing and modern Machine Learning (ML) workloads, sequences of dependent General Matrix Multiplications (GEMMs) often dominate execution time. While state-of-the-art BLAS libraries aggressively optimize individual GEMM calls, they remain constrained by the BLAS API, which requires each call to independently pack input matrices and restore outputs to a canonical memory layout. In sequential GEMMs, these constraints cause redundant packing and unpacking, wasting valuable computational resources. This paper introduces LP-GEMM, a decomposition of the GEMM kernel that enables packing-layout propagation across sequential GEMM operations. This approach eliminates unnecessary data repacking while preserving full BLAS semantic correctness at the boundaries. We evaluate LP-GEMM on x86 (AVX-512) and RISC-V (RVV 1.0) architectures across MLP-like and Attention-like workloads. Our results show average speedups of 2.25x over OpenBLAS on Intel x86 for sequential GEMMs and competitive gains relative to vendor-optimized libraries such as Intel MKL. We demonstrate the practicality of the approach beyond microbenchmarks by implementing a standalone C++ version of the Llama-3.2 inference path using exclusively BLAS-level GEMM calls. These results confirm that leveraging data layout propagation between operations can significantly boost performance. Index Terms—Compilers, Machine Learning, High Performance Computing
I. I NTRODUCTION Physics [1], chemistry [2], and protein modeling [3], as well as novel Machine Learning (ML) systems [4]–[6], are examples of applications that leverage High-Performance Computing (HPC) techniques to make such problems tractable on modern computer systems. These applications involve a large volume of data typically represented as matrices, enabling the use of linear algebra to compute the outcomes of their mathematical models. The computation in such applications is based either on a single Matrix Multiplication (MM), a sequence of dependent MM operations, or a batch of independent MM operations (batched-MM). Furthermore, operations not immediately related to MM, such as convolutions, can be correctly computed by MM operations if the data is properly reorganized (i.e., packed) before execution. In computing, Matrix Multiplication (MM) is generally implemented as the General Matrix Multiply (GEMM) operation found in linear algebra libraries. It generalizes the MM operation with an additional scaling parameter α and a scaling-accumulation parameter β. Due to its relevance, various optimization approaches targeting different aspects of
computer systems have been explored. For example, [7]–[9] offloads MM operations to specialized hardware architectures; [10], [11] investigates code-level optimization techniques applied to naive implementations; and there are also numerous algorithm-level investigations of MM operations [12]. Furthermore, linear algebra libraries are typically vendor-dependent. Hardware vendors, such as Intel (MKL [13]) and NVIDIA (cuBLAS [14]), provide closed-source libraries with routines highly optimized for their hardware, often compliant with the Basic Linear Algebra Subprograms (BLAS) Application Programming Interface (API). The OpenBLAS 1 library is one of the most famous opensource efforts to provide highly optimized linear algebra routines for different architectures. In particular, it inherits from the Goto and Van de Geijn [10] GEMM optimization approach, which provides a mathematical analysis that directs a sequence of tiling, packing, and loop unrolling optimization steps over the naive MM kernel to utilize the underlying architecture more effectively. However, the BLAS API makes no assumptions about subsequent operations, simply expecting the library to provide a highly optimized implementation for the current operation. Consequently, the layout of the input data in memory must be preserved in the operation’s output for consistency. Nevertheless, linear algebra workloads are rarely composed of a single operation. The Multi-Layer Perceptron (MLP) and the Transformer modules of recent ML models are examples of workloads consisting of a sequence of dependent MM operations. During execution, each MM in these modules will individually execute all steps of the library’s optimized GEMM kernel (i.e., tiling, packing, and loop unrolling). In particular, this work observes that when MM operations are executed sequentially, redundant packing steps are performed for each of them, as shown in Figure 1a. Therefore, we propose the Layout Propagation GEMM (LP-GEMM), a decomposition of the OpenBLAS GEMM kernel into three specialized kernels that avoid executing redundant packing steps. Specifically, the GEMM operations are divided into: (1) the Initial GEMM (ini-GEMM), which performs packing but propagates the layout; (2) the intermediate GEMM (midGEMM), which receives the data already packed from iniGEMM and retains this layout for the next operations; and (3) the Ending GEMM (end-GEMM), which unpacks the data into the original layout. Figure 1b shows how LP-GEMM can 1 https://github.com/OpenMathLib/OpenBLAS
OpenBLAS Input Pack
μKernel Redundant
FMA
Algorithm 1 Naive GEMM Implementation
OpenBLAS Input Pack
Unpack
μKernel FMA
Unpack
(a) Consecutive GEMM executions using OpenBLAS. The illustration depicts only two GEMMs, but the same pattern generalizes to an arbitrary number of consecutive operations. LP-GEMM (Ini)
LP-GEMM (Mid)
LP-GEMM (Mid)
LP-GEMM (End)
Input Pack μKernel
μKernel
μKernel
μKernel
FMA
FMA
FMA
FMA
Unpack
(b) Consecutive GEMM executions using LP-GEMM. The figure illustrates a scenario with more than three GEMMs, where intermediate kernels (MID) are used. When only two GEMMs are executed, only the INIT and END kernels are required.
Fig. 1: Comparison of OpenBLAS and LP-GEMM kernel execution of consecutive GEMM operations. The relative block sizes do not represent execution time. Weight packing is omitted for clarity.
simplify the BLAS GEMM kernel, removing unnecessary operations by leveraging layout propagation between operations. LP-GEMM was evaluated on x86 (AVX-512) and RISCV (RVV 1.0) platforms with their vector extensions enabled, resulting in average performance speedups of up to 2x for x86 and up to 5x for RISC-V for Attention-like workloads. For isolated GEMMs extracted from real-world applications, LP-GEMM achieved results comparable to those of highly specialized, vendor-distributed BLAS libraries for x86, and a considerable speedup in relation to other state-of-the-art approaches in RISC-V. The main contributions of this work are as follows: The LP-GEMM decomposition of the OpenBLAS GEMM operation into new kernel operations, named iniGEMM, mid-GEMM, and end-GEMM, which leverage data layout propagation. • A standalone implementation of LLaMA 3.2 [5] in C++ using the BLAS API for GEMM operations 2 . • The implementation of other common matrix operations in ML workloads (i.e., Softmax, RoPE, RMSNorm) on AVX-512, using the LP-GEMM propagated data layout.
•
The paper is structured as follows. First, the background of this work is discussed in Section II. Then, our LP-GEMM approach is described in Section III. Section IV describes how layout propagation was integrated into the Attention layer. LPGEMM experimental results are presented and discussed in Section V. A review of the literature is given in Section VI. Finally, concluding remarks are provided in Section VII. 2 https://github.com/czariv/simple attention
Require: Matrices A ∈ Rm×k , B ∈ Rk×n , C ∈ Rm×n , scalars α, β Ensure: C ← αAB + βC 1: for i = 0 to m − 1 do 2: for j = 0 to n − 1 do 3: Ci,j ← β · Ci,j 4: for l = 0 to k − 1 do 5: Ci,j ← Ci,j + α · Ai,l · Bl,j 6: end for 7: end for 8: end for
II. BACKGROUND This section provides the necessary background to understand the proposed LP-GEMM approach. A. GEneral Matrix Multiplication – GEMM GEMM, as described in Equation 1, refers to a standard matrix multiplication (MM) of two operands A ∈ Rm×k and B ∈ Rk×n that produces a matrix C ∈ Rm×n , extended with two additional scaling parameters α and β. C = αA × B + βC
(1)
When translating a GEMM computation into code, a naive implementation may use a three-level loop nest that iterates over the m, n, and k dimensions (Algorithm 1). However, such an implementation is not optimal. It fails to effectively exploit the available memory hierarchy, causing the operands to be loaded multiple times with little to no data reuse. For example, if matrix B is stored in row-major format, almost every access to its elements can result in a cache miss. There are multiple ways to improve the naive implementation. When possible, computation may be offloaded to specialized hardware [9], [15] or optimized using HPC techniques to enhance single-core data reuse and multi-core parallelism. Goto and Van de Geijn [10] revolutionized matrix multiplication optimization with GotoBLAS by systematically analyzing GEMM’s computational intensity and its interaction with the memory hierarchy, and by organizing the computation around hardware-driven design principles. A key contribution of this approach is the systematic adoption of an outerproduct formulation as the organizing principle for registerlevel computation, where GEMM is expressed as a sequence of rank-1 updates that accumulate the product of a column of A and a row of B into tiles of C, enabling partial results to remain resident in registers across multiple updates. This outer-product organization aligns naturally with modern processors that provide hardware support for Fused Multiply-Add (FMA) instructions. By expressing the update C ← C + A:,l × Bl,: using FMA operations on small registerresident tiles, GotoBLAS maximizes arithmetic throughput while minimizing instruction overhead, while also exposing opportunities for register blocking and vectorization. These
nc
N nr
Bt
B
kc
kr
K
Bt
kc
K
kr
A
mr
C
At
Ct
kc
nc
mc
M
(a) GotoBLAS’s outer tiling. Kernel
At
Ct
mc
(b) GotoBLAS’s inner tiling. micro-kernel (µKernel)
1: for j = 0 to N step nc do 2: for l = 0 to K step kc do 3: Pack mc rows of AT into sa 4: for jj = j to j + nc step nr do
Pack nr columns of BT into sb CT ← AT · BT {µKernel w/ sa, sb} end for for i = mc to M step mc do Pack next AT into buffer sa CT ← AT · BT {µKernel w/ sa, sb} 11: end for 12: end for 13: end for 5: 6: 7: 8: 9: 10:
(c) GotoBLAS’s algorithm.
Fig. 2: Visual representation of GotoBLAS approach [10]. (a) shows how matrices are tiled for the architecture’s memory hierarchy, while (b) depicts how each block At and Bt is further tiled and packed for better usage of the processor’s registers during CT computing of µKernel. Finally, (c) shows the final GotoBLAS GEMM algorithm. principles form the foundation of the highly optimized computation structures employed in GotoBLAS and later adopted by modern BLAS implementations. For simplicity, we assume α = 1 and β = 0 for the remainder of the paper. a) Tiling: Loop tiling, or blocking, is used to optimize nested loops. It enhances data locality within the memory hierarchy by adjusting the iteration space to operate on smaller blocks rather than the entire data set at once. To accomplish this, new outermost loop levels are created to iterate over blocks, and the iteration space of the original element-level loops is adjusted to iterate over the elements within each block [16], [17]. b) Packing: Packing optimizes memory hierarchy usage by buffering tiled blocks into contiguous memory regions and reordering their elements to match the execution stride. This minimizes Translation Lookaside Buffer (TLB) misses and reduces cache conflict misses [18]. c) Loop Unrolling: Loop unrolling replicates the body of a loop multiple times within a single iteration. This reduces loop control overhead and exposes more independent instructions to the processor, thereby improving instructionlevel parallelism and instruction scheduling [19]. Fig. 2 shows the GotoBLAS GEMM optimization strategy. The process starts with the outer tiling of the GEMM operation, referred to as the kernel (Fig. 2a), which tiles all MMrelated dimensions using parameters nc , mc , and kc derived from the processor’s memory hierarchy, and parameters nr , mr , and kr derived from register sizes. Subsequently, as shown in Figure 2b, a micro-kernel (µKernel) computes a given CT tile from blocks AT and BT . During computation, the AT and BT tiles are first packed (as indicated by the arrows in Figure 2b) into contiguous temporary buffers, thereby placing them at the appropriate level of the memory hierarchy. Finally, the CT results are generated according to this packed layout, requiring an unpacking step before they are written back to
memory. This approach is also detailed in the algorithm in Figure 2c. B. Basic Linear Algebra Subprograms – BLAS The Basic Linear Algebra Subprograms (BLAS) define a standard set of routines that serve as building blocks for performing vector and matrix operations. These routines are categorized into three levels: • Level 1: Performs scalar, vector, and vector-vector operations. • Level 2: Performs matrix-vector operations. • Level 3: Performs matrix-matrix operations. Due to their efficiency, portability, and widespread availability, BLAS routines are commonly used as the foundation for highquality linear algebra software [20]. In general, different processor vendors provide linear algebra libraries that are highly tuned for their hardware, such as MKL (closed-source) [13] and oneDNN [21] for Intel processors, and cuBLAS for NVIDIA devices [14]. An opensource alternative BLAS implementation is the OpenBLAS library. It extends the GotoBLAS optimization approach to all BLAS API operations, supporting a wide range of architectures including LoongArch, MIPS, ARM, x86, and RISC-V. It provides a common memory-hierarchy tiling strategy used for all architectures. However, the innermost tiling is guided by the processor’s register sizes (i.e., the micro-kernel or µK), which is implemented by experts for each target architecture. This work focuses on the GEMM implementation in OpenBLAS targeting Intel x86 and RISC-V processors, leveraging the AVX-512 and RVV v1.0 vector extensions, respectively. An OpenBLAS GEMM consists of input packing, computation via the µKernel, and output unpacking to restore the standard memory layout. When executing two or more consecutive matrix multiplications using OpenBLAS, where the output of one GEMM serves as the input to the next, each intermediate
result is first unpacked by the producer and then immediately repacked by the consumer. As illustrated in Fig. 1a, this results in redundant packing and unpacking steps at every GEMM boundary, except for the final operation. In contrast, LP-GEMM is explicitly designed to eliminate this inefficiency by enabling layout propagation across consecutive GEMM invocations. As depicted in Fig. 1b, the Ini-kernel performs the initial packing and establishes the internal layout, which is preserved across all Mid-kernels without intermediate unpacking. Only in the End-kernel is the accumulated result converted back to the standard layout. This design minimizes unnecessary data movement and improves efficiency in workloads characterized by chains of dependent GEMM operations. C. Machine Learning Workloads ML workloads are dominated by a small set of highly structured key operations that typically reduce to tensor/matrix multiplications or element-wise operations [22]. Liu et al. [23] categorized ML operations into three groups, depending on how they interact with the data layout: (1) operations that are oblivious to the data layout (e.g., element-wise operations); (2) operations that are layout-tolerant (e.g., GEMM and Softmax); and (3) operations that modify the data layout (e.g., Flatten and Reshape). However, such operations do not appear in isolation. Deep ML models are composed of sets of operations with complex dependencies. A Multi-Layer Perceptron (MLP), one of the most common constructs in ML, consists of a sequence of matrix-multiplication (MM) layers interleaved with activation functions. The same holds for recent Transformer architectures [4], which contain projection layers, multi-head attention layers, and MLPs. In this work, we focus on optimizing sequences of GEMMs. That is, given an input matrix X and a set of S weight matrices
Wi and activation functions Fi , the output matrix O can be computed as O = FS WS × FS−1 (· · · × F1 (W1 × X) · · · ) . (2) Using this notation, we describe two primary deep learning architectures as groups of sequential GEMMs: 1) Multi-Layer Perceptrons (MLPs): These networks follow the formulation in Eq. 2, where the output of one layer serves as the input to the next layer’s GEMM operation. 2) Transformers: The Transformer’s main building block, the multi-head attention mechanism (Algorithm 2), can be described as follows: (1) three initial MM operations that serve as input projections for Q, K, and V (Lines 1–3); (2) the attention logic (Lines 4–9), which involves two additional MM operations, namely the score calculation (Q · K) and the weighted summation (tmp · V ), interleaved with Softmax normalization; and (3) the final output projection (Line 10). III. P ROPAGATING L AYOUT IN GEMM O PERATIONS Typical BLAS libraries are designed to operate as isolated kernels, producing outputs in the same layout as the naive implementation, leading to some redundant operations. For this reason, LP-GEMM removes those redundancies, allowing the packing layouts constructed in earlier kernels to be propagated to subsequent uses of the same matrix. Figure 3 shows two ways that sequential matrix multiplications can be performed by the OpenBLAS library to generate E, defined by E = D · C with C = A · B. This figure also illustrates the overall layout used in those distinct ways to express the sequential GEMMs. The top approach corresponds to the previous equations, where the output of one MatMul is the multiplicand of the subsequent one. The bottom approach
1st GEMM
B
A
C
2nd GEMM
D
Algorithm 2 Multi-head Attention Require: Matrices X ∈ Rm×k , Wq ∈ Rk×k , Wk ∈ Rk×m , Wv ∈ Rk×m , Wo ∈ Rk×k , scalar heads Ensure: Matrix Y ∈ Rm×k 1: Q ← X · Wq {Q ∈ Rm×k } 2: K ← X · Wk {K ∈ Rk×k } 3: V ← X · Wv {V ∈ Rk×k } 4: Apply rotatory position embedding (RoPE) to Q and K 5: Replicate K and V head-wise by m/k 6: for all headi in heads do 7: Qh , Kh , Vh , Yh ← head(h, Q, K, V, Y ) 8: tmp ← Qh · Kh 9: tmp ← scale(tmp) ∗ mask {mask is optional} 10: tmp ← softmax(tmp) 11: Yh ← tmp · Vh 12: end for 13: Y ← Y · Wo
1st GEMM
AT
BT
CT
C
E
2nd GEMM
CT
DT
ET
Fig. 3: Overview of sequential GEMM operations using different data layouts. The results are the same as using OpenBLAS with column-major or row-major, respectively.
is slightly different: by using the transposes of the matrices, it is possible to use the output of one MatMul as the multiplier of the next one. This latter approach exhibits a clear common layout between the matrix generated in the first GEMM and how it is consumed by the second GEMM, as shown by the highlighted tiles in both instances of Ct . By design in OpenBLAS, the output shares a similar tile structure to the one used by the multiplier, as illustrated by the arrow showing CT compute order in Fig. 4a. This structure allows for a very straightforward layout propagation, which LPGEMM can leverage. Alternatively, if matrix E is produced using the traditional approach to sequential MatMuls, storing the output matrix would require complex packing, generating additional overhead. For this reason, this paper will assume the use of transposed matrices. A. Kernel Implementation All kernels used by LP-GEMM are derived from the OpenBLAS kernels, which we refer to as the default implementation for both the kernel and the micro-kernel. For reference, Figure 2c provides a pseudo-code representation of how OpenBLAS performs matrix multiplication. To understand why layout propagation is not supported in BLAS-style GEMMs, it is important to distinguish three concepts: 1) Packed layout used for reading inputs, which is determined by the packing strategy and specifies the layout expected by the micro-kernel. 2) Iteration order used during computation, which dictates the order in which sub-blocks of the output are produced. 3) Final stored layout, i.e., the layout in which the output matrix is written to memory and returned to the user. BLAS does not mandate a specific computation order for GEMM (i.e., it does not constrain (2)), and it also does not require any particular packed layout (1), since packing is an internal optimization detail. BLAS only prescribes the final visible output layout (3). However, in most BLAS implementations based on GotoBLAS, (1) and (2) are tightly coupled: the packed layout (1) determines how the micro-kernel accumulates partial results, which, in turn, defines the computation order (2). Meanwhile, the final stored layout (3) must conform to the BLAS API and, therefore, often differs from both (1) and (2). As a consequence, implementations must perform an additional unpacking/reordering step to convert the output produced according to (2) into the required layout (3). This relationship is illustrated in Figures 4a and 4b. LP-GEMM aims to unify these three aspects – making (1) = (2) = (3), so that no packing or unpacking-related reordering is necessary. By aligning the packed layout, the computation order, and the final stored layout, LP-GEMM enables the output of one GEMM to be consumed directly by the next without redundancy. Data can then be stored sequentially in memory using this common layout, drastically boosting spatial locality, as demonstrated in Figure 4c. This design choice, however, violates the BLAS requirement that
results be returned in a specific canonical layout. Therefore, LP-GEMM is not compliant with the BLAS API. To safely propagate layouts across GEMM operations while preserving correctness, LP-GEMM defines three types of kernels: the Initial Kernel, the Intermediate Kernel, and the Ending Kernel. These kernels rely on two micro-kernels: the Propagate-Layout µkernel, which maintains the propagated layout across GEMMs, and the Default µkernel, which terminates propagation and restores the BLAS-style output layout when needed. The following subsections explain the modifications made to the original kernel, detailed in Figure 2c, in order to create three different kernel versions that together implement the layout propagation strategy. 1) Initial Kernel: This kernel should come before all the others, as it is responsible for starting the layout propagation. The kernel itself is not much different from the one in OpenBLAS. The only difference is that it uses the propagate layout micro-kernel to perform the GEMM, which leads to the output being stored in the propagated layout of LP-GEMM. 2) Intermediate Kernel: This kernel assumes that the multiplier is already packed into the layout that will be used by the micro-kernel, allowing the kernel to skip packing this matrix. For this reason, this kernel can only be called after packing the multiplier, which can be done either by an Initial Kernel or by directly packing it before calling this kernel. Besides that, this kernel continues the layout propagation by storing its output in the same order it is calculated, which allows for most of the GEMMs in a sequence of MatMuls to be solved by this kernel. The difference in implementation comes from removing lines 3-7 and starting the loop of line 8 at position 0 instead of mc. 3) Ending Kernel: This kernel is responsible for stopping the layout propagation by producing an output in the original data layout. This kernel is structured the same way as the Intermediate Kernel, with the same changes to the implementation. The difference is that it uses the Default µkernel, which itself resets the data back to the original layout. B. Micro-kernel Design Both the Propagate Layout and the Default µkernels use the same structure to calculate the values of C; the arrow in Figure 4a demonstrates the order in which the values of C are calculated. The default µkernel implementation, shown in Figure 4b, has to save C in an order different from the one in which it is computed, so that the resulting output has the same layout as the original operator. The LP-GEMM layout propagator micro-kernel takes advantage of the notion that C is computed in a packed layout, which can be expressed as: N M nc mc nr mr (3) nc mc nr mr This means that data is contiguous in the nr × mr dimension, which is important for micro-architecture-specific register population. But most importantly, C will be contiguous in
nc
nc
nr
Bt
kr
kc
mc
kr mr
At
Ct
mc
N sa sb CT
M
nc ⋅ mc
(a) Micro-kernel compute order.
(b) Default saving order, going back to original layout. N⋅M
CT
nc ⋅ mc
(c) Contiguous storage of C, propagating packing layout.
Fig. 4: Micro-kernel Layout. the nc × mc dimension, which is the same dimension that will be used to calculate data in the case of a subsequent consumer GEMM operator. As data is not stored in its original layout, it is required that the following are true: (1) If C already contained data before the first GEMM call, and β is not zero; C must be packed into the layout of equation 3. (2) Calls to the µkernel need to take into account that C will not be stored in the original layout, and the regions of this matrix passed to the micro-kernel need to reflect this notion. The LP-GEMM kernels that use layout propagation (e.g., initial and intermediate) already solve (2). The scenario described by (1) cannot be solved by LPGEMM without significant changes to the kernel and µkernel implementations. Luckily, scenarios like this are rare, especially in AI models, which are the primary motivators for this work. For this reason, we decided not to address this corner case in our proposed implementation. C. Strided Loads and Stores The flexibility of matrix multiplications means that the use cases for it are numerous, and LP-GEMM had to be able to replicate naive GEMM behavior for those cases. Although one of the most common ways to think of an MM is to consider multiplying contiguous matrices in memory, it is not always the case, as sparse or tiled matrices may be involved. The OpenBLAS kernel can seemingly deal with these cases through the manipulation of kernel parameters such as M, N, K, and the sizes (lda, ldb, and ldc) of the matrices. This is possible because, by default, data is stored requiring some stride in memory accesses in those kernels.
Because LP-GEMM stores data in the same order in which it is computed, sometimes such behavior cannot seemingly be adapted to work in those kernels. In response to this, it is necessary to use some additional function parameters for those scenarios. One of those parameters is simply responsible for changing the order of the blocks CT that will be saved sequentially. The second defines a stride for data to be stored in memory, so that a GEMM call leaves some empty positions between data from a single block CT , crucial when calculating only a fraction of the output matrix. IV. L AYOUT CHANGES IN THE ATTENTION L AYER To test the generalizability of LP-GEMM, we developed a mock of the attention layer of the LLaMA transformer using C++, so that the layout could be propagated freely while testing the correctness of the results compared to the baseline execution of the actual model. To assess the validity of this implementation, the randomly generated inputs, all intermediate calculations, and the weights of the LLaMA 3.2 (1B parameters) 3 version were exported from the PyTorch implementation to binary files. These same inputs and weights were read by the C++ implementation, and all intermediate results were compared to those extracted from PyTorch. A. Layout Propagation in Matrix Operations A key feature of LP-GEMM is its ability to propagate the packing layout from one GEMM operation to the next. In practice, however, back-to-back matrix multiplications are uncommon. Machine-learning workloads illustrate this well, 3 https://huggingface.co/meta-llama/Llama-3.2-1B
as after almost every matrix multiplication, an activation or other non-linear transformation is applied. To evaluate how LP-GEMM interacts with such operators, we use the attention layer of transformer models as a representative scenario. Multi-head attention performs several matrix multiplications that can be grouped as a sequence of GEMMs, making it a suitable candidate for LP-GEMM. Layout propagation across these operations, however, is subtle. Because the Q, K, and V matrices are split into multiple heads, each head consumes different slices of the original matrices. This is inconsequential without layout propagation (see Section III-C), but LP-GEMM stores its output in the exact layout expected by the consumer GEMM. As a result, the kernel must know the correct strides to generate a layout compatible with downstream consumers. Fortunately, these strides are highly predictable, depending only on the input dimensions and the per-head projection size, allowing the compiler to infer them statically. As shown in Algorithm 2, several matrix operations appear between the GEMMs in the attention layer. These include: Scale, which is layout-oblivious; and RoPE and Softmax, which are layout-tolerant but require adjustments to preserve correctness under a propagated layout. The problem is that the optimal layout for these operators differs from the one preferred by the subsequent consumer GEMMs. A naive address translation would introduce substantial overhead due to irregular memory access. It is therefore necessary to adapt these intermediate operations so that they better exploit the LP-GEMM layout. a) Rotatory Position Embedding (RoPE): naturally aligns with the layout produced for the following GEMM. Rotations are applied independently per row within each head, which is the same pattern used when the LP-GEMM kernel loads data. As a result, RoPE can operate efficiently on the propagated layout with minimal modification. However, it can actively produce better results if multiple rows are calculated simultaneously using SIMD, taking advantage of the row interleaving done in the propagation layout. b) Softmax: poses a greater challenge. It is computed independently for each row, whereas the propagated layout partitions the row dimension into tiles for outer-product processing. Direct propagation, therefore, introduces addressjump overhead at tile boundaries. However, because the propagated layout interleaves rows, Softmax can be reorganized to operate over multiple rows at once, reducing the penalty associated with non-sequential accesses. More generally, other matrix operations can also be adapted to operate directly on the LP-GEMM layout, and the compiler can determine the required transformations at compile time. For row-major operations that rely on full-row reduction and cannot be decomposed into partial results, such as Softmax, some overhead is expected when using the propagated layout of Equation 3. Nonetheless, as demonstrated in Section V-C, the performance gains from LP-GEMM outweigh these additional costs.
CPU info CPU Family Vector Extension mc , nc and kc mr and nr L1 cache size (KiB) L2 cache size (KiB) L3 cache size (KiB) CPU Frequency Vector Registers Size FMA Throughput (GFLOPS/s)
Intel Xeon Gold 6252 Skylakex (x86-64) AVX512 448, 16384, 448 16, 4 32 1024 35.8 shared 3.6 512 223.3
Spacemit X60 RV64GCVB RVV1.0 128, 16385, 128 16, 8 32 512 shared 1.6 256 25.6
TABLE I: Summary of evaluated systems.
V. E VALUATION OF LP-GEMM This section evaluates LP-GEMM along two primary dimensions: performance and generalizability. First, the experimental setup is described, including the target x86-64 and RISC-V platforms and the set of optimized GEMM implementations used for comparison. Next, we assess performance using isolated GEMM executions, comparing LPGEMM against state-of-the-art approaches on both architectures. We then evaluate LP-GEMM within the Attention layer of the LLaMA 3.2 model to study its behavior in a realistic machine learning workload. Finally, we analyze a sequence of three consecutive GEMM operations to compare LP-GEMM with existing approaches for optimizing sequences of matrix multiplications. A. Experimental Setup The experiments aim to evaluate LP-GEMM on two representative architectures that provide explicit vector register support and enable Single Instruction Multiple Data (SIMD) execution, which is standard for high-performance GEMM implementations. A summary of the architectural characteristics relevant to GEMM optimization is provided in Table I. 1) Execution Environment: a) RISC-V: Experiments on the RISC-V platform were conducted using a Banana Pi BPI-F3 equipped with a SpacemiT K1 octa-core X60 processor (RV64GCVB). This processor supports SIMD execution through the RISC-V Vector Extension (RVV) version 1.0. b) Intel x86: Experiments on the x86 platform were conducted using an Intel Xeon Gold 6252 processor, which provides native SIMD support via the AVX-512 instruction set. All performance evaluations use OpenBLAS as the primary baseline implementation. In addition to LP-GEMM, we compare against several widely used and state-of-the-art GEMM implementations, including BLIS [24], Intel MKL [13], oneDNN [21], and FlashGEMM [11]. Due to platform support constraints, Intel MKL, oneDNN, and FlashGEMM are evaluated only on the x86-64 platform. LP-GEMM, OpenBLAS, and BLIS were evaluated on both x86-64 and RISC-V architectures. All implementations are invoked through the BLAS Level3 GEMM interface whenever it is supported. For LP-GEMM and FlashGEMM, minor adaptations were required to ensure
To evaluate the performance of the proposed kernels against state-of-the-art GEMM implementations, we conducted a single-GEMM execution experiment using matrix sizes drawn from the gemmbench [25] dataset. Figure 5 shows the results for an x86 and riscv platforms. The points used to create the boxplot represent the means of 10 executions for each implementation using each of the matrix sizes from gemmbench. All speedups are computed relative to OpenBLAS. The simplicity of this setup allows for the comparison of a wide range of implementations under the same execution model. Although LP-GEMM kernels are not designed for this usage pattern, as they expect data to already be in the propagated layout, this experiment still provides a useful perspective. It offers a lower-bound estimate of the potential gains from LP-GEMM in realistic pipelines and helps clarify where its performance advantages originate. The Initial kernel (Ini) delivers performance close to OpenBLAS. This is expected, since, aside from removing the unpacking step and slightly altering the store pattern, it largely mirrors the baseline behavior. For large matrices, these changes alone are not sufficient to generate substantial speedups. The Intermediate and Ending kernels (Mid and End) reveal the real benefit of layout propagation for both RISC-V and x86. By fully eliminating the packing of one of the matrices, they achieve a median speedup of about 1.5× over OpenBLAS, with third-quartiles exceeding 2×. These gains highlight that packing, rather than the micro-kernel itself, is often the dominant cost and show how impactful layout propagation can be for optimizing GEMM execution. The LP-GEMM kernels are derived from the open-source OpenBLAS implementations, so improvements relative to OpenBLAS mainly come from removing packing rather than from deeper micro-kernel tuning. This contrasts with libraries such as BLIS [24] and FlashGEMM [11], whose median speedups exceed those of the Initial LP-GEMM kernel because their underlying kernels are more heavily optimized. Still, when compared to the Mid and End kernels, both BLIS and FlashGEMM show noticeably lower speedups due to their packing overhead. Highly tuned commercial and vendor libraries like Intel MKL and OneDNN exhibit strong performance for x86, clearly outperforming OpenBLAS (as seen in Fig. 5). Their advantage over all three LP-GEMM kernels—particularly the Initial one—comes primarily from superior kernel optimization. Nevertheless, the gap between these libraries and LPGEMM’s Mid/End kernels is modest, suggesting that layout propagation alone can close a substantial portion of the per-
Speedup (based on OpenBLAS)
B. Single GEMM Execution
5
120
133
4
120
118
3
142
98
2 1
43 31
5
) ) ) BLIS M (Ini M (Mid (End M M E M EM LP-G LP-G LP-GE
L EMM tel MK neDNN O nI
hG Flas
Speedup (based on OpenBLAS)
functional equivalence during evaluation, as these implementations do not strictly conform to the standard BLAS API. All experiments were conducted in a single-threaded configuration to avoid introducing bias due to differences in parallel runtime behavior.
(a) Intel Xeon Gold 6252 29
3
68
2
69
1
1
BLIS
LP-
M GEM
(Ini) LP-
M GEM
)
(Mid
nd)
(E EMM G P L
(b) Spacemit X60
Fig. 5: Speedup of a single GEMM extracted from gemmbench [25] using different state-of-art kernels compared to the three separate LP-GEMM kernels. formance difference relative to highly specialized implementations. A final advantage of LP-GEMM is its generality: layout propagation is orthogonal to micro-kernel design and can be combined with more advanced kernels to achieve similar or greater speedups. For example, if the kernels used by MKL were open-source, LP-GEMM could be integrated with them, likely yielding even stronger results. C. Attention Layer One of the prime motivators for this paper is layout propagation in the attention layer of a transformer network. This experiment aims to test how LP-GEMM performs in this situation and how it interacts with other matrix operations, as described in section IV-A. Figure 6a shows the performance of the attention layer implementation proposed in section IV using LP-GEMM and modified matrix operations to deal with layout propagation, compared to using OpenBLAS with no layout propagation, executed on x86. Results in both graphs of Figure 6 were calculated as the mean of 50 analogous executions, for both architectures. The x86 results obtained in this experiment highlight a crucial aspect of LP-GEMM and layout propagation as a whole: the improvements in layout propagation scale inversely with the size of the matrices involved in the GEMM operations. This aspect is to be expected, as packing operations
Speedup (over OpenBLAS)
2.0
5
1.8
4
1.6
3
1.4
2
1.2 1.0
Attention MLP Total
1
0
100
200
300
ntokens
400
500
0
100
(a) Intel Xeon Gold 6252
200
300
ntokens
400
500
(b) Spacemit X60
Fig. 6: Speedup of Attention layer from LLaMA 3.2 model, with embedded dimension of 2048, and MLP weights with dimension of 8129. Comparing LP-GEMM and layout propagation to OpenBLAS with no propagation. have a complexity of O(n2 ), while the micro-kernel that performs the multiplications has a complexity of O(n3 ) . This characteristic is not inherently a downside to layout propagation optimization, but it is something that needs to be considered when dealing with this type of optimization. Taking that into consideration, the results obtained by integrating LP-GEMM into the attention layer are still impressive, with speedups of up to 2x for Multi-Headed Attention with small input sizes and still significant improvements for larger inputs. The curve for speedup in the multi-headed attention layer has an exponential-like shape due to some operations performed in this region using a matrix of dimensions num tokens×num tokens. The curve for the MLP speedup tends to stay around 1.1, as the operations performed in this region mostly depend on the dimensions of the weights used. The graphs focus on sequence lengths of up to 500 tokens, as, in practical Edge AI workloads, latency constraints often preclude massive context windows. Furthermore, recent empirical studies suggest that extending context length indefinitely is not strictly beneficial for reasoning tasks. Levy et al. [26] demonstrated that reasoning performance on identical tasks degrades significantly as input length increases, and Du et al. [27] found that this ”distraction” effect persists even when retrieval is perfect. Consequently, optimizing for moderate context windows, where the hardware is most efficient and the model is cognitively most effective, remains the primary objective for edge deployment. The RISC V results, shown in Figure 6b, follow a noticeably different trend. The absolute performance gap between RISCV and x86 is largely explained by architectural differences. The processor in the Banana Pi BPI F3 provides significantly lower FMA throughput and memory bandwidth compared to the Intel Xeon Gold 6252, which naturally limits GEMM performance due to its high arithmetic intensity. Beyond the SpacemiT K1 used in this experiment, other RISC-V implementations are targeting similar workloads, such as the Kendryte K230 (featuring a dual-core C908 with RVV 1.0 for AIoT) and the SiFive Intelligence X280, which is increasingly adopted for AI acceleration in automotive and datacenter offload scenarios. However, the shape of the speedup curve on RISC V is
governed by a different factor. The nearly linear improvement observed as ntokens increases is primarily a consequence of inefficiencies in the baseline RISC-V kernel used by OpenBLAS. This kernel performs the final unpacking step through out-of-order memory accesses, which become increasingly costly as matrix sizes grow. LP-GEMM avoids this overhead entirely by producing the output directly in a contiguous layout. As the penalty from scattered accesses accumulates in the reference implementation, the relative advantage of LP GEMM increases proportionally with problem size. In summary, the hardware differences explain the performance gap between the two architectures, but the scaling behavior on RISC-V is driven mostly by the inefficiency of the reference kernel in OpenBLAS. This makes layout propagation especially effective on this platform as the size of the problem increases, in contrast to x86, where speedups tend to saturate once packing and unpacking costs become negligible relative to the highly optimized micro-kernels. D. Consecutive GEMMs As previously mentioned, it is uncommon to find a sequence of pure consecutive GEMMs in machine learning, as the structurally adjacent matrix multiplications (such as those found in DNN bottleneck blocks) are typically separated by essential non-linear activation layers, which break data-flow continuity. Nevertheless, other work has established benchmarks composed of this challenging sequence of three consecutive GEMM operations by extracting the input/output matrix sizes from the convolutional layers found in common DNN frameworks [11]. This methodology creates a specialized, linear benchmark scenario that enables the study of fusion across multiple matrix multiplications by abstracting away the intermediate non-linear operations inherent in the original models. In the benchmarks used for this experiment, most of the execution is spent on the first GEMM in the case of LPGEMM, using the Initial Kernel. As mentioned in Section V-B, most of the speedup from LP-GEMM is in the other two kernels. In other words, this is not the best scenario for LPGEMM, but we still want to compare our approach with FlashGEMM to understand whether layout propagation can
70
Usage (% to Peak)
60 50 40 30 20 10 0
1
2
3
4 5 6 7 Benchmark Number
FlashGEMM OpenBLAS LP-GEMM 8 9 10
Fig. 7: Usage comparison of LP-GEMM with FlashGEMM and OpenBLAS, using benchmarks extracted from DNN [11].
bring some benefits even in these circumstances. As depicted in Fig. 7, LP-GEMM is still able to outperform both OpenBLAS and FlashGEMM on most of the proposed benchmarks. Once again, it demonstrates the power of layout propagation in a sequence of GEMM operations. VI. R ELATED W ORK Matrix multiplication is a fundamental building block of modern machine learning workloads and remains one of the most heavily used operations across HPC applications. Historically, GEMM optimization has focused on maximizing the performance of individual calls, treating each GEMM as an isolated computation. This design philosophy is exemplified by traditional BLAS libraries, whose implementations, often derived from GotoBLAS [10], perform packing and unpacking on every GEMM invocation, regardless of the broader workload structure. Even highly specialized libraries such as Intel MKL, which exploit extensive low-level tuning to achieve high processor utilization, optimize each GEMM independently and do not account for opportunities across consecutive operations or propagate data layouts through a computation pipeline. Data layout propagation itself is not new, particularly in contexts involving tensor graphs and deep-learning operators. NeoCPU [23] proposes a graph-level framework that identifies profitable data layouts and determines regions where such layouts can be maintained. However, its approach is tailored specifically to ML workloads and does not generalize easily to other domains. Moreover, its layout selection relies on an analysis pass and a search procedure, creating a tradeoff between search cost and layout quality that may result in suboptimal layouts being chosen. OneDNN [21] provides one of the most mature implementations of layout propagation. When enabled, the framework infers optimized data formats to be used in subsequent operators. While originally optimized for Intel processors, OneDNN has since become an open-source, architecture-agnostic project.
Like NeoCPU, it depends on analysis to identify propagation boundaries and requires users to specify where reordering is permitted, relying on explicit reorder primitives to transition between layouts. In contrast, LP-GEMM integrates the initial reorder step directly into the GEMM operation itself via the Initial kernel, reducing the overhead and complexity of managing layout transitions. FlashGEMM [11], similar in spirit to LP-GEMM, targets workloads consisting of sequential GEMMs rather than optimizing each GEMM call in isolation. Its novelty lies in an aggressive fusion of GEMMs to exploit data reuse and locality, thereby enabling layout propagation across fused operations. FlashGEMM also includes profitability analysis to determine whether packing should be performed based on memory constraints. However, its reliance on fusion makes integration into real ML workloads challenging: intermediate non-GEMM layers are not directly supported and would need to be reimplemented inside fused kernels to preserve correctness. Moreover, the fusion strategy disallows partial results, limiting the set of matrix dimensions that can be tiled efficiently. VII. C ONCLUSION AND F UTURE W ORK This work introduced LP-GEMM, a general and effective approach for propagating data layouts across sequences of GEMM operations. By exploiting the natural layout continuity present in many workloads, LP-GEMM improves performance without requiring low-level microarchitectural tuning. Across the evaluated architectures and workflows, LP-GEMM consistently delivers speedups comparable to those of highly specialized GEMM implementations, underscoring its portability and practical relevance. Our results highlight the importance of reducing redundant work, particularly unnecessary packing and unpacking, when optimizing matrix operations. The performance gains achieved by LP-GEMM stem entirely from eliminating these overheads, demonstrating that understanding how GEMMs interact within broader computation pipelines can unlock significant efficiency improvements beyond isolated kernel optimization. LP-GEMM preserves a structure close to BLAS-style GEMMs to ease adoption, but it is not a drop-in replacement, as it introduces additional parameters for strided memory accesses (Section III-C) and produces outputs in a propagated layout. Nevertheless, integration into existing code requires only modest changes, as demonstrated in the Attention-layer case study (Section V-C). Although LP-GEMM is currently implemented for x86 64 and RISC-V and builds on OpenBLAS kernels, extending it to additional architectures, combining layout propagation with low-level tuning, and integrating it more deeply with compilerdriven optimization frameworks are natural directions for future work. More broadly, LP-GEMM suggests a shift from optimizing isolated GEMM calls toward optimizing end-toend workloads, with layout propagation beyond GEMM, such as for convolutions, representing a promising avenue for future research.
R EFERENCES [1] G. Karniadakis, Y. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics, pp. 1–19, 2021. [2] Y.-F. Shi, Z.-X. Yang, S. Ma, P.-L. Kang, C. Shang, P. Hu, and Z.-P. Liu, “Machine learning for chemistry: Basics and applications,” Engineering, vol. 27, pp. 70–83, 2023. [3] J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, S. Bodenstein, D. A. Evans, C.-C. Hung, M. O’Neill, D. Reiman, K. Tunyasuvunakool, Z. Wu, A. Žemgulytė, E. Arvaniti, C. Beattie, O. Bertolli, A. Bridgland, A. Cherepanov, M. Congreve, A. I. Cowen-Rivers, A. Cowie, M. Figurnov, F. B. Fuchs, H. Gladman, R. Jain, Y. A. Khan, C. M. R. Low, K. Perlin, A. Potapenko, P. Savy, S. Singh, A. Stecula, A. Thillaisundaram, C. Tong, S. Yakneen, E. D. Zhong, M. Zielinski, A. Žı́dek, V.-. tor Bapst, P. Kohli, M. Jaderberg, D. Hassabis, and J. M. Jumper, “Accurate structure prediction of biomolecular interactions with alphafold 3,” Nature, vol. 630, pp. 493 – 500, 2024. [4] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS’17, 2017. [5] Meta’s Llama 3 team, “The llama 3 herd of models,” arXiv e-prints, 2024. [6] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT’2019, 2019. [Online]. Available: https://doi.org/10.18653/ v1/n19-1423 [7] H. Genc, S. Kim, A. Amid, A. Haj-Ali, V. Iyer, P. Prakash, J. Zhao, D. Grubb, H. Liew, H. Mao, A. J. Ou, C. Schmidt, S. Steffl, J. C. Wright, I. Stoica, J. Ragan-Kelley, K. Asanovic, B. Nikolic, and Y. S. Shao, “Gemmini: Enabling systematic deep-learning architecture evaluation via full-stack integration,” in DAC’2021, 2021. [8] M. Li, C. Liu, J. Liao, X. Zheng, H. Yang, R. Sun, J. Xu, L. Gan, G. Yang, Z. Luan, and D. Qian, “Towards optimized tensor code generation for deep learning on sunway many-core processor,” Frontiers Comput. Sci., vol. 18, p. 182101, 2024. [9] H. Kim, G. Ye, N. Wang, A. Yazdanbakhsh, and N. S. Kim, “Exploiting intel advanced matrix extensions (AMX) for large language model inference,” IEEE Comput. Archit. Lett., vol. 23, no. 1, pp. 117–120, 2024. [10] K. Goto and R. A. van de Geijn, “Anatomy of high-performance matrix multiplication,” ACM Transactions on Mathematical Software (TOMS), vol. 34, pp. 1–25, 2008. [11] J. Zhang, W. Yang, J. Fang, D. Dong, and X. Chen, “Flashgemm: Optimizing sequences of matrix multiplication by exploiting data reuse on cpus,” ACM Trans. Archit. Code Optim., vol. 22, no. 4, Dec. 2025. [Online]. Available: https://doi.org/10.1145/3760784 [12] J. Alman, R. Duan, V. Vassilevska Williams, Y. Xu, Z. Xu, and R. Zhou, “More asymmetry yields faster matrix multiplication,” in SODA’2025, Y. Azar and D. Panigrahi, Eds., 2025, pp. 2005–2039. [13] Intel Corporation, “Intel math kernel library (intel mkl),” https:// www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html, 2024, accessed: 2025-12-10. [14] NVIDIA Corporation, “cublas: Cuda basic linear algebra subroutines,” https://developer.nvidia.com/cublas, 2024, accessed: 2025-12-10. [15] S. Remke and A. Breuer, “Hello sme! generating fast matrix multiplication kernels using the scalable matrix extension,” in SC24-W:, 2024, pp. 1443–1454. [16] M. D. Lam, E. E. Rothberg, and M. E. Wolf, “The cache performance and optimizations of blocked algorithms,” ACM SIGOPS Operating Systems Review, vol. 25, no. Special Issue, pp. 63–74, 1991. [17] M. J. Wolfe, C. Shanklin, and L. Ortega, High Performance Compilers for Parallel Computing. Addison-Wesley Longman Publishing Co., Inc., 1995. [18] C. S. Rohwedder, N. Henderson, J. ao P. L. De Carvalho, Y. Chen, and J. N. Amaral, “To pack or not to pack: A generalized packing analysis and transformation,” in CGO’23, 2023, p. 14–27. [19] J. L. Hennessy and D. A. Patterson, Computer Architecture: A Quantitative Approach, 5th ed. Morgan Kaufmann Publishers Inc., 2011. [20] The Netlib, “BLAS (Basic Linear Algebra Subprograms),” https://www. netlib.org/blas/, 2025, accessed: 2025-11-25.
[21] J. Li, Z. Qin, Y. Mei, J. Cui, Y. Song, C. Chen, Y. Zhang, L. Du, X. Cheng, B. Jin, Y. Zhang, J. Ye, E. Lin, and D. Lavery, “onednn graph compiler: A hybrid approach for high-performance deep learning compilation,” in IEEE/ACM International Symposium on Code Generation and Optimization, CGO 2024, Edinburgh, United Kingdom, March 2-6, 2024, T. Grosser, C. Dubach, M. Steuwer, J. Xue, G. Ottoni, and ernando Magno Quintão Pereira, Eds. IEEE, 2024, pp. 460–470. [Online]. Available: https://doi.org/10.1109/CGO57630.2024.10444871 [22] A. Thangamani, M. A. A. Shahid, A. Siemieniuk, R. Morel, R. Golin, and A. Heinecke, “Library liberation: Competitive performance matmul through compiler-composed nanokernels,” arXiv e-prints, 2025. [23] Y. Liu, Y. Wang, R. Yu, M. Li, V. Sharma, and Y. Wang, “Optimizing CNN model inference on CPUs,” in USENIX ATC’19, 2019, pp. 1025– 1040. [24] F. G. Van Zee and R. A. van de Geijn, “BLIS: A framework for rapidly instantiating BLAS functionality,” ACM Transactions on Mathematical Software, vol. 41, no. 3, pp. 14:1–14:33, June 2015. [Online]. Available: https://doi.acm.org/10.1145/2764454 [25] Lucas Alvarenga, “GEMMBench: A lightweight benchmarking framework for evaluating custom GEMM microkernels.” https://github.com/ LucasFernando-aes/gemmbench, 2025, accessed: 2025-11-25. [26] M. Levy, A. Jacoby, and Y. Goldberg, “Same task, more tokens: the impact of input length on the reasoning performance of large language models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, L. Ku, A. Martins, and V. Srikumar, Eds. Association for Computational Linguistics, 2024, pp. 15 339–15 353. [Online]. Available: https://doi.org/10.18653/v1/2024.acl-long.818 [27] Y. Du, M. Tian, S. Ronanki, S. Rongali, S. B. Bodapati, A. Galstyan, A. Wells, R. Schwartz, E. A. Huerta, and H. Peng, “Context length alone hurts LLM performance despite perfect retrieval,” in Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng, Eds. Suzhou, China: Association for Computational Linguistics, Nov. 2025, pp. 23 281–23 298. [Online]. Available: https://aclanthology.org/2025.findings-emnlp.1264/