Operating System Support for Accelerators as First-Class Compute Elements in Heterogeneous Systems

Operating System Support for Accelerators as First-Class Compute Elements in Heterogeneous Systems Mohammad Shahidzadeh† and Lesley Shannon‡

arXiv:2610.33380v1 [cs.AR] 24 Oct 2026

Simon Fraser University, Canada†, Simon Fraser University, Canada‡ msa417@sfu.ca Abstract—Heterogeneous systems-on-chip (SoCs) increasingly rely on domain-specific hardware accelerators (HAs) to deliver performance and energy-efficiency gains beyond what generalpurpose processors can provide. However, operating systems (O/S), such as Linux, assume that processors are the Compute Elements (CEs) and HAs are peripheral devices rather than specialized CEs. As a result, HA execution incurs substantial software (SW) overheads that reduce the effective benefits of hardware (HW) acceleration. This work demonstrates that significant inefficiencies arise from “O/S peripheral device taxes” (O/S taxes), including SW stack-accelerator communication, invocation, interrupt handling, and scheduling. We built MEERKAT1 , an Field-Programmable Gate Array (FPGA)-based framework for configurable multicore systems that include HAs; it allows the user to migrate Linux O/S support from SW to HW and identify, study and quantify the impact. Here we study four major sources of O/S tax and introduce techniques to mitigate them. These include a virtual-memoryaware (VMA) communication method that enables efficient HA access to application data, a lightweight HA invocation interface, Interrupt Policy Adaptation (IPA) for dynamic polling and interrupt selection, and an HA-aware scheduler that dynamically distributes work across Central Processing Units (CPUs) and multiple HA instances. Experimental results demonstrate up to 260× faster communication and 2–6× performance improvements in wall clock time over conventional Schedulers. Furthermore, our study reveals that overall system performance depends not only on HA speed, but also on O/S overheads, resource availability, and scheduling decisions. These findings highlight the importance of treating HAs as first-class computational resources and motivate a more accelerator-centric O/S design for future heterogeneous systems.

I. I NTRODUCTION With the end of Dennard scaling, heterogeneous System-onChips (SoCs) that integrate general-purpose processors with specialized hardware (HW) accelerators have emerged as a key approach for sustaining performance growth and improving energy efficiency [8], [10]. As a result, considerable research effort has been devoted to designing increasingly faster and more specialized accelerators. However, these accelerators are often developed and evaluated in isolation, with limited consideration of the applications they ultimately serve or the system-level environment in which they operate. In particular, relatively little attention has been paid to how Operating System (O/S) abstractions, resource management policies, and 1 https://github.com/mohammadshahidzade/MEERKAT

software (SW) interfaces influence accelerator utilization and overall application performance [11], [25]. Despite their growing importance, accelerators continue to be treated as peripheral devices within most O/Ss. Modern O/Ss were designed around a CPU-centric execution model in which Central Processing Unit (CPU) cores are the primary Compute Elements CE responsible for scheduling, resource management, and execution. Accelerators, in contrast, are typically accessed through device-driver interfaces and remain outside O/S abstractions [18]. As accelerators increasingly perform a substantial fraction of application computation, We argue that accelerators should increasingly be treated as firstclass CE rather than passive devices [39]. The lack of first-class O/S support introduces a variety of overheads that reduce the effective benefits of HW acceleration [11], [25]. These overheads can significantly impact application performance, particularly for fine-grained accelerators and latency-sensitive workloads. In this work, we refer to such overheads as “O/S peripheral device taxes” (O/S taxes). There are four major sources of overhead in tightly integrated heterogeneous SoCs: communication, invocation, interrupt handling, and scheduling overheads. One major source of O/S tax arises from communication between SW and HW accelerators [9], [23], [41]. In particular, enabling accelerators to operate directly on virtualmemory-managed data introduces three fundamental challenges. First, accelerators require access to physical memory locations while applications operate using virtual addresses. Second, application data is often distributed across noncontiguous physical pages, complicating efficient accelerator data movement. Third, tightly integrated accelerators must maintain memory consistency with processor caches. To address these challenges, we introduce Virtual-Memory-Aware (VMA) method consisting of three complementary mechanisms: a Custom Instruction Virtual Address Translation (CIVAT) for accelerator-side address translation, a Scatter-Gather Page Transfer (SGPT) mechanism for efficient non-contiguous memory access, and a HW-Managed Accelerator Coherence (HMAC) protocol that maintains consistency between accelerator and processor caches. Together, these mechanisms enable accelerators to operate directly on virtual-memory-managed application data without requiring expensive SW-mediated address translation, data copying, or coherence management.

Accelerator execution is also accompanied by invocation and completion overheads [39]. Unlike CPU execution, which is directly managed by the cores with a simple function call, accelerator execution typically requires traversing multiple SW layers including user-space libraries, system calls, device drivers, and runtime frameworks before computation can begin. Similarly, accelerator completion is commonly communicated through interrupts that trigger additional O/S processing and context switches. While these overheads may appear negligible for long-running accelerators, they can become a significant fraction of total execution time for finegrained workloads and frequently invoked accelerators. To reduce these costs, we replace expensive Input/Output Control (IOCTL)-based accelerator requests with a lightweight function-call interface that minimizes SW overhead during accelerator invocation. Furthermore, we introduce Interrupt Policy Adaptation (IPA), a runtime mechanism that dynamically selects between polling and interrupt-driven completion handling according to the predicted accelerator execution time. Together, these techniques reduce invocation latency, avoid unnecessary context switches, and lower the O/S tax associated with accelerator execution. Existing O/S schedulers introduce another source of inefficiency [13], [27]. Traditional schedulers were developed for homogeneous multicore systems in which execution resources are largely interchangeable. In heterogeneous systems, however, CPUs and accelerators exhibit widely different performance and energy characteristics. Nevertheless, users are often responsible for explicitly selecting accelerators, while the O/S remains largely unaware of accelerator capabilities and availability. This can lead to suboptimal resource utilization and increased execution time [26], [27]. In this work, we demonstrate that treating CPUs and accelerators as a unified pool of computational resources enables more effective scheduling decisions. By dynamically selecting between CPUs and multiple accelerator instances based on workload characteristics and resource availability, the scheduler can improve system utilization and overall application performance. Furthermore, we show that, similar to multicore CPU systems, the benefits of multiple accelerators can only be fully realized when they are managed by an accelerator-aware scheduler. The key contributions of this paper are summarized as follows: • We develop MEERKAT, an Field-Programmable Gate Array (FPGA)-based Linux research framework for systematically studying O/S taxes in heterogeneous SoCs, enabling detailed analysis of CPU–accelerator interactions and cross-layer performance bottlenecks. • We propose a HW–SW co-designed communication method consisting of CIVAT, SGPT, and HMAC, enabling accelerators to efficiently access virtual-memorymanaged application data while achieving up to 260× lower communication overhead with near-zero HW cost. • We reduce accelerator invocation and completion overheads through a lightweight function-call-based execution interface and IPA, a runtime mechanism that dynamically

selects between polling and interrupt-driven completion based on predicted accelerator execution time. • We extend the O/S scheduler with accelerator-aware resource selection that treats CPUs and multiple accelerator instances as a unified pool of computational resources, achieving up to 2–6× performance improvement in wall clock time over conventional accelerator-selection approaches. • We characterize the system-level behavior of heterogeneous multi-accelerator platforms and demonstrate that application performance depends not only on accelerator speed, but also on O/S overheads, resource availability, and scheduling decisions, highlighting the importance of system-level optimization for future accelerator-rich systems. The remainder of this paper is organized as follows. Section II reviews existing frameworks and prior work on O/S taxes. Section III presents our framework and the techniques employed to reduce these taxes. Section IV and V evaluates the proposed approach, quantifies the resulting performance improvements, and demonstrates how treating accelerators as first-class system resources can substantially reduce the performance gap imposed by O/S overheads. II. R ELATED W ORK AND M OTIVATION While numerous prior works have addressed individual sources of overhead in heterogeneous systems, including accelerator invocation latency, communication overhead, cache coherence, interrupt handling, and accelerator-aware scheduling, these challenges are typically studied in isolation. Among the closest efforts, Mozart [39] demonstrates that SW overheads can substantially reduce the benefits of HW acceleration and introduces the concept of accelerator taxes associated with accelerator communication in NOC and its invocation. Similar to Mozart, we argue that SW overheads play a critical role in determining system performance. However, whereas Mozart primarily focuses on the runtime overheads of supporting disaggregated accelerators, our work takes a broader O/S perspective by systematically characterizing the end-to-end SW overheads encountered by shared-memory accelerators. A. Evaluation Framework A wide range of infrastructures have been proposed for evaluating heterogeneous systems, each with different tradeoffs between flexibility, execution speed, and architectural visibility. Simulation frameworks such as gem5 [4] and gem5Aladdin [37] provide extensive architectural flexibility but incur low execution speed, limiting full-system Linux evaluation and long-running workloads. FPGA-based platforms, including FireSim [20], HERO [25], ESP [28], LiteX [21], and commercial FPGA SoCs, provide significantly higher execution speeds but typically rely on large processor cores not optimized for FPGA implementation. This leads to lower achievable frequencies, higher FPGA resource utilization, and limited flexibility for modifying processor architecture and operating-system interactions.

B. Communication Methods Traditionally, communication between CPUs and HW accelerators has been performed through a copy-based method which uses SW-managed Direct Memory Access (DMA) transfers using dedicated physically contiguous memory regions shared between the host and the accelerator [24], [27], [31]. Although this approach requires no additional HW support and performs well for applications with modest data-transfer requirements, SW-managed data copies introduce significant latency and CPU overhead. To eliminate these SW-managed copies, prior work has explored shared virtual memory (SVM), which allows accelerators to directly access virtual-memory-managed application data [23], [41]. SVM has been extensively studied in heterogeneous systems ranging from GPUs and programmable many-core accelerators to FPGA-based computing platforms [2], [32], [41]. By removing explicit data movement and enabling a shared address space between SW and HW, SVM improves both programmability and performance. Existing SVM implementations generally rely on IOMMUs, translation caches, and accelerator-side memory-management HW to perform virtual-to-physical address translation [7], [33], [38]. While effective, these solutions incur substantial HW complexity and resource overhead, making them difficult to deploy in resource-constrained FPGA-based embedded systems. To address these challenges, several works have proposed lightweight FPGA-oriented SVM solutions. [41] introduced a configurable FPGA-resident IOMMU and demonstrated up to a 13× improvement over conventional copy-based communication. More recently, [23] evaluated an IOMMU-based shared virtual addressing architecture for RISC-V heterogeneous systems and reported approximately 3× improvement compared to copy-based method. These results demonstrate the potential of SVM for FPGA accelerators. However, both approaches still rely on dedicated addresstranslation HW and associated translation overheads. Furthermore, the relatively low effective memory bandwidth reported by recent soft-processor-based IOMMU implementations suggests that lightweight FPGA-oriented SVM architectures have not yet reached their full performance potential.

Updated Interrupt

Updated CSR

Atomic Data $ Instructions Invalidation

Arbiter

Data $ TLB/MMU

CPU0 CPU1

Instruction $ TLB/MMU

CPU2

Base CVA5

SOC L1 I$ L1 D$ L1 I$ L1 D$ L1 I$ L1 D$ L1 I$ L1 D$

Arbiter Arbiter Arbiter Arbiter +Snooping Logic

Architecture

CPU3

In contrast, our objective is to study operating-system overheads in heterogeneous systems, which requires a highly configurable platform capable of migrating OS functionality and SW ”taxes” into HW to improve performance. To achieve this, we build our framework around the lightweight CVA5 soft processor [30]. Compared to larger Linux-capable soft processors, CVA5 provides an excellent balance of FPGA resource efficiency, configurability, and performance, leaving sufficient resources for large HW accelerators while remaining fully modifiable [29]. A comparative study of Linux-capable soft processors also found that CVA5 delivers the highest performance among existing designs [16].

Platform

Litex UartTimer-SDcard DDR (Linux Image)

Simulation

PLIC CLINT ROM (BootLoader) BRAM (Stack Data)

Emulation VCU118

DMA ACC

Fig. 1. Overview of the proposed MEERKAT. Green blocks indicate modifications to the original CVA5 [30] architecture and SoC while the blue blocks show the updated blocks.

C. Interrupt/Polling Trade-offs Prior work has explored the trade-off between interruptdriven and polling-based event handling in operating systems, particularly in networking and storage systems. Linux’s NAPI framework [36] dynamically switches between interrupts and polling to reduce interrupt overhead under high event rates, while hybrid interrupt-polling schemes have demonstrated that polling can provide higher throughput and lower processing overhead than pure interrupt-driven execution when events occur frequently [34], [35]. More recently, studies in lowlatency storage systems have shown that polling can outperform interrupts in both latency and energy efficiency for fast I/O operations [15]. However, these approaches typically make decisions based on device load, traffic characteristics, or fixed thresholds rather than the expected completion time of individual operations. To the best of our knowledge, prior work has not explored using execution-time prediction to dynamically select between polling and interrupt-driven completion handling for HW accelerators. Our work addresses this gap by leveraging scheduler-provided execution-time estimates to select the completion mechanism that minimizes operatingsystem overhead for each accelerator invocation. D. Scheduling Scheduling in heterogeneous systems is a well-studied problem. Prior works have demonstrated substantial performance improvements through runtime scheduling across multiple CPUs and GPUs by dynamically or statically selecting the most suitable computing resource for each task [1], [3], [5], [6], [12], [14], [17], [19], [40], [42], [43]. However, comparatively little attention has been given to the overheads incurred by the scheduling infrastructure itself. More recent frameworks such as IRIS and CEDR [22], [27] employ worker-based execution models in which the scheduler and each heterogeneous computing resource are managed through dedicated worker threads. While effective, these approaches introduce additional SW overheads that can become significant, particularly for

A. Communication Tax The framework supports three communication mechanisms between the CPU and HW accelerators. Regardless of the communication model, three fundamental challenges must be addressed, highlighted by the purple boxes in Figure 2. First, user applications operate on virtual addresses, while accelerators require physical addresses to access memory. Second, data coherence must be maintained between copies of data residing in processor caches and data accessed by accelerators. Finally, O/S memory allocation produces physically non-contiguous pages, requiring a mechanism for accelerators to efficiently access scattered memory regions. The first communication model (copy-based method) allocates a dedicated memory region for accelerator use. As illustrated by the split DDR memory organization in Figure 2, the CPU and accelerator operate on separate memory spaces. The accelerator region is physically contiguous, non-virtualized, and uncached, allowing direct DMA access without requiring

L1 D$

NonContiguous Memory

L1$

SG DMA1

3 Problem 3: NonContiguous Read

REG

3

2231122

CPU Traditional Copy Memory Based Method Data Path

Contiguous Memory

Arbiter

ACC1

2

SG DMA0

L1 I$

2

Problem 2: Coherency Issue

3

L1$

L1 D$

1 Problem 1: Virtual to Virtual Physicall Translation Address

L2$

Scrachpad ACC0

L1 I$

Arbiter

L1 D$

2 Arbiter

L1 I$

REG

1 1

Arbiter

L1 D$

Arbiter

CPU0 CPU1 Arbiter CPU2

To systematically study O/S taxes in heterogeneous SoCs, we developed MEERKAT, a Linux-capable FPGA-based research framework built on the CVA5 RISC-V processor. CVA5 is a highly configurable RV32IMFD soft processor designed for FPGA deployment and extensibility. To support Linux execution and accelerator-aware O/S research, substantial modifications were made to both the processor architecture and the surrounding SoC infrastructure. These additions include support for virtual memory, privilege modes, interrupt management, cache coherence, multicore execution, and accelerator integration. Figure 1 presents an overview of MEERKAT, with the modifications to the original CVA5-based system highlighted in green and updated blocks marked in blue. To enable rapid design-space exploration and system evaluation, MEERKAT supports both simulation and FPGA emulation. For simulation, we leverage the LiteX platform, which provides fast Verilator-based cycle-accurate simulation of the complete SoC. For FPGA emulation, we implement the SOC using Vivado Block Design and deploy it on the Xilinx VCU118 development board. To maximize SW portability across both environments, the same Universal Asynchronous Receiver-Transmitter (UART), timer, interrupt controller, and peripheral infrastructure are used in both the LiteX and Vivado implementations. Consequently, a single SW stack—including the first-stage bootloader, OpenSBI, device tree, and Linux kernel—can be executed without modification on either platform. The Vivado-based implementation additionally enables the integration of vendor IP blocks, flexible clocking configurations, and high-performance FPGA interfaces, providing a practical environment for evaluating realistic heterogeneous SoC deployments.

4

CPU3

III. P ROPOSED MEERKAT

L1 I$

Scrachpad

Physicall Address

fine-grained accelerator invocations. In this work, we quantify these scheduling overheads and demonstrate techniques to reduce their impact.

1

1 3 1 Virtual Memory Aware Communication Data Path

2 4 2

Acc Tightly Coupled Memory Communication Data DDR

Path

BUS

Fig. 2. Overview of the three CPU–accelerator communication models supported by the framework and the operating-system challenges associated with each.

address translation or cache-coherence support. The primary disadvantage of this approach is the additional data movement overhead. Before execution, application data must be copied from the CPU’s virtual-memory space into the acceleratoraccessible region. After computation completes, the results must be copied back into the application’s address space. These extra memory transfers increase execution time and consume memory bandwidth, particularly because accesses to the uncached region incur full DDR latency. Despite these limitations, dedicated accelerator memory remains attractive for storing accelerator-private data structures, such as neuralnetwork weights, lookup tables, and configuration data that are infrequently accessed by the CPU. The second communication model employs a tightly coupled interface between the CPU and accelerator. In this approach, the accelerator is attached directly to on-chip memory accessible through dedicated uncached load and store operations. CVA5 supports this capability by allowing selected address regions to bypass the cache hierarchy and access FPGA BRAM resources directly. Because all data movement is performed by the CPU, virtual-memory translation, cache coherence, and memory protection are handled transparently by the processor and O/S, eliminating the need for acceleratorside support. The main limitation of this approach is the limited capacity of FPGA on-chip memory. Application data typically resides in DDR memory and must therefore be explicitly transferred between DDR and the accelerator-accessible Block RAM (BRAM). While accesses to the tightly coupled interface can complete in a single cycle, the cost of moving large

Allow Custom

Decode

Wait for TLB/MMU Results

Mem Read

Mem Read

PTE1.PPN Offset

Mem Read

Address Exist

PTE0.PPN VPN1

Offset

TLB

Satp.ppn VPN0

rs1 func3 rd Opcode offset

LSQ FIFO Pop and Write Back Send to to rd L1$ register Load Store Unit

VPN1 VPN0

Preprocessing Prevent Custom Offset(rs1) Instruction Address FIFO

Decode

Incorrect Create Permission Page TLB/MMU

Valid PermissionYes NO

Physicall Address

Zero

Write Back to rd Register

Custom Block

Fig. 3. CIVAT instruction datapath. A virtual address supplied in and translated using the processor’s existing TLB/MMU mechanisms, and the resulting physical address is forwarded to the custom execution unit for writeback to the destination register.

Hot Pages Cold Pages 100

26 1,850

71.05x Speedup

Mostly dominated by the page fault

101

Custom Instruction Software (IOCTL)

1.50x Speedup 10,509 15,800

104 102 103 Average Clock Cycles (Log Scale)

105

Fig. 4. Virtual-to-physical address translation latency using the conventional Linux SW path and the proposed CIVAT instruction.

datasets between DDR and on-chip memory limits scalability. Consequently, this communication model is best suited for fine-grained accelerator invocations and workloads with small working sets. B. VMA Communication Method To eliminate the overheads associated with explicit data copying, the framework also supports a SVM communication model. In this approach, accelerators directly access application data structures using virtual addresses, allowing CPU and accelerator resources to operate on the same memory space without intermediate copies. However, enabling this model requires mechanisms for virtual-to-physical address translation, coherence management, and efficient access to physically noncontiguous memory pages. The design and implementation of this mechanism are described in the following section. C. Custom Instruction Virtual Address Translation (CIVAT) Obtaining the physical address corresponding to a virtual address is a significant source of overhead when accelerators require direct memory access. In Linux, if the target page has not yet been allocated, the processor must first service a page fault and create the corresponding page-table entry. Even when the page is already present, user-space applications cannot directly access physical addresses. Instead, a kernel call is required to walk the page tables, locate the physical

page, and return the translation. As shown in Figure 4, this process requires thousands to tens of thousands of cycles for a single address translation, making it a substantial O/S tax for accelerator communication. To reduce this overhead, we implemented CIVAT within the CVA5 processor. Figure 3 illustrates the required architectural modifications. The instruction uses the custom RISC-V opcode 0x7B and accepts a virtual address through the source register (rs). When decoded, the instruction is simultaneously forwarded to the custom execution unit and the load-store unit. The load-store unit performs the normal address translation process using the existing TLB and MMU infrastructure but suppresses the generation of any memory read or write requests. Once translation completes, the resulting physical address is forwarded directly to the custom execution unit, which writes the translated address back to the destination register (rd). This approach preserves the protection guarantees of the O/S. If the calling process lacks permission to access the requested page, the processor raises a page fault exactly as it would for a conventional memory access. Similarly, if the page has not yet been allocated, the normal page-fault handling mechanism is invoked, allowing Linux to allocate and map the page before the translation is returned. Consequently, the custom instruction exposes only physical addresses that are already accessible to the requesting process while leveraging the existing virtual-memory infrastructure. Figure 4 shows that the proposed mechanism reduces translation overhead by up to 71× when the page is already present in memory. Even in the page-fault case, where O/S page allocation dominates execution time, the proposed approach still achieves a 1.5× improvement over the conventional SWbased translation path. D. Scatter-Gather Page Transfer (SGPT) To address the challenge of non-contiguous physical pages in virtual-memory systems, we leverage the scatter-gather DMA (SGDMA) engine provided by Vivado and develop a SW driver that automatically converts virtual memory regions into DMA-accessible descriptor chains. Algorithm 1 presents a simplified view of the driver operation. The SGDMA engine requires a transmit (TX) chain describing the memory regions to be read and a receive (RX) chain describing the destination regions to be written. Each chain consists of a list of physical-address and transfer-size pairs stored in memory, with the physical address of the chain head provided to the DMA engine. Since the accelerator memory region shown in Figure 2 occupies a fixed physical-address range, both the TX and RX descriptor chains are allocated within this region, allowing the DMA HW to access them directly. To construct these chains, the driver receives a list of arrays or matrices that will either be transferred to the accelerator or populated with accelerator results. As shown in Lines 22– 23 of Algorithm 1, the driver invokes the BuildChain() procedure to generate the corresponding DMA descriptors.

For each matrix, the procedure iterates through all rows and obtains the physical address of the first element using the proposed CIVAT instruction. Because a row may span multiple virtual-memory pages, the driver determines the remaining bytes available in the current 4 KB page and creates a descriptor covering only the contiguous portion of memory within that page. The process then advances to the next page and repeats until the entire row has been described. This procedure converts a virtually contiguous data structure into a sequence of DMA transactions. An additional advantage of the proposed driver is its support for two-dimensional memory transfers. Rather than requiring SW to manually repack, transpose, or reorient matrix tiles before accelerator execution, the driver can directly describe each row through the descriptor chain. This capability is particularly beneficial for matrix-multiplication accelerators, where input matrices tiles are frequently accessed in different orientations. By allowing the DMA engine to gather data directly from the original matrix layout, the driver eliminates SW-side data rearrangement overhead and reduces unnecessary memory copies. After the descriptor chains have been generated, the driver programs the SGDMA engine with the addresses of the TX and RX chain heads and initiates the transfer. As a result, the accelerator can efficiently access application data regardless of the underlying physical-page layout, eliminating the requirement for physically contiguous allocations while remaining fully compatible with Linux virtual memory. Unlike traditional approaches that require contiguous DMA buffers or explicit copies into accelerator memory, the proposed SGDMA driver enables accelerators to operate directly on application data allocated by the Linux virtual-memory system. E. HW-Managed Accelerator Coherence (HMAC) Cache coherence presents another challenge when accelerators access data residing in the CPU’s virtual-memory space. CVA5 employs write-through L1 data caches and a snoopingbased coherence mechanism to invalidate cache lines modified by other processor cores. A straightforward solution would be to expose every accelerator memory access to the snooping bus so that corresponding cache lines can be invalidated. However, this approach does not scale well as the number of accelerators in the system increases. Furthermore, DMA engines typically issue burst transactions over AXI, meaning that only the starting address and burst length are visible on the bus rather than every individual address accessed by the accelerator. In contrast, processor stores naturally generate individual memory transactions that can be directly observed by the coherence infrastructure. To address this challenge, we designed the HW module shown in Figure 5. The module monitors AXI transactions generated by each accelerator and extracts the starting address and transfer size of every burst. This information is stored in a dedicated address and size register associated with each accelerator. The coherence unit then generates cache-line invalidation requests on behalf of the DMA engine by traversing

Algorithm 1 Scatter-Gather DMA Handling 1: procedure B UILD C HAIN(matList, ll) 2: for each mat ∈ matList do 3: for row ← 0 to rows − 1 do 4: vaddr ← ROW S TART(mat, row) 5: bytes ← ROW S IZE(mat) 6: while bytes > 0 do 7: paddr ← VirtToPhys(vaddr) 8: pageLef t ← 4096 − (paddr mod 4096) 9: chunk ← min(bytes, pageLef t) 10: A PPEND(ll, paddr, chunk) 11: vaddr ← vaddr + chunk 12: bytes ← bytes − chunk 13: end while 14: end for 15: end for 16: F INALIZE C HAIN(ll) 17: end procedure 18: procedure T RANSFER(dst, src, test ) 19: R ESET DMA 20: tx ← [srcA , srcB , . . . ] 21: rx ← [dstA , dstB , . . . ] 22: B UILD C HAIN(tx, txLL) 23: B UILD C HAIN(rx, rxLL) 24: ovhd ← S ETUP OVERHEAD 25: irq ← (test − ovhd) > Tswitch 26: if irq then 27: tok ← E NABLE IRQ 28: end if 29: S ET C HAIN H EADS(txLL, rxLL) 30: C OMMIT TAILS(txLL, rxLL) 31: if irq then 32: WAIT F OR IRQ(tok) 33: else 34: while ¬T RANSFER D ONE do 35: end while 36: C LEAR F LAGS 37: end if 38: S TOP DMA 39: end procedure

the address range covered by the burst. Starting from the burst address, invalidation requests are issued at cache-line granularity until the entire transfer region has been covered. As a result, all cache lines potentially modified by the accelerator are invalidated from the processor caches, ensuring coherence between accelerator-generated data and CPU accesses. An additional optimization is included to efficiently handle DMA transfers that are split into multiple consecutive AXI bursts. When a newly observed burst begins immediately after the address range of a previously tracked burst, the coherence unit merges the two transactions by extending the stored transfer size rather than creating a new invalidation region. This behavior, illustrated by input two of the size multiplexer in Figure 5, makes sure the registers would not override an already existing one. F. Invocation and Scheduling Tax Previous work has shown that a significant portion of accelerator invocation overhead originates from the SW stack required to access the accelerator, particularly the use of IOCTL calls to communicate with kernel-space drivers. To reduce this invocation tax, we designed the accelerator interface as a user-space library rather than a traditional device driver. As

+$ line size

Accelerator Function

Address Register Address Register IF Address 2 Register Address 3 Register S1 S0 S0 Register S1Change Choice 0

Address Mux

Process/ Thread Process/ Thread Func1 Process/ Thread Process/

1

(Address,Size) (Address,Size) Arbiter (Address,Size)

IF

Arbiter

Invalidation Address

3

Size Register 2

Size Mux

+

(Address,Size)

1

-$ line size

0

IF

Size Register

Func1 Acc() Thread Func1 Acc() Func2 Func1 Acc() Func2 Acc() Func2 Func2

IF

Size Register Size Register

Fig. 5. Overview of HMAC, a HW-assisted cache-coherence mechanism that monitors accelerator DMA transactions and generates cache-line invalidation requests for the CVA5 snooping-based cache hierarchy.

illustrated in Figure 6, applications invoke accelerators through a function call, eliminating the kernel transition and IOCTL overhead associated with conventional accelerator frameworks. We also investigated the scheduling mechanisms employed by existing open-source heterogeneous computing frameworks and observed that many dedicate a separate scheduling process or worker thread in addition to one or more worker threads responsible for accelerator management. While this design simplifies implementation, it introduces additional O/S overheads. In many workloads, scheduling decisions occur infrequently, causing the scheduler thread to spend most of its execution time waiting for work while still consuming system resources through polling, context switches, or synchronization operations. To eliminate this scheduler tax, we integrated the scheduling logic directly into the accelerator invocation path, as shown in Figure 6. Instead of forwarding requests to a dedicated scheduler process, the application thread invokes the scheduler immediately before accelerator execution. This approach removes the need for a separate scheduler thread, reducing context-switch overhead and eliminating idle scheduler execution. Integrating scheduling into the application thread introduces concurrency challenges because scheduling decisions rely on shared system-wide state, including accelerator availability and performance statistics. To ensure correctness, we maintain scheduler and accelerator locks within a shared data structure accessible by all processes. These locks serialize access to shared scheduling information while allowing competing processes to sleep rather than spin while waiting for accelerator resources. As a result, the framework avoids both the overhead of dedicated scheduler threads and the CPU utilization penalties associated with busy-wait synchronization. G. Interrupt Tax Another source of O/S overhead is the context switch associated with accelerator completion interrupts. In a conventional interrupt-driven execution model, every accelerator comple-

Phase1: Scheduling

Sleep: Awaits Lock NO

Start Open or Create a Shared Memory Section

Acquire Scheduler Lock

Release the Scheduler Lock

Acquire Accelerator Lock

Calculate Busy Time+ Estimated Time for each Resource

Select Optimal Accelerator

YES

Phase2: Virtualization YES

RUN DRIVER

NO

Sleep: Awaits Lock

Release Accelerator Lock

End

Fig. 6. Overview of the accelerator invocation and scheduling path. The design consists of two main components, scheduling and virtualization, coordinated through a shared-memory interface accessible by all processes and threads.

tion generates an interrupt that forces the O/S to perform interrupt handling, sleep the current process, and execute one or more context switches. For short accelerator executions, this overhead can become comparable to or even exceed the accelerator execution time itself, causing the processor to spend a significant fraction of its time performing O/S work rather than useful computation. Our experiments revealed that no single completion strategy is optimal across all workloads. For short-running accelerator tasks, actively polling for completion is often more efficient than sleeping and waiting for an interrupt because it avoids the context-switch overhead entirely. In contrast, for longrunning accelerator executions, interrupt-based completion is preferable because it allows the processor to execute other work while the accelerator is running. Consequently, we implemented IPA that dynamically selects between polling and interrupt-driven execution. The proposed policy leverages the execution-time estimates already generated by the scheduler. As shown in Lines 31– 37 of Algorithm 1, the estimated accelerator execution time is compared against a predefined threshold. If the expected execution time exceeds this threshold, the process is suspended and later resumed through an interrupt. Otherwise, the application polls the accelerator completion status directly, avoiding the overhead of interrupt processing and context switching. A practical challenge is that accelerator interrupts are connected to the PLIC and managed through a Linux device driver. Consequently, an interrupt would still trigger O/S handling even when polling is selected. To avoid this unnecessary overhead, the framework dynamically enables or disables accelerator interrupts before execution, as shown in Lines 25– 28 of Algorithm 1. When polling is selected, the interrupt source is disabled, preventing the generation of an unnecessary context switch. When interrupt-driven execution is selected, the interrupt is enabled and normal O/S notification is used. This approach ensures that the completion mechanism itself does not introduce avoidable O/S overhead.

IV. E XPERIMENTAL S ETUP We evaluate the proposed framework on a Xilinx VCU118 FPGA platform with all system components operating at 100 MHz. The system boots using the LiteX built-in BIOS as the first-stage bootloader, followed by OpenSBI v0.8 and Linux kernel v5.18.0-rc7. All experimental results were collected using this configuration.

TABLE II F RAMEWORK OVERHEAD AND MEMORY THROUGHPUT AT 100 MH Z Operation / Benchmark lat_syscall null bw_mem 1m wr bw_mem 1m rd bw_mem 1m rdwr

Raw Metric 2.97 µs 271.09 MB/s 66.59 MB/s 53.76 MB/s

Performance in Clock Cycles 297 cycles 0.37 cycles/byte 1.50 cycles/byte 1.86 cycles/byte

A. Framework The baseline evaluation builds upon the CVA5 processor core, configured with two-way set-associative instruction and data caches (512 lines, 8-word width), a 512-entry branch predictor, an 8-entry return address stack, and 64entry TLBs. To support Linux execution, some modifications and updates were made to the original CVA5 architecture. As detailed in Table III, extending the baseline CVA5 to a full Linux-capable core increases resource utilization by only 1 BRAM and 211 CLBs (805 versus 594 CLBs), with no additional DSP footprint. The surrounding framework infrastructure—including the multicore wrapper, PLIC, CLINT, and LiteX IP—consumes only 615 CLBs total without using BRAMs or DSPs. This lightweight hardware modification enables efficient OS-level execution without incurring significant architectural latency. As shown in Table II, low-level system operations inside the framework remain minimal: a null system call (lat_syscall) resolves in 297 clock cycles (2.97 µs). Furthermore, memory throughput benchmarks (bw_mem) demonstrate low-overhead data handling, requiring 0.37 cycles/byte for write operations and 1.50 cycles/byte for reads. Building on this optimized single-core baseline, the framework scales predictably in multicore configurations. Table I presents the parallel CoreMark execution results across up to four cores. Crucially, the software and framework layer introduce negligible overhead: individual core performance remains pinned at 2.59 CoreMark/MHz, matching the reported baremetal baseline [16]. Doubling the workload from one to two cores increases total execution time by only 0.18 s (1.1%), while a four-core configuration maintains 94.7% scaling efficiency, confirming the overall framework scalability. TABLE I CVA5 MULTI -C ORE SCALING AND C ORE M ARK PERFORMANCE AT 100 MH Z Core Config. 1 Core 2 Cores 3 Cores 4 Cores

Parallel Threads 1 2 3 4

Total Iters 4,000 8,000 12,000 16,000

Elapsed Time (s) 15.45 15.63 15.78 16.32

CoreMark Score 258.88 511.71 760.55 980.69

CoreMark/MHz (at 100 MHz) 2.59 5.12 7.61 9.81

Scaling Eff. (%) 100.0% 98.8% 97.9% 94.7%

B. Applications To evaluate the solution to the proposed taxes, we implemented four HW accelerators and four representative applications generating accelerator requests. The benchmark suite includes two communication-bound and two compute-bound applications, enabling evaluation of invocation mechanism,

TABLE III HW RESOURCE UTILIZATION BREAKDOWN Component Baseline CVA5 (1 Core) Linux CVA5 (1 Core) CIVAT Multicore Wrapper HMAC PLIC CLINT Litex IP

DSP 4 4 – 0 – 0 0 0

BRAM 19.5 20.5 – 0 – 0 0 0

CLB 594 805 26 294 138 101 88 132

LUT 2905 4593 2 1001 422 489 324 587

FF 1908 2745 38 448 126 223 365 607

interrupt policy, and scheduler. Table IV summarizes the characteristics of a single application execution; additional repetitions used in specific experiments are stated separately. The communication-bound applications are: Pulse Doppler(PD): Performs FFT and inverse FFT operations using the Xilinx Vivado FFT IP. Since the FFT core is fully pipelined, performance is primarily limited by data movement rather than computation. As shown in Table IV, each invocation transfers approximately 1 KB and achieves a 15× speedup over CPU execution. • Memory Copy(MC): Uses a FIFO connected directly to the DMA engine to isolate communication overhead without performing computation. Random-sized transfers between 1 B and 250 KB (125 KB average) achieve a 9× speedup over SW memcpy. For maximum-size transfers of 1 MB, the DMA-based implementation achieves a 26× speedup compared to CPU memcpy. •

The compute-bound applications are: Matrix Multiplication(MM): Implements a tiled GEMM accelerator in Vivado HLS with AXI-Stream interfaces connected to the DMA engine. The benchmark performs 512 × 512 matrix multiplications while varying the tile size to evaluate different computation-tocommunication ratios. Tile sizes of 64 and 128 achieve 238× and 770× speedups, respectively, over the CPU implementation. • Synthetic Accelerator(SA): A configurable accelerator with programmable execution latency. It emulates accelerators with different computation times and speedups, enabling controlled evaluation of the scheduler and the proposed Interrupt Policy Adaptation (IPA). Each invocation uses a random latency, allowing requests to favor either polling or interrupt-based completion. •

TABLE V P ERFORMANCE COMPARISON OF COPY- BASED , THE PREVIOUS IOMMU- BASED SVM IMPLEMENTATION [23], AND THE VMA METHOD . Metric

CPU SVM Our Max Copy-based [23] VMA Performance Data Size 1 MB 64 KB 1 MB 1 MB Bus Width 32 32 64 64 Cycles 57.29M 2.00M 219.8K 140.8K Cycles/B 57.29 30.52 0.220 0.141 BW (MB/s) 1.75 3.28 454.96 710.23 Peak BW 400 400 800 800 Eff. (%) 0.44 0.82 56.87 88.78 Speedup 1.00× 1.88× 260.63× 406.86×

C. Scheduling Policies We tested five scheduling policies to find the most effective one. • CPU Only(CO): Every task is executed on available CPUs, regardless of accelerator availability or system load. • Accelerator Only(AO): Every task is dispatched to one designated accelerator. If the accelerator is busy, incoming tasks wait until it becomes available. • Busy-Aware (Fuse): Adapted from Fuse [18], this policy dispatches a task to the accelerator whenever it is idle; otherwise, the task is immediately executed on the CPU. • Random: For each task, the scheduler randomly selects one of the available computing resources (CPU or accelerator) without considering execution time, resource utilization, or queue occupancy. • Earliest Finish Time (EFT): Our proposed scheduler estimates the completion time on every available computing resource and assigns the task to the resource predicted to finish first. The estimate accounts for both the expected execution time of the task and the current workload of each resource, enabling dynamic load balancing across heterogeneous processors and accelerators. V. E XPERIMENTAL R ESULTS A. Communication tax results We evaluated the three communication mechanisms introduced in Section III-A. Figure 7 compares their communication latency across different transfer sizes. The tightly coupled interface provides the lowest latency for small transfers because the accelerator communicates directly with on-

Tightly Connection vs Copy-based vs VMA 572857

106

Copy-based VMA Our Tightly Coupled Setup Time (VMA) Speedup Factor

105 104

5745

2198 6370

608 568 264 302 100.95x 790 250 259 260 166 111 120 300 102 150 150 150 150 170 137 19.02x 45 48 42 0.46x 0.64x 2.30x 0.44x 1

10

100

1k

189572350

300 57338 260.63x1937463500 250

103

101

Speedup Factor (Copy-based vs VMA)

App Calls Avg. Tx HW CPU Time ACC Total HW × MM 40 49KB GEMM64 4004 4003 16.8 238 MM 40 196KB GEMM128 4004 4003 5.2 770 MC 10KB 125K FIFO 40.8 40.4 4.5 9.0 PD 512 1KB FFT 4.6 3.0 0.2 15

Execution Time (us) - Log Scale

TABLE IV B ENCHMARK CHARACTERISTICS . C ALLS DENOTES THE NUMBER OF ACCELERATOR INVOCATIONS , AVG . T X THE AVERAGE TRANSFERRED DATA PER INVOCATION , CPU T IME THE CPU- ONLY EXECUTION TIME , ACC T OTAL THE ACCELERATOR - BASED EXECUTION TIME INCLUDING COMMUNICATION OVERHEAD , HW THE ACCELERATOR EXECUTION TIME , AND × THE SPEEDUP OVER CPU EXECUTION . T HE UNIT OF TIME IN THIS TABLE IS IN SECONDS .

10k 100k 1M

Transfer Size (Bytes)

200 150 100 50

10M 100M 0

Fig. 7. Communication latency of the three accelerator communication mechanisms supported by MEERKAT.

chip BRAM without accessing external memory. However, its scalability is fundamentally limited by the available onchip memory resources. In our implementation, the maximum transferable data size is only 1 KB. The copy-based communication method exhibits significantly higher latency, and the performance gap widens as the transfer size increases. This approach requires the CPU to explicitly copy data between the application’s virtual memory and the accelerator’s dedicated memory region, causing the communication overhead to grow linearly with the amount of transferred data. Furthermore, the maximum transferable data size is limited by the size of the reserved accelerator memory in DDR. In contrast, the proposed VMA method combines the low SW overhead of direct memory sharing with the scalability of off-chip memory. Since the accelerator directly accesses the application’s virtual memory, no explicit memory copies are required, and the transfer size is no longer constrained by a dedicated accelerator memory region. As shown in Figure 7, VMA outperforms both previous communication methods for large transfers while remaining scalable to arbitrarily large data sizes. Figure 7 also separates the communication time into address translation and data transfer. Interestingly, approximately 33% of the total communication time is spent translating virtual addresses, while the remaining 66% is devoted to the actual DMA transfers. This observation suggest that further optimization of the translation mechanism could substantially improve performance. Table V compares our implementation against both the conventional CPU copy-based approach and the only previously published SVM implementation for RISC-V soft processors [23]. VMA achieves an effective bandwidth of 454.96 MB/s, corresponding to 56.9% of the theoretical 64bit memory bandwidth. Without address translation, the same communication path reaches 710.23 MB/s (88.8% efficiency). Compared to the conventional copy-based approach, the

Speedup Factor

4

CPU

100

Cor 2 es

20 f APPs er o umb

N

1

1

Fig. 8. Performance comparison between the proposed scheduler and the worker-based scheduler used in CEDR [27].

Scheduler Effect on Memory Bound Applications

Pulse Doppler Application

Memcopy Application

166 S 150

400 118 S

100

85 S

83 S

50 0

CO

FUSE

EFT

AO

Configuration / Scheduler

Execution Time (S)

Figure 8 compares the performance of our scheduler with the worker-based scheduler used by CEDR [27] using the PD benchmark. Across all configurations, our scheduler improves application wall clock time by 1.5×–5.8×. The largest improvement is observed when the system executes only a small number of applications. For example, with a single CPU core and a single application, our framework achieves a 5.8× speedup. In this scenario, there is little opportunity for parallel execution, making the dedicated scheduler thread in CEDR largely idle while still consuming CPU resources. Furthermore, CEDR allocates an additional worker thread for each HW accelerator, which also remains mostly unused. As a result, a significant fraction of the available processing resources is devoted to scheduler management rather than useful application execution. As the number of concurrent applications increases, the scheduler and accelerator worker threads become increasingly utilized, reducing the relative overhead of the worker-based design. Consequently, the performance gap narrows. Nevertheless, even when executing 100 PD applications simultaneously, our scheduler still achieves a 1.5×–2× improvement over CEDR by eliminating the scheduler worker thread and reducing scheduling overhead. Finally, PD is primarily CPU-bound because the FFT accelerator is fully pipelined and spends relatively little time computing compared to the SW processing surrounding each invocation. Consequently, increasing the number of CPU cores consistently improves throughput for both scheduling approaches.

Our Method

400 200 100 50 20 10 5 2 1

Execution Time (S)

B. Schedluing Tax

CEDR (Baseline)

Execution Time (sec)

proposed VMA method reduces the communication overhead by up to 260×. It also substantially outperforms the previous IOMMU-based SVM implementation [23], requiring only 0.22 cycles/byte compared to 30.52 cycles/byte reported in their work while the CIVAT consumes only 26 CLBs and HMAC 138 III. The results for the previous SVM implementation are taken directly from the published paper, as we did not reimplement their design.

388 S

300 221 S

200 100 0

CO

FUSE

107 S

107 S

EFT

AO

Configuration / Scheduler

Fig. 9. Different Scheduler effect on the overall application execution time. For the PD there are 100 of calls while for the MC there is 20.

C. Communication Bound Applicaions Figure 9 presents the results for two communication-bound applications. For the PD application, the SoC was configured with two FFT accelerators available to the schedulers; similarly, the Memcpy application was configured with two FIFO accelerators. However, in both cases, advanced scheduling yields no positive impact on performance. Because bus bandwidth is constrained, data transfers from one accelerator force the other to wait. As a result, throughput remains virtually identical to a system with AO policy. D. Computation Bound Applicaions For compute-bound workloads, we evaluate both MM and SA. Figure 10 illustrates the performance of the evaluated scheduling policies under different accelerator speedups relative to the CPU.

Across all configurations, the proposed EFT scheduler achieves peak performance by dynamically scheduling across all available computing resources. For example, when an accelerator is 4× faster than the CPU, fully configured EFT significantly outperforms both the AO policy and EFT(2ACC)—which restricts decisions to two accelerators. This demonstrates that for compute-bound workloads, cooptimizing CPUs and accelerators yields substantially higher throughput than relying on accelerators alone. An interesting observation is the strong performance of the Fuse scheduler despite its simple heuristic. When the accelerator provides a moderate speedup, immediately executing tasks on the CPU whenever the accelerator is busy is often sufficient to achieve performance close to that of EFT. However,

Impact of Polling Threshold on Execution Time (2 ACC, 4 CPU | ACC 2x Faster | Random Delay)

Fuse EFT(2ACC) Random EFT(2ACC,4CPU) AO CO (a) Accelerator 4x (b) Accelerator 50x (c) Accelerator 1x 400 Faster than CPU 250 Faster than CPU 150 Faster than CPU 350 125 200 300 100 250 150 200 75 100 150 50 100 50 25 50 0 12345 10 20 0 12345 10 20 0 12345 10 20 Number of Apps Number of Apps Number of Apps

Fig. 10. Performance of different scheduling policies for SA as the accelerator speedup relative to the CPU varies.

1GEMM vs 2GEMM 1GEMM 2GEMM

Execution Time(s)

200 150

129

100 50

17

0 1

40 27.1

2

60 40.4

3

82.8 54.7

4

102.3 102

149.3 124

175.4

192.6

141

148.6

8

9

209 164

75.05

5

6

Number of Apps

7

10

Fig. 11. Effect of accelerator replication on a tiled MM workload.

this heuristic becomes ineffective as the accelerator speedup compare to the CPU increases. In these cases, executing a task on the CPU may take significantly longer than waiting for the accelerator to become available, making Fuse’s scheduling decision suboptimal. Conversely, when the CPU and accelerator provide similar execution times, exploiting both resources becomes increasingly beneficial. In this regime, scheduling tasks across both CPUs and accelerators nearly doubles the achievable throughput compared to policies that dispatch work exclusively to accelerators, demonstrating the importance of using all the available resources in the system. To evaluate the scheduler on a realistic workload, we implemented MM application. Figure 11 evaluates the impact of scheduling across multiple GEMM64 accelerators. We compare systems containing one and two identical GEMMs with the tile size of 64 using the proposed EFT scheduler. When multiple applications execute concurrently, distributing requests across two accelerators improves overall throughput by approximately 21% compared to a single-accelerator configuration. These results demonstrate that accelerator replication provides an additional dimension of parallelism. Similar to increasing the number of processor cores, adding multiple accelerators allows independent application requests to execute concurrently, enabling the scheduler to further increase system throughput.

Execution Time (s)

Execution Time (S)

Scheduler Effect on SA 66 64 62

Optimal Threshold (5e+03 ns: 57.8 s)

60 58

0 5 × 101 5 × 102 5 × 103 5 × 104 5 × 105 5 × 106 5 × 107 5 × 1010 (Interrupt) (Polling)

Polling Threshold (ns)

Fig. 12. Performance of the proposed IPA for different interrupt-to-polling transition thresholds in SA.

E. Interrupt+Polling Our experiments showed that polling consistently outperforms interrupts for short-running workloads such as FFT and memory-copy operations, where the context-switch overhead is comparable to the accelerator execution time. In contrast, interrupt-driven execution is preferable for compute-bound applications, where accelerator execution dominates the overall latency and the processor can perform useful work while waiting for completion. To evaluate IPA, we constructed a benchmark containing both short- and long-running accelerator tasks, forcing the scheduler to make completion-policy decisions across a wide range of execution times. Figure 12 compares several threshold values used to switch between polling and interrupts. The results show that a threshold of 5000 ns consistently provides the best performance. This value closely matches the measured context-switch overhead of our interrupt-handling path, indicating that the optimal transition point occurs when the expected accelerator execution time becomes comparable to the operating-system cost of servicing an interrupt. VI. C ONCLUSION AND F UTURE W ORKS This paper demonstrates that treating accelerators as firstclass CE rather than CPU-managed peripherals can substantially reduce the SW taxes that limit heterogeneous system performance. Using our FPGA-based research framework, we identified four major sources of overhead—communication, memory translation, interrupt handling, and scheduling—and proposed HW-SW co-designed mechanisms that reduce these costs, achieving up to 260× lower communication overhead and improving application wall clock time by 2–6× with a better scheduler. Future work will further increase accelerator autonomy by offloading virtual-memory translation through hardware IOMMU support, introducing streaming DMA to overlap communication with computation, exploring hardware-assisted scheduling, and extending the framework to dynamically reconfigurable heterogeneous systems. Together, these directions move toward heterogeneous platforms where CPUs and accelerators operate as equal, composable computing resources.

R EFERENCES [1] S. Aldegheri, S. Manzato, and N. Bombieri, “Enhancing performance of computer vision applications on low-power embedded systems through heterogeneous parallel programming,” in 2018 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC). IEEE, 2018, pp. 119–124. [2] AMD Inc., “2nd Generation AMD Embedded R-Series APU,” Product Brief, 2017, accessed: Jun. 2026. [3] C. Augonnet, S. Thibault, R. Namyst, and P.-A. Wacrenier, “Starpu: a unified platform for task scheduling on heterogeneous multicore architectures,” in European Conference on Parallel Processing. Springer, 2009, pp. 863–874. [4] N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti et al., “The gem5 simulator,” ACM SIGARCH computer architecture news, vol. 39, no. 2, pp. 1–7, 2011. [5] C. Bolchini, S. Cherubin, G. C. Durelli, S. Libutti, A. Miele, and M. D. Santambrogio, “A runtime controller for opencl applications on heterogeneous system architectures. sigbed rev. 15, 1 (march 2018), 29– 35,” 2018. [6] H. J. Choi, D. O. Son, S. G. Kang, J. M. Kim, H.-H. Lee, and C. H. Kim, “An efficient scheduling scheme using estimated execution time for heterogeneous computing systems,” The Journal of Supercomputing, vol. 65, no. 2, pp. 886–902, 2013. [7] Y.-k. Choi, J. Cong, Z. Fang, Y. Hao, G. Reinman, and P. Wei, “A quantitative analysis on microarchitectures of modern cpu-fpga platforms,” in Proceedings of the 53rd Annual Design Automation Conference, 2016, pp. 1–6. [8] J. Cong, Z. Fang, M. Huang, P. Wei, D. Wu, and C. H. Yu, “Customizable computing—from single chip to datacenters,” Proceedings of the IEEE, vol. 107, no. 1, pp. 185–203, 2018. [9] E. G. Cota, P. Mantovani, G. Di Guglielmo, and L. P. Carloni, “An analysis of accelerator coupling in heterogeneous architectures,” in Proceedings of the 52Nd Annual Design Automation Conference, 2015, pp. 1–6. [10] H. Esmaeilzadeh, E. Blem, R. St. Amant, K. Sankaralingam, and D. Burger, “Dark silicon and the end of multicore scaling,” in Proceedings of the 38th annual international symposium on Computer architecture, 2011, pp. 365–376. [11] H. Genc, S. Kim, A. Amid, A. Haj-Ali, V. Iyer, P. Prakash, J. Zhao, D. Grubb, H. Liew, H. Mao et al., “Gemmini: Enabling systematic deeplearning architecture evaluation via full-stack integration,” in 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 2021, pp. 769–774. [12] P. Georgiev, N. D. Lane, K. K. Rachuri, and C. Mascolo, “Leo: Scheduling sensor inference algorithms across heterogeneous mobile processors and network resources,” in Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, 2016, pp. 320–333. [13] A. A. Goksoy, A. Krishnakumar, M. S. Hassan, A. J. Farcas, A. Akoglu, R. Marculescu, and U. Y. Ogras, “Das: Dynamic adaptive scheduling for energy-efficient heterogeneous socs,” IEEE Embedded Systems Letters, vol. 14, no. 1, pp. 51–54, 2021. [14] C. Gregg, M. Boyer, K. Hazelwood, and K. Skadron, “Dynamic heterogeneous scheduling decisions using historical runtime data,” in Workshop on Applications for Multi-and Many-Core Processors (A4MMC), 2011, pp. 1–12. [15] B. Harris and N. Altiparmak, “When poll is more energy efficient than interrupt,” in Proceedings of the 14th ACM Workshop on Hot Topics in Storage and File Systems, 2022, pp. 59–64. [16] C. Heinz, Y. Lavan, J. Hofmann, and A. Koch, “A catalog and inhardware evaluation of open-source drop-in compatible risc-v softcore processors,” in 2019 International Conference on ReConFigurable Computing and FPGAs (ReConFig). IEEE, 2019, pp. 1–8. [17] C. Hsieh, A. A. Sani, and N. Dutt, “Surf: Self-aware unified runtime framework for parallel programs on heterogeneous mobile architectures,” in 2019 IFIP/IEEE 27th International Conference on Very Large Scale Integration (VLSI-SoC). IEEE, 2019, pp. 136–141. [18] A. Ismail and L. Shannon, “Fuse: Front-end user framework for o/s abstraction of hardware accelerators,” in 2011 IEEE 19th annual international symposium on field-programmable custom computing machines. IEEE, 2011, pp. 170–177.

[19] D. Kadjo, R. Ayoub, M. Kishinevsky, and P. V. Gratz, “A controltheoretic approach for energy efficient cpu-gpu subsystem in mobile platforms,” in Proceedings of the 52nd Annual Design Automation Conference, 2015, pp. 1–6. [20] S. Karandikar, H. Mao, D. Kim, D. Biancolin, A. Amid, D. Lee, N. Pemberton, E. Amaro, C. Schmidt, A. Chopra et al., “Firesim: Fpgaaccelerated cycle-exact scale-out system simulation in the public cloud,” in 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA). IEEE, 2018, pp. 29–42. [21] F. Kermarrec, S. Bourdeauducq, J.-C. L. Lann, and H. Badier, “Litex: an open-source soc builder and library based on migen python dsl,” arXiv preprint arXiv:2005.02506, 2020. [22] J. Kim, S. Lee, B. Johnston, and J. S. Vetter, “Iris: A portable runtime system exploiting multiple heterogeneous programming systems,” in 2021 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2021, pp. 1–8. [23] C. Koenig, E. Zelioli, and L. Benini, “Evaluating iommu-based shared virtual addressing for risc-v embedded heterogeneous socs,” in 2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 2025, pp. 1–7. [24] A. Kurth, A. Tretter, P. A. Hager, S. Sanabria, O. Göksel, L. Thiele, and L. Benini, “Mobile ultrasound imaging on heterogeneous multicore platforms,” in Proceedings of the 14th ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia, 2016, pp. 9–18. [25] A. Kurth, P. Vogel, A. Capotondi, A. Marongiu, and L. Benini, “Hero: Heterogeneous embedded research platform for exploring risc-v manycore accelerators on fpga,” arXiv preprint arXiv:1712.06497, 2017. [26] C.-K. Luk, S. Hong, and H. Kim, “Qilin: exploiting parallelism on heterogeneous multiprocessors with adaptive mapping,” in Proceedings of the 42nd Annual IEEE/ACM international symposium on microarchitecture, 2009, pp. 45–55. [27] J. Mack, S. Hassan, N. Kumbhare, M. Castro Gonzalez, and A. Akoglu, “Cedr: A compiler-integrated, extensible dssoc runtime,” ACM Transactions on Embedded Computing Systems, vol. 22, no. 2, pp. 1–34, 2023. [28] P. Mantovani, D. Giri, G. Di Guglielmo, L. Piccolboni, J. Zuckerman, E. G. Cota, M. Petracca, C. Pilato, and L. P. Carloni, “Agile soc development with open esp,” in Proceedings of the 39th International Conference on Computer-Aided Design, 2020, pp. 1–9. [29] E. Matthews, Z. Aguila, and L. Shannon, “Evaluating the performance efficiency of a soft-processor, variable-length, parallel-execution-unit architecture for fpgas using the risc-v isa,” in 2018 IEEE 26th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, 2018, pp. 1–8. [30] E. Matthews and L. Shannon, “Taiga: A new risc-v soft-processor framework enabling high performance cpu architectural features,” in 2017 27th International Conference on Field Programmable Logic and Applications (FPL). IEEE, 2017, pp. 1–4. [31] P. Meloni, G. Deriu, F. Conti, I. Loi, L. Raffo, and L. Benini, “A high-efficiency runtime reconfigurable ip for cnn acceleration on a mid-range all-programmable soc,” in 2016 International Conference on ReConFigurable Computing and FPGAs (ReConFig). IEEE, 2016, pp. 1–8. [32] NVIDIA Corporation, “NVIDIA Jetson TX2 Delivers Twice the Intelligence to the Edge,” NVIDIA Parallel Forall Blog, Mar. 2017, accessed: Jun. 2026. [Online]. Available: https://developer.nvidia.com/ blog/jetson-tx2-delivers-twice-intelligence-edge/ [33] A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. P. Gopal, J. Gray et al., “A reconfigurable fabric for accelerating large-scale datacenter services,” ACM SIGARCH Computer Architecture News, vol. 42, no. 3, pp. 13– 24, 2014. [34] K. Salah, K. El-Badawi, and F. Haidari, “Performance analysis and comparison of interrupt-handling schemes in gigabit networks,” Computer Communications, vol. 30, no. 17, pp. 3425–3441, 2007. [35] K. Salah and A. Qahtan, “Implementation and experimental performance evaluation of a hybrid interrupt-handling scheme,” Computer Communications, vol. 32, no. 1, pp. 179–188, 2009. [36] J. H. Salim, R. Olsson, and A. Kuznetsov, “Beyond softnet,” in 5th Annual Linux Showcase & Conference (ALS 01), 2001. [37] Y. S. Shao, S. L. Xi, V. Srinivasan, G.-Y. Wei, and D. Brooks, “Codesigning accelerators and soc interfaces using gem5-aladdin,” in 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 2016, pp. 1–12.

[38] J. Stuecheli, B. Blaner, C. Johns, and M. Siegel, “Capi: A coherent accelerator processor interface,” IBM Journal of Research and Development, vol. 59, no. 1, pp. 7–1, 2015. [39] V. Suresh, B. Mishra, Y. Jing, Z. Zhu, N. Jin, C. Block, P. Mantovani, D. Giri, J. Zuckerman, L. P. Carloni et al., “Mozart: Taming taxes and composing accelerators with shared-memory,” in Proceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques, 2024, pp. 183–200. [40] H. Topcuoglu, S. Hariri, and M.-Y. Wu, “Performance-effective and low-complexity task scheduling for heterogeneous computing,” IEEE transactions on parallel and distributed systems, vol. 13, no. 3, pp. 260– 274, 2002. [41] P. Vogel, A. Marongiu, and L. Benini, “Exploring shared virtual memory for fpga accelerators with a configurable iommu,” IEEE Transactions on Computers, vol. 68, no. 4, pp. 510–525, 2018. [42] Y. Wen, Z. Wang, and M. F. O’boyle, “Smart multi-task scheduling for opencl programs on cpu/gpu heterogeneous platforms,” in 2014 21st International conference on high performance computing (HiPC). IEEE, 2014, pp. 1–10. [43] H. Zhou and C. Liu, “Task mapping in heterogeneous embedded systems for fast completion time,” in Proceedings of the 14th International Conference on Embedded Software, 2014, pp. 1–10.

Related documents

• Operating System Support for Accelerators as First-Class Compute Elements in Heterogeneous Systems#1455861
• Argus: Agentic, Reference-Calibrated, Tree-Guided, System-Software-Level Bottleneck Localization#1130089
• Argus: Agentic, Reference-Calibrated, Tree-Guided, System-Software-Level Bottleneck Localization#1108860
• Higher-level synthesis methodology for heterogeneous software and hardware architectures#1312755
• Minimum RISC-V: Maximum FPGA Efficiency#1356888