The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics Yiwei Yang
Andi Quinn
UC Santa Cruz USA [email protected]
UC Santa Cruz USA [email protected]
arXiv:2607.26335v1 [cs.DC] 28 Jul 2026
Abstract Modern GPU clusters are no longer only collections of accelerators behind a host CPU. They are fabrics: GPU kernels expose execution phase, warp behavior, page faults, KV blocks, and tensor semantics; NICs and DPUs expose queue pairs, work queues, completions, retransmits, and congestion; CXL switches and memory devices expose placement, sharing, ordering, persistence, and near-memory movement. Yet today’s control planes split these signals across mutually blind policy systems. The GPU scheduler does not know which RDMA queue carries decode KV fetches, and the network or CXL fabric does not know whether a transfer belongs to prefill, decode, MoE routing, checkpointing, or background training. We present fabric_ext, an eBPF middleware compiler and runtime for extensible OS policies over GPU–CXL fabrics. fabric_ext lets one policy program execute across GPU hooks, driver/runtime hooks, DPU/NIC hooks, and CXL switch or near-memory hooks. The key abstraction is a semantic movement graph: edges describe bytes, stride, reuse distance, read/write ratio, source and destination, ordering requirement, alias set, ownership, and transformations such as Move, Quantize, Compress, Checksum, Filter, Reduce, Scatter/Gather, Replicate, and Persist. The compiler lowers this graph into per-device eBPF programs, verifier obligations, consistency-classed BPF maps, and artifacts for dputime. At the fabric edge, fabric_ext treats a near-Type-2 small core as a hardware-JIT and state manager: it specializes verified movement descriptors into local copy, placement, ordering, and transformation commands, while the surrounding Von Neumann island of memory, DMA, and compute engines performs the dataflow. Because this dataflow is data-driven, fabric_ext also places observation beside the island, where queues, DMA completions, memory placement, and ownership transitions are visible as they happen. The canonical stress case is LLM prefill: attention streams KV blocks and reductions while FFN streams activations, weights, and compressible intermediates, forcing one request to cross GPU tensor execution, DPU/NIC event execution, and CXL or switch-local dataflow islands. fabric_ext’s central contribution is not a shared map. It is a cross-device policy contract. Telemetry maps may be eventually consistent; scheduling hints use bounded-staleness epochs; ownership and placement maps require atomic epoch
handoff; command maps require exactly-once completion. A cross-device effect verifier checks both SIMT GPU execution and event-driven fabric execution: bounded helpers and RDMA/CXL operations, no waits on completions triggered by the same handler, no GPU–fabric dependency cycles, tenant/QP/region capabilities, retry TTLs, and atomic policy updates across devices. We prototype fabric_ext in Damer with dputime as the eBPF frontend. On 1,000 generated movement microbenchmarks, the compiler produces verified plans and BPF C in microseconds. Across 14 LLM fabric workloads—including Qwen 27B decode, long-context CXL KV cache, pipeline and tensor parallelism, LoRA adapter updates, Qwen scaleout from 2 to 32 GPUs, MoE all-to-all, prefill/decode colocation, remote KV cache, and training/checkpoint traffic— fabric_ext compiles 86 events into BPF objects and validates remote Arm compilation on a BlueField DPU. The expanded suite reports a modeled 2.34× data-movement E2E speedup over staged host/NIC control. A copy-path optimizer that selects DPU inline, GPU pre-transform, and direct GPU-peer/CXL paths improves the modeled speedup to 3.47×. These results are not token-latency claims; they show that semantic, verified fabric policies can expose where end-to-end data movement improves when the fabric, not the host CPU alone, becomes the cluster driver.
1
Introduction
GPU clusters are becoming programmable fabrics rather than host-attached islands. An LLM serving request may start as prefill on one GPU, decode on another, fetch remote KV blocks from CXL memory, route tokens through MoE experts over RDMA, and periodically share the same fabric with background training and checkpoint traffic. Each layer sees a useful part of the truth. GPU kernels and runtimes see execution phase, streams, warps, page faults, KV block identifiers, expert IDs, and whether a token is on the latency-critical path. NICs, DPUs, and switches see queue pairs, WQEs, CQEs, retransmissions, congestion, flow timeouts, and placement pressure. CXL switches and memory devices see fabric routes, shared memory windows, persistence domains, and ordering constraints. No single layer can currently express a policy that combines these facts. This split causes a control-plane mismatch. A GPU policy can decide that decode KV fetches are urgent, but it cannot directly pace a congested RDMA queue or redirect a transfer
Yiwei Yang and Andi Quinn
away from an overloaded expert replica. A DPU or NIC policy can prioritize a QP or react to congestion, but it cannot tell whether the bytes are decode, prefill, MoE tokens, gradients, checkpoint records, or CXL-memory migration. A CXL fabric manager can configure memory windows and routing, but it lacks application semantics such as reuse distance, alias ownership, or the phase in which a page fault occurs. The result is a set of local controllers that each make reasonable decisions with incomplete information. fabric_ext takes the opposite view: the policy boundary should be the fabric, not the device. It extends the eBPF programming model from one GPU or one network endpoint to a cross-device policy system spanning GPU kernels, GPU drivers, DPU/NIC execution, CXL switches, and CXLattached memory. The goal is not to replace RDMA, GPUDirect, CXL fabric management, or existing GPU runtimes. Those mechanisms already provide high-performance data paths. fabric_ext contributes the missing safe, dynamic, semantic policy layer that decides what movement should happen, where transformations should execute, and when fabric operations should be delayed, redirected, replicated, or persisted. For CXL and switch-attached execution, the target is a near-Type-2 small core rather than a full host CPU. That core acts as a hardware-JIT and state manager: it consumes verified movement descriptors, maintains epochs and ownership, emits local commands, and lets the adjacent Von Neumann island of memory, DMA, and simple compute engines carry out the dataflow. Since that dataflow is driven by queue entries, memory completions, and buffer readiness rather than host polling, fabric_ext places observation beside the island as well. The same fabric-side runtime that programs movement can observe queue pressure, completion timing, ownership transitions, and local placement state at the point where they affect data movement. The strongest use case is LLM prefill rather than ordinary QoS. In attention, one request streams KV blocks, page-table state, reductions, and remote-cache hits. In FFN, the same request streams activations, weights, quantized or compressed intermediates, and tensor-parallel exchanges. These movements are data-driven and phase-dependent, so the right policy point changes within a single request. GPU hooks see layer, phase, KV block, and tensor-shard semantics. DPU/NIC hooks see RDMA pressure, queue completion, congestion, and remote GPU readiness. A near-Type-2 fabric core sees CXL placement, local queues, DMA completion, and ownership transitions. fabric_ext lets one verified policy span all three architectures instead of approximating the request as a QP priority. Consider semantic RDMA scheduling for MoE inference. Today the fabric sees packets or QPs. The GPU sees expert IDs, token batches, and which experts are on the decode critical path. With fabric_ext, a GPU hook publishes a fabric hint keyed by stream, rank, and expert. A fabric hook on a DPU, NIC, or CXL switch-attached compute engine uses that
One logical eBPF policy, split across fabric execution points GPU application LLM runtime GPU hooks phase / warp / page KV / expert / stream
Damer compiler movement graph BPF C + object runtime manifest
Consistency-classed BPF maps telemetry / hint ownership / command
DPU / NIC hooks WQE / CQE / congestion retry / timeout CXL switch or near-memory hooks place / persist / reduce ownership
GPU peer hooks ready / receive / copy collective boundary Cross-device effect verifier capabilities / TTL epoch update Bulk data stays on RDMA / GPUDirect / CXL paths; policy decisions move to the nearest safe hook.
Figure 1. fabric_ext compiles one logical eBPF policy into GPU, driver, DPU/NIC, and CXL switch or near-memory handlers. Bulk data continues to use RDMA, GPUDirect, and CXL data paths; fabric_ext provides the verified semantic policy layer above them.
hint when WQEs are submitted or completions arrive: critical expert tokens are scheduled first, traffic to congested experts is paced, idle replicas receive redirected tokens, and transfers to a destination GPU that is not ready can be deferred before they build up in CQ or GPU buffers. The same policy can run on the best available execution point: a BlueField DPA handler for RDMA queue decisions, a host dputime handler for driver-level fallback, or a switch/near-memory handler for CXL placement and persistence decisions. The abstraction fabric_ext exposes is a semantic datamovement graph. Nodes include host memory, CXL memory devices, GPU memory, accelerators, and switch compute engines. Edges carry bytes, stride, reuse distance, read/write ratio, source and destination, ordering requirement, alias set, ownership, and a transformation. fabric_ext supports fused actions that recur in GPU-fabric workloads: Move, Quantize, Compress, Checksum, Filter, Reduce, Scatter/Gather, Replicate, and Persist, always represented as movement plus an optional transformation. These are not packet classifiers. They are movement intents that the compiler can place on GPUs, DPUs, CXL memory devices, or switch-local engines. The hard part is safety. Extending eBPF across the fabric cannot simply share maps between devices. Prior GPU eBPF systems showed that CPU–GPU hierarchical maps are useful for statistics and hints, but they mostly tolerate eventual consistency. Fabric policies also manipulate queueing, placement, ownership, and retries. A stale statistic is acceptable; a stale ownership transfer or unbounded retry loop is not. fabric_ext therefore introduces consistency-classed BPF maps and a cross-device effect verifier. Telemetry maps are eventually consistent. Hint maps carry bounded-staleness epochs. Ownership and placement maps use epoch-atomic handoff.
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
Command maps expose exactly-once completion. The verifier checks both GPU SIMT constraints and fabric-handler constraints, including bounded helper calls, bounded RDMA/CXL operations per event, no dependency cycles across GPU and fabric programs, tenant/QP/region capabilities, and retry TTLs. This paper makes five contributions: • We identify the semantic split in GPU fabrics: GPUs know execution meaning but not fabric state; DPUs, NICs, and CXL switches know fabric state but not execution meaning. • We design fabric_ext, a cross-device eBPF policy system that lowers semantic movement graphs into GPU, driver, DPU/NIC, and CXL-switch policy hooks. • We introduce a near-Type-2 fabric execution model in which a small core performs hardware-JIT specialization, policy state management, and adjacent observation while the surrounding memory/compute island executes data-driven dataflow. • We introduce consistency-classed BPF maps and a cross-device effect verifier for policies whose control flow crosses SIMT GPU execution and event-driven fabric execution. • We implement a prototype in Damer using dputime as the eBPF frontend and evaluate it on 1,000 generated movement microbenchmarks plus 14 LLM fabric workloads, showing where semantic policies improve modeled end-to-end data movement.
2
Background and Motivation
2.1
GPU Fabrics Are Already Cross-Layer
Modern LLM systems are not only GPU compute pipelines. They are data-movement systems whose critical paths cross GPU memory, host memory, CXL memory pools, RDMA queues, and collective communication libraries. PagedAttention serving engines maintain logical-to-physical KV block tables [7]. NCCL and RDMA transports move activations, gradients, tokens, and checkpoints across GPUs [5, 6]. CXL 3.x adds fabric capabilities, switching, peer-to-peer access, memory sharing, and fabric manager mechanisms that make memory placement a fabric concern rather than a host-local concern [1, 2]. DPU and GPU-centric network APIs can remove the CPU from parts of the packet path, for example by enabling GPU-triggered network operations and direct GPU memory transfers [10, 11]. These mechanisms solve data-path problems. They do not solve the policy problem. The data path can move bytes efficiently once a transfer is chosen. The difficult question is whether a transfer should be prioritized, delayed, transformed, redirected, replicated, persisted, or placed near the requester. That decision needs execution semantics from the GPU side and fabric state from the network and CXL side.
2.2
What Each Layer Can and Cannot See
GPU view. GPU kernels and runtimes see the semantic structure of computation. They know whether a stream is running prefill, decode, all-reduce, checkpoint, or MoE routing. They can expose warp-level stalls, kernel phase, page faults, KV block reuse, sequence lifetime, expert ID, token batch, and whether a transfer is on the request critical path. However, a GPU hook does not directly observe RDMA retransmissions, remote QP depth, CXL switch congestion, remote GPU readiness, or memory-pool pressure. Fabric view. DPUs, NICs, CXL switches, and memory devices see fabric state. They observe WQE and CQE timing, queue depth, retransmissions, timeouts, routing, memorywindow ownership, persistence domains, and placement conflicts. Some devices also provide programmable execution near the data path: BlueField DPA exposes event-triggered threads, asynchronous operations, RDMA primitives, memory services, and ordering plus notification control [10]; CXL switches and switch-attached near-memory engines provide a natural point for placement and copy policies when the traffic is CXL memory movement. But the fabric sees a stream of operations, not their high-level meaning. A QP does not reveal whether its bytes are decode KV blocks, MoE expert tokens, prefill activations, gradients, or checkpoint records. Host view. The host CPU can coordinate runtimes, drivers, and fabric managers, but it is the wrong critical-path policy point for many decisions. Host-mediated control adds latency, loses event granularity, and often observes state after queue buildup has already happened. The host remains useful for admission control, policy loading, audit, and fallback. fabric_ext moves per-event decisions to the fabric while keeping the host as the policy owner. 2.3
Why eBPF Is the Right Frontend
eBPF provides a mature model for safe, dynamically loaded OS extensions. It gives programmers maps, helpers, bounded execution, verifier checks, and attach points. Prior work extended this model toward GPUs, showing that eBPF can expose GPU-side observability and policy hooks [14]. fabric_ext keeps eBPF as the frontend because fabric policies need the same properties: dynamic loading, per-tenant capability boundaries, bounded execution, and safe helper APIs. The difference is scope. A GPU-local eBPF policy can tune one accelerator. fabric_ext treats the fabric as the programmable object. One logical policy is split into devicespecific handlers: • GPU hooks publish semantic hints at kernel, stream, warp, page-fault, and phase boundaries. • Driver hooks bind streams, memory regions, QPs, CXL windows, and tenants to policy capabilities. • DPU/NIC hooks run at WQE submit, CQE complete, congestion update, timeout, and peer-failure events.
Yiwei Yang and Andi Quinn
• CXL switch or near-memory hooks run at placement, copy, persistence, reduction, and ownership-transfer events. 2.4
From Shared Maps to Consistency Classes
The tempting design is a shared cross-device BPF map. That is insufficient. Different policy state has different correctness requirements. Approximate telemetry such as bytes moved or queue depth can be eventually consistent. A decode hint can tolerate bounded staleness if it carries an epoch. Ownership of a CXL memory window cannot be eventually consistent; a stale owner can violate isolation or ordering. A retry command cannot be unbounded; it must have a TTL and exactly-once completion semantics. fabric_ext therefore treats map consistency as part of the policy type system. The programmer chooses a consistency class, and the compiler/verifier checks which helpers may read or write that map from each device. The runtime implements the chosen class with GPU-local maps, host mirrors, BlueField/DPU memory, CXL memory, or switch-local state depending on the target. 2.5
These examples are all data-movement policies. They do not require inventing a new packet path or a new CXL protocol. They require a safe way for execution semantics and fabric state to meet.
Motivating Workloads
LLM prefill attention and FFN.. LLM prefill is the canonical fabric_ext workload because the critical dataflow crosses all three execution points inside one request. The attention path moves KV blocks, gathers remote cache lines, performs reductions, and depends on sequence-local readiness. The FFN path moves activations and weights, often with shrinkable intermediates that can be quantized, compressed, filtered, or reduced before crossing the fabric. GPU hooks expose the semantic phase and tensor objects; DPU/NIC hooks observe RDMA queues, congestion, and remote readiness; near-Type-2 CXL or switch-local hooks observe placement, DMA completion, and ownership beside the dataflow island. No packet-only or page-only policy can see this full state.
3
Design
3.1
Overview
fabric_ext compiles one logical fabric policy into a set of device-specific eBPF handlers and a shared movement plan. The programmer writes policy over semantic events: GPU phase changes, page faults, stream boundaries, RDMA workqueue submission, completion events, congestion updates, CXL placement events, and ownership transfers. The compiler lowers this policy into: 1. a semantic movement graph; 2. per-device eBPF programs for GPU, driver, DPU/NIC, and CXL switch or near-memory hooks; 3. consistency-classed BPF maps and helper calls; 4. verifier obligations for bounded execution and crossdevice effects; and 5. runtime manifests for dputime-based loading and observation. The design principle is simple: data-path mechanisms remain device-specific, but policy meaning is shared. A DPU handler may post or defer an RDMA operation. A CXL switch handler may choose placement or trigger a copy to a memory pool. A GPU handler may publish decode or expert metadata. All of them execute one policy contract with explicit consistency and capability rules. 3.2 Semantic Movement Graph The compiler’s internal representation is a movement graph. Nodes describe where data resides or where transformations can execute: Node
Meaning
MoE all-to-all. GPU hooks identify expert ID, token batch, source/destination rank, critical-path experts, and destination readiness. Fabric hooks prioritize critical expert tokens, redirect tokens to idle replicas, pace congested experts, and avoid sending to a GPU whose receive buffer is not ready.
host memory CXL memory device GPU memory accelerator switch compute engine
CPU DRAM or pinned staging memory CXL Type-3 memory, pooled memory, or persistent memory HBM or GPU-visible allocation GPU, DPU, or other compute-capable endpoint near-Type-2 small core plus local DMA/compute engines
Prefill–decode colocation. When training, prefill, decode, and checkpoint traffic share the same GPU fabric, a QP-only policy is too coarse. fabric_ext classifies decode KV fetches as deadline-critical, prefill activations as medium priority, training all-reduce as paceable, and checkpoint traffic as deferrable under congestion.
Field
Purpose
bytes, stride reuse distance read/write ratio source/destination ordering requirement alias set ownership transformation
data volume and access shape whether placement or prefetch is useful copy, reduction, or writeback bias endpoints and routing candidates none, program order, acquire-release, or total memory-equivalence and conflict class borrowed, source-owned, destination-owned, or shared operation fused with movement
Remote KV cache and CXL memory. GPU page-fault and KV-block hooks reveal reuse distance and sequence lifetime. CXL switch or near-memory hooks can prefetch blocks, choose replicas, redirect failed memory servers, and place hot blocks near the GPU that will reuse them.
Edges describe a movement intent:
fabric_ext currently recognizes nine movement actions: Move Move+Checksum Move+Scatter/Gather
Move+Quantize Move+Filter Move+Replicate
Move+Compress Move+Reduce Move+Persist
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
These actions are deliberately small. They are expressive enough to cover common GPU-fabric transfers, but constrained enough for effect verification. For example, a remote KV-cache policy may compile a page fault into prefetch-like behavior expressed as movement with replicate or persist semantics, depending on whether the target is a cache replica or durable CXL memory. An MoE token routing policy compiles into scatter/gather movement plus optional filtering or replication. Checkpoint and memory-tiering policies compile into movement with compression and persistence. 3.3
Near-Type-2 Hardware JIT
fabric_ext separates fabric control from fabric dataflow. The control point is a near-Type-2 small core: a programmable core near a CXL Type-2 endpoint, a switch-attached engine, or an equivalent accelerator-side control processor. This core is not intended to copy every byte with scalar code. Instead, it acts as a hardware-JIT for movement descriptors. Given a verifier-approved edge, it specializes the descriptor into device-local commands: DMA copy, CXL placement, persistence, reduction, scatter/gather, pacing, or replica selection. It also owns the local slice of policy state: map epochs, ownership handoff, command sequence numbers, TTLs, and capability checks. The dataflow then runs in the surrounding Von Neumann island. By this we mean the local memory, load/store engine, DMA engines, queue machinery, and small fixed-function or programmable transform units around the Type-2 endpoint. The island still follows a conventional memory-command execution model, but fabric_ext changes who programs it: the host no longer stages every policy decision, and the network no longer sees only packets or QPs. The fabric small core turns semantic eBPF decisions into dataflow actions close to where CXL memory, GPU buffers, and switch-local resources meet. Observation must be co-located with this execution point. The dataflow is data-driven: work appears as queue entries, DMA descriptors, memory completions, ready bits, and ownership handoffs. A host-side poller sees these signals late and after aggregation. fabric_ext therefore treats observation as a sidecar to the near-Type-2 hardware-JIT. The small core observes local queue depth, completion timing, placement state, retry/redirect outcomes, and map-epoch transitions while it programs the local dataflow engines. Those observations update telemetry and hint maps without moving the policy loop back to the host. 3.4
LLM Prefill Across Three Architectures
Prefill attention and FFN are the clearest example of why fabric_ext is a fabric policy system rather than a DPU-only offload. Within one prefill request, attention movement is dominated by KV-block lookup, gather, placement, reduction, and readiness; FFN movement is dominated by activation and weight movement plus shrinkable intermediate tensors.
fabric_ext maps these data-driven flows across three architecture points: Execution point
What it observes
What it controls
GPU SIMT/ten- layer phase, tensor sor execution shard, KV block, page fault, stream deadline
HBM pre-transform, phase hints, block reuse, readiness publication DPU/NIC event WQE/CQE timing, QP RDMA priority, pacing, execution depth, retransmit, con- redirect, retry, replica gestion, remote GPU selection readiness near-Type-2 fab- CXL placement, DMA hardware-JIT comric island completion, queue mands for copy, place, state, ownership reduce, persist, repliepoch, persistence cate, and observe state
The same logical policy can therefore treat attention KV fetches as deadline-critical, pace FFN activation spill when queues build up, pre-transform shrinkable FFN intermediates on the GPU, and move cold KV or persistent state through a CXL-local dataflow island. The observability point follows the dataflow: the GPU observes tensor semantics, the DPU observes network events, and the near-Type-2 core observes local queue and memory-state transitions. 3.5
Policy Hooks
fabric_ext exposes hooks at the semantic points where information appears. GPU hooks publish execution meaning; fabric hooks act on communication and memory-placement events. The following example marks decode traffic at a GPU phase boundary: SEC("gpu/kernel_phase") int mark_phase(struct gpu_kernel_ctx *ctx) { struct fabric_hint hint = { .tenant = ctx->tenant, .stream = ctx->stream, .phase = GPU_PHASE_DECODE, .deadline = ctx->deadline, .priority = LATENCY_CRITICAL, }; bpf_fabric_publish(ctx->stream, &hint); return 0; }
A fabric handler consumes that hint when a movement operation reaches the fabric: SEC("fabric/rdma_submit") int schedule_wqe(struct fabric_wqe_ctx *ctx, struct fabric_hint *hint) { if (hint && hint->phase == GPU_PHASE_DECODE) return BPF_FABRIC_HIGH_PRIORITY; return BPF_FABRIC_ALLOW; }
Yiwei Yang and Andi Quinn
Map class
Consistency contract Typical state
Telemetry
eventual
Hint Ownership Command
bytes, queues, retries
decode, expert load, readiness bounded-stale epoch
epoch-atomic exactly-once
CXL window, GPU buffer, QP cap redirect, retry, persist, reduce
The verifier checks that helpers and target devices respect the declared map class.
Figure 2. Consistency-classed BPF maps. Each map class exposes a different synchronization contract, allowing telemetry to remain cheap while ownership, placement, and commands receive stronger guarantees.
The same policy may attach to multiple fabric hook kinds: Hook
Typical decision
rdma_wqe_submit rdma_cqe_complete rx_message/tx_message congestion_update remote_gpu_ready flow_timeout peer_failure cxl_place cxl_copy_complete
priority, defer, redirect, pace update readiness, release ownership packet or message classification pacing and replica choice avoid CQ and buffer buildup retry, failover, or TTL expiry route around failed GPU or memory server choose CXL memory device or switch path publish placement epoch
3.6
Consistency-Classed BPF Maps
The most important design choice is that fabric_ext does not present one undifferentiated shared map. Each map declares a consistency class: Class
Contract
Telemetry
Eventually consistent counters and histograms. Used for bytes, queue depth, retry counts, or phase statistics. Bounded-staleness values with an epoch and expiration. Used for priority, decode/prefill/MoE phase, expert load, and remote readiness. Epoch-atomic ownership or placement state. Used for CXL memory windows, GPU buffer ownership, QP capability binding, and alias sets. Exactly-once command completion with sequence numbers. Used for redirect, retry, persist, copy, reduce, and placement commits.
Hint
Ownership
Command
The runtime implements these classes with hierarchical physical storage. A GPU may keep a fast local map for phase hints. The host mirror records policy ownership and audit state. A DPU may keep hot QP and congestion entries in DPA memory. A CXL switch or near-memory engine may keep
Movement placement preference by transform and target Move
fallback
ok
ok
fallback
Quantize
best
fallback
ok
fallback
Compress
best
fallback
ok
fallback
Checksum
fallback
best
ok
fallback
Filter
best
fallback
ok
fallback
Reduce
best
ok
best
fallback
Scatter/ Gather
ok
best
best
fallback
Replicate
fallback
best
best
fallback
Persist
fallback
ok
best
fallback
GPU pre-transform
DPU/NIC inline
CXL switch near-memory
Host fallback
Figure 3. Placement preference for common movement transformations. GPU pre-transform is preferred when the transform shrinks HBM-resident data; DPU/NIC inline execution is preferred for network-path scheduling and copy control; CXL switch or near-memory execution is preferred for placement, persistence, and memory-local operations.
placement and ownership epochs close to the memory pool. The compiler generates synchronization code only where the class requires it. Telemetry never pays ownership-map cost; ownership never degrades into best-effort statistics. 3.7
Placement and Lowering
fabric_ext chooses where a movement-side transformation runs by considering source, destination, transform type, byte volume, reuse distance, ordering, and device capability. The placement problem is not a global optimizer in the first prototype. It is a constrained decision checked by the verifier: • GPU pre-transform is preferred when source data is in HBM and the transform shrinks data, such as quantize, compress, filter, or reduce. • DPU/NIC inline execution is preferred for RDMA queue scheduling, replica selection, pacing, checksum, and copy operations already on the network path. • CXL switch or near-memory execution is preferred for placement, persist, copy, reduce, and scatter/gather operations whose source or destination is CXL memory. • Host execution is a fallback for unsupported devices, policy loading, and audit. For example, the following switchlet expresses a CXL-local quantizing move: damer.switchlet @kv_pack( %src : memref<?xf16, "cxl">, %dst : memref<?xi8, "cxl">) { %v = damer.read %src %q = damer.quantize %v damer.write %q, %dst }
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
If the CXL memory device or switch-attached engine supports the transform, fabric_ext keeps the operation near CXL memory. If not, it may pre-transform on the GPU or fall back to host/DPU execution, but the verifier preserves ordering, alias, and ownership rules. 3.8
Cross-Device Effect Verifier
The verifier extends conventional eBPF bounded-execution checks with fabric effects. It validates the combined policy, not only each handler in isolation. The checked properties are: 1. DPU, switch, and GPU handlers have bounded instruction count, helper count, and stack usage. 2. A handler cannot wait for a completion event that can only be produced by an operation the same handler has not yet issued. 3. GPU and fabric handlers cannot form a dependency cycle, such as GPU waiting for DPU completion, DPU waiting for a GPU-ready flag, and the GPU-ready flag being updated only after that completion. 4. Every QP, tenant, GPU memory region, and CXL window access is protected by a capability. 5. Each input event emits a bounded number of RDMA, DMA, or CXL memory operations. 6. Redirect and retry paths carry a TTL to prevent packet, WQE, or placement livelock. 7. GPU and fabric programs update through an epochatomic switch: either all devices execute the old policy or all execute the new policy for a given epoch. This is the core verification problem fabric_ext targets: a single policy can have control flow that crosses SIMT GPU execution and event-driven fabric execution. Without this verifier, semantic fabric policies become dangerous ad hoc offloads. With it, fabric_ext can allow dynamic policies while preserving tenant isolation and progress.
4
Implementation
fabric_ext is implemented in Damer as an eBPF middleware compiler. The prototype is intentionally independent of CIRCT: eBPF is the frontend and runtime interface, while Damer owns the movement graph, placement decisions, helper ABI, verifier summary, and generated artifacts. 4.1
Compiler Pipeline
The compiler accepts two input forms. First, a JSON event format describes movement requests emitted by GPU runtimes, drivers, or fabric monitors. Second, a compact switchlet syntax describes high-level movement intent, as in the CXL-local kv_pack example in Section 3. Both are normalized into the same internal movement graph. For each movement edge, the compiler emits: • a semantic plan JSON containing nodes, edges, placement, properties, and verifier results;
• dputime-facing eBPF C with a policy section name and helper calls; • a dputime runtime manifest describing maps, helpers, and attach sections; • a BPF object compiled with clang -target bpf; and • an optional offload bundle for host/GPU/DPU observation. The generated BPF C does not move user data itself. It emits decisions and movement descriptors through two compact Damer helpers. This keeps eBPF programs small and verifiable while leaving bulk transfer to RDMA, GPUDirect, CXL, DMA engines, or switch-local copy mechanisms. 4.2
dputime Frontend
fabric_ext uses dputime as the userspace eBPF execution substrate and the dputime fork as the experimental GPU/DPU observation path. On the GPU host, dputime can be built with CUDA attach support so CUDA module loading and runtime events become observable policy hooks. On BlueField/Arm, dputime provides the native eBPF runtime used to observe DPU-side hooks and validate that generated BPF objects compile on the DPU operating system. The runtime report records two observer roles: Observer
Role
GPU host BlueField DPU CXL switch target
observe CUDA/GPU-side policy hooks and memory movement intent observe RDMA/NIC placement, queue events, and offload bundle state consume movement plans for switch or near-memory execution
The third target is currently a backend interface rather than a vendor-specific switch firmware implementation. The compiler represents it as switch_compute_engine and emits placement, consistency, and verifier metadata that a CXL switch-attached executor or near-memory controller can consume. This is deliberate: fabric_ext should not bake BlueField DPA into the abstraction boundary. DPUs are one execution point; CXL fabric points are another. For the CXL target, the emitted artifact is a hardware-JIT input rather than a monolithic kernel: it contains a verified movement descriptor, map-class requirements, command-map completion rules, and the local state that a near-Type-2 small core needs to program the adjacent dataflow engines. It also contains observer metadata for data-driven events next to those engines, including queue updates, DMA completion, memoryplacement completion, ownership handoff, and retry or redirect expiry. 4.3
Runtime Artifacts
An offload bundle contains all state needed to load and audit a policy: • the source workload event or switchlet; • generated movement plans; • BPF C and BPF objects; • dputime manifests;
Yiwei Yang and Andi Quinn
• verifier summaries; • helper IDs and map declarations; • target roles and observer build information; and • policy epoch metadata. This bundle is the unit of cross-device update. When a policy is installed, each target first loads the new program into an inactive epoch. Only after all required targets acknowledge verification and load success does the host publish the new epoch. Fabric events then carry an epoch tag, so a GPU hint, a DPU queue decision, and a CXL placement decision agree on which policy version they are executing. 4.4
Helper ABI
fabric_ext exposes helper APIs at the movement level rather than at raw packet or page granularity. Table entries below use short names; generated code uses the corresponding bpf_fabric_* or bpf_cxl_* symbol. Important helpers include: Helper
Purpose
publish
publish a semantic hint with epoch and TTL read a hint or ownership record under its map class query destination readiness before enqueueing movement emit a movement decision descriptor choose an alternate QP, GPU, replica, or CXL target request CXL memory placement or migration commit completion for exactly-once command maps
lookup ready emit redirect cxl_place commit
The verifier checks helper use against map class and target capability. For example, a GPU hook may publish a hint but may not directly commit ownership of a CXL memory window unless it holds the corresponding capability. A CXL placement hook may update an ownership map only through an epoch-checked helper. A DPU handler may emit a bounded number of RDMA operations and must attach TTL to redirect or retry decisions. 4.5
CXL Switch and Near-Memory Backend
fabric_ext treats CXL as both a memory target and a policy target. When an edge’s source or destination is a CXLmemory node, the compiler considers CXL-local placement. When a transform is bandwidth-reducing or persistencerelated, it may place the transform at a switch compute engine if the target capability exists. The intended backend is a near-Type-2 control core that performs hardware-JIT specialization and state management, then dispatches the operation to the local Von Neumann island: CXL memory, queues, DMA, and simple transform engines. Observation is placed next to the same island. The backend records state
changes where the dataflow is produced, not only after the host observes a completed transfer. The first prototype models this backend with the same movement-plan interface used for DPU and GPU targets. The plan records: • the selected CXL memory device or pool; • whether the operation requires program order, acquirerelease, or total ordering; • alias and ownership epochs; • expected byte volume after transformation; • data-driven observation points beside queue, DMA, and placement engines; • fallback targets if switch-local execution is unavailable; and • command-map completion conditions. This lets the evaluation ask a concrete question before vendor-specific switch support is available: where would a near-Type-2 hardware-JIT and its local dataflow island reduce end-to-end movement, and which events are still dominated by GPU or DPU paths?
4.6
Failure and Fallback
fabric_ext assumes policy handlers are fail-stop and bounded by the verifier. If a target cannot load a policy, the epoch switch is aborted and the previous policy remains active. If a target disappears at runtime, capability checks force dependent events onto fallback paths. For example, if the CXL switch-local executor is unavailable, Move+Persist can fall back to DPU inline copy or host-mediated placement, but ownership epochs still prevent two devices from claiming the same memory window. If the DPU runtime is unavailable, GPU and host hooks can continue publishing hints and using conservative default fabric decisions.
5
Evaluation
We evaluate fabric_ext with three goals. First, can the compiler generate verified eBPF artifacts across the movement primitive space? Second, can the same workload be compiled into host/GPU/DPU-facing artifacts rather than remaining a paper-only abstraction? Third, where does semantic fabric policy improve modeled end-to-end data movement, especially when CXL memory and switch-local placement are considered?
5.1
Experimental Setup
Prototype. The prototype is implemented in Damer. It compiles movement events and switchlets into semantic plans, BPF C, BPF objects, and runtime manifests. The dputime fork of dputime is used as the GPU/DPU observation runtime. The GPU-host path is built with CUDA attach support when CUDA headers and a visible NVIDIA GPU are available.
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
The BlueField path is built natively on Arm Ubuntu and validates BPF compilation on the DPU OS. The CXL switch backend is evaluated through the switch_compute_engine placement model and verifier metadata because commodity programmable CXL-switch firmware is not part of the prototype.
Workloads. We use 14 fabric workloads: • Qwen-style serving: logits movement, KV packing, prefill activation spill, decode KV fetch, tensor shard exchange, KV replication, and checkpoint persistence. • Qwen DPU decode: decode-critical KV fetch, token scatter, retry/redirect, replica refresh, and append-log persist decisions visible to the DPU. • Qwen long-context CXL: CXL-backed KV fetch, prefix-cache replication, cold-KV eviction, sequence migration, and attention reduction. • Qwen parallelism: pipeline activation movement, CXL activation spill, tensor all-reduce, weight streaming, microbatch scatter, and checkpoint persist. • Qwen LoRA adapter: adapter delta reduction, quantized adapter push, optimizer-state compression, decode KV fetch, adapter replica refresh, and checkpoint persist. • Qwen scale-out: 2/4/8/16/32 logical GPU variants with increasing collective, scatter/gather, replica, redirect, and checkpoint fanout. • MoE all-to-all: critical expert tokens, background expert traffic, expert output reduction, replica refresh, and checkpoint persist. • Prefill/decode colocation: decode KV fetch, prefill activation spill, background training all-reduce, checkpoint throttle, and shared tensor movement. • Remote KV cache: KV prefetch, compact/persist, failed memory-server redirect, sequence migration, and placement updates. • Training/checkpoint: gradient reduction, parameter broadcast, optimizer shard exchange, activation checkpoint compression, and step checkpoint persist.
Metrics. For artifact-level evaluation we report generated cases, verifier success, BPF object count, and compiler latency. For E2E evaluation we use the explicit data-movement model recorded in the benchmark report. The baseline is staged host/NIC-visible movement with separate transform and control steps. The fabric_ext path uses fused placement and dputime offload control. A second optimizer evaluates copy-path choices on top of fabric_ext: DPU inline, GPU pre-transform, and GPU peer/CXL direct paths. These are modeled data-movement results, not application tokenlatency measurements.
Table 1. Generated movement microbenchmark coverage. Metric
Value
Cases Movement kinds Node classes Placement targets Max RDMA/CXL ops per event Median compile time 95th percentile compile time Median BPF C emit time
1,000 9 5 5 8 8.5 𝜇s 9.7 𝜇s 4.1 𝜇s
Table 2. Verifier regression coverage. Expected rejections count as passing tests when the verifier rejects the input. Suite
Expected result
Pass rate
Coverage
Generated positives LLM workload events Fuzzer mix Targeted negatives
Accept Accept Accept/reject Reject
1,000/1,000 86/86 512/512 3/3
9 actions, 5 nodes, 5 placements; max fanout 8 14 workloads, 10 movement kinds, 3 placements; max fanout 64 404 accepted, 108 rejected, 0 unexpected; 204 kinds, 40 multi-edge transform bound, RDMA/CXL fanout bound, retry TTL
5.2
Microbenchmark Coverage
The generated microbenchmark suite spans all nine movement kinds and five node classes. It validates that the compiler can parse events, build movement plans, run verifier checks, emit BPF C, and produce bounded artifacts. Table 1 shows that the compiler path is lightweight enough for policy iteration. The point is not that policy compilation occurs on every packet or WQE. Rather, fabric operators can generate, verify, and load specialized movement policies at runtime without treating the compiler as an offline-only tool. 5.3
Verifier Coverage
We evaluate the verifier as a compiler component: a test passes when an input with a legal effect contract is accepted, or when an input with an illegal effect contract is rejected with a diagnostic. Table 2 summarizes positive, fuzzed, and targeted negative coverage. The fuzzer mixes JSON events, compact switchlet syntax, and multi-edge pipelines. The targeted negative suite exercises the implemented hard effect checks: transform-count bounds, RDMA/CXL fanout bounds, and retry/redirect TTL requirements. The LLM workloads produce no verifier diagnostics. Each accepted plan carries a bounded-effect summary; in this suite the generated helper bound is two helper calls per event and the largest event fanout is 64, below the configured bound of 128. The targeted negative cases are intentionally small, because their purpose is not random discovery but regression protection for specific safety obligations.
Workload
Events
fabric_ext us
Speedup
Qwen-style serving Qwen DPU decode Qwen long-context CXL Qwen pipeline/tensor parallel Qwen LoRA adapter Qwen scale-out 2 GPU Qwen scale-out 4 GPU Qwen scale-out 8 GPU Qwen scale-out 16 GPU Qwen scale-out 32 GPU MoE all-to-all Prefill/decode colocation Remote KV cache/CXL memory Training/checkpoint
9 6 6 6 6 6 6 6 6 6 5 6 6 6
10,146.8 3,780.3 21,884.2 52,957.2 9,586.4 12,965.2 27,236.4 59,973.2 142,224.1 373,834.6 2,394.0 18,616.8 3,621.4 71,159.5
2.77× 2.79× 4.10× 3.06× 3.65× 2.76× 2.66× 2.50× 2.26× 1.96× 3.90× 2.87× 3.84× 2.56×
Aggregate
86
810,380.1
2.34×
Latency under scale-out
102
101 2
4
8 16 Logical GPUs
End-to-End Workload Compilation
Across the 14 workloads, fabric_ext compiles 86 semantic movement events into 86 BPF objects. The same artifact bundle is shipped to the BlueField host for remote Arm compilation; all 86 generated BPF C programs recompile successfully on the DPU OS. This validates the cross-device packaging path: the generated policy is not only a local plan but a set of BPF artifacts that can be observed by host and DPU runtimes. Table 3 reports the first-level fabric_ext speedup. The largest relative gains appear in long-context CXL, MoE, remote KV cache, and LoRA adapter workloads because semantic routing, placement, and transformation reduce unnecessary movement and queueing. The largest absolute latency remains in training/checkpoint, pipeline/tensor parallel, and scale-out traffic because those events carry large byte volumes. This distinction is important: semantic policies help critical small transfers and large background transfers for different reasons. 5.5
Qwen Scale-Out
The scale-out workloads increase tensor collective size, scatter/gather fragments, KV replica fanout, redirect fanout, and checkpoint bytes from 2 to 32 logical GPUs. Figure 4 shows that fabric_ext preserves a 1.96–2.76× modeled speedup as fanout grows. The decline at 16 and 32 GPUs is expected: replica refresh and checkpoint persistence become bytevolume dominated, so semantic policy cannot hide all fabric work. Copy-path optimization still adds 1.44–1.54×, primarily by selecting DPU inline replica and checkpoint persistence plus direct GPU-peer scatter/gather paths. 5.6
32
Speedup under scale-out 4.26x
4.07x
3 2.76x
2.66x
4
3.77x 2.50x
2
fabric_ext + copy path 3.34x 2.82x 2.26x
1.96x
1 0
2
4
8 16 Logical GPUs
32
Figure 4. Qwen 27B scale-out experiment. Labels show firstlevel fabric_ext speedup and speedup after copy-path optimization. Table 4. Copy-path optimization on top of fabric_ext. Workload
5.4
5
Baseline fabric_ext + copy path
Speedup over baseline
Table 3. Modeled data-movement E2E speedup over staged host/NIC control.
Modeled movement latency (ms, log)
Yiwei Yang and Andi Quinn
Current us
Optimized us
Extra speedup
Qwen-style serving Qwen DPU decode Qwen long-context CXL Qwen pipeline/tensor parallel Qwen LoRA adapter Qwen scale-out 2 GPU Qwen scale-out 4 GPU Qwen scale-out 8 GPU Qwen scale-out 16 GPU Qwen scale-out 32 GPU MoE all-to-all Prefill/decode colocation Remote KV cache/CXL memory Training/checkpoint
10,146.8 3,780.3 21,884.2 52,957.2 9,586.4 12,965.2 27,236.4 59,973.2 142,224.1 373,834.6 2,394.0 18,616.8 3,621.4 71,159.5
6,865.3 2,504.4 10,523.1 35,618.1 5,269.2 8,398.3 17,772.1 39,665.3 96,034.6 259,105.0 895.0 12,779.4 1,943.9 49,300.0
1.48× 1.51× 2.08× 1.49× 1.82× 1.54× 1.53× 1.51× 1.48× 1.44× 2.67× 1.46× 1.86× 1.44×
Aggregate
810,380.1
546,673.8
1.48×
• DPU inline: keep copy or persistence control on the BlueField/GPUDirect side and reduce staged control overhead. • GPU pre-transform: perform shrinking transformations such as compress, quantize, filter, or reduce on the GPU before fabric movement. • GPU peer/CXL direct: use direct GPU–GPU or GPU– CXL paths when the edge does not need host staging. Table 4 shows that copy-path optimization raises aggregate modeled speedup from 2.34× to 3.47× over the staged baseline. The path mix is 49 DPU inline choices, 24 GPU pre-transform choices, and 13 direct GPU-peer/CXL choices. Critical-priority events improve by 1.51× over the current fabric_ext path. Table 5 answers the practical optimization question. Large persistent or replicated transfers benefit most from DPU inline copy control. GPU-source compress and reduce events benefit from pre-transform because fewer bytes cross PCIe, RDMA, or CXL. MoE all-to-all benefits from direct peer routing, but its absolute savings are smaller unless expert traffic is on the critical path.
Where Copy-Path Optimization Matters
We next ask where further DPU/GPU/CXL copy-path optimization helps on top of the current fabric_ext placement. The optimizer considers three additional candidates:
5.7
Case Studies
LLM prefill attention and FFN.. Prefill is the most complete use case because attention and FFN exercise different
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
Table 5. Largest copy-path savings over current fabric_ext placement.
spill are larger, so their improvement comes from DPU inline copy and GPU-side compression. This validates the need for multiple policy classes within one workload.
Event
Best path
Qwen 32-GPU KV replica refresh Qwen 32-GPU checkpoint persist Qwen 32-GPU tensor scatter Qwen 16-GPU checkpoint persist Qwen 16-GPU tensor scatter Qwen 16-GPU KV replica refresh Qwen 32-GPU tensor all-reduce Qwen long-context cold KV evict
DPU inline DPU inline GPU peer/CXL direct DPU inline GPU peer/CXL direct DPU inline GPU pre-transform GPU pre-transform
Remote KV over CXL memory. Remote KV cache events combine page-fault semantics from the GPU with placement and persistence decisions in the CXL fabric. The largest gains come from compact/persist and failed-server redirect paths. This is the main example where CXL switch or near-memory placement is the correct policy target: the decision is about memory location and ownership, not only about RDMA priority.
Saved 45.33 ms 31.32 ms 25.08 ms 15.66 ms 12.54 ms 11.48 ms 8.95 ms 8.59 ms
DPU/GPU Copy-Path Optimization Hotspots
5.8
Takeaways
45.33
qwen27b 32gpu kv replica refresh 31.32
qwen27b 32gpu checkpoint persist 25.08
qwen27b 32gpu tensor scatter 15.66
qwen27b 16gpu checkpoint persist 12.54
qwen27b 16gpu tensor scatter
11.48
qwen27b 16gpu kv replica refresh 8.95
qwen27b 32gpu tensor allreduce qwen27b longctx cold kv evict...
8.59
qwen27b pp checkpoint persist
7.83
qwen27b 8gpu checkpoint persist
7.83 7.83
train step checkpoint persist
6.27
qwen27b 8gpu tensor scatter
0
10
20 30 Additional saving over current Damer (ms)
40
Figure 5. Event-level copy-path optimization hotspots. The largest absolute E2E savings come from DPU inline persist/replicate paths and GPU-side pre-transform for compress/reduce traffic.
parts of the same fabric policy. Attention KV fetches and reductions benefit from GPU semantic hints plus DPU/NIC readiness and pacing. Long-context KV placement and coldblock movement benefit from the CXL or switch-local target because ownership and placement are memory-side decisions. FFN activation spill and tensor-parallel exchange benefit from GPU pre-transform when intermediates shrink, and from DPU inline control when the edge is already on the RDMA path. This is exactly the three-architecture split fabric_ext is designed for: GPU tensor execution names the data, DPU/NIC event execution schedules network movement, and the near-Type-2 island JITs and observes local dataflow commands. Semantic MoE scheduling. The MoE workload has the highest extra speedup after copy-path optimization. Critical expert tokens use high-priority fabric decisions, while background expert traffic can use GPU-peer/CXL-direct movement. The policy is semantic: it does not merely prioritize a QP, but ties fabric scheduling to expert ID, token criticality, and destination readiness. Prefill/decode colocation. Decode KV fetches are small but deadline-sensitive, so their improvement is mostly queueing and readiness. Checkpoint throttle and prefill activation
The evaluation supports three claims. First, the eBPF middleware compiler can cover the movement primitive space and produce real BPF artifacts. Second, semantic policies expose E2E effects that packet/QP-only policies miss, especially MoE routing, decode/prefill colocation, and remote KV placement. Third, the right execution point varies by event. DPU inline execution is best for large copy and persist paths; GPU pre-transform is best when HBM-side shrinking reduces fabric bytes; CXL switch or near-memory execution is best when the policy is placement, ownership, persistence, or memory-local reduction.
6
Related Work
eBPF beyond the kernel. eBPF has become a practical substrate for safe OS extension because it combines dynamic loading, helper APIs, maps, and verifier-enforced bounded execution. Recent work has explored eBPF deployment outside the kernel and into heterogeneous environments, including GPU observability and programmability [14]. fabric_ext builds on this direction but changes the unit of programmability. Rather than programming one GPU or one host kernel hook, it programs a fabric policy that spans GPU execution, driver binding, DPU/NIC events, and CXL placement. The main new mechanism is the cross-device contract: consistency-classed maps and a verifier for effects that cross device boundaries. GPU runtime and cluster scheduling. Systems such as Salus, Tiresias, Gavel, and SIA schedule GPU clusters using host-visible metrics and workload-level objectives [4, 9, 12, 15]. They are complementary to fabric_ext. A cluster scheduler decides which job or model should receive resources. fabric_ext decides, at fabric-event granularity, how semantically different transfers within a job should move through GPU, RDMA, and CXL paths. For example, a scheduler may co-locate prefill and decode on a GPU pool; fabric_ext can still prioritize decode KV fetches over checkpoint writes inside that shared fabric. GPU communication and RDMA.. NCCL and RDMA transports provide high-throughput GPU communication [5,
Yiwei Yang and Andi Quinn
6]. DOCA GPUNetIO and related GPU-centric networking mechanisms reduce CPU involvement in packet and RDMA paths [11]. BlueField DPA provides event-triggered execution, RDMA primitives, memory services, and ordering plus notification control near the network path [10]. fabric_ext does not replace these mechanisms. It supplies a policy layer above them, allowing scheduling and placement decisions to use GPU execution semantics instead of only QP, packet, or byte-level information. CXL and disaggregated memory. CXL enables memory expansion, pooling, switching, peer-to-peer access, and fabric management [1, 2]. Systems such as DirectCXL, Pond, TPP, and related tiered-memory work study how to expose and manage CXL memory from hosts and applications [3, 8, 13]. fabric_ext is not a CXL memory allocator by itself. It is a policy compiler that can use CXL memory and switch-local execution as targets when a GPU workload exposes semantic movement intent such as KV block reuse, ownership, persistence, or reduction. LLM serving systems. LLM serving engines such as vLLM expose structured memory behavior through KV block tables and PagedAttention [7]. This structure is exactly the kind of semantic signal fabric_ext wants to carry into the fabric. The difference is boundary. Serving engines usually optimize within the GPU runtime or application scheduler. fabric_ext lets those signals influence RDMA, DPU, and CXL decisions without hard-coding every policy into the serving engine. Near-data and switch execution. Near-data processing has long argued that computation should move toward data when movement dominates cost. fabric_ext applies the same idea to fabric policies rather than general-purpose computation. It does not require arbitrary application kernels to run inside a switch. Instead, it targets small bounded movement-side operations—copy, persist, reduce, checksum, filter, replicate, and placement commit—that are suitable for verification and execution near RDMA or CXL data paths.
7
Discussion
7.1
Why Not a DPU-Only Design?
A DPU is a powerful policy point, but it is not the fabric. DPU/NIC hooks are ideal for RDMA queue scheduling, retransmission response, pacing, and network-path copy control. They are the wrong place to observe warp stalls, kernel phase, KV block lifetime, and GPU page faults. They are also not always the closest place to CXL memory placement or persistence. fabric_ext therefore treats the DPU as one executor among several. The abstraction boundary is the semantic movement graph plus verified policy contract, not a particular BlueField generation or a particular NIC API.
7.2
Why Not Just Use a Fabric Manager?
CXL fabric managers configure routes, devices, memory windows, and topology. They are necessary, but they usually operate at management timescales and with host-visible information. fabric_ext targets per-event policy decisions inside running GPU workloads. For example, a fabric manager can make a CXL memory pool available; fabric_ext can decide that a specific remote KV block should be prefetched, replicated, or persisted because the GPU just exposed its reuse distance and sequence lifetime. The two systems are complementary: the fabric manager owns topology and administrative policy, while fabric_ext handles verified movement decisions within those boundaries. 7.3
Policy Stability and Debugging
Cross-device policies are harder to debug than local eBPF programs. fabric_ext addresses this by making policy state explicit. Every generated artifact records helper IDs, map classes, capability requirements, verifier summaries, placement decisions, and policy epochs. Telemetry maps are intentionally cheap and approximate so operators can observe policy behavior without perturbing ownership or command state. For stronger classes, fabric_ext records epoch transitions and command completions so policy changes can be audited after a failure or performance anomaly. 7.4
Security and Isolation
The main isolation risk is that a semantic policy might affect resources outside its tenant or memory region. fabric_ext uses capabilities at every cross-device boundary. A handler cannot access an RDMA QP, GPU region, CXL window, tenant hint, or ownership map unless the loader grants the capability. Map class also matters for security. Telemetry writes cannot mutate ownership. Hints expire and carry epochs. Ownership and command maps require checked helpers and cannot be updated through raw map stores. 7.5
Limitations
The current implementation is a prototype. It generates real BPF artifacts and validates host/GPU/DPU packaging, but the near-Type-2 hardware-JIT backend is modeled through the movement-plan interface rather than deployed on a production programmable CXL switch or Type-2 device. The E2E speedups in Section 5 are modeled data-movement speedups, not token-latency measurements from a full serving stack. This distinction is important: the results identify where semantic policy should improve end-to-end movement, but real deployment will add device firmware constraints, runtime integration cost, and workload-specific scheduling effects. fabric_ext also assumes fail-stop behavior for policy handlers. The verifier bounds execution and effects, but it does not prove arbitrary semantic correctness of user policy. A bad but bounded policy can still choose poor priorities. Finally,
The Fabric Is the Cluster Driver: Cross-Layer eBPF Policies for GPU–CXL Fabrics
some fabric operations require vendor-specific attach points. fabric_ext’s contribution is the common policy contract and compiler structure; each hardware target still needs a backend that maps verified decisions to device-specific mechanisms. 7.6
Future Work
Three directions are immediate. First, a production CXL/Type2 backend should map switch_compute_engine placement decisions to a hardware-JIT interface on the fabric small core and to real switch, Type-2, or near-memory firmware APIs. Second, the copy-path optimizer should be connected to hardware counters from GPUs, DPUs, and CXL devices so modeled bandwidth and queue costs become measured online values. Third, higher-level LLM runtimes should expose richer semantic hints—expert replica health, KV block temperature, sequence deadlines, and decode batch structure—so fabric_ext can make better fabric decisions without reverseengineering application state.
8
Conclusion
fabric_ext argues that the programmable object in modern GPU systems is the fabric, not an individual device. GPU kernels expose execution semantics that the network and CXL fabric cannot infer from packets or memory transactions. DPUs, NICs, and CXL switches expose congestion, placement, ordering, and ownership state that GPU-local policies cannot see. Treating these as separate control systems leaves performance and correctness decisions at the wrong layer. fabric_ext provides a cross-layer eBPF middleware compiler for this setting. It represents movement as a semantic graph, lowers one logical policy into GPU, driver, DPU/NIC, and CXL switch or near-memory hooks, and protects crossdevice state with consistency-classed BPF maps. Its verifier checks bounded execution, bounded fabric effects, capability access, retry TTLs, dependency cycles, and epoch-atomic program updates across devices. At the fabric edge, the key execution point is a near-Type-2 small core that performs hardware-JIT specialization, state management, and adjacent observation, while the surrounding Von Neumann island executes data-driven movement dataflow. LLM prefill attention and FFN form the canonical use case: one request exposes tensor semantics on the GPU, network pressure on the DPU/NIC, and placement plus ownership transitions at the CXL or switch-local island. The prototype shows that this design can generate real dputime-facing artifacts and expose meaningful E2E optimization points. Across 14 LLM fabric workloads and 86 generated BPF objects, semantic placement and offload reduce modeled data-movement time by 2.34× over staged host/NIC control; copy-path selection across DPU inline, GPU pre-transform, and direct GPU-peer/CXL paths raises the modeled speedup to 3.47×. The central lesson is that the
fabric can make better decisions when it sees GPU semantics, and GPU policies become more effective when they can safely act on fabric state.
References [1] CXL Consortium. Compute express link (cxl) specification 3.0. Technical report, Compute Express Link Consortium, August 2022. [2] CXL Consortium. Compute express link (cxl) specification 3.1. Technical report, Compute Express Link Consortium, January 2024. [3] Donghyun Gouk, Sangwon Lee, Miryeong Kwon, and Myoungsoo Jung. Direct access, high-performance memory disaggregation with directcxl. In Proceedings of the 2022 USENIX Annual Technical Conference (ATC ’22), pages 287–294. USENIX, 2022. [4] Juncheng Gu et al. Tiresias: A gpu cluster manager for distributed deep learning. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2019. [5] Zhiyi Hu, Siyuan Shen, Tommaso Bonato, Sylvain Jeaugey, Cedell Alexander, Eric Spada, Jeff Hammond, and Torsten Hoefler. Demystifying nccl: An in-depth analysis of gpu communication protocols and algorithms. arXiv preprint arXiv:2507.04786, 2025. [6] Sylvain Jeaugey. Nccl 2.0. In GPU Technology Conference, 2017. [7] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP), pages 611–626, 2023. [8] Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Johannes Weiner, Niket Agarwal, Pallab Bhattacharya, Chris Petersen, Mosharaf Chowdhury, Shobhit Kanaujia, and Prakash Prakash. Tpp: Transparent page placement for cxl-enabled tiered-memory. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’23), volume 3, pages 742–755. ACM, 2023. [9] Deepak Narayanan, Keshav Santhanam, et al. Heterogeneity-aware cluster scheduling for deep learning workloads. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020. [10] NVIDIA. NVIDIA DOCA DPA Documentation. https://docs.nvidia. com/doca/archive/2-9-1/doca+dpa/index.html, 2025. Accessed July 21, 2026. [11] NVIDIA. NVIDIA DOCA GPUNetIO Documentation. https://docs. nvidia.com/doca/sdk/doca-gpunetio/index.html, 2026. Accessed July 21, 2026. [12] Suhas J. Subramanya, Daiyaan Arfeen, et al. Sia: Heterogeneity-aware, goodput-optimizing cluster scheduling for deep learning. In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP), 2023. [13] Huaicheng Wang, Mingxing Liang, Yonggang Lee, Xing Yang, Yiwei Zhu, Huisheng Li, and Wei Liu. Pond: Cxl-based memory pooling systems for cloud platforms. Proceedings of the VLDB Endowment, 16(4):752–765, 2022. [14] Yiwei Yang, Tong Yu, Yusheng Zheng, and Andrew Quinn. egpu: Extending ebpf programmability and observability to gpus. In Proceedings of the 4th Workshop on Heterogeneous Composable and Disaggregated Systems, pages 73–79, 2025. [15] Peifeng Yu and Mosharaf Chowdhury. Salus: Fine-grained gpu sharing primitives for deep learning applications. In Proceedings of MLSys, 2020.