Meganeura: Portable GPU Training and Inference through Vulkan and Metal

Meganeura: Portable GPU Training and Inference through Vulkan and Metal Dzmitry Malyshau Independent Researcher kvark@fastmail.com ORCID: 0009-0005-6410-4276 Project and artifact: https://github.com/kvark/meganeura

arXiv:2608.01563v1 [cs.LG] 3 Aug 2026

Abstract

profiling, and correctness work. It also makes adaptation on the deployed device unusually difficult. Consumer graphics APIs offer a tempting common substrate for the alternative in Figure 1. Vulkan covers desktop, mobile, and embedded GPUs across vendors, while Metal exposes Apple GPUs. Both offer general compute and increasingly expose matrix hardware. They do not, however, provide the complete compiler, kernel library, graph-capture, and debugging ecosystem available to CUDA applications. A portable system must decide which layers it can replace, which hardware features it can use safely, and how much performance it loses. This paper studies that tradeoff through Meganeura. The same typed graph representation, automatic differentiation, compiler, shader generator, memory planner, checkpoint format, and runtime support inference and optimizer-backed training. Vulkan is used on Linux, Windows, and Android, and Metal on Apple platforms. The deployed application requires neither CUDA nor ROCm nor a Python runtime. The intended setting is a native application, robot, creative tool, or edge device with mostly static tensor shapes and a reason to keep inference, fine-tuning, or personalization in one codebase. Meganeura is not positioned as a replacement for dynamic Python research workflows, distributed cloud-scale training, or the full operator breadth of mature frameworks. Within that setting, the measured answer is more positive than the conventional wisdom about graphics APIs suggests: both AMD devices reach nearparity or better with compiled ROCm PyTorch on most workloads, minimal-batch latency favors the portable stack on most devices, and the only correctness-gate failures in fifty audited cells are two modes of one APU workload whose cross-device gradients strongly implicate the reference path. The gaps that remain — chiefly backward passes on NVIDIA and Apple hardware — are profiled down to the responsible kernel families, and the paper reports them with the same prominence as the wins. A further motivation is temporal. Deployed models today are frozen at export: the training loop lives in a datacenter, the shipped artifact cannot learn, and adaptation means a round trip through a separate stack — whereas biological learners adapt in place, continuously. Colocating a real training loop with inference on the deployed device

Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries. Meganeura asks whether one compact native compiler can span both phases on consumer GPUs. Its typed static graph, automatic differentiation, optimizer, checkpoint, memory planner, and runtime lower specialized programs through Vulkan and Metal. We compare five matched workloads with PyTorch on NVIDIA and AMD discrete GPUs, an AMD APU, Apple silicon, and an Intel iGPU. The protocol separates strict f32 from validated fast paths and gates forward and backward independently. Forty-eight of 50 device–workload– mode cells pass both gates; the other two share one unresolved backward-reference disagreement on a newly supported APU. In strict f32, Meganeura wins 12 of 20 GPUreferenced minimal-latency cells and has a median valid training gap of 1.8×. On the discrete AMD GPU, four of five inference workloads are within 1.10× of compiled ROCm PyTorch and three training workloads are faster. Under accelerated contracts, the worst training gap is 4.6×. Compilation takes 0.1–2.4 s versus 6–96 s for torch.compile on supported GPU paths; the stripped binary is 13 MiB. Dispatch profiles localize the largest gaps to convolution derivatives and attention backward. A physical Android XR case study transfers a Meganeura-trained decoder into an Adreno/OpenXR application sharing the graphics queue. The results show that general consumer graphics APIs can support a compact shared train-to-deploy stack at useful, sometimes vendor-competitive performance. The measured gaps point to kernel coverage, scheduling, and arithmetic policy rather than an identified API limitation.

1

Introduction

The software path used to train a model is often not the path used to deploy it. Training commonly relies on PyTorch [11] and vendor libraries; deployment may use TensorRT, Core ML, a mobile runtime, a browser engine, or a bespoke application stack. Each boundary duplicates model conversion, operator coverage, numerical policy, 1

CUDA / TensorRT deployment Common split stack

Training graph PyTorch + vendor libraries

Export, convert, recapture, revalidate

Core ML / Metal deployment

separate operator, precision, profiling, and debug policies

Mobile / browser deployment

Meganeura shared stack

One graph representation + one checkpoint

Shared compiler autodiff + specialization + plan

Blade runtime

Vulkan NVIDIA, AMD, Intel, Android Metal Apple

Figure 1: The systems question. A conventional train/deploy boundary can split one model into several runtimespecific paths. Meganeura studies the alternative: keep graph semantics, automatic differentiation, specialization, checkpointing, and execution in one native stack while changing only the graphics backend. The top lane is illustrative, not a feature-equivalence claim about the named products. is a precondition for studying that kind of live adaptation, and a shared train/deploy compiler is the substrate it requires. This paper builds and measures the substrate; live-adaptation behavior on top of it is deliberately left to follow-up work. We ask three research questions:

precision policy that produce both near-reference cases and the largest remaining gaps. • Measured engineering results for deployment closure — including a physical Android XR train-to-deploy case study — automatic static execution, rewrite strategies, shader authoring boundaries, and their present debugging tradeoffs.

1. Performance portability: how close does one implementation come to PyTorch for inference and forward– loss–backward execution across workloads, arithmetic policies, vendors, and device classes?

2

Scope and Design Goals

2. Causes: which specializations recover performance, One stack for two phases. Inference and training and which kernels, launches, or arithmetic constraints are modes of one compiler rather than separate products. explain the remaining gaps? Training differentiates the optimized forward graph, pre3. Systems cost: what does a shared train- serves the forward outputs, appends one gradient output ing/deployment stack change about the artifact, ap- per parameter, and applies an SGD or Adam update in the plication integration, correctness work, optimization same static session. Parameters and optimizer state can be stored in a safetensors checkpoint. A separately compiled strategy, and compiler authoring experience? inference session with the same named parameters can This work makes four contributions: load that checkpoint; it omits derivative and optimizer • A compact executable system whose shared nodes and may apply inference-only fusions. The perforgraph/compiler path supports inference, reverse-mode mance tables time forward–loss–backward separately from differentiation, SGD and Adam updates, and check- the optimizer so that kernel/compiler cost is comparable, while Section 5.1 tests the complete workflow. point transfer into a fresh inference session. • A controlled comparison with five matched model families, explicit arithmetic contracts, raw timing samples, and independent forward and backward correctness gates, frozen at one revision pair across five devices from three GPU vendors and Apple silicon — 48 of 50 cells passing both gates, with cross-device gradient records localizing the two exceptions to the newly enabled reference path, subject to the need for a third implementation.

Portable execution, specialized programs. Portability does not imply one untuned kernel. Meganeura generates specialized programs for matrix shape, transpose mode, tile geometry, input format, prologue, and epilogue. The invariant is that specialization remains behind the same graph/runtime interface and lowers through the graphics stack.

Correctness before peak numbers. Every reported • A decomposition of performance into the kernel cover- cell is conditional on a numerical contract. Inference and age, dispatch structure, cooperative-matrix use, and backward execution are validated independently because 2

an accelerated forward pass can remain accurate while small derivative operands underflow in a reduced input format.

Native builder or ONNX / NNEF importer

Typed static graph

Graph rewrites and ordering

Operational model and observability. Reducing host launch overhead in PyTorch can involve compiler integration or CUDA Graph capture, whose warmup, static-shape, memory-address, and mutation constraints are a recurring integration concern.1 Meganeura starts from a static graph: session construction fixes the dispatch order, buffer ownership, pipelines, and synchronization, and each step() executes that plan. There is no separate capture API or application-managed graph-safe input buffer. This is an automation and usability distinction, not a claim that Blade pre-records a native command buffer or eliminates all host submission overhead. The tradeoff is observability. PyTorch provides a mature eager mode, operator-level inspection, autograd diagnostics, and integrated profilers. Meganeura currently provides graph and dispatch-plan dumps, WGSL source-located parser errors, API validation, per-dispatch GPU timings, gradient summaries, and Perfetto traces, but a failure that crosses generated WGSL, Naga, Blade, and a vendor driver is still hard to localize. Source mapping, automatic graph bisection, captured intermediates, and a single operator replay tool are future work.

Inference forward graph

Autodiff forward + backward Kernel archetypes + target specialization device capabilities + precision policy Generated WGSL validated Naga IR Static dispatch + memory plan

Vulkan

Metal

Figure 2: The shared training and inference compilation pipeline. Training adds autodiff to the optimized forward graph; both modes then use the same specialization, validation, scheduling, memory-planning, and backend path.

3

Meganeura Architecture

3.1

Typed graph and automatic differentiation

Graph nodes carry an operation, typed tensor shape, inputs, and a precision policy. Builders cover dense and quantized parameters, pointwise operations, reductions, normalization, matrix multiplication variants, convolution, attention, indexing, and losses. ONNX and NNEF importers lower to the same graph used by native model builders. The node vocabulary is deliberately mid-level, and the choice is applied consistently: an operation becomes a node when its differentiation rule is local, its lowering maps to one kernel archetype or a short fusion of them, and importers translate near one-to-one — matrix products, convolutions, attention, and normalizations are nodes; their scalar decompositions are not. Both ends of the granularity spectrum carry real costs. Very low-level primitive sets keep the IR minimal but push all performance recovery into search: fused kernels must be rediscovered from clouds of primitives, which is precisely the regime where global techniques such as equality saturation become load-bearing. Monolithic high-level ops at the other end duplicate kernel work per model family and starve the rewriter of reusable structure. The mid-level choice keeps Meganeura’s profitable rewrites local and shallow — one reason greedy rewriting suffices in Section 7 — at the cost of a larger node vocabulary and a per-node autodiff rule

Scale of the implementation. “Compact” is meant as a measurable property rather than a stylistic one. At the frozen revision, the system is 34.5 KLOC of Rust spanning the typed graph, automatic differentiation, rewrites, compiler, code generation, memory planner, runtime, optimizer, ONNX/NNEF importers, and model builders, plus 6.1 KLOC of WGSL across 75 shader files; tests and examples add a further 14.8 KLOC. These are physical source-line counts. That is the budget within which every result below was produced. It is also part of the explanation for them, in both directions: a kernel library of this size cannot cover what a vendor stack covers, so the gaps analyzed in Section 7.3 are better read as a consequence of scope than as a property of the graphics APIs — while the cells at parity show what the same budget already suffices for.

Non-goals. Meganeura is not a complete replacement for PyTorch, does not cover arbitrary dynamic Python programs or distributed training, and does not claim the breadth of CUDA, cuDNN, or large production compilers. The evaluation asks what this deliberately compact system achieves and where it stops.

1 PyTorch CUDA Graphs documentation.

3

to maintain. Training first optimizes and topologically sorts the forward graph, then applies reverse-mode automatic differentiation. Multiple derivative paths are combined explicitly in the graph. Backward-only nodes include transposed matrix products, normalization derivatives, attention derivatives, convolution input/weight derivatives, scatters, and shaperouting operations. A precision bit is attached to derivative regions. It prevents an optional f16-input cooperative-matrix promotion from rounding small gradient operands, while still allowing a native f32 cooperative path where a device exposes one. This policy was added after the Whisper workload showed that forward agreement alone did not bound training error (Section 4.2).

3.2

3.4

WebGPU is a standardized, safety-oriented graphics and compute API exposed by browsers and native implementations. wgpu is a Rust implementation of that API; Naga is its shader translation and validation library.2 Meganeura does not execute through a browser or wgpu. It uses Blade, a smaller native graphics abstraction, to submit compute through Vulkan on Linux, Windows, and Android and Metal on Apple platforms.3 Blade issues global barriers at pass boundaries instead of tracking per-resource state; a companion cross-vendor study measures that model’s costs and headroom [9]. Blade and wgpu nevertheless share Naga as a shader boundary, though the integrations differ in two safety-relevant ways: Blade consumes the validated module with Naga’s runtime safety instrumentation disabled (no injected bounds checks, unlike wgpu), and the binding model is driven by Blade-side data declarations rather than reflected from the shader. The shader boundary is validation and translation, not sandboxing — appropriate for a trusted, compiler-generated kernel set, and part of what keeps the dispatch path thin. This paper treats Blade as the device, resource, command, and presentation substrate. Meganeura owns the ML graph, autodiff, rewrites, kernel generation, precision policy, dispatch/memory plan, optimizer, and evaluation studied here; Blade’s general graphics architecture is outside the paper’s scope. Generated or templated WGSL—the WebGPU shading language—is parsed and validated by Naga, then consumed by Blade as a Naga module for SPIR-V/Vulkan or Metal execution. WGSL is therefore an authoring and diagnostic boundary, not evidence that the runtime itself is WebGPU and not a required source round trip at execution time. Meganeura previously constructed Naga modules directly. This exposed arena handles, expression-emission ranges, and errors such as “Expression is not cached” without a useful source location. Immediately before retreating from that design, the direct-IR code generator had reached 6,359 Rust lines. Generated WGSL replaced it with 949 Rust lines and 1,402 WGSL lines, a roughly 63% reduction for the corresponding implementation. Section 8 discusses this negative result.

Graph rewriting

Meganeura recognizes algebraic and implementation-level forms such as x σ(x) → SiLU(x), decomposed SiLU-gating → SwiGLU, matrix-product plus residual, and packed projection forms. A tensor-traffic cost estimates bytes read and written by an extracted graph. The compiler supports deterministic greedy rewriting and several equality-saturation modes built with egglog [26]. The latter include fixed windows, repeated-region outlining, and whole-graph saturation. Equality saturation is established prior art [23, 24]; our contribution is the measured result that it is not the right production default for the current rewrite space. Section 7 compares the modes.

3.3

WGSL, Naga, and graphics backends

Kernel archetypes and specialization

Rather than treating every operation as an unrelated shader, the code generator uses a small conceptual set of archetypes: • pointwise DAGs with unary/binary inputs and chain fusion; • workgroup reductions with composable per-element and output logic;

• matrix products with transpose modes, storage formats, prologues, epilogues, scalar tiles, GEMV, and 3.5 Static execution and memory planning cooperative tiles; Compilation emits a fixed dispatch sequence with explicit • implicit-GEMM and spatial convolution kernels and logical buffers. The runtime groups dispatches at dependency barriers, computes buffer lifetimes, and aliases their derivatives; step-local intermediates with disjoint live ranges. Parame• tiled attention forward, dQ, and fused dK+dV kernels. ters, graph outputs, gradients, and stateful buffers remain Specialization is shape- and capability-aware. For exam- pinned. Device-local storage is preferred on discrete GPUs; ple, a matrix epilogue is part of the same pipeline/geometry host-visible storage remains available for unified-memory key as its matrix kernel. A cooperative implementation devices and diagnostics. Calling step() walks this precompiled sequence without stages accumulator tiles through workgroup memory before rediscovering the graph or asking the application to capture applying a scalar pointwise DAG. Weighted f16/Q4/Q8 it. Blade still creates an encoder, records the selected kernels remain on their own code-generation path until that path can compose with epilogues without changing the B-buffer interpretation.

2 https://github.com/gfx-rs/wgpu/tree/trunk/naga 3 https://github.com/kvark/blade

4

compute passes, and submits work for the step; the current implementation is therefore graph-static but not equivalent to native CUDA Graph command-buffer replay. Capabilities select scalar, small-tile, cooperative-matrix, generated attention, and storage-format-specific pipelines. This selection also fixes workgroup geometry and padding. Keeping these decisions atomic prevents a failure mode in which a scalar epilogue pipeline is dispatched with cooperative geometry.

4

remains f32 unless a future experimental path passes the same gradient gate. This distinction repaired a concrete error. An earlier Whisper accelerated run had small forward error but large gradient disagreement because derivative operands were rounded before cooperative matrix multiplication. Marking the automatically differentiated region full precision reduced the development run’s total-gradient relative error from approximately 22% to 0.023%. The frozen accelerated runs confirm the repair: Whisper’s total-gradient error is 0.023% on both discrete devices where the cooperative path engages (Section 5.4).

Evaluation Methodology

Inferena is the open-source cross-framework harness used for the evaluation.4 It constructs matched inputs and objectives, invokes each framework runner, synchronizes timing boundaries, retains raw samples, and records precision, environment, revision, output, and gradient metadata. The harness does not supply kernels to either engine. Every artifact identifies its exact Git revision alongside the Meganeura revision.

4.3

Each series uses five untimed warmups and at least 20 retained samples. We report median, interquartile range, minimum, maximum, and raw samples. GPU execution is synchronized at timing boundaries. Gap profiles are collected only after the ordinary series and never replace its latency. Blade hardware timestamps are retained for repeated executions with one compute pass per plan dispatch, together with the selected pipeline, workgroup geometry, logical input/output bytes, and driverreported executable statistics where available. The artifact records the profiled wall time, timestamped GPU sum, and their ratio to the ordinary grouped-pass median, because pass-level instrumentation itself can be expensive. Vulkan intervals include the inter-pass barrier before the following dispatch; Metal uses compute-encoder boundary counter samples. We therefore use these profiles to rank end-to-end dispatch costs, not as instruction-level kernel timings. Inference is one complete no-gradient forward pass. Latency is an agreed minimal shape, not an enginespecific shortcut. Forward–loss–backward includes those three phases but no optimizer update. This isolates graph/compiler and kernel work from a choice of SGD or Adam; Section 5.1 separately verifies a real update and checkpoint handoff. Compile includes graph construction, optimization, and GPU pipeline construction for the measured sessions; model download and parameter upload are excluded or reported separately.

Reference system. PyTorch is the primary reference because the central experiment needs matched forward, loss, and backward execution for every workload, in addition to accelerated vendor backends. It is a much broader framework, and the comparison does not imply feature equivalence. An inference-only runtime would be informative for a different question but could not serve as the common reference for the shared training/inference claim. Likewise, the footprint comparison in Section 5.7 measures deployment closure, not equal functionality.

4.1

Matched workloads

Table 1 summarizes the audited graphs. PyTorch and Meganeura use matching shapes, objectives, canonical parameter names, and deterministic inputs. Physical matrix layouts are transposed during initialization where required.

4.2

Timing and validity

Arithmetic contracts

Inferena defaults to the practical accelerated configuration; passing –strict disables documented reduced-input matrix paths. The paper reports both. Accelerated mode asks how fast each engine’s normal quality-validated hardware path is, while strict f32 is the controlled arithmetic comparison. Accelerated mode is not a matched-format experiment. PyTorch strict mode requests its highest float32 matmul precision and disables CUDA matmul and cuDNN TF32. The practical default permits TF32. Meganeura strict mode disables f16 cooperative matrix and cooperative f16 attention. Its practical default permits eligible forward matrix, convolution, and attention inputs to be rounded locally to f16, accumulates in f32, and stores f32. Backward

Memory. Each result also records GPU memory, and the two engines account for it differently enough that every per-phase figure carries an explicit basis label. Meganeura reports what its execution plan physically allocates after lifetime-based aliasing, together with the same plan’s logical buffer total; PyTorch reports its caching allocator’s peak allocated and peak reserved bytes. Those quantities are not interchangeable and are never placed in one column. The cross-engine figure is per-process device memory, taken from VK_EXT_memory_budget or Metal’s current allocated size for Meganeura and from NVML, amd-smi, or the MPS driver-allocated size for PyTorch. It is the comparable one because it is reported by the driver rather than by either engine: it includes context, pipeline, and staging overhead

4 https://github.com/kvark/inferena

5

Workload SmolLM2-135M

Parameters 135M

Full shape

Objective and scope

batch 1, sequence 128

30-layer decoder-only transformer; token cross entropy 16-layer action expert; output MSE timestep-conditioned convolutional U-Net with self/crossattention; noise MSE inference-folded normalization representation; class cross entropy four encoder layers, 1,500 output positions; output MSE

SmolVLA action expert [17]

99.85M

Scaled Stable Diffusion 1.x U-Net [14]

10.93M

ResNet-50 [2]

25.53M

batch 4, 3 × 224 × 224

Whisper-tiny encoder [13]

8.21M

batch 1, 80 × 3000 mel input

batch 1, 50 action tokens, 16 VLM tokens batch 1, 4×32×32 latent; context 77 × 768

Table 1: Audited workloads. The diffusion graph preserves the characteristic latent, timestep, and text-conditioning paths, but reduces width, depth, and block count and is not checkpoint-compatible with SD 1.5. Whisper omits the decoder. Property

Practical default

–strict

Persistent tensors PyTorch Meganeura scalar path Meganeura eligible forward matrices Meganeura backward matrices Output

f32 TF32/high permitted f32 f16 input, f32 accumulate f32 f32

f32 TF32 disabled f32 f32 f32 f32

Table 2: Arithmetic permissions. TF32 and IEEE f16 have different exponent ranges; accelerated results are therefore reported separately. that neither internal accounting observes, and being per- ure invalidates that cell without discarding a valid inference process it is unaffected by an unrelated workload sharing cell. the device. A backend that cannot report a figure records it as absent rather than as zero. Two sampling limits accompany every memory cell. Meganeura samples at phase boundaries rather than continuously within a step, so its per-process figure bounds residency from below. PyTorch’s phases share one allocator 4.4 Device and revision controls pool without releasing it, so each peak includes residency established by earlier phases. To keep qualitative language falsifiable, a valid result is Every artifact records clean Meganeura and Inferena recalled competitive only when its median is no more than 2× visions, GPU and driver identifiers, OS, API/backend, the PyTorch median in the same device, workload, shape, toolchain versions, precision switches, optimizer mode, inand arithmetic mode. Exact ratios are always reported; put metadata, timing samples, output fingerprints, and the threshold does not enter an aggregate metric. validation diagnostics. The frozen matrix (Table 3) spans For SmolLM, the 128-token full forward is reported five machines: NVIDIA and AMD discrete GPUs, an explicitly as prefill. The current one-token graph does not AMD APU, an Intel integrated GPU, and Apple silicon. carry a KV cache and is therefore labeled stateless one- All 50 result artifacts — five devices, five workloads, two token latency, not decode latency. This paper makes no arithmetic modes — carry the same clean revision pair decode claim; a decode result would require both engines (Meganeura 7561a64, Inferena 7ca9c5c7); no historical to use a matched cache layout and cache length. cells are mixed in. An integrated or edge-class device is PyTorch is the numerical reference. Forward validity included only if both runners complete the audited workrequires matching output shape, relative output L2 error loads: the Intel machine qualifies because the PyTorch below 1%, and symmetric relative loss error below 1%. A reference completes there on its documented CPU fallback, forward–loss–backward cell additionally requires matching which the tables label explicitly, and the AMD APU qualicanonical trainable parameter sets, total-gradient-norm fies because its reference completes every workload — its error below 5%, and relative L2 error below 5% over the backward-validation failure on one workload is a reported vector of per-parameter gradient norms. A backward fail- result (Section 5.9), not an exclusion. 6

Device

Class

OS

Meganeura backend PyTorch build (backend, mode)

NVIDIA GeForce RTX 5070 AMD Radeon RX 7900 XT (RADV) AMD Radeon 780M (RADV) Intel Graphics (RPL-U) Apple M3

discrete, 12 GB discrete, 20 GB integrated integrated unified SoC

Linux Linux Linux Linux macOS 15.7.3

Vulkan, 595.71.05 Vulkan, Mesa 26.0.3 Vulkan, Mesa 25.2.8 Vulkan, Mesa 26.0.3 Metal

2.13.0+cu130 (CUDA 13.0, compiled) 2.10.0+rocm7.1 (ROCm 7.1, compiled) 2.12.0+rocm7.14.0 (ROCm 7.14, compiled) 2.11.0+xpu (CPU, eager) 2.11.0 (MPS, eager)

Table 3: The frozen device matrix. Each PyTorch build is the newest vendor wheel that functioned on that machine, which is why versions differ. “Compiled” means the measured series ran under torch.compile; “eager” means the compiler was unavailable there (unsupported for MPS in the harness; Inductor fell back on the Intel machine’s CPU device). On the Intel machine the +xpu wheel installs but exposes no usable XPU device, so its reference executes on the CPU.

5

Results

5.1

Functional train-to-deploy check

The performance protocol deliberately stops before an optimizer update. To verify the broader systems claim, the artifact also contains a deterministic executable check in examples/train_deploy.rs. It compiles a two-class linear model, performs 40 GPU SGD steps over eight separable points, saves the parameters as a safetensors checkpoint, constructs a fresh inference-only session, loads that checkpoint, and runs the same inputs without autodiff or optimizer dispatches. On the development RTX 5080, cross-entropy fell from 0.693147 to 0.026063 and the reloaded inference session classified 8/8 points correctly. This is a capability and checkpoint-compatibility test, not evidence of model quality or generalization. Its value is that it exercises the complete graph→autodiff→update→checkpoint →freshinference path that the timing columns intentionally de- Figure 3: Quest 3S casting capture of DinoVision’s live RGB reconstruction. This qualitative image documents compose. physical deployment; the correctness and timing claims use the protocols described below.

5.2

Application-scale train-to-deploy case study: DinoVision

Host training and held-out evaluation. The frozen We evaluate Meganeura beyond isolated operators with encoder caches features on an NVIDIA host, after which DinoVision, an Android XR application that reconstructs Meganeura trains a 2,012,547-parameter decoder-only passthrough imagery from intermediate DINOv3 [19] fea- batch graph. Deployment uses the same decoder contures (Figure 3). The application center-crops and resizes struction and learned parameters in a joined batch-one an RGB frame to 224 × 224, executes the first three lay- graph. The paths therefore share graph operators, comers of a frozen DINOv3 ViT-S/16 encoder, rearranges its piler, memory planner, and runtime, but are not literally 14 × 14 × 384 patch features, and decodes them to RGB. the same complete graph. On device, the encoder and decoder form one batch-one We preserve Imagenette’s upstream split. Three indeinference graph, and two asynchronous eye sessions share pendently initialized runs each use a class-balanced 2,500Blade’s Vulkan context and queue with OpenXR rendering. image training subset, 12,000 batch-eight Adam updates, This case study exercises patch projection, learned pre- and all 3,925 validation images. Seed zero was selected for fix tokens, axial RoPE, multi-head attention, LayerNorm, deployment before validation; seed one later scored highest. GELU MLPs, LayerScale, residual connections, convolu- Across seeds, global validation PSNR is 21.86 ± 0.11 dB, tion, group normalization, SiLU, upsampling, autodiff, median-image PSNR is 22.45±0.12 dB, median RGB SSIM Adam, parameter serialization, Android cross-compilation, is 0.6405 ± 0.0076, and median MAE is 0.04672 ± 0.00070 and compute/graphics co-tenancy. It is evidence about (mean ± sample standard deviation). The public artifact stack breadth and deployment closure, not a matched retains every per-image record; host wall time is excluded PyTorch comparison or a capture-to-photon measurement. because the interactive machine experienced recorded sus7

The experiment does not measure on-device trainTable 4: Meganeura encoder agreement with the indepening, PyTorch-on-Android speedup, quantitative headsetdent reference. camera quality, or capture-to-photon latency. The curEncoder depth Relative L2 CLS cosine Worst patch cosine rent path includes CPU camera conversion and patchi1 0.000781 1.000000 0.999989 fication, explicit GPU transfers, output readback, tem3 0.001403 1.000000 0.999995 poral smoothing, and renderer upload. Full weights, 12 0.002253 0.999998 0.999995 raw samples, manifests, environment records, and validation scripts are available at https://huggingface. co/mad-bot/dinovision; source is available at https: pension gaps. //github.com/kvark/dinovision. The artifact also provides the full-resolution casting captures and a 39-second Independent correctness gate. The independent gate live video. proved necessary: an initial LayerScale layout mismatch updated only the first token even though the output images remained plausible. We rejected all affected weights and 5.3 Strict-f32 results measurements, corrected the layout, added exact attention Table 6 and Figure 4 answer RQ1 with raw per-device and LayerScale CPU tests, and retrained all replicates. An independent Torch/Transformers reference and values rather than a universal-win claim, and the valMeganeura consume the same normalized f32 tensor and ues support more than parity-chasing. Against compiled checkpoint. Predeclared thresholds are relative L2 ≤ 0.01, CUDA PyTorch on the RTX 5070, Meganeura is faster on CLS cosine > 0.999, and every patch-token cosine > 0.999. two of five inference cells (diffusion U-Net 0.82×, ResNetTable 4 shows that the deployed and full-depth controls 50 0.90×) and three of five one-token cells; the remaining inference gaps are 1.84–2.65×, and training runs 1.8–2.9× pass. behind. On the RX 7900 XT the portable stack effectively reaches the vendor stack: four of five workloads Physical Android correctness. Before timing, the are within 1.10× for inference (SmolVLA is 0.67×, i.e. host and Quest execute one public fixed RGB frame with 33% faster) and three of five training cells are outright identical encoder and preselected decoder weights. Prepro- faster (SmolVLA 0.73×, diffusion U-Net 0.89×, Whisper cessed patches are bit-exact. Encoder output has relative 0.90×), with ResNet-50 the outlier in both (1.85× and L2 = 0.001059, cosine 0.999999441, and minimum to- 2.41×). On the Radeon 780M APU — the machine whose ken cosine 0.999973147; spatial decoder input has relative vendor support is newest — Meganeura wins three of five L2 = 0.001063 and cosine 0.999999438; reconstruction has inference cells at 0.72× and two training cells, with only relative L2 = 0.000457 and cosine 0.999999938. the convolution pair beyond 1.25×. On the Apple M3, inference lands at 0.93–1.88× and training at 1.20–2.84× Submission chunking under graphics co-tenancy. against eager MPS — the weakest surface in the matrix, A native benchmark first measures the joined 2.359-GMAC and the one that improved most from the profile-guided graph in isolation. Each chunk cell retains 20 synchronized optimization pass discussed in Section 7.3. On the Intel samples after five warmups in three fresh processes. We machine the comparison inverts: the vendor XPU path then run a live-worn stereo OpenXR sweep for 90 seconds does not function, and the portable stack outperforms the per cell at inference interval zero. Both sweeps use the resulting CPU reference in 12 of 15 cells while being the predeclared order 4, 16, 1, 12, 2, 8 to expose time drift. Ta- only functional GPU path on that device. ble 5 reports application render submissions, not composiThree cross-cutting observations. First, minimal-shape tor/display rate; worker latency is not capture-to-photon latency favors the static dispatch plan: Meganeura is latency. After discarding two initial five-second windows, outright faster in 12 of the 20 GPU-referenced oneevery cell retains 16 windows. All state snapshots report token/minimal cells, most visibly on ROCm, where Pyan awake, worn headset and thermal status zero; peak Torch’s one-token SmolLM step takes 6.65 ms against reported GPU temperature is 63.9 ◦ C. 1.93 ms (3.4×). This small shape is especially sensitive Chunking incurs monotonic isolated cost, reaching 18.5% to launch and synchronization structure, and the static at 16 chunks. Under live co-tenancy, two chunks instead dispatch sequence is faster despite torch.compile; the improve render submissions by 18.8% and the lower-eye measurement does not separate host launch cost from update rate by 8.7%, while reducing median worker latency small-kernel quality. The exceptions are compute-bound by 20.2% relative to one chunk. Larger counts regress minimal shapes (Whisper’s full-length encoder, batch-1 non-monotonically, so the result establishes chunking as a ResNet-50 on the M3), where kernel quality decides inmeasurable, caller-selected scheduling control rather than stead. Second, the convolution workloads are the only ones a universal two-chunk optimum. The current runtime that resist parity on the AMD devices: ResNet-50 training partitions ordered compiler barrier groups evenly by count; spans 1.60–2.41× across the GPU-referenced machines, it does not estimate their duration or observe renderer while every non-convolution AMD training cell sits at or latency. below 1.22× — implicating convolution derivative cover8

Table 5: Isolated submission cost and live Blade/OpenXR co-tenancy on Quest 3S. Brackets are interquartile ranges; overhead is relative to one chunk. Chunks Isolated ms [IQR] Overhead Render submit Hz Lower-eye Hz / worker ms [IQR] 1 2 4 8 12 16

96.78 [96.02, 97.41] 100.79 [99.78, 101.46] 105.76 [104.75, 107.49] 108.25 [107.33, 109.99] 112.91 [112.06, 114.43] 114.68 [113.67, 115.48]

baseline +4.1% +9.3% +11.9% +16.7% +18.5%

10.83 12.87 12.07 12.70 12.61 11.87

Compile (s)

Full / prefill (ms)

Ours

Ours

Minimal / one-token (ms)

PT

×

NVIDIA GeForce RTX 5070 — Meganeura Vulkan vs. PyTorch CUDA, compiled SmolLM2-135M 1.52 50.2 12.41 6.44 1.93 2.89 3.21 0.90 47.00 SmolVLA 1.20 23.7 4.46 2.43 1.84 1.81 1.49 1.21 13.10 Diffusion U-Net 0.65 28.8 2.69 3.27 0.82 2.73 3.29 0.83 10.59 ResNet-50 0.99 13.1 7.26 8.06 0.90 4.67 4.90 0.95 36.63 Whisper-tiny 0.74 7.3 9.04 3.41 2.65 9.03 3.42 2.64 33.27

16.35 5.78 5.94 16.33 11.97

2.87 2.27 1.78 2.24 2.78

AMD Radeon RX 7900 XT — Meganeura Vulkan vs. PyTorch ROCm, compiled SmolLM2-135M 0.47 96.0 10.79 9.78 1.10 1.93 6.65 0.29 39.80 SmolVLA 0.34 18.1 3.05 4.57 0.67 1.39 3.96 0.35 9.29 Diffusion U-Net 0.09 26.7 2.31 2.36 0.98 2.34 2.35 0.99 7.89 ResNet-50 0.23 20.8 6.83 3.69 1.85 4.28 3.46 1.24 38.73 Whisper-tiny 0.21 6.7 6.75 6.34 1.06 6.74 6.49 1.04 25.04

32.68 12.72 8.82 16.06 27.91

1.22 0.73 0.89 2.41 0.90

AMD Radeon 780M — Meganeura Vulkan vs. PyTorch ROCm, compiled SmolLM2-135M 1.06 78.0 40.61 56.73 0.72 14.67 19.61 0.75 SmolVLA 0.76 38.7 16.03 22.40 0.72 11.00 14.07 0.78 Diffusion U-Net 0.25 50.5 11.60 5.59 2.07 11.56 4.64 2.49 ResNet-50 0.77 28.1 76.53 41.52 1.84 22.30 10.87 2.05 Whisper-tiny 0.60 19.3 50.42 69.88 0.72 50.41 69.91 0.72

161 54.68 20.10 211 229

170 58.37 12.41 131 187

0.95 0.94 1.62 1.60 1.22†

Intel Graphics (RPL-U) — Meganeura Vulkan vs. PyTorch CPU fallback, eager SmolLM2-135M 1.19 0.0 148 201 0.74 22.97 36.84 0.62 835 SmolVLA 1.12 0.0 58.89 105 0.56 18.95 18.73 1.01 229 Diffusion U-Net 0.23 0.0 18.10 37.78 0.48 18.69 37.65 0.50 74.47 ResNet-50 0.71 0.0 162 310 0.52 43.86 84.56 0.52 707 Whisper-tiny 0.61 0.0 187 307 0.61 188 291 0.65 1231

832 339 107 893 889

1.00 0.68 0.70 0.79 1.39

94.34 29.06 20.47 92.34 97.34

2.00 2.06 1.20 2.34 2.84

PT

PT

Ours

Apple M3 — Meganeura Metal vs. PyTorch MPS, eager SmolLM2-135M 0.35 0.0 41.06 31.08 1.32 7.08 SmolVLA 0.24 0.0 19.95 10.63 1.88 5.24 Diffusion U-Net 0.08 0.0 6.76 7.29 0.93 6.76 ResNet-50 0.19 0.0 49.94 34.36 1.45 15.97 Whisper-tiny 0.14 0.0 60.40 39.77 1.52 59.72

PT

11.65 8.21 7.60 10.30 40.16

×

F+L+B (ms) Ours

Workload

×

5.99 / 149.51 [148.37, 150.99] 6.51 / 119.25 [118.73, 119.84] 6.06 / 129.69 [128.11, 142.42] 6.37 / 128.94 [127.26, 130.13] 6.33 / 131.57 [126.65, 134.12] 5.94 / 143.37 [140.24, 145.27]

0.61 0.64 0.89 1.55 1.49

188 59.73 24.56 216 276

Table 6: Strict-f32 results: medians over 20 samples after 5 warmups. × is the Meganeura/PyTorch ratio; bold marks cells where Meganeura is faster. F+L+B is forward, scalar loss, and backward without an optimizer update. Every cell passes the forward gate of Section 4, and every F+L+B cell except the marked one passes the backward gate. †: invalid — the Whisper backward comparison on the 780M fails the 5% gradient gate; Section 5.9 presents cross-device evidence implicating the reference path, but the ratio is shown for completeness only. Whisper’s minimal shape equals its full shape, so its two forward columns coincide. The Intel rows compare the integrated GPU against PyTorch’s CPU fallback and are reported as a support result, not an engine-efficiency result. age rather than a uniform graphics-API tax. Section 7.3 directly confirms that kernel-family concentration for the largest NVIDIA gap. Third, the correctness margins are wide: across all 25 strict cells the worst forward relative L2 error is 0.0043% against a 1% gate, and across the 24 valid backward comparisons the worst per-parameter gradient-norm error is 0.64% against a 5% gate. Several per-cell error values reproduce to three digits across all five devices, indicating that the residual disagreement is a deterministic operation-ordering difference between the two engines rather than platform noise.

5.4

Accelerated arithmetic

Table 7 compares each engine’s documented fast path under the same correctness gates. The instructive result is where each contract engages. TF32 changes PyTorch only on NVIDIA (ResNet-50 inference gains 3.1×, SmolLM and the diffusion U-Net about 1.9×); its ROCm timings are unchanged and its MPS timings move only within run-torun variation, confirming the permission is CUDA-specific in practice. Meganeura’s f16 cooperative-matrix path engages exactly where the driver reports the capability — the NVIDIA and both AMD GPUs (Whisper inference gains 9

Full / prefill inference

Forward–loss–backward

RTX 5070 (CUDA) 1.93

SmolLM2-135M

2.87

1.84

SmolVLA

2.27

0.82

Diffusion U-Net

1.78

0.90

ResNet-50

2.24

Whisper-tiny

2.65

2.78

RX 7900 XT (ROCm) 1.10

SmolLM2-135M

1.22

0.67

SmolVLA

0.73 0.98

Diffusion U-Net

0.89 1.85

ResNet-50 Whisper-tiny

2.41

1.06

0.90

Radeon 780M (ROCm) SmolLM2-135M

0.72

0.95

SmolVLA

0.72

0.94 2.07

Diffusion U-Net

1.62

1.84

ResNet-50 Whisper-tiny

1.60 1.22†

0.72

Intel RPL-U (CPU ref.) 0.74

SmolLM2-135M

Diffusion U-Net

1.00

0.56

SmolVLA

0.68

0.48

ResNet-50 Whisper-tiny

0.70

0.52

0.79 0.61

1.39

Apple M3 (MPS) 1.32

SmolLM2-135M

2.00 1.88

SmolVLA

2.06

0.93

Diffusion U-Net

1.20 1.45

ResNet-50 Whisper-tiny

2.34

1.52

0.5×

1×

2.84

2×

0.5×

1×

2×

Figure 4: Strict-f32 Meganeura/PyTorch median ratios for all 25 device–workload cells (log scale; the solid line is parity). Teal bars extend left of parity: Meganeura is faster. Orange bars extend right: PyTorch is faster. The hollow bar (†) is the invalid 780M Whisper backward cell of Section 5.9. One-token latency, where Meganeura wins 12 of 20 GPU-referenced cells, is tabulated in Table 6. Generated from the same artifacts as the tables. 1.8× and 1.4× on the discrete cards; SmolLM gains 1.3× on the APU, putting it at 0.55× the compiled reference) — and is absent on the M3 and the Intel iGPU, whose accelerated timings match strict within noise. The net effect is that the NVIDIA gap widens (training up to 4.6× on ResNet-50, where TF32 convolutions compound the strict-mode conv-derivative gap), while both AMD devices keep near-parity: on the RX 7900 XT, four of five inference cells sit at or below 0.99× and four of five valid training cells between 0.75× and 1.16×. Two honest wrinkles. On the RX 7900 XT, SmolVLA regresses from 3.05 to 3.82 ms when the cooperative path is permitted, and the same regression reproduces on the 780M: the current selection treats an available cooperative tile as always profitable, and for these shapes on RADV it is not — a per-shape cost model is the indicated fix. And Meganeura’s ResNet-50 and SmolVLA cells gain nothing on NVIDIA because their hot matrix sites do not currently promote to cooperative geometry (Section 7). Correctness margins remain wide in this mode: among valid cells, worst forward L2 error 0.59% (gate 1%), worst gradient-

norm figures 0.18% total and 0.64% per-parameter (gate 5%), and the Whisper precision repair of Section 4.2 holds at 0.023% total-gradient error on both engaged discrete devices.

5.5

Compilation and startup

Preparing a workload for execution — graph construction, optimization, autodiff, and GPU pipeline creation for the inference, one-token, and training sessions — takes Meganeura 0.08–1.5 s per workload in strict mode and at most 2.4 s with cooperative variants enabled, on every device in the matrix. Where torch.compile functions it takes 6.5–96 s for the corresponding specializations (Table 6), the maximum being SmolLM on ROCm: 96 s against 0.47 s. This is not only a development-loop difference: sub-second specialization is what makes compile-on-device practical for the deployment scenarios of Section 6, where a shipped application cannot assume a warm compiler cache or tolerate a minute of stall.

10

Full / prefill (ms) Workload

Ours

PT

Minimal / one-token (ms)

×

Ours

PT

×

F+L+B (ms) Ours

PT

×

NVIDIA GeForce RTX 5070 — Meganeura Vulkan vs. PyTorch CUDA, compiled SmolLM2-135M 9.90 3.45 2.87 2.86 3.22 0.89 44.28 11.21 3.95 SmolVLA 4.45 1.72 2.59 1.80 1.45 1.24 13.02 5.75 2.26 Diffusion U-Net 2.18 1.72 1.27 2.28 1.74 1.31 10.26 6.08 1.69 ResNet-50 7.29 2.64 2.76 4.67 1.60 2.91 36.77 7.94 4.63 Whisper-tiny 4.91 2.81 1.75 4.91 2.81 1.75 29.14 10.47 2.78 AMD Radeon RX 7900 XT — Meganeura Vulkan vs. PyTorch ROCm, compiled SmolLM2-135M 9.77 9.86 0.99 1.96 6.69 0.29 38.14 32.84 1.16 SmolVLA 3.82 4.56 0.84 1.38 3.96 0.35 9.59 12.71 0.75 Diffusion U-Net 2.13 2.36 0.90 2.10 2.36 0.89 7.84 8.77 0.89 ResNet-50 6.62 3.68 1.80 4.23 3.46 1.22 38.62 16.18 2.39 Whisper-tiny 4.78 6.32 0.76 4.79 6.49 0.74 23.60 28.02 0.84 AMD Radeon 780M — Meganeura Vulkan vs. PyTorch ROCm, compiled SmolLM2-135M 31.42 56.65 0.55 14.69 19.45 0.76 151 169 SmolVLA 19.27 22.36 0.86 11.09 14.20 0.78 55.15 58.19 Diffusion U-Net 11.36 5.65 2.01 11.16 4.73 2.36 19.77 12.41 ResNet-50 78.75 41.44 1.90 22.49 10.84 2.07 207 132 Whisper-tiny 52.41 69.73 0.75 52.34 69.86 0.75 231 186

0.90 0.95 1.59 1.57 1.24†

Intel Graphics (RPL-U) — Meganeura Vulkan vs. PyTorch CPU fallback, eager SmolLM2-135M 162 302 0.54 22.97 25.70 0.89 859 968 0.89 SmolVLA 59.02 101 0.59 18.91 25.24 0.75 222 323 0.69 Diffusion U-Net 18.25 36.21 0.50 18.73 31.28 0.60 74.24 84.64 0.88 ResNet-50 169 283 0.60 43.93 80.91 0.54 731 853 0.86 Whisper-tiny 194 312 0.62 195 295 0.66 1269 912 1.39 Apple M3 — Meganeura Metal vs. PyTorch MPS, eager SmolLM2-135M 41.22 31.09 1.33 7.49 11.81 SmolVLA 20.07 11.49 1.75 5.45 7.84 Diffusion U-Net 6.80 7.44 0.91 6.80 7.58 ResNet-50 50.52 34.38 1.47 16.06 10.33 Whisper-tiny 58.55 39.58 1.48 58.51 39.74

0.63 0.69 0.90 1.55 1.47

189 59.89 24.56 216 274

94.54 29.06 20.13 93.28 97.34

2.00 2.06 1.22 2.32 2.82

Table 7: Accelerated results under the same gates: PyTorch may use TF32, Meganeura may round eligible forward matrix inputs to f16 with f32 accumulation. The two fast paths are not format-equivalent, so this table is reported separately from Table 6 rather than merged with it. Validation matches the strict table: every cell passes except the marked 780M Whisper backward comparison (†, Section 5.9). Compile times match the strict table within 2 s and are omitted. Training (MiB) Workload SmolLM2-135M SmolVLA Diffusion U-Net ResNet-50 Whisper-tiny

Inference (MiB)

Logical Phys. Saved Logical Phys. Saved 1974 1024 159 1640 978

1707 978 118 861 558

14% 4% 26% 48% 43%

875 596 121 877 287

742 563 84 283 61

15% 6% 30% 68% 79%

Table 8: The memory-planner ablation, measured: logical buffer totals versus physically allocated bytes after lifetimebased aliasing (Section 3.5). Plans are shape-deterministic, so these values vary by at most 1% across backends; the artifact records all devices. Neither engine’s timed training session holds optimizer state.

5.6

Memory

Artifact size is only one deployment cost; the memory a workload holds while running decides whether it fits on a shared device at all. LlamaWeb treats its memory reduction as a headline portability result for exactly this reason [5]. Every result artifact therefore records, per device, workload, and phase, the quantities defined in Section 4: Meganeura’s planned physical allocation, the same plan’s logical total, and per-process device memory for both engines.

The recorded logical/physical pair turns the planner design of Section 3.5 into a measured ablation without a separate run (Table 8). Lifetime aliasing recovers 4–48% of training residency and 6–79% of inference residency, and the spread confirms the design intuition: the saving tracks how much of a graph is step-local intermediate rather than pinned parameter or gradient state, peaking for the activation-heavy ResNet-50 and Whisper graphs and nearly vanishing for the parameter-dominated SmolVLA. Against PyTorch’s caching-allocator training peak on CUDA, the static plan costs more on four of five workloads (e.g. ResNet-50 861 versus 498 MiB, Whisper 558 versus 275 MiB) and less on one (diffusion U-Net 118 versus 164 MiB). A dynamic allocator can retire each activation the moment backward consumes it, while the current plan reserves its high-water configuration for the whole step; the return is that a session performs no allocation at step time and cannot fragment or fail at step N . The bases differ, so these numbers are read side by side, never as one column. The driver-reported per-process figure adds what internal accounting misses. For SmolLM on the RTX 5070 the benchmark process peaks at 3,420 MiB while the plans of its three sessions sum to 3,165 MiB: context, pipelines, staging, and heap retention cost roughly 255 MiB on that driver. The figure is a whole-benchmark-process peak —

11

Linux x86-64 artifact Meganeura benchmark executable, stripped PyTorch + Triton packages only PyTorch CUDA runtime dependency closure

Installed size 12.8 MiB 1.75 GiB 4.66 GiB

Inference

Minimal

F+L+B

Workload

Ours

PT

Ours

PT

Ours

PT

SmolLM2-135M SmolVLA Diffusion U-Net ResNet-50 Whisper-tiny

0.79 0.74 0.82 0.70 0.69

0.87 0.75 0.78 0.83 0.83

1.00 0.96 0.77 0.73 0.70

0.55 0.65 0.79 0.84 0.84

0.62 0.68 0.76 0.52 0†

0.99 0.85 0.90 0.95 0.98

Table 9: Deployment footprint, measured on the Linux development machine. Model weights, Python itself, ordinary OS libraries, and the GPU driver are excluded from mean 0.75 0.81 0.83 0.73 0.52 0.93 both sides. PyTorch is a much broader framework, so this is a deployment-cost comparison rather than a featureTable 10: Pennycook performance portability over the equivalence claim. frozen five-machine set, strict mode. Application efficiency on each machine is measured against the best valid time eithe harness keeps the inference and one-token sessions ther engine achieved there, so values are ≤ 1 and PyTorch’s resident together, and the allocator can retain heap from CPU fallback on the Intel machine counts as its own rea dropped session — so it upper-bounds any single-session sult on that machine. †: an invalid cell is not a result, deployment. Even so it cuts both ways by workload: for so Meganeura’s Whisper training entry scores zero under the diffusion U-Net on NVIDIA the whole Meganeura pro- the metric’s discipline — even though Section 5.9 presents cess peaks at 281 MiB against PyTorch’s 802 MiB, and for evidence implicating the reference path. Accelerated-mode Whisper on the M3 at 618 MiB against 1,236 MiB, while values are in the artifact; they shift the column means by for SmolLM the three-session Meganeura process is the at most 0.05 and change no ordering. larger one. The artifact retains every figure with its basis label.

5.7

Deployment footprint

The uniform stack also changes what an application must ship. Table 9 reports a development measurement on Linux x86-64. The Meganeura entry is the stripped Inferena runner, which embeds the graph compiler, autodiff, runtime, model builders, and shader sources in one executable. It dynamically links ordinary OS libraries and relies on the installed GPU driver, but has no Python, CUDA, or ROCm userspace dependency. The PyTorch closure was computed recursively from active Python package metadata rather than from the complete development virtual environment: active Requires-Dist edges and requested extras were followed from torch, and each installed file was counted once. The packages-only row sums the torch and triton distributions. The roughly two-orders-of-magnitude difference is relevant to edge and application embedding, but it is not a substitute for model coverage or performance. TinyIREE similarly treats runtime and artifact size as first-class deployment results, although it studies embedded inference rather than portable GPU training [7]. What the closure costs in practice. Assembling the reference stack for this study was itself a measurement of the fragmentation in Figure 1. Each machine needed a different vendor package channel: an extra wheel index for CUDA (cu130), an extra index plus exact version pins for ROCm (PyPI’s default torch resolves to the CUDA build otherwise), and a full index replacement for Intel’s +xpu build. The resulting PyTorch versions span 2.10–2.13 (Table 3) because each vendor channel lags differently. The outcomes also differ: on the Intel machine the +xpu wheel

installs cleanly yet exposes no usable device, so the measured reference is its CPU fallback, and torch.compile was exercised only on the CUDA and ROCm machines. The portable side of the same experiment is one statically specialized binary per platform plus the system’s installed graphics driver; no per-vendor package source, version pin, or compute runtime was involved on any of the machines. The Radeon 780M APU makes the asymmetry sharpest; Section 5.9 reports its bring-up cost and the one workload responsible for both correctness failures in the matrix, whose cross-device gradients implicate the reference path.

5.8

Performance portability

Table 10 aggregates the per-device results with the Pennycook metric [12]: application efficiency per machine (relative to the best valid time observed on that machine), combined by harmonic mean over the frozen set, with zero for a machine on which an application produces no valid result. The metric’s severity is visible in one cell: Meganeura’s single invalid backward comparison zeroes its entire Whisper training score, cutting the training mean from 0.62 to 0.52 — even though the cross-device evidence implicates the reference path (Section 5.9). We apply the discipline to ourselves and report the zero. The remaining structure is symmetric in an informative way. For forward–loss–backward the incumbent is clearly more portable (mean 0.93 versus 0.52): vendor training libraries are good wherever they run. For full-shape inference the two are close (0.81 versus 0.75). For minimal-shape latency the portable stack is the more performance-portable system (0.83 versus 0.73). The result is consistent with the static plan reducing sensitivity to backend launch and synchronization costs, although the end-to-end measurements also include kernel quality.

12

5.9

The newest vendor path: bring-up and A shared stack changes the product boundary. A native application can construct or import a graph, train a localized anomaly

The Radeon 780M machine is a full member of the frozen matrix, and it is also the machine whose vendor support is newest — ROCm reached this APU class only recently, and standing the reference up required a current Linux firmware payload and a PyTorch wheel from the separate ROCm 7.14 channel, after which the runs still emit persistent rocSHMEM warnings. The portable stack ran unchanged against the installed Mesa driver. That freshness shows up in the matrix’s one recurring correctness anomaly. Whisper’s forward pass on this machine validates at 0.0016%, but its backward comparison fails the gate at 17.8% per-parameter gradient-norm error, in both arithmetic modes — and the recorded artifacts localize the disagreement. Meganeura’s per-parameter gradient norms on this machine are identical, at recorded precision, to its own norms on the RX 7900 XT, where they pass against that machine’s ROCm reference at 0.035%; PyTorch’s norms meanwhile differ by 16.3% between its own two ROCm machines. This cross-device evidence strongly implicates the newly enabled reference path, and the independent forward/backward gates did exactly what they were designed to do: an accurate forward pass did not launder a backward disagreement. We report the cell as invalid — the protocol has no mechanism to bless our own result without a reference — and do not claim that these records alone prove which implementation is correct. A third implementation is needed to resolve the anomaly; until then, the Whisper row in Table 10 correctly scores a zero. The rest of the machine’s column is unremarkable in the best way: the reference’s per-process memory figure is recorded as absent per the absent-not-zero rule of Section 4, and performance follows the pattern of the other AMD device — Meganeura ahead on the transformer-family workloads, behind about 2× on the convolution pair.

6

Implications for Systems and Products

The API is only one layer of the result. A strong result on one workload would not prove that Vulkan or Metal is intrinsically as fast as CUDA, and a slow result would not prove the opposite. The measured unit is the complete stack: graph representation, fusion and specialization coverage, generated kernels, dispatch structure, driver compiler, and arithmetic policy. The useful systems conclusion is more specific. General consumer GPU APIs expose enough compute and matrix capability for a compact compiler to be competitive on covered graphs; the remaining distance can then be attributed to concrete missing work rather than to “portability” as an indivisible tax.

or adapt it, save the resulting parameters, and run an inference-only plan without introducing a Python service or converting to a second model representation. That is particularly relevant to robotics, creative applications, games, and private on-device personalization, where the ML runtime must coexist with rendering, sensors, and application logic. The compact artifact and Blade contextsharing path make Meganeura a plausible embeddable component, but productization still depends on broader import/operator coverage, versioned checkpoints, stronger diagnostics, and sustained-device profiling.

What an edge demonstration should prove. For a device such as Meta Quest, a more informative artifact than a standalone kernel benchmark would collect a small amount of user-specific data, update a compact model head on-device, checkpoint it, and immediately use it for inference in the same application. A personalized hand/controller gesture classifier is a bounded example: it exercises Android/Vulkan deployment and the train-toinfer handoff without requiring a large generative model. Such a result should report memory, sustained latency, thermal behavior, and interference with the render loop. DinoVision (Section 5.2) evaluates the preceding hostto-edge boundary on a physical Quest: a host-trained decoder is checkpointed and deployed through the same compiler/runtime in an application that shares the graphics queue. It does not update parameters on the headset, so on-device adaptation and its memory budget remain future work.

7

Ablations and Gap Analysis

7.1

Greedy rewriting versus equality saturation

The current rewrite set is small and mostly locally profitable. In CPU-only ablation runs, greedy rewriting reaches essentially the same active node counts as windowed or outlined egglog. Representative SmolLM inference optimization took 0.089 ms with greedy rewriting, 32.6 ms with windowed egglog, and 2.94 ms with repeated-region outlining. Whole-graph saturation took 56.2 ms for inference and 7.43 s for the differentiated graph. End-to-end GPU time for the e-graph modes was within run-to-run variation of greedy. This is a negative result for a strong “e-graphs make Meganeura fast” claim. The rewrite rules provide small benefits, but equality saturation has no demonstrated runtime advantage over their deterministic fixed point. We therefore use greedy rewriting as the production default and retain egglog as research infrastructure for future non-local alternatives. This complements TENSAT and Glenside [24, 21]: equality saturation becomes compelling when the representation and rewrite set create meaningful

13

global choices, not merely because a graph is a tensor b1405a3, it showed backward attention consuming 58.3% graph. of GPU time (FlashGradKV alone 141.9 ms of a 330 ms step, FlashGradQ 57.5 ms), with backward matrix products a distant second at 14.2%. That breakdown motivated the 7.2 Cooperative matrices and epilogues Metal optimization pass that landed in the frozen revision Two lessons about cooperative-matrix promotion came at 7561a64, which cut the frozen cell to 274 ms — a 1.20× measured cost. The first is that pipeline variant, workgroup improvement that moved Apple’s worst training ratio from geometry, padding, and epilogue must be selected atomi- 3.4× to 2.8×. The remaining Apple gaps (1.20–2.84× cally: an early path that combined cooperative workgroup training against an eager reference, with every Apple incounts with a scalar epilogue pipeline could leave output ference cell at or below 1.88×) stay concentrated in the rows unwritten, and the safe replacement stages accumula- backward kernel set, where no cooperative path engages. tor tiles through workgroup memory as one selection unit (Section 3.5). The second is that historical performance Structure, not kernels, where Meganeura already numbers are meaningless without graph-level inspection: wins. The third cause is dispatch structure. The ResNetrestoring the once-removed cooperative-epilogue capability 50 inference profile records 177 barrier groups over 198 did not restore the SmolLM inference ratio associated with dispatches — nearly one barrier per dispatch — and our it, because the packed-SwiGLU rewrite had meanwhile companion cross-vendor study of Blade’s barrier model eliminated the epilogue-bearing matmuls it applied to — quantifies the cost of redundant compute-pass barriers and the frozen accelerated cell stands at 2.87×, and the recov- the headroom from eliding them [9]. Those measurements ery now depends on cooperative promotion at the current predate Blade’s newer global resource-access tracking, so graph’s sites, gated by the cost model that the SmolVLA we cite them as demonstrated headroom rather than as a regression (Section 5.4) shows is necessary. gain this stack currently banks. Together with the frozen la-

7.3

Where the remaining time goes

The largest frozen gaps are ResNet-50 training on NVIDIA (4.6× accelerated, 2.2× strict), SmolLM training on NVIDIA (4.0× accelerated, 2.9× strict), and Whisper training on the M3 (2.8×). For the first and third we captured per-dispatch profiles with the mechanism of Section 4; both were taken at the immediately preceding revision b1405a3, a disclosure that carries different weight in each case.

tency results (Section 5.3), the picture is consistent: where launch and synchronization structure dominates, Meganeura’s static plan already wins; where a single kernel family dominates, coverage decides. Cooperative-matrix promotion is the remaining policy gap: the accelerated contract helps only where hot matmuls promote (Whisper 1.8× on NVIDIA, SmolLM 1.3× on the APU), leaves NVIDIA ResNet-50 and SmolVLA unimproved, and regresses SmolVLA on both RADV devices — the promotion decision needs a per-shape cost model, not merely wider coverage. The improvement paths are therefore concrete and falsifiable: unified convolution derivatives, Metal backward attention and matrix tuning, barrier elision informed by the companion study, and cost-model-gated cooperative promotion — each an absent specialization or policy, none an API limitation identified so far.

Convolution derivatives own the ResNet gap. The NVIDIA ResNet-50 accelerated training profile is directly comparable to the frozen cell: its unprofiled control median (36.81 ms) matches the frozen table (36.77 ms) within 0.1%. Of its timestamped GPU time, backward spatial convolution takes 73.0% and forward conLessons from Direct Naga-IR Auvolution another 16.3% — convolution is 89% of the 8 step. Three derivative shaders account for 28.5 ms of thoring the 36.8 ms step (Conv2dGradWeightGemmSmall 12.6 ms over just 12 dispatches, Conv2dGradInputGemm 8.8 ms, Naga’s internal representation is an arena-based, explicConv2dGradWeightGemm 7.2 ms); matrix, normalization, itly typed module structure: each function owns arenas pointwise, and data movement together are under 11%. of expressions and statements, cross-references are integer The weight-gradient “small” variant averaging over 1 ms handles into those arenas, and expression evaluation is per dispatch marks the concrete occupancy target. This scheduled by explicit emission ranges rather than implied confirms, at dispatch granularity, that the worst gap in the by tree shape. That design serves machine construction, matrix is a coverage-and-tuning property of the current validation, and translation between WGSL, SPIR-V, and convolution-derivative implementations rather than a uni- MSL well — and it is an effective portability boundary form graphics-API overhead — TF32 cuDNN convolutions — but we found it a poor human authoring boundary for on the reference side then widen it. this project. Direct construction required explicit types, globals, bindings, arenas, expression handles, statements, Attention backward owned the Metal gap — and and emission ranges, and a missing range could surface durpaid for the fix. The M3 Whisper training profile ing a later backend transformation with no source-located predates the frozen revision deliberately: captured at diagnostic. Retiring direct construction removed the 6,359 14

lines of IR-building Rust it had grown to, replacing them with 949 lines of text generation plus 1,402 lines of WGSL templates — a net reduction of roughly 63% for the corresponding surface — while making every generated kernel independently inspectable. The temporary Naga-IR→WGSL→Naga-IR round trip was a normalization workaround, not an architectural requirement. We eventually fixed direct module submission, yet retained WGSL authoring because:

TensorFlow.js demonstrated browser-resident training and inference through WebGL [20]; WebLLM shows that WebGPU can retain a substantial fraction of native LLM inference performance [16]. LlamaWeb is the closest recent broad graphics-API performance-portability study: it implements memory-efficient, multi-precision browser inference and evaluates 10 models on 16 devices from eight vendors [5]. A complementary WebGPU dispatch study separates API, framework, and shader overhead across Vulkan and Metal implementations [8]. Meganeura in• generated kernels remain recognizable and indepen- stead studies native Vulkan/Metal, several model families, dently inspectable; and automatically differentiated training as well as inference. We claim neither graphics-API machine learning nor • Naga’s parser owns expression-arena and emission portable automatic differentiation in isolation as novel. invariants; • diagnostics identify source locations; • templates still express target constants, prologues, epilogues, and cooperative variants.

The current system still has 75 WGSL files and 6.1 KLOC, so retreating from direct IR did not solve shaderfamily proliferation. During this work we removed separate dK and dV shaders after making fused dK+dV the compiler invariant. The longer-term target is one typed generator per major archetype, with specialized kernels retained only when profiles justify them. The broader lesson is that the best interchange IR and the best authoring IR need not be the same representation.

9

Related Work

Portable ML compilation and runtimes. TVM established graph/operator optimization and learned scheduling for heterogeneous targets [1]. Glow and MLIR demonstrate typed, multi-level compiler architectures and static planning [15, 4]; Triton exposes a tile-level language for high-performance deep-learning kernels [22]. RAF generates and optimizes training graphs, including automatic differentiation and mixed precision, for vendor GPU systems [25]. Meganeura is narrower in framework breadth and execution substrate: it asks how far a compact native Vulkan/Metal stack can take a shared inference and backward path. TinyIREE makes compiled-artifact and runtime footprint a first-class result for embedded inference [7]; Meganeura measures the same deployment concern for a graphics-API executable that also includes training. Burn is a close open-source implementation analogue: it is a Rust training-and-inference framework whose composable automatic differentiation supports GPU backends including Vulkan, Metal, and WebGPU [18]. Its broader framework and backend design establish that portable Rust autodiff is not itself new. The distinction measured here is one frozen native graphics-stack path evaluated across devices with matched forward/backward gates, vendor references, compile and deployment costs, and physical application co-tenancy.

On-device deployment and training. ExecuTorch is the closest production analogue to the deployment half of this work: a PyTorch program is exported once and executed across mobile, embedded, and desktop backends without reimplementation, at very large deployed scale [10]. Its scope is inference. IREE similarly compiles a model through MLIR into host scheduling logic and device executables, and its Vulkan/SPIR-V path is the closest architectural precedent for compiling machine learning to a portable graphics API.5 Meganeura differs in what crosses the deployment boundary rather than in the ambition to cross it: the same graph, compiler, memory plan, and runtime also produce the backward pass and the optimizer update, so adaptation can occur on the deployed device instead of in a separate training stack. Research on training within edge budgets approaches the same goal from the algorithm side, co-designing quantized sparse updates for microcontroller-class memory [6]; Meganeura retains full-precision reverse-mode differentiation and instead asks how much a portable GPU stack can carry. Tensor graph search. TASO searches verified graph substitutions [3]. egg and egglog provide reusable equalitysaturation and fixpoint-reasoning infrastructure [23, 26]. TENSAT applies equality saturation and global extraction to tensor graphs [24], while Glenside uses access patterns for low-level tensor rewriting [21]. Meganeura contributes neither a new e-graph data structure nor a pure tensor IR; its useful result here is the production ablation against greedy rewriting. Performance portability. Pennycook et al. argue that portability must be measured over an explicit platform set and propose an aggregate that becomes zero when a platform is unsupported [12]. We follow that discipline by freezing the device set, reporting per-device results, and separating arithmetic contracts before considering an aggregate.

15

5 https://github.com/iree-org/iree

10

Limitations and Threats to Va- no value to the paper’s aggregate performance-portability metric. lidity

Five workloads do not establish complete operator or model coverage. The diffusion workload has the latent, timestep-conditioning, self-attention, and 77-token text cross-attention structure of Stable Diffusion 1.x, but uses three reduced-width levels, one residual block per stage, and GELU in place of GEGLU. It is not an SD 1.5 topology or checkpoint and excludes the text encoder, VAE, and sampler. Whisper omits the decoder; ResNet represents folded inference normalization rather than training-time batch statistics. Synthetic deterministic inputs test computation but not task quality. PyTorch is a moving reference whose compiler and libraries vary by version, driver, and platform, and the frozen matrix makes that concrete: the four machines run PyTorch 2.10–2.13 because each vendor wheel channel lags differently, and torch.compile was exercised only on the CUDA and ROCm machines. The eager-only Apple and Intel references make those ratios generous to Meganeura; we note that the conclusion least favorable to us on Apple — training 1.4–3.4× behind — survives the asymmetry, since the handicapped reference still wins. The Intel machine’s CPU reference also shows up to ∼1.5× swings between arithmetic modes whose switches are no-ops on CPU, so Intel ratios should be read as coarse. Conversely, graphics drivers may optimize SPIR-V or Metal differently. Exact revisions, raw samples, and environment metadata make the experiment reproducible but do not eliminate this external validity threat. Strict f32 controls arithmetic permissions, not operation order or bitwise identity. Accelerated mode intentionally compares different fast formats and must not be interpreted as format equivalence. Gradient-norm gates can miss elementwise cancellation; the artifact should add sampled or full gradient vector comparisons where memory permits. Two SPIR-V decorations bear on this and are currently unexploited through Blade/Naga: NoContraction, which would pin FMA contraction and tighten cross-vendor reproducibility of the strict contract, and RelaxedPrecision, a portable reduced-precision hint distinct from explicit f16 storage; both are future work at the shader boundary. The implementation has been optimized most heavily on NVIDIA hardware. Results on AMD, Apple, and integrated devices are therefore both a portability test and a maturity test — which makes the AMD parity result more surprising, not less. The frozen matrix contains one machine per vendor and class, AMD excepted (one discrete and one integrated machine), so per-device results should not be read as vendor-wide generalizations. The frozen matrix contains no Windows device: the Vulkan path is the same code there, but this paper makes no measured performance claim for it. Android is represented by the DinoVision physical-device case study (Section 5.2), not by a matched PyTorch/Meganeura matrix cell; it therefore supports deployment and co-tenancy claims but contributes

Naga’s internal IR validation and Vulkan’s validation of emitted SPIR-V are distinct boundaries. The currently pinned backend triggers VUID-StandaloneSpirv-None-10684, an open upstream Naga/wgpu explicit-layout issue that wgpu suppresses but Blade reports.6 Execution and numerical checks pass on every device, but we do not treat those checks as proof of fully valid SPIR-V; the diagnostic is retained as an explicit artifact limitation until a verified upstream correction is consumed.

11

Conclusion

Can one portable GPU stack span training and deployment at useful performance? On the evidence of the frozen matrix, yes — with measured, attributable exceptions. A typed graph, automatic differentiation, specialized kernels, static execution, checkpointing, and lifetime-based memory planning fit in 34.5 KLOC of Rust and 6.1 KLOC of WGSL; 48 of 50 audited cells pass both forward and backward gates across five consumer devices. The stack deploys as a 13 MiB binary that compiles workloads in seconds. Where kernel coverage matches a workload, the graphics-API path reaches a mature vendor-native reference: training and inference near-parity or wins on both AMD devices against compiled ROCm PyTorch, wins against compiled CUDA PyTorch on two strict inference cells, and the stronger Pennycook portability score for minimal-batch latency. On the machine where vendor support arrived newest, the roles inverted: the portable stack ran everything, and cross-device gradient records strongly implicate the reference path in the matrix’s only failed gates. Where coverage runs out, the gap is profiled, not guessed: convolution derivatives are 89% of the worst step, Metal attention backward was 58% of the second-worst before a profile-guided pass reduced it, while cooperative-matrix coverage and selection explain several arithmetic-policy shifts. These are concrete specialization and policy targets; no API limitation is identified by the measurements. The shared stack does not eliminate systems work; it moves that work from several conversion/runtime boundaries into one inspectable compiler and artifact. The experience identifies the boundaries that currently scale: precision policy must propagate through autodiff, pipeline geometry and epilogues must be selected together, static execution can hide graph capture from the application, greedy rewriting is more practical than equality saturation for the present local rule set, and generated WGSL is more maintainable than direct Naga-IR authoring. The result is therefore both a usable implementation and a map of the remaining work required for portable training and inference to become routine.

16

6 https://github.com/gfx-rs/wgpu/issues/7696

Artifact Availability Source code is available in the public Meganeura and Inferena repositories.78 Every benchmark-matrix table and the ratio figure derive from the revision pair Meganeura 7561a64 and Inferena 7ca9c5c7 (both preserved under the public tag paper-arxiv-1), recorded in each result file. The artifact includes the raw JSON for all 50 cells — timing samples, correctness diagnostics, memory figures with basis labels, and environment metadata — plus the per-dispatch profile sidecars of Section 7.3 (which record their own revision), together with the script that regenerates the benchmark-matrix tables and ratio figure from those files. The DinoVision case study (Section 5.2) is its own frozen artifact: source at https://github.com/kvark/dinovision, weights and evidence records at https://huggingface.co/mad-bot/ dinovision, with its evidence revisions documented there. The repository includes machine-readable citation metadata, and both benchmark revisions have public frozen tags.

Acknowledgments The author thanks the wgpu community for maintaining Naga, the shader validation and translation library that anchors Meganeura’s portable shader boundary, and thanks their family for support and patience throughout this work.

AI-Assistance Disclosure Generative AI tools were used for code review, benchmarkharness development, experimental debugging, and editorial assistance. The author designed the study, reviewed generated changes, executed the experiments, verified the reported claims and references, and accepts responsibility for the manuscript.

[3] Zhihao Jia, Oded Padon, James Thomas, Todd Warszawski, Matei Zaharia, and Alex Aiken. TASO: Optimizing deep learning computation with automatic generation of graph substitutions. In Proceedings of the 27th ACM Symposium on Operating Systems Principles, pages 47–62, 2019. [4] Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache, and Oleksandr Zinenko. MLIR: Scaling compiler infrastructure for domain specific computation. In 2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pages 2–14, 2021. [5] Reese Levine, Rithik Sharma, Nikhil Jain, Abhijit Ramesh, Zheyuan Chen, Neha Abbas, James Contini, and Tyler Sorensen. Llamas on the web: Memory-efficient, performance-portable, and multiprecision llm inference with WebGPU. arXiv preprint arXiv:2605.20706, 2026. [6] Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. On-device training under 256KB memory. In Advances in Neural Information Processing Systems, volume 35, 2022. [7] Hsin-I Cindy Liu, Marius Brehler, Mahesh Ravishankar, Nicolas Vasilache, Ben Vanik, and Stella Laurenzo. TinyIREE: An ML execution environment for embedded systems from compilation to deployment. IEEE Micro, 42(5):9–16, 2022. [8] Jędrzej Maczan. Characterizing WebGPU dispatch overhead for LLM inference across four GPU vendors, three backends, and three browsers. arXiv preprint arXiv:2604.02344, 2026. [9] Dzmitry Malyshau. Global pass barriers without perresource RHI tracking: A cross-vendor study with Blade. arXiv:2607.26506, 2026.

[10] Mergen Nachin, Digant Desai, Sicheng Stephen Jia, et al. ExecuTorch: A unified PyTorch solution to run AI models on-device. arXiv preprint arXiv:2605.08195, 2026. [1] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, [11] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor and Arvind Krishnamurthy. TVM: An automated Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, end-to-end optimizing compiler for deep learning. In Alban Desmaison, Andreas Kopf, Edward Yang, 13th USENIX Symposium on Operating Systems DeZachary DeVito, Martin Raison, Alykhan Tejani, sign and Implementation (OSDI 18), pages 578–594. Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie USENIX Association, 2018. Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In [2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Advances in Neural Information Processing Systems, Sun. Deep residual learning for image recognition. volume 32, 2019. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016. [12] Simon J. Pennycook, Jason D. Sewall, and Victor W. 7 https://github.com/kvark/meganeura Lee. A metric for performance portability. In Pro8 https://github.com/kvark/inferena ceedings of the 7th International Workshop in Per-

References

17

formance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS), 2016.

Corrado, Fernanda B. Viégas, and Martin Wattenberg. TensorFlow.js: Machine learning for the web and beyond. In Proceedings of Machine Learning and Systems, volume 1, 2019.

[13] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. [21] Gus Henry Smith, Andrew Liu, Steven Lyubomirsky, Scott Davidson, Joseph McMahan, Michael Taylor, arXiv preprint arXiv:2212.04356, 2022. Luis Ceze, and Zachary Tatlock. Pure tensor pro[14] Robin Rombach, Andreas Blattmann, Dominik gram rewriting via access patterns. In Proceedings of Lorenz, Patrick Esser, and Björn Ommer. Highthe 5th ACM SIGPLAN International Symposium on resolution image synthesis with latent diffusion modMachine Programming, 2021. els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages [22] Philippe Tillet, H. T. Kung, and David Cox. Triton: An intermediate language and compiler for tiled neu10684–10695, 2022. ral network computations. In Proceedings of the 3rd [15] Nadav Rotem, Jordan Fix, Saleem Abdulrasool, GarACM SIGPLAN International Workshop on Machine ret Catron, Summer Deng, Roman Dzhabarov, Nick Learning and Programming Languages, pages 10–19, Gibson, James Hegeman, Meghan Lele, Roman Lev2019. enstein, Jack Montgomery, Bert Maher, Satish Nadathur, Jakob Olesen, Jongsoo Park, Artem Rakhov, [23] Max Willsey, Chandrakana Nandi, Yisu Remy Wang, Oliver Flatt, Zachary Tatlock, and Pavel Panchekha. and Misha Smelyanskiy. Glow: Graph lowering comegg: Fast and extensible equality saturation. Propiler techniques for neural networks. arXiv preprint ceedings of the ACM on Programming Languages, arXiv:1805.00907, 2018. 5(POPL), 2021. [16] Charlie F. Ruan, Yucheng Qin, Akaash R. Parthasarathy, Xun Zhou, Ruihang Lai, Hongyi Jin, [24] Yichen Yang, Phitchaya Phothilimthana, Yisu Wang, Max Willsey, Sudip Roy, and Jacques Pienaar. EqualYixin Dong, Bohan Hou, Meng-Shiun Yu, Yiyan ity saturation for tensor graph superoptimization. In Zhai, Sudeep Agarwal, Hangrui Cao, Siyuan Feng, Proceedings of Machine Learning and Systems, voland Tianqi Chen. WebLLM: A high-performance ume 3, 2021. in-browser LLM inference engine. arXiv preprint arXiv:2412.15803, 2024. [25] Cody Hao Yu, Haozheng Fan, Guangtai Huang, Zhen Jia, Yizhi Liu, Jie Wang, Zach Zheng, Yuan Zhou, [17] Mustafa Shukor, Dana Aubakirova, Francesco CaHaichen Shen, Junru Shao, Mu Li, and Yida Wang. puano, Pepijn Kooijmans, Steven Palma, Adil ZouiRAF: Holistic compilation for deep learning model tine, Michel Aractingi, Caroline Pascal, Martino Russi, training. arXiv preprint arXiv:2303.04759, 2023. Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. SmolVLA: A vision- [26] Yihong Zhang, Yisu Remy Wang, Oliver Flatt, David language-action model for affordable and efficient Cao, Philip Zucker, Eli Rosenthal, Zachary Tatlock, robotics. arXiv preprint arXiv:2506.01844, 2025. and Max Willsey. Better together: Unifying datalog and equality saturation. In Proceedings of the [18] Nathaniel Simard, Louis Fortier-Dubois, Dilshod Tad44th ACM SIGPLAN Conference on Programming jibaev, Guillaume Lagrange, and Burn Framework Language Design and Implementation, 2023. Contributors. Burn. Software, version 0.21.0, 2026. [19] Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. DINOv3. arXiv preprint arXiv:2508.10104, 2025. [20] Daniel Smilkov, Nikhil Thorat, Yannick Assogba, Ann Yuan, Nick Kreeger, Ping Yu, Kangyi Zhang, Shanqing Cai, Eric Nielsen, David Soergel, Stan Bileschi, Michael Terry, Charles Nicholson, Sandeep N. Gupta, Sarah Sirajuddin, D. Sculley, Rajat Monga, Greg 18

Related documents

• WarpGuard: Towards Control-Flow Attestation for Heterogeneous CPU-GPU Execution#370276
• Entwine: Coordinating Tiled Computation and Fine-Grained Communication across GPUs#999083
• VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU#155230
• Accelerating Dynamic Graph Clustering on GPU Architectures with cuGraph#429327
• Microbenchmark-Driven Analytical Performance Modeling Across Modern GPU Architectures#168296