arXiv CSopen access

Beyond CPU-GPU Frequency: Memory-Clock and Tail Effects in Edge Inference Latency Estimation

Beyond CPU–GPU Frequency: Memory-Clock and Tail Effects in Edge Inference Latency Estimation Jaehoon Kang

arXiv:2606.16106v1 [cs.PF] 15 Jun 2026

Cleinsoft [email protected] Abstract—Frequency-aware latency estimators enable deadline-aware DVFS for edge ML inference by modeling latency over CPU and GPU frequencies. We present a measurement study on an NVIDIA Jetson Orin Nano showing three phenomena outside this modeling scope. (1) The memory clock is a missing axis: the EMC domain exposes only four lockable operating points on our Orin Nano Super under L4T R36.5, and across the realistic upper range (2133→3199 MHz) it shifts median latency by +11% to +48% depending on workload; for a synthetic L2-resident compute kernel at the top GPU clock we observe a reproducible non-monotonic case (−9%), where lowering the clock lowers latency. On the bottom step (665.6→204 MHz), latency inflates by roughly 3× for every workload, including one with near-zero DRAM traffic. A GPU-frequency estimator profiled under one power profile and deployed under another consequently underestimates latency by up to 32%. Tabulating the four lockable points repairs three of four workloads; a parametric 1/femc term does not. (2) Aggregate miss rates hide bursts: at fixed clocks — with and without memory contention — 100k-cycle runs show knife-edge latency distributions whose deadline-miss cliffs span ∼1 ms, yet misses are temporally clustered far beyond independence—at a deadline with 0.1% aggregate miss rate, the probability that the next cycle also misses reaches 74% — 740× the independent baseline. In this dataset, generalized Pareto fits from the first half of each run track the second half’s extreme quantiles (p99.99, ∼5 order statistics) within 6%; deadlines set by Gaussian µ+3σ margins overshoot a 0.1% miss-rate target by 13–29×; GPD margins stay within ∼2× the target (worst 2.02×; some cells conservative by up to 6×) in all eight measured configurations. (3) Frequency actuation is not free: per-domain transition stalls stay below our 100 µs relevance criterion, but the new operating point takes 1 ms (CPU), 5 ms (GPU), and 8 ms (EMC) to take effect—a substantial fraction of typical inference periods for per-inference governors. We release the full measurement harness and discuss implications for the next generation of frequency-aware estimators and governors.

I. I NTRODUCTION Deadline-constrained ML inference on embedded SoCs increasingly relies on DVFS governors guided by frequencyaware latency estimators: models that predict inference latency across processor frequency settings and pick the most efficient setting that meets the deadline. The state of the art models perlayer latency over the CPU×GPU frequency grid, explicitly capturing asynchronous CPU–GPU coupling [1], following earlier GPU-frequency latency models for embedded DNN inference [2].

These estimators share a structural form — a frequencydependent term plus a frequency-independent constant; in the state-of-the-art per-layer model, T (f ) = k/f + b, where b absorbs “frequency-independent” overheads, explicitly including memory transfer delays [1]. On integrated SoCs, however, the memory subsystem has its own DVFS domain (the EMC on NVIDIA Jetson [3]), so b is in fact b(femc ). Joint memory– compute frequency scaling is known to matter for energy [4]; what is missing is a characterization of what its omission does to latency estimation and deadline guarantees. Likewise, governor evaluations report aggregate QoS metrics — latencyQoS percentages [1] or average frame rates [5] — which compress away the temporal structure of misses, and perinference frequency switching assumes the new setting takes effect immediately, though switching costs are documented on discrete GPUs [6] and CPUs [7]. We measure all three assumptions on a Jetson Orin Nano Super — a resource-constrained SKU of the Orin family whose larger members (AGX Orin, Orin NX) validated the state of the art [1]. Our study is an existence proof by construction: a single platform suffices to show that these phenomena exist and are large; we make no claim about their magnitude elsewhere, and our harness is released for replication on other SKUs. We do not claim that the state of the art is incorrect within its stated CPU×GPU scope; we show that deployed deadline control on integrated SoCs depends on additional state variables that a CPU×GPU estimator cannot observe. Contributions. (1) A four-point characterization of the EMC axis across six workloads spanning the roofline [8] (arithmetic intensity ∼1 to L2-resident), showing a universal ∼3× inflation on the bottom EMC step (665.6→204 MHz), workload-dependent +11–48% shifts in the realistic range, a reproducible −9% non-monotonic inversion, and a direct demonstration that a GPU-frequency estimator slice underestimates out-of-scope latency by up to 32%/79%, and that a four-point tabulation repairs three of four workloads; in tracedriven governor simulation, the blind policy breaks its missrate promise on up to 25% of deadlines — missing 55–81% of cycles when it does — while the repair cuts violations to 3–5% (§IV, §VII). (2) A 100k-cycle-per-cell tail study under controlled memory contention: miss cliffs ∼1 ms wide, miss clustering up to 740× the independent baseline, Gaussian

margins overshooting a 0.1% miss target by 13–29× while out-of-sample GPD margins stay within ∼2× it (§V). (3) Perdomain frequency actuation lag measurements (1/5/8 ms for CPU/GPU/EMC) with transition stalls below the 100 µs relevance criterion, reframing switching cost for integrated SoCs (§VI). (4) A reproducible measurement harness, including the pitfalls that silently corrupt such measurements (RT-throttling artifacts, BPMP rate rounding, I/O-buffer flushes in timed loops) (§III). II. BACKGROUND AND R ELATED W ORK A. Frequency-aware latency estimation FLAME [1] exemplifies the state of the art in frequencyaware latency estimation for embedded inference. It models per-layer latency on each processor as T (f ) = k/f + b, where b captures frequency-independent overheads — in the authors’ words, “pipeline stalls and cache misses on CPU, or kernel launch latencies and memory transfer delays on GPU.” The modeled frequency space is CPU×GPU: on the Jetson AGX Orin, 29 CPU steps and 11 GPU steps yield 319 combinations, and the memory clock is not a dimension of the model. Within this scope the estimator is accurate — over 85% of CPU and 88% of GPU estimation errors fall within 10% — and the deadline-aware governor built on it improves latency guarantees over the learning-based zTT governor [5] by 4.35%. Earlier work in the same class models DNN inference latency as a function of GPU frequency alone on the Jetson Xavier NX and Orin Nano [2]. Two properties of this line of work motivate our study. First, evaluation is by aggregate QoS percentages: in FLAME’s FPS-constraint experiments, “most methods achieve near 100% latency QoS rates.” Second, FLAME’s online adaptation is latency-level drift calibration — a sliding window estimates a systematic bias between predicted and measured latency, smoothed by an exponentially weighted moving average. This is a meanlevel correction to the estimate, not a model of the distribution around it. We measure precisely the terms this formulation holds fixed: the content of b (§IV), the tail around the estimate (§V), and the cost of acting on a new frequency choice (§VI). B. Memory-frequency effects on embedded inference That the memory clock matters on Jetson-class devices is established. Han et al. [4] jointly scale memory and computing frequency for DNN inference on the Jetson TX1 and Orin Nano, fitting inference time as a sum of power-law terms in each frequency; their objective is energy reduction, and their latency measurements are averages. Earlier characterization work swept CPU and GPU frequency combinations on the Jetson Nano and Xavier NX and found CPU frequency to have a smaller effect on inference latency than GPU frequency [9]; the memory domain was not swept. We claim no novelty for the observation that memory frequency affects inference. Our contribution is to quantify what omitting the axis does to latency estimation and deadline guarantees: how large and how workload-dependent the error folded into b is, whether the frequency–latency relation is even monotonic, and how

few lockable operating points the axis actually has on this platform. C. Contention and real-time GPU inference The real-time community studies deadline misses for GPU inference directly. DeepRT compares deadline-miss rates across schedulers under concurrent inference requests on a Jetson TX2 [10]; DARIS reports per-policy miss rates of roughly 2–25% for real-time DNN inference [11]; RTGPU provides schedulability analysis for hard-deadline parallel GPU tasks [12]. On integrated CPU–GPU SoCs specifically, Ali and Yun show that memory-bandwidth-intensive CPU corunners slow GPU kernels by as much as 3× on a Jetson TX2, using the IsolBench Bandwidth benchmark [13] as the adversary [14]; our contention design follows this lineage (§III). Recent profiling of concurrent vision inference on the Orin Nano documents counterintuitive power and throughput behavior attributed to DVFS acting near the module power cap [15]. The mitigation side of this literature is also well developed: memory-bandwidth regulation (MemGuard [16]) throttles offending cores to protect latency-critical work, and our contention results say nothing against it — they characterize the unregulated baseline such mechanisms defend. Across the GPU-inference scheduling literature the reported metric is an aggregate miss rate. The real-time community has long formalized why that is insufficient: (m, k)-firm deadlines [17] and weakly-hard constraints [18] specify which miss patterns a task tolerates, and consecutive misses in particular determine control stability [19]. What is missing is measurement: none of the GPU-inference works above reports the temporal structure its misses actually exhibit on integrated-SoC platforms. §V measures exactly that — in weakly-hard terms, which ⟨m, k⟩ constraints the platform empirically violates. D. Tail-aware latency reporting Within real-time systems, applying extreme-value theory to measured execution times is an established line: statistical WCET estimation [20] and measurement-based probabilistic timing analysis (MBPTA) [21] fit EVT models to observed timing to extrapolate exceedance probabilities, with the surveyed caveats — independence, stationarity, representativity — that make iid-grade pWCET guarantees hard to establish [22]. Our use of GPD margins (§V) is MBPTA in spirit but deliberately weaker in claim: we use the fit as a marginal quantile heuristic, validated out-of-sample, precisely because our exceedances are temporally clustered and the iid preconditions do not hold. Outside the embedded niche, reporting latency as percentiles and distribution tails rather than averages is established practice: Dean and Barroso frame service latency entirely in terms of distribution tails [23], and MLPerf Inference standardizes the measurement methodology — untimed warmup, fixed and disclosed system configuration, replicability — for inference benchmarking [24]. Percentiles, however, summarize the marginal distribution: two systems with identical tail percentiles can differ entirely in whether their misses cluster. The workload trend raises the stakes

for the memory axis as well: batch-1 transformer decode is memory-bandwidth-bound, with per-token latency dominated by streaming weights and KV cache [25], [26], so this workload class is governed by precisely the clock domain that frequency-aware estimators omit. Our GEMV decode proxy and quantized-SLM workload (§III) instantiate it. III. M ETHODOLOGY A. Platform and clock control All measurements ran on a single NVIDIA Jetson Orin Nano Super developer kit under L4T R36.5 (kernel 5.15.185tegra) [3]: six Cortex-A78AE cores, an Ampere-architecture integrated GPU, and 8 GB of shared LPDDR5 with a nominal peak bandwidth of 102.4 GB/s (128-bit LPDDR5 at 6400 MT/s; NVIDIA quotes 102 GB/s). We pinned all six CPU cores at 1728 MHz via cpufreq and the GPU at 1020 MHz via its devfreq node, leaving the EMC — the memory controller’s own DVFS domain on Jetson [3] — as the manipulated variable. We locked the EMC through the BPMP debugfs interface by asserting mrq_rate_locked and writing the target to rate. The lockable set is smaller than the BPMP’s own tables suggest. The dvfs_table lists seven voltage operating points, but only four rates lock in practice: 204, 665.6, 2133, and 3199 MHz. Off-table requests round up silently — a request for 1600 MHz locks at 2133 MHz, succeeds from the writer’s perspective, and is visible only in the readback. Our launcher therefore runs a pre-flight gate that locks every rate in the experiment matrix and asserts an exact readback before the first cell, repeats the lock-and-readback at each cell boundary, and re-verifies all clocks after each cell completes; tegrastats traces recorded during every run confirm that the GPU and EMC clocks held. The 204 MHz floor is likewise our own observation, read back from the BPMP min_rate file rather than taken from documentation. B. Workloads across the roofline Table I lists the six workloads. Two are real vision models run in fp32 through the onnxruntime (ORT) CUDA execution provider [27] (EP): MobileNetV2 [28] and ViT-Small [29] (timm vit_small_patch16_224). One is a real small language model: Qwen2.5-1.5B-Instruct (Q4 K M) [30] decoding through llama.cpp’s CUDA [31] backend at 24.2 ms per token at 3199 MHz — an effective ∼43 GB/s, the dequantization arithmetic of the 4-bit format diluting its bandwidth sensitivity. The remaining three are synthetic ONNX anchors that pin the corners of the roofline. The GEMV decode proxy chains twelve 4096×4096 fp16 matrix–vector products, streaming 384 MiB of weights per inference at a measured ∼81 GB/s — about 80% of nominal peak, at arithmetic intensity (AI) near 1 FLOP/byte. The compute-bound anchor took two attempts, and the failed first attempt is itself a finding about roofline reasoning at batch 1. Version 1 chains four 20483 fp16 GEMMs with per-layer weights; its nominal AI is ∼683 FLOP/byte, far above the ridge. Measured, it delivers 6.2 TFLOPS while

TABLE I T HE SIX WORKLOADS . P 50 IS THE MEDIAN COMPUTE LATENCY AT EMC 3199 MH Z WITH CPU/GPU PINNED (1 K ITERATIONS ); “—” = TRAFFIC NOT INSTRUMENTED ; ( EST.) = INFERRED FROM MEASURED DRAM RATE AND TILED -GEMM PANEL - REUSE ARITHMETIC , NOT DIRECTLY MEASURED . Workload

Runtime

Traffic/inf.

Character

MobileNetV2 ViT-Small GEMV 12×40962 GEMM 4×20483 GEMM 64×10243 Qwen2.5 1.5B Q4

ORT fp32 ORT fp32 ORT fp16 ORT fp16 ORT fp16 llama.cpp

— — 384 MiB ∼1 GiB (est.) 2 MiB ∼1 GiB/tok

mixed mixed bandwidth-bound AI 683→65 realized L2-resident compute bw-bound + dequant

p50 (ms) 4.20 11.40 4.54 10.96 18.60 24.26

streaming DRAM at ∼81 GB/s — a realized AI of only ∼65, consistent with panel re-reads in a tiled GEMM against a few-MiB cache hierarchy pulling the kernel back toward the bandwidth roof. Nominal arithmetic intensity lies at batch size 1. Version 2 instead shares one 2 MiB weight across 64 chained 10243 GEMMs; measured DRAM streaming is near zero — consistent with cache residency — and the kernel is compute-bound in realized terms. We kept both in the campaign because they bracket claimed and actual computeboundedness. This asymmetry — real models plus synthetic-only compute anchors — is forced by the workload class: batch-1 decode is bandwidth-bound by construction [25], [26], and every real model we measured leans memory-bound (§IV), so the compute-bound corner is reachable only synthetically. C. Measurement harness Each cell ran a periodic-release loop: a thread under SCHED_FIFO priority 80, pinned to core 5 with its address space locked via mlockall, released one inference per period and timestamped every cycle into release-jitter, compute, and response components. We recorded full percycle distributions rather than summaries, so a deadline is a post-hoc analysis parameter: any deadline can be evaluated against the same raw runs, in line with percentile-oriented latency reporting [23], [24]. The task model is implicit-deadline periodic with absolutetime releases: release i is scheduled at t0 + i · P , and the harness sleeps to that absolute instant, so an overrun never delays subsequent releases. In Part B, MobileNetV2 runs at P = 20 ms (50 Hz) and the GEMV proxy at P = 40 ms (25 Hz); the maximum observed response in any cell (9.3 ms) is below every period, so utilization stays under ∼30% and backlog carryover is impossible — a job never starts late because its predecessor overran. The miss clustering of §V therefore cannot be a queueing artifact of the harness; it is platform behavior. The campaign has two parts. Part A sweeps the four lockable EMC points across all six workloads (24 cells: 1 000 timed iterations per ONNX cell, 500 decode tokens per SLM cell). Part B runs eight 100 000-cycle cells for the tail study: MobileNetV2 at 2133 and 3199 MHz under 0, 2, and 4 adversary threads, and the GEMV proxy at 2133 MHz under 0

D. Pitfalls that corrupt clock measurements Five failure modes produced, or would have produced, data that looked publishable and was wrong. We report each as symptom, mechanism, detection, and remedy. (P1) RT throttling injects phantom stalls. Symptom: busy-waiting SCHED_FIFO probes intermittently report 46– 51 ms outliers, indistinguishable from genuine clock-transition stalls. Mechanism: the kernel’s real-time throttling budget (sched_rt_runtime_us, default 950 ms per 1 s period [32]) suspends a spinning FIFO thread once per period. Detection: our noise-floor control windows — identical probe windows with no frequency change — showed the same artifacts, which no transition effect can produce. Remedy: set sched_rt_runtime_us to −1 for the duration of a run. (P2) Silent BPMP rate rounding. Symptom: a cell labeled 1600 MHz runs at 2133 MHz while every write appears to succeed. Mechanism: the BPMP rounds off-table requests up to the next lockable rate (§III). Detection and remedy: the launch pre-flight locks and reads back every rate in the matrix, and each cell re-asserts the readback; any mismatch invalidates the cell. (P3) stdio flushes inside timed loops. Symptom: a reproducible 4.8 ms “stall” — roughly 48× our pre-specified 100 µs relevance criterion — recurring at fixed cycle intervals. Mechanism: per-cycle logging fills a 4 MB stdio buffer and the flush lands inside the timed region. Detection: the artifact’s period tracked the logging volume, not the clocks. Remedy: accumulate samples in preallocated memory and write only after the timed loop ends. (P4) mlockall ordering versus CUDA. Symptom: CUDA initialization fails with allocation errors in an otherwise correct real-time setup. Mechanism: with MCL_FUTURE in force, the large mappings the CUDA driver creates at initialization must be locked, and on Tegra these allocations fail. The failure is loud, but the obvious fix — dropping mlockall — silently reintroduces page-fault noise. Remedy: initialize the CUDA context first, then lock. (P5) Silent CPU-EP fallback. Symptom: none — runs complete and write plausible latency distributions. Mechanism: in our configuration, onnxruntime sessions completed on the CPU execution provider, without error, when the CUDA provider failed to initialize. Detection: only a magnitude check against known-GPU latencies, after the data already exists.

p50 latency / p50 at 3199 MHz

and 4 threads. A thermal gate held the start of every cell until the GPU read below 55 ◦ C, so cells begin from comparable thermal state. The contention adversary is an IsolBench-style streamingwrite benchmark [13]: threads sequentially writing large arrays — the access pattern known to cause worst-case interference on multicore platforms, used as the contention adversary on an integrated CPU–GPU SoC [14] — on cores disjoint from the harness core, self-reporting 19–25 GB/s depending on the EMC point. We adopted it after a stress-ng memory stressor moved the victim’s median by only ∼0.3%; a co-runner that saturates no shared resource tests nothing.

101

100 204

665.6

2133 3199

EMC frequency (MHz, locked; CPU/GPU pinned) GEMM 20483 Qwen2.5-1.5B decode

decode proxy (GEMV) ViT-Small

MobileNetV2 GEMM L2-resident

Fig. 1. Median latency vs. locked EMC frequency (normalized to 3199 MHz; CPU/GPU pinned; 1k iterations/cell). Every workload, including the nearzero-DRAM L2-resident GEMM, slows several-fold at low memory clocks; the L2-resident GEMM is faster at 2133 than at 3199 MHz.

GEMM L2-resident -9.1% +11.7%

MobileNetV2 Qwen2.5-1.5B decode

+15.3%

GEMM 20483

+18.1%

ViT-Small

+18.0% +44.9%

decode proxy (GEMV) 10

0

10

20

30

40

50

p50 shift, EMC 3199 2133 MHz (%)

Fig. 2. Workload-dependence of the EMC effect in the realistic upper range (3199→2133 MHz): +45% for bandwidth-bound decode to −9.1% (faster at the lower clock) for L2-resident compute. A frequency-independent constant cannot represent this term.

Remedy: query the session’s resolved providers at startup and hard-fail on any mismatch with the request. P1 and P3 apply to any Linux real-time measurement loop, P2 and P4 to the Tegra family, and P5 to any onnxruntime deployment. IV. RQ1: T HE M EMORY-C LOCK A XIS Figure 1 shows median latency for all six workloads at the four lockable EMC operating points. We pinned the CPU and GPU clocks, locked each EMC point through the BPMP interface with readback verification (§III), and ran 1k iterations per cell. At the bottom of the range the effect is universal: stepping down from 665.6 to 204 MHz raises median latency by +163% (MobileNetV2) to +224% (GEMV decode proxy, L2-resident GEMM) — roughly a 3× inflation on this sin-

gle segment for every workload we measured, regardless of arithmetic intensity. The most instructive curve is the one that should have been flat. The L2-resident GEMM is constructed so that its weights are served from the GPU’s L2 with near-zero DRAM streaming (§III), yet at 204 MHz it runs 8.3× slower than at its fastest operating point, while tegrastats readback across the run shows the GPU clock held at 1016–1018 MHz. The slowdown is therefore not a hidden GPU-clock excursion, and with near-zero DRAM traffic it cannot be a bandwidth shortfall in the usual sense: the EMC domain evidently gates resources beyond DRAM bandwidth — consistent with the interconnect fabric or memory-command path scaling with the EMC clock. Whatever the mechanism, the working assumption that a compute-bound kernel is immune to the memory clock is false on this SoC. The 204 MHz floor is, however, a point a deadline-driven governor would rarely select; the decision-relevant range is the upper pair, 2133 → 3199 MHz. Here the effect becomes strongly workload-dependent (Fig. 2): lowering the clock from 3199 to 2133 MHz shifts the median by +45% for the GEMV decode proxy, +18% for ViT-Small and the GEMM proxy, +12% for MobileNetV2, and +15.3% for the real SLM (Qwen2.5-1.5B) — a spectrum spanning +11% to +48% across our campaign and pilot runs. The real SLM sits well below its GEMV anchor because Q4 K M dequantization adds per-weight compute that dilutes the memory-boundedness of batch-1 decode [25], [26]. These shifts are resolved far above measurement noise: at 3199 MHz the p99/p50 ratio is at most 1.019 in every cell. Then there is the inversion. The L2-resident GEMM is 9.1% faster at 2133 MHz than at 3199 MHz (16.91 vs. 18.60 ms median). We reproduced this three times independently: in the initial pilot (−9.9%), in a cold re-run started at 48 ◦ C with the GPU clock verified at 1016–1018 MHz throughout, and in the full campaign (−9.1%). It is thus neither a thermal artifact nor a GPU-clock excursion. It is, however, specific to the top GPU clock: replicated under the 25 W profile (GPU pinned at 918 MHz, a different firmware clock table), the same workload shows +6.3% — normal ordering — while the rest of the curve replicates (mobilenet +10.4%, proxy +32.7%, and a near-identical low-end collapse). The inversion is an interaction between the GPU and memory operating points, not a property of the EMC axis alone — consistent with §IV-A, where the GPU-frequency model’s cproxyv2 error shrinks at lower GPU clocks. We do not know the mechanism; plausible candidates include EMC-point-dependent memory-controller timing or arbitration settings and clock-domain-crossing costs between the GPU and the fabric, and we bound our claim to the reproducible observation. For estimation purposes the observation suffices: on this platform, latency is not monotone in the memory clock. Three consequences follow for frequency-aware estimators. First, the constant b that absorbs “memory transfer delays” [1] is in reality b(femc , w): a term that moves the median by +45% for one workload and −9.1% for another, with CPU

TABLE II EMC BEHAVIOR UNDER THE STOCK CONFIGURATION . “O BSERVED ” IS THE EMC CLOCK DURING 30–90 S RUNS WITH CPU/GPU PINNED , SAMPLED AT 10 H Z ; NO TRANSITION WAS OBSERVED IN ANY TRACE . N OTE THE OPERATIVE POINT DIFFERS ACROSS PROFILES (2133 UNDER MAXN SUPER, 3199 UNDER 25 W) IN A WAY UNRELATED TO WORKLOAD DEMAND .

nvpmodel profile

EMC config

observed under load

trans.

15 W 25 W MAXN SUPER

max 2133 MHz max 3199 MHz uncapped

2133 MHz (all 4 traces) 3199 MHz in all four traces 2133 MHz (all 4 traces)

0 0 0

and GPU clocks fixed, is neither frequency-independent nor workload-independent. Second, frequency-search procedures that assume latency is non-increasing in clock frequency are unsafe across the inversion: a governor that raises the memory clock to buy deadline margin can lose margin instead — and because the inversion appears only at the top GPU clock, even per-axis monotonicity depends on the other axis’s operating point, so safe search needs measured curves rather than assumed shapes. Third, the fix is concretely cheap: on this board the EMC exposes four lockable points, so adding the axis multiplies profiling cost by four, not by a continuum. A. What the omission costs an estimator A natural defense of the CPU×GPU formulation is that if the EMC is fixed during both profiling and deployment, b is simply a constant of that configuration. The EMC is indeed not governed dynamically here: under the stock governor with CPU and GPU pinned, we observe zero EMC transitions across idle, CNN, GEMV, and SLM workloads — the clock sits at 2133 MHz throughout, even at 45% EMC utilization under SLM decode in the uncapped power mode — counterintuitively below the 3199 MHz the capped 25 W profile pins (Table II); we report the observation without claiming to explain the governor’s policy. But which constant it is belongs to the platform, not the model (Table II): the same binary deployed under the 15 W and 25 W nvpmodel profiles sees 2133 and 3199 MHz memory clocks respectively. A CPU×GPU estimator has no input that distinguishes the two deployments; our 2133↔3199 comparisons quantify exactly this profile pair. To measure the cost directly, we fit T (fgpu ) = k/fgpu + b per workload on an eight-point GPU devfreq sweep (306– 1020 MHz, 300 iterations per cell, CPU pinned) at EMC 3199 MHz, then evaluated the fitted model at the other lockable points (Table III). This is deliberately the GPU-frequency slice of the model family, not a replication of the state of the art’s coupled CPU–GPU formulation [1]: it is a lowerbound demonstration that even the simplest slice, evaluated only at frequencies it was not profiled at on a single other axis, degrades as follows — a richer CPU×GPU static model has no frequency input that distinguishes the two deployments; only a runtime feedback path (such as online drift calibration [1]) could partially compensate, a point we quantify in §VII. In scope, the form is excellent — median residual 0.3–2.5%

MobileNetV2 ViT-Small decode proxy GEMM L2-resident

A: GPU-only

B: +m/femc

C: 2-cell refit

5.3 / 12.8 5.1 / 17.6 3.4 / 32.2 1.5 / 2.8

5.4 / 7.1 10.5 / 14.1 19.9 / 30.9 17.8 / 34.5

2.4 / 3.4 4.2 / 6.7 9.2 / 20.9 2.5 / 4.8

— consistent with the accuracy this model class reports on its own terms [1]. One lockable point away, at 2133 MHz, the median error grows to 3.4–5.3% with maxima of 12.8– 32.2%; at 665.6 MHz the median is 38–69%. The signed error is negative for every workload: the estimator underestimates latency, the direction that converts model error into deadline misses. Repairing the model is not as simple as adding a term. A parametric extension T = k/fgpu + m/femc + b, fitted on the 3199 and 665.6 MHz sweeps and interpolated to the held-out 2133 MHz point, is worse than the unrepaired model for three of four workloads (Table III): the EMC response is not 1/f shaped, and for the L2-resident GEMM it is not even monotonic, so no monotone parametric form can represent it. Tabulation works where the parametric form fails: refitting (k, b) from just two cells measured at the target EMC point bounds the median error to 2.4–4.2% for three of four workloads, and the same repair holds at the second held-out point (665.6 MHz: median 1.8%/1.0%/0.0% for MobileNetV2/ViT/proxy, 8.1% for the L2-resident GEMM). The bandwidth-bound GEMV proxy resists even this (9.2%/20.9%): its GPU-frequency dependence itself changes with the memory clock — the terms interact — so it needs the full per-point sweep. The practical prescription is a per-lockable-point table, T = k(femc )/fgpu + b(femc , w): at four lockable points, a 4× profiling multiplier, and the same profiling pass yields the tail margins of §V. V. RQ2: TAILS AND B URSTS UNDER C ONTENTION To measure tail behavior under contention, we ran 100,000 timed cycles in each of eight cells spanning two workloads (MobileNetV2 and the GEMV decode proxy), two locked EMC frequencies (2133 and 3199 MHz), and the streamingwrite adversary of §III on zero, two, or four cores (18.9– 24.6 GB/s self-reported). The locked-clock distributions are knife-edge: p99.99 exceeds the median by at most 10% (p99.99/p50 ≤ 1.10) in all eight cells. Post-hoc deadline-miss curves consequently fall off a cliff roughly 1 ms wide: with the adversary active at 2133 MHz, MobileNetV2 misses every cycle at a 5.0 ms deadline, 0.25–0.26% of cycles at 5.5 ms, and at most 0.002% at 6.0 ms (Fig. 3). Contention’s dominant effect is to move this cliff, not to fatten the tail: the twocore adversary shifts the MobileNetV2 median at 2133 MHz by +13% (4.70→5.30 ms), while p99.99/p50 grows only from 1.087 to 1.099. For a deadline governor the consequence is stark — near the cliff, a few hundred microseconds of

miss rate (%), 100k cycles

workload

MobileNetV2 @ 50 Hz: the miss cliff

102 101 100 10 1 10 2 10 3

4.0

4.5

5.0

5.5

6.0

6.5

deadline (ms)

EMC 2133, 0 adv EMC 2133, 2 adv

EMC 2133, 4 adv EMC 3199, 0 adv

7.0

EMC 3199, 2 adv EMC 3199, 4 adv

Fig. 3. Post-hoc deadline-miss curves (100k cycles/cell). Locked-clock distributions are knife-edge: the transition from always-miss to never-miss spans ∼1 ms. Contention shifts the cliff location (+13% median) more than it fattens the tail.

MobileNetV2, EMC 2133 + adversary, deadline = p99.9 number of bursts

TABLE III M EDIAN / MAX RELATIVE P 50 ERROR (%) ON THE HELD - OUT EMC 2133 MH Z SWEEP. A: k/fgpu +b FITTED AT 3199 MH Z ; B: +m/femc FITTED AT {3199, 665.6}; C: (k, b) REFITTED FROM TWO CELLS AT 2133 MH Z .

observed independent misses (geometric)

101 100 10 1 10 2 0

2

4

6

8

10

12

14

burst length (consecutive misses)

16

Fig. 4. Miss-burst lengths at a p99.9-tight deadline vs. the geometric distribution that independent misses would produce. Observed bursts reach 16 consecutive misses; continuation probability is 0.74 vs. 0.001 under independence.

estimation bias separates 100% QoS from total failure, so the aggregate miss rate is maximally sensitive exactly where the estimator must operate. Aggregate rates also hide when misses happen. Because the harness records full per-cycle distributions, the deadline is a post-hoc analysis parameter; we anchored it at each cell’s empirical p99.9, fixing the aggregate miss rate at 0.1% (100 misses in 100k cycles), and examined the miss sequence. If misses were independent, the probability that the cycle immediately after a miss also misses (the continuation probability) would be 0.001, and bursts would be geometric with mean

GPD prediction from first 50k (ms)

Out-of-sample tail validation, all 8 cells p99.9 p99.99 ±6% band

9 8 7 6 5 4 4

5

6

7

8

observed quantile, last 50k cycles (ms)

9

Fig. 5. Out-of-sample tail validation: GPD fitted on the first 50,000 cycles of each Part B cell predicts the last 50,000 cycles’ p99.9 and p99.99 within the ±6% band, across all eight cells (16 predictions). The in-sample survival illustration is in Appendix C.

length ≈1.001. Instead, the continuation probability reaches 0.74 — 740× the independent baseline — in the worst cell (MobileNetV2, 2133 MHz, two-core adversary), where the 100 misses collapse into 26 bursts of mean length 3.85 and maximum length 16 (Fig. 4). Across the six MobileNetV2 cells the clustering ratio spans 40–740×; the proxy cells sit at 10–50×. Notably, contention is not required: the uncontended MobileNetV2 cell at 2133 MHz clusters at 540× (continuation probability 0.54), and clustering is not monotone in adversary count (740× at two cores, 340× at four) — the adversary moves the cliff’s location more than it creates the bursts. Each cell is a single 100k-cycle run, so burst statistics carry no run-to-run replication; the same qualitative signature does, however, appear in an independent 1.6M-cycle dataset collected on this board three days earlier with a different stressor suite (Appendix A). Burst arrivals are themselves clustered at 2133 MHz (inter-burst-interval CV up to 2.6, in three of five cells) but consistent with a Poisson process at 3199 MHz (CV 0.88–1.08): the lower memory clock makes misses both more clustered within bursts and more episodic in their arrival. The workloads order opposite to what bandwidth sensitivity predicts. The GEMV proxy has the largest median EMC sensitivity of any workload (+45% across 3199→2133 MHz, §IV), so one might expect it to suffer the worst contention tails. The opposite holds. The proxy’s tails are short in every threshold-robust measure: its continuation probability never exceeds 0.05, its longest burst is 3 cycles, and its extrapolated p99.99 sits within 3% of its median. MobileNetV2 — the least EMC-sensitive of the real models at the median (+12%) — shows the heaviest clustering and the widest extrapolated tails. (We deliberately rest these claims on quantiles and burst statistics rather than on the GPD shape parameter: ξ point estimates are not threshold-stable at these sample sizes —

the contended proxy’s ξ flips sign between fitting thresholds — which is why our margins are quantile predictions, not shape claims.) On this platform, then, the axis that moved the median was not the axis that broke the tail: median frequency sensitivity — what frequency-aware estimators measure — failed to predict tail risk in this two-workload contrast; whether the anticorrelation generalizes is an open question for more workloads. We looked for a board-level mechanism and report a bounded negative result. Joining per-cycle timestamps against tegrastats telemetry at 500 ms resolution, we compared burst windows (bursts of length ≥ 2; 4–57 joined samples per cell) against each cell’s baseline. EMC busy fraction, GPU temperature, and RAM occupancy are indistinguishable during bursts (e.g., 37.0% vs. 37.0% EMC busy and 54.3 vs. 54.2 ◦ C in the worst cell). Whatever produces multi-cycle miss bursts operates below 500 ms resolution — consistent with GPUor driver-level events rather than thermal or memory-pressure drift — and we do not identify it here. In this dataset, GPD margins are substantially better calibrated than Gaussian margins, and the comparison survives out-of-sample validation; we treat the GPD fits as marginal quantile heuristics, not iid pWCET guarantees. A GPD fitted to the exceedances above p99 of the first 50,000 cycles of each run predicts the last 50,000 cycles’ p99.9 within ±2.7% and p99.99 within ±5.7% across all eight cells (Fig. 5); the predicted quantile is stable to the threshold choice — fitting at p98.5, p99, or p99.5 moves it by less than 2% — even where the shape estimate itself is not, and we therefore treat the extrapolated quantile, not ξ, as the estimator’s deliverable. Because exceedances cluster (the burst structure above), we use the GPD as a marginal quantile heuristic, not as iidEVT inference: our ordinary-bootstrap CIs (∼1% half-width) ignore the dependence and indeed cover the empirical p99.99 in only four of eight cells — a concrete coverage failure, not a hypothetical one — while the empirical p99.99 of a 100kcycle run is itself only ∼10 order statistics. As a dependence check, runs-declustering (refitting on cluster maxima) moves the predicted p99.99 by at most 5% across gap parameters 1–10 — most in the most-clustered cell — within the outof-sample error band above. The deployment consequence is direct: setting a deadline from the first half of each run to target a 0.1% miss rate, the Gaussian µ+3σ margin achieves 1.3–2.9% on the second half — 13–29× its design target — in every cell, while the GPD margin achieves 0.016– 0.20%, staying within ∼2× the target (worst 2.02×), with some cells conservative by up to 6× — the failure mode shifts from missed deadlines to conservatism. A plain empirical p99.9 from the same profiling window is competitive in most cells but reaches 4.4× the target in its worst cell, where the GPD margin stays at 1.8×: the parametric tail buys worstcase boundedness rather than average accuracy. The tail is also compute-side, not scheduler-side: release jitter at p99.9 is ∼23–37 µs across cells, against compute times of 4.3– 8.9 ms, so neither the periodic harness nor CPU scheduling explains the bursts. Two prescriptions follow. Quantile margins

actuation lag (settle p50)

workload stall (raw p95)

100 µs kill line

EMC GPU CPU 10 3

10 2

10 1

ms (log)

100

101

Fig. 6. Per-domain transition costs. Workload-observed stalls stay below the pre-specified 100 µs relevance line (raw p95 ≤92 µs, against matched probe-noise floors of up to 84 µs), but the new frequency takes 1/5/8 ms (CPU/GPU/EMC) to take effect; the EMC’s firmware bookkeeping lags a further ∼13 ms.

for deadline governors should come from EVT fits rather than µ+kσ assumptions — measurement-based pWCET practice [22] applied at the governor level — and QoS reporting, which has already moved from means to percentiles [23], [24], should pair the aggregate rate with the miss-pattern statistics that weakly-hard analysis consumes [18], [19], e.g. the continuation probability or empirical ⟨m, k⟩ violations: a 0.1% miss rate delivered in bursts of sixteen consecutive misses presents a different failure mode to a downstream consumer than 0.1% delivered independently. VI. RQ3: F REQUENCY ACTUATION L AG We pre-specified a demotion criterion for this experiment before running it: per-domain transition stalls would be reported as a headline result only if the workload-observed stall exceeded 100 µs at p95 on some domain. The criterion fired against us, and we report the stalls as the null result they are. We measured two frequency pairs per domain (CPU 1728 ↔ {115.2, 729.6} MHz, GPU 1020 ↔ {306, 612} MHz, EMC 3199 ↔ {2133, 665.6} MHz), 150 (CPU) or 200 (GPU, EMC) transitions per direction, using cycle-counted probes that timestamp fixed work quanta on CLOCK_MONOTONIC around the frequency write: a dependent-multiply chain on the CPU, a clock64 spin kernel on the GPU, and DRAMstreaming chunks for the EMC. The worst raw stall at p95 is 49 µs on the CPU, 92 µs on the GPU, and 24 µs on the EMC (Fig. 6); matched no-transition control windows put the probes’ own noise floors at 10–84 µs, so the GPU’s 92 µs sits barely above its 84 µs floor. (We report raw quantiles with the noise floor alongside rather than their difference, which is not itself a quantile.) All domains fall below the 100 µs criterion, consistent in magnitude with the tens of microseconds (roughly 10–70 µs) measured for Intel CPU

frequency transitions [7]. Rewriting a clock does not stall a running workload on this board in any way that matters at millisecond deadlines. What the same traces do show is actuation lag. The probe’s per-quantum rate settles at the new operating point about 1 ms after the write on the CPU (settle p50 0.99–1.40, p95 ≤1.65 ms across pairs and directions), 4.8–5.3 ms (p95 ≤6.3) on the GPU, and 7.5–8.7 ms (p95 ≤9.7) on the EMC — the lag distribution is itself tight, so the 1/5/8 ms framing carries to the tail. During this window the workload does not stall— it simply keeps executing at the old frequency. Switching on this SoC is therefore nearly free in the stall sense, but not in the control sense studied on discrete GPUs [6]: a governor’s decision takes roughly 1/5/8 ms (CPU/GPU/EMC) to become true. One methodological finding deserves emphasis. The BPMP debugfs rate readback does not report the target EMC frequency until 12.9–13.8 ms (median) after the write—roughly 5 ms after the workload demonstrably runs at the new rate. The readback is firmware bookkeeping, not hardware state; we treat the pto_counter readout as ground truth for the EMC rate — its value (e.g., 3,191,887,872 against a requested 3,199,000,000) reflects a measured clock rather than the request table. The same caution applies on every domain: cpufreq, devfreq, and BPMP readback files are driver or firmware state, useful for verifying that a lock took hold, and should never be cited as a measurement of when—or whether—the hardware transitioned. VII. T RACE -D RIVEN G OVERNOR E VALUATION To connect the estimator errors of §IV-A to governor decisions, we replay the measured sweep through a deadline governor in simulation — an illustrative decision replay, not a deployed governor. The scenario is the power-mode pair of Table II: the policy is profiled at the 25 W memory point (3199 MHz) and deployed at the 15 W point (2133 MHz). Each policy is a (point model, margin model) pair: # k,b: blind=fit @3199; aware=2-cell @2133 # r: margin ratio q/median from profiling # (Gauss = mu+3sigma; GPD = p99.9) for f in GPU freqs, ascending: T = k/f + b M = T * (r - 1) if T + M <= D: return f return f_max # infeasible: run at max

This is a decision-level trace replay, not a deployed governor: no online adaptation runs, and no power is measured. The deadline grid spans 60 points from 0.95× the fastest median to 1.3× the slowest p99.9 at the deployment point, per workload; the grid, candidate frequencies, and scoring are emitted verbatim by the released simulation script. Each choice is scored against the measured 300-cycle distribution of the chosen (fgpu , 2133) cell: a violation is an achieved miss rate above the 1/300 resolution floor. At this granularity the simulation evaluates point-model feasibility near the deadline cliff, not calibrated 10−3 tail behavior (§V covers that regime).

TABLE IV G OVERNOR SIMULATION ACROSS THE DEADLINE GRID (G AUSSIAN - MARGIN VARIANTS SHOWN ; GPD VARIANTS DIFFER BY ≤4 PERCENTAGE POINTS OF VIOLATION RATE ). VIOL % IS OVER THE FULL GRID , [ BRACKETS ] OVER FEASIBLE DEADLINES ONLY; “ MISS @ VIOL” IS THE MEAN ACHIEVED MISS RATE ON VIOLATED DEADLINES ; ∆f¯ THE MEAN SELECTED - FREQUENCY INCREASE OF THE AWARE POLICY ( NO POWER MEASURED ; FREQUENCY IS A NOMINAL PROXY ). C URVES FOR ALL FOUR WORKLOADS : A PPENDIX C. aware (2-cell @2133)

workload

viol%

miss@viol

viol%

miss@viol

∆f¯

MobileNetV2 ViT-Small decode proxy GEMM L2-resident

25 [22] 22 [18] 7 [2] 8 [5]

69% 81% 55% 80%

3 [0] 5 [0] 5 [0] 3 [0]

100% 68% 67% 100%

+7% +9% +14% +2%

achieved miss rate (%) at EMC 2133

blind (fit @3199)

102

MobileNetV2: governor outcomes, profiled @3199

101

VIII. I MPLICATIONS FOR E STIMATORS AND G OVERNORS

100

1/300 floor

10 1 10 2

14% higher mean selected frequency for this — a feasibility price; we measure no power, so selected frequency stands in for energy only nominally — and when an aware violation does occur it crashes equally hard, because the cliff punishes every policy alike. Third, the margin model is second-order here: Gaussian and GPD variants differ by at most a few points of violation rate, because at 300-cycle resolution the cliff dominates the within-cell tail. The margin distinction matters in the 10−3 –10−4 regime quantified in §V. Finally, deployed estimators are not fully static: the state of the art ships a latency-level drift calibrator [1] whose converged effect resembles the C1 offset repair of §IV-A — recovering MobileNetV2 and the L2-resident GEMM (4.2%/2.0% median at the held-out point) but leaving ViT and the decode proxy at 6.7–17.2%, because an offset cannot repair a slope that changes with the memory clock. Online calibration narrows the blind policy’s gap; it does not close it, and it spends its convergence window (∼100–300 iterations) missing.

4

6

8

10

deadline (ms)

blind + Gauss blind + GPD

12

14

aware + Gauss aware + GPD

Fig. 7. Achieved miss rate vs. deadline for MobileNetV2 under the four policies (deployment at EMC 2133 MHz). Each red spike is the blind policy stepping down to a lower frequency as soon as its biased model predicts feasibility — and landing on the wrong side of that frequency’s miss cliff. The dotted line is the 1/300 resolution floor.

Three observations (Table IV, Fig. 7). First, the blind policy violates feasibility on 7–25% of the deadline grid, and when it violates it does not degrade gracefully: the mean achieved miss rate on violated deadlines is 55–100% in this replay — an illustration of feasibility failure near the §V cliff, where a few hundred microseconds of point-model bias places the workload on the wrong side. (At the 300-cycle resolution of these cells this is a decision-replay illustration, not a calibrated tail measurement.) The grid deliberately includes infeasible deadlines; conditioning on feasible ones (those the maximum-frequency cell meets at the 1/300 floor) sharpens the contrast: the aware policy’s violations vanish — 0% on all four workloads, every residual violation having sat on an infeasible deadline — while the blind policy still violates 2–22%. Second, the two-cell EMC-aware refit pays a 2–

The memory axis is bounded to add — as a table, not a term. RQ1 implies a structural fix: T = k(femc )/fgpu + b(femc , w), with both coefficients indexed by the memory point (§IV-A). The indexing must be tabular: the measured parametric alternative (+m/femc ) is worse than no repair for three of four workloads, while a two-cell-per-point refit repairs three of four — cheap, but not always two-cell cheap: the bandwidth-bound decode proxy needs the full perpoint sweep (Table III). The cost is bounded by the action space: this board exposes exactly four lockable EMC points ({204, 665.6, 2133, 3199} MHz), so the full repair multiplies profiling cost by four, not by a continuum. For pipelines whose profiling is already reduced to minutes [1], a 4× factor is a bounded cost — though the required per-point profiling density remains workload-dependent (§IV-A); the lockable count is board-specific but discoverable with the pre-flight gate of §III. Monotonicity is not a safe search assumption. The −9.1% inversion means “higher frequency, lower latency” fails on the EMC axis. A governor that reacts to a predicted deadline violation by raising the memory clock—the natural hill-climbing move—makes the L2-resident workload slower, stepping away from the feasible operating point. Frequency search should run over measured per-workload curves rather than assumed shapes; with four points per workload, measuring the curve costs little more than assuming it. Aggregate QoS hides the failure mode. Two systems with identical 0.1% miss rates can differ enormously in burst behavior: at that rate our cells span continuation probabilities from 0.01 to 0.74 (vs. 0.001 under independence), with bursts up to 16 consecutive misses. Real-time GPU and inference schedulers report aggregate miss rates [11], [12]; governor evaluations likewise report aggregate attainment — latencyQoS percentages [1] or average achieved frame rates [5]; but the weakly-hard literature establishes that miss patterns, not rates, determine whether a control loop survives [17], [18],

[19]. Our measurements supply the missing empirical layer: at a deadline with a 0.1% rate, this platform delivers runs of up to 16 consecutive misses — violating, e.g., any ⟨m, k⟩ constraint with k≤16 windows that a rate-based analysis would certify with ease. We recommend reporting the aggregate rate together with P [miss | miss] or empirical ⟨m, k⟩ violation counts, in both evaluations and governor objectives. Margins should come from extreme-value fits. The textbook Gaussian margin µ + 3σ — which we construct as the natural first-order baseline, not a documented practice of any cited system — fails to cover even p99.9 in all eight of our 100k-cycle cells, because locked-clock distributions are knifeedge yet sometimes heavy-tailed (ξ up to 0.50). A generalized Pareto fit over the ∼500 exceedances above p99 of a 50k-cycle profiling window predicts the out-of-sample p99.99 within 6% and hits a 0.1% miss target within 2×, where the Gaussian margin overshoots it 13–29×—a tail margin obtainable from a standard profiling pass at no extra measurement cost. Such margins complement, rather than duplicate, existing online adaptation: FLAME’s drift calibration (sliding-window bias with EWMA smoothing) corrects the mean level of the latency estimate [1]; it offers no quantile protection. End-to-end, the point model dominates the failure mode (§VII): the EMC-blind policy breaks its feasibility promise on 7–25% of the deadline grid (2–22% of feasible deadlines) and crashes when it does; the two-cell repair’s violations on feasible deadlines are zero. Actuation lag bounds governor reaction time. A perinference governor cannot act faster than the platform actuates: about 1 ms (CPU), 5 ms (GPU), and 8 ms (EMC) on this board (§VI). At inference periods of 10–30 ms—the range four of our six workloads occupy at the top clocks (medians ∼11–24 ms) and typical of deadline sweeps in governor evaluations (e.g., 12–20 ms for ResNet50 [1])—an EMC decision takes effect a large fraction of one period after it is made; the inference it was meant to protect may complete under the old clock. Governors must either actuate ahead of need or carry deadline margins that absorb the lag, and the margin must be largest exactly where RQ1 shows the knob matters most: on this platform the memory clock is at once the most impactful and the least agile axis. IX. L IMITATIONS Single platform. Every measurement in this paper came from one Jetson Orin Nano Super. Within the board we replicated the RQ1 curve under a second firmware clock configuration (the 25 W nvpmodel profile: different CPU/GPU caps and voltage tables), which reproduced the curve shape and bounded the inversion to the top GPU clock (§IV); crossboard and cross-SKU replication remain open. Our claims are existence proofs: on this board the phenomena exist and are large. Cross-SKU validation on an additional Orin-family platform (Jetson Orin NX 16 GB) is ongoing. The magnitudes — the +11–48% EMC shifts, the −9.1% inversion, the 1/5/8 ms actuation lags — are specific to this SKU, its LPDDR5 configuration, and this software release, and we

make no claim that they transfer. Because the state-of-the-art estimator was validated on the AGX Orin and Orin NX [1], our study is a same-family complement to its published results, not a replication; we release the harness so that other SKUs can be profiled directly. Unidentified mechanisms. We characterized the 2133vs-3199 MHz inversion and the miss-burst process without identifying their causes. The inversion reproduced across three independent runs with the GPU clock held and thermal state controlled, which excludes the obvious confounds but names no mechanism. For bursts, board-state telemetry shows no correlated movement in EMC utilization, GPU temperature, or memory footprint at its 500 ms resolution, bounding the responsible process below that timescale — consistent with a GPU- or driver-level effect — but not isolating it. Percycle hardware counters (the EMC activity monitor, PTO clock counters) are the natural next instruments and are future work. Software stack. The DNN workloads ran on the ONNX Runtime CUDA execution provider in fp32 and the SLM through llama.cpp; TensorRT engines [33] or other runtimes schedule memory traffic differently and may shift the magnitudes we report. Quantized SLM. The language-model workload is Qwen2.5-1.5B in Q4 K M quantization, whose per-byte dequantization compute dilutes pure bandwidth sensitivity. We quantified the dilution — the SLM shifts by +15.3% over 3199→2133 MHz where the fp16 GEMV decode proxy shifts by +45% — but higher-precision deployments, which sit closer to the proxy on the roofline, were not measured. One adversary class. Contention was generated by a single adversary type: IsolBench-style streaming writes from CPU cores [13]. Moving from two to four adversary cores leaves its self-reported bandwidth essentially unchanged (19.7 GB/s at 2133 MHz, 24.5–24.6 GB/s at 3199 MHz), indicating the adversary itself saturates near 19–25 GB/s; stronger or differently shaped interference — GPU co-runners, read-dominated or random-access patterns — remains unexplored. X. C ONCLUSION On one integrated edge SoC, the three modeling assumptions we tested are each incomplete for deployed deadline control: valid within the CPU×GPU profiling scope, they omit state that deployment depends on, by margins that matter. The memory clock, absorbed into a frequency-independent constant by current estimators [1], moves median latency by +11–48% across its realistic upper range — and, for a synthetic L2-resident kernel at the top GPU clock, in the wrong direction (−9.1%). The aggregate miss rates that governor evaluations report conceal temporal structure: at a deadline missed 0.1% of the time, the cycle after a miss also misses with probability up to 0.74. And a commanded frequency takes 1/5/8 ms (CPU/GPU/EMC) to take effect — a large fraction of one period for per-inference governors. An estimator over CPU×GPU frequencies alone is therefore calibrated to the mean of a distribution whose location, shape, and actuation all depend on state it does not see. The repair is bounded

10 4 10 5 10 6

5.4

5.5

5.6

5.7

5.8

5.9

response time (ms)

6.0

6.1

GPD above p99 ( =0.39)

empirical survival Gaussian ( +3 =5.41 ms)

Fig. 8. In-sample illustration of the heaviest cell’s tail: the Gaussian survival collapses where the empirical tail extends; the GPD tracks it. The quantitative (out-of-sample) validation is Fig. 5. MobileNetV2

102 101

101

100 10 1 10 2

100 10 1

4

6

8

10 12 deadline (ms)

10 2

14

decode proxy (GEMV)

102

15

20

25 30 deadline (ms)

35

101

100 10 1

100 10 1

6

8

10

12 14 deadline (ms) blind+G

16

18 blind+P

10 2 aware+G

20

30 40 deadline (ms) aware+P

50

A PPENDIX B W HAT IS RULED OUT

Fig. 9. Governor trace replay (§VII) for all four workloads.

The paper reports three phenomena without a mechanism. The following causes are excluded by measurement; the remainder is genuinely open.

A PPENDIX C S UPPLEMENTARY FIGURES

phenomenon

ruled out (evidence)

still open

inversion (§IV)

thermal (cold re-run, 48◦ C) GPU-clock excursion (held 1016–1018 MHz) DRAM-traffic shift (EMC busy% identical) single-run noise (3× reproduced)

MC arbitration, fabric clock ratios

miss bursts (§V)

harness backlog (max resp. < period) release jitter (p99.9 ∼30 µs) RT throttling (disabled; §III-D) logging I/O (no I/O in timed loop) CPU-EP fallback (hard-fail guard) board state at 500 ms (EMC%/temp/RAM flat)

sub-500 ms GPU/ driver events

EMC pinning (Table II)

workload demand (45% busy, no upclock)

BPMP/actmon policy internals

40

GEMM L2-resident

102

101

10 2

ViT-Small

102

miss %

Three days before the main campaign, the same board ran a 16-cell campaign (100k cycles each, 1.6M total) with a different runner configuration and a stress-ng-based stressor suite instead of the streaming-write adversary. Computing the same continuation statistic at p99.9-anchored deadlines reproduces the signature: the memory-pressured cells cluster (P [miss | miss] = 0.40 under the memory stressor, 0.19 and 0.14 under combined memory/VM stressors, 0.11 under an IRQ storm — 110–400× independence), while baseline, CPU-only, and cache-only cells sit at 0.00–0.07. Clustering under memory pressure and near-independence without it replicate across the two campaigns’ different stressors, dates, and runner configurations. Raw data ships in the repository (results/p3*).

10 3

miss %

A PPENDIX A I NDEPENDENT BURST REPLICATION

P(response > x)

All raw data (32 campaign cells totaling 822,000 timed cycles — Part A 20×1000 inference cycles and 4×500 decode tokens, Part B 8×100,000 cycles — plus the estimatorevaluation sweep (96×300 cycles, §IV-A), the governor traces, and all pilot runs), the harness, clock control, adversary, probes, and analysis scripts are available at https://github.com/ dankang21/jetson-latency-lab; bulk probe traces that exceed repository limits are archived separately on Zenodo (https: //doi.org/10.5281/zenodo.20694688). Every external claim in this paper was verified against its source text during drafting.

10 2

miss %

R EPRODUCIBILITY

MobileNetV2, EMC 2133 + adversary: tail fits

miss %

on this board: the EMC axis adds only four lockable points to profiling, out-of-sample generalized Pareto margins stay within ∼2× a 0.1% miss target (worst 2.02×) where Gaussian margins overshoot it 13–29×, and burst hazard is one number reported next to the aggregate rate. We release the harness and all raw data: 822,000 timed cycles across the 32 campaign cells, plus all pilot runs.

60

R EFERENCES [1] J. Chen, J. You, Z. Liu, and Z. Li, “Taming asynchronous CPU-GPU coupling for frequency-aware latency estimation on mobile edge,” 2026, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2604.15357 [2] Y. Han, Z. Nan, S. Zhou, and Z. Niu, “DVFS-aware DNN inference on GPUs: Latency modeling and performance analysis,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2502.06295 [3] NVIDIA Corporation, “NVIDIA Jetson Linux Developer Guide (Release 36.5): Platform Power and Performance — Jetson Orin Nano Series, Jetson Orin NX Series and Jetson AGX Orin Series,” https://docs.nvidia.com/jetson/ archives/r36.5/DeveloperGuide/SD/PlatformPowerAndPerformance/ JetsonOrinNanoSeriesJetsonOrinNxSeriesAndJetsonAgxOrinSeries. html, 2026, accessed 2026-06-11. [Online]. Available: https://docs.nvidia.com/jetson/archives/

r36.5/DeveloperGuide/SD/PlatformPowerAndPerformance/ JetsonOrinNanoSeriesJetsonOrinNxSeriesAndJetsonAgxOrinSeries. html [4] Y. Han, Z. Nan, S. Zhou, and Z. Niu, “Joint memory frequency and computing frequency scaling for energy-efficient DNN inference,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/ 2509.17970 [5] S. Kim, K. Bin, S. Ha, K. Lee, and S. Chong, “zTT: Learningbased DVFS with zero thermal throttling for mobile devices,” in Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ’21). Virtual Event, Wisconsin: ACM, 2021, pp. 41–53. [Online]. Available: https://doi.org/10.1145/3458864.3468161 [6] D. Velicka, O. Vysocky, and L. Riha, “Methodology for GPU frequency switching latency measurement,” 2025. [Online]. Available: https://arxiv.org/abs/2502.20075 [7] A. Mazouz, A. Laurent, B. Pradelle, and W. Jalby, “Evaluation of CPU frequency transition latency,” Computer Science – Research and Development, vol. 29, no. 3–4, pp. 187–195, 2014. [Online]. Available: https://doi.org/10.1007/s00450-013-0240-x [8] S. Williams, A. Waterman, and D. A. Patterson, “Roofline: An insightful visual performance model for multicore architectures,” Communications of the ACM, vol. 52, no. 4, pp. 65–76, 2009. [9] A. Dutt, S. P. Rachuri, A. Lobo, N. Shaik, A. Gandhi, and Z. Liu, “Evaluating the energy impact of device parameters for DNN inference on edge,” in Proceedings of the 14th International Green and Sustainable Computing Conference (IGSC ’23). ACM, 2023, pp. 52–55, accessed 2026-06-11. [Online]. Available: https: //doi.org/10.1145/3634769.3634809 [10] Z. Yang, K. Nahrstedt, H. Guo, and Q. Zhou, “DeepRT: A soft real time scheduler for computer vision applications on the edge,” in Proceedings of the 6th ACM/IEEE Symposium on Edge Computing (SEC), 2021, pp. 271–284. [Online]. Available: https://arxiv.org/abs/2105.01803 [11] A. F. Babaei and T. Chantem, “DARIS: An oversubscribed spatiotemporal scheduler for real-time DNN inference on GPUs,” in Proceedings of the 62nd ACM/IEEE Design Automation Conference (DAC). IEEE, 2025, pp. 1–7, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2504.08795 [12] A. Zou, J. Li, C. D. Gill, and X. Zhang, “RTGPU: Real-time GPU scheduling of hard deadline parallel tasks with fine-grain utilization,” IEEE Transactions on Parallel and Distributed Systems, vol. 34, no. 5, pp. 1450–1465, May 2023. [Online]. Available: https://ieeexplore.ieee.org/document/10012550 [13] P. K. Valsan, H. Yun, and F. Farshchi, “Taming non-blocking caches to improve isolation in multicore real-time systems,” in 2016 IEEE RealTime and Embedded Technology and Applications Symposium (RTAS), 2016, pp. 161–172, accessed 2026-06-11. Origin of the IsolBench benchmark suite, including the Bandwidth memory-bandwidth adversary benchmark; code at https://github.com/CSL-KU/IsolBench. [Online]. Available: https://ieeexplore.ieee.org/document/7461361 [14] W. Ali and H. Yun, “Protecting real-time GPU kernels on integrated CPU-GPU SoC platforms,” in 30th Euromicro Conference on Real-Time Systems (ECRTS 2018), ser. Leibniz International Proceedings in Informatics (LIPIcs), vol. 106. Schloss Dagstuhl– Leibniz-Zentrum für Informatik, 2018, pp. 19:1–19:22, accessed 202606-11. [Online]. Available: https://drops.dagstuhl.de/entities/document/ 10.4230/LIPIcs.ECRTS.2018.19 [15] A. Chakraborty, W. Tavernier, A. Kourtis, M. Pickavet, A. Oikonomakis, and D. Colle, “Profiling concurrent vision inference workloads on NVIDIA Jetson – extended,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2508.08430 [16] H. Yun, G. Yao, R. Pellizzoni, M. Caccamo, and L. Sha, “MemGuard: Memory bandwidth reservation system for efficient performance isolation in multi-core platforms,” in Proceedings of the 19th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2013). IEEE, 2013, pp. 55–64. [17] M. Hamdaoui and P. Ramanathan, “A dynamic priority assignment technique for streams with (m,k)-firm deadlines,” IEEE Transactions on Computers, vol. 44, no. 12, pp. 1443–1451, 1995. [18] G. Bernat, A. Burns, and A. Llamosı́, “Weakly hard real-time systems,” IEEE Transactions on Computers, vol. 50, no. 4, pp. 308–321, 2001. [19] M. Maggio, A. Hamann, E. Mayer-John, and D. Ziegenbein, “Controlsystem stability under consecutive deadline misses constraints,” in Proceedings of the 32nd Euromicro Conference on Real-Time Systems

(ECRTS 2020), ser. LIPIcs, vol. 165. Schloss Dagstuhl – LeibnizZentrum für Informatik, 2020, pp. 21:1–21:24. [20] S. Edgar and A. Burns, “Statistical analysis of WCET for scheduling,” in Proceedings of the 22nd IEEE Real-Time Systems Symposium (RTSS 2001). IEEE Computer Society, 2001, pp. 215–224. [21] L. Cucu-Grosjean, L. Santinelli, M. Houston, C. Lo, T. Vardanega, L. Kosmidis, J. Abella, E. Mezzetti, E. Quiñones, and F. J. Cazorla, “Measurement-based probabilistic timing analysis for multi-path programs,” in Proceedings of the 24th Euromicro Conference on Real-Time Systems (ECRTS 2012). IEEE Computer Society, 2012, pp. 91–101. [22] R. I. Davis and L. Cucu-Grosjean, “A survey of probabilistic timing analysis techniques for real-time systems,” Leibniz Transactions on Embedded Systems (LITES), vol. 6, no. 1, pp. 03:1–03:60, 2019. [23] J. Dean and L. A. Barroso, “The tail at scale,” Communications of the ACM, vol. 56, no. 2, pp. 74–80, Feb. 2013. [Online]. Available: https://doi.org/10.1145/2408776.2408794 [24] V. J. Reddi, C. Cheng, D. Kanter, P. Mattson, G. Schmuelling, C.-J. Wu, B. Anderson, M. Breughe, M. Charlebois, W. Chou, R. Chukka, C. Coleman, S. Davis, P. Deng, G. Diamos, J. Duke, D. Fick, J. S. Gardner, I. Hubara, S. Idgunji, T. B. Jablin, J. Jiao, T. St. John, P. Kanwar, D. Lee, J. Liao, A. Lokhmotov, F. Massa, P. Meng, P. Micikevicius, C. Osborne, G. Pekhimenko, A. T. R. Rajan, D. Sequeira, A. Sirasao, F. Sun, H. Tang, M. Thomson, F. Wei, E. Wu, L. Xu, K. Yamada, B. Yu, G. Yuan, A. Zhong, P. Zhang, and Y. Zhou, “MLPerf inference benchmark,” in Proceedings of the ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), 2020, pp. 446–459, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/1911.02549 [25] R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, “Efficiently scaling transformer inference,” in Proceedings of Machine Learning and Systems, D. Song, M. Carbin, and T. Chen, Eds., vol. 5. Curan, 2023, pp. 606–624. [Online]. Available: https://proceedings.mlsys.org/paper files/paper/2023/ hash/c4be71ab8d24cdfb45e3d06dbfca2780-Abstract-mlsys2023.html [26] S. Kim, C. Hooper, T. Wattanawong, M. Kang, R. Yan, H. Genc, G. Dinh, Q. Huang, K. Keutzer, M. W. Mahoney, Y. S. Shao, and A. Gholami, “Full stack optimization of transformer inference: a survey,” 2023, accessed 2026-06-11. [Online]. Available: https: //arxiv.org/abs/2302.14017 [27] Microsoft, “ONNX runtime,” https://github.com/microsoft/onnxruntime, release v1.23.0, CUDA execution provider. Accessed 2026-06-12. [28] M. Sandler, A. G. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018, pp. 4510–4520. [29] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in 9th International Conference on Learning Representations (ICLR 2021), 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy [30] Qwen Team, “Qwen2.5 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2412.15115 [31] G. Gerganov and contributors, “llama.cpp: LLM inference in C/C++,” https://github.com/ggml-org/llama.cpp, commit ac4cdde, CUDA backend, Q4 K M GGUF. Accessed 2026-06-12. [32] The Linux Kernel documentation, “Real-time group scheduling,” documentation/scheduler/sched-rt-group.rst. Accessed 2026-06-12. [Online]. Available: https://docs.kernel.org/scheduler/sched-rt-group.html [33] NVIDIA, “TensorRT developer guide,” https://docs.nvidia.com/ deeplearning/tensorrt/, accessed 2026-06-12.