Precision - Preserving Subtensor Inference for a 22B Joint Audio - Video Generator under a Fixed GPU Working Set

Precision-Preserving Subtensor Inference for a 22B Joint Audio–Video Generator under a Fixed GPU Working Set Hanyue Shen and Xinling Liao

arXiv:2611.08023v1 [cs.DC] 5 Nov 2026

YH Intelligence Technology, Co., Ltd. Beijing, China hanyueshen@yh-intel.com; xinlingliao@yh-intel.com ORCID: H. Shen 0009-0009-5934-6384; X. Liao 0009-0000-3137-2825

Abstract—Complete-block offloading reduces model residency but requires each transferred block to fit the GPU budget. We present Precision-Preserving Subtensor Inference (PSI), a fixed-arena runtime that combines sub-block weight streaming with separately managed intermediate state for the BF16 LTX2.5 22B joint audio–video transformer. The contribution is a construction whose managed GPU footprint is determined by reusable pages, live state, and bounded workspace, together with controls that separate resource use, transport correctness, and numerical fidelity. Its partitioned operators preserve the dense real-arithmetic construction under stated assumptions. Across nine matched full-prompt cases on RTX A6000, modified PSI reduces sampled transformer-process peak memory from 8,578 to 360 MiB relative to official MATH, while mean time for the eleven transformer invocations rises from 247.11 to 349.00 s. Its managed arena is 80 MiB. Complete native generation and a complete-block FLASH path both succeed under a fixed 3 GiB available-memory cap. The matched A6000 comparison and separate DGX Spark service suite both achieve 100% success in complete audiovisual decoding. FFN reuse lowers mean block time by 4.0% with unchanged BF16 outputs; a fixed-arithmetic one-token transport control preserves all saved states bitwise. Targeted bias and attention changes improve local and one-token numerical comparisons, while mean endpoint error increases across nine full-prompt cases. The construction enables BF16 audiovisual inference with an explicit transformer-memory budget and makes numerical fidelity a separate, testable property of the implementation. Index Terms—resource-constrained inference, diffusion transformer, GPU memory, weight streaming, audiovisual generation

I. I NTRODUCTION Large generative models face two GPU-memory constraints: the checkpoint may exceed accelerator capacity, and the weights and state needed by one operation or block may exceed the budget assigned to a request. Complete-block offloading addresses model residency while retaining the block as the transfer and execution unit. PSI schedules operations inside that block using smaller weight slices and stores inactive intermediate state on the host. Joint audio–video generation makes this distinction Accepted to the CONSTRAINT 2026 Workshop, held in conjunction with IEEE BigData 2026.

useful because conditioning, transformer execution, and media decoding have different live-state requirements. Lightricks’ LTX-2.5 setup guide lists at least 32 GB of GPU memory and recommends an A100 (80 GB) or H100 [1]. Prior systems manage activations, tensor placement, and tile reuse across the memory hierarchy [2]–[6]. PSI applies these established techniques to a BF16 multimodal diffusion transformer through a fixed device arena and an indexed host journal. The execution question is how to schedule each operator within that arena. The measurement question is how this managed working set relates to the transformer process and the complete generation pipeline. Our prototype targets the official distilled BF16 LTX-2.5 22B checkpoint [7], a successor to the joint audio–video LTX-2 model [8]. It declares the BF16 write boundaries and execution dependencies used by each operator. Matched controls separate transport scheduling from numerical evaluation: a transport-only comparison fixes the arithmetic, while baseline and modified PSI are compared with the official MATH transformer on onetoken and full-prompt inputs. The modified variant evaluates targeted numerical changes to bias addition and attention accumulation. The paper makes four contributions: • We construct a fixed-arena runtime that combines reusable weight pages with indexed host state. Real-arithmetic identities establish the partitioned operators, and explicit event dependencies support byte-preserving transport. • We measure an 80 MiB managed arena and a 360 MiB transformer-process peak across eighteen full-prompt native trajectories. Allocation traces, capacity probes, and complete-block controls explain these resource scopes. • We separate numerical and scheduling effects through same-input operator probes, matched trajectories, and repeated block measurements. These controls identify floating-point differences and test optimizations with arithmetic held fixed. • We demonstrate complete generation under a fixed 3 GiB available-memory cap, with a successful FLASH control. The matched A6000 study produces 27/27 successfully

decoded outputs, and a separate Spark GB10 service suite produces 30/30; media similarity and prompt adherence are evaluated separately.

For the next disjoint tile T , set m′ = max(m, maxj∈T sj ) and ′ η = em−m , then update X ′ ℓ′ = ηℓ + esj −m , j∈T

II. D ESIGN ′

A. Managed working-set contract Let Slive be the reserved capacity for the current operator’s state and scratch, including alignment and unused capacity, W a fixed reserved library workspace, P the maximum page size, and ϵ bounded metadata. With two reusable pages, the declared managed working set is

z = ηz +

X

′

esj −m vj .

(4)

j∈T ′

′

The identity em−m esj −m = esj −m proves the invariant by induction from the first nonempty tile. Since ℓ > 0, z/ℓ equals dense softmax attention for any complete key partition. This established IO-aware construction [9] applies here to unmasked, noncausal, dropout-free attention with at least one key per active p Mmanaged ≤ Slive + W + 2P + ϵ. (1) row. Full-row RMS yi = γi xi / s/d + ε using Preplay P computes 2 Equation (1) accounts for PSI-managed device allocations for a the global s = J i∈J xi , total width d, and one positive selected workload. Independence from checkpoint size requires denominator for every tile. Split RoPE transforms (a, b) = parameter consumers partitionable into pages of at most P , (xj , xj+d/2 ) to (ca − sb, sa + cb) when the original pair values, bounded state/scratch reservation and library workspace, and table indices, and layout are retained. These finite-sum and page reuse ordered after the last consumer. Device-side journal pair identities establish the real-arithmetic construction. staging buffers are included in Slive or W . Host storage, C. Floating-point differences total host RSS, CUDA context, external allocations, and Floating-point implementations evaluate these identities other audiovisual pipeline phases lie outside this contract; its conditions must be checked for each operator and token length. through finite-precision reductions and intermediate writes. Four rules define the contract: PSI-managed device alloca- Changing a reduction order or a BF16 write boundary can tions come from the arena; each weight page has at most one change the result. For BF16 round-to-nearest, ties-to-even B, unreleased slice request; reuse follows the required copy and let z = 257/256 and b = 1/256. Then consumer dependencies; and operators declare their BF16 write B(B(z) + b) = 1 ̸= 129/128 = B(z + b). (5) boundaries. In the measured configuration, two 8 MiB pages and state/workspace/scratch share an 80 MiB arena. Figure 1 The extra BF16 write changes the result although the realshows the allocation boundary and page lifecycle. arithmetic expressions are identical. This mechanism motivates the implementation choices below. Actual projection operands B. Exact arithmetic equivalence and matched trajectories quantify their numerical effects in Section IV-B. For fixed checkpoint values and scalar functions, subtensor evaluation preserves the dense construction over the reals. Let D. Ownership and journaling X ∈ RN ×d , U ∈ Rh×d , V ∈ Rr×h , with biases a, b and A weight page is assigned to one outstanding slice request a complete, disjoint hidden-channel partition {J}. For the at a time. Its conceptual lifecycle is free (available for writing), coordinatewise tanh-GELU function ϕ, filling (host-to-device copy in progress), ready (copy complete), T and consuming (used by computation). Copy completion HJ = ϕ(XUJ,: + 1aT J ), X permits consumption; completion of the last consumer permits (2) T T F (X) = HJ V:,J + 1b . the next write. Stream ordering and cross-stream event waits J enforce these dependencies. The full-prompt implementation Each hidden coordinate receives its complete dot product and copies weights on the compute stream. In the asynchronous bias before ϕ. Concatenating the HJ therefore recovers the weight streamer, release records an event after all uses have dense hidden tensor; the disjoint down-projection sum recovers been enqueued and ends the request’s ownership; the next copy every output entry, with output bias added once. Computing waits for that event to complete before overwriting the page. Recording the event therefore schedules a dependency while each HJ once and reusing it preserves these operands. For one attention query with finite active scores sj = αq T kj , device work may still be in progress. Intermediate state follows a separate path. An indexed host online softmax retains the following invariant after a nonempty journal stores BF16 values by block, branch, and tensor role. processed key set A: The device-to-host copy waits for its producer; host commit X m = max sj , ℓ= esj −m , waits for that copy to finish before moving the staged bytes into j∈A j∈A their journal location. Device-buffer overwrites and later hostX (3) sj −m to-device reloads retain the previous-consumer dependencies. z= e vj . A later operator reloads the same stored BF16 values. j∈A

GPU PROCESS / sampled NVML HOST Checkpoint slices

PSI ARENA: 80 MiB H2D slices

Page A: 8 MiB

Page B: 8 MiB

Indexed BF16 journal

CUDA context and external allocations

Workspace 8 MiB

State / scratch: 56 MiB commit / reload

Host storage and RAM outside device contract PAGE LIFECYCLE

free

filling

ready

consuming

Next write follows completion of the last consumer

Fig. 1. Fixed-arena construction. Checkpoint slices enter reusable pages; state is committed to and reloaded from indexed host buffers. The next write to a weight page follows completion of its last consumer. The arena is inside the GPU process; external allocations and CUDA context are outside its 80 MiB accounting. Box areas are schematic.

For a fixed deterministic arithmetic dependency graph, bytepreserving transport with these dependencies preserves outputs: in topological order, every operation receives the same operand bytes and computes the same result. Producer completion precedes consumption, and the last consumer completes before its buffer is overwritten. III. I MPLEMENTATION AND METHOD A. Numerical implementation choices Both native variants write the biased up projection as BF16 before GELU, then write GELU’s output as BF16. Baseline PSI also rounds the audio matrix-product output to BF16 before adding bias; modified PSI adds that bias in FP32 before the preGELU BF16 write. Modified PSI also replaces BF16 numerator exponential weights with FP32 online accumulation for 64dimensional attention heads, changing the reduction schedule. These two changes are evaluated together in trajectories and separately in same-input operator probes. B. Data flow and scheduling A weight slice enters a device page, the operator consumes it, and intermediate state is retained on device or stored in the host journal for later use. Three scheduling optimizations act on this data flow. FFN reuse computes each 1,024-channel hidden tile once, journals its BF16 values, and reuses them across down-projection output tiles. This removes repeated hidden computation while retaining the ordered down-projection operations. Asynchronous journal transport stages state through a pinned host ring, allowing copies to overlap independent computation while event waits preserve producer and consumer order. Keeping updated state resident removes redundant transfers. Weight prefetch prepares upcoming slices in two alternating pages; computation waits for the required copy, and the next

page write waits for its previous consumer. The journal and prefetch variants are tested separately on top of FFN reuse. C. Workloads and comparisons The native CUDA prototype targets LTX-2.5 22B distilled with BF16 weights throughout. Each transformer invocation in the full-prompt cohort contains 48 blocks and separate 1,024row video and audio conditioning contexts. Stage 1 uses 1,232 video-state tokens; Stage 2 uses 4,928. Both use 126 audio-state tokens. The paired block studies use a first-block input with 4,928 video-state tokens, 126 audio-state tokens, and a frozen one-token text context. Their performance measurements apply to that block input. Table I summarizes the experiments, and Appendix A provides supporting measurements. Each local candidate compares eight stage tensors and the block’s final video and audio outputs, giving ten BF16 probes. Matched trajectories share conditioning, initial state, random tensors, sigma schedules, and preprocessing/upsampling functions; subsequent states evolve separately in each implementation. Same-input operator comparisons are analyzed separately. In the one-token replay, all eleven official denoising states reproduce the frozen BF16 reference values. The full-prompt cohort fixes three service prompts—a spokesperson, a chef and an airliner—and three unsigned 64bit request seeds per prompt before execution. The seeded repetitions are labeled run #1, run #2, and run #3 within each prompt category. All 27 trajectories use 1,024-row native projected contexts shared with the official MATH transformer; valid tokenizer lengths are 87, 76 and 77, respectively, with the remaining rows padded. Native text conditioning and the common native media decoders are held fixed to isolate the transformer comparison.

TABLE I E XPERIMENT OVERVIEW. A LL STUDIES USE RTX A6000 EXCEPT THE S PARK SERVICE SUITE .

Experiment

Setup

Measurement

Transformer memory One-token replay Full-prompt comparison Full-pipeline budget Transport control Scheduling Capacity and block offload Spark service

18 native full-prompt trajectories; both stages Official MATH, baseline PSI, modified PSI; 8+3 steps 3 prompts × 3 seeds × 3 transformer paths Native PSI and FLASH; 3 GiB physical free pool One-token replay; fixed arithmetic 4 block variants; 5 measured runs each 6 arena/page settings; official block controls DGX Spark GB10; 3 runs × 10 requests; single slot

Managed arena and process peak Trajectory error, time, and memory Numerical fidelity and media validation Completion, time, and memory Bitwise equality of saved states Block time and bitwise equality Feasibility, memory, and numerical error Media validation, throughput, and adherence

IV. R ESULTS A. Memory scopes Across the nine full-prompt cases, baseline and Modified PSI each complete an 8+3 trajectory, giving eighteen native trajectories. Every transformer invocation uses an 80 MiB arena and reaches a sampled per-process NVML peak of 360 MiB. These measurements cover both denoising stages, including the larger Stage-2 video state, and describe the transformer process; complete-pipeline memory is measured separately below. Table II pairs time and memory for the same nine full-prompt cases. Modified PSI uses 95.8% less sampled transformerprocess memory than official MATH, with 41.2% greater mean summed invocation time. The common outer timer and equal measurement pauses support this descriptive comparison; cases use a fixed implementation order. A separate first-block CUPTI trace observes four simultaneously live dynamic device allocations: 80 MiB, 8 MiB, 128 KiB, and 1 KiB, totaling 88.126 MiB. The three smaller allocations precede arena creation; source order associates them with library initialization, while the trace itself records allocation sizes and lifetimes. This dynamic-allocation total remains distinct from the sampled process NVML peak. The reserved 80 MiB arena consists of two 8 MiB pages, 8 MiB declared workspace, and 56 MiB scratch; the full-prompt

TABLE II M ATCHED FULL - PROMPT RTX A6000 TIME – MEMORY COMPARISON , NINE COMPLETED TRAJECTORIES PER PATH . WALL TIME SUMS ELEVEN INVOCATIONS , INCLUDING STARTUP, INPUT / OUTPUT, AND A 1.5 S MEASUREMENT PAUSE PER INVOCATION . C ONDITIONING , STATE UPDATES , UPSAMPLING , AND DECODING ARE OUTSIDE THIS TIMER . S AMPLE SD DESCRIBES VARIATION ACROSS CASES . E ACH PATH ’ S SAMPLED PROCESS PEAK IS IDENTICAL ACROSS ITS NINE CASES . Mean ± SD (s) Peak (MiB)

Path

Official MATH 247.11 ± 11.65 Baseline PSI 349.77 ± 9.16 Modified PSI 349.00 ± 3.22

8578 360 360

1,916

2,000

GPU memory (MiB)

The full-pipeline budget control uses spokesperson run #1 and the same request seed. We compare official FLASH followed by native PSI under a fixed 3 GiB physical free pool. A direct CUDA allocation reserves the rest of the GPU before each run, leaving an initial physical free pool within 2 MiB below this budget. The reservation stays fixed through tokenizer and native conditioning, regenerated initial/noise tensors, the 8+3 transformer schedule, shared preparation and upsampling, native decoding, and media validation. All prompt-dependent stages run again under the reservation; input bytes are checked against the matching full-prompt case. Each path competes for this pool with its CUDA context and library allocations; the reservation helper’s own context is reported separately. Separate allocation probes verify small allocations and over-budget rejection. FLASH uses completeblock CPU staging and checked zero-mask elision without a MATH fallback. This is an A6000 available-memory emulation with one attempt per setting.

1,500 1,000

360

500

80 0

Managed arena

Transformer process

Complete pipeline

Fig. 2. RTX A6000 memory scopes. The 80 MiB managed arena and 360 MiB transformer-process peak apply to both denoising stages of all eighteen full-prompt native trajectories. The 1,916 MiB sampled workload-process peak describes complete native generation under the fixed 3 GiB available-memory cap; the FLASH path has the same measured peak. The reservation helper is excluded.

native trajectories reach a maximum of 52.94 MiB scratch use. Figure 2 compares their managed arena and transformerprocess peak with the full-pipeline process peak under the fixed 3 GiB budget. B. Numerical discrepancies and targeted changes All nine full-prompt cases complete the official MATH, baseline PSI, and modified PSI trajectories. Figure 3 shows every endpoint comparison. Modified PSI lowers relative L2 error in

5/9 video cases and 3/9 audio cases; the mean paired changes (modified minus baseline PSI) are +0.0378 and +0.0395. The changes improve individual cases while increasing the mean endpoint error in both modalities. All compared endpoint tensors are finite. Independent CPU decoding of the BF16 values reproduces the trajectory error metrics. The fixed one-token diagnostic (random seed 185) gives a different outcome. All eleven official states reproduce the frozen reference bitwise. The native variants retain an 80 MiB explicit application arena and a 360 MiB sampled process NVML peak. Baseline PSI ends at video/audio relative L2 errors of 0.5057/0.3339; Modified PSI ends at 0.4262/0.2739 (Table III). The same-input diagnostics identify concrete numerical boundaries. Fusing FP32 bias addition before the audio FFN up-projection’s BF16 write eliminates 267,593 differing elements among 1,032,192 output elements in the full-shape projection probe. Switching the 64-dimensional-head attention implementation from BF16 numerator exponential weights to FP32 online accumulation reduces the tested audio-selfattention relative L2 error from 1.28 × 10−3 to 1.26 × 10−5 , and differing BF16 elements from 68,979 to 45 among 258,048 output elements. The fallback also changes the reduction schedule. These are operator comparisons on identical inputs; the full trajectory includes both changes and subsequent state evolution. Actual projection operands also expose a finite-precision evaluation discrepancy. At the first mismatching Q coordinate in the baseline PSI diagnostic at denoising state 5 and the first transformer block, with 256-row tiles and 512 output columns per group, the initial normalized/affine tensor matches the official input bitwise. Exact rational evaluation of all 4,096 BF16 products and the bias gives 46489387 , 239 175 177 yofficial = − 21 . ynative = − 21 , 2 2 y∗ = −

(6)

The native value is the nearest BF16 value to y∗ ; the official projection implementation produces a different BF16 value. Independent integer decoding of the original operands reproduces this result. Thus, identical stored inputs and a common realvalued formula can produce different actual outputs through finite-precision evaluation. Matching a particular official implementation is a distinct target from rounding the exact real sum once. Internal timer intervals differ: the official timer spans input loading through output serialization, while the native timer spans the 48 blocks and final projection.

TABLE III M ATCHED ONE - TOKEN 8+3 REPLAY ON RTX A6000. P EAK MEMORY IS SAMPLED PER - PROCESS NVML. I NVOCATION WALL SUMS ALL ELEVEN TRANSFORMER SUBPROCESSES , INCLUDING INITIALIZATION , OUTPUT WRITING , AND A MEASUREMENT PAUSE OF 0.5 S / CALL FOR OFFICIAL AND 1.5 S / CALL FOR NATIVE . Metric

Official Baseline PSI Modified PSI

Invocation wall (s) 227.385 Process peak (MiB) 8578 Final video rel. L2 0 Final audio rel. L2 0

306.352 360 0.5057 0.3339

302.163 360 0.4262 0.2739

TABLE IV R EPEATED RTX A6000 FIRST- BLOCK SCHEDULING STUDY. E ACH ROW HAS FIVE MEASURED SAMPLES ; EVERY RUN PRESERVES ALL TEN TEST TENSORS BITWISE . A SYNCHRONOUS JOURNAL TRANSPORT AND WEIGHT PREFETCH SEPARATELY EXTEND FFN REUSE . Variant Scheduling baseline FFN reuse Reuse + async. journal Reuse + weight prefetch

Mean (s)

Sample SD (s)

1.5428 1.4816 1.4349 1.4753

0.0188 0.0296 0.0156 0.0357

The ten BF16 output tensors match the scheduling baseline in all 24 runs, including warm-ups. Table IV reports native block-wall means and sample standard deviations; these timers include eight intermediate-output writes and exclude process initialization and the post-work measurement pause. FFN reuse lowers the mean block time by 4.0% relative to the scheduling baseline. Adding asynchronous journal transport lowers the mean by a further 3.2%; adding weight prefetch gives a 0.43% reduction, which is small relative to the observed dispersion. The asynchronous-journal variant keeps updated state resident and omits redundant transfers, reducing activation H2D traffic and host-journal writes by 163,545,088 bytes each per block in the repeated measurements. Its blocking synchronizations decrease from 422 to 297. To isolate transport over a full trajectory on the fixed onetoken input, we hold baseline PSI arithmetic fixed and replace asynchronous journal transport with a synchronous computestream implementation. All twelve saved video/audio states across the 8+3 run, including the inter-stage state, are bitwise equal between the two versions. D. Capacity sweep and complete-block control

The Modified PSI first-block construction is tested at six discrete arena/page capacities with head, row, output, and FFN tiles fixed. Three configurations complete and produce the same ten tensors bitwise (Table V). The smaller arenas exhaust scratch; reducing each page to 4 MiB fails when an FFN weight tile no longer fits. Increasing capacity without changing C. Repeated scheduling measurements the computation leaves the successful block times near 1.45 s. We repeat the fixed first-block scheduling studies on a GPU These are single capacity probes of this fixed construction. reserved for the benchmark. The input has 4,928 video tokens, The complete-block control stages the same original check126 audio tokens, and one-token conditioning. Four scheduling point parameters on CPU, moves the unmodified official block variants with identical arithmetic run once for warm-up and then and inputs to CUDA, and uses the original SDPA math backend. for five measured cycles, rotating execution order each cycle. It runs out of memory at PyTorch allocator caps of 80, 89,

3 prompts × 3 seed runs

Baseline PSI

(a) Video

Spokesperson

Chef

Airliner

Modified PSI

(b) Audio

Δ

Δ

run #1

+0.0631

−0.0035

run #2

−0.0009

+0.0222

run #3

+0.0272

+0.0318

run #1

+0.0536

+0.0291

run #2

−0.0542

+0.0172

run #3

−0.0052

+0.0068

run #1

−0.0280

−0.0041

run #2

−0.0735

−0.0033

run #3

+0.3579

+0.2595

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.1

0.2

0.3

Endpoint relative L2 error to official MATH (smaller is closer) Δ = modified − baseline; negative means lower error. Panel scales differ.

Fig. 3. Endpoint relative L2 errors against the official MATH transformer for three prompts and three seed runs per prompt on RTX A6000. Each row pairs baseline PSI (open circle) with modified PSI (filled circle); smaller values indicate closer agreement. ∆ is modified minus baseline error. Video and audio use different horizontal scales. All paths share native projected conditioning, initial state, and random inputs within each case.

TABLE V RTX A6000 CAPACITY PROBES ON THE 4,928/126- TOKEN BLOCK TEST INPUT. PAGE CAPACITY IS PER PAGE ; THERE ARE TWO PAGES . S UCCESSFUL OUTPUTS MATCH THE 80/8 M I B CONFIGURATION BITWISE . Arena (MiB) 48 64 80 96 80 96

Page Wall (s) (MiB) 8 8 8 8 4 16

– – 1.4458 1.4481 – 1.4497

Outcome Scratch exhausted Scratch exhausted Complete Complete Weight tile exceeds page Complete

TABLE VI S UCCESSFUL BLOCK CONTROLS ON IDENTICAL FROZEN INPUTS . R ELATIVE L2 USES THE OFFICIAL MATH BLOCK AS REFERENCE . M EMORY IS SAMPLED PROCESS NVML. FLASH USES VERIFIED ZERO - MASK ELISION . Implementation

Peak (MiB) Video error Audio error

Official MATH Official FLASH adapter Modified PSI native (80/8)

8360 1818 360

0 0.002246 0.003195

0 0.002441 0.001501

TABLE VII F ULL - PIPELINE A6000 CONTROL UNDER A FIXED 3 G I B PHYSICAL FREE POOL . B OTH PATHS PRODUCE SUCCESSFULLY DECODED AUDIOVISUAL OUTPUT. WALL TIME INCLUDES RUNTIME CONSTRUCTION , FINAL VALIDATION , AND ELEVEN 1.5 S MEASUREMENT PAUSES ; RESERVATION SETUP IS EXCLUDED . P EAK IS SAMPLED WORKLOAD - PROCESS NVML, EXCLUDING THE RESERVATION HELPER .

1,024, and 4,096 MiB, and completes at 8,192 MiB with outputs bitwise equal to the official block reference. The 4,096 MiB failure occurs in dense SDPA’s 2.90 GiB allocation; the 80 and 89 MiB failures occur while moving the prepared inputs. The 89 MiB cap is a rounded-up comparison to the separately traced native dynamic-allocation peak, with framework-external allocations still outside the PyTorch cap. These outcomes characterize complete-block residency with the math backend.

E. Full-pipeline physical-budget control

The FLASH block control uses a compatibility adapter that checks each of the four masks is floating-point, finite, and identically zero before omitting it from the attention call. Under this mask convention, the official block completes at a 2,048 MiB allocator cap: peak framework allocation is 1,231.55 MiB and sampled process NVML is 1,818 MiB. Table VI reports its numerical difference from the successful MATH reference alongside the Modified PSI native block. Backend selection therefore changes both memory use and floating-point outputs for this input.

Both paths complete the fixed spokesperson case under a 3 GiB physical free pool and produce successfully decoded 121-frame, 704×448 audiovisual files with 48 kHz stereo audio (Table VII). Both paths have a sampled workload-process NVML peak of 1,916 MiB, excluding the reservation helper. The native and FLASH pipeline times are 610.37 and 459.59 s. These are single observations of this prompt and seed under the same fixed-reservation procedure. At 3 GiB, all twelve saved native states and the raw RGB and natural audio outputs exactly reproduce the same unconstrained

Path

Wall (s) Peak (MiB)

Official FLASH Modified PSI native

459.59 610.37

1916 1916

native case. Against the shared-conditioning MATH reference, native and FLASH final video/audio relative L2 errors are 0.3893/0.05439 and 0.3608/0.06663. Their encoded-video PSNR/SSIM values are 30.53 dB/0.9427 and 30.35 dB/0.9478. Table IX in Appendix A-C gives the direct comparison and waveform errors. F. Matched full-prompt media

TABLE VIII S INGLE - SLOT DGX S PARK GB10 SERVICE RESULTS . P IPELINE DURATION COVERS EACH JOB ’ S GENERATION ; THROUGHPUT USES THE COMPLETE ORCHESTRATION SPANS OF THE THREE RUNS . Metric

Spark GB10

Runs × requests Active execution setting Decoded outputs / requests Mean pipeline time (s) Decoded outputs/GPU-hour All-five-clear scores (A/B)

3 × 10 Single slot 30/30 524.143 6.862 8/30; 14/30

We evaluate three distinct properties. Numerical fidelity measures tensor differences from the matched reference. Media similarity uses encoded-video PSNR and SSIM as image-similarity proxies. Prompt adherence measures whether generated content satisfies the prompt’s criteria, with inde- fresh deepseek-flash contexts use the same frozen rules pendent ASR for requested speech. Complete audiovisual with different instruction and criterion order. Each criterion decoding establishes file validity; similarity and adherence receives 0 (absent or contradicted), 1 (partial or ambiguous), 2 (clearly supported), or an unassessable label. An all-five-clear scores characterize the decoded content. All 27 matched full-prompt outputs pass complete result requires five scores of 2. The two contexts return 8/30 and 14/30 all-five-clear results. video/audio decoding, with 121 frames at 704×448 and 48 kHz stereo audio. Against the same-case official-transformer Criterion scores agree on 134/150 items, and the all-fiveendpoint decoded by the common native decoder, Modified clear decisions agree on 24/30 clips. These repeated modelPSI has higher encoded-video PSNR in 5/9 cases and higher assisted evaluations disagree on borderline prompt-adherence SSIM in 4/9 cases than baseline PSI. Airliner run #3 decreases judgments. Independent ASR exactly matches the requested dialogue affrom 23.92 to 19.70 dB PSNR and from 0.8351 to 0.7741 ter case and punctuation normalization for all 6/6 Spark speech SSIM. Two fresh deepseek-flash contexts assess all 121 clips. ASR measures recovered words; speaker attribution and frames against each prompt’s five original criteria; all-five-clear lip synchronization require separate audiovisual assessment. requires a score of 2 on every criterion. Context A returns 5/9, Appendix A-B specifies the evaluation protocol. 4/9 and 3/9 for official-transformer, baseline PSI and Modified V. R ELATED WORK AND POSITIONING PSI media; context B returns 6/9 for each arm. The contexts The residency unit determines which memory constraint agree on 21/27 strict decisions and 119/135 criterion scores. Neither context marks any airliner clip all-five-clear. Both offloading can relieve. In training, vDNN manages activation assign airliner run #3’s takeoff criterion 1 for Modified PSI lifetimes across CPU and GPU, while SwapAdvisor jointly and 2 for the other arms. All nine independent dialogue ASR selects operator order, allocation, and tensor swaps [2], [3]. transcripts exactly match the requested four-word sentence ZeRO-Infinity explicitly decomposes oversized operators into after the fixed normalization. Scores were finalized before parameter tiles fetched and released sequentially [4]. For autoregressive inference, FlexGen optimizes tensor placement implementation identities were revealed. and access over GPU, CPU, and disk [5]. These precedents G. Spark service delivery distinguish total model storage from the parameters and live On DGX Spark GB10, requests execute one at a time; the state needed for the current computation. Limiting that live state also requires scheduling intermediates suite covers ten prompt categories in each of three seeded runs. All 30 outputs are verified unchanged from the original and data movement within and across operators. Welder’s service outputs and pass container probing and complete tile graphs coordinate intra- and inter-operator reuse across a video/audio decoding. Decoded-output throughput uses the memory hierarchy that can include the host [6]. FlashAttention three run makespans; mean pipeline time and throughput appear reduces HBM–SRAM traffic through tiled exact attention [9], and Blockwise Parallel Transformers carries sequencein Table VIII, with dispersion statistics in Appendix A-A. The Spark configuration uses BF16 without quantization, a block attention into feed-forward computation [10]. Operator 1,024-token dynamic context, and an 80 MiB arena. Its unified- tiling, intermediate reuse, and online attention are therefore memory accounting is reported separately from the RTX A6000 established ingredients of memory-efficient execution. Diffusion systems combine these choices under latency and process NVML measurements. memory budgets. Streamlined Inference uses feature slicing H. Spark prompt-adherence evaluation and operator grouping for activation memory, with a separate We assess each of the 30 successfully decoded Spark outputs step-reuse mechanism [11]. Xema couples selective blockagainst the five criteria in its original prompt. Each API weight offload with localized activation chunking and fusion request contains all 121 original frames at 704×448, their and static lifetime-based memory placement [12]. ChunkFlow timestamps, and independent Whisper small.en transcripts for splits layer prefetch to yield to distributed collectives and speech prompts. Hardware and prior scores are hidden. Two permits partial parameter residency [13]; its transfer chunks

refine communication scheduling while computation proceeds Spark service suite is separate from the RTX A6000 matched layerwise. Diffusers exposes block- and leaf-level offload with full-prompt and fixed-budget experiments. stream prefetch [14], and WeeLLM documents unquantized The 17 native source files retained on Spark are bytelayer streaming and lists LTX-2.5 support [15]. Lightricks’ identical to the corresponding baseline copies. Spark records reference pipeline also documents FP8 casting with CPU or 330 successful native transformer invocations, eleven per job, disk offload for constrained GPU memory [16]. each completing 48 blocks. The available implementation PSI organizes operator partitioning and host-backed state combines FFN reuse and asynchronous journal transport; it for the BF16 LTX-2.5 22B audiovisual transformer [7], [8] uses the compute stream for weight copies, so it does not using fixed-size weight pages, separately indexed host state, include the separate weight-prefetch optimization. and explicit producer/consumer dependencies. Its contribution A. Service records and timing distributions is this construction and its validation: a managed-arena bound Spark executes one request at a time. All 30 original MP4 with stated state and workspace assumptions; separate arena, process, and complete-pipeline measurements; and numerical files pass container and stream probing and full video/audio deand media checks that distinguish real-arithmetic identities, coding to end of file. Validation uses FFmpeg 6.1.1 configured transport-only byte preservation, BF16 execution differences, to stop on decoding errors; no errors occur. Every file contains and successfully decoded audiovisual outputs. The fixed-budget 121 video frames at 704×448 and 24 fps, with a 5.041667 s controls include both PSI and a complete-block FLASH path. video/container duration and 48 kHz stereo AAC audio. The media were validated on 30 October 2026. VI. C ONCLUSION The monotonic pipeline timer starts after setup and ends PSI executes a 22B BF16 joint audio–video transformer by after decoding and multiplexing; it excludes time waiting in scheduling weight slices through a fixed arena and retaining the service queue. These pipeline times have a sample standard intermediate state in indexed host buffers. Its partitioned deviation of 4.419 s and a linearly interpolated P95 of 529.961 operators preserve the dense real-arithmetic construction under s. Each job includes eight Stage-1 and three Stage-2 transformer the stated assumptions. Event-ordered transport preserves invocations. Spark’s throughput uses the 30 decoded outputs operand bytes; the fixed one-token control reproduces twelve and the three measured orchestration spans: 5261.613492, saved states bitwise. 5231.609223, and 5246.526462 s. These include polling and Across eighteen full-prompt native trajectories on RTX orchestration overhead beyond the worker intervals. A6000, both denoising stages use an 80 MiB managed arena B. Semantic evaluation protocol and reach a sampled 360 MiB transformer-process peak. The Spark evaluation covers all 3,630 frames from the 30 Complete native generation and the complete-block FLASH control both succeed under a fixed 3 GiB available-memory decoded videos and uses 60 separate API calls made on 30 Octocap. The matched three-path study produces 27/27 successfully ber 2026. The API identifies the model as deepseek-flash; decoded audiovisual outputs, and the separate Spark service service documentation identifies that deployment as DeepSeeksuite produces 30/30. These results distinguish transformer V4.1-Flash-1017. No previous judgment is supplied to a later residency, complete-pipeline memory, and decoded-output call. Evaluation records retain the input prompts, raw responses, and timestamped evidence for each criterion. Both contexts validity. Finite-precision evaluation accounts for the same-input use the same model. Whisper small.en runs independently in CPU INT8 mode projection discrepancies. Targeted bias and attention changes provide local numerical improvements and lower both one- with no requested-text prompt. Across the six Spark speech token endpoint errors; across nine full-prompt cases, mean error clips, the transcripts have zero word edits among 36 requested increases in both modalities. With arithmetic fixed, FFN reuse words. lowers mean block time by 4.0% in the five-repeat scheduling C. Fixed-budget state and media comparisons study. The construction thus supports small managed residency Table IX compares the completed 3 GiB paths with the samewith explicit scheduling and numerical tradeoffs. The matched case unconstrained MATH transformer and with each other. All media results cover three prompt categories with common use common native conditioning, decoding, and decoder noise. conditioning and decoders; broader perceptual evaluation Relative L uses the named reference’s norm. RGB metrics 2 and localization of the remaining trajectory differences are cover all 114,487,296 decoded BF16 values before clipping directions for further study. or encoding; audio metrics cover 240,480 natural samples A PPENDIX A per stereo channel without alignment, normalization, or mux S UPPORTING MEASUREMENTS AND DIAGNOSTICS padding. The direct FLASH-to-native encoded comparison Baseline PSI includes the Q/K normalization and rotary- gives 29.92 dB PSNR and 0.9471 SSIM over all 121 frames. position materialization changes evaluated at denoising state 5 The constrained native rerun matches all twelve same-native in the first transformer block. Modified PSI adds the bias and unconstrained states and raw RGB/audio samples exactly; its attention changes described in the numerical comparison. The encoded frames have zero error. Per-state, per-frame, and perchannel records accompany the retained inputs and outputs.

TABLE IX R ELATIVE L2 IN THE COMPLETED FIXED - BUDGET CASE . MATH DENOTES THE SAME - CASE UNCONSTRAINED MATH TRANSFORMER WITH THE COMMON NATIVE DECODER ; NATIVE DENOTES THE 3 G I B M ODIFIED PSI PATH . Candidate / reference

Video

Audio

RGB Waveform

Native / MATH FLASH / MATH FLASH / native

0.3893 0.05439 0.1658 0.3608 0.06663 0.1684 0.3637 0.07136 0.1796

0.5874 0.8468 0.8249

ACKNOWLEDGMENT Hanyue Shen and Xinling Liao contributed equally to all aspects of the work. The authors declare no conflicts of interest. OpenAI Codex assisted with implementation diagnostics, evidence analysis, and manuscript preparation. DeepSeek supplied the model-assisted semantic judgments described above. The authors are responsible for the technical content and final manuscript. R EFERENCES [1] Lightricks, “LTX-2.5: System requirements,” Official LTX documentation, 2026, accessed Nov. 4, 2026. [Online]. Available: https://docs.ltx.io/ open-source-model/getting-started/system-requirements [2] M. Rhu, N. Gimelshein, J. Clemons, A. Zulfiqar, and S. W. Keckler, “vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design,” in Proc. 49th IEEE/ACM Int. Symp. Microarchitecture (MICRO), 2016, pp. 1–13. [Online]. Available: https://doi.org/10.1109/MICRO.2016.7783721 [3] C.-C. Huang, G. Jin, and J. Li, “SwapAdvisor: Pushing deep learning beyond the gpu memory limit via smart swapping,” in Proc. 25th Int. Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2020, pp. 1341–1355. [Online]. Available: https://doi.org/10.1145/3373376.3378530 [4] S. Rajbhandari, O. Ruwase, J. Rasley, S. Smith, and Y. He, “ZeRO-Infinity: Breaking the GPU memory wall for extreme scale deep learning,” in Proc. Int. Conf. High Performance Computing, Networking, Storage and Analysis (SC), 2021, pp. 1–14. [Online]. Available: https://doi.org/10.1145/3458817.3476205 [5] Y. Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, B. Chen, P. Liang, C. Ré, I. Stoica, and C. Zhang, “FlexGen: Highthroughput generative inference of large language models with a single GPU,” in Proc. 40th Int. Conf. Machine Learning (ICML), ser. PMLR, vol. 202, 2023, pp. 31 094–31 116. [Online]. Available: https://proceedings.mlr.press/v202/sheng23a.html [6] Y. Shi, Z. Yang, J. Xue, L. Ma, Y. Xia, Z. Miao, Y. Guo, F. Yang, and L. Zhou, “Welder: Scheduling deep learning memory access via tile-graph,” in Proc. 17th USENIX Symp. Operating Systems Design and Implementation (OSDI), 2023, pp. 701–718. [Online]. Available: https://www.usenix.org/conference/osdi23/presentation/shi [7] Lightricks, “LTX-2.5 model card,” Hugging Face model repository, 2026, accessed Oct. 30, 2026. [Online]. Available: https://huggingface. co/Lightricks/LTX-2.5 [8] Y. HaCohen et al., “LTX-2: Efficient joint audio-visual foundation model,” arXiv preprint arXiv:2601.03233, 2026. [Online]. Available: https://arxiv.org/abs/2601.03233 [9] T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré, “FlashAttention: Fast and memory-efficient exact attention with IO-awareness,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 16 344–16 359. [Online]. Available: https://doi.org/10.52202/068431-1189 [10] H. Liu and P. Abbeel, “Blockwise parallel transformers for large context models,” in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 8828–8844. [Online]. Available: https: //doi.org/10.52202/075280-0386

[11] Z. Zhan, Y. Wu, Y. Gong, Z. Meng, Z. Kong, C. Yang, G. Yuan, P. Zhao, W. Niu, and Y. Wang, “Fast and memory-efficient video diffusion using streamlined inference,” in Advances in Neural Information Processing Systems, vol. 37, 2024, pp. 13 660–13 684. [Online]. Available: https://doi.org/10.52202/079017-0437 [12] X. Kang, G. Xiang, S. Li, Y. Wang, S. Shi, L. Zhang, and X. Chu, “Xema: Efficient diffusion serving through fine-grained memory management and auto-configuration,” arXiv preprint arXiv:2607.11136, 2026. [Online]. Available: https://arxiv.org/abs/2607.11136 [13] H. Meng, D. W. Liu, and D. Li, “ChunkFlow: Communication-aware chunked prefetching for layerwise offloading in distributed diffusion transformer inference,” arXiv preprint arXiv:2605.11335, 2026. [Online]. Available: https://arxiv.org/abs/2605.11335 [14] Hugging Face, “Diffusers: Reduce memory usage,” Official documentation, 2026, accessed Oct. 30, 2026. [Online]. Available: https://huggingface.co/docs/diffusers/optimization/memory [15] Jit-Roy, “WeeLLM: Layer-streaming inference for large diffusion models,” GitHub repository (evaluated revision), 2026, accessed Oct. 30, 2026. [Online]. Available: https://github.com/Jit-Roy/WeeLLM [16] Lightricks, “LTX-2 reference implementation: Quick start and optimization tips,” Official GitHub repository, 2026, accessed Nov. 4, 2026. [Online]. Available: https://github.com/Lightricks/LTX-2/blob/ main/README.md

Related documents

• StreamEval: Evaluating Interactive Streaming Video Generation Models#1460417
• Audio Preprocessing Effects on Stuttering Detection: A Class-Specific Analysis#1116616
• VoRTeC: Taming Foundation Flow for One-step Real time Video Compression#925268
• VoRTeC: Taming Foundation Flow for One-step Real time Video Compression#921032
• Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging#1142386