Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation

arXiv:2611.05642v1 [cs.DC] 3 Nov 2026

RUNWAY C HARACTERS : S ERVING A 57B V IDEO D IFFUSION T RANSFORMER FOR R EAL -T IME C ONVERSATION Rohan Agarwal 1 Robin Andeer 1 Alex Armbruster 1 Bennett Arthur 1 Michail Christos Doukas 1 Piero Esposito 1 Qiming Fang 1 Aron Filkey 1 Bryan Fox 1 Anastasis Germanidis 1 Jonathan Granskog 1 Corina Gurau 1 Robin Kahlow 1 Taras Khakhulin 1 Nasir Mohammad Khalid 1 Tomasz Łakomy 1 Mike Leisz 1 Kathleen M. Lewis 1 Daniil Merkulov 1 Nicolas Neubert 1 Ryan Phillips 1 Yining Shi 1 Axel Örn Sigurðsson 1 Erica Simmons 1 Mark Sturley 1 Ziyi Tang 1 Michail Tarasiou 1 Jamie Umpherson 1 Cindy Wang 1 Jimei Yang 1 Hudson Yeo 1 Wei Zhang 1 Zejia Zheng 1

A BSTRACT Conversational video characters must generate video as fast as it plays, for one user at a time: at 24 fps, a sustained 166.7 ms per four-frame chunk at batch size one, with only a few chunks of buffering. We describe the system behind Runway Characters, which serves a ≈57B-parameter autoregressive video diffusion transformer (four denoising passes per chunk) at 704×1088 and 24 fps on four H100 GPUs with a voice agent in the loop. Each chunk takes 240 attention calls, each reading a 5,788-slot KV cache in which only 757 tokens are new, and 29% of the original step was whole-cache work repeated in every call. We move that work to write time (a ring cache, RoPE and FP8 quantization applied once per write, and a delta rotation when positions reset), fuse the tensor-parallel all-reduce into the FP8 GEMM with in-switch reduction, and give the decoder that turns latents into video frames its own GPU. Without the fused kernel, our optimizations cut the diffusion step from 240.9 to 159.3 ms (1.51×, 24.5 fps sustained) with no detected degradation in lip-sync scores; the fused kernel, rolled back once in production and since re-enabled, reaches 147.4 ms (1.63×, 43% of FP8 peak). In seven weeks of production the worker averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second, and our pipeline takes 0.49 s of a 2.1 s median conversational turn. On the same GPUs, the two open-source real-time video systems we benchmarked, at four denoising passes, miss real time or add over a second of latency (one by estimate), at a third to half of our resolution. 1

I NTRODUCTION

A conversational video character is a generated face that listens and talks back in real time: the user speaks, a voice agent answers, and a video model animates the character to that answer while it plays. Runway Characters (Runway Research and Engineering Team, 2026) serves such characters with a ≈57B-parameter autoregressive (AR) video diffusion transformer. Each user gets a private stream: every 4 video frames (166.7 ms at 24 fps) the model must denoise one new latent frame (a compressed frame that a separate decoder turns into the 4 video frames), conditioned on recent audio and video, at that rate for the whole session. The player buffers only a few chunks; falling behind by more freezes the picture, and since each frame carries its slice of audio, the voice stalls with it. We treat the model as a fixed workload. Runway Characters 1

Runway, New York, NY, USA. Correspondence to: Zejia Zheng <zejia@runwayml.com>. Preprint.

animates any single image, from a photorealistic human to a cartoon mascot or fantasy creature, in HD at 24 fps (Runway Research and Engineering Team, 2026), and we chose model capacity for visual fidelity and temporal consistency across such characters in long, high-resolution conversations. Our goal is to serve that model at interactive rates in a deployed product without degrading its output. Table 1 lists what real time means here. It differs from the serving problems the systems literature has studied most. LLM servers amortize weights by batching requests; here the deadline is per user and per chunk, and one session already saturates its GPUs. Video diffusion servers parallelize long sequences offline (Fang et al., 2024), or pipeline frames and denoising steps across GPUs for throughput (Feng et al., 2026; Huang et al., 2025b), which multiplies per-chunk latency by the pipeline depth. At batch size one, each AR step runs 4 denoising passes × 60 blocks = 240 attention calls. Each call writes 757 new tokens into its own 5,788-slot KV cache and reads all of it, so work done per read touches 7.6× the tokens of work

Runway Characters: Real-Time Conversational Video at Batch Size One Table 1. What real time means for a conversational video character. Lip sync: detectability thresholds, + = audio leads; turn gaps: means (agent: a commercial voice agent). Requirement

Target / reference

Source

Sustained rate Batch size Lip-sync detection Human turn gap Agent turn gap

166.7 ms per chunk one session +45 / −125 ms 208 ms 1.30 s

24 fps compute-bound (§2) ITU-R, 1998 Stivers et al., 2009 Lin et al., 2025a

done per write, in every call. In the original implementation, 29% of the step was such work: re-applying rotary position embeddings (RoPE) to the cached keys, re-quantizing the cache to FP8, and shifting the cache window. The next cost after the GEMMs was the tensor-parallel all-reduce, in series with every block. Attention itself was 6% of the step. Contributions. • A characterization of conversational video generation as a serving workload: a sustained per-chunk rate at batch size one, 240 attention calls per chunk that each re-read a mostly unchanged cache, and a turn budget shared with a voice agent (§2). • A write-time KV-cache design: a ring cache with a device-side write position, RoPE and FP8 quantization applied once at write with per-(layer, head) scales, and a delta rotation at position resets. It removes 91% of cache upkeep (§4). • A fused FP8 GEMM + all-reduce for odd tensorparallel degrees on Hopper and Blackwell that reduces output tiles in the NVLink switch while later tiles are computed (§5). • The production system and its measurement: seven weeks of telemetry, a per-stage turn breakdown, quality against a run-to-run floor, and comparisons with open systems and Blackwell (§6–§8). On four H100s our optimizations without the fused kernel run the diffusion step in 159.3 ms (240.9 ms without them), which halved the GPUs per session against the 8GPU launch; the fused kernel, rolled back once in production and since re-enabled, reaches 147.4 ms. On 53 clips and in 10-minute conversations we detect no degradation in lip-sync scores. In production the worker averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second, and of a 2.1 s median turn our pipeline takes 0.49 s.

2

BACKGROUND AND W ORKLOAD

Few-step AR video diffusion. The Runway Characters model generates one latent frame at a time, conditioned

Table 2. GPU time per AR step on one diffusion GPU (4×H100, TP-3; 10 profiled steps of a live-length session; ms). Cache upkeep: whole-cache RoPE, padding, window shifts and FP8 requantization. Fused: the output projection’s GEMM and all-reduce run as one kernel; FP8 GEMMs is the remaining GEMMs. Component

Baseline

Prod. Fused

FP8 GEMMs Fused GEMM + all-reduce KV-cache upkeep TP all-reduce (NCCL) Activation quantization Attention kernel AdaLN modulation Other

84.4 – 69.7 28.6 19.7 13.7 11.8 11.4

85.9 – 6.2 21.2 16.9 14.1 0.8 8.8

54.1 40.9 6.4 – 18.5 15.0 0.7 3.4

GPU busy

239.2 153.9

138.9

through a KV cache on the frames before it. Sampling takes four denoising passes per latent frame. A step (one AR step) denoises one latent frame, a four-frame chunk, in four forwards at batch size one. The model works on compressed latent frames; a separate decoder network (the decoder of a variational autoencoder, VAE) turns each denoised latent frame into 4 video frames at 704×1088, a step we call decoding. The speech the character says during those frames conditions the step. The model. The model has 60 blocks. Under tensor parallelism (TP) each block’s input projections are columnparallel and its output projection row-parallel, so each block needs one all-reduce. The block linears, which run in every forward, hold ≈57B parameters and run in FP8 with per-token activation and per-channel weight scales. The model keeps one KV cache per layer and denoising pass, so each of a step’s 240 attention calls writes and reads its own. Each has 1,246 global slots, written once per session, and a window of 6 latent frames of 757 tokens: 5,788 slots, all read by attention. Temporal RoPE positions slide by one latent per step and reset every 42 steps, keeping them within a fixed range. The step budget. One AR step is 240 block forwards over 757 tokens: ≈343 TFLOP of GEMMs and ≈30 TFLOP of attention, or 93.3 TFLOP per video frame of useful work (attention over all 5,788 slots). At 24 fps a session sustains 2.24 PFLOP/s, more than one H100’s dense FP8 peak (1.98 PFLOP/s). We estimate that three diffusion GPUs are the minimum for our layout on H100: at TP-3 the FP8 GEMMs alone take 86 ms, and scaling the whole TP-3 step by 3/2 gives ≈230 ms at TP-2, though communication and fixed costs need not scale that way. We run batch size one because the GEMMs already run near two-thirds of FP8 peak at 757 tokens, so a second session in the batch would nearly double the step.

Runway Characters: Real-Time Conversational Video at Batch Size One

Browser

LiveKit (WebRTC)

GPUs 1–3 diffusion transformer, TP-3 GPUs 1–3

diffuse i GPU 0 decode i−1 play i−2 Pacer

features latent

Streaming process voice agent, VAD, 24 fps pacer

shifts its schedule by the delay rather than dropping frames, and the audio inside the frames waits with it.

audio chunks

Why tensor parallelism and a separate decoder GPU. Pipelining layers or denoising passes across G GPUs, as StreamDiffusionV2 (Feng et al., 2026) and Live Avatar (Huang et al., 2025b) do, puts about G× steps pipeline ticks between an audio chunk and its frames: an estimated 3.2 s for StreamDiffusionV2 at four steps on four H100s (§7.5). Sequence parallelism (Fang & Zhao, 2024) would split only 757 query tokens. Tensor parallelism shortens each forward, at one all-reduce per block. Decoding on a diffusion GPU would add its 114 ms to every step, since the TP group waits for its slowest member; on B200 we measured that it misses real time (§7.4).

frames

GPU 0 audio encoder, decoder (latents to video frames)

diffuse i+1

diffuse i+2

decode i play i−1

decode i+1 play i time

166.7 ms

Figure 1. Top: one session on four H100 GPUs. Bottom: in steady state three steps are in flight; decoding of step i−1 into video frames and playback of step i−2 overlap the denoising of step i.

Where the original step went. In the baseline (Table 2) the GEMMs already ran at ≈67% of FP8 peak. Cache upkeep took 69.7 of 239.2 ms (29%): keys were cached before RoPE, so every block of every denoising pass padded the queries to the cache length and re-applied RoPE to all cached keys; FP8 attention re-quantized the whole cache in each of the 240 calls; and the window advanced by copying it. The all-reduce (28.6 ms) followed each down-projection GEMM. The attention kernel took 13.7 ms.

3

S YSTEM OVERVIEW

A session runs on four H100 GPUs (Figure 1). The browser connects over WebRTC to a LiveKit (LiveKit, 2026b) room. A streaming process hosts the connection to a third-party voice agent, a local voice-activity detector for idle handling, and the pacer that publishes frames and audio at 24 fps. GPU 0 runs the audio encoder and the decoder (latents to video frames) and broadcasts each step’s audio features; GPUs 1–3 run the transformer with TP-3 and send the denoised latent back. Per-step schedule. Three steps are in flight: while GPUs 1–3 denoise latent i, GPU 0 decodes latent i−1 (114 ms) and the pacer plays latent i−2. GPU 0 encodes the audio for step i (9 ms) and broadcasts it before decoding latent i−1, so in a live session latent i−1 is decoded only once audio i has arrived. Decode overlaps diffusion fully, leaving a serial remainder of 1.6 ms per step. Every queue has depth one with backpressure. With the queued audio, denoising, decoding and the playout buffer, 3–5 chunks of speech are in flight, which sets the latency of a turn and of an interruption (§7.2). On an underrun the pacer holds the last frame and

4

M OVING W ORK FROM R EAD T IME TO W RITE T IME

The common thread of this section is to do once per write what was done once per read. Figure 2 gives the effect of each change, applied cumulatively in the order the code’s dependencies allow. 4.1

A ring cache with a device-side write position

In the original code (the baseline of Table 2), the cache was one fixed-size allocation per layer and denoising pass, and once full it shifted the whole window left with two overlapping copies per step. We stack all layers into one allocation, keep per-layer views, and write each new frame into a circular slot with index_copy_. The write position is a device tensor, so the compiled step and its CUDA graph capture a pointer rather than a Python integer and replay for every step. A ring is correct only if each cached frame keeps the position it was written with. The original code re-applied RoPE in physical slot order, which stops being chronological once the ring wraps: combined with the ring it silently permutes the temporal positions of cached frames. A test now pins that combination as invalid. 4.2

RoPE at write, and a delta rotation at reset

We apply RoPE to each new token’s queries and keys once, before the cache write, and attention consumes cached keys as they are. This removes the query padding, the full-cache RoPE and the full-length concatenations, about 0.5 GB of memory traffic per block forward. Temporal positions cycle within a fixed range, and every 42 steps every cached frame’s temporal index moves by the same ∆ = −42. The temporal angle of a window frame is affine in its index t, θj (t) = ωj (at + b), so the reset

Runway Characters: Real-Time Conversational Video at Batch Size One

is one rotation of each cached window key by ωj a∆ per frequency j; spatial angles and global tokens do not change. In an FP8 cache the key is dequantized, rotated in fp32 and requantized with its unchanged per-head scale: the rotation identity is exact, and the stored key changes by one FP8 rounding, at most once per key (Appendix G). A reset costs one pass over the window every 42 steps instead of a fullcache RoPE in each of 240 calls per step. It shows up as one slower step in 42 (263 ms in the production configuration, §7.1), which the playout buffer absorbs. 4.3

Scales are calibrated once per session, from the amax of the bf16 cache filled by the first six steps, times a margin of 1.5; values beyond it saturate. Because K and V are projections of RMS-normalized tokens, input gain cannot move their amax, but content can. Rare clipping alone does not bound attention error, so §7.3 compares with a bf16 cache; a rolling amax cost a step (§8). Scales are per head because value heads differ widely in range: one per-tensor scale would leave the quietest heads few significant bits. Fused pre-attention

The remaining work between the up-projection and attention is a chain of small memory-bound operations: split, reshape, RMS-normalize Q and K, RoPE, quantize K and V per head, write them into the ring. Three CuTe DSL kernels (Q, K, V) fuse it; the K and V kernels read the write position from device memory. The fused RMS norm squares in fp32 where the reference squares in bf16, which flips the last bf16 bit of about 2% of elements. At RoPE cancellation points (x0 cos θ ≈ x1 sin θ) that bit dominates, so 0.3% of FP8 codes differ by a few codes while the absolute error stays within two bf16 ulps of the head scale; tests therefore bound absolute, not relative, error. 4.5

Constant folding over a fixed schedule

The four-pass schedule is fixed, so all 60 blocks’ AdaLN modulations are computed in one batched GEMM outside the compiled region and cached across steps (11.8 → 0.8 ms). The four forwards of a step share their conditioning, so RoPE tables and conditioning embeddings are built once per step.

238.1

+AdaLN

233.0 (−5.1)

+RoPE@write

208.9 (−24.1)

+Ring cache

182.1 (−26.8)

+FP8 KV

166.3 (−15.8)

+Cond. cache

166.2 (−0.2)

+Fused pre-attn (prod)

161.3 (−5.0)

+Fused GEMM+AR

150.4 (−10.9)

0 100 200 300 Diffusion p50 per AR step (ms)

FP8 at write

FP8 attention originally recomputed a per-tensor scale and re-quantized the whole cached K and V in every call. We store K and V in float8_e4m3 and quantize only the new frame, with per-(layer, head) scales: the descale granularity that FlashAttention-3’s FP8 kernels take as arguments (Shah et al., 2024; Dao-AILab, 2026), so they need no separate rescaling kernel. Only the queries are quantized per call.

4.4

24 fps budget

Baseline

Figure 2. Cumulative waterfall on one 4×H100 box (TP-3): diffusion time per AR step and the change from the previous configuration: medians over three runs per arm in random order, each a fresh worker and one 180 s session; every arm’s three values lie within 1.0 ms. The fused GEMM + all-reduce is §5.

Effect. The three cache changes save 66.7 of the 76.7 ms the production configuration saves, and cut cache upkeep from 68 to 6 ms. The conditioning cache does not move the step: the host work it removes was already hidden behind GPU work. RoPE at write is also what introduces the reset step; the baseline re-applies RoPE every step and has none.

5

H IDING T ENSOR -PARALLEL C OMMUNICATION

With the cache work gone, the all-reduce after each downprojection was the largest cost after the GEMMs: at TP-3 the FP8 down-projection followed by an NCCL all-reduce of its output takes 208 µs, 240 times per step. Swapping the collective did not help. Replacing NCCL with an NVLink SHARP (NVLS) multimem all-reduce, which offloads reduction and multicast to the NVSwitch, saved 0.4 ms per step. The FP8 GEMM could not write into the symmetric buffer, forcing a copy, and the reduction still started only after the whole GEMM. Fusing the reduction into the GEMM. We extend the fused GEMM + all-reduce of ParallelKittens (Sul et al., 2026), built on ThunderKittens (Spector et al., 2025). A persistent kernel fills all 132 SMs. Compute SMs write each output tile into their symmetric-memory buffer and bump a counter at the tile’s owner rank; once all N partials have landed, the owner reduces the tile in the switch with multimem.ld_reduce and writes the sum to every rank with multimem.st (Figure 6). Tiles are reduced while later tiles are still being computed, so most communication leaves the critical path. Four changes to the upstream kernel

were needed: 1. FP8 GEMM. Upstream’s Hopper kernel is bf16, and a bf16 down-projection costs more than the whole allreduce (+8.3 ms per step). Ours takes FP8 inputs with per-token and per-channel scales, accumulates in fp32 and reduces in bf16, since multimem.ld_reduce has no FP8 form on Hopper.

Speedup over NCCL (×)

Runway Characters: Real-Time Conversational Video at Batch Size One

2. Vectorized reduction. Upstream moved 4 bytes per thread per round trip and needed 32 of 132 SMs to keep pace; we use the 16-byte ld_reduce.v4.bf16x2 form with two accesses in flight. With a static com3. Shared tail queue. pute/communication split the tiles of the last GEMM wave, finishing together, waited on a few communication SMs (34 of 176 µs). Compute SMs now drain into a shared reduction queue, which made the split nearly irrelevant: 2 communication SMs give 153.0 µs, 8 give 155.4 µs; 16 or more starve the GEMM (184–187 µs). 4. Odd world sizes. TP-3 and TP-7 groups live on devices 1..N , so the world size is a template parameter and symmetric memory comes from PyTorch rather than upstream’s handshake, which assumes rank equals device index. Microbenchmarks. At the production shape (M =757 tokens, TP-3) the fused kernel takes 153.8 µs against 208.2 µs for _scaled_mm + NCCL. Its own GEMM is 5–22% slower than _scaled_mm; in five runs timing both alone, the fused call hides 54–68% of the all-reduce’s time. Fusion needs a GEMM large enough to hide the reduction under and a reduction large enough to matter: fused vs NCCL is 0.84× at M =64, 1.35× at 757 and 1.01× at 11,136. The benefit grows with the TP degree (Figure 3), as the ring’s traffic grows and the GEMM shrinks. Blackwell. The Blackwell kernel, which we run on B200, rebuilds the GEMM on tcgen05 MMAs with 2-CTA clusters and 256×256 tiles in tensor memory and keeps the Hopper reduction, at a finer grain: reducing whole tiles left most compute clusters idle while the last wave was reduced, so it reduces 128×64 sub-tiles. B200’s FP8 GEMM is 1.9× faster than H100’s but its GEMM + NCCL call only 1.76×, so at TP-2 there is less GEMM to hide the reduction under (1.11×); from TP-4 up the gain is 1.5–1.8× (Blackwell pitfalls: Appendix F). Correctness under compilation. Two bugs passed every eager test and failed only when compiled. The operator wrote the symmetric buffer without declaring it (declaring it lets functionalization substitute a clone in ordinary memory,

H100 B200

1.8 1.6 1.4 1.2

NCCL (scaled_mm + all_reduce)

1.0 2

3

4 5 6 Tensor-parallel degree

7

8

Figure 3. Fused GEMM + all-reduce vs FP8 GEMM then NCCL all-reduce at the production shape (M =757, each rank’s slice of the down-projection; Appendix D). H100 TP-2 to TP-4 were measured on a 4-GPU box, TP-5 to TP-8 on an 8-GPU node.

where multimem silently does not reduce), and all calls at one shape share the buffer, so Inductor legally moved one call’s read past the next call’s write: 96% of elements wrong with 60 calls in a graph. The operator now returns a copy of the result, at no measurable cost. And CUDA graph trees copy unmarked symmetric buffers into their own pool, so the captured kernel wrote partials its peers never read; the buffers must be marked as static. Status. The kernel served production in August 2026 until, after a PyTorch and NCCL upgrade, workers ran out of memory binding NVLS multicast buffers; reverting to NCCL was a configuration change (§8). The kernel kept buffers for every token count, and a session’s first forward has a prompt-dependent length, so workers bound new buffers per prompt length, which fixed-prompt benchmarks never do. Now only the steady-state shape is fused and first forwards use NCCL. After a 15-hour canary in which every session with a measured worker fps averaged 24.0 fps or more, the kernel was re-enabled on 8 October 2026. In the 15 days since, 0.7% of sessions of at least a second averaged 24 fps or less, against 7–23% in each of the five weeks before, and 0.08% averaged 23.5 fps or less.

6

O PERATING I T IN P RODUCTION

Deployment. Runway Characters launched on eight H100s per session and then served on four B200s, both without the optimizations of §4; with them it serves on four H100s, half the launch configuration. Workers pull sessions from a queue and serve one at a time, so capacity is the number of warm workers. Worker start-up. A worker takes a median 6.5 min from pod creation to ready, 4.2 min of it compiling and warming

Runway Characters: Real-Time Conversational Video at Batch Size One

up, much longer than a typical session, so capacity cannot follow demand per session and must be held warm. Session start. A new session resets diffusion state, not the worker: it encodes the new reference image and prompt and clears the dynamic cache (0.4–0.6 s), while compiled graphs and static buffers persist. The first six steps run eagerly while the cache fills (4.0–4.5 s). In live sessions frames were released 5.4–9.9 s after the worker began the task (queue time excluded). Health. The worker measures its own fps and logs every session that streams below target. A policy that replaces a worker after two consecutive slow sessions is implemented but switched off until its decisions have been checked against production logs.

7

E VALUATION

Setup. Unless noted, we use one 4×H100 SXM box with NVSwitch, PyTorch 2.13 (CUDA 13.0) and NCCL 2.29.7, in the production layout. A benchmark harness drives the production worker’s components in production order (setup and warm-up, then per session the task reset and an AR step loop fed from recorded speech) without the queue, LiveKit or voice agent. It runs unpaced unless stated, so its step period is capacity. Each arm is a fresh worker; sessions are 300 s (≈1,800 steps) with the window full and resets included, and steady state excludes the first 60 and the profiled steps. Baseline turns every optimization off; it is the launch software, which served on eight GPUs per session, replayed here on four. Production is the set deployed while we measured, and fused adds the fused GEMM + all-reduce. We also use live sessions on the staging app and seven weeks of production telemetry. Production results use performance telemetry only: aggregate frame-rate and timing counters and per-session frame-rate records; no user audio, video, images or text were accessed. Live sessions and latency runs were the authors’ own, driven by synthesized speech. 7.1

Real-time delivery

Benchmark. The production configuration runs the diffusion step in 159.3 ms, 1.51× faster than the baseline, which cannot run in real time on this layout; the fused kernel reaches 1.63× (Table 3, Figure 4). The period exceeds diffusion by only 1.6 ms, because decoding and audio encoding overlap it. The steps over budget are the resets (2.4%) and a few of 167–189 ms (0.4%); with them the configuration sustains 24.5 fps, 2% above real time. The contract is therefore a sustained rate with bounded excursions: a reset runs 96 ms over budget, and a simulated 24 fps player sees no stalls when generation may lead playback by two steps, one finished step queued (with one, each reset stalls it for

Table 3. Live-length benchmark sessions (2 × 300 s per arm, mean of the two). Capacity is 4000/period p50; sustained is frames generated over elapsed time, resets included. Per AR step Diffusion p50 (ms) Period p50 (ms) Period p90 (ms) Capacity (fps) Sustained (fps) Reset step, every 42 (ms) TFLOP/s per diffusion GPU % of dense FP8 peak

Baseline

Prod. Fused

240.9 159.3 242.8 161.0 245.2 163.4 16.5 24.8 16.4 24.5 – 263 517 781 26.1 39.5

147.4 149.1 151.7 26.8 26.4 245 844 42.7

Table 4. Production, 4×H100 workers, 16 Aug – 4 Oct 2026: perseries sums of observed Prometheus counters (Appendix A); fractions at bucket edges. Sessions ≥1 s, worker avg. ≤23.5 fps 0.55% Report intervals ≤23.5 / ≤20 fps 2.3% / 0.8% Frames (1 in 5) within 1/10/100 ms of slot 99.52/99.84/99.98%

≈100 ms; Appendix B). Live sessions on stage matched the benchmark within 2 ms of diffusion time, and the browser showed 24 fps with no dropped frames. Production. We report 4×H100 workers from 16 August to 4 October 2026 (the fused kernel on until 31 August; Table 4). While streaming, the worker reports its fps per 10 s interval and every fifth frame’s drift from its slot on the pacer’s clock; after a late frame the pacer shifts its clock, so a stall counts once, at its length. The interval in which a stream ends also counts its idle tail, so intervals overstate slow generation (in three hours of logs: 2.2% at 23.5 fps or less, against 0.3% of logged windows cut at the last frame). Of the sessions that streamed at least a second, 0.55% averaged 23.5 fps or less on the worker. 0.16% of sampled frames (sampling meets resets at every phase) were more than 10 ms off their slot, a quarter of the 0.6% that a stall at every reset would cause, so the playout buffer absorbs most resets in production too. 7.2

Latency

We joined server traces with the browser’s trace for 56 turns in 12 live sessions and excluded five whose first speaking frame fell in the profiler’s window; of the other 51, 46 (11 sessions) ran the production configuration and five (one session) the fused kernel (Figure 5). Server stages and client totals are each on one clock; only the split between the first and last stage depends on the clock-sync offset (±100 ms). The median turn, from the user’s end of speech to the avatar’s audio at the client, is 2.13 s (p10–p90 1.45–3.00 s) and the voice agent takes 1.42 s, over the 43 turns with a user endof-speech mark. Our pipeline takes 493 ms from the agent’s

Runway Characters: Real-Time Conversational Video at Batch Size One

position reset every 42 steps (7 s of video)

225 200 175

24 fps budget

150

Fraction of steps ≤ x

250

1.0

warm-up

Step period (ms)

275

0.8

24 fps budget

0.6 0.4 0.2

Baseline Production Fused

0.0 0

200 400 600 800 1000 1200 1400 1600 1800 AR step index (production, one 300 s session)

150 175 200 225 250 275 Steady-state step period (ms)

Figure 4. Left: step period over one 300 s production-configuration session; the spikes are position resets every 42 steps. Right: distribution of the steady-state period; the dashed line is the 24 fps budget.

1418

0

250

289

188

500 750 1000 1250 1500 1750 2000 Median latency per stage (ms)

Voice agent Hand-off

Wait + diffusion Playout buffer

Decode to video Network + client

Figure 5. Per-stage median turn latency, production configuration (46 turns, 11 live sessions; 43 for stages from the user’s end of speech), from the user’s end of speech to the avatar’s audio at the client. Stage medians need not sum to the median total.

first audio to the first speaking frame captured by LiveKit (p10–p90 421–562 ms); the stage medians are 289 ms to the denoised latent, waiting for the in-flight step included, 117 ms decoding, 14 ms hand-off and 74 ms in the playout buffer. Delivery takes another ≈190 ms. The fused kernel does not shorten the turn: under pacing the leader decodes latent i only after audio i+1 arrives, so saved diffusion time becomes waiting. It buys capacity, not latency. When the user interrupts, the 3–5 voiced latents already in flight still play, and the last voiced frame reaches LiveKit on the server a median 0.78 s after the agent’s interruption signal (0.70–1.01 s, 11 cases), not timed at the client. 7.3

Quality

Three optimizations change numerics: the FP8 cache, the fused pre-attention and the fused all-reduce. We generated all 53 clips of our evaluation set (50 reference images in photoreal, cartoon, claymation, fantasy and non-human styles; 954 s of speech) with baseline, production and fused, with production four times to measure run-to-run variation, and with a bf16 cache to isolate the FP8 cache. Lip sync is Sync-

Table 5. Lip sync: paired per-clip differences (mean, 95% bootstrap interval) over the 51 clips SyncNet scores in every run. Absolute values: LSE-D 8.26–8.29 (lower is better), LSE-C 7.32–7.36. Last row: each of three reruns. LSE-D LSE-C Prod.−base −.02 [−.07,.02] +.03 [−.01,.07] Fused−base .00 [−.04,.04] +.01 [−.03,.06] bf16 KV−prod. +.02 [−.02,.06] −.01 [−.05,.03] Reruns−prod. −.01/+.01/+.02 +.01/−.01/−.01

Net (Chung & Zisserman, 2016) distance and confidence (LSE-D, LSE-C) with Euclidean distance on video resampled to 25 fps, as in the standard protocol (Prajwal et al., 2020) but with our own crop. SyncNet does not track every style, so arms are compared on the 51 clips it scores in every run (Appendix C). We detected no degradation in lip sync (Table 5). Mean differences are at most 0.03 and every 95% interval lies within ±0.08; the four production runs span 0.024 LSE-D and 0.018 LSE-C, so differences of this size also occur between repeated runs. SyncNet is weaker on the 15 stylized clips it tracks (LSE-C 5.5, photoreal 8.1), with intervals 2–5× as wide. The intervals bound the changes but do not prove equivalence. Long sessions. The clips are short (median 18.5 s, at most 40 s), while production sessions can last much longer. As a targeted stress test we ran four 10-minute sessions per arm that alternate speaking and listening as the worker does (Appendix C): a photoreal and a non-human character, each calibrated once while listening and once while speaking, for production, a bf16 cache and a production rerun on identical inputs. Over 130 speaking periods the bf16 cache changes LSE-D by +0.02 [−0.10, +0.14] and the rerun by +0.05 [−0.05, +0.15]. These intervals resample segments, not sessions, so they understate the uncertainty. Per session (Appendix C) LSE-D moves as much with the rerun as with

Runway Characters: Real-Time Conversational Video at Batch Size One

the bf16 cache, and neither difference grows from the first two minutes to the last beyond the rerun’s own variation; four sessions cannot exclude a trend. Pixels stop separating runs after seconds. An AR model turns any numeric difference, even nondeterminism, into a different trajectory: after 10 s two production runs differ in PSNR as much as any two configurations (Appendix H), so an optimization must be judged on task metrics against a measured run-to-run floor. 7.4

Scaling and layout

Production packs two 4-GPU workers per 8-GPU node; two fused workers started together on one node ran diffusion at 148.6 and 148.1 ms (147.4 ms alone). On eight H100s (decoder + TP-7, heads split unevenly; Appendix D) diffusion falls to 118.2 ms (98.9 ms fused), but the period stops at 128.7 ms (31 fps): the single decoder GPU becomes the bottleneck, and the fused kernel’s 19 ms does not raise throughput. A channels-last decoder (cuDNN’s native NDHWC kernels) cuts decoding from 117 to 96 ms, lifting the 8-GPU period to 119.4 and 109.0 ms fused. Sharing a GPU between decoding and diffusion misses real time on B200: with the fused kernel, two B200s take 195.0 ms even with an FP8 decoder, against 148.8 ms on three with a dedicated bf16 decoder. The shared GPU stays at its 1000 W power cap, slowing its diffusion kernels by 18–26% and stalling the other rank in every all-reduce. 7.5

Open-source systems on the same hardware

We ran StreamDiffusionV2 (Feng et al., 2026) and Live Avatar (Huang et al., 2025b), 14B models built on Wan 2.1 and Wan 2.2-S2V, from their public code on the same box (Table 6; recipes in Appendix E). At four steps StreamDiffusionV2’s four-stage pipeline either misses real time (480×832) or adds an estimated 2.2 s of latency (5122 ), since a chunk crosses 16.5 pipeline ticks. Live Avatar’s released recipe needs five GPUs; on four it reaches 21.3 fps, and a 3+1 layout reaches 42.3 fps only with a three-step schedule it was not distilled for, at 1.1 s latency. At 704×1088 the 14B model needs 166 TFLOP per frame against our 93 and runs at 6.3 fps. These are feasibility comparisons of complete systems on four GPUs, our production unit: model, tokenizer, resolution, schedule and layout all differ, and we did not compare output quality, so the case for our techniques rests on the ablations above. 7.6

to an avatar through its LiveKit plugin, while a viewer on the same host and clock times the avatar’s first voiced audio (Appendix I); a mock avatar with a known delay shows the harness adds 112 ms. Runway Characters takes 0.56 s at the median (p10–p90 0.46–0.78 s, including the 112 ms), close to the 0.68 s sum of the live sessions’ stage medians from agent audio to client audio (493 + ≈190 ms, §7.2). The phase at which agent audio meets the 166.7 ms step contributes to the spread. We report our public-API measurement and local open-system benchmarks; other commercial avatar APIs are not included. The open Live Avatar encodes its whole audio file up front, so as released it cannot be driven live. Adding the 112 ms to its paced latency on five H100s (1.41 s from a block’s scheduled audio arrival to its frames on the host) gives an estimate of 1.52 s if transport simply adds; for our system the same sum falls 150 ms short of the measurement. 7.7

Blackwell

On an 8×B200 node with the same stack, three B200s run production precision in real time: 154.3 ms per step (decoder + TP-2), 142.9 ms with the fused kernel and a bf16 cache; with those, four take 107.4 ms and eight, with an FP8 decoder too, 65.5 ms. Two H100 optimizations behave differently. The FP8 cache saves 17.0 ms on H100 but costs 2.4 to 6.7 ms on B200, where our build has no FP8 attention that takes its per-head scales, so the cache is dequantized whole before every call. cuDNN’s MXFP8 attention does not fit per-write quantization (Appendix G). The fused kernel matters most where decoding is off the critical path: 1.20× end to end at TP-7. With NVFP4 matmuls (Alvarez et al., 2025) and FP8 decoding two B200s reach real time (141.4 ms), but NVFP4 output sits ≈3 dB below the run-to-run PSNR floor, so we treat it as a speed bound.

8

L ESSONS

Profile the current build. The multimem all-reduce was sized from an old trace under a heavy profiler (≈150 µs per all-reduce); on the current build it was 76 µs, partly waiting on the slowest rank; only fusion recovered it.

Test custom collectives under the compiler. Both serious fused-kernel bugs (§5) were invisible in eager mode, where issue order is execution order. Every kernel test now chains calls under torch.compile with CUDA graphs.

End-to-end latency

Published figures cover different intervals (Appendix I). To measure Runway Characters through its public API we built a harness around a LiveKit Agents (LiveKit, 2026a) worker that sends 30 recorded agent responses (about 3, 8 and 15 s)

One synchronization per step costs a step. One .item() per step, to check a rolling FP8 amax, took the step from 168.5 to 278 ms by stopping the CPU from running ahead of the GPU.

Runway Characters: Real-Time Conversational Video at Batch Size One Table 6. Real-time video systems on one 4×H100 box. fps is capacity (unpaced). Latency (p50): a chunk’s arrival to its frames on the host, with arrivals paced at the fps in parentheses (u: unpaced, from rank 0’s pickup). TFLOP/s per GPU: model TFLOP per frame × capacity fps ÷ 4. PP: pipeline over layers; TPP: pipeline over denoising passes; + dec.: a GPU of its own decodes latents to video frames; shared dec.: decoding shares a diffusion GPU. † Re-enabled in production on 8 October 2026. ‡ Index-matched; alignment of the decoded output unvalidated. System Runway Characters, prod. Runway Characters, fused† StreamDiffusionV2 StreamDiffusionV2 StreamDiffusionV2 Live Avatar (released) Live Avatar (3-step) Live Avatar

Params Resolution Steps Layout 57B 704×1088 57B 704×1088 14B 480×832 14B 512×512 14B 512×512 14B 384×704 14B 384×704 14B 704×1088

Keep the plain path behind every switch. The fused kernel worked for weeks, then failed in production on input variety our benchmarks lacked (§5). Every optimization has a switch in one configuration object that keeps the plain path, so turning it off was a configuration change.

Measure the measurement. Two instruments misled us until we cross-checked them. GPU spans timed from a CUDA event recorded at worker start had drifted 58–65 ms from host time hours later, so we re-anchor them to host events on the same rank. And the browser’s “avatar audio ended” marker follows turn messages, not sound, so bargein latency starts from the agent’s interruption signal.

9

fps Real time

4 TP-3 + dec. 24.8 4 TP-3 + dec. 26.8 4 PP-4 20.6 4 PP-4 30.2 1 PP-4 57.2 4 TPP-4, shared dec. 21.3 3 TPP-3 + dec. 42.3 4 TPP-4, shared dec. 6.3

R ELATED W ORK

Real-time autoregressive video diffusion. CausVid (Yin et al., 2025) and Self Forcing (Huang et al., 2025a) distill bidirectional video diffusion into few-step causal generators that extend video through a KV cache. LongLive (Yang et al., 2026), Rolling Forcing (Liu et al., 2026), Deep Forcing (Yi et al., 2026) and Infinity-RoPE (Yesiltepe et al., 2026) reach long horizons with attention sinks, cache recaching and positional schemes; Deep Forcing and InfinityRoPE re-position cached keys as the window moves, and LongLive’s later relative-RoPE code (NVlabs, 2026) stores unrotated keys and applies RoPE at every read, work we move to write time. LongLive-2.0 (Chen et al., 2026) and Matrix-Game 3.0 (Wang et al., 2026) also give decoding its own GPU, and Quant VideoGen (Xi et al., 2026) and Forcing-KV (Ji et al., 2026) quantize AR video caches. Seaweed APT2 (Lin et al., 2025b) (8B, eight H100s, 720p, 24 fps), MAGI-1 (Sand.ai et al., 2025) (24B, 24 H100s) and Krea Realtime (Millon, 2025) (14B, one B200, 11 fps) stream video. These works improve models and schedules; we serve a larger model under a conversational deadline.

Latency TFLOP/s/GPU

yes 293 ms (24) yes 292 ms (24) no 3.20 s (u)‡ yes 2.18 s (u)‡ yes 0.34 s (u)‡ no 1.43 s (20) yes 1.13 s (25) no –

579 625 257 227 108 205 305 263

Real-time talking avatars. Live Avatar (Huang et al., 2025b), SoulX-FlashTalk (Shen et al., 2025), SoulXLiveAct (Zhen et al., 2026), TalkingMachines (Low & Wang, 2025), StreamAvatar (Sun et al., 2026) and AvatarForever (Li et al., 2026a) animate faces from speech with 5– 22B models on one to eight GPUs, several with a dedicated decoder GPU. Humans answer after a mean 208 ms (Stivers et al., 2009); from the end of user speech, Full-DuplexBench (Lin et al., 2025a) measures 0.27 s for Moshi (Défossez et al., 2024) and 1.30 s for a commercial voice agent. Diffusion serving. StreamDiffusionV2 (Feng et al., 2026) serves streaming video-to-video with a pipeline across GPUs and a host-managed ring-buffer cache, and SlackServe (Xia et al., 2026) manages resources and quality across streaming video requests. xDiT, PipeFusion and DistriFusion (Fang et al., 2024; 2025; Li et al., 2024) parallelize large images and videos for offline requests, and USP (Fang & Zhao, 2024) long-context sequences. Overlapping communication with computation. Our tensor partitioning follows Megatron-LM (Shoeybi et al., 2019). Communication has been overlapped with computation by fusing collectives into GEMM kernels (Chang et al., 2024; Zheng et al., 2025b;a), with hardware support (Pati et al., 2024), and by pipelining decomposed operations (Zhu et al., 2025; Wang et al., 2024). ParallelKittens (Sul et al., 2026) fuses GEMM and all-reduce with inswitch reduction, and we extend its Hopper kernel; NCCL offers NVLS collectives (NVIDIA, 2026b;c); TensorRTLLM (NVIDIA, 2026d) ships a fused GEMM + all-reduce; concurrent work extends it to visual generation with FP8 block scales (NVIDIA, 2026a) and builds multi-node fused kernels (Mao et al., 2026). We contribute an FP8 variant for odd TP degrees with a shared tail queue, a Blackwell port, and its behavior inside a compiled, CUDA-graphed production model.

Runway Characters: Real-Time Conversational Video at Batch Size One

KV caches, positions and quantization. PagedAttention (Kwon et al., 2023) and attention sinks (Xiao et al., 2024) shaped KV management for LLM serving. Caching keys after RoPE (Su et al., 2024) and shifting them is established in LLM engines (llama.cpp contributors, 2023; Hugging Face, 2024), as are FP8 and low-bit KV caches (vLLM Team, 2026; Liu et al., 2024; Hooper et al., 2024) and FP8 attention with per-head descales (Shah et al., 2024; DaoAILab, 2026). Concurrent work fuses QK-norm, RoPE and the cache write (FlashInfer contributors, 2026) and precomputes AdaLN tables (Li et al., 2026b). In our model the positions slide every step and reset every 42 steps, which is why the launch configuration re-applied RoPE at read time; caching post-RoPE keys needed the delta rotation, and CUDA-graph capture a device-side write position.

10

C ONCLUSION

Serving a 57B video diffusion transformer to one user at a time, at a sustained 166.7 ms per chunk, differs from serving it for throughput. At batch size one the cost that matters is work repeated in each of a step’s 240 attention calls over a cache that is mostly unchanged, and the communication in series with each block. Moving that work to write time took the diffusion step from 240.9 to 159.3 ms and halved the GPUs per session; folding the all-reduce into the GEMM reached 147.4 ms, with no detected degradation in lip sync. In production the worker averages above 23.5 fps in 99.5% of sessions of at least a second, and of a 2.1 s turn our pipeline takes half a second. What remains is mostly outside the step: the voice agent, the pipeline depth that delays barge-in, and a fleet sized for its peak.

R EFERENCES Alvarez, E., Almog, O., Chung, E., Layton, S., Stosic, D., Krashinsky, R., and Aubrey, K. Introducing NVFP4 for efficient and accurate low-precision inference. NVIDIA Technical Blog, https://developer.nvidia.c om/blog/introducing-nvfp4-for-efficie nt-and-accurate-low-precision-inferen ce/, 2025. June 24, 2025. Chang, L.-W., Bao, W., Hou, Q., Jiang, C., Zheng, N., Zhong, Y., Zhang, X., Song, Z., Yao, C., Jiang, Z., Lin, H., Jin, X., and Liu, X. FLUX: Fast software-based communication overlap on GPUs through kernel fusion. arXiv preprint arXiv:2406.06858, 2024. URL https: //arxiv.org/abs/2406.06858. Chen, Y., Wang, L., Huang, W., Yang, S., Zhang, B., Xiao, Y., Chu, R., Mao, W., Hu, Q., Liu, S., Zhao, Y., Mao, H., Chen, Y.-C., Xie, E., Qi, X., and Han, S. LongLive2.0: An NVFP4 parallel infrastructure for long video

generation. arXiv preprint arXiv:2605.18739, 2026. URL https://arxiv.org/abs/2605.18739. Chern, E., Hu, Z., Tang, B., Su, J., Chern, S., Deng, Z., and Liu, P. LiveTalk: Real-time multimodal interactive video diffusion via improved on-policy distillation. arXiv preprint arXiv:2512.23576, 2025. URL https://ar xiv.org/abs/2512.23576. Chung, J. S. and Zisserman, A. Out of time: Automated lip sync in the wild. In Computer Vision – ACCV 2016 Workshops, 2016. Dao-AILab. FlashAttention-3 Hopper API (hopper/flash_api.cpp). GitHub DaoAILab/flash-attention, commit 99589e5a, 2026. URL https://github.com/Dao-AILab/fla sh-attention/blob/99589e5a669ea4688c afd4c71ddb947456454ff4/hopper/flash_ api.cpp. Q/K/V descales of shape (batch, KV heads); the build we deploy. Défossez, A., Mazaré, L., Orsini, M., Royer, A., Pérez, P., Jégou, H., Grave, E., and Zeghidour, N. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037, 2024. URL https: //arxiv.org/abs/2410.00037. Fang, J. and Zhao, S. USP: A unified sequence parallelism approach for long context generative AI. arXiv preprint arXiv:2405.07719, 2024. URL https://arxiv.or g/abs/2405.07719. Fang, J., Pan, J., Sun, X., Li, A., and Wang, J. xDiT: an inference engine for diffusion transformers (DiTs) with massive parallelism. arXiv preprint arXiv:2411.01738, 2024. URL https://arxiv.org/abs/2411.0 1738. Fang, J., Pan, J., Li, A., Sun, X., and Wang, J. PipeFusion: Patch-level pipeline parallelism for diffusion transformers inference. In Advances in Neural Information Processing Systems (NeurIPS), 2025. Feng, T., Li, Z., Yang, S., Xi, H., Li, M., Li, X., Zhang, L., Yang, K., Peng, K., Han, S., Agrawala, M., Keutzer, K., Kodaira, A., and Xu, C. StreamDiffusionV2: A streaming system for dynamic and interactive video generation. In Proceedings of Machine Learning and Systems (MLSys), 2026. FlashInfer contributors. feat(rope): fuse QK norm, RoPE, and paged KV append. GitHub pull request flashinferai/flashinfer #5405, September 2026. URL https:// github.com/flashinfer-ai/flashinfer/p ull/5405. Opened 22 September 2026; open as of 2026-10-04.

Runway Characters: Real-Time Conversational Video at Batch Size One

Hooper, C., Kim, S., Mohammadzadeh, H., Mahoney, M. W., Shao, Y. S., Keutzer, K., and Gholami, A. KVQuant: Towards 10 million context length LLM inference with KV cache quantization. In Advances in Neural Information Processing Systems (NeurIPS), 2024. Huang, L., Wu, Z.-F., Shi, Y., Wang, W., Feng, M., He, J., Xie, C.-W., Liu, Y., Zhou, J., Wang, A., Zhang, B., Ai, B., Liang, C., Yu, C., Zhong, C., Qi, J., Zhu, K., Li, P., Zhang, P., Zhang, W., Cheng, X., Huang, Y., Zheng, Y., Bao, Y., Wang, Y., and Bi, Z. Wan-Streamer v0.2: Higher resolution, same latency. arXiv preprint arXiv:2607.04443, 2026. URL https://arxiv.org/abs/2607.0 4443. Huang, X., Li, Z., He, G., Zhou, M., and Shechtman, E. Self Forcing: Bridging the train-test gap in autoregressive video diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2025a. Huang, Y., Guo, H., Wu, F., Wang, W., Zhang, S., Huang, S., Gan, Q., Liu, L., Zhao, S., Chen, E., Liu, J., and Hoi, S. Live Avatar: Streaming real-time audio-driven avatar generation with infinite length. arXiv preprint arXiv:2512.04677v6, 2025b. URL https://arxiv. org/abs/2512.04677v6. Version 6, 31 July 2026. Hugging Face. Utilities for generation: SinkCache and StaticCache (Transformers v4.40.0 documentation). Hugging Face documentation, 2024. URL https://hugg ingface.co/docs/transformers/v4.40.0 /en/internal/generation_utils. Accessed 2026-10-04. ITU-R. Relative timing of sound and vision for broadcasting. Recommendation ITU-R BT.1359-1, International Telecommunication Union, November 1998. URL https://www.itu.int/rec/R-REC-BT.1359 -1-199811-I/en. Accessed 2026-10-04. Ji, Y., Zhong, Z., Zhang, J., Yang, Q., Jin, X., Qin, Y., Luo, W., Mao, S., Liu, W., and Li, H. Forcing-KV: Hybrid KV cache compression for efficient autoregressive video diffusion models. arXiv preprint arXiv:2605.09681, 2026. URL https://arxiv.org/abs/2605.09681. Kravtsov, A., Ziganshin, D., Poletaev, V., Balitskiy, G., Tikhonova, A., Burkov, E., and Lebedev, V. AVTR1: Open stack for real-time interactive avatars. arXiv preprint arXiv:2609.22913, 2026. URL https://ar xiv.org/abs/2609.22913. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), pp. 611–626, 2023. doi: 10.1145/3600006.3613165.

Li, M., Cai, T., Cao, J., Zhang, Q., Cai, H., Bai, J., Jia, Y., Li, K., and Han, S. DistriFusion: Distributed parallel inference for high-resolution diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7183–7193, 2024. Li, R., Yang, T., Ma, Z., Ai, F., Wen, S., and Zhang, L. Avatar-Forever: Decoupled parallel training for high-quality real-time infinite avatars. arXiv preprint arXiv:2608.12107, 2026a. URL https://arxiv.or g/abs/2608.12107. Li, Y., Yu, J., Chen, J., Li, H., Xue, S., Liu, H., Luo, P., Han, S., and Xie, E. Sol-H3: Recursive self-improvement for MiniMax-H3 inference acceleration on Sol-Engine across cloud and edge. arXiv preprint arXiv:2609.35110, 2026b. URL https://arxiv.org/abs/2609.35110. Lin, G.-T., Lian, J., Li, T., Wang, Q., Anumanchipalli, G., Liu, A. H., and yi Lee, H. Full-Duplex-Bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities. In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 1–8, 2025a. doi: 10.1109/ASRU65441.2025.11433838. Lin, S., Yang, C., He, H., Jiang, J., Ren, Y., Xia, X., Zhao, Y., Xiao, X., and Jiang, L. Autoregressive adversarial post-training for real-time interactive video generation. In Advances in Neural Information Processing Systems (NeurIPS), 2025b. Liu, K., Hu, W., Xu, J., Shan, Y., and Lu, S. Rolling Forcing: Autoregressive long video diffusion in real time. In International Conference on Learning Representations (ICLR), 2026. Liu, Z., Yuan, J., Jin, H., Zhong, S., Xu, Z., Braverman, V., Chen, B., and Hu, X. KIVI: A tuning-free asymmetric 2bit quantization for KV cache. In Proceedings of the International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 32332–32344, 2024. LiveKit. LiveKit Agents: A framework for realtime voice and video AI agents. GitHub release livekitagents@1.8.3, 2026a. URL https://github.c om/livekit/agents/releases/tag/livek it-agents@1.8.3. Accessed 2026-10-05. LiveKit. LiveKit: End-to-end realtime stack for connecting humans and AI. GitHub repository livekit/livekit, 2026b. URL https://github.com/livekit/livek it. Accessed 2026-10-04. llama.cpp contributors. llama_kv_cache_seq_shift delta does not appear to be calculated properly. GitHub issue

Runway Characters: Real-Time Conversational Video at Batch Size One

ggml-org/llama.cpp #3825, October 2023. URL https: //github.com/ggml-org/llama.cpp/issue s/3825. Opened 28 October 2023, closed 29 October 2023. Accessed 2026-10-04. Low, C. and Wang, W. TalkingMachines: Real-time audiodriven FaceTime-style video via autoregressive diffusion models. arXiv preprint arXiv:2506.03099, 2025. URL https://arxiv.org/abs/2506.03099. Mao, Z., Zhang, Y., Chew, S. W., Ma, S., Raiciu, C., Zhou, Y., Shenker, S., and Stoica, I. mKernel: Fast multi-GPU, multi-node fused kernels. arXiv preprint arXiv:2609.13585, 2026. URL https://arxiv.or g/abs/2609.13585. Millon, E. Krea Realtime 14B: Real-time, long-form AI video generation. Krea blog, October 2025. URL https: //www.krea.ai/blog/krea-realtime-14b. Code: https://github.com/krea-ai/realti me-video. Accessed 2026-10-04.

overlap of compute & collectives. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS), pp. 1146–1164, 2024. doi: 10.1145/3620665.3640410. Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., and Jawahar, C. V. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM International Conference on Multimedia, 2020. Runway Research and Engineering Team. Building Runway Characters: Real-time conversational video agent from a single image. Runway engineering blog, May 2026. URL https://runway.com/news/engineering/ building-runway-characters. Published 4 May 2026. Accessed 2026-10-04.

NVIDIA. [feat] Extend Fused GeMM+AllReduce support into VisualGen, Add FP8 BlockScales variant. TensorRTLLM GitHub pull request #19518, September 2026a. URL https://github.com/NVIDIA/Tensor RT-LLM/pull/19518. Opened 21 September 2026; open as of 2026-10-04.

Sand.ai, Teng, H., Jia, H., Sun, L., Li, L., Li, M., Tang, M., Han, S., Zhang, T., Zhang, W. Q., Luo, W., Kang, X., Sun, Y., Cao, Y., Huang, Y., Lin, Y., Fang, Y., Tao, Z., Zhang, Z., Wang, Z., Liu, Z., Shi, D., Su, G., Sun, H., Pan, H., Wang, J., Sheng, J., Cui, M., Hu, M., Yan, M., Yin, S., Zhang, S., Liu, T., Yin, X., Yang, X., Song, X., Hu, X., Zhang, Y., and Li, Y. MAGI-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505.13211, 2025. URL https://arxiv.org/abs/2505.1 3211.

NVIDIA. NCCL: Optimized primitives for collective multiGPU communication. GitHub repository NVIDIA/nccl, 2026b. URL https://github.com/NVIDIA/nc cl. Accessed 2026-10-04.

Shah, J., Bikshandi, G., Zhang, Y., Thakkar, V., Ramani, P., and Dao, T. FlashAttention-3: Fast and accurate attention with asynchrony and low-precision. In Advances in Neural Information Processing Systems (NeurIPS), 2024.

NVIDIA. NCCL environment variables: NCCL_NVLS_ENABLE (NVLink SHARP). NCCL 2.32.3 documentation, 2026c. URL https://docs .nvidia.com/deeplearning/nccl/user-g uide/docs/env.html#nccl-nvls-enable. Accessed 2026-10-04. NVIDIA. TensorRT-LLM. GitHub repository NVIDIA/TensorRT-LLM, 2026d. URL https://gi thub.com/NVIDIA/TensorRT-LLM. Accessed 2026-10-04. NVlabs. LongLive: relative-RoPE causal model (causal_model_infinity.py). GitHub NVlabs/LongLive, commit 7add41d6 (10 January 2026), 2026. URL https://github.com/NVlab s/LongLive/blob/7add41d68fdc665e1d98 04cdd65a6a1ee7a6f467/wan/modules/cau sal_model_infinity.py. Stores unrotated keys and applies RoPE when reading the cache. Pati, S., Aga, S., Islam, M., Jayasena, N., and Sinclair, M. D. T3: Transparent tracking & triggering for fine-grained

Shen, L., Qiao, Q., Yu, T., Zhou, K., Yu, T., Zhan, Y., Wang, Z., Tao, M., Yin, S., and Liu, S. SoulX-FlashTalk: Real-time infinite streaming of audio-driven avatars via self-correcting bidirectional distillation. arXiv preprint arXiv:2512.23379, 2025. URL https://arxiv.or g/abs/2512.23379. Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., and Catanzaro, B. Megatron-LM: Training multibillion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019. URL https://arxiv.org/abs/1909.08053. Spector, B. F., Arora, S., Singhal, A., Parthasarathy, A., Fu, D. Y., and Ré, C. ThunderKittens: Simple, fast, and Adorable kernels. In International Conference on Learning Representations (ICLR), 2025. Stivers, T., Enfield, N. J., Brown, P., Englert, C., Hayashi, M., Heinemann, T., Hoymann, G., Rossano, F., de Ruiter, J. P., Yoon, K.-E., and Levinson, S. C. Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences, 106(26): 10587–10592, 2009. doi: 10.1073/pnas.0903616106.

Runway Characters: Real-Time Conversational Video at Batch Size One

Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. doi: 10.1016/j.neucom.2023.127063.

Xu, S., Chen, G., Guo, Y.-X., Yang, J., Li, C., Zang, Z., Zhang, Y., Tong, X., and Guo, B. VASA-1: Lifelike audiodriven talking faces generated in real time. In Advances in Neural Information Processing Systems (NeurIPS), 2024.

Sul, S. H., Arora, S., Spector, B. F., and Ré, C. ParallelKittens: Systematic and practical simplification of multiGPU AI kernels. In Proceedings of Machine Learning and Systems (MLSys), 2026.

Yang, S., Huang, W., Chu, R., Xiao, Y., Zhao, Y., Wang, X., Li, M., Xie, E., Chen, Y., Lu, Y., Han, S., and Chen, Y. LongLive: Real-time interactive long video generation. In International Conference on Learning Representations (ICLR), 2026.

Sun, Z., Peng, Z., Ma, Y., Chen, Y., Zhou, Z., Zhou, Z., Zhang, G., Zhang, Y., Zhou, Y., Lu, Q., and Liu, Y.J. StreamAvatar: Streaming diffusion models for realtime interactive human avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. vLLM Team. Quantized KV cache. vLLM documentation, 2026. URL https://docs.vllm.ai/en/late st/features/quantization/quantized_k vcache/. Accessed 2026-10-04. Wang, Y., He, H., Wright, L., Wehrstedt, L., Liu, T., and Liang, W. [distributed w/ TorchTitan] introducing async tensor parallelism in PyTorch. PyTorch Forums, September 2024. URL https://discuss.pytorch.or g/t/distributed-w-torchtitan-introdu cing-async-tensor-parallelism-in-pyt orch/209487. Accessed 2026-10-04. Wang, Z., Liu, Z., Li, J., Huang, K., Xu, B., Kang, F., An, M., Wang, P., Jiang, B., Wei, Y., Xietian, Y., Pei, J., Hu, L., Jiang, B., Xue, H., Wang, Z., Sun, H., Li, W., Ouyang, W., He, X., Liu, Y., Li, Y., and Zhou, Y. Matrix-Game 3.0: Real-time and streaming interactive world model with long-horizon memory. arXiv preprint arXiv:2604.08995, 2026. URL https://arxiv.org/abs/2604.0 8995.

Yesiltepe, H., Meral, T. H. S., Akan, A. K., Oktay, K., and Yanardag, P. Infinity-RoPE: Action-controllable infinite video generation emerges from autoregressive self-rollout. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. Yi, J., Jang, W., Cho, P. H., Nam, J., Yoon, H., and Kim, S. Deep Forcing: Training-free long video generation with deep sink and participative compression. In Proceedings of the International Conference on Machine Learning (ICML), 2026. Yin, T., Zhang, Q., Zhang, R., Freeman, W. T., Durand, F., Shechtman, E., and Huang, X. From slow bidirectional to fast autoregressive video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22963–22974, 2025. Zhen, D., Zheng, X., Zhang, R., Jiang, Z., Yan, Y., Tao, M., and Yin, S. SoulX-LiveAct: Towards hour-scale real-time human animation with neighbor forcing and ConvKV memory. arXiv preprint arXiv:2603.11746, 2026. URL https://arxiv.org/abs/2603.11746.

Xi, H., Yang, S., Zhao, Y., Li, M., Cai, H., Li, X., Lin, Y., Zhang, Z., Zhang, J., Li, X., Xu, Z., Wu, J., Xu, C., Stoica, I., Han, S., and Keutzer, K. Quant VideoGen: Auto-regressive long video generation via 2-bit KV-cache quantization. In Proceedings of the International Conference on Machine Learning (ICML), 2026.

Zheng, S., Bao, W., Hou, Q., Zheng, X., Fang, J., Huang, C., Li, T., Duanmu, H., Chen, R., Xu, R., Guo, Y., Zheng, N., Jiang, Z., Di, X., Wang, D., Ye, J., Lin, H., Chang, L.-W., Lu, L., Liang, Y., Zhai, J., and Liu, X. Triton-distributed: Programming overlapping kernels on distributed AI systems with the Triton compiler. arXiv preprint arXiv:2504.19442, 2025a. URL https://arxiv.org/abs/2504.19442.

Xia, Y., Yuan, H., Ling, S., Sun, H., Zhang, H., Miao, X., Fu, F., and Cui, B. Adaptive resource management and quality control for streaming video generation. arXiv preprint arXiv:2606.15319, 2026. URL https:// arxiv.org/abs/2606.15319. Introduces the SlackServe system.

Zheng, S., Fang, J., Zheng, X., Hou, Q., Bao, W., Zheng, N., Jiang, Z., Wang, D., Ye, J., Lin, H., Chang, L.-W., and Liu, X. TileLink: Generating efficient computecommunication overlapping kernels using tile-centric primitives. In Proceedings of Machine Learning and Systems (MLSys), 2025b.

Xiao, G., Tian, Y., Chen, B., Han, S., and Lewis, M. Efficient streaming language models with attention sinks. In International Conference on Learning Representations (ICLR), 2024.

Zhu, K., Gao, Y., Zhao, Y., Zhao, L., Zuo, G., Gu, Y., Xie, D., Tang, T., Xu, Q., Ye, Z., Kamahori, K., Lin, C.-Y., Wang, Z., Wang, S., Krishnamurthy, A., and Kasikci, B. NanoFlow: Towards optimal large language model

Runway Characters: Real-Time Conversational Video at Batch Size One

serving throughput. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 749–765, 2025.

Runway Characters: Real-Time Conversational Video at Batch Size One

A

M EASUREMENT METHODOLOGY

All measurements come from four tools; their raw data, traces and output videos are archived internally. Our model weights and production telemetry are proprietary; Appendix E lists what is needed to rerun the external systems. • Session benchmark. Drives the production worker’s components in production order (setup, warm-up, persession reset, AR step loop fed from recorded speech) on the real topology, records per-step host and CUDAevent timings on every rank, and optionally captures profiler traces of a step window. Used for the livelength sessions, the waterfall and the Blackwell runs; the fused kernel has a separate microbenchmark. • Quality scorer. Generates each evaluation clip with its reference image, prompt and speech, then scores lip sync (SyncNet, Euclidean distance, video resampled to 25 fps, with our own crop instead of S3FD) per arm, and PSNR / SSIM per pair of arms overall and per 10 s window over the frames that differ (identical frames are counted separately), with bootstrap intervals over clips. • Turn analyzer. Joins the server’s trace spans with the browser’s trace using the client’s clocksynchronization offset, follows each turn through its first speaking latent, re-anchors GPU spans to host events on the same rank, excludes turns that overlap the profiler window, and times barge-ins from the voice agent’s interruption signal. • Telemetry puller. Reads every Prometheus counter and histogram bucket per series at weekly boundaries and sums each series’ last value minus its value a week earlier (a new series counts from zero; no counter reset occurred); fractions are at bucket edges. The metric populations are observed, not a reconciled census: in one week sessions of at least a second outnumber sessions with frames by 2%.

B

P LAYBACK SIMULATION

The simulator replays a session’s measured unpaced step periods through one generation stage and a 24 fps player; decoding and delivery add a constant offset and are not modeled. With slack s, step k starts generating once step k−1 is done and step k−s has started playing: s=1 overlaps generating a step with playing the previous one, and s=2 also queues one finished step (166.7 ms of video). Playback starts, with nothing queued, when step 0 is ready; a step that misses its slot stalls playback and shifts every later slot, as the pacer does. In the production configuration, with

s=1 a 161 ms step that starts when its predecessor starts playing is ready 5.7 ms before its slot, so a 263 ms reset step is 96 ms late and playback stalls (≈10 stalls per minute of ≈100 ms, some from slow ordinary steps). With s=2 the generator gains 5.7 ms per step until it leads its slot by 172 ms; a reset cuts the lead to 70 ms, which is rebuilt within 18 steps, before the next reset, so there are no stalls.

C

Q UALITY COHORTS AND LONG SESSIONS

A clip enters the lip-sync comparisons only if SyncNet returns a finite score for it in every run. SyncNet finds no track in one photoreal clip in any run, and tracks one nonhuman clip in five of the seven runs with LSE-D 12.1–14.2 (12.1–13.7 across production runs alone); counting that clip wherever both runs of a pair score it moves the bf16-cache LSE-D difference to +0.06 [−0.01, 0.16]. Table 7 gives absolute means on these 51 clips. Table 7. Absolute lip-sync means over the 51 clips SyncNet scores in every run. Run

LSE-D

LSE-C

Baseline 8.294 7.321 Production 8.270 7.353 Production reruns 8.261–8.285 7.345–7.362 Fused 8.293 7.336 bf16 cache 8.288 7.342

The long-session test alternates speaking and listening periods of 4–14 and 4–16 s, near the 52–61% listening of our live sessions. It has four sessions per arm, one per character and calibration, each with one seed shared across arms. Lip sync is scored per speaking period of at least 4 s (SyncNet tracks 92–94 of the 130 in both runs of a pair). The intervals in §7.3 come from a bootstrap over these segments, which treats segments of one session as independent; Table 8 gives the effects per session. Table 8. Long sessions: mean paired LSE-D difference from the first production run per session, with the number of periods SyncNet scores in both runs. Periods: speaking periods (scored by SyncNet in production). Character

Calibrated Periods bf16 cache

Photoreal Photoreal Non-human Non-human

listening speaking listening speaking

D

Rerun

33 (33) −0.07 (33) +0.01 (33) 32 (32) −0.01 (32) −0.05 (32) 33 (22) +0.22 (15) +0.17 (16) 32 (14) +0.06 (12) +0.23 (13)

T ENSOR - PARALLEL SHAPES

Column- and row-parallel weights are split on whole heads, without padding. At TP-7 and TP-8 the split is uneven, so the ranks with more heads set the step. FLOP and peak

Runway Characters: Real-Time Conversational Video at Batch Size One

figures count only the model’s own work, as there is no head padding. The kernel microbenchmark uses one K on every rank, the average slice rounded up to the 128-element reduction block; at TP-7 it is 4.4% below the larger ranks’ K.

E

E XTERNAL SYSTEMS

StreamDiffusionV2: code at commit 6961a5cf, weights Wan2.1-T2V-14B a064a6c7 and StreamDiffusionV2 2373eb2b, PyTorch 2.6, its four-GPU pipeline with -schedule_block, 1 or 4 steps, upstream defaults otherwise; our patch adds per-chunk timestamps and a clean exit, and keeps the input video on the host, copying each chunk to the GPU when rank 0 picks it up, as a live stream would. Output chunks are matched to inputs by message index, which we have not checked against the output misalignment its decoder shows on four GPUs. Live Avatar: code at commit c3c47d03, weights Live-Avatar 92cdccd1 and Wan2.2-S2V-14B dab4e9c5, PyTorch 2.8 with torch.compile, its -fp8 option, Euler sampler without guidance, 48-frame clips in 12-frame blocks; our patch adds timestamps, optional pacing and a conditioncache size fix needed above 384×704. For the paced latency of §7.6 (384×704, five GPUs: four DiT ranks, one per denoising pass, and one decoder rank), rank 0 starts block g no earlier than t0 + 0.48 g s, as if its audio had just arrived whole, and latency runs to the block’s frames on the host; audio features are computed from the whole file before generation; the audio is the first 60 s of our evaluation speech. The four-GPU runs used the box of Table 6, the five-GPU run an 8×H100 node.

F

F USED KERNEL DETAILS

Numerics. The GEMM is bit-exact against torch._scaled_mm with the same scales; the reduced output differs from GEMM + NCCL by bf16 rounding of the sum (at most ≈1 bf16 ulp of the output scale, mean absolute difference 0.005 at output magnitude 10).

Hopper tuning. Five pipeline stages instead of four (163.9 → 154.9 µs): the GEMM waits on weight tiles, not tensor cores. The activation-scale vector is padded to whole 128-row tiles, since each consumer warpgroup of the last row-block loads its full slice; a 64-row pad read past the tensor and faulted only at M =1671. ThunderKittens must be built with -fno-strict-aliasing: it builds TMA descriptors in host code by type punning, and strict aliasing at -O3 produced descriptors that fault inside the kernel, with byte-identical device code.

Blackwell deadlock and tracing. The deadlock appeared as a silent hang at 100% GPU utilization. ThunderKittens’ tensor-memory allocator waits on a cluster barrier in its destructor, so the post-GEMM reduction must run inside the allocator’s scope, or warp 0 (in the destructor) and the other warps (at the reduction’s barrier) wait on each other. Separately, the compiled path must not import the extension module, which the tracer cannot follow under fullgraph=True; the eager preparation step publishes the kernel geometry for the traced path to read.

G

C ACHE UPDATE AND POSITION RESET

Each of the 240 caches (60 layers × 4 denoising passes) holds 1,246 global slots, written once per session, and a ring of six latent slots of 757 tokens: 5,788 slots, all read by attention. In every step each block computes Q, K and V for the 757 new tokens, RMS-normalizes Q and K, applies RoPE at the new latent’s temporal index, quantizes K and V with the cache’s per-(layer, head) scales and writes them into the slot at the device-side write position, then attends over the whole cache. Scales are set once per session, on entering the seventh step, from the amax of the bf16 cache the first six filled, times 1.5. Every 42 steps every cached frame’s temporal index moves by ∆ = −42. A window frame at temporal index t has angle θj (t) = ωj (at + b) per RoPE frequency j, and the global tokens’ positions do not depend on the window. At a reset, before each denoising pass’s blocks read their caches, the window keys are dequantized with their head’s scale, rotated by ωj a∆ in fp32 (spatial components are unchanged), clamped to the FP8 range and requantized with the same scale; values are not touched. The window (6 latents) is shorter than the reset period (42 steps), so no key is requantized twice. On B200, cuDNN’s MXFP8 attention takes per-32-element block scales instead of per-head ones, but blocks values along the sequence, so each 757-token write would share blocks with older tokens, and padding writes to whole blocks needs a key mask that the wrapper and cuDNN build we use do not offer.

H

P IXEL DIVERGENCE

Two production runs, and production versus a bf16 cache, are bit-identical for the first 21 frames (0.875 s) of every clip and reach 37.6 and 36.8 dB over the first 10 s, against 34.7–35.3 dB for configurations that change numerics from the first step; after 10 s every pair sits near 30, then 28, then 25 dB. PSNR means leave identical frames out, whole-clip and per window. Whole-clip PSNR between configurations ranges from 32.6 to 35.5 dB, with two production runs the most similar. Per 10 s window (Figure 7) the difference is

Runway Characters: Real-Time Conversational Video at Batch Size One (a)

row block

column blocks of 128 0

1

0

0

6

1

1

7

2

2

8

3

3

9

4

4

10

5

5

11

2

reduction owner ···

768 rows (757 tokens + pad) task t = row t mod 6, col. ⌊t/6⌋ (b)

TP rank 0 TP rank 1 TP rank 2 owner = t mod 3; every rank computes a partial of every tile

TP rank 0 GEMM on own K slice TMA-store partial to own C0 ; wait FP8 K-tiles of 128 TP rank 1 (reduction owner of this tile) GEMM on own K slice TMA-store partial to own C1 ; wait FP8 K-tiles of 128

counter on TP rank 1 +1 per partial

TP rank 2 GEMM on own K slice TMA-store partial to own C2 ; wait FP8 K-tiles of 128

multimem.ld_reduce: claim tile; switch sums partials spin until = 3 multimem.st: sum to every rank

Figure 6. The fused FP8 GEMM + NVLS all-reduce on Hopper at the production shape (TP-3). (a) The 757-row output is covered by 128×128 tiles in six row blocks, with storage padded to 768 rows. Compute CTA b ∈ [0, 129] (of 132 SMs) processes tasks t = b + 130j in column-wise order. Every rank computes a partial for every tile; t mod 3 assigns the reduction owner. (b) For one tile, every rank multiplies its K slice, TMA-stores the bf16 partial into its own symmetric buffer, waits for the store to complete and adds one to the tile’s counter on the owner. The owner’s reduction warps (two communication CTAs, joined by each compute CTA once it runs out of GEMM work) claim owned tiles in order from a work queue, wait until all three partials have landed, and reduce the tile in the NVSwitch with multimem.ld_reduce, writing the sum to every rank with multimem.st. Early tiles are reduced while later tiles are still in the GEMM. After the call an epilogue kernel resets the counters and holds all ranks at an exit barrier, so no rank overwrites a partial a peer has not yet reduced.

only when runs diverge.

I

P UBLISHED END - TO - END LATENCY

Harness. The worker (LiveKit Agents 1.8.3 and the avatar’s official plugin) plays 30 clips cut at pauses from our evaluation speech, 10 each of about 3, 8 and 15 s, in five sessions of at most 120 s, each opened by a warm-up turn that is excluded and started 5 s after the avatar’s video appears. It sends each clip as 20 ms frames of 16 kHz audio as fast as the plugin accepts them, as streaming speech synthesis would, with transcript synchronization off. A viewer in the same LiveKit Cloud room, on the same host and clock, marks the avatar’s voice at the first two consecutive 10 ms frames above −40 dBFS; latency is that time minus the agent’s first sent frame and the clip’s own lead-in, found with the same detector. A mock avatar that returns the audio after 300 ms measured 412 ms through the cloud (112 ms of overhead), and 367 and 882 ms for 300 and 800 ms on a local server.

Published figures. Table 9 lists the published latency figures, each over the interval its source defines, next to ours over the closest interval we measure. The intervals seldom match exactly, so the table is context rather than a comparison.

Runway Characters: Real-Time Conversational Video at Batch Size One Table 9. Latency of real-time avatar systems as each source defines it, and ours over the closest measured interval. n/r: not reported. System

Interval (as defined by the source)

Video side: speech audio in → first video frame out Runway Characters, audio chunk → its frames on host benchmark Runway Characters, live agent’s first audio → first speaking frame sent Runway Characters, harness agent’s first audio frame → avatar’s (§7.6) voice at a viewer AVTR-1 (Kravtsov et al., 2026) agent’s first speech → first speaking frame sent VASA-1 (Xu et al., 2024) “preceding latency” SoulX-FlashTalk (Shen et al., 2025) SoulX-LiveAct (Zhen et al., 2026) StreamAvatar (Sun et al., 2026) Live Avatar (Huang et al., 2025b)

Median PSNR (dB)

30 25

0

493 ms p50 – 557 ms p50 – 390–454 ms mean – 170 ms – 0.87 s –

“latency” (not defined)

0.94 s –

input chunk → first frame (incl. 0.48 s buffer) audio arrival → first frame

1.20 s –

user unit → response, + 350 ms assumed network end of speech → start of response

prod vs prod (noise floor) prod vs bf16 KV prod vs fused baseline vs prod

35

293 ms p50 –

start-up, one generation cycle

Reported interactive-system response latency (endpoints differ) Runway Characters, live end of user speech → avatar audio at the client LiveTalk (Chern et al., 2025) latency per turn (not defined) Wan-Streamer v0.2 (Huang et al., 2026) Commercial voice agent (Lin et al., 2025a)

Latency Voice and language

10 20 Window start (s)

30

Figure 7. Median PSNR between two runs per 10 s window over the frames that differ, over the clips long enough for each window.

1.21 s –

2.13 s p50 third-party agent 1.16 s Qwen3-Omni ≈0.55 s integrated model 1.30 s mean –

Setting

4×H100, 704×1088, 24 fps same public API, LiveKit Cloud 1 GPU, 153M model, 25 fps 1×RTX 4090, 5122 , 40 fps 8×H800, 720×416, 32 fps 2×H100/H200, 20 fps 2×H800, 928×704, 25 fps 5×H100, 720×400, 45 fps p10–p90 1.45–3.00 s video model 1 GPU (system n/r), 5122 , 25 fps 640×368, 25 fps Full-Duplex-Bench

Related documents

• VC-1K: A Diverse Video Test Set for Real-Time Communication#1535744
• PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion#1155418
• AoiZora: Compiler-Agnostic, Placement-Aware Auto-Parallelization for Video Diffusion Transformers#1536158
• markov-ai/computer-use-large#786822
• From Perception to Prediction and Intervention: Benchmarking Proactive Video Assistants in Real-Life Scenarios#1356904