arXiv CS open access

Resource-Efficient Reinforcement Learning Post-Training through Elastic Expert Parallelism

Jia Wei et al. · arxiv_cs

arXiv CS · Papers

cloud distributed-computing parallel-computing

Reinforcement learning post-training improves the capabilities of large language models, but generating training responses is a major computational bottleneck. In mixture-of-experts models, workers that finish their assigned responses may remain occupied because their experts are still needed by wor

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

R ESOURCE -E FFICIENT R EINFORCEMENT L EARNING P OST-T RAINING THROUGH E LASTIC E XPERT PARALLELISM Jia Wei Ziyi Qiu Yi Xu Xuehai Qian * Department of Computer Science and Technology, Tsinghua University

arXiv:2611.06984v1 [cs.AI] 4 Nov 2026

weijia4473@tsinghua.edu.cn, qiuzy21@mails.tsinghua.edu.cn xuyi23@mails.tsinghua.edu.cn, xuehaiq@tsinghua.edu.cn * Corresponding author, Xuehai Qian

A BSTRACT Reinforcement learning post-training improves the capabilities of large language models, but generating training responses is a major computational bottleneck. In mixture-of-experts models, workers that ﬁnish their assigned responses may remain occupied because their experts are still needed by workers processing longer responses. Consequently, approximately one-third of occupied worker time in the baseline occurs after local response generation ﬁnishes. Here we present FlexFloor, a system that enables earlier resource release during synchronous response generation while preserving the expert computation required by ongoing responses. FlexFloor jointly adapts response scheduling and the active group of workers. It uses historical response lengths and memory constraints to guide batching and prompt placement, then progressively releases workers while maintaining expert availability and correct communication. Compared with the verl baseline, FlexFloor generates approximately 65.40% more response tokens per unit of occupied computing time and increases response generation throughput by 52.74%, with little observed variation in rollout reward. By making reinforcement learning post-training more efﬁcient, FlexFloor supports faster development and iterative improvement of large language models and AI agents. 1

I NTRODUCTION

Reinforcement-learning (RL) post-training generates multiple responses to a prompt, scores them, and updates a shared policy (Ouyang et al., 2022, Shao et al., 2024). Sparse mixture-of-experts (MoE) models increase parameter capacity while activating only selected experts for each token (Shazeer et al., 2017, Lepikhin et al., 2021, Fedus et al., 2022). Expert parallelism (EP) distributes those experts across cooperating ranks. Long-tail generation leaves locally ﬁnished ranks waiting for the rollout to end. It must nevertheless remain in expert dispatch, computation, and combine because unﬁnished responses elsewhere still require access to its experts. This coupled tail creates two related but distinct inefﬁciencies. Long responses extend rollout wall time. Completed ranks also remain allocated even when remaining responses ﬁt on fewer ranks that collectively host the full expert set. We call this completed-but-coupled interval dummy run. The name describes allocation rather than claiming that every kernel is a no-op. Reclaiming the interval requires changing expert placement and collective membership, not simply declaring the device idle.

We evaluate FlexFloor through two primary metrics. Rollout throughput counts generated response tokens per second. Rank-time resource efﬁciency counts the same response work per actually occupied rank-second. The second metric includes both faster execution and fewer occupied ranks. It charges preparation, transition, cleanup, and restoration whenever the primary job owns the rank. Releasing devices from dummy execution saves rank-time regardless of subsequent use. Tail-aware batching and prediction reduce response skew (Gao et al., 2026, He et al., 2026), while worker-level reuse exploits slack between workers (Hu et al., 2026). FlexFloor targets the remaining coupling inside the EP group of an active worker. Three requirements must be satisﬁed together. Prompt placement must advance a completion quorum rather than merely minimize pairwise length gaps. Survivors must retain expert availability and sufﬁcient KV capacity. Finally, the runtime must change and restore the distributed world without violating collective ordering or the assumptions of its decode graphs. FlexFloor combines historical length prediction and guarded epoch batching with ﬂoor-conditioned KV admis-

sion and quota-aware prompt matching. For each step it selects a feasible survivor ladder and maps prompt groups to physical rank roles. Its elastic runtime waits for observed completions, detaches ﬁnished ranks, and continues on fewer ranks hosting all model experts. Direct graph reuse is unsafe across these topology changes. We design topology-aware full-decode recapture and native-sleep lifecycle management for FlexFloor, keeping expert placement, communicators, weights, and KV state consistent with each capture. We implement two variants to trade throughput against rank-time resource efﬁciency. Natural constructs survivor resources on demand to favor resource reclamation, whereas Planned pre-provisions them to reduce transition overhead and favor throughput. On the same graph-enabled backend, the evaluated Planned variant improves throughput by 52.74% and rank-time resource efﬁciency by approximately 65.40% over the verl baseline. Both ﬁgures use the same conﬁguration and generated-work accounting. Controlled experiments additionally isolate quotaaware placement, compare conﬁgured eager and graph paths, and demonstrate actual sidecar inference with exact paired primary outputs. An additional code workload improves throughput by 22.02% under matched frozen-policy rollout, with sampled pass@1 of 70/160 versus 71/160. The paper makes three contributions. • We identify EP-coupled rank occupancy as a resourceefﬁciency problem beyond rollout wall time and establish consistent throughput and occupied rank-time accounting. • We design a latency-guarded, KV-aware planner that jointly chooses completion quorums, prompt placement, and feasible EP ﬂoors. • We implement lossless elastic EP execution with topology-aware graph recapture and lifecycle management, and evaluate throughput, rank-time efﬁciency, and quality on mathematics and code workloads using a common backend.

P ROBLEM AND O PPORTUNITY

2.1

Local Completion Does Not Release a Rank

Consider an EP group of N ranks and a step batch Bt containing b prompts, with n responses generated per prompt. An assignment partitions the batch into rank-local prompt groups. Let Mr be the local request-completion time. The generation tail is held by the slowest rank, but a locally ﬁnished rank cannot independently leave an EP collective or stop providing its resident experts. Figure 1 shows this dummy state in Worker4. FlexFloor

Token

Dispatch

Experts

Top-2 selected

Compute

Combine

Worker1 E2 E3 Worker2 E4 E5 Worker3 E6 Dummy Run Worker4

Local requests E8 Still serving remote tokens complete

Figure 1. Dummy execution in an illustrative EP group. Worker4 has completed its local requests but remains allocated to serve Worker3’s tokens through expert E7.

waits for enough ranks to ﬁnish before continuing the remaining rollouts on a smaller EP group that still serves all experts. Memory-dependent reuse additionally waits for allocation release, not merely detachment from EP execution. 2.2

Two Primary Metrics

Let W be the response tokens actually generated and retained, excluding prompt tokens, padding, and duplicate P work from recomputation. Let T = s |Is | sum the measured rollout windows. For step s, deﬁne as,r (t) = 1 while the primary rollout occupies rank r and zero during an eligible released interval. Occupied rank-time is H=

N Z XX s

r=1

as,r (t) dt.

(1)

W . H

(2)

The two primary outcomes are Q=

W , T

Q has units of response tokens/s and E has units of response tokens/rank-s. For a ﬁxed-world baseline, H = N T . Writing the average occupied rank count as N̄ = H/T gives Q E= . (3) N̄ Lower average occupancy ampliﬁes the resource-efﬁciency gain beyond the throughput gain. The two factors combine multiplicatively. Every occupied interval is charged. Preparation, preﬁll, decode, expert transitions, graph cleanup, and native sleep are

included before release. If a rank later rejoins full-world restore, its renewed occupancy through the end of the window is also included. A ﬁrst-detach timestamp is not a valid end of lifetime when the same rank is reacquired. For an evaluated method A and reference baseline B, relative throughput and resource-efﬁciency gains are Q̄A GQ = − 1, Q̄B

ĒA GE = − 1, ĒB

(4)

where bars denote means across independent seeds. For each seed, Q and E are computed from total response tokens divided by total rollout time and total occupied ranktime, respectively. We report 100GQ and 100GE as percentage improvements over B. 2.3

Why Release Area Is a Planning Surrogate

For a ladder ⟨f0 = N, f1 , . . . , fS ⟩, an idealized release score is b= R

S X

(fj−1 − fj )(Tb − b tj ).

(5)

j=1

b lowers the proxy For ﬁxed predicted duration, larger R b b b H = N T − R. This motivates quorum-aware planning. It is not the ﬁnal resource-accounting formula. Actual H uses observed release-ready and reacquisition boundaries and includes transition costs. A smaller ﬂoor also concentrates expert state and KV on fewer survivors. Local completion order alone therefore cannot establish a useful release. FlexFloor must ﬁnd a KV-feasible assignment that reaches the required quorums and is supported by the runtime. Graph validity adds an execution constraint at topology and allocation boundaries, which Section 4 addresses.

3.1

(k)

For prompt i in epoch k, the observed tail is Mi = (k) (k) maxj≤n Li,j , where Li,j is the length of response j. Long responses delay rank completion, making this maximum more relevant than the mean. The predictor uses c(k+1) = αM c(k) + (1 − α)M (k) , M i i i

R ESOURCE -E FFICIENT P LANNING

FlexFloor plans at two levels. It ﬁrst groups prompts with similar predicted completion times and applies TailGuard to control rollout tails. Within each batch, it jointly selects prompt placement and a memory-feasible shrink schedule to advance completion quorums and maximize predicted released rank-time. Figure 2 separates epoch shaping, ﬂoor-speciﬁc KV admission, and quorum-aware placement. Tail shaping limits repeated exposure to long prompts. Admission excludes groups that exceed survivor memory. Quorum matching advances collective shrink rather than merely balancing local completion times. The cached plan contains the assignment, caps, and ladder needed by the executor.

0 ≤ α < 1. (6)

The latest observation receives weight 1 − α. We retain the response-length multiset separately for KV estimation. Predictions are keyed by dataset item, not just prompt template. An initial rollout without shrinking bootstraps the length history, which is updated after each subsequent epoch for planning the next epoch. ci into steps of target size b groups Sorting prompts by M similar tails and reduces repeated exposure to long prompts. This inexpensive initial layout is reﬁned by subsequent KVaware prompt pairing and rank placement to advance completion quorums. When needed, bounded swaps between adjacent steps restore feasibility while limiting disruption to the tail-shaped layout. TailGuard bounds unexpectedly long responses in a predicted short step. Using one-sided historical ratios ri+ = ci + ϵ)), it estimates the q-quantile ρq over max(1, Mi /(M the recent K epoch transitions. For step t, let St = ci . Its cap is maxi∈Bt M ρq St Ct = min Lmax , max Cmin , δC . (7) δC The quantile q, history window K, minimum cap Cmin , rounding unit δC , and global limit Lmax are conﬁgurable. Naturally long steps retain the global limit. TailGuard limits response lengths in predicted short steps, while ﬂoorconditioned KV admission determines memory-feasible prompt assignments. 3.2

History-Based Steps and a Calibrated Tail Guard

Floor-Conditioned KV Admission

The pair-assignment formulation uses b = 2N , with conﬁgurable world size, response count, and ﬂoor ladders. Each candidate pair e = (i, j) initially occupies one rank. Its ci , M cj ). For predicted completion proxy is Me = max(M the pair’s historical response length multiset Le , a responseside active-token peak is Ae = max (τ |{ℓ ∈ Le | ℓ ≥ τ }|) . τ

(8)

This proxy counts tokens belonging to simultaneously unﬁnished responses. Prompt KV and allocation overhead require separate accounting. Before applying the test, lengths are inﬂated by a conﬁgurable safety factor µ ≥ 1 and clipped to the applicable response cap. Let the resulting peak be Aµ,e .

1 Inputs and constraints Prompt 1

2a Epoch-level tail shaping

Prompt 2 Prompt 3

Guard

… Prompts

…

Prompt i

Historical traces " " "#$ " " 𝑀! = 𝛼𝑀! + 1 − 𝛼 𝑀!

Prompt 1 Prompt 2 Prompt 3

…

KV budget by floor Admissible floors C_eff(f, m)

N > f1 > ... > fS

expert residency

f1 f2 fS fS

+ KV headroom

…

Batch1

Batch 2

…

f1 f2 fS fS

EP f

Switch EP Group

Maximize

Reuse Released Ranks

min 𝑇epoch 𝜋 . safe

Minimize epoch rollout time

EP g

Transfer Experts Release Memory

Sidecar

Safety Gate Before Reuse

A released rank is handed to sidecar work only after it is decoupled from the active EP group.

Prompt assignment | Selected ladder | Survivor ranks | Comm groups | Tail cap

"∈$

Enough Finished Ranks

Trigger Shrink

Per-step Rank Plan

⋆ First find 𝑇epoch =

Apply Planned Caps Monitor Completion

Batch t

2b Step-level floor feasibility Rank 0 Rank 1 Floor f, KV feasibility Batch

… Batch

EMA Predicted Length

+ communication

…

Guard

2c Release-area optimization

Prompt i

Objective

3 Runtime executorGuard

2 Offline planner

! Then maximize 𝑅 𝜋 = ∑) ∑%+' 𝑓%&',) − 𝑓%,)

KV safety is hard

𝑇) − 𝑡%,)

Maximize released rank-time

Figure 2. FlexFloor’s planning and execution pipeline. History produces guarded steps, ﬂoor-speciﬁc KV feasibility graphs, and quorumaware rank plans. The runtime uses observed completions to shrink and exposes released ranks subject to a reuse safety gate. Floorspeciﬁc KV budgets and admissible ladders are runtime inputs rather than ﬁxed device counts. The reuse block describes the consumer contract instantiated by the sidecar experiments in Section 5.7.

For ﬂoor f and residency policy m, the runtime supplies a eﬀ calibrated effective budget Cf,m . The planner forms Gt,f = (Bt , Et,f ),

eﬀ Et,f = {e | Aµ,e ≤ Cf,m }. (9)

Illegal pairs never enter matching. The capacity proﬁle reﬂects expert residency, communication workspaces, and usable KV, rather than dividing total HBM by KV bytes. Runtime admission also accounts for prompt tokens. Each residency policy supplies an admissible ﬂoor set Fm ⊆ {1, . . . , N }. Planned residency reserves additional memory for prepared resources, changing both the KV budgets and feasible ﬂoors relative to Natural residency. Only ﬂoors supported by the runtime and its calibrated memory proﬁle are candidates. If length variation, block granularity, or policy drift exhausts KV capacity after predicted admission, the scheduler preempts and recomputes requests. The runtime retains this normal recovery and charges its performance cost, distinguishing preemption from allocator failure. 3.3

Joint KV Safety and Completion Planning

Figure 3 separates the two obstacles using four ranks and two responses per prompt. Pairing p3 with p4 looks attrac-

tive because their maximum lengths, 50 and 52, are close. Yet at token position 39 all four responses remain live, producing a 156-token peak above the 144-token budget. Pairing p0 with p3 instead has a larger tail gap but a peak of only 78. Length proximity cannot replace a KV feasibility test. Even among KV-safe pairs, local skew does not determine release timing. The illustrated skew-oriented matching ﬁnishes its ﬁrst rank at 22 but reaches the two-rank completion quorum only at 60. The release-oriented matching ﬁnishes its ﬁrst rank later, at 50, but reaches that quorum at 52. Both retain the same tail of 93. Their idealized released areas are 2(93 − 60) = 66 and 2(93 − 52) = 82 rank-time units. The gain comes from the quorum, not an earlier ﬁrst completion or a shorter ﬁnal response. 3.4

Quorum-Aware Matching and Floor Selection

EP communication libraries restrict the group sizes used to host the complete expert set. For example, Huawei CANN’s MoE dispatch operator requires the routed expert count to be divisible by the number of ranks hosting routed experts (Huawei Technologies, n.d.). Without dedicated shared-expert ranks, this requires the EP group size to divide the routed expert count. Resource release there-

1 (a) Toy batch and KV constraint Prompt

Response lengths

m_i

[21, 19]

[22, 4]

[47, 7]

[50, 39]

[52, 48]

[60, 16]

[73, 68]

[93, 16]

4 ranks, 8 prompts 2 prompts per rank 2 responses per prompt KV cap = 144 Target shrink: 4 -> 2

𝐴 𝑖, 𝑗

= max 𝜏 × !

2 (b) KV-safe edges require active-

3 (c) Feasible matchings under same

Unsafe near-neighbor pair Pair: p3 + p4 Combined responses: [52, 50, 48, 39] Response Alive at length 𝜏 = 39 𝜏 = 39 52 1 50 1 48 1 39 1

Skew-oriented matching

token peak

At τ = 39, all 4 responses are alive. A({p3,p4}) = 39 × 4 = 156 > 144

KV cap

Rank r0 r1 r2 r3

# 𝑟𝑒𝑠𝑝𝑜𝑛𝑠𝑒𝑠 𝑤𝑖𝑡ℎ 𝑙𝑒𝑛𝑔𝑡ℎ ≥ 𝜏

Peak at 𝜏 = 39 with 2 alive: A({p0,p3}) = 39 × 2 = 78 ≤ 144

A({i,j}) 57 141 117 144

Ordered exit times

Skew 1 26 10 41 60 73

Exit 22 73 60 93 93

4 -> 2 shrink point = 60

Release-point-aware matching

Feasible sparse pair Pair: p0 + p3 Combined responses: [50, 39, 21, 19] Response Alive at length 𝜏 = 21 𝜏 = 39 𝜏 =21 𝜏 =39 50 1 1 39 1 1 21 1 0 19 0 0

Pair p0,p1 p2,p6 p3,p5 p4,p7 22

Rank r0 r1 r2 r3

Pair p0,p3 p1,p4 p2,p6 p5,p7

Ordered 50 52 exit times

A({i,j}) 78 96 141 120

Skew 29 30 26 33 73

Exit 50 52 73 93 93

4 -> 2 shrink point = 52

Figure 3. Didactic example of KV-safe pairing and completion quorums. Eight prompts form four ranks under a 144-token KV cap. A nearby-length pair is unsafe, and two safe matchings reach the same two-rank completion quorum at different times. Minimizing pairwise skew is therefore not the same objective as advancing a legal shrink point.

fore proceeds between admissible group sizes and requires enough ranks to ﬁnish their local responses before each transition. Local balance is therefore not the release objective. For an admissible ladder N = f0 > f1 > · · · > fS , reaching stage s requires a completion quorum of N − fs ranks. Given candidate quorum times τ1 ≤ · · · ≤ τS and step tail proxy Tt , the release score is bf (τ ) = R S

S X

(fs−1 − fs )(Tt − τs ).

(10)

s=1

Quorum times come from distinct feasible pair-completion proxies. A binary matching variable xe must satisfy X

xe = 1

∀i ∈ Bt ,

(11)

xe ≥ N − fs

∀s.

(12)

e∋i

e | Me ≤τs

The ﬁrst constraint pairs every prompt once. The second requires enough completed pairs for each stage. For each candidate ﬂoor, FlexFloor examines shrink schedules in descending order of predicted release area and selects the ﬁrst schedule whose completion quotas admit a KV-safe prompt matching. Within that schedule, a weighted matching cost

favors smaller within-rank tail gaps, earlier pair completion, and lower active-token peaks while satisfying all completion quotas. When a lower ﬂoor’s higher expert residency tightens KV constraints, the planner revises prompt pairings. If these pairings delay earlier release stages, they offset the ﬁnal shrink’s beneﬁt. FlexFloor therefore selects the ﬂoor with the largest total predicted release area rather than always choosing the smallest ﬂoor. When release areas are equal, it selects the larger ﬂoor to reduce shrink transitions. The full N -rank world remains the non-shrinking fallback. A multistage plan includes every preceding stage, not just its ﬁnal shrink. The ILP determines how prompts are paired. The planner then assigns these pairs to physical ranks according to their predicted completion times. Pairs expected to ﬁnish earlier are assigned to ranks scheduled for earlier release, while longer-running pairs are assigned to ranks expected to remain active after the ﬁnal shrink. Residency preparation and actual survivor selection are distinct decisions. Actual survivor selection follows observed unﬁnished requests, independently of the residency policy. A predicted completion never authorizes early exit. Observed completions and legal collectives govern execution.

Under the same KV constraints and length predictions, FlexFloor considers admissible MinSkew matchings alongside other feasible pairings. It jointly selects pairs and a ﬂoor to maximize predicted release area under each stage’s completion quorum.

responding capacity graph while runtime completion still gates release. This design uses resident experts rather than assuming that on-demand CPU or remote expert loading has the same performance. 4.2

3.5

Bounded Repair and the Objective’s Scope

Epoch construction calls the complete step planner, not a weaker feasibility check. If a step has a feasible plan at any allowed ﬂoor, the planner retains the selected plan without changing its prompt membership, even if the smallest ﬂoor is infeasible. Otherwise, bounded swaps with the preceding sorted step reduce the rejected step’s KV demand while disturbing the sorted layout as little as possible. Affected caps and complete plans are recomputed. With b prompt ver tices, a pair graph has at most 2b candidate edges. Other group cardinalities require a corresponding grouping solver rather than unchanged pair matching. Planning is off the per-token path. More precisely, the examined safe layouts are ranked by b lexmin Tbepoch (π), −R(π) . (13) π∈Πsafe

This preserves the predicted latency objective while seeking release within the chosen layout. For a ﬁxed predicted tail, release lowers the occupancy proxy in Section 2.2. During execution, resource efﬁciency W/H reﬂects the generated response work, the token-generation rate at each ﬂoor, and the rank-time spent preparing, releasing, and restoring resources. The elastic runtime carries out these transitions to realize the planned resource release.

Progressive Detach and Full-World Restore

The driver installs the cached prompt placement, cap, and survivor plan before rollout. Ranks report whether they still own unﬁnished requests. A transition is legal only after observed completions satisfy its quorum and survivors satisfy full-expert coverage and KV admission. Execution follows the selected ladder N = f0 > f1 > · · · > fS . Ranks agree on group creation and switching order, including ranks that stop serving tokens. When prediction errors invalidate a planned transition, the runtime delays or skips shrinking without discarding unﬁnished requests. An exiting worker detaches from the active expert group but stays alive for restoration. Admitted sidecar tasks reuse its released compute capacity and reclaimed memory while survivors continue generation, improving resource utilization. Actual KV pressure is handled by normal scheduler admission and recomputation. All ranks return to the full world before reward processing and policy training require that topology. The next rollout uses the newly synchronized policy, including refreshed expert replicas. 4.3

Sidecar Admission and Resource Reuse

E LASTIC EP E XECUTION AND R ANK -T IME R EUSE

Sidecar admission checks the released compute capacity, the device memory actually reclaimed, and the time available before restoration. This check distinguishes free request-level blocks from backing KV storage still allocated on the device. Before restoring the full expert group, FlexFloor stops the sidecar and waits for conﬁrmation that its resources have been released.

4.1

Lossless Expert Availability and Residency Policies

4.4

Shrinking EP must preserve every logical expert available to remaining tokens. FlexFloor uses resident replicas prepared from the current policy weights. For X logical experts in a layer, the average ownership per survivor increases to X/f at ﬂoor f . The KV proﬁle counts all resident capacity, including redundant slots preallocated before shrink, rather than only primary expert rows. KV capacity is therefore determined by the memory occupied by all resident expert weights, including those not selected by the current tokens. Natural mode constructs survivor resources on demand. Planned residency uses preparation and a separate residency-aware capacity proﬁle, with additional headroom charged against KV. Residency preparation does not by itself ﬁx actual survivor identities. The planner uses the cor-

Graph-Compatible Execution

FlexFloor captures the full decode path, including Attention and MoE. Preﬁll, recomputation, ﬂoor decisions, group creation, and allocation remain outside the graph. This separates a dynamic control plane from a replayable data plane without requiring elastic decode to remain eager. FlexFloor adds topology- and allocation-aware orchestration on top of backend-provided capture and Attention operators. Graph validity depends on survivor identities, communicators, expert maps, current weights, and KV addresses, not only ﬂoor and batch size. The runtime installs the role plan, current weights, and applicable KV allocation before capture. A legal topology transition invalidates the old execution state and prepares a capture for the new world. Stable worlds replay their captures. A recapture request is ac-

knowledged only after capture succeeds. Each rank has local captures for the required batch buckets and topology epochs, not a universal graph for an entire RL step. When releasing rollout memory for training, FlexFloor waits for pending device operations to ﬁnish, then releases the old decode graphs and their memory pools. Expert placement and active-expert metadata are retained for the next rollout. Before capturing new graphs, FlexFloor allocates KV storage and loads the updated policy weights, ensuring that each graph uses the current memory addresses and expert conﬁguration. The planner selects decode buckets from historical activebatch traces to reduce predicted padding under a conﬁgurable capture budget. This trades padding against capture and memory costs. Audits check full-decode dispatch without silent eager fallback. Correctness additionally requires token/logit checks across wake, shrink, and restore. Section 5.5 compares the conﬁgured eager and graph paths.

E VALUATION

We primarily report rollout throughput Q and rank-time resource efﬁciency E. All experiments run on vLLM 0.14.1. 5.1

Setup and Statistical Protocol

The platform has 16 software-visible Ascend devices with 64 GiB HBM each. The BF16 rollout model is Qwen3-30BA3B with 128 experts per MoE layer, TP1, PP1, DP16, and EP16 (Yang et al., 2025). We use the vLLM 0.14.1 engine and local Ascend/VERL integration. Graph execution uses native FULL_DECODE_ONLY, capturing both Attention and MoE during decode. We select the task-queue (TQ) setting with the best measured performance for each execution mode, using TQ1 for graph execution and TQ2 for eager execution. These settings apply throughout the evaluation, including sidecar inference, unless otherwise stated. Every primary method uses graph execution, native sleep2, explicit graph lifecycle management, KV-ﬁrst wake, and history-only eight-bucket selection. Operator and allocator settings are common to the methods. Each epoch has 160 DeepScaleR mathematical reasoning prompts, arranged as ﬁve 32-prompt steps with 16 responses per prompt. Prompt and response limits are 1,024 and 16,384 tokens. Sampling uses temperature 0.9, top-p 0.9, and top-k 50. Every arm starts from a common permanent epoch0 checkpoint and history. Epochs1 and2 update the actor at learning rate 10−6 , with epoch2 continuing from that arm’s own checkpoint and observations. The pair-assignment solver places two prompts and 32 responses on each rank. We use EMA weight α = 0.3 and KV safety factor µ = 1.16. TailGuard uses q = 0.95,

K = 3, Cmin = 4096, and δC = 512, with Lmax = 16384. Rollout seeds are 0, 2, and 3, while actor, reference, and data seeds are 42. For each seed, we pool the work and time of its two epochs before computing Q and E. We then compare means of the three seed-level metrics. Throughput gains and efﬁciency gains are ratios of these means, not averages of step-level percentages. The primary comparison contains 48 epochs, 240 steps, and 122,880 responses. The no-TailGuard control adds six epochs and 15,360 responses, reusing the already measured guarded control. Natural admits ﬂoors {2, 4, 8, 16}, while Planned admits {4, 8, 16}. Both use realized unﬁnished ranks for survivor selection, including the P4 residency conﬁguration, rather than ﬁxing predicted identities in advance. Full16 has 380,800 KV tokens per rank. Natural ﬂoor2 and ﬂoor4 have 131,072 and 280,576. Planned ﬂoor4 has 133,120 after its residency reserve. The frozen matrix, including ﬂoor8 families, is in the appendix. Planned-F2 is budget-excluded before timing, not a measured OOM or a ninth timed arm. T sums the logged rollout intervals. It includes rollout preparation and cleanup but excludes actor training, checkpoint serialization, and ofﬂine epoch planning. W uses saved response masks, excluding a proven ﬁrst padding EOS at short caps. Prompt tokens and retokenized text are not used as response work. Reward is task scoring on training rollouts, not held-out generalization. 5.2

Occupied Rank-Time Accounting

We reconstruct H per rank and step from runtime events. A rank is charged from rollout start until shrink release completes, including the shrink RPC and local resource cleanup. Its released interval ends at the start of primary topology restoration or an earlier resource wake. We charge the remaining time within the rollout window, and the full window for ranks that never detach. Topology restoration follows completion of generation and rollout resource release. It rebuilds communication groups and expert mappings, while KV allocation and currentweight loading occur before the next rollout. Restore start is taken from an explicit request or reconstructed from the logged completion time and RPC duration, using the earlier time when both are available. Timestamp resolution gives lower and upper bounds for H, including preparation, cleanup, topology changes, and restoration. The primary and TailGuard accounting covers 54 epochs and 4,320 rank-step windows, including normal scheduler preemption in the Full16 controls and one Fixed-F4 step. It does not infer completion from response length or assume constant token latency. All 1,680 detached-rank windows show reacquisition for restoration within the measured rollout window. Omitting these tails would overstate efﬁciency.

The appendix provides the event ledger and uncertainty equations. 5.3

Throughput and Rank-Time Resource Efﬁciency

The verl baseline retains Full16 without epoch tail shaping or shrink. LengthSort and LS+TG retain the same world while shaping tails. Fixed-F4 uses a ﬁxed ﬂoor policy. MinSkew changes within-step pairing without quota-aware release optimization. Flex-N4 and Flex-N2 use Natural residency, and Flex-P4 uses Planned residency. Each minimum ﬂoor is a lower bound, not a required shrink target. Figure 4 and Table 1 show the two beneﬁts jointly. Flex-P4 achieves 5668.7 ± 42.8 response tokens/s against the verl baseline’s 3711.3 ± 33.0, a 52.74% gain. Its rank-time efﬁciency is about 383.6 response tokens/rank-s against 232.0 for the baseline. The reconstructed relative gain lies in [65.23%, 65.57%], summarized as approximately 65.40%. Both gains use the same conﬁguration. Flex-P4 reduces rollout time from 4152.2 to 2710.1 s and mean occupied ranks from 16 to approximately 14.8. Both increase E = Q/N̄ . Occupied rank-time falls from 66,435.5 to approximately 40,003 to 40,083 rank-s per epoch, reducing occupied cost per response token by about 39.5%. The largest raw release area is not necessarily the best efﬁciency point. Natural-F2 exposes more raw release opportunity than Flex-P4 but has lower Q and mean E. Natural-F4 has slightly higher mean E, while Flex-P4 has the highest mean Q. Small mean differences do not establish statistical superiority or dominance across resource objectives. Flex-N2 reaches 5471.2 tokens/s with the corrected ﬂoor8 budget, a 47.42% throughput gain. Its efﬁciency bounds imply a 64.08% to 64.51% gain, including restoration reoccupation. 5.4

Mechanism Contributions

With H = 16T , Full16 methods have identical relative Q and E gains. Lower mean occupancy additionally improves elastic methods’ efﬁciency. Against LS+TG, Flex-P4 improves throughput by 8.57% and rank-time resource efﬁciency by approximately 17.56%. Its corresponding rollout-time reduction is 7.9%. The paired seed-level time difference is 232.3 s, with a 95% Student-t conﬁdence interval of [123.9, 340.6] s (NIST/SEMATECH, 2012). We compare quota-aware assignment with MinSkew under identical Planned-F4 weights, prompts, KV constraints, expert residency, and graph settings. Only prompt pairing and rank placement differ. Constructed response-length limits with EOS ignored ﬁx work at 1,710,400 tokens across 32

prompts with 16 responses each. MinSkew minimizes within-rank length differences, whereas quota-aware assignment advances completion across ranks. The ﬁrst eight-rank quorum arrives at 194.29 s rather than 229.56 s. Waiting after local completion falls from 750.26 to 290.65 rank-s (61.26%). Earlier release raises E from 356.93 to 383.59 tokens/rank-s (7.47%), while Q improves from 4358.73 to 4433.20 tokens/s (1.71%). Both windows include transitions, cleanup, and restoration but exclude initial loading and graph capture. Detailed protocol and waiting components are in the appendix. Composition with Worker-Level Reuse. We evaluate two EP8 rollout workers over ﬁve seed0 steps from the frozen common actor, with 16 prompts per worker per step and 16 responses per prompt. TLT-style worker-level reuse (Hu et al., 2026) exposes 1013.3 rank-s between worker completion times. FlexFloor adds 3315.0 rank-s within unﬁnished workers. Together, these disjoint windows provide 4328.3 rank-s, or 4.27× worker-level reuse alone. 5.5

Conﬁgured Graph-Path Performance

We add seed0 eager controls for the verl baseline and FlexP4, each with ﬁve real actor updates from the same epoch0. Ordered inputs and actual per-rank KV/ﬂoor settings match the corresponding graph epoch. Both paths retain native sleep2 and lifecycle management. Table 2 shows Q gains of 90.76% and 124.88%. Each updated-policy arm computes Q from its own response-token count. The main 52.74% result instead compares methods already sharing the graph backend. 5.6

TailGuard and Rollout Quality

We disable dynamic response guards and the auxiliary short-step cap in Flex-P4, keeping EOS stopping and the global 16,384-token limit. Epoch1 prompt/rank assignments, ﬂoors, graph buckets, and non-cap plan ﬁelds match the guarded arm. Epoch2 follows each arm’s own updated policy and history. The ablation is not token-paired frozenpolicy continuation. TailGuard improves mean throughput by 3.65% and mean resource efﬁciency by approximately 1.68% (Table 3). These are descriptive trends. The paired guarded-minusunguarded rollout-time difference is −94.5 s with 95% interval [−231.9, 42.9] s. The reward difference is +0.124 percentage points with interval [−0.317, 0.564]. Neither interval establishes signiﬁcance or non-inferiority. No-TailGuard exposes more release opportunity but lower mean Q and E. Cap-boundary counts and their interpretation are in the appendix.

(a) Rollout Throughput

Q (response tokens/s)

33.43% 40.69%

44.77% 45.33%

50.22% 47.42% 52.74%

400

5000 4000 3000 2000 Budget excluded

1000

E (response tokens/rank-s)

6000

(b) Rank-Time Resource Efficiency

60.26% 61.51%

66.13% 64.29% 65.40%

33.43% 40.69%

300

200

100

Budget excluded

0 verl LS baseline F16

LS Fixed Min Flex +TG F4 Skew N4

Flex N2

Flex P4

verl LS baseline F16

Flex P2

LS Fixed Min Flex +TG F4 Skew N4

Flex N2

Flex P4

Flex P2

Figure 4. The two primary v0.14 outcomes on a common graph-enabled backend. Higher is better in both panels. Throughput is Q = W/T , measured in response tokens/s. Throughput bars show seed means with sample SD. Resource-efﬁciency bars show midpoint estimates with sample SD across seeds, while Table 1 reports the separate recorded-clock bounds. Percentage annotations show gains over the verl baseline. Planned-F2 is budget-excluded, not a timed method. Table 1. Primary outcomes and supporting quality/timing results. Throughput, wall time, and reward show mean ± sample SD across three seed-level two-epoch summaries. Efﬁciency ranges are outward-rounded bounds from log timestamp resolution, not conﬁdence intervals. Restore reoccupation is included in occupied rank-time. Method verl baseline LengthSort LS+TG Fixed-F4 MinSkew Flex-N4 Flex-N2 Flex-P4

Throughput (tok/s)

Efﬁciency (tok/rank-s)

Rollout (s)

Reward

3711.3 ± 33.0 4952.2 ± 246.4 5221.3 ± 43.2 5372.9 ± 29.5 5393.7 ± 35.2 5575.1 ± 44.2 5471.2 ± 52.2 5668.7 ± 42.8

[231.9, 232.0] [309.5, 309.6] [326.3, 326.4] [371.2, 372.2] [374.1, 375.2] [384.8, 385.9] [380.6, 381.6] [383.2, 384.1]

4152.2 ± 25.7 3112.0 ± 165.2 2942.3 ± 26.5 2860.7 ± 11.2 2849.1 ± 17.6 2762.5 ± 25.6 2807.1 ± 27.5 2710.1 ± 20.4

0.8333 ± 0.0026 0.8366 ± 0.0030 0.8332 ± 0.0014 0.8342 ± 0.0017 0.8336 ± 0.0024 0.8343 ± 0.0007 0.8345 ± 0.0007 0.8346 ± 0.0028

Table 2. Eager and graph response throughput, seed0 epoch1. Each arm performs ﬁve updates. Method Eager Graph Gain verl baseline 1964.13 3746.71 +90.76% Flex-P4 2515.24 5656.35 +124.88%

Table 3. Flex-P4 TailGuard ablation. The guarded control is reused from the primary suite. Resource-efﬁciency ranges describe timestamp resolution, not statistical conﬁdence intervals. +TailGuard No TailGuard Throughput (tok/s) 5668.7 ± 42.8 5469.2 ± 122.4 Efﬁciency (tok/rank-s) [383.2, 384.1] [376.9, 377.7] Reward 0.8346 ± 0.0028 0.8333 ± 0.0020 Mean length 6000.7 ± 11.6 5989.5 ± 15.2

Held-out Accuracy After Five Updates. To assess model quality after training, we evaluate one checkpoint per training conﬁguration after ﬁve additional policy updates. All checkpoints answer the same 256 questions excluded from training, using identical nonelastic eager serving with greedy n = 1 and native sleep2. Table 4 reports accuracies of 82.42% for the verl baseline, 81.25% for FlexP4, and 82.81% for Flex-N2. Relative to the baseline’s 211 correct answers, Flex-P4 answers three fewer and Flex-N2 one more. All three checkpoints produce nonempty answers. 5.7

Sidecar Rank-Time Reuse

We test three auxiliary models under the same seed0 FlexN4 post-training protocol, each over two updated-policy epochs and ten steps. Both primary and sidecar use graph execution. Sidecar serves GSM8K with greedy n = 1 and a 4,096-token limit. A history-only predictor requires a forecast strictly greater than 60 s before lending eight inactive

Table 4. Accuracy on 256 held-out questions after ﬁve additional policy updates. All checkpoints use the same greedy evaluation conﬁguration. Training conﬁguration Correct / 256 Accuracy verl baseline, eager Flex-P4, eager Flex-N2, graph + sidecar

211 208 212

82.42% 81.25% 82.81%

Table 5. Additional inference during Flex-N4 post-training. Requests include shutdown drain. Params denotes total parameters. Model Params (B) Requests Accuracy Qwen2.5-1.5B Qwen3-8B PanGu-MoE

1.54 8.2 72

28,730 218 168

75.30% 96.79% 90.48%

devices, without additional NPUs. Table 5 reports completed requests and answer accuracy. Counts include shutdown drain and exclude unﬁnished requests. Available windows differ across runs, so these are service demonstrations, not matched model benchmarks. Outputs match durable completion records and actual FULL replay. Physical process-group exit and matching ACK precede primary restoration. Each primary run produces 5,120 nonempty responses with 160 rank-step FULL audits and zero KV preemptions. These results establish additional service and safe handback. Layouts, scoring, and output recovery are in the appendix. 5.8

Table 6. Frozen code rollout, seed401. Pass@1 scores one ﬁxed answer per problem against all tests. Flex-N2 releases ranks in step1 only. Method Q (tok/s) E (tok/rank-s) Pass@1 verl baseline Flex-P4 Flex-N2

4389.08 274.32 44.38% 5494.14 343.38 43.75% 5355.59 [357.51, 357.64] 43.75%

R ELATED W ORK AND L IMITATIONS

Tail-aware RL batching and prediction address synchronization skew (Gao et al., 2026, He et al., 2026). TLTstyle reuse exploits slack between workers (Hu et al., 2026). FlexFloor targets additional EP-coupled occupancy inside an active worker. MoE systems optimize expert dispatch, placement, and sparse computation (Rajbhandari et al., 2022, Hwang et al., 2023, Gale et al., 2023). FlexFloor instead coordinates completion and expert-world transitions to release ranks. Paged KV and iteration-level serving improve inference (Kwon et al., 2023, Yu et al., 2022), while RL frameworks coordinate training and inference (Sheng et al., 2025, Hu et al., 2024). Our contribution is planner/runtime co-design that reduces occupied rank-time per generated response token. Evaluation covers Qwen3-30B-A3B mathematics and code rollouts on 16 Ascend devices. Expert residency and KV capacity determine feasible ﬂoors. Separate controls characterize graph paths and isolate quorum-aware placement. Sidecars demonstrate additional inference service and safe handback. Short training and frozen-policy code experiments motivate broader validation.

Additional Code-Generation Workload

We compare one frozen-policy seed401 epoch on 160 code problems from Eurus-2-RL, with ﬁve steps and 16 responses per problem. Both arms share the code epoch0 checkpoint, inputs, response limit 16,384, and graph backend with native sleep2. Flex-N2 improves response throughput by 22.02% and reduces rollout time by 17.77%, while generating 0.34% more response tokens (Table 6). The baseline incurs 61 recovered KV preemptions versus zero for Flex-N2. Flex-N2 plans ﬂoors [4, 16, 16, 16, 16] and actually shrinks through eight to four ranks in step1, releasing 12 ranks. Resource efﬁciency improves by approximately 30.35%. The four remaining steps retain Full16, limiting release opportunity. The earlier Flex-P4 control is also shown. Quality uses each problem’s prespeciﬁed sample0 answer under OS-isolated full test suites. We obtain 71/160 versus 70/160 passes. Failures and resource limits remain in the denominator. This is sampled pass@1, not best-of-16, RL convergence, or statistical equivalence. Judge schema repair and limits are documented in the appendix.

C ONCLUSION

FlexFloor coordinates response scheduling and elastic execution to accelerate MoE RL rollouts while releasing resources earlier. On a common graph-enabled backend, it improves throughput by 52.74% and rank-time resource efﬁciency by approximately 65.40%, with little observed variation in rollout reward. Beyond TLT-style reuse between workers, FlexFloor releases resources within unﬁnished workers, exposing complementary capacity for useful work. This ﬁner-grained reuse turns long-tail slack into opportunities for productive computation. Faster response generation and lower occupied resource costs support more economical training iterations and the continued advancement of large language models and AI agents. Future work will explore more diverse sidecar tasks, deeper integration with speculative decoding, and extensions to agentic RL systems.

R EFERENCES Fedus, W., Zoph, B., and Shazeer, N. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efﬁcient Sparsity. J. Mach. Learn. Res., 23:120:1– 120:39, 2022. URL https://jmlr.org/papers/ v23/21-0998.html. Gale, T., Narayanan, D., Young, C., and Zaharia, M. MegaBlocks: Efﬁcient Sparse Training with Mixture-ofExperts. In Conference on Machine Learning and Systems, 2023. URL https://proceedings.mlsys. org/paper_files/paper/2023/hash/ 5a54f79333768effe7e8927bcccffe40Abstract-mlsys2023.html. Gao, W., Zhao, Y., An, D., Wu, T., Cao, L., Xiong, S., Huang, J., Wang, W., Yang, S., Su, W., Wang, J., Qu, L., Zheng, B., and Wang, W. RollPacker: Taming Long-Tail Rollouts for RL Post-Training with Tail Batching. In USENIX Symposium on Networked Systems Design and Implementation, pp. 849–866, 2026. URL https://www.usenix.org/conference/ nsdi26/presentation/gao-wei. He, J., Li, T., Feng, E., Du, D., Liu, Q., Liu, T., Xia, Y., and Chen, H. History Doesn’t Repeat Itself but Rollouts Rhyme: Accelerating Reinforcement Learning with RhymeRL. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 929–945, 2026. doi: 10. 1145/3779212.3790172. URL https://doi.org/ 10.1145/3779212.3790172. Hu, J., Wu, X., Wang, W., Xianyu, Zhang, D., and Cao, Y. OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework. arXiv preprint arXiv:2405.11143, 2024. doi: 10.48550/ARXIV. 2405.11143. URL https://doi.org/10.48550/ arXiv.2405.11143. Hu, Q., Yang, S., Guo, J., Yao, X., Lin, Y., Gu, Y., Cai, H., Gan, C., Klimovic, A., and Han, S. Taming the Long-Tail: Efﬁcient Reasoning RL Training with Adaptive Drafter. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 1933–1948, 2026. doi: 10.1145/3779212.3790231. URL https://doi. org/10.1145/3779212.3790231. Huawei Technologies. CANN MoE Dispatch Operator Documentation. Ascend Extension for PyTorch API Reference, release 7.2.0, n.d. URL https://github.com/Ascend/op-plugin/ blob/7.2.0/docs/context/torch_npunpu_moe_distribute_dispatch_v2.md. Accessed October 30, 2026.

Hwang, C., Cui, W., Xiong, Y., Yang, Z., Liu, Z., Hu, H., Wang, Z., Salas, R., Jose, J., Ram, P., Chau, J., Cheng, P., Yang, F., Yang, M., and Xiong, Y. Tutel: Adaptive Mixture-of-Experts at Scale. In Conference on Machine Learning and Systems, 2023. URL https://proceedings.mlsys. org/paper_files/paper/2023/hash/ 5616d34cf8ff73942cfd5aa922842556Abstract-mlsys2023.html. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., and Stoica, I. Efﬁcient Memory Management for Large Language Model Serving with PagedAttention. In ACM Symposium on Operating Systems Principles, pp. 611–626, 2023. doi: 10. 1145/3600006.3613165. URL https://doi.org/ 10.1145/3600006.3613165. Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., and Chen, Z. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding. In International Conference on Learning Representations, 2021. URL https:// openreview.net/forum?id=qrwe7XHTmYb. NIST/SEMATECH. e-Handbook of Statistical Methods. National Institute of Standards and Technology, 2012. URL https://www.itl.nist.gov/div898/ handbook/prc/section3/prc312.htm. April 2012 revision. Section 7.3.1.2, Conﬁdence intervals for differences between means. Accessed November 4, 2026. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., and Lowe, R. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022. URL http://papers. nips.cc/paper_files/paper/2022/hash/ b1efde53be364a73914f58805a001731Abstract-Conference.html. Rajbhandari, S., Li, C., Yao, Z., Zhang, M., Aminabadi, R. Y., Awan, A. A., Rasley, J., and He, Y. DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. In International Conference on Machine Learning, pp. 18332–18346, 2022. URL https://proceedings. mlr.press/v162/rajbhandari22a.html. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open

Language Models. arXiv preprint arXiv:2402.03300, 2024. doi: 10.48550/ARXIV.2402.03300. URL https: //doi.org/10.48550/arXiv.2402.03300. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. V., Hinton, G. E., and Dean, J. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations, 2017. URL https:// openreview.net/forum?id=B1ckMDqlg. Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. HybridFlow: A Flexible and Efﬁcient RLHF Framework. In European Conference on Computer Systems, pp. 1279–1297, 2025. doi: 10.1145/3689031.3696075. URL https://doi. org/10.1145/3689031.3696075. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, M., Xue, M., Li, M., Zhang, P., Wang, P., Zhu, Q., Men, R., Gao, R., Liu, S., Luo, S., Li, T., Tang, T., Yin, W., Ren, X., Wang, X., Zhang, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Zhang, Y., Wan, Y., Liu, Y., Wang, Z., Cui, Z., Zhang, Z., Zhou, Z., and Qiu, Z. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388, 2025. doi: 10.48550/arXiv.2505.09388. URL https://arxiv. org/abs/2505.09388. Yu, G.-I., Jeong, J. S., Kim, G.-W., Kim, S., and Chun, B.G. Orca: A Distributed Serving System for TransformerBased Generative Models. In USENIX Symposium on Operating Systems Design and Implementation, pp. 521–538, 2022. URL https://www.usenix.org/ conference/osdi22/presentation/yu.

AI U SAGE D ISCLOSURE OpenAI Codex was used to assist with grammar and wording reﬁnement, manuscript writing and revision, and implementing parts of the system. The authors retain responsibility for reviewing and validating AI-assisted text, code, and analyses, and for all technical claims, references, experimental results, and conclusions reported in this paper.

A PPENDIX Table 7. Frozen effective KV token budgets per rank. These are usable conﬁgured capacities after residency reservations, not literal graph bytes. Floor / family Natural Planned 2 4 8, Fixed/MinSkew 8, N2/N4/P4 16

131,072 Excluded 280,576 133,120 288,256 N/A 349,952 235,264 380,800 380,800

S COPE AND F ROZEN C APACITY M ATRIX

This appendix supports the paper’s two primary outcomes, throughput Q = W/T and rank-time resource efﬁciency E = W/H. All empirical values use the completed v0.14 runs. The reused mechanism ﬁgures and toy example are schematics, not empirical measurements from another backend. The primary suite has eight timed methods, seeds 0/2/3, and two sequential updated-policy epochs per seed. The no-TailGuard control adds three trajectories of two epochs. The combined accounting covers 27 trajectories, 54 epochs, 270 steps, and 138,240 responses. The guarded control is not counted twice. Planned-F2 is a budget-excluded category, not a timed arm. Planned ﬂoor4 reserves 147,456 more tokens of headroom than Natural ﬂoor4, and its ﬂoor8 reserve is 114,688 tokens. Applying the ﬂoor4-style reserve to ﬂoor2 yields 131072 − 147456 = −16384, explaining the conﬁgured Planned-F2 exclusion. This is not an exhaustive claim about every possible implementation of that residency policy. Natural-F2 uses the corrected ﬂoor8 budget, equal to Natural-F4. A separate three-step capacity diagnostic actually allocates 131,072, 349,952, and 349,952 tokens per rank, with real actor updates and 48 FULL rank-step audits. It is excluded from performance results. Other families retain their calibrated withholding. A zero extra withholding offset at another ﬂoor does not imply zero physical graph memory. Cold bootstrap blocks also do not describe the actual per-step cache after adaptive resizing.

P LANNER F ORMULATION

A step has 32 vertices and at most 496 candidate promptpair edges. Only pairs passing ﬂoor-speciﬁc calibrated KV admission enter matching. Each selected edge assigns one pair to an initial rank. Perfect-matching constraints cover every prompt exactly once. Cumulative completion quotas

of 8, 12, and 14 implement the possible 16-to-8-to-4-to-2 ladder. Schedules are enumerated in descending modeled release area. For a ﬁxed schedule, a secondary preference minimizes X c e xe , ce = 106 gape + 103 Me + Aµ,e . (14) e

Here gape is predicted pairwise tail difference, Me is pair completion, and Aµ,e is calibrated active-token peak. These are tie preferences within a release schedule, not a global lexicographic dominance proof for arbitrary length ranges. The solver’s feasible set includes every admissible MinSkew candidate on the same graph. This concerns the ﬁxed-length release model. It does not prove lower measured T or H, which also depend on compute rate, group construction, capture, cleanup, and restore. Physical placement maps early pairs to donor roles and long pairs to ﬁnal survivors. A safe shallower ﬂoor remains a valid plan when deeper admission or completion conditions are not met. The epoch repair loop invokes the complete step planner and only changes adjacent sorted steps when no allowed ﬂoor yields a valid assignment. Feasibility repair is not an unrestricted second permutation optimizer.

O CCUPIED R ANK -T IME E VENT L EDGER

For each rollout window we charge all initial ranks from its logged start. A detached rank becomes eligible for release only after its own shrink RPC and subsequent native sleep complete. Its released interval ends at the earliest observed restore or wake start. A retained rank is charged for the entire window. The measured progressive ladder has one released interval per detached rank and subsequent reacquisition before window end. Native-sleep timestamps are logged at second resolution. Restore completion records include elapsed RPC duration. A completion-time interval [t− , t+ ] and duration d imply a start interval [t− − d+ , t+ − d− ], including duration rounding. Explicit earlier restore requests and wake records also close the free interval when present. Let [ℓ− , ℓ+ ] bound release readiness and [u− , u+ ] bound reacquisition, clipped to the rollout window. The free interval satisﬁes F − = max(0, u− − ℓ+ ), F + = max(0, u+ − ℓ− ).

(15)

Consequently, H ∈ [N T −

F +, N T −

when converting efﬁciency gain to cost reduction. The two headline gains are complementary but not additive.

F − ],

E ∈ [W/H + , W/H − ].

(16)

Only conﬁrmed lifecycle intervals are subtracted. A live process or allocated communication handle is not independently equated with full compute occupancy, and incomplete native cleanup is never counted as free. The metric is primary compute-rank allocation under this declared protocol, not a claim that all HBM is free or every consumer ﬁts. All 4,320 rank-step ledgers are checked, and all 1,680 detached-rank windows show restore reacquisition before rollout end. The parser checks complete per-rank runtime audits, uniqueness of release, positive occupied duration, and absence of logged primary decode inside a credited free interval. It accounts for normal request preemption through the observed occupied duration without reconstructing local completion from token counts. Why restore must be charged. For Flex-P4, renewed occupancy contributes approximately 236 to 276 rank-s per epoch. A rule that stops charging at ﬁrst native sleep and credits the entire remaining sufﬁx would omit this cost and overstate resource efﬁciency. Including restoration gives approximately 65.40%, whereas omitting it would give approximately 66.4%. The primary results use the full lifecycle. These ranges describe recorded-clock resolution, not statistical conﬁdence intervals. They do not certify universal sidecar admission latency. If more than one free interval occurs in a future execution mode, all occupied segments must be integrated without overlapping or double counting.

AGGREGATION AND W ORK ACCOUNTING

For seed s and two measured epochs, we compute Qs =

Ws,1 + Ws,2 , Ts,1 + Ts,2

Es =

Ws,1 + Ws,2 . Hs,1 + Hs,2

(17)

Seed means are equally weighted. Relative gains are ratios of these means. Efﬁciency bounds propagate occupancy uncertainty before aggregation. The plot uses interval midpoints and between-seed sample SD, while the main table gives separate clock-resolution envelopes. Averaging step percentages would overweight short windows and is not used. The per-run inverse cost Cs = Hs /Ws is computed independently. A mean cost must not be obtained by inverting an already averaged efﬁciency. The same caution applies

The earlier allocation fraction 1 − D/(N T ) answers a different question about trapped dummy time inside a provisioned node window. It is not W/H and is not the resourceefﬁciency outcome in this revision. Released time is removed from H rather than also added to W . Saved response masks supply generated work. Only a veriﬁed ﬁrst padding EOS beyond a short cap is removed. Retokenized text and prompt-inclusive console throughput are not used. Preempted-token recomputation does not increase retained output work, but its time remains charged. For paired seed intervals we use three trajectory pairs and t0.975,2 = 4.30265. Two sequential updated-policy epochs are not independent replicates. Matched engine seeds do not imply matched per-request random draws after sorting or shrink.

S UPPORTING T IMING , R ELEASE , AND Q UALITY R ESULTS

Fixed-F4 seed3 epoch2 has one normal KV preemption. The original output and recovery time are retained. LengthSort and LS+TG incur 109 and 118 preemptions, whose delays are included in both primary outcomes. No NPU OOM occurs in the reported suites. A recovered scheduler preemption is not an allocator failure and is not discarded as an unfavorable run. In the TailGuard control, 24 of 12,288 responses under short caps reach a short boundary. Another 592 reach the global limit, versus 591 without the guard. These are boundary proxies rather than exact stop-reason labels. Counting all 616 guarded boundary responses as extra guard truncation would be incorrect. Mean length and reward do not establish long-run training equivalence or unbiased prediction from censored histories. The unguarded arm exposes more raw release opportunity but lower Q and E. The supporting proxy is useful for mechanism inspection but cannot replace occupied ranktime normalization.

G RAPH AND N ATIVE -S LEEP D ETAILS

The main paper keeps graph support within the elastic runtime rather than presenting a separate compiler contribution. Shape equality is insufﬁcient for capture reuse across different survivor identities, communicators, expert maps, weights, or KV addresses. Role synchronization and current-policy preparation precede capture, and recapture acknowledgement follows success. Preﬁll and scheduler recomputation use the eager meta-

4000 -25.1%

-29.1%

-31.1%

-31.4%

-33.5%

-32.4%

-34.7%

3000 2000 1000

Exit-based Release Proxy (rank-s)

7.89% vs TG

Budget excluded

Epoch Rollout Time (s)

5000

8000 7000

(b) Released Rank-Time Opportunity 16 → 8 8→4 4→2 5.52k

6000

5.75k

5.78k 5.30k

5000

4.11k

4000

Budget excluded

(a) Rollout Critical Path

3000 2000 1000

0 verl LS baseline F16

LS Fixed Min Flex +TG F4 Skew N4

Flex N2

Flex P4

verl LS baseline F16

Flex P2

LS Fixed Min Flex +TG F4 Skew N4

Flex N2

Flex P4

Flex P2

Figure 5. Supporting wall time and exit-start release opportunity using the original HPCA plot style with v0.14 data. The right panel is a proxy extending to rollout output, not the free intervals used in H. Native cleanup and restore reacquisition explain the difference. Table 8. Supporting primary-footprint and work measurements. Occupancy and cost ranges describe recorded-clock resolution. Release is an exit-start opportunity proxy. Preemptions total the six measured epochs per method. Method Occupied rank-s/epoch Cost (rank-s/1M tok) Mean length Exit proxy (rank-s) Preempt. verl baseline LengthSort LS+TG Fixed-F4 MinSkew Flex-N4 Flex-N2 Flex-P4

[66435.5, 66435.6] [49792.0, 49792.1] [47076.9, 47077.0] [41300.7, 41396.8] [40967.9, 41072.0] [39917.5, 40013.6] [40246.7, 40350.8] [40002.7, 40082.8]

[4311.3, 4311.4] [3236.2, 3236.3] [3064.4, 3064.5] [2687.0, 2693.4] [2665.9, 2672.8] [2592.0, 2598.3] [2620.6, 2627.5] [2604.0, 2609.3]

6019.4 6009.4 6000.8 6004.0 6002.7 6015.7 5999.0 6000.7

0.0 0.0 0.0 5518.2 5749.4 5304.3 5775.7 4106.0

0 109 118 1 0 0 0 0

data path rather than inheriting decode buffers. Decode updates token positions, sequence lengths, and KV slot mappings under the backend Attention contract. FlexFloor preserves this contract across distributed world and allocation changes.

Execution-mode audits establish actual full-decode dispatch, not token/logit parity at every boundary or zero physical graph overhead. A zero additional KV-withholding offset does not imply graph allocations are absent.

Cleanup synchronizes outstanding work before retiring graph wrappers, outputs, task handles, and pool references. References through compiled wrappers matter as well as ordinary module children. Expert maps, active expert masks, logical-to-physical maps, and relevant expert counts have different lifetimes and are preserved for model preparation. They are not stale execution handles.

Native sleep2 discards rollout allocations at the RL phase boundary. KV-ﬁrst wake selects the per-step cache before weight reload competes for residual memory. Capacityaware pool identity prevents incompatible remapping. Parameter metadata and normal post-load initialization remain intact, and current weights and replicas are installed before capture. The native path is not replaced by manual freeing.

R EPRODUCTION AND S COPE

The separate reproduction archive includes the frozen primary and TailGuard summaries, source digests, per-rank occupancy ledgers, ofﬂine parser, tables, and plot code. It excludes private weights, checkpoint shards, prompt text, raw generated responses, and execution caches. Its scripts rebuild the ﬁgures and metrics from this evidence. Regenerating raw event extraction requires the local archived logs referenced by the ledger. Completed eager controls, quota mechanisms, held-out quality, and actual consumer work are documented below. They are separate protocols, not added replicates for the primary suite. All summaries and their hashes are included in that archive.

Table 9. One controlled quota pair. Waiting includes dummy execution and metadata synchronization, not isolated kernel busy time. Metric MinSkew Quota Window time (s) Q (response tokens/s) H (occupied rank-s) E (response tokens/rank-s) Completion-to-exit waiting (rank-s) Floor8 data quorum (s) Floor8 coordinated RPC end (s) Floor4 data quorum (s) Floor4 coordinated RPC end (s)

392.41 385.82 4358.73 4433.20 4791.98 4458.97 356.93 383.59 750.26 290.65 229.56 194.29 245.68 210.41 293.41 287.43 308.59 303.11

C ONTROLLED Q UOTA -P LANNING M ECHANISM

The prespeciﬁed constructed-length case uses 32 real rawtoken prompts, 16 responses per prompt, and the same frozen tensor-audited common actor. Both arms share a 398-edge feasible matching graph, 133,120 KV tokens per rank, 32-row physical Planned-F4 expert capacity with eight initial primary rows at EP16, ﬁxed eight-bucket captures, request seeds, and native sleep2/lifecycle1. Only matching and placement change. Ignore-EOS response lengths ﬁx generated work at 1,710,400 tokens in both arms. This isolates scheduling under a disclosed length pattern, not natural response quality. Waiting starts at each rank’s local completion and ends at its exit-RPC start, or global generation end for retained ranks. Actual native-sleep completion opens each free interval, and restore closes it. All 16 ranks have positive allocator release observations in each arm. Both execute FULL decode after ﬂoor4 and complete the EP16 restore. The controlled window includes transition capture, cleanup, sleep, and restore, but not initial checkpoint loading and initial capture. Its absolute throughput therefore does not share the main epoch window. The coordinated EP-detach proxy increases by 226.24 ranks, decomposable into 304.13 from earlier stage endpoints and −77.89 from the shorter tail. This is an algebraic decomposition, not a ﬁxed-tail causal rerun, and the proxy is not substituted for physically released intervals in H. One pair provides no population conﬁdence interval or measured global optimality proof.

Table 10. Disjoint reusable rank-time in the two-worker composition run. All entries are rank-seconds. Step Worker-level Within-worker Combined

W ORKER -L EVEL AND W ITHIN -W ORKER R EUSE

One seed0 frozen-policy run uses the common math epoch0 checkpoint and 160 DeepScaleR prompts over ﬁve steps. Two independent EP8 workers share the same 16device allocation, each receiving 16 prompts per step and

1 2 3 4 5

235.5 59.5 430.9 138.6 148.7

493.6 810.3 557.2 1453.9 0.0

729.1 869.9 988.1 1592.5 148.7

Total

1013.3

3315.0

4328.3

16 responses per prompt. The planner sorts the global step buckets, alternates prompts between workers, and applies quota-aware pairing within each worker. Two bounded cross-step swaps satisfy prompt-inclusive KV admission. Natural ﬂoor2, ﬂoor4, and ﬂoor8 capacities are 131,072, 280,576, and 349,952 tokens per rank. Both workers select ﬂoors [2, 2, 4, 4, 8]. This mechanism control uses ﬁxed capture sizes [1, 2, 4, 8, 12, 16, 24, 32]. Each step exercises resource release and subsequent wake while policy updates and sidecar tasks are skipped. The experiment measures the worker-level reuse boundary using only the primary MoE rollout. Let ri be rank i’s conﬁrmed resource release time, Bw = maxi∈w ri the release-completion time of worker w, and G = maxw Bw . With P eight ranks per worker, worker-level reuse is Rinter = 8 w (G − Bw ). For the set Dw of ranks that actually detach before worker completes, FlexFloor additionally Pw P exposes Rintra = w i∈Dw (Bw − ri ). The intervals are disjoint. Ranks that never detach contribute no withinworker credit for differences in cleanup timing. Restore time and the shared handoff gap after G are excluded from both components. The run completes with exit0 and produces 2,560 nonempty responses and 15,358,973 response tokens. All 80 rankstep audits conﬁrm FULL decode, with zero KV preemptions. Each of the four shrinking steps restores all 16 ranks to their original EP8 groups, yielding 64 topologyrestoration records. The ﬁnal step retains EP8 and completes the driver restoration barrier without rebuilding its unchanged groups. Ofﬂine scoring with the original reward function yields 0.833594. The retained log also records a TBE repository daemon ProcessLookupError during process teardown after the completed run. Release events, perrank audits, source hashes, and the shutdown diagnostic are preserved in the composition evidence.

C ONFIGURED E AGER AND G RAPH C ONTROLS

The seed0 controls perform ﬁve actual actor updates from the common epoch0. The corresponding primary graph

epoch is reused, not a two-epoch average. Each method matches ordered input/ground-truth pairs, 80 actual perrank KV/ﬂoor conﬁgurations, native sleep2/lifecycle1, and historical execution source ﬁngerprints. Execution modes follow the main experimental setup. The verl baseline takes 7839.10 s eager versus 4126.04 s FULL, generating 15,397,030 versus 15,459,094 response tokens. Flex-P4 takes 6091.05 versus 2699.78 s, generating 15,320,458 versus 15,270,884 response tokens. Each throughput uses its own actual generated work. Each arm follows its own updated policy, so the epoch is not a frozen token-paired replay. Before the ﬁrst new update, all 512 verl-baseline answer texts and the entire length vector match. Flex-P4 has 355/512 identical texts and different lengths. These performance controls do not establish Flex-P4 token parity, and text equality alone is not a token/logit parity test. Boundary checks must separately cover ﬁrst decode, weight reload, shrink, and restore. The Flex-P4 eager child originally completed all ﬁve steps with exit0. An ofﬂine parent validator returned 3 because it incorrectly required a graph-only invalidation log in eager mode. The original failure is retained. Independent native-sleep completion and positive allocator release counters were audited across all 80 rank-steps without rerunning training or fabricating graph events. This distinction is recorded in the bundled evidence.

S HORT-T RAINING H ELD -O UT Q UALITY

All three endpoints are evaluated on the same prespeciﬁed 256 questions, disjoint from the 160 training prompts. Evaluation is nonelastic eager, native sleep2, greedy n = 1 for every endpoint. Checkpoint loading on all 16 ranks and question/ground-truth identity were veriﬁed. The verl baseline and Flex-P4 were trained eager for ﬁve additional updates. Flex-N2 uses its ﬁrst primary-graph/graph-sidecar training epoch under the corrected 349,952-token ﬂoor8 budget, also ﬁve updates. This is not an epoch2 endpoint. The correct counts are 211, 208, and 212, respectively, with no empty answers. Between the ﬁrst two, seven questions are correct only for the verl baseline and four only for FlexP4, a −1.17 percentage-point difference. Flex-N2 is +0.39 points relative to the baseline but changes multiple components. These are local short-training observations from one seed, not evidence of non-inferiority, ﬁnal convergence, or a graph-speciﬁc quality gain.

Table 11. Flex-N4 graph-sidecar windows. Times are seconds. Forecast is at admission, lease is launch to primary stop, ready is launch to all eight replicas ready, exit is stop to conﬁrmed process exit, and queries exclude drain. Epoch/step Forecast Lease Ready Exit Queries 1/3 1/4 2/2 2/3 2/4

90.00 82.31 162.00 178.64 95.43 94.54 126.00 122.94 112.00 115.93

56.61 4.65 46.72 4.63 46.75 4.47 46.67 4.95 47.33 4.59

2466 10513 4565 5877 5062

F LEX -N4 G RAPH S IDECAR WITH A S TRICT 60-S ECOND G ATE

The seed0 Natural-F4 run starts from the protected common math epoch0, updates the actor at 10−6 over ﬁve steps, then resumes its own epoch1 checkpoint and history for ﬁve more updates. Primary workload, sampling, TailGuard, KV matrix, eight historical buckets, native sleep2/lifecycle1, KV-ﬁrst wake, and graph execution follow the main protocol. There are 5,120 primary responses with zero preemptions and 160 actual rank-step FULL audits. Epoch rollout times are 2866.53 and 2856.94 s, response throughputs 5389.55 and 5351.65 tokens/s, and rewards 0.829688 and 0.834375. The pooled primary throughput is 5370.63 tokens/s versus 5524.38 without sidecar, a descriptive 2.78% decrease. The reference updates its own policy, so these values do not identify causal interference. Mean response length is 6003.64 tokens, with no empty primary outputs or KV preemptions. Each admitted 16-to-8 handoff lends eight inactive devices to independent Qwen2.5-1.5B-Instruct TP1 replicas. Sidecar uses eager preﬁll and capture sizes [1, 2, 4, 8, 16, 32, 64, 128]. Greedy sampling, 4,096-token response limit, 6,144-token model limit, maximum 128 sequences, and memory fraction 0.80 are unchanged. Sidecar does not perform FlexFloor shrink, policy updates, or native sleep. Only the primary changes expert world. Optional FX fusion is disabled without disabling native FULL decode capture. Admission uses only historical estimates, completed earlier-step lengths, and observed pre-shrink decode progress. It requires forecast > 60 s, not ≥ 60 s. Exactly 60 s skips. Epoch1 starts steps3/4 and skips steps1/2 with forecasts 32.16/47.15 s. Epoch2 starts steps2/3/4 and skips step1 at 30.57 s. Both step5 runs retain Full16. The ﬁve start decisions lead to actual model execution, not enabledbut-skipped windows. Every window contains 64 capture events and FULL replay observations on all eight replicas, rather than relying on graph-enable ﬂags. Startup includes capture and process initialization, not certiﬁed uncached disk loading. Ob-

served all-eight readiness is 46.67 to 56.61 s, not an upper bound for all future loads. Stop-to-exit is 4.47 to 4.95 s and stop-to-ACK is 4.49 to 4.97 s. Five SIGTERM events follow grace, with no SIGKILL. Each started lease has durable output and conﬁrmed physical process-group exit before its matching ACK and primary restore. Before primary stop, the ﬁve windows complete 28,483 jobs. Drain adds 247 completions, giving 28,730 ﬁnal completions. Final partial queues contain 410 and 318 jobs by epoch, excluded from completions. Jobs repeat questions across dataset cycles and epochs. Partial preﬁxes are resumed only within an epoch, not counted as separate completed work. The eight-device assigned leases sum to 4,754.82 rank-s before stop, including startup. Drain adds 186.29 rank-s. These are assigned-device time, not AICore utilization or a physical-memory-release fraction. Skipped windows are retained in the admission record. We do not add tokens from different models into a combined throughput or infer graph acceleration by dividing query totals from different window schedules. Scoped sidecar graph-correctness evidence. A reused single-replica preﬂight checks 16 greedy queries on the same model and input data. For identical padded decode inputs, 465 pre-replay and 465 post-replay hidden/logit checks have zero absolute difference and no argmax mismatch. Diagnostic instrumentation does not change graphgenerated token IDs. However, free-running eager and padded-graph trajectories match exactly for only 12/16 queries. These facts are reported together, not converted into a universal greedy-parity or standalone-qualityequivalence claim. The two updated-policy epochs do not isolate a graph-only speedup or establish matched standalone quality equivalence. Completion-conditioned answer accuracy describes the served jobs, not a ﬁxed benchmark score. Additional auxiliary models and answer scoring. The auxiliary-model table in the main paper uses three separate runs under the same seed0 Natural-F4, two-epoch posttraining protocol. The auxiliary layouts are eight independent TP1 replicas for Qwen2.5-1.5B, one internal DP2/TP4 engine for Qwen3-8B, and one internal DP8/TP1 engine with EP8 for PanGu-MoE. All use eight released devices, greedy GSM8K train requests, the same response limit, and graph execution. Available windows are not frozen across runs. Answer accuracy uses the existing GSM8K numeric extractor on the ﬁnal 300 output characters, without requiring a particular answer delimiter. Correct counts are 21,634/28,730 for Qwen2.5-1.5B, 211/218 for Qwen3-8B,

and 152/168 for PanGu-MoE. The rule is applied uniformly to all completed requests, including drain. These completion-conditioned results do not rank model quality on a shared ﬁxed evaluation set. Qwen3-8B counts are reconstructed from the two DP leader output shards after aggregate-ﬁle collection failed. Every counted output is unique, matches a durable completion record and its source prompt, and belongs to a lease with actual FULL replay and physical exit before restore ACK. Unﬁnished and inﬂight requests are excluded. This reconstruction validates completed service without asserting that the original full-arm collector passed. PanGu-MoE completes 168 requests in one of its ﬁve startup windows. The other four end before all workers are ready. Both models pass a ﬁxed 16-request standalone eager/graph token-parity preﬂight, which does not establish full served-workload quality equivalence.

C OMPLETED C ODE -G ENERATION PAIR

The additional task uses 160 code problems from the validation split of Eurus-2-RL, covering APPS, CodeContests, Codeforces, and TACO. Problem selection, seed401, ﬁve steps, 16 samples per problem, a 16,384-token response limit, and ﬁxed sample0 quality scoring are prespeciﬁed. Both arms start from the same permanent code epoch0 and keep the actor frozen. All 2,560 original prompt/groundtruth/sample identities match across arms. Both use graph execution with native sleep2 and eight history-selected buckets. The verl baseline generates 27,187,974 response tokens in 6194.46 s. Flex-N2 generates 27,281,462 in 5094.01 s. Its response work is 0.34% larger, so 17.77% lower time and 22.02% higher throughput are distinct quantities. The baseline has 61 KV preemptions, all recovered normally. FlexN2 has zero. These costs remain included in the measured rollout intervals. Flex-N2’s planned ﬂoors are [4, 16, 16, 16, 16], with no response cap below 16,384. Worker completion logs conﬁrm eight-rank and four-rank groups in step1. Twelve ranks actually complete native sleep before restoration. The driver’s optional shrink-preﬁx counter is not a physical release test. Occupied rank-time is 99,111.41 versus [76,283.91, 76,307.92] rank-s, and efﬁciency is 274.32 versus [357.51, 357.64] response tokens per occupied rank-s. The E gain is [30.33%, 30.38%] with outward rounding. The four other steps retain Full16 because their predicted peaks exceed the ﬂoor8 budget, limiting cross-task release opportunity. All 160 combined rank-step FULL audits and both completed ﬁve-step driver exits pass. Quality is scored separately from the timed log’s placeholder zero reward. One prespeciﬁed sample0 answer

per problem runs against the complete test suite in an OS-isolated sandbox with 1 GiB address space, 64 MiB tmpfs, 8 s CPU, and 20 s wall limits. Failures and resourcelimit exits remain in the 160-problem denominator. The observed pass counts are 71/160 (44.38%) and 70/160 (43.75%). No best-of-16 selection or NPU generation retry is used, and the one-question difference does not establish statistical equivalence or code-task training convergence. The reused baseline’s original grading exposed an infrastructure mismatch between native list-valued tests and the pinned PRIME judge’s string-wire representation. A lossless adapter repaired 21 schema-affected baseline answers, retaining the other 139 scores. New Flex-N2 answers use the same adapter and unchanged judge, test values, expected outputs, and limits. All 160 are scored once. Preliminary baseline counts are not reused. The earlier valid Flex-P4 control remains a separate table row, with 5494.14 tokens/s, 343.38 tokens/rank-s, and 70/160 passes. It does not actually shrink. The lower N2 throughput does not imply less rank-time reuse. Coverage across completed experiments. All main and supplemental result tables use v0.14 measurements. The primary suite’s 3,840 FULL rank-step audits and TailGuard’s 480 cover 138,240 responses. Fixed-F4’s single preemption and LengthSort/LS+TG’s 109/118 preemptions remain charged rather than discarded. Separate frozen controls, updated-policy endpoints, and sidecar windows retain their own protocols. They do not become extra replicates of the primary suite. Natural-F2 versus Natural MinSkew has paired epoch-time difference −42.0 s with interval [−140.3, 56.3] s, which does not isolate the planner mechanism. The controlled pair in Section H addresses that attribution under ﬁxed lengths.

Resource-Efficient Reinforcement Learning Post-Training through Elastic Expert Parallelism

Analyze & explore related research in Conceptio

Related documents