Scaling VLA Serving Under Robotic Execution Constraints

S CALING VLA S ERVING U NDER ROBOTIC E XECUTION C ONSTRAINTS

A BSTRACT Vision-language-action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation. In our production setting, hundreds of VLA-powered robots are already operating in real-world environments, creating an immediate need for efficient fleet serving. We present XI NFER, a production-deployed VLA serving system designed to reduce the inference infrastructure cost of large-scale robotic deployments. XI NFER jointly coordinates heterogeneous VLA computation with the evolving response requirements of robotic execution. By exploiting computational parallelism while managing interference, XI NFER improves GPU efficiency while protecting time-critical robotic feedback. We evaluate XI NFER across three VLA models using production-derived workloads, achieving 1.26–3.75× the effective robot capacity (ERC) of state-of-the-art systems under physical-execution constraints. Our implementation is publicly available at https://github.com/ X-Square-Robot/xinfer.

Figure 1. Production parcel handling. A carton (top) and a soft parcel (bottom) are reoriented and pushed onto the conveyor. Each row follows the same parcel from left to right.

I NTRODUCTION

Vision-language-action (VLA) models are bringing generalpurpose robotic manipulation into logistics and manufacturing, where multiple robots can share GPU infrastructure to reduce serving cost. Figure 1 shows representative parcelhandling operations from our production deployment. Recent inference engines improve the efficiency of individual VLA queries (Su, 2026; Xu et al., 2026; Tan et al., 2025). However, higher inference throughput does not necessarily translate into greater physical productivity, as delayed feedback can disrupt robotic execution. We use effective robot capacity (ERC), the maximum number of concurrent robots a GPU can serve while meeting specified latency and reliability requirements, as the serving objective. Shared VLA serving requires exploiting concurrency without compromising time-critical feedback. VLA requests traverse heterogeneous stages with distinct computational demands, creating opportunities to overlap independent work across robots. However, concurrent GPU execution

FleetServe @ 16 31.2k parcels/h

FleetServe (ERC=18) FlashRT (ERC=5)

(103 parcels/h)

1

Projected throughput

arXiv:2611.14969v1 [cs.RO] 11 Nov 2026

X Square Robot Team

30 FleetServe @ 7 13.8k parcels/h

20 13.0k target

10 FlashRT @ 16 13.0k parcels/h

0 1

4

7

8

12

16

Number of concurrent robots

Figure 2. Projected parcel throughput on one GPU (VLA-2B), combining measured latency with single-robot trials. Guides mark 13k parcels/h; ERC requires response P50 < 150 ms and zero request errors. Shading reflects trial variation; dashed curves and hatching mark extrapolation.

also introduces interference: admitting a new stage may delay stages already in progress. Since these stages are

XInfer

non-preemptive, prioritizing urgent requests alone cannot prevent deadline violations caused by previously admitted work. Effective fleet serving therefore requires two coordinated decisions: which requests should be prioritized, and which stages can be admitted without jeopardizing the deadlines of current and forthcoming requests.

for Slack-Gate, 14 for FIFO, and 12 for EDF (§6). Mechanism ablations isolate the effects of asynchronous execution and scheduling decisions. Our contributions are: • An asynchronous VLA serving engine that exposes cross-request and cross-stage parallelism by advancing work as its dependencies complete.

The cost of this interference is not uniform across requests. Robots continue executing previously predicted actions while awaiting new responses, and the consequences of delayed feedback depend on their ongoing physical execution. As a result, requests with similar computational demands may have substantially different response-time requirements, which evolve throughout task execution. Moreover, response timing influences subsequent request arrivals, coupling scheduling decisions with future workload dynamics. Fleet serving must therefore balance concurrency against interference under changing per-request deadlines, rather than optimize inference throughput or average latency alone. Prior systems adapt action horizons and request priorities to execution context (Dai et al., 2026; Anonymous Authors, n.d.), coordinate serving for factory productivity (Jiang et al., 2026), or overlap encoding and diffusion in lockstep rounds (Adamopoulos et al., 2026). Yet effective fleet serving must reconcile the time-varying urgency of robotic feedback with the distinct computational demands of VLA stages. This calls for fine-grained scheduling that matches stage execution to available GPU capacity while controlling interference with time-critical requests. We present XI NFER, a VLA serving system that couples asynchronous execution with fine-grained, execution-aware scheduling. Its dependency-driven engine exposes parallelism across requests and heterogeneous VLA stages, allowing computation to progress as dependencies resolve. The scheduler coordinates these execution opportunities with the time-varying urgency of robotic feedback by decoupling request prioritization from stage admission. At stage boundaries, it combines per-request response-time budgets, remaining computation, and predicted interference to admit ordinary work only when protected requests remain predicted feasible. It further accounts for anticipated earlierdeadline arrivals using request history, balancing immediate concurrency gains against future scheduling demands. We evaluate VLA-2B, π0.5 , and Wall-OSS-0.5 using production-derived closed-loop workloads and controlled experiments. With error-free responses, XI NFER achieves ERCs of 18 VLA-2B and 24 π0.5 robots, versus 10 and 15 for vLLM-Omni; Robion serves 19 π0.5 robots. In a separate VLA-2B evaluation with heterogeneous response-time budgets, all policies share the same backend and execution resources. Under the same real-time feasibility criterion, XI NFER achieves an ERC of 18 robots, compared with 15

• Execution-aware cross-stage scheduling that coordinates shared GPU computation by separating request priority from stage admission, accounting for interference on current and anticipated urgent requests. • Production deployment and empirical evaluation. XI NFER is deployed in production. We characterize closed-loop workloads, compare ERC across three VLA backends, and evaluate heterogeneous-budget scheduling and its mechanisms.

2

BACKGROUND

2.1

Heterogeneous VLA Computation

Vision-language-action (VLA) models map observations and instructions to robot actions. We study models coupling a vision-language backbone to an action expert that generates continuous actions through flow matching or diffusion (Lipman et al., 2023; Chi et al., 2023; Black et al., 2025a; Yu et al., 2026; Zhai et al., 2025). As Figure 3(a) shows, the backbone computes conditioning keys and values once; the expert reuses them over K denoising steps to generate an action chunk of H future keyframes (Zhao et al., 2023). We distinguish three computational stages: ViT encodes visual observations, Prefill performs language-model computation after visual encoding, and Action generates actions through denoising. These stages differ in computation and memory-access patterns. The expert’s weights differ from the backbone’s, and its evolving action state is private to each request. 2.2

Closed-Loop Robot Control

Real-time chunking (RTC) (Black et al., 2025b) aligns independently generated chunks by freezing the next chunk’s first d keyframes to committed actions (Figure 3(b)). For a request issued at tk with keyframe spacing ∆, the planned handoff is tk + d∆ and its budget is d∆. A late chunk can cause an action discontinuity if applied without alignment to current execution progress. If the action queue empties, our controller pauses execution until a replacement chunk becomes usable. Longer commitment permits more inference time but postpones feedback, even when d adapts to recent latency (§3.1). Appendix B.1 gives the inpainting and handoff details.

XInfer

Action: K steps

ViT / Prefill: once

Response

Images ViT

Next send Switch tk + TE

Send tk

Backbone QKV

K/V

Expert QKV

Chunk k

reuse

Prefix attention Backbone FFN Per-layer K/V are cached for all denoising steps

Time

Noisy action state

Text / state

Chunk k+1

Action attention L transformer layers

Robot

Expert FFN

Execute current chunk

Execute returned chunk

L

Update action state

Handoff budget

Chunk: H keyframes

Frozen

(a) VLA computation and KV reuse

d∆

Guided

Fresh

Executed

(b) Real-time chunking

Figure 3. VLA computation and real-time chunking. (a) Conditioning keys and values are reused across denoising steps. (b) The frozen prefix aligns the next chunk with committed actions while inference overlaps execution. The illustrated early response satisfies L < d∆ < TE , so the next send occurs at tk + TE .

Fleet Workload and Request Contract

A fleet shares inference resources across robots, each running its own control loop. A request carries observations, robot state, and model inputs. Each request j is associated with a response-time budget bj that expresses its feedback requirement. The budget is supplied as request metadata and may be assigned by the robot controller, a task planner, or a policy at the serving frontend. RTC requests additionally supply the aligned previous action chunk and commitment length d. The response budget, RTC handoff budget d∆, and minimum request period TE are distinct quantities: they specify the serving requirement, the planned chunk handoff, and the earliest next send. Robots continue their current chunks while requests are processed. We use a minimum-period request policy with at most one outstanding request per robot. Let sj be the actual send time of request j and rj its response arrival time. Abstracting local client-side handling, the next request is sent at sj+1 = max(sj + TE , rj ). (1) The next send occurs when the minimum interval has elapsed and the previous response has arrived. Thus, N robots offer at most N/TE requests per unit time; late responses reduce both offered load and action-update frequency. Figure 4 overlays five streams from deployed robots, including two sessions from one robot. Requests occupy only 45–53% of each dedicated instance’s timeline; combined arrivals show gaps and bursts. Objective and service metrics. We use effective robot capacity (ERC), denoted N ∗ , to measure the maximum number of concurrent robots a GPU can serve while meeting specified latency and reliability requirements. Each

S1 S5

In flight

2.3

5 4 3 2 1 0

mean 2.45

0

500

1,000

1,500

2,000

Time (ms)

Figure 4. Five separately recorded request streams aligned on a common 2-s axis. Top: bars mark requests in flight and ticks mark sends. Bottom: the superimposed in-flight count.

comparison fixes the model, workload, and acceptance criterion. Response latency measures feedback delay. SLO attainment is the fraction of responses meeting their supplied budgets; on-time goodput counts such responses per unit time. Section 6.1 specifies the acceptance criteria and measurement windows for each experiment.

3

M OTIVATION

Sharing GPU capacity across independent robots must preserve timely feedback. The workload in Section 2 motivates three requirements: express changing feedback needs, expose independent work, and control concurrent interference. 3.1

Feedback Needs Change During Execution

In our delay-injection study, adding 200 ms before action delivery reduces completed parcel output by 29.6%. Separate fixed-count studies observe additional attempts for parcel reorientation and conveyor pushing, both contact operations. These results connect delayed feedback to reduced physical progress; Appendix A gives the protocols and results.

XInfer

Acceptable delay varies during execution. Contact operations, such as grasping, parcel flipping, and conveyor pushing, or interaction with a changing scene may require frequent updates. Predictable free-space transport may tolerate a longer execution horizon. Increasing the RTC commitment length buys response time but postpones feedback; it cannot replace timely service when rapid adaptation is needed. A fixed priority per robot cannot capture these changes. These observations call for per-request feedback requirements. Planned motion, execution progress, and applicationspecific signals, such as gripper events or contact observations, provide cues about how quickly feedback is needed. A budget-selection policy can map this execution context to a response-time budget for each request. The contract in Section 2.3 exposes these budgets to the scheduler through a uniform interface. Closed-loop throughput and goodput. By Equation 1, robot i’s kth response latency Li,k determines send interval τi,k = max(TE , Li,k ). Over robot i’s requests, define mean cycle overrun hi = E[(Li − TE )+ ] and attainment pi = Pr(Li ≤ bi ), where bi is the request’s budget and (x)+ = max(x, 0). For N continuously active robots with stationary, ergodic request sequences and finite mean latency, long-run throughput X and on-time goodput J satisfy X=

N X

1 , T + hi i=1 E

J=

N X

pi . T + hi i=1 E

(2)

3.3

Throughput Alone Does Not Protect Deadlines

Throughput gains need not improve tail latency. In a separate 32-robot π0.5 ablation, adding independent progression to multi-stream execution raises QPS by 9.7%, but P99 rises from 467.07 to 560.28 ms (Table 6). A wide-budget request’s Prefill can consume GPU resources needed by a tight-budget Action. Deadline ordering (Liu & Layland, 1973) chooses which queued request runs next, but moving the urgent request to the front does not release resources held by non-preemptive work. Urgency must account for remaining computation and interference, and starting another stage must depend on its effect on that work. Protecting only requests already present can also be too late: a long-running stage may still be active when an earlierdeadline request arrives. In matched experiments with alternating task phases, arrival-aware admission raises mean SLO attainment by 5.0 and 11.3 percentage points at 16 and 18 robots, respectively (Table 8). Conversely, holding all other work whenever a request becomes urgent can forgo useful overlap. The scheduler must therefore separate request protection from stage admission. Admission should permit useful overlap while keeping the deadlines of current and anticipated urgent requests predicted feasible. Urgency alone should not stop all other work.

4

D ESIGN

These long-run rates follow from one response per mean cycle TE + hi , with fraction pi on time. Delaying wide-budget requests can lengthen request cycles even when responses remain on time. Higher attainment can therefore coexist with lower goodput; scheduling must preserve update frequency as well as deadline attainment. Section 6.3 evaluates both effects.

XI NFER separates the availability of parallel work from the decision to execute it. An asynchronous engine makes work ready as its dependencies complete. A central scheduler identifies requests that need protection, then admits ordinary work only when the resulting interference preserves predicted deadline feasibility. This division addresses the tension in Section 3: exploiting parallelism without delaying urgent action updates.

3.2

4.1

Independent Requests Offer Parallelism

The streams in Figure 4 leave gaps in each dedicated instance yet produce bursts when combined. Sharing can fill these gaps by coordinating independent work. Each request follows ViT, Prefill, and Action dependencies, but CPU preparation can overlap GPU execution, and another request’s ViT or Prefill can progress during Action. Serializing complete requests or advancing stages in common rounds introduces waits that these dependencies do not require. At 24 robots, enabling the full execution engine on π0.5 raises throughput from 54.90 to 131.53 req/s (2.40×, Figure 9). These results motivate exposing work as its dependencies complete.

System Overview

The server consumes per-request response-time budgets and RTC inputs through the request contract in Section 2.3. Budget assignment and execution-horizon selection are separate from the scheduler’s request prioritization and stage admission. Figure 5(a) shows the ViT, Prefill, and Action pipeline, preceded by CPU preparation and followed by output processing. The engine maintains dependencies, ready queues, and bounded execution resources; the central scheduler controls admission at each stage entry. Solid arrows carry data, dashed arrows convey admission decisions, and dotted arrows return scheduler state.

XInfer (a) Overall architecture XI NFER Central scheduler

SLOs Arrival history

Request prioritization

CPU input

ViT Ready / run

Progress Resources

SLO-aware admission

Prefill Ready / run

Context process

Action Ready / run

CPU output

Action process

Shared GPU (MPS)

Robot clients (1, . . . , N) Robot request State / Images / Language SLO / RTC Observe

Execute

Action chunk RTC handoff

Independent control loops

(b) Asynchronous execution and admission A: protected ViT

D

A D

Prefill Action Robot A

Send A

Overlap cost C D (ii) A’s response deadline C (i) (iii) B waits B C

B

B A D

Current chunk

(i) Hold B’s Prefill to protect A (iii) A completes; recheck B

A

B C Response + switch Returned chunk

Time (ii) Admit C’s ViT alongside A

Figure 5. XI NFER architecture and illustrative execution. (a) Central scheduling controls admission at the ViT, Prefill, and Action entries. (b) Admission permits concurrent work when A’s response is predicted to meet its deadline.

streams support concurrency within the ViT/Prefill process; MPS (NVIDIA, 2026) permits the processes to execute concurrently on the shared GPU. Requests reuse read-only weights and retain private intermediate state. Bounded state slots propagate backpressure when a downstream stage cannot accept work. Dispatch admitted work. The engine dispatches admitted work at backend-supported execution boundaries. Each execution unit advances on its own completion events, without a common round barrier, and runs to completion. Request state remains private. Section 5 maps computational stages to execution units and specifies their dispatch boundaries. 4.3

Request Prioritization

The scheduler protects queued and running requests according to their remaining slack and the work that could delay them. Progress-aware service-time estimation. At server arrival aj , request j with budget bj receives scheduling deadline ℓj = aj + bj − δ, where δ reserves estimated time outside the server. Given projected concurrent stage counts n, a calibrated model Rj (n) estimates remaining service time, covering the current stage’s residual, subsequent stages, and output processing, excluding future queueing. At time t, the slack available for waiting and additional interference is σj (n) = ℓj − t − Rj (n).

The policy makes two decisions: which requests need protection (§4.3) and which ready stages may start (§4.4). Both use request progress and shared resource state; the latter also considers arrival history. Issued GPU units are nonpreemptive. Section 5 specifies each backend’s execution boundaries. In Figure 5(b), A has an urgent Action, B a ready Prefill, and C additional vision work; D is preceding ordinary work.

(3)

The predictor tracks each running stage’s progress by accumulating calibrated execution rates as concurrency changes, and refreshes Rj at each decision. Appendix C.1 specifies the model and calibration; Appendix C.2 evaluates remaining-time prediction for running stages.

Advance work by its dependencies. Each request follows ViT, Prefill, and Action dependencies without a common round barrier. Once Prefill supplies its conditioning state, Action can run alongside another request’s ViT or Prefill. CPU preparation and output processing can overlap GPU work from other requests. Admission controls interference before additional GPU work starts.

Blocking-aware promotion. Following non-preemptive response-time analysis (Audsley et al., 1993; Jeffay et al., 1991), let Bj be the maximum predicted remaining time of another request’s running ordinary unit, or zero if none exists. When σj < Bj , waiting for that unit could exhaust j’s slack, so the scheduler promotes j. In Figure 6(a), A is promoted because its 10 ms slack is below BA = 14 ms. Promotion persists while the request is live. Effective protection is withheld while the request is predicted late even in isolation; it remains eligible for service and can regain protection if the prediction changes.

Separate submission paths. ViT and Prefill share a process; Action runs in another, with independent host submission loops and private execution state. This avoids serializing their host work through one submission loop. CUDA

R EFRESH P RIORITY returns the effective protected set H, including queued and running requests, and foreseeable load nf : running units plus protected ready work that available resources can issue. Ready requests are ordered by protected status and then deadline.

4.2

Asynchronous Stage Execution

XInfer A: ℓA − t = 50 ms, RA = 40 ms Slack

σA = 10

1. Check Prefill u 6 extra delay

4 left

∆A (u) = 6 ≤ 10

Admit u

Ordinary running units (remaining ms)

ViT

4

Prefill Action

Refresh state

14 8

BA = max(4, 14, 8) = 14 ms σA < BA ⇒ Protect A

w passes the current-request check

Protected A: shared slack = 10 ms

Forecast e has an earlier deadline: ℓe < ℓw ℓe ℓw t

w

If admitted now

e

Forecast

2. Check ViT v 4 left

Arrival

+2

Defer v

∆A (v) = 6 > 4

(a) Priority promotion

(b) Shared-slack admission

Extra delay

Predicted miss ⇒ Defer w (c) Arrival-aware admission

Figure 6. Scheduling decisions (illustrative predictions). (a) Protect A using remaining slack. (b) Refresh shared slack after each launch. (c) Defer w for an earlier-deadline forecast. Panel (b) fixes t and H = {A}, with additive costs and other checks passing. Dashed outlines mark forecasts; hatching marks excess cost in (b) and extra delay in (c).

4.4

Algorithm 1 SLO-aware stage dispatch

SLO-Aware Admission

Protected work starts when resources permit; ordinary candidates must pass current-request and forecast checks to limit interference on urgent work. This adapts slack stealing (Lehoczky & Ramos-Thuel, 1992) to concurrent GPU stages, using parallel capacity when the affected deadlines remain predicted feasible. Current-request protection. Consider an ordinary candidate unit u at stage s, not yet included in foreseeable load nf . At a fixed decision time, let n+ = nf + es add that unit. Its predicted interference on protected request h and the resulting slack satisfy ∆h (u) = Rh (n+ ) − Rh (nf ), σh (n+ ) = σh (nf ) − ∆h (u).

(4)

Require: State S: ready queues, running work, arrival history 1: Record arrivals and completions in S 2: for s ∈ [Action, Prefill, ViT] do 3: (H, nf ) ← R EFRESH P RIORITY(S) 4: for u in S CAN R EADY(s, S, H) do 5: if request(u) ∈ / H then 6: n+ ← n f + e s 7: if not C URRENT F EASIBLE(H, n+ ) then 8: continue {protect current requests} 9: end if 10: if not F ORECAST F EASIBLE(u, n+ ) then 11: continue {protect forecast arrivals} 12: end if 13: end if 14: I SSUE(u); update queues and running state in S 15: (H, nf ) ← R EFRESH P RIORITY(S) 16: end for 17: end for

C URRENT F EASIBLE accepts exactly when ∆h (u) ≤ σh (nf ) for every h ∈ H. Local property. A passing candidate leaves the current protected set predicted feasible under this decision’s snapshot. Proof. The inequality gives σh (n+ ) ≥ 0, hence t + Rh (n+ ) ≤ ℓh for every h ∈ H. A failed check leaves the candidate queued while other ready work is considered. Coordinate decisions across stages. Arrivals, completions, and periodic ticks trigger Algorithm 1. The driver visits Action, Prefill, and ViT. S CAN R EADY starts from priority-ordered ready work and yields candidates while the stage has capacity, retaining deferred work for later scans. Each launch updates shared execution state and refreshes protection before the next check. Figure 6(b) shows why: under the illustrative additive predictor, admitting u consumes 6 ms of A’s 10 ms slack, leaving 4 ms. Candidate v would add another 6 ms and is deferred. Independent checks against the original snapshot would admit both. Appendix C.1 specifies the priority update and stage iteration rules.

Arrival-aware admission. In Figure 6(c), w passes the current-request check but would delay forecast request e past its earlier deadline. F ORECAST F EASIBLE uses recent arrivals grouped by response budget to forecast requests during the candidate’s execution, retaining those whose deadlines precede the candidate’s. Each retained forecast must satisfy δ + f + RV (n′e ) ≤ be , where f reserves front-end time, RV estimates the remaining pipeline from ViT, and n′e includes the candidate and preceding retained forecasts.

5

I MPLEMENTATION

Model adapters connect input preparation, kernels, CUDA graphs, and output processing to the runtime. VLA-2B and Wall-OSS-0.5 use MXFP8 ViT/Prefill linear operations (Rouhani et al., 2023) and BF16 Action; π0.5 uses native FlashRT FP8 with explicit state-text conditioning (Appendix B.1). The central scheduler dispatches each stage separately; the three-model backends combine ViT and Prefill as Context. Each Action dispatch runs a complete, nonpreemptive denoising trajectory with fixed request member-

XInfer Table 1. Input configuration. Token segments sum to the full prefix; action tokens and sequence padding are excluded. Input

VLA-2B Wall-OSS-0.5

Camera 1 (pixels) 224×352 Cameras 2/3 (pixels) 224×320 Patch size (pixels) 16×16 Visual tokens 217 Instruction tokens 41 State tokens 72 Template/special tokens 116 Full prefix 446

π0.5

196×308 196×308 196×280 196×280 14×14 14×14 217 217 41 40 74 127 113 11 445 395

ship; central Action dispatches contain one request. Client timestamps measure response latency; server timers and event traces diagnose execution. Runtime controls toggle GPU-stage overlap, cohort barriers, and CPU–GPU overlap (Section 6.4). Appendix C.1 records the central predictor’s calibration and constants.

6

E VALUATION

6.1

Experimental Setup

Models and serving comparisons. Each serving run uses NVIDIA consumer-grade Blackwell GPUs. The threemodel study covers VLA-2B, Wall-OSS-0.5, and π0.5 . VLA-2B uses a Qwen3.5-2B backbone and non-public weights. We use requests representative of our deployment setting and keep inputs fixed across serving methods within each model. Each request contains three camera views, a shared language instruction, and a 14-dimensional synthetic robot state. Table 1 summarizes the input shapes and actual prefix lengths. All models encode 868 image patches into 217 visual tokens; text lengths reflect their native tokenizers and templates. Appendix B.1 details input preprocessing. For each model, we compare XI NFER with integrations of SGLang (Zheng et al., 2024) and vLLMOmni (Yin et al., 2026), FlashRT (served through its Nexus frontend) (Su, 2026), and a Robion single-GPU reimplementation (Adamopoulos et al., 2026). The latter combines true stage batches, lockstep progression, and calibrated Context SM quotas; Context group g overlaps Action group g − 1, with a barrier between rounds. Comparisons share the checkpoint, workload, precision, and compute backend. Appendix B.3 lists the Robion reproduction settings; Appendix B.1 provides the measurement protocol. Central-scheduling comparison. All SLO policies share the three-stage VLA-2B backend, model, kernels, and allocated GPU resources. FIFO orders ready requests by arrival. EDF orders each stage queue by deadline with fixed perstage concurrency limits. Slack-Gate uses deadline queues and defers non-urgent work when another request has less

Table 2. ERC (robots) under the response-latency criterion. π0.5 Serving method Wall-OSS-0.5 VLA-2B XI NFER SGLang vLLM-Omni FlashRT Robion

18 10 10 5 11

15 5 8 4 10

24 14 15 9 19

than 40 ms estimated slack. Appendix C.4 details baseline configurations and parameter selection. Policy-specific bounds and admission decisions determine actual concurrency. XI NFER applies the same prediction and admission rules across workloads. Matched policy comparisons use the same GPU. Load and metrics. Each emulated robot follows a closedloop request stream with at most one request in flight and a minimum send interval of 180 ms (Equation 1). Response latency spans sending a packed request to receiving the complete response; capture, packing, and parsing are excluded. The three-model runs use ten denoising steps and 16 future action points. The RTC commitment length is fixed at d = 8 in both studies; response-time budgets in the central-scheduling replay follow each workload. Both serving studies measure requests over 180 s after 30 s of warmup. We report ERC as the largest validated integer robot population. The three-model comparison requires zero request errors and response P50 below 150 ms in all three windows; the central-scheduling study requires at least 95% SLO attainment. Three-model QPS counts successful requests sent and completed within the window; centralscheduling goodput counts on-time responses to requests sent within it. Both rates divide these counts by 180 s. Appendix B.1 details aggregation and measurement-window boundaries; Appendices B.2 and C.4 detail ERC validation. ERC criteria and physical task throughput. We apply the deployment-derived 150 ms threshold across models: 50 ms baseline response plus 100 ms injected delay, near the delay–throughput knee (Appendix A.1). For heterogeneous budgets, the 95% attainment target defines a common high-attainment operating point allowing at most 5% deadline misses; goodput captures the accompanying response-rate tradeoff. Figure 2 projects fleet parcel throughput from measured serving latency and physical calibration (Appendix A.1). 6.2

Serving-System Comparison

XI NFER achieves ERCs of 18, 15, and 24 robots across the three models, exceeding all compared systems (Table 2). Compared with Robion’s 11, 10, and 19 robots, these ERC values are 63.6%, 50%, and 26.3% higher.

XInfer FleetServe

Response P50 (ms, log)

SGLang

(a) VLA-2B

1500

vLLM-Omni

FlashRT

150 ms

(c) π0.5

(b) Wall-OSS-0.5

500 150 50 20

11

Throughput (req/s)

Robion

18

10

15

19

100

80

160

50

40

80

0

0 1

8

16

24

32

24

0 1

8

16

24

32

1

8

16

24

32

Number of concurrent robots

Figure 7. Serving performance across three models. Latency uses a log scale; dashed lines mark 150 ms. Large markers and guides identify confirmed ERCs for XI NFER and Robion.

For each model, all five systems are evaluated at the same robot counts. At low load, throughput follows the aggregate request rate; after saturation, responses delay subsequent sends and latency rises sharply (Figure 7). Appendix B.2 details ERC validation. These complete-system comparisons combine multiple mechanisms; Section 6.4 separates their contributions within a common runtime. 6.3

Serving Heterogeneous SLOs

Alternating task phases. In our block-stacking tasks, alignment and contact are more sensitive to delayed feedback than transport. Motivated by this observation and the recorded phase patterns (Appendix C.3), we alternate each robot’s response budget between 90 and 400 ms every 30 completed requests, with staggered robot phases and the closed-loop trigger in Equation 1. We report SLO attainment, on-time goodput, and mean wide-budget response latency. XI NFER achieves an ERC of 18 robots under the 95% SLO criterion, compared with 15 for Slack-Gate, 14 for FIFO, and 12 for EDF (Figure 8). EDF degrades gracefully under overload: it keeps 88.6–90.8% attainment from 16 to 20 robots, where FIFO falls to about 50%, and exceeds SlackGate at 20 robots (88.6 versus 75.9%). Appendix C.4 gives detailed results. At 15 robots, both XI NFER and SlackGate satisfy the 95% overall SLO target. XI NFER reduces mean wide-budget latency by 33% and increases goodput from 69.3 to 80.2 responses/s. The closed-loop relation in Equation 2 explains how shorter request cycles can increase goodput alongside improved attainment.

Bursty robot arrivals. To test transient overload, we start with 12 robots and add six robots with 90 ms budgets at 60 s and again at 120 s, each issuing 55 requests. Figure 8(d) shows the resulting attainment over time. Within the designated 60–80 s and 120–140 s windows, XI NFER attains 96.6%, versus 95.9% for EDF and 85.1% for Slack-Gate. Multiple response budgets. To broaden the budget distribution, 15 robots cycle through 80, 120, 160, 250, and 400 ms budgets every 30 responses, with staggered phases. XI NFER attains 96.1% overall versus 95.1% for EDF and 92.5% for Slack-Gate, with the per-budget breakdown in Figure 8(e). Appendix C.4 reports per-round results and aggregation details. Parcel handling. The parcel-handling workload derives response-indexed budget sequences from gripper events in 23 episodes recorded in production. Sequences advance with successful responses, yielding 61–63% tight requests and intervals in which all robots carry tight budgets (Appendix C.5). Under the 95% SLO criterion, XI NFER achieves an ERC of 14 robots, versus 12 for Slack-Gate and 10 for FIFO. EDF has no confirmed ERC within the tested range of 10–20 robots (Figure 8(f)). From 16 to 20 robots, observed SLO attainment gains for XI NFER over SlackGate span 3.8–11.4 percentage points, while wide-budget responses return 7–39 ms sooner. 6.4

Execution Engine Ablations

CPU–GPU overlap in Figure 9 permits host work during GPU execution, but retains serial GPU execution. Execution-aware concurrency enables the selected intra-

XInfer FleetServe

Slack-Gate

EDF

FIFO

95% target

Alternating task phases

(a) SLO attainment (%)

(b) Mean wide-budget latency (ms)

100 90 80 70 60 50

400 300

-33%

200 100 0 8

12

15

18

20

8

12

Robots

+6 robots

16

20

+6 robots

80 60

Multiple response budgets (e) SLO attainment (%)

120

Robot send time (s)

180

16

20

Parcel handling (f) SLO attainment (%)

100

100

80

80 60 Overall: FleetServe 96.1% Slack-Gate 92.5%

40

40 60

12

Robots

60

30

8

Robots

Bursty robot arrivals (d) SLO attainment (%) 100

(c) Goodput (req/s) 80 70 60 50 40 30

400 ms budget

80

120

160

250

400

40 10

Response budget (ms)

12

14

16

18

20

Robots

Figure 8. Four workloads with matched resources: (a–c) show three metrics for alternating phases; (d–f) show attainment for the other workloads. Curves show means with min–max whiskers. Large markers denote ERC in (a) and 33% lower mean latency for XI NFER versus Slack-Gate at 15 robots in (b). Shading in (d) marks added robots’ measured active periods.

stage multi-stream mode, raising throughput from 59.52 to 104.23 req/s (1.75×). Cross-stage coordination then combines Context–Action overlap with independent progression, reaching 131.53 req/s (1.26×) and reducing P50 from 229.45 to 142.35 ms (1.61× speedup). These increments depend on the enable order. Closed-loop triggering limits throughput headroom. The 180 ms minimum send interval caps 24 robots at 24/0.18 ≈ 133.33 req/s, only 27.9% above the penultimate configuration. The full engine reaches 131.53 req/s. Responses arriving before the next eligible trigger cannot advance the next request (Equation 1), so latency reductions need not yield proportional QPS gains. Appendix B.4 details the controls and protocol. 6.5

Scheduler Analysis

Policy controls and component ablations. Figure 10 compares four controls with full XI NFER. Removing interference modeling replaces concurrent-stage predictions with isolated durations, lowering mean attainment by 45.5 and 51.4 percentage points for alternating phases and parcel handling at 18 robots. Strict priority gating blocks ordinary work whenever a protected request exists, reducing attainment by 7.5–30.5 points across eight settings. Arrival-aware and stage-wise admission primarily protect tight-budget requests: disabling arrival-aware checks or checking only before ViT weakens this protection as load approaches saturation. The losses grow with the tight-budget share and

concentrate within bursts; strict gating instead starves ordinary work. Appendix C.3 reports workload-level results and the latency and goodput effects of arrival-aware admission. Running-stage prediction. Causal replay of recorded stage transitions at 8/18/20 robots uses only the concurrency history available at each sample. Mean absolute errors for the remaining execution time of running stages are 2.0/2.6/2.8 ms. Appendix C.2 details the measurement protocol and remaining-time error distributions. Scheduling overhead. In chronological CPU replay, a policy update followed by an admission check takes 73/77 µs at P99 with 15/18 robots, below 0.1% of the 90 ms response budget. This measures CPU policy time, not added response latency; GPU execution proceeds asynchronously. Appendix C.3 gives the replay protocol.

7

R ELATED W ORK

Robot and VLA serving. Prior robot-serving systems use execution context to guide scheduling: Kairos adapts execution horizons, Armory uses action-queue state, and ROSA coordinates placement, batching, and request rates (Dai et al., 2026; Anonymous Authors, n.d.; Jiang et al., 2026). Robion overlaps encoding and action diffusion with fixed stage batches and lockstep progression, and selects model queues by their head requests’ SLOs (Adamopoulos et al., 2026). XI NFER also exploits stage overlap, but ad-

XInfer

π0.5 | 24 robots All disabled

435.4

+ CPU-GPU overlap

401.9

+ Execution-aware Concurrency

229.5 142.3

+ Cross-stage Coordination 0

150

54.9

1.08×

1.08×

59.5

1.75×

1.75×

104.2

1.61×

300

1.26×

131.5 450

(a) P50 response latency (ms)

0

50

100

150

(b) Throughput (req/s)

Figure 9. Cumulative ablation on π0.5 at 24 robots. Bars show per-window means, rows retain prior mechanisms, and arrows show incremental gains. Execution-aware concurrency enables intra-stage concurrency. Cross-stage coordination combines Context–Action overlap and independent progression. Full engine: 2.40× throughput and 3.06× P50 speedup over All-Off.

w/o interference modeling Strict priority gating

w/o arrival-aware admission Entry-only admission

SLO attainment drop (pp)

60

40

20

0

Alternating phases

Bursty arrivals

Multiple budgets

controlled kernel preemption (Strati et al., 2024; Han et al., 2022). Our admission tests adapt slack stealing and nonpreemptive blocking analysis from real-time scheduling (Lehoczky & Ramos-Thuel, 1992; Audsley et al., 1993; Jeffay et al., 1991). They use request progress, calibrated concurrent interference, and arrival history to protect current and anticipated deadlines at VLA stage boundaries. Issued stages remain non-preemptive, and feasibility is predicted rather than established by worst-case timing bounds.

Parcel handling

8 Figure 10. Scheduler ablations. Mean SLO attainment drop relative to full XI NFER for each scheduler variant. Appendix C.3 gives workload settings.

vances requests independently and separates request priority from interference-aware stage admission. Stage-aware inference. Iteration-level scheduling and chunked prefill improve LLM serving (Yu et al., 2022; Agrawal et al., 2024), while DistServe, Splitwise, and semiPD separate prefill and decoding resources (Zhong et al., 2024; Patel et al., 2024; Hong et al., 2025a). In VLA serving, encoding and denoising use different model components, and closed-loop control couples response time to subsequent arrivals. Optimized inference runtimes and modellevel work reduction (Su, 2026; Su et al., 2026; Xu et al., 2026; Tan et al., 2025) complement our service; RTC maintains action-chunk continuity during ongoing motion (Black et al., 2025b). XI NFER coordinates shared execution using per-request response-time budgets. SLO-aware GPU sharing. Clockwork uses predictable execution to meet request latency targets (Gujarati et al., 2020). SLOs-Serve allocates work to meet application- and stage-specific SLOs (Chen et al., 2025; Hong et al., 2025b), while Orion limits interfering kernels and REEF supports

C ONCLUSION

XI NFER is a production-deployed VLA serving system that coordinates shared GPU computation with the changing feedback needs of robotic execution. Its asynchronous engine exposes parallelism across requests and stages, while execution-aware scheduling separates request priority from stage admission to control interference on current and anticipated urgent requests. Across three VLA models, XI NFER achieves 1.26–3.75× the effective robot capacity of the compared systems under a common response-latency criterion. Under heterogeneous SLOs, it also improves attainment and on-time goodput. These results demonstrate how coordinating parallelism and interference around executiondependent feedback requirements enables more efficient serving of robot fleets.

C ONTRIBUTORS Fleetserve is a collaborative effort of the X Square Robot team. The full contributor list is given below; ∗ denotes core contributors, † denotes the project lead, and ‡ denotes the corresponding author. Yue Zhang∗ , Zicong Ye∗ , Lufang Chen∗† , Dance Yang, Mingyang Li, Zhenwei Duan, Michael Zhao, Ziye Deng, Starrick Liu, Ruizhi Chengze, YongHua Chen, SongAo Han, Yu Sun, Ran Yu, Xiaofan Li, Ruyi Gan, Hao Wang‡

XInfer

R EFERENCES Adamopoulos, D., Chanpaisit, N., Fakhri, B., and Giannoula, C. Efficient vision-language-action management and serving for robot factories. arXiv preprint arXiv:2609.12075, 2026. URL https://arxiv. org/abs/2609.12075. Agrawal, A., Kedia, N., Panwar, A., Mohan, J., Kwatra, N., Gulavani, B. S., Tumanov, A., and Ramjee, R. Taming throughput-latency tradeoff in LLM inference with Sarathi-Serve. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pp. 117–134. USENIX Association, 2024. URL https://www.usenix.org/conference/ osdi24/presentation/agrawal. Anonymous Authors. Armory: Action chunk scheduling for batched robot policy serving, n.d. URL https:// actionchunkscheduling.github.io/. Project page, under review. Accessed October 30, 2026. Audsley, N., Burns, A., Richardson, M., Tindell, K., and Wellings, A. J. Applying new scheduling theory to static priority pre-emptive scheduling. Software Engineering Journal, 8(5):284–292, 1993. doi: 10.1049/sej.1993. 0034. Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Galliker, M. Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A. Z., Shi, L. X., Smith, L., Springenberg, J. T., Stachowicz, K., Tanner, J., Vuong, Q., Walke, H., Walling, A., Wang, H., Yu, L., and Zhilinsky, U. π0.5 : a vision-language-action model with open-world generalization. In Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp. 17–40. PMLR, 2025a. URL https://proceedings.mlr. press/v305/black25a.html. Black, K., Galliker, M., and Levine, S. Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems, volume 38, pp. 33383– 33407. Curran Associates, Inc., 2025b. doi: 10.52202/ 085713-1122. Chen, S., Jia, Z., Khan, S., Krishnamurthy, A., and Gibbons, P. B. SLOs-Serve: Optimized serving of multi-SLO LLMs. arXiv preprint arXiv:2504.08784, 2025. URL https://arxiv.org/abs/2504.08784. Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., and Song, S. Diffusion Policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics:

Science and Systems (RSS), 2023. doi: 10.15607/RSS. 2023.XIX.026. Dai, Y., Ananthanarayanan, G., Cox, L., Foukas, X., Radunovic, B., and Netravali, R. Kairos: A scalable serving system for physical AI. arXiv preprint arXiv:2605.11381, 2026. URL https://arxiv. org/abs/2605.11381. Gujarati, A., Karimi, R., Alzayat, S., Hao, W., Kaufmann, A., Vigfusson, Y., and Mace, J. Serving DNNs like Clockwork: Performance predictability from the bottom up. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pp. 443–462. USENIX Association, 2020. URL https://www.usenix.org/conference/ osdi20/presentation/gujarati. Han, M., Zhang, H., Chen, R., and Chen, H. Microsecond-scale preemption for concurrent GPUaccelerated DNN inferences. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pp. 539–558. USENIX Association, 2022. URL https://www.usenix.org/conference/ osdi22/presentation/han. Hong, K., Chen, L., Wang, Z., Li, X., Mao, Q., Ma, J., Xiong, C., Wu, G., Han, B., Dai, G., Liang, Y., and Wang, Y. semi-PD: Towards efficient LLM serving via phase-wise disaggregated computation and unified storage. arXiv preprint arXiv:2504.19867, 2025a. URL https://arxiv.org/abs/2504.19867. Hong, K., Li, X., Chen, L., Mao, Q., Ning, X., Dai, G., Yan, S., Liang, Y., and Wang, Y. SOLA: Optimizing SLO attainment for large language model serving with stateaware scheduling. In Proceedings of Machine Learning and Systems, volume 7. MLSys, 2025b. URL https: //mlsys.org/virtual/2025/poster/3231. Jeffay, K., Stanat, D. F., and Martel, C. U. On nonpreemptive scheduling of periodic and sporadic tasks. In Proceedings of the Twelfth IEEE Real-Time Systems Symposium (RTSS), pp. 129–139, 1991. doi: 10.1109/ REAL.1991.160366. Jiang, W., Clemons, J., O’Flaherty, R., Hadfield, H., Degirmenci, A., Song, S., Narang, Y., and Kozyrakis, C. ROSA: A robotics foundation model serving system for robot factories. arXiv preprint arXiv:2607.01088, 2026. URL https://arxiv.org/abs/2607.01088. Lehoczky, J. P. and Ramos-Thuel, S. An optimal algorithm for scheduling soft-aperiodic tasks in fixed-priority preemptive systems. In Proceedings of the IEEE Real-Time Systems Symposium (RTSS), pp. 110–123, 1992. doi: 10.1109/REAL.1992.242671.

XInfer

Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023. URL https://openreview.net/ forum?id=PqvMRDCJT9t. Liu, C. L. and Layland, J. W. Scheduling algorithms for multiprogramming in a hard-real-time environment. Journal of the ACM, 20(1):46–61, 1973. doi: 10.1145/321738. 321743. NVIDIA. Multi-process service, 2026. URL https://docs.nvidia.com/deploy/mps/ latest/index.html. Documentation. Last updated September 9, 2026. Accessed October 30, 2026. Patel, P., Choukse, E., Zhang, C., Shah, A., Goiri, Í., Maleki, S., and Bianchini, R. Splitwise: Efficient generative LLM inference using phase splitting. In ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA), pp. 118–132, 2024. doi: 10.1109/ISCA59077.2024.00019. Rouhani, B. D., Zhao, R., More, A., Hall, M., Khodamoradi, A., Deng, S., Choudhary, D., Cornea, M., Dellinger, E., Denolf, K., Stosic, D., Elango, V., Golub, M., Heinecke, A., James-Roxby, P., Jani, D., Kolhe, G., Langhammer, M., Li, A., Melnick, L., Mesmakhosroshahi, M., Rodriguez, A., Schulte, M., Shafipour, R., Shao, L., Siu, M., Dubey, P., Micikevicius, P., Naumov, M., Verrilli, C., Wittig, R., Burger, D., and Chung, E. Microscaling data formats for deep learning. arXiv preprint arXiv:2310.10537, 2023. URL https://arxiv.org/abs/2310.10537. Strati, F., Ma, X., and Klimovic, A. Orion: Interferenceaware, fine-grained GPU sharing for ML applications. In Proceedings of the Nineteenth European Conference on Computer Systems (EuroSys), pp. 1075–1092, 2024. doi: 10.1145/3627703.3629578. Su, L. FlashRT, 2026. URL https://github.com/ flashrt-project/FlashRT. Software repository. Accessed October 30, 2026. Su, L., Chen, D., Li, J., Guo, K., and Lv, T. FlashRTPAI: Fast kernels are not fast pipelines for physical AI inference. arXiv preprint arXiv:2610.24349, 2026. URL https://arxiv.org/abs/2610.24349. Tan, X., Yang, Y., Ye, P., Zheng, J., Bai, B., Wang, X., Hao, J., and Chen, T. Think twice, act once: Tokenaware compression and action reuse for efficient inference in vision-language-action models. arXiv preprint arXiv:2505.21200, 2025. URL https://arxiv. org/abs/2505.21200.

Xu, L., Li, B., Wu, H., Han, C., Li, X., Hua, M., Jiang, S., Cao, T., Li, C., Zhong, S., and Wang, S. Embodied.cpp: A portable inference runtime of embodied AI models on heterogeneous robots. arXiv preprint arXiv:2607.02501, 2026. URL https://arxiv. org/abs/2607.02501. Yin, P., Zhu, J., Gao, H., Zheng, C., Huang, Y., Zhou, T., Yang, R., Liu, W., Chen, W., Guo, C., Deng, D., Mo, Z., Wang, C., Cheng, J., Wang, R., and Liu, H. vLLM-Omni: Fully disaggregated serving for any-to-any multimodal models. arXiv preprint arXiv:2602.02204, 2026. URL https://arxiv.org/abs/2602.02204. Yu, G.-I., Jeong, J. S., Kim, G.-W., Kim, S., and Chun, B.G. Orca: A distributed serving system for TransformerBased generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pp. 521–538. USENIX Association, 2022. URL https://www.usenix.org/conference/ osdi22/presentation/yu. Yu, R., Zhang, P., Liu, S., Liu, B., Kang, M., Li, S., Shi, L., Ma, E., Yang, P., Pan, C., Chen, J., Liu, D., Sun, R., Guo, M., Zhang, B., Zhou, H., Xu, Z., Chen, V., Huang, H., Wang, J., Kuzi, D., Zhai, A., Su, H., Gan, R., Liang, L., Wang, H., and Wang, Q. Wall-OSS-0.5 technical report. arXiv preprint arXiv:2605.30877, 2026. URL https://arxiv.org/abs/2605.30877. Zhai, A., Liu, B., Fang, B., Cai, C., Ma, E., Yin, E., Wang, H., Zhou, H., Wang, J., Shi, L., Liang, L., Wang, M., Wang, Q., Gan, R., Yu, R., Li, S., Liu, S., Chen, S., Chen, V., and Xu, Z. Igniting vlms toward the embodied space. arXiv preprint arXiv:2509.11766, 2025. Zhao, T. Z., Kumar, V., Levine, S., and Finn, C. Learning fine-grained bimanual manipulation with low-cost hardware. In Proceedings of Robotics: Science and Systems (RSS), 2023. doi: 10.15607/RSS.2023.XIX.016. Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., and Sheng, Y. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, volume 37, pp. 62557–62583. Curran Associates, Inc., 2024. doi: 10.52202/079017-2000. Zhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X., Jin, X., and Zhang, H. DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pp. 193–210. USENIX Association, 2024. URL https://www.usenix.org/conference/ osdi24/presentation/zhong-yinmin.

XInfer

AI U SAGE D ISCLOSURE The authors led the system and experimental designs. We used OpenAI ChatGPT/Codex and Anthropic Claude to generate and iteratively revise implementation code and experimental, analysis, and plotting scripts based on the authors’ specifications. The tools also assisted with literature review and drafting and revising multiple manuscript sections. The authors determine the final designs and content and take full responsibility for the accuracy and integrity of the paper.

A

R ESPONSE L ATENCY AND P HYSICAL TASK E FFICIENCY

A.1

Sustained Parcel Handling

Protocol and latency criterion. We use one robot and VLA-2B BF16 (execute horizon=7), holding inference settings fixed. Each of six added-delay settings has three five-minute trials. Delay is inserted between inference completion and response delivery; baseline response latency is approximately 50 ms. Table 3 distinguishes all handled parcels from those reaching the required task endpoint.

P50 reaches 505.4 ms at 16 robots. Beyond 250 ms, dashed curves extrapolate per-parcel handling time: qext (∆) =

3600 , t200 + k(∆ − 200)

0 25 50 100 150 200

1968 1944 2020 1892 1552 1376

1920 1912 1968 1812 1516 1352

86.5 43.3 73.3 66.1 96.2 129.4

Completed output falls 29.6% at 200 ms added delay. Handling throughput drops 6.3% from 50 to 100 ms, then 18.0% from 100 to 150 ms. Selecting 100 ms before this steeper loss, plus the 50 ms baseline, motivates the 150 ms responseP50 criterion (Section 6.1). Predicted parcel-handling throughput. Figure 2 maps serving latency to fleet parcel throughput for 1–16 robots. es (N ) Let q(∆) interpolate the handled rates in Table 3, and R interpolate mean per-run response P50s across robot counts for system s (Figure 7). Both interpolations are piecewise linear. Up to 250 ms response latency,   b s (N ) = N q max{0, R es (N ) − 50 ms} . Q (5) Subtracting 50 ms removes baseline latency; faster responses use the zero-injection rate. FlashRT’s measured

(6)

Here ∆ is added delay in milliseconds, qext is in parcels/h, and t200 = 3600/1376 = 2.6163 seconds per parcel. The slope k = 0.0071353 seconds per millisecond is fitted to 3600/q(∆) at 100, 150, and 200 ms using only physical calibration data. Adjacent-interval slope fits give FlashRT 12,137–13,940 parcels/h at 16 robots, so its comparison with seven FleetServe robots depends on the extrapolation slope. Uncertainty and scope. Using Table 3’s sample SDs sj , we linearly interpolatepvariance v(∆) within calibration and shade N [q(∆) ± v(∆)], using s20 below baseline. Beyond 250 ms, independent sample-SD perturbations of the calibration means q̄100 and q̄200 propagate through both the anchor and fitted slope, giving first-order half-width v u u B(N, ∆) = N t

X j∈{100,200}

Table 3. Hourly parcel rates under added response delay. Handled and completed are measured aggregate rates; handled SD is estimated from three trial markers digitized from the source plot. Figure 2 uses handled rates. Added delay Handled Completed Handled (ms) parcels/h parcels/h SD

∆ > 200.



∂qext ∂ q̄j

2

s2j .

(7)

Serving P50s remain fixed. These bands quantify sensitivity to single-robot variation, not confidence intervals or fleetlevel SDs; they exclude digitization, latency, and modelform uncertainty. The projection transfers BF16, horizon-7 calibration to MXFP8 Context/BF16 Action with RTC d = 8, assuming identical, independent robots with fixed task mix and control behavior. It treats P50 as constant delay and omits jitter and shared physical bottlenecks; it is distinct from measured multi-robot task output and ERC. A.2

Manipulation Attempts by Parcel Type

Protocol. We evaluate parcel reorientation (barcode initially down) and conveyor pushing (barcode initially up) with VLA-2B BF16 and execute horizon=7; baseline model latency is approximately 45 ms. Each task tests the four parcel types in Table 4, with five trials per type at each added delay from 0 to 500 ms in 50 ms increments. A pushing attempt is one complete push; injected delay differs from full response latency. Effect on manipulation effort. Attempt counts vary by parcel type and need not increase monotonically with delay (Figure 11). Mean flip attempts rise from 1.05 without added delay to 1.75 at 350 ms; mean push attempts rise from 1.00 to 1.50–1.70 at 350–500 ms. Table 4 gives the complete counts.

XInfer Mean of four types

Range across types

Carton A

Gray bag

(a) Parcel reorientation Attempts per task

2.5

Carton B

Long carton C

(b) Conveyor pushing

Mean: 1.05 at 0 ms, 1.75 at 350 ms (+67%)

Mean: 1.00 at 0 ms, 1.70 at 400 ms (+70%)

2.0

1.5

1.0 0

100

200

300

400

500

0

Added response delay (ms)

100

200

300

400

500

Added response delay (ms)

Figure 11. Response delay increases physical effort, with different effects across parcel types. Each point averages five tasks of one type; the black line is the mean of the four types and the band their range. Zero denotes no added delay. Full counts are in Table 4.

Table 4. Manipulation attempts (flip/push) by parcel type under added response delay. Type columns give totals over five tasks; Mean gives averages over twenty tasks. Added delay Carton Gray Carton Long carton (ms) A bag B C Mean 0 50 100 150 200 250 300 350 400 450 500

6/5 5/5 6/5 5/5 5/5 6/8 6/7 5/8 6/6 6/7 8/5 6/8 7/7 10/6 11/8 8/8 7/8 9/11 12/9 7/11 9/8 8/7

5/5 6/5 6/5 7/6 8/6 6/5 9/6 9/7 5/5 5/5 6/7

5/5 1.05/1.00 6/5 1.15/1.00 7/5 1.20/1.15 6/5 1.20/1.30 6/5 1.30/1.20 5/6 1.25/1.20 7/5 1.65/1.20 7/7 1.75/1.50 5/10 1.30/1.70 5/8 1.45/1.65 6/10 1.45/1.60

B

S ERVING E VALUATION : D ETAILED R ESULTS

B.1

Protocol and Configuration

starvation means available actions run out before replacement. A trigger miss occurs when sj + TE passes while request j remains in flight (Equation 1). Input preprocessing. VLA-2B and Wall-OSS-0.5 merge image patches in their trained encoders. π0.5 instead averages non-overlapping 2×2 patch blocks after the full ViT, producing 77, 70, and 70 tokens across the three cameras. Equal visual-token counts do not imply equal total FLOPs. Its native FlashRT backend uses per-tensor FP8 E4M3 with BF16 inputs and outputs; Section 5 specifies the other precision settings. The shared instruction retains its original metadata. WallX converts, normalizes, and discretizes the all-zero 14dimensional state into 20 values, encoded as 72/74 tokens for VLA-2B/Wall-OSS-0.5. π0.5 uses identity normalization and its native state-as-text formatter, padding to 32 dimensions and discretizing into 256 bins. This synthetic state is not a calibrated physical-robot state policy. State labels, punctuation, and special markers belong to the template column of Table 1.

Measurement and aggregation. Section 6.1 defines the workload and timing boundaries. Warmup uses the target robot population. The 180 ms request period corresponds to six 30 ms model steps in production. Latency statistics include responses completed after the measurement window for requests sent within it. We average per-window P50/P95/P99 and QPS; whiskers show the observed min– max range. Each repeat restarts the client, while the model service may remain resident.

Serving interfaces. SGLang and vLLM-Omni retain their queues. FlashRT uses its native single-consumer model session over HTTP with JSON/base64; the other paths use WebSocket with MessagePack. Transport costs are included in response latency. π0.5 enables Context-completion notification.

RTC handoff. After freezing the first d keyframes, RTC (Black et al., 2025b) guides subsequent keyframes toward the remaining old trajectory with decaying weight and generates the rest freely. Excessive commitment is over-freezing; a response after tk +d∆ causes a handoff violation, whereas

We apply the criterion in Section 6.1 to unrounded windowlevel values: every window at the reported ERC must pass, and the adjacent larger population must fail in at least one window. For π0.5 , XI NFER passes at 24 robots with P50s of 133.470–136.592 ms and mean QPS 132.31; all three

B.2

ERC Validation

XInfer

windows at 25 robots fail (163.104–166.842 ms). VLA-2B Robion at 12 robots and π0.5 Robion at 20 robots each pass in only two of three windows, despite passing on mean P50. B.3

Robion Reproduction Settings

Table 5 specifies our single-GPU reproduction of Robion (Adamopoulos et al., 2026). B.4

Execution Mechanisms

Cumulative protocol. Figure 9 follows the sequence in Section 6.4 at 24 robots with fixed inputs, backend, and resources. All-Off and all-enabled each use two windows, while intermediate configurations use one, totaling six displayed windows under Appendix B.1. Values average perwindow P50 and QPS. Execution-aware concurrency enables intra-stage concurrency with the round barrier retained. Cross-stage coordination combines Context–Action overlap with independent progression, removing that barrier. Gains depend on the enabling order. This study does not re-estimate ERC. Incremental controls. Table 6 evaluates six configurations at 8, 16, and 32 robots, each with a separate service start and one window. All-Off serializes GPU stages, allows one request per stage, and retains a round barrier. M denotes execution-aware concurrency with the selected intrastage streams, concurrency, and batching limits. C overlaps CPU preparation and output processing with GPU work. R removes only the round barrier. G overlaps combined ViT– Prefill (Context) with Action on the GPU. R and G are the two components of cross-stage coordination. M+C, M+R, and M+G each add one mechanism to M. All-enabled combines them. Lockstep states share a round-admission rule, and completed results can return without waiting for the barrier. Inputs, weights, precision, kernels, workspaces, KV storage, and captured graphs remain fixed with completion notification enabled. Throughput and tail latency. At 32 robots, M+R improves QPS and P50 over M while increasing P95/P99. All-enabled remains above the capacity latency threshold (Table 6). GPU execution timelines. Nine separate profiler runs compare three states per model at N = 32: independent progression with CPU overlap disabled/enabled, and lockstep with CPU overlap enabled. Environments and configurations match within each model; profiler configurations differ from the ablations above. CPU controls toggle preparation/output overlap with GPU work; progression controls only the round barrier, retaining cross-stage overlap. Two host-clock markers align the traces; Table 7 summarizes 61–69 s and Figure 12 shows an excerpt. With CPU overlap

(a) CPU-GPU overlap off CPU ViT+Prefill Action

Idle

(b) CPU-GPU overlap on CPU ViT+Prefill Action 0

100

200

300

400

500

Time from 61 s (ms)

Figure 12. CPU–GPU overlap reduces idle gaps in a separate profiler diagnostic. VLA-2B serves 32 robots with overlap (a) disabled and (b) enabled. The figure shows the 61–61.5 s excerpt; the outlined gap marks an observed GPU idle interval. Table 7 reports statistics for the full profiling window.

disabled, CPU/GPU intersection is zero at display precision (below 0.001% for π0.5 ). These shares measure activity, not SM occupancy or MFU; profiler throughput is excluded from ERC.

XInfer Table 5. Robion configuration used in the serving-system comparison. Setting

Configuration

Scope Execution units Round progression Result publication

Single-model serving; distributed placement and routing are excluded. Context and Action each use a sequential CUDA stream with a dedicated CPU submission thread. Context work in round k overlaps Action work from round k − 1; the next round starts after both finish. Publish completed Action results after the round barrier (a reproduction choice); CPU output processing may overlap the next GPU round. An offline lookup selects the Context SM quota that minimizes full-round CUDA completion time; Action remains unrestricted. Capture supported execution-unit and Context SM configurations before measurement. Keep input preparation pipelined with GPU execution.

SM allocation CUDA Graphs CPU preparation

Table 6. Incremental ablations on π0.5 using a multi-stream baseline. M denotes execution-aware concurrency, C CPU–GPU overlap, R independent progression, and G GPU-stage overlap. C, R, and G are enabled individually on top of M. All-Off disables all four mechanisms. QPS is measured in requests/s. P50, P95, and P99 are client response latencies in ms. N Configuration

QPS

P50

P95

P99

8 All-Off M M+C M+R M+G M+C+R+G

44.26 44.25 44.26 44.25 44.27 44.27

20.00 19.51 19.87 20.40 19.67 19.85

26.92 24.50 24.82 31.38 20.54 21.19

32.46 30.54 27.34 33.66 21.08 21.71

16 All-Off M M+C M+R M+G M+C+R+G

54.81 305.80 350.92 364.93 87.98 145.05 184.75 208.85 88.44 85.00 97.11 103.96 87.46 152.56 193.70 224.76 88.44 109.63 151.40 172.86 88.51 39.46 61.53 73.70

32 All-Off M M+C M+R M+G M+C+R+G

53.77 581.14 672.59 691.47 90.57 351.93 428.52 467.07 100.79 327.11 352.05 362.57 99.33 311.91 461.74 560.28 96.67 327.68 423.47 475.40 132.24 235.18 310.03 378.06

Table 7. Wall-time shares (%) at N = 32 over 61–69 s. Ind./Lock.: independent/lockstep; off/on: CPU overlap. Either/Both: union/intersection of Context/Action kernel intervals. CPU/GPU: kernel overlap with instrumented CPU preparation/output scopes. Model

State

VLA-2B

Ind., off 85.49 58.36 Ind., on 99.72 79.73 Lock., on 90.79 25.45

Either Both CPU/GPU 0.00 56.85 11.28

Wall-OSS-0.5 Ind., off 82.05 55.35 Ind., on 98.78 87.78 Lock., on 87.55 68.99

0.00 26.88 14.49

π0.5

0.00 19.60 4.04

Ind., off 72.79 19.08 Ind., on 94.73 28.80 Lock., on 72.44 18.26

C

SLO-AWARE S CHEDULING : M ODEL AND D ETAILED R ESULTS

C.1

Prediction and Dispatch Model

The central backend schedules non-preemptive units at three boundaries: ViT, post-vision Prefill, and a complete Action denoising trajectory. Its evaluation uses batch size one. The response-latency SLO bj is converted to a server deadline ℓj = aj + bj − δ, where aj is server arrival time and δ = 4 ms reserves estimated time outside the server. The request does not carry its robot-side send timestamp. Progress-aware remaining-time model. For stages s ∈ {V, P, A} in pipeline order, define qs (n) = hs +

Ws , ρs (n̄)

n̄s = min(ns , cs ).

(8)

Ws is calibrated stage work in isolation-time units, ρs is a dimensionless rate table, and hs is stage-issue overhead. Every rate lookup is clipped to the calibrated bounds c. For a running stage s, the supplied counts include request j’s own unit; its rate therefore uses n−j = [n − es ]+ , where es is the unit vector for stage s and the positive part is componentwise. Waiting and downstream stages use qs (n) without subtraction. In foreseeable load nf , protected ready units already included by the scheduler remain in this lookup, even for their own requests. The supplied load approximates future concurrency. Let nr (u) be the scheduler-observed running counts at time u, including request j’s own unit, and define n−j r (u) = [nr (u)−es ]+ . For a stage issued at τj , its modeled compute start is bj,s = τj + hs . Completed work and residual time are Z max(t,bj,s ) w bj,s (t) = ρs (n−j r (u)) du, bj,s (9) [Ws − w bj,s (t)]+ rej,s (t, n) = [bj,s − t]+ + . ρs (n−j ) Here [x]+ = max(0, x). The integral is a sum over intervals between observed stage transitions. Historical rates deter-

XInfer

mine completed work; the supplied load determines the rate used for the remaining work. The remaining pipeline time is X qk (n), Rj (n) = hout + rj,s (n) + k>s

( rj,s (n) =

qs (n), waiting, rej,s (t, n), running.

(10)

The term hout covers output processing. A completion event advances the request to its next stage. Future stages use the same projected load, without an additional prediction of future queueing delay. The counters describe scheduler in-flight units, including observation delay. Calibration. The rate table comes from a microbenchmark with RTC d = 8 and B1 Action trajectories. It measures isolated and mixed-stage execution at 167 concurrency combinations, repeated three times per calibration instance. Rates are derived from isolated work divided by measured unit duration. The frozen table incorporates stage corrections and a common work factor of 1.02 fitted during simulator calibration. This yields (WV , WP , WA ) = (4.328931, 8.907749, 12.116308) ms. The model uses (hV , hP , hA ) = (2.1, 0.25, 1.1) ms and hout = 1.2 ms; hA includes preparation and KV copying. The front-end reserve f = 8.5 ms is added only for forecast arrivals below. These parameters were frozen before the reported final-policy GPU comparisons and are not fitted to their outcomes. Current-request protection. Algorithm 2 implements the promotion and protection rules of Section 4.3, refreshing the effective protected set H and foreseeable load nf as the promoted set P grows. Ordinary dispatch must preserve every protected request’s predicted feasibility; these predictions are not worst-case timing bounds or SLO guarantees. Arrival-aware admission. Recent arrivals are grouped by observed SLO value. For a value seen k > 0 times in the rolling window P = 180 ms, forecasts start at max(t, alast + P/k) and repeat every P/k, without robot identities or predefined tight/wide classes. Before issuing ordinary unit u, F ORECAST F EASIBLE (Section 4.4) checks forecasts during u’s estimated execution, retaining only those with ℓe < ℓu . Its projected load n′e adds u to foreseeable load and accumulates earlier retained ViT forecasts. The front-end reserve is f = 8.5 ms; physical resources still bound concurrency. Termination of priority refresh. With time, live requests J , and execution state fixed, each nonempty update only adds members to P, so the loop terminates within |J | − |P0 | + 1 evaluations including the final empty update, where

Algorithm 2 Refresh protected requests and foreseeable load Require: State S: live requests J , running units I, promoted set P, and current time t 1: P ← P ∩ J 2: L ← {j ∈ J : ℓj − t < Rj (0)} 3: repeat 4: H ←P \L 5: nf ← F ORESEEABLE L OAD(S, H) 6: Compute Bj for each j ∈ J from other requests’ lowpriority units 7: U ← {j ∈ J \ P : σj (nf ) < Bj } 8: P ← P ∪ U 9: until U = ∅ 10: Save P in S; return H, nf

P0 is the initial promoted set after intersection with J , without requiring monotonic calibrated rates. In Algorithm 1, S CAN R EADY initially orders each stage’s ready work by protected status, deadline, then request ID. After each launch updates occupancy and refreshes priority, Action retains its remaining order, while Prefill and ViT rebuild it. Failed checks advance the scan while leaving candidates queued; both checks apply only to ordinary work. C.2

Running-Stage Prediction Accuracy

Protocol and causal replay. We evaluate the remaining execution time of running stages using Equation 9, with one recorded run each at 8, 18, and 20 robots under the alternating 90/400 ms workload. We use the frozen calibration and lookup bounds of Appendix C.1. At the first policy observation at least 20 ms after the preceding sample, we capture request state and final foreseeable load, without adding predictor evaluations to the scheduling path. We retain valid running-stage forecasts for requests sent in [30, 210) s and track each sampled stage through its observed completion. Replay integrates scheduler-observed dispatch and retirement transitions strictly before forecast time t and uses the captured nf . Tests verify that changing future events cannot change a prediction. The artifact retains the pre-run protocol, instrumentation, analysis scripts, and raw traces. Stage endpoints and aggregation. ViT and Prefill end when completion is observed by the scheduler. Action ends after its RPC and CPU output processing. The stageendpoint estimate rbj,s (t) equals rej,s (t, nf ) from Equation 9, with the fixed hout added for Action. The signed prediction error is ej,s,t = tactual bj,s (t). (11) j,s,end − t − r Positive error denotes underprediction. Primary mean absolute errors (MAEs) weight each forecast equally. Requestbalanced summaries give each represented request equal total weight within a robot population or stage, divided

XInfer

Results. Across all running-stage samples, current-stage MAE is 2.04/2.62/2.79 ms at 8/18/20 robots. With equal request weights within each stage, MAE is 1.91–2.49 ms for ViT and Prefill and 2.15–3.05 ms for Action. Figure 13 shows both error directions.

CPU decision cost. On a Xeon Gold 6530, we replay queued decision states from 40–45 s of the simulation four times in chronological order, with no serving process running. For 8/15/18 robots, FleetServe’s policy update and admission check (including arrival observation) take 41/63, 52/73, and 55/77 µs (median/P99), respectively; these are CPU policy times, not added response latency.

C.3

C.4

equally among its forecasts.

GPU Ablations and Scheduling Cost

Recorded task phases. Eight recorded π0.5 blockstacking episodes motivate the alternating phase workload. Marking 1.0 s before to 0.5 s after each gripper switch as critical covers 15–72% of each trace. The evaluation abstracts these phase patterns into controlled 90/400 ms budgets and 30-response phases. Workloads and measurement. The GPU controls use the workloads in Section 6.3, with batch-size-one execution, a 180 ms minimum send interval, and RTC d = 8. Appendix C.5 specifies the parcel-handling trace. Response latency spans the client send to receipt of the complete response. We retain requests sent during [30, 210) s through completion and divide their on-time response count by 180 s for goodput. Unless stated otherwise, bursty results cover this full window; burst-only results use 60–80 s and 120– 140 s. All measured requests complete successfully. Component controls. Five controls change one mechanism or a specified policy combination while matching the full policy’s execution resources. Removing interference modeling uses isolated-stage durations without changing actual execution. Strict priority gating replaces both slack stealing and arrival-aware admission: ordinary work waits whenever a protected request exists. Removing arrivalaware admission retains current-request protection. Entryonly admission checks before ViT; Prefill and Action advance in priority/deadline order when resources permit. Zero-slack promotion sets the blocking term to zero. Each of three campaigns rotates policy order across three matched rounds, with a 75 s premeasurement smoke run. In Table 8, full-policy means pool nine rounds, while each difference uses its own campaign’s three paired rounds. Zeroslack promotion has mixed effects. Arrival-aware admission. A separate experiment isolates arrival-aware admission in six matched rounds with rotated policy order and identical model instances, stage computations, and execution resources. These rounds are not pooled with the component campaigns. Table 9 retains both attainment and the latency and goodput effects: protecting tight-budget requests can increase wider-budget latency and need not increase goodput.

Scheduling Policy Comparison

Baseline configuration. All policies share allocated execution resources and per-request stage computation. Simulation selects EDF’s smallest per-stage concurrency limits preserving the best SLO attainment under alternating task phases at 15 robots; these limits are fixed for GPU evaluation. Slack-Gate’s 40 ms threshold is selected from 20, 40, and 80 ms jointly with ViT concurrency, using the same workload and bursty arrivals. Among candidates within 1 percentage point of the best attainment, selection favors lower mean latency. Comparison and aggregation. Each policy uses three rounds with a fixed configuration. FIFO and Slack-Gate rotate within matched rounds; EDF and FleetServe are measured separately. Comparisons use equally weighted round means, with reused measurements counted once. Referenceload checks under unchanged configurations show meanattainment variation of −1.02 to +1.00 percentage points. Alternating task phases use 1,200 requests per robot; measurement boundaries follow Appendix C.3. Table 10 reports per-round attainment and differences in three-round mean goodput. ERC. ERC is the largest tested population with at least 95% attainment in each of three rounds: FleetServe passes at 18 robots but not at 20; 19 was not measured. This criterion does not imply that every smaller population passes. Temporal and budget aggregation. Figure 8(d) averages attainment in 2 s request-send-time bins across rounds. Section 6.3 reports attainment over the 60–80 s and 120–140 s burst windows; Table 10 uses the full [30, 210) s send window. For multiple response budgets (Figure 8(e)), overall attainment aggregates all requests within each round before averaging rounds, rather than weighting budget classes equally. C.5

Parcel handling

Client budgets follow recorded contact timing during parcel picking, flipping, and passing. We use 23 validation episodes, excluding one collection site with constant gripper readings that obscure contact timing. A contact interval spans 1.0 s before to 0.5 s after any gripper switch, and overlapping intervals merge; contact covers 36–78% of

XInfer (a) 8 robots

(b) 18 robots

(c) 20 robots

Cumulative fraction

1.00 0.75 0.50 0.25 0.00 -10

-1

0

1

10

-10

Actual - predicted (ms)

-1

0

1

10

-10

Actual - predicted (ms)

-1

0

1

10

Actual - predicted (ms)

Figure 13. Remaining-time prediction errors for running stages. Positive error is underprediction. Each represented request has equal total weight within its robot population. Curves include both tails; the horizontal scale is logarithmic outside [−1, 1] ms. Table 8. GPU component ablations. Strict gating jointly replaces slack stealing and arrival-aware admission. FS: SLO attainment of full FleetServe (mean of the nine rounds in the three campaigns, %). Other columns: gain of full FleetServe over each variant (percentage points, mean of three rounds paired with full FleetServe in the same campaign), with the number of rounds in which full FleetServe is higher. Bursty robot arrivals use the full measurement window. Entry-only Zero-slack Strict No arrivalNo interference Workload FS model gating aware adm. admission promotion Phases, N = 15 Phases, N = 16 Phases, N = 18 Phases, N = 20 Bursty arrivals Multiple budgets Parcel, N = 16 Parcel, N = 18

96.49 95.57 95.96 94.23 98.28 96.01 94.78 88.77

+24.29 (3) +45.74 (3) +45.47 (3) +45.68 (3) +9.92 (3) +14.24 (3) +56.62 (3) +51.44 (3)

+7.73 (3) +11.16 (3) +21.34 (3) +30.49 (3) +7.54 (3) +22.29 (3) +18.55 (3) +29.90 (3)

−0.40 (2) +5.02 (3) +11.31 (3) +13.80 (3) +0.58 (2) +1.47 (2) +16.97 (3) +18.05 (3)

+4.87 (3) +15.97 (3) +12.13 (3) +11.07 (3) +0.71 (2) +5.64 (3) +13.77 (3) +11.12 (3)

+0.27 (1) +0.19 (1) +1.04 (3) −0.29 (1) +2.26 (3) −0.25 (2) −0.77 (1) +2.01 (2)

Table 9. Arrival-aware GPU ablation, six-round means. Each cell gives U / T: FleetServe without / with arrival-aware admission under matched execution resources. Tight budgets are 90 ms for alternating phases and bursty arrivals, and 80 ms for multiple budgets. Wide-budget latency is for the 400 ms class. Bursty results use the full measurement window. Tight attainment Wide mean latency Goodput Overall attainment Workload (%) (%) (ms) (responses/s) Phases, N = 15 Phases, N = 16 Phases, N = 18 Bursty arrivals Multiple budgets

97.3 / 97.8 89.2 / 96.6 84.3 / 96.5 97.7 / 97.8 94.6 / 96.1

each episode (64% on average). Robot i replays episode i mod 23 from a seeded start offset and loops it. Contact intervals carry the 90 ms budget and the remaining time the 400 ms budget. Interval durations convert to response counts at the 180 ms period, because phases advance by successful responses as in alternating task phases. Each robot sends 1,200 requests with the same staggered starts, and each of up to 20 robots replays a different episode. Tight requests make up 61–63% of each scenario, and the fraction of robots in contact varies between 20% and 100%. FIFO and Slack-Gate are measured at 10–20 robots in three rounds; EDF comes from its separate campaign and Fleet-

94.5 / 95.6 78.3 / 93.2 68.7 / 93.0 95.7 / 96.0 75.0 / 81.9

114 / 148 200 / 208 260 / 262 94 / 95 139 / 236

80.4 / 80.4 74.0 / 79.0 68.7 / 78.4 66.6 / 66.5 77.7 / 74.7

Serve from a separate run, all with the protocol of Appendix C.4. Repeated runs of the same configuration differ by up to 2.9 percentage points in three-round mean at 16 and 18 robots, so comparisons across runs carry uncertainty of this size. All 941,655 requests sent in the measurement windows succeed. Table 11 reports every round. At 16 robots, tight-budget attainment is 56.6% for EDF, 92.6% for FleetServe, and 83.8% for Slack-Gate. From 14 robots upward, FleetServe returns wide-budget responses 7–56 ms sooner than Slack-Gate and increases goodput by 4.1–9.1 requests/s.

XInfer

Table 10. Scheduling policy comparison under matched execution resources. SLO attainment (%) is reported as R1 / R2 / R3. The N rows use alternating task phases; bursty arrivals use the full [30, 210) s send window. ∆goodput is the difference between three-round means (FleetServe − Slack-Gate; on-time responses/s). ∆ goodput Workload FIFO EDF Slack-Gate FleetServe (responses/s) N =8 74.86 / 99.96 / 99.81 99.86 / 99.92 / 99.99 99.97 / 100.00 / 99.90 99.94 / 99.91 / 99.94 N = 10 98.94 / 97.98 / 96.17 99.94 / 99.92 / 99.98 99.98 / 99.97 / 99.96 99.90 / 98.95 / 99.93 N = 12 99.77 / 99.75 / 99.58 99.66 / 98.45 / 99.70 99.92 / 99.86 / 99.27 99.89 / 97.99 / 99.79 N = 14 96.30 / 96.01 / 97.49 91.05 / 88.72 / 96.71 98.86 / 99.26 / 97.93 95.82 / 98.22 / 99.48 N = 15 81.19 / 81.36 / 82.35 87.00 / 90.61 / 91.35 95.88 / 97.28 / 95.88 96.85 / 97.14 / 99.19 N = 16 50.14 / 49.98 / 52.38 92.20 / 91.23 / 88.00 95.69 / 95.68 / 94.77 96.12 / 96.89 / 97.59 N = 18 49.97 / 49.99 / 50.00 88.92 / 89.42 / 94.05 89.99 / 90.33 / 93.63 96.12 / 95.56 / 97.43 N = 20 50.01 / 49.99 / 50.02 85.71 / 88.87 / 91.22 75.28 / 75.43 / 76.99 95.93 / 93.43 / 94.90 Bursty robot arrivals 88.89 / 89.00 / 88.37 98.81 / 97.99 / 97.22 95.61 / 95.61 / 97.50 98.66 / 98.79 / 99.27 Multiple response budgets, N = 15 85.49 / 86.31 / 79.02 94.13 / 94.80 / 96.44 92.16 / 92.47 / 92.79 96.61 / 95.01 / 96.58

Table 11. Parcel handling: overall SLO attainment (%) as R1 / R2 / R3 under matched execution resources. Robots 10 12 14 16 18 20

FIFO

EDF

Slack-Gate

FleetServe

99.79 / 99.84 / 99.68 98.79 / 99.74 / 94.03 97.46 / 96.57 / 94.39 54.46 / 38.09 / 47.65 37.65 / 37.82 / 37.65 36.94 / 37.12 / 36.84

99.79 / 92.78 / 95.77 97.48 / 99.60 / 86.81 90.26 / 89.49 / 93.02 72.45 / 73.19 / 73.24 65.32 / 69.96 / 73.45 50.65 / 52.13 / 55.79

99.88 / 99.91 / 98.35 99.26 / 99.70 / 99.83 95.87 / 95.80 / 93.89 89.25 / 87.64 / 92.76 81.46 / 80.64 / 83.59 56.90 / 55.18 / 58.40

98.07 / 99.76 / 99.74 99.81 / 99.26 / 99.66 95.49 / 99.10 / 98.67 95.21 / 94.50 / 96.51 86.68 / 89.43 / 90.97 62.66 / 66.61 / 68.99

−0.03 −0.14 +1.86 +5.34 +10.84 +10.62 +14.94 +21.90 +4.08 +12.28

Related documents

• Scaling VLA Serving Under Robotic Execution Constraints#1648999
• Human-to-Robot Transfer via Bridging Representation and Action Space in VLAs#1588674
• AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models#1648465
• Towards VLA-Dreamer: Refining VLA Behavior Using World Models#1105871
• T3R: Training-Free Two-Stage Token Refinement Towards Efficient and Robust VLA Models#1624181