AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks

AsymSpec: Efficient Cloud–Edge Speculative Decoding over Asymmetric Networks Guotao Yang, Hao Chen, Rui Guo, Xinyu Li, Liang Zheng, Sheng Chen, Yitao Hu* , and Keqiu Li

arXiv:2608.04974v1 [cs.DC] 5 Aug 2026

Tianjin University, China

Abstract—Cloud–edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative block. Under a constrained uplink, candidate messages may queue while the verifier is idle. Stop-and-wait scheduling leaves edge compute underutilized; optimistic same-request runahead can waste work when a rejection or an unexpected bonus token invalidates dependent drafts. We present AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms. Its asymmetric verification protocol keeps the common-path acceptance upload compact and moves richer, rejection-only correction information to the downlink. A total-variation (TV) certificate for the residual distribution determines whether a small target top-K response suffices; if not, the protocol progressively escalates through proposal-based exact recovery before falling back to the full distribution. Its confirmedprefix pipeline exposes only independent, valid requests to the edge scheduler and lets the cloud re-batch arrived blocks, hiding verification waits when another confirmed-prefix request is ready without using same-request runahead. Across three draft–target pairs, two workloads, and three asymmetric network profiles, our end-to-end evaluation shows that AsymSpec delivers 2.82–28.03× the output-token throughput of the strongest baseline. Index Terms—large language model serving, speculative decoding, asymmetric communication, multi-tenant scheduling

(a) Prior Works: Edge (Draft model)

Draft Req-A

Resource Idleu

Verify Req-A

(b) AsymSpec(Ours):

Cloud (Target model)

Draft Req-A Lower Data

Edge Drafting

Low Utilization

Insufficient BW Cloud (Target model)

Edge (Draft model)

Draft Req-A

Cloud Verification

Requests in Ready Pool Draft Req-B

Draft Req-C

Information exchange Verify Req-A

Draft Req-A

Network Bandwidth

High Utilization Edge-Cloud Data Size

Fig. 1. Conceptual comparison of prior cloud–edge speculative serving and AsymSpec. Prior designs can leave edge resources idle while a request is verified. AsymSpec reduces common-path communication and drafts independent requests from the ready pool during the verification wait, improving edge utilization without drafting descendants of an unresolved block.

replaced by one parallel verification pass, reducing the number of target invocations per output token. Crucially, the standard acceptance test and residual correction rule preserve the target model’s output distribution exactly, so the acceleration comes without sacrificing output quality [6], [7]. The gains of speculative decoding within a single request I. I NTRODUCTION motivate a natural follow-up question: when many requests are Large language models (LLMs) now power dialogue [1], served concurrently, can the draft–verify split also be distributed code generation [2], knowledge-intensive question answeracross machines? Recent systems place draft generation on ing [3], and complex reasoning [4]. A growing class of an edge device and target verification in the cloud [8]–[12]. applications wraps these models in agentic loops that plan, This placement matches a two-sided resource asymmetry: edge invoke external tools, and incorporate the returned observations devices usually cannot host a high-quality target model but can before generating again [5]. Unlike a conventional query that run a lightweight draft model, while cloud servers provide the produces one response and exits, an agent task repeatedly GPU memory and compute that verification demands. The edge alternates model generation, tool execution, and feedback handles inexpensive proposal generation, the cloud concentrates incorporation, and each round initiates another autoregressive on verification across concurrent requests, and otherwise idle generation phase. Because every token can be sampled only edge capacity is put to use. Cloud–edge execution is therefore after its predecessor has been fixed, the serial cost that is attractive, but nontrivial: its benefit depends on how effectively modest for a single reply compounds across the many rounds the system converts asymmetric network and compute resources of an agent interaction. Multi-round LLM workloads therefore into output tokens [13], [14]. The cloud target must still verify amplify the throughput cost of autoregressive decoding. every accepted candidate, and rejection correction is either One way to relieve this bottleneck is speculative decoding [6], exact or explicitly bounded. [7]. A lightweight draft model proposes a short continuation, Two concrete obstacles stand in the way of this goal. and a high-quality target model verifies multiple candidate Limitation 1: the verification path does not match positions in parallel. The longest accepted prefix is committed; any rejection is resolved with a target-guided correction before asymmetric bandwidth. Every speculative block must traverse the next round. In effect, several serial target decoding steps are the constrained uplink before the cloud can verify it, although common-path acceptance needs only candidate tokens and their * Corresponding author: Yitao Hu (e-mail: yitao@tju.edu.cn). draft probabilities. Rejection is less frequent but may require

much richer target-side information for correction, which can instead use the stronger downlink. Treating both outcomes as one symmetric exchange therefore either inflates every commonpath upload or returns excessive correction state. This mismatch can queue block uploads and starve the verifier while downlink capacity remains available. Moreover, a compact correction response still needs a residual-specific guarantee: target top-K coverage alone is insufficient because the draft may match the target on the head while their positive difference lies in the tail. Section III-B characterizes this directional load condition. Limitation 2: dependent runahead can turn overlap into invalid work. The next draft round of a request must begin from the tokens confirmed by its current verification. Waiting for that result creates a draft-compute bubble. Optimistic runahead hides the wait by drafting from an unconfirmed prefix, but a rejection then invalidates the dependent tokens, their transmissions, and possibly their cloud verification [10], [15]–[17]. Prior systems already interleave requests and batch verification [9], [18], [19]. The remaining dependency question is whether overlap can be restricted to the confirmed frontier: while one request awaits verification, another independent request may be ready without creating any descendant of an unresolved block. To address these two gaps, we present AsymSpec, which pairs a communication mechanism with a scheduling invariant. On the communication side, it progressively expands target support under a certificate for the realized residual distribution, composes each bounded correction into a request-level fidelity budget, and falls back to exact recovery when the certificate is insufficient. On the scheduling side, it admits only confirmedprefix requests and re-batches their arrived blocks at the cloud. We call a speculative block open from the start of edge drafting until it is committed or aborted; AsymSpec allows at most one such block per request. This invariant forbids dependent runahead rather than merely rescheduling it; when another request is ready, the scheduler uses that confirmed-prefix work to cover the verification wait. Figure 1 contrasts this confirmedprefix pipeline with prior request-internal execution: the edge uses independent ready requests to cover verification waits while the common path carries less data. This paper makes three contributions: • Problem analysis. We quantify the directional payload imbalance on the verification path and characterize the validity boundary between idle waiting and requestinternal work that a rejection or unseen bonus token in an ancestor block can invalidate. • Protocol and system design. We specify AsymSpec, combining an acceptance-sufficient common-path uplink with rejection-only progressive correction. The protocol certifies the residual-distribution error rather than relying on target top-K coverage alone, and composes each bounded correction into a request-level fidelity budget. A confirmed-prefix cross-request pipeline then decouples edge drafting from cloud verification batches without generating descendants of unresolved blocks. • Experimental evaluation. We evaluate AsymSpec across three draft–target model pairs, two workloads, and three

asymmetric network profiles. Relative to the strongest baseline, AsymSpec delivers 2.82–28.03× the outputtoken throughput. II. BACKGROUND A. LLM Inference and the Decoding Bottleneck An autoregressive large language model (LLM) generates an output by repeatedly predicting the next token conditioned on the confirmed prefix [3]. Its inference has a parallel prefill phase over the input prompt and a sequential decode phase over the output. During decoding, a key–value (KV) cache reuses states from the confirmed prefix instead of recomputing them [20], [21]. This cache reduces redundant computation within each step, but it does not remove the dependency across steps: token xn+1 must be selected before the model can generate xn+2 . Conventional decoding therefore invokes the full model once for each output token, leaving little opportunity to exploit parallelism across positions within one sequence. B. Speculative Decoding Speculative decoding targets this serial decode bottleneck without changing the target model’s autoregressive semantics. A lightweight draft model first supplies provisional future tokens, allowing one target-model forward pass to evaluate several candidate positions in parallel. Because the draft and target distributions differ, however, parallel scoring alone cannot determine which tokens may be committed. Speculative sampling closes this gap with a target-guided acceptance-andcorrection rule that converts the parallel scores into the same output distribution as ordinary target decoding [6], [7]. Let x1:n denote the confirmed prefix. The draft model extends it with a block of provisional tokens. At speculative position i, let qi denote the actual post-processed draft distribution conditioned on the confirmed prefix and preceding draft candidates, and let pi denote the target distribution evaluated at the same candidate position. Both are normalized categorical distributions over the same vocabulary V, represented by the canonical finite-precision values actually used for sampling. The endpoints bind each block to tokenizer, model, and samplingconfiguration versions; each endpoint must reproduce its own distribution exactly under that configuration. We focus on sampling; greedy decoding can be handled as a special case. The draft samples x̃i ∼ qi . The target accepts it with probability   pi (x̃i ) ai = min 1, . (1) qi (x̃i ) At the first rejected position, standard speculative sampling draws the correction token from the residual distribution X [pi (v) − qi (v)]+ ri (v) = , Zi = [pi (u) − qi (u)]+ . (2) Zi u For normalized pi and qi , Zi = DTV (pi , qi ), where DTV denotes total-variation distance, and Zi equals the rejection probability for a candidate sampled from qi . If Zi = 0, the correction path is never entered at that position. Replacing (2) with an uncertified compressed approximation can therefore change the target distribution.

Ignoring end-of-sequence tokens and maximum-length truncation, a block of draft depth γ commits every consecutively accepted candidate and one correction or bonus token. On the all-accepted path, the bonus is sampled with fresh randomness from the full target distribution after the accepted block. Let Cγ denote the number of tokens committed by a block of draft depth γ. If αi is the conditional acceptance probability at position i, then E[Cγ ] = 1 +

γ Y j X

αk .

(3)

Draft compute Target compute 33%

55%

1.74 s

Medium 150/1000 Mbps

40%

51%

2.54 s

Weak 50/150 Mbps

46%

51%

9.73 s

0

C. Cloud–Edge Collaborative Speculative Inference Cloud–edge speculative inference places the lightweight draft model at the edge and the larger target model in the cloud [8]– [12]. This placement matches their resource requirements: the edge handles inexpensive proposal generation, whereas the cloud provides the memory and compute required for target verification. In each round, the edge drafts from a confirmed prefix and uploads a candidate block; the cloud verifies the block and returns the committed result. Both endpoints then advance to the same prefix. The split places the draft distribution qi at the edge and the target distribution pi in the cloud. Given a shared confirmed prefix and sampling configuration, the cloud can evaluate acceptance using only mup i = (x̃i , qi (x̃i )),

(4)

plus compact metadata. Exact correction, by contrast, may depend on qi (v) for tokens other than x̃i . The common acceptance path and the conditional correction path can therefore impose substantially different communication requirements. Round-wise synchronization creates a separate dependency between drafting and verification. Waiting for the cloud response leaves the edge idle, whereas same-request runahead from an unresolved prefix can be invalidated by a rejection or an unseen bonus token [15]–[17]. An independent request with a confirmed prefix does not have this dependency. These pathdependent communication and work-validity considerations motivate the next section. III. M OTIVATION Cloud–edge speculative decoding couples model execution with network communication and round-wise synchronization, so its bottlenecks cannot be inferred from model compute alone. We first quantify the end-to-end latency composition and then analyze the two dominant sources of inefficiency revealed by this breakdown. A. End-to-End Bottleneck Breakdown We instrument a representative setup (Qwen3 4B/32B on GSM8K) and measure its latency breakdown under progressively weaker asymmetric network conditions. Figure 2 shows

Waiting

Strong 250/2500 Mbps

j=1 k=1

Under an approximation in which position-wise acceptance events are independent and identically distributed with common probability α, αγ estimates only the probability that all draft candidates survive; it is not the total block yield.

Upload Download

25 50 75 End-to-end latency composition (%)

100

Fig. 2. End-to-end latency breakdown under progressively weaker asymmetric network conditions. Communication (upload and download) and verification waiting together dominate total latency, while local draft and target compute remain a minor fraction. Segment labels indicate per-component shares; values at the bar ends report total latency. TABLE I D IRECTIONAL PAYLOADS OF A FULL - DISTRIBUTION VERIFICATION EXCHANGE . N IS THE TOTAL NUMBER OF CANDIDATE POSITIONS IN A TRANSMITTED BATCH . T HE CORRESPONDING DOWNLOAD PAYLOADS FOR N = 4, 32, AND 256 ARE 64 B, 512 B, AND 4,096 B, RESPECTIVELY. Model

|V|

Upload payload (MiB)

Up/down

N = 4 N = 32 N = 256 Qwen3 151,936 GLM-4 151,552 Llama 3.1 128,256

1.159 1.156 0.979

9.273 9.250 7.828

74.188 74.000 62.625

18,992× 18,944× 16,032×

that total latency rises from 1.74 s to 2.54 s and 9.73 s as the network profile weakens. The upload share grows from 33.3% to 40.2% and 46.0%, while the waiting component remains the largest at 50.6–54.7%. Together, communication and waiting account for over 85% of end-to-end latency under all conditions, far exceeding local draft and target compute. The breakdown exposes two distinct bottlenecks. First, every speculative block must traverse the uplink before verification, making the upload a substantial part of the critical path. Second, the large waiting component indicates that round-wise synchronization leaves considerable edge compute unproductive. The following two subsections analyze these communication and scheduling bottlenecks, respectively. B. Limitation 1: The Verification Path Does Not Match Asymmetric Bandwidth Existing cloud–edge speculative verification can place a disproportionate amount of traffic on the constrained uplink. Every speculative block must arrive at the cloud before verification can begin, and an exchange that uploads a vocabularysized distribution for each candidate position incurs a payload that grows with both the candidate count and the model vocabulary. The reverse message carrying the verification result is substantially smaller. As shown in Table I, even a batch containing only four candidate positions produces a 0.98–1.16 MiB upload, compared with a 64 B download. At N = 256, the upload reaches 62.63–74.19 MiB, while the download remains 4,096 B. The

(a) Conventional Edge-cloud Speculation: Stop-and-wait Edge Request A Draft A1 (Draft model)

Draft A2

Idleu

Cloud (Target model)

Verify A1

Idleu Edge Drafting

Verify A2

(b) Optimistic Runahead (Same Request) Edge Request A Draft A1 (Draft model) Cloud (Target model)

Draft A2

Draft A3

Verify A1

Pvalid < 1 Rejection => A2/A3 Invalidated

Verify A2

Request A Draft A1 Edge Request B (Draft model)

Request C Cloud (Target model)

Draft A2 Draft B2

Draft B1

Pvalid ≤ αγ .

u

For 0 < α < 1, the upper bound decreases exponentially with γ, but it may still be loose because, even when every candidate in A1 is accepted, the target may append an unseen bonus token that changes the prefix from which A2 should have been drafted. A rejection or unseen bonus can therefore invalidate dependent draft tokens, their transmissions, associated verification work, and KV suffixes. Together, Figure 3(a) and (b) expose the limitation of request-internal execution. Stop-and-wait preserves validity but leaves the edge idle, whereas runahead increases utilization by producing work whose validity depends on an unresolved block. Raw utilization can consequently increase without a corresponding improvement in throughput.

Invalid Work Data

Draft C1 Mixed verification batch (A1,B1,C1)

Cloud Verification

Idle Bubble

(c) AsymSpec: Confirmed Prefix Multi-requests Scheduling

Pvalid = 1 No Invalid => High Util.

Pvalid denote the probability that the dependent drafts A2–A3 in Figure 3(b) remain valid. If A1 contains γ candidates and each candidate is accepted with probability α, approximating the acceptance events as independent gives

Invalid data

Fig. 3. Execution alternatives during verification. (a) Stop-and-wait leaves the edge idle until A1 is confirmed. (b) Same-request runahead drafts A2 and A3 from an unresolved prefix, so their work may be invalidated when A1 changes the committed prefix. (c) Blocks drafted from independent confirmed prefixes provide a dependency-free comparison. The timeline is schematic and not to scale.

(6)

upload-to-download ratio is therefore 16,032–18,992× across D. Takeaways the three models. Increasing N from 4 to 256 increases both The analysis yields two conclusions: (1) the verification the candidate count and the upload payload by 64×, showing exchange should match asymmetric bandwidth and event frethat the uplink cost scales linearly with N |V|. quency, avoiding vocabulary-scale traffic on the recurring uplink This vocabulary-scale upload is disproportionate to the path when richer information is needed only conditionally; information needed on the common verification path. As and (2) scheduling efficiency should be measured by valid summarized in (4), acceptance testing at each candidate position committed work rather than raw utilization, since stop-andrequires only the sampled token, its draft probability, and wait leaves compute idle while dependent runahead may spend compact metadata, whereas richer distribution information resources on work that is later invalidated. is needed only when rejection requires correction. A fullIV. A SYM S PEC D ESIGN distribution exchange therefore places this conditional information cost on every recurring uplink message, which can gate A. Design Overview verification even when cloud compute and downlink capacity AsymSpec consists of an edge gateway and a cloud verifier. remain available. The edge maintains confirmed request prefixes, permits at most one open block per request, generates draft blocks, and retains C. Limitation 2: Dependent Runahead Creates Invalid Work Existing request-internal execution faces two unsatisfactory the associated draft state. The cloud collects asynchronously choices while a speculative block is being verified. Stop-and- arrived blocks, dynamically forms verification batches across wait preserves the confirmed prefix but leaves the edge idle, requests, and verifies them with the target model. The design whereas same-request runahead fills the wait with descendants combines an asymmetric verification protocol with a requestof an unresolved block whose results may later become invalid. decoupled execution pipeline. During execution, the edge selects a ready request, generates Figure 3(a) and (b) illustrate these two cases, respectively. a draft block, and uploads token–scalar pairs (x̃i , qi (x̃i )) with Under stop-and-wait in Figure 3(a), the edge drafts A1 compact metadata. The cloud dynamically batches incoming and then waits for upload, cloud queueing and verification, blocks and verifies their candidate positions in parallel. On the downlink, and commit before beginning A2. Let Tedge denote all-accepted path, it returns the accepted length and a bonus the drafting time and Tof f the interval during which the request token; at the first rejection, it invokes progressive correction is off the edge critical path. The edge utilization attributable using top-K support, exact proposals, or full fallback. After to this request is bounded by the result is resolved, the edge and cloud commit the same Tedge single Uedge ≤ . (5) suffix, and the request becomes ready for its next block. Tedge + Tof f As communication and verification delays increase, Tof f B. Asymmetric Verification Protocol Insight and approach. Acceptance and correction differ in dominates the denominator and the idle interval in Figure 3(a) when they are required and in their information demands. expands. Same-request runahead, shown in Figure 3(b), increases Every speculative block must be checked, but acceptance raw utilization by drafting A2 and A3 before A1 is resolved. requires only each candidate token and its draft probability; However, these descendants are conditioned on the assumption a rejection is conditional and may require richer targetthat A1 commits exactly the prefix used to draft them. Let side information for correction. AsymSpec therefore uploads

only acceptance-sufficient token–probability pairs and returns correction information on demand over the downlink. After a rejection, it starts with a small target top-K. Using the retained draft distribution, the edge bounds the TV distance between the support-truncated correction distribution defined by these K tokens and the standard exact residual distribution. We call this bound the residual-distribution TV certificate. The protocol expands K until the bound fits the request’s error budget and otherwise falls back to an exact path.

PProposition 1 (Residual-TV certificate). Let Ztail = v ∈S / [p(v) − q(v)]+ . Then Ztail δp DTV (r, r̂) = . (8) ≤ ZS + Ztail ZS + δ p | {z } ϵ̄K

Restricting r to S moves exactly Ztail /(ZS +Ztail ) probability mass, and Ztail ≤ δp , which proves the bound. The certificate is computable at the edge from the returned target support and retained draft state, so the downlink support can be expanded 1) Acceptance-Sufficient Uplink: Equation (1) shows that the until the computable certificate meets the requested error budget. cloud requires only (x̃i , qi (x̃i )) for candidate i. A depth-γ block A valid implementation must accumulate these statistics exactly therefore uploads O(γ) token–scalar pairs instead of O(γ|V|) or conservatively round δp upward and ZS downward, so draft probabilities. The full draft distribution remains at the edge numerical error cannot understate the certificate. through a queryable draft state Qi , defined by Qi (v) ≜ qi (v) For correction j following pre-correction history h, set for every v ∈ V; after a rejection, the edge evaluates Qi (v) only ϵj (h) = ϵ̄Kj (h) when bounded correction uses selected support for tokens selected by the cloud’s correction response. This size Kj (h), and set ϵj (h) = 0 when exact correction is used. converts an unconditional distribution upload into a demandA bounded draw is allowed only when ZS > 0 and ϵj (h) fits driven downlink support query. the request’s remaining budget. Otherwise, the cloud extends The target evaluates all candidate positions in parallel and the same ordered support; increasing K cannot increase δp applies (1) from left to right. An all-accepted block follows the or decrease ZS and therefore cannot worsen the certificate. A compact result path, whereas the first-rejection path retains the failed certificate ultimately enters an exact mode rather than target distribution and accepted-prefix state until one correction relaxing fidelity. token is chosen. If Qi cannot reproduce the agreed draft Corollary 1 (Request-level budget). For a finite request probabilities, the block aborts. Every transmitted acceptance under a common fixed stopping and abort policy and with scalar, certificate statistic, and exact-recovery probability uses a request-level TV budget ϵreq , let Sexact and Sbounded a lossless encoding of the canonical normalized finite-precision denote the distributions of completed model-token traces under value used for sampling; lossy encodings are out of scope. exact correction throughout and under the budgeted protocol, Block identifiers bind every response to the request, base-prefix respectively. Suppose that, for every correction j and every version, candidate position, and sampling configuration that reachable pre-correction history h, the TV distance between produced it. This binding makes delayed or duplicated messages the exact and implemented conditional correction distributions detectable and prevents a correction from being applied after is at most ϵj (h). For an online path, write h<j for its history the request has advanced. Retransmission is idempotent: the before correction j. If every online path satisfies X endpoints reuse cached acceptance decisions, target supports, ϵj (h<j ) ≤ ϵreq , (9) and random outcomes rather than sampling again. These rules j do not change the underlying acceptance law, but they make the distributional argument meaningful in an asynchronous then DTV (Sexact , Sbounded ) ≤ ϵreq . A coupling that charges implementation where messages can be retried or arrive late. the first differing correction proves this implication. Each realized bounded correction is charged once; exact corrections 2) Certified Progressive Correction: At the rejected position, consume zero budget. If abort is externally observable, it is write p, q, r, and Z for the corresponding pi , qi , ri , and Zi . The included as an output symbol in both distributions. cloud first returns a small target top-K. The edge combines this 3) Exact Recovery: When the certificate is insufficient, support with local draft probabilities to construct a bounded immediately sending the full target distribution would forfeit the residual approximation. Crucially, target top-K coverage alone communication benefit. AsymSpec first uses a proposal-based is not a distribution-fidelity guarantee: p and q may agree on exact path: with randomness independent of the triggering most of the returned head while the positive residual [p − q]+ rejection and previous trials, the cloud draws Y ∼ p and sends lies in the omitted tail. AsymSpec therefore certifies error in only (Y, p(Y )), while the edge retrieves q(Y ) and accepts with  the correction distribution itself.   1 − q(v) , p(v) > 0, Order tokens by decreasing p(v), break ties by token ID, p(v) + b(v) = and let S = TopK(p). From the returned (v, p(v)) pairs and  (10) 0, p(v) = 0, local q(v) lookups, the edge computes X P(Y = v, accept) = p(v)b(v) = [p(v) − q(v)]+ . δp = 1 − p(v), av = [p(v) − q(v)]+ , The uniform variate used for acceptance is independent v∈S X (7) of Y and all other protocol randomness and is compared ZS = av . against b(Y ). Conditioned on success, the proposal is therefore v∈S distributed exactly as the residual r in (2); this instantiates For ZS > 0, define r̂(v) = av /ZS on S and zero outside S. the standard rejection-sampling identity as a two-endpoint

correction protocol. If the protocol attempts Nprop mutually independent proposals, P(all Nprop proposals fail | p, q) = (1 − Z)Nprop . The first accepted proposal is exact; if all attempts fail, an independent full residual draw is also exact. Repeated delivery of the same proposal attempt reuses its cached outcome rather than resampling. This cap is important because high draft– target agreement makes rejection rare but can make proposal acceptance, whose mass is Z, small once rejection occurs. Bounded top-K, proposal-based exact recovery, and full fallback transmit O(K), O(Nprop ), and O(|V|) entries across one correction, respectively. Thus the protocol pays for a full distribution only when neither a certified support nor a bounded number of exact proposals suffices. The three modes form a staged runtime decision rather than a fixed global choice of K: start from a small support, expand it monotonically when the certificate is too loose, and reserve exact recovery for the remaining cases. Because residual-distribution error rather than target coverage alone determines sufficiency, the same K may pass one rejection and fail another. Per-event certification thus avoids a workloadspecific threshold while capping conditional communication. C. Request-Decoupled Pipeline Insight and approach. The next block of a request depends on the current block’s committed prefix, whereas blocks from different requests have no prefix dependency on one another. Based on this observation, AsymSpec builds a requestdecoupled pipeline that overlaps edge drafting, network transfer, and cloud verification across requests rather than speculating ahead within one request. It combines two mechanisms: confirmed-prefix scheduling admits at most one open block per request and switches the edge to another ready request while verification is pending; decoupled verification batching collects asynchronously arrived blocks in a global queue and dynamically re-batches them at the cloud, independently of their arrival grouping. Together, these mechanisms preserve per-request order and eliminate cross-block invalidation caused by unresolved prefixes while converting verification stalls and arrival skew into useful cross-request parallelism. 1) Confirmed-Prefix Frontier: A block is open from the start of drafting until it commits or aborts, and for every request ρ, AsymSpec enforces Nopen (ρ) ≤ 1.

(11)

A request is on the confirmed frontier only when it has no open block and both KV cursors match its canonical prefix. It leaves the frontier when drafting starts and returns only after the block resolves. No cross-block invalidation. Under (11), a rejection can invalidate only the uncommitted suffix of the current block: no draft, transmission, or cloud verification for a dependent successor exists. Waiting time is instead covered by another frontier request whose prefix is already valid. This trades away single-request overlap; when no other request is ready, the edge waits rather than speculating from an unresolved prefix.

Any fairness policy may choose among ready requests without changing this property. 2) Decoupled Verification Batching: Requests produce blocks at different times because their draft depths, edge service times, and network delays differ. Preserving their arrival grouping would transfer this skew to the target verifier. AsymSpec instead places every arrived block whose base prefix still matches the request’s confirmed prefix in a global verification queue and forms target batches from currently available work. This decoupling turns staggered cross-request arrivals into batching opportunities while keeping per-request order at commit. Conventional queue thresholds control launch timing without relaxing the confirmed-frontier or correction-fidelity invariants. V. E VALUATION A. Evaluation Setup 1) Implementation and Testbed: We implement AsymSpec as two ZeroMQ-connected Python services [22]. The edge runs draft generation and request scheduling on one RTX 3090; the cloud performs target verification and independent re-batching on four A100 GPUs. 2) Models and Workloads: We evaluate three draft/target pairs: Qwen3 (4B/32B) [23], GLM-4 (9B/32B) [24], [25], and Llama 3.1 (8B/70B) [26]. These pairs span three model families and 32B–70B targets. GSM8K [27] and HumanEval [2] represent mathematical reasoning and code generation. 3) Network Conditions: Using Linux traffic control and NetEm [28], we emulate three representative cloud–edge access scenarios with asymmetric uplink/downlink capacity. Their bandwidth ranges are consistent with measurements from commercial 4G/5G networks [29]–[31]. • Weak—50/150 Mbps: congested or LTE-like cellular access with scarce uplink capacity. • Medium—150/1,000 Mbps: sub-6 GHz 5G access with a faster downlink but still-limited uplink capacity. • Strong—250/2,500 Mbps: high-capacity 5G/mmWave access, where network transfer is less likely to dominate. 4) Baselines: We compare AsymSpec against three representative designs: • Standard Spec [6], [7]: canonical speculative decoding and the algorithmic reference. • CoSine [18]: collaborative cloud–edge speculative inference with decoupled execution. • PipeInfer [15]: asynchronous pipelined speculative inference. 5) Metrics: We report output-token throughput (tokens/s), end-to-end latency, time to first token (TTFT), and time per output token (TPOT). B. End-to-End Throughput AsymSpec attains the highest output-token throughput at all 18 model–workload–network operating points in Figure 4, achieving 2.82–28.03× the throughput of the strongest baseline

Standard Spec

PipeInfer

AsymSpec

GLM-4 9B/32B

Llama 3.1 8B/70B

GSM8K

1000 300 100 30 10 1000 HumanEval

Output-token throughput (tokens/s, log scale)

Qwen3 4B/32B

CoSine

300 100 30 10

Strong 250/2500

Medium 150/1000

Weak 50/150

Strong 250/2500

Medium 150/1000

Strong 250/2500

Weak 50/150

Medium 150/1000

Weak 50/150

Network profile (uplink/downlink in Mbps)

C. Component Analysis Table II compares the Full configuration with variants that remove or replace asymmetric correction, top-K support, and request-decoupled pipelining. Across the recorded aggregates, the Full configuration has the highest throughput and the lowest end-to-end latency, TTFT, and TPOT under all three network profiles. Even against the strongest reduced configuration in

10

−1

10

−3

Residual TV

Target tail mass

(a) Residual fidelity 1.00 Recall

at each point (geometric mean, 7.96×). Geometrically averaged over the three network profiles, this ratio ranges from 4.20× for GLM-4 9B/32B on HumanEval to 12.55× for Qwen3 4B/32B on GSM8K. Across the 18 operating points, AsymSpec sustains 201.31–823.65 output tokens/s. Its geometric-mean throughput across the six model–workload pairs is 438.17, 443.81, and 446.36 tokens/s under the Strong, Medium, and Weak profiles, respectively. We next examine how each method’s throughput changes as the network degrades. From Strong to Weak, geometricmean throughput falls by 78.6% for Standard Spec, 71.4% for CoSine, and 71.5% for PipeInfer: every baseline loses roughly three-quarters of its throughput as the profile changes from 250/2,500 to 50/150 Mbps. AsymSpec instead increases by 1.9% over the same range. This pattern is consistent with its acceptance-sufficient common path and rejection-only downlink correction. Under the tested profiles, AsymSpec’s throughput is therefore much less sensitive to network degradation than that of the baselines. Differences across model–workload pairs reflect draft cost, acceptance behavior, and available batching work. Across the tested profiles, AsymSpec maintains stable geometric-mean throughput from Strong to Weak, unlike the large declines observed for all three baselines.

Error (log)

Fig. 4. End-to-end output-token throughput under asymmetric network conditions. Each panel compares the four serving methods across the Strong, Medium, and Weak profiles; network labels report uplink/downlink bandwidth. Panels are organized by workload (rows) and draft/target model pair (columns). AsymSpec attains the highest throughput at all 18 operating points, up to 28.03× the throughput of the strongest baseline, and is the only method whose geometric-mean throughput does not drop from Strong to Weak. Bars share a logarithmic y-axis with a lower limit of 10 tokens/s; higher is better.

0.75 0.50

Top-1 recall 5

16

Top-5 recall

Top-10 recall

64 256 1024 Returned target support K (b) Returned-support coverage

8192

Fig. 5. Correction fidelity on GSM8K with a Qwen3 4B draft and 32B target. Panel (a) reports residual TV and target-tail mass; (b) reports top-1, top-5, and top-10 recall within the returned support.

each profile, Full raises throughput by 13.0–20.1% while also reducing end-to-end latency, TTFT, and TPOT. The communication-side mechanisms become increasingly important as bandwidth declines. Jointly removing asymmetric correction and top-K support reduces throughput by 93.7%, 96.0%, and 98.6% relative to Full from Strong to Weak. The same configuration raises end-to-end latency to 32.1×, 50.4×, and 152×; its TTFT and TPOT overheads similarly reach 87.6× and 148× in the Weak profile. Removing top-K alone is less severe but still reduces throughput by 25.5–76.1% and raises Weak-profile end-to-end latency to 7.7×. Across these

TABLE II C OMPONENT ANALYSIS UNDER THREE ASYMMETRIC NETWORK PROFILES . E NTRIES REPORT ABSOLUTE VALUES FOLLOWED BY VALUES RELATIVE TO THE COMPLETE A SYM S PEC CONFIGURATION , LABELED F ULL IN THIS TABLE , IN PARENTHESES . A RROWS INDICATE THE PREFERRED DIRECTION ; BOLD AND UNDERLINING MARK THE BEST AND SECOND - BEST RESULTS IN EACH METRIC – NETWORK GROUP. Metric

Throughput tokens/s ↑

E2E latency seconds ↓

TTFT seconds ↓

TPOT seconds ↓

Qwen3 Accuracy (%)

Network profile (uplink/downlink)

Configuration

Strong 250/2,500 Mbps

Medium 150/1,000 Mbps

Weak 50/150 Mbps

Full

751.02 (100.0%)

760.14 (100.0%)

749.87 (100.0%)

No asymmetric correction / top-K No top-K No pipeline No pipeline + runahead

47.46 (6.3%) 559.45 (74.5%) 664.62 (88.5%) 572.99 (76.3%)

30.42 (4.0%) 499.29 (65.7%) 632.72 (83.2%) 580.88 (76.4%)

10.15 (1.4%) 179.59 (23.9%) 637.06 (85.0%) 571.91 (76.3%)

Full

19.347 (1.00×)

19.217 (1.00×)

19.556 (1.00×)

No asymmetric correction / top-K No top-K No pipeline No pipeline + runahead

621.154 (32.11×) 38.281 (1.98×) 23.725 (1.23×) 33.360 (1.72×)

968.581 (50.40×) 44.461 (2.31×) 25.932 (1.35×) 32.677 (1.70×)

2,966.391 (151.69×) 149.660 (7.65×) 25.775 (1.32×) 32.736 (1.67×)

Full

0.432 (1.00×)

0.434 (1.00×)

0.443 (1.00×)

No asymmetric correction / top-K No top-K No pipeline No pipeline + runahead

7.044 (16.32×) 1.062 (2.46×) 0.531 (1.23×) 0.651 (1.51×)

11.826 (27.27×) 1.151 (2.66×) 0.627 (1.45×) 0.640 (1.48×)

38.801 (87.64×) 1.482 (3.35×) 0.616 (1.39×) 0.565 (1.28×)

Full

0.074 (1.00×)

0.069 (1.00×)

0.078 (1.00×)

No asymmetric correction / top-K No top-K No pipeline No pipeline + runahead

2.408 (32.47×) 0.146 (1.97×) 0.091 (1.23×) 0.128 (1.73×)

3.752 (54.30×) 0.170 (2.46×) 0.099 (1.44×) 0.126 (1.82×)

11.481 (147.95×) 0.581 (7.49×) 0.099 (1.27×) 0.126 (1.63×)

GLM-4

admitting independent work across requests. Together, the throughput and latency columns in Table II show that asymmetric correction and top-K control address communication amplification, whereas the decoupled pipeline converts the resulting slack into useful cloud-side concurrency. The ablation effects need not add linearly: changing correction traffic also changes block arrival times and the amount of work available for a verification batch, while changing the scheduler alters when link capacity is exercised. The component study therefore supports a division of roles rather than an exact numerical decomposition of total speedup. Communication control is most visible as the network weakens; request decoupling remains useful across profiles because arrival skew and per-request dependency persist even when bandwidth is less restrictive.

Llama 3.1

75 50

Accuracy (%)

(a) GSM8K

75 50 16

128

512 1024 8192 Correction support K (b) HumanEval

40k

Full

D. Correction Sensitivity The returned support K trades correction payload against residual fidelity. We examine its effect on protocol fidelity, task accuracy, and end-to-end latency. The fidelity experiment Fig. 6. Task accuracy across correction-support settings on (a) GSM8K and (b) HumanEval. Lines identify the three draft/target model pairs. uses a denser grid between K = 5 and K = 8,192, while the accuracy and latency experiments use seven settings from configurations, bounded correction support is associated with K = 16 through Full. lower startup and token-generation overhead as bandwidth 1) Protocol Fidelity: Figure 5 evaluates the Qwen3 4B draft declines. and 32B target pair on GSM8K. Residual TV and target-tail The scheduling variants expose a complementary effect. mass decrease rapidly before K = 256 and then flatten. TopRemoving the decoupled pipeline lowers throughput by 11.5– 1 recall is 1.0 throughout the tested range from K = 5 to 16.8% and raises the three latency metrics to 1.2–1.4× Full. K = 8,192. At K = 16, top-5 and top-10 recall already reach Replacing it with runahead retains only about 76% of Full’s 0.987 and 0.925, with a residual TV of 0.0116. At K = 256, throughput and raises the same metrics to 1.3–1.8× Full, so both recall values reach 1.0 and residual TV falls to 4.85 × same-request runahead alone cannot recover the benefit of 10−4 . Since the Qwen3 32B target has 151,936 vocabulary

Strong (250/2500 Mbps)

Medium (150/1000 Mbps)

Weak (50/150 Mbps)

Qwen3 1.0× 1.1× 0.9× 1.2× 1.1× 1.5× 1.5×

1.0× 0.9× 0.9× 0.9× 1.0× 1.5× 1.3×

1.0× 1.0× 0.9× 0.9× 1.0× 4.9× 5.1×

GLM-4 1.0× 1.0× 1.0× 1.0× 1.0× 1.0× 1.3×

1.0× 1.0× 1.0× 1.1× 1.0× 1.0× 1.0×

1.0× 1.0× 1.0× 1.0× 1.1× 1.1× 1.3×

Llama 3.1 1.0× 1.0× 1.1× 1.0× 1.0× 1.1× 1.3×

1.0× 1.0× 1.0× 1.1× 1.0× 1.1× 1.9×

1.0× 1.0× 1.0× 1.0× 1.0× 2.0× 3.9×

16

16

16

128

512 1024 8192 40k

Full

128

512 1024 8192 40k

Full

128

512 1024 8192 40k

Full

Fig. 7. End-to-end latency for 100 requests at 4 requests/s. Each row is normalized independently to K = 16 for the same model and network profile; labels give ratios rounded to one decimal place. Full denotes full-vocabulary correction.

entries, this support provides 593.5× compression. Expanding to K = 8,192 reduces residual TV by only another 3.67×10−4 . Even at K = 8,192, the returned support provides 18.55× compression relative to the full 151,936-entry vocabulary. Thus, compact support already provides high correction fidelity with a substantially smaller payload. 2) Task Quality: Figure 6 compares bounded correction support with Full for each model and workload. Across the 36 within-pair comparisons, bounded support has an unweighted mean difference of −0.52 percentage points relative to Full. Separated by workload, the mean bounded-minus-Full difference is −1.35 percentage points on GSM8K and +0.30 percentage points on HumanEval. At least one bounded setting matches or exceeds Full in five of the six model–workload pairs; in the remaining Qwen3–GSM8K case, the best bounded setting achieves 75.0% versus 77.0% with Full. The direction of the pointwise difference varies across K, models, and workloads. 3) Runtime Sensitivity: Figure 7 normalizes each row independently to the K = 16 latency of the same model and network profile. The bounded settings from K = 128 to K = 8,192 remain close to this reference, spanning 0.88– 1.19× for Qwen3, 0.95–1.06× for GLM-4, and 0.95–1.06× for Llama 3.1. The K = 40,000 setting is non-monotonic: for Qwen3 it reaches 1.50×, 1.47×, and 4.93× under Strong, Medium, and Weak, respectively, while Llama 3.1 reaches 2.01× under Weak. Full support becomes most expensive under the Weak profile, where latency rises to 5.1×, 1.3×, and 3.9× the respective per-model references. Thus, supports up to K = 8,192 add little latency within each model configuration, whereas K = 40,000 and full-vocabulary support produce larger overheads in specific model–network combinations, most prominently for Qwen3 under Weak.

instead addresses the communication and dependency costs that arise when drafting and verification run on opposite sides of an asymmetric network. Cloud–edge collaborative inference. Recent systems place draft generation at the edge and target verification in the cloud [8]–[12]. The verification path runs through every speculative block, but cloud–edge bandwidth is asymmetric: compact acceptance inputs repeatedly consume the constrained uplink, whereas richer correction information is needed only after rejection and can use the stronger downlink. Prior work sparsifies or adaptively compresses probability messages [11], [35], while Quantize-Sample-and-Verify offers an exact communication mechanism [36]. AsymSpec is complementary: it tests whether the returned target support is sufficient for the realized residual, composes bounded correction error across a request, and falls back to exact recovery when the certificate fails. Serving and pipelining. Continuous batching and iterationlevel scheduling improve accelerator utilization for centralized LLM serving by admitting and retiring requests at fine granularity, interleaving prefill and decode work, or assigning these phases to different resources [20], [21], [37]. Distributed speculative systems further overlap drafting and verification, interleave requests, or speculate beyond unresolved blocks [9], [15]–[19]. In contrast, AsymSpec permits at most one open block per request and fills verification waits only with confirmed-prefix work from independent requests, eliminating same-request invalidation cascades by construction.

VII. C ONCLUSION

AsymSpec combines certified progressive correction with a confirmed-prefix pipeline for cloud–edge speculative decoding. Speculative decoding. Classical speculative decoding uses It keeps the common acceptance path compact, expands cora small draft model to propose tokens and a larger target rection support only after rejection, and retains exact fallback; model to verify them in parallel while preserving the target meanwhile, independent requests fill verification waits without distribution [6], [7]. Later work improves proposal quality same-request runahead. Bounded supports through K = 8,192 and verification efficiency through tree-structured candidates, closely track the task quality of full-vocabulary correction and multi-token heads, and adaptive draft structures [32]–[34]. the corresponding per-model, per-network K = 16 latency Tree structures expose several candidate continuations to one reference, whereas very large or full-vocabulary supports can verification pass, multi-token heads predict multiple future become costly on a weak downlink. Across three model positions without repeatedly invoking a separate drafter, and pairs, two workloads, and three asymmetric network profiles, adaptive structures vary the candidate topology with the ex- AsymSpec achieves 2.82–28.03× the output-token throughput pected verification gain. These methods primarily reduce target- of the strongest baseline, while its geometric-mean throughput model invocations within a local inference stack; AsymSpec changes by 1.9% from Strong to Weak across the tested profiles. VI. R ELATED W ORK

R EFERENCES [1] R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha et al., “LaMDA: Language models for dialog applications,” arXiv preprint arXiv:2201.08239, 2022. [2] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021. [3] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 1877–1901. [4] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 824–24 837. [5] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in The Eleventh International Conference on Learning Representations, 2023. [6] Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in Proceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 202, 2023, pp. 19 274–19 286. [7] C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318, 2023. [8] J. Ning, C. Zheng, and T. Yang, “DSSD: Efficient edge-device LLM deployment and collaborative inference via distributed split speculative decoding,” arXiv preprint arXiv:2507.12000, 2025. [9] J. Park, S. Cho, and D. Han, “SpecEdge: Scalable edge-assisted serving framework for interactive LLMs,” arXiv preprint arXiv:2505.17052, 2025. [10] Y. Han, Y. Gao, B. Hu, M. B. Mashhadi, Y. Duan, P. Xiao, and Y. Zhang, “PipeSD: An efficient cloud-edge collaborative pipeline inference framework with speculative decoding,” arXiv preprint arXiv:2605.13319, 2026. [11] C. Zheng and T. Yang, “Communication-efficient collaborative LLM inference via distributed speculative decoding,” arXiv preprint arXiv:2509.04576, 2025. [12] C. Zheng, K. Zhang, C. Sun, W. Zhang, Q. Liu, and A. A. Tesfay, “Fast collaborative inference via distributed speculative decoding,” arXiv preprint arXiv:2512.16273, 2025. [13] Y. Lyu, B. Irukulapati, and J. P. Champati, “Speculation at a distance: Where edge-cloud speculative decoding actually pays off,” arXiv preprint arXiv:2606.25091, 2026. [14] D. Xu, W. Yin, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu, “LLMCad: Fast and scalable on-device large language model inference,” arXiv preprint arXiv:2309.04255, 2023. [15] B. Butler, S. Yu, A. Mazaheri, and A. Jannesari, “PipeInfer: Accelerating LLM inference using asynchronous pipelined speculation,” in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis, 2024, pp. 1–19. [16] B. McDanel, “AMUSD: Asynchronous multi-device speculative decoding for LLM acceleration,” arXiv preprint arXiv:2410.17375, 2024. [17] T. Liu, Y. Li, Q. Lv, K. Liu, J. Zhu, W. Hu, and X. Sun, “PEARL: Parallel speculative decoding with adaptive draft length,” arXiv preprint arXiv:2408.11850, 2024. [18] L. Gao, J. Liu, X. Zhang, G. Gao, and Y. Liao, “CoSine: Enhancing LLM serving via collaborative and decoupled speculative inference,” in IEEE INFOCOM 2026 - IEEE Conference on Computer Communications, 2026, pp. 1–10. [19] X. Li, J. Fan, Q. Wang, D. Spatharakis, S. Ghafouri, H. Vandierendonck, D. John, B. Ji, A. R. Butt, and D. S. Nikolopoulos, “WISP: Wasteand interference-suppressed distributed speculative LLM serving at the edge via dynamic drafting and SLO-aware batching,” arXiv preprint arXiv:2601.11652, 2026. [20] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” in Proceedings of the 29th Symposium on Operating Systems Principles, 2023, pp. 611–626. [21] Y. Zhong, S. Liu, J. Chen, J. Hu, Y. Zhu, X. Liu, X. Jin, and H. Zhang, “DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving,” in 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, 2024, pp. 193–210.

[22] P. Hintjens, ZeroMQ: Messaging for Many Applications. O’Reilly Media, 2013. [23] Qwen Team, “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025. [24] Team GLM, “ChatGLM: A family of large language models from GLM130B to GLM-4 All Tools,” arXiv preprint arXiv:2406.12793, 2024. [25] Z.ai, “GLM-4-0414 model series,” Official model repository, 2025. [Online]. Available: https://github.com/zai-org/GLM-4 [26] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang et al., “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [27] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021. [28] S. Hemminger, “Network emulation with NetEm,” in Proceedings of the 6th Australian National Linux Conference (linux.conf.au 2005), Canberra, Australia, Apr. 2005. [29] D. Xu, A. Zhou, X. Zhang, G. Wang, X. Liu, C. An, Y. Shi, L. Liu, and H. Ma, “Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption,” in Proceedings of the 2020 ACM SIGCOMM Conference. ACM, 2020, pp. 479–494. [Online]. Available: https://doi.org/10.1145/3387514.3405882 [30] Z. J. Kong, Q. Xu, J. Meng, and Y. C. Hu, “AccuMO: Accuracycentric multitask offloading in edge-assisted mobile augmented reality,” in Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, 2023, pp. 1–16. [31] A. Narayanan, X. Zhang, R. Zhu, A. Hassan, S. Jin, X. Zhu, X. Zhang, D. Rybkin, Z. Yang, Z. M. Mao, F. Qian, and Z.-L. Zhang, “A variegated look at 5G in the wild: Performance, power, and QoE implications,” in Proceedings of the 2021 ACM SIGCOMM Conference. ACM, 2021, pp. 610–625. [Online]. Available: https://doi.org/10.1145/3452296.3472923 [32] X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi, C. Shi, Z. Chen, D. Arfeen, R. Abhyankar, and Z. Jia, “SpecInfer: Accelerating large language model serving with tree-based speculative inference and verification,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. ACM, 2024, pp. 932–949. [33] T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao, “Medusa: Simple LLM inference acceleration framework with multiple decoding heads,” in Proceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 235, 2024, pp. 5209–5235. [34] Z. Chen, A. May, R. Svirschevski, Y. Huang, M. Ryabinin, Z. Jia, and B. Chen, “Sequoia: Scalable, robust, and hardware-aware speculative decoding,” arXiv preprint arXiv:2402.12374, 2024. [35] P. Bhattacharjee, F. Tian, M. Zhong, G. Zhang, O. Simeone, and R. Tandon, “Conformal sparsification for bandwidth-efficient edge-cloud speculative decoding,” arXiv preprint arXiv:2510.09942, 2025. [36] G. Zhang, Y. Cai, G. Yu, P. Popovski, and O. Simeone, “Quantizesample-and-verify: LLM acceleration via adaptive edge-cloud speculative decoding,” arXiv preprint arXiv:2507.00605, 2025. [37] A. Agrawal, N. Kedia, A. Panwar, J. Mohan, N. Kwatra, B. S. Gulavani, A. Tumanov, and R. Ramjee, “Taming throughput-latency tradeoff in LLM inference with Sarathi-Serve,” in 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, 2024, pp. 117–134.

Related documents

• PriDyG: Privacy-preserving Dynamic Graph Inference with LLM-GNN Collaboration#429283
• Enhanced Feedback Mechanisms for Resource-Efficient Incremental Redundancy#1061750
• Analysis of Federated Aggregation under Model Poisoning and Backdoor Attacks: A Reconstructed Cross-Dataset and Cross-Architecture Benchmark#447890
• Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries#987757
• MineDraft: A Framework for Batch Parallel Speculative Decoding#821432