1
Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge
arXiv:2606.04581v1 [cs.DC] 3 Jun 2026
Haotian Zheng, Zhanwei Wang, Mingyao Cui, Chang Cai, Hongyang Du, and Kaibin Huang, Fellow, IEEE
Abstract—Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs). In this work, we propose its distributed deployment to enable cooperative token generation in a multiuser edge system; its advantage is to effectively balance computational loads between resource-constrained devices and servers. The resulting architecture, termed Multi-access SPIN (Multi-SPIN), utilizes ondevice small language models (SLMs) to generate and upload candidate token drafts, alongside an edge server that operates the LLM to verify these drafts in parallel batches. Given the severe heterogeneity in users’ computation and communication (C2 ) capabilities, the draft length emerges as a critical control variable that influences node-level computation loads and multi-access latency, thereby governing the sum token goodput. Consequently, considering frequency-division multiple access, we investigate the problem of so called multi-access draft control, referring to joint optimization of draft-length control and bandwidth allocation to maximize the sum token goodput. We examine two cases: (1) homogeneous draft lengths across users to facilitate serverside batching, and (2) heterogeneous draft lengths to introduce a new dimension for goodput enhancement. By developing decomposition methods, we systematically reduce these complex optimizations into tractable sub-problems, which allow efficient draft control algorithms to be derived in closed-form. Our analysis shows that the optimal bandwidth allocation compensates users with weaker C2 capabilities in the homogeneous case due to the batching synchronization requirements, whereas its heterogeneous-case counterpart rewards users with higher acceptance rates by relaxing such requirements. Experiments utilizing Llama-2 and Qwen3.5 model pairs across tasks demonstrate that Multi-SPIN improves goodput by up to 88% compared to heterogeneity-agnostic baselines. Index Terms—Edge networks, speculative inference, multiple access, draft length control, bandwidth allocation
I. I NTRODUCTION Deploying large language models (LLMs) at the sixthgeneration (6G) network edge enables highly efficient, lowlatency generative AI services for mobile devices [1], [2]. However, resource-constrained edge servers often struggle with the heavy computational demands of LLMs containing tens to hundreds of billions of parameters [3]. Speculative inference (SPIN) mitigates this by leveraging distributed ondevice computing: devices use lightweight small language models (SLMs) to generate candidate token sequences, or drafts, which are then uploaded to the server for parallel LLM verification [4]. As such single-pass verification is significantly less complex than sequential autoregressive generation, SPIN H. Zheng, Z. Wang, M. Cui, C. Cai, H. Du, and K. Huang are with the Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China (Email: {htzheng, zhanweiw, mycui, changcai, duhy, huangkb}@eee.hku.hk). Corresponding authors: K. Huang; Z. Wang.
drastically reduces the server’s computational burden without sacrificing accuracy [5]. Nevertheless, scaling SPIN for multiuser systems introduces two key challenges. The first is a local computation-and-communication (C2 ) bottleneck driven by on-device draft generation and multi-access uplink transmissions. The second is a global computation bottleneck resulting from the simultaneous verification of multiple drafts at the server. To overcome these limitations, we propose a multi-access SPIN (Multi-SPIN) framework that utilizes draft batching for efficient parallel server-side verification. As a key component of this framework, we present novel algorithms to jointly optimize draft lengths and uplink bandwidth allocation, with the objective of maximizing the token goodput. The advantages of SPIN can be clearer when compared with conventional edge LLMs. In standard server-hosted LLM generation, each token requires a computationally expensive forward pass and extensive memory access to retrieve the “KV cache”, referring to historical intermediate data essential for autoregressive generation [3]. Repeating this sequential process over a long token sequence and for multiple users severely strains resource-constrained edge servers [3], [6]. In contrast, the Multi-SPIN framework offloads much of this workload to many increasingly powerful mobile processors by using local SLMs to generate draft token sequences [7], [8]. Although these drafts naturally deviate from the LLM’s target distribution [9], [10], the edge server resolves this through a highly efficient verification process. By computing the target token probabilities of multiple user drafts simultaneously in a single batched forward pass, the server accepts valid tokens and calibrates any discrepancies to guarantee LLM-level quality before returning the finalized tokens to the devices. The current mainstream approach for alleviating computational load on edge servers is so called split inference [11]. It offloads early layers of a global model (i.e., deep convolutional neural network) to an edge device while processing the remainder at the server. The split point can be dynamically adjusted to balance computing loads and control wireless transmission overhead. While effective for sensing and object recognition, this architecture incurs severe latency for autoregressive token generation in transformer-based LLMs as explained later. Existing split inference research focuses on reducing communication overhead via split-point optimization [12] and feature compression and transmission [13]–[15]. Recent applications to edge LLMs have introduced practical techniques such as memory-aware layer placement [6], overthe-air tensor parallelism [16], and activation-based routing of expert sub-models [17], [18]. However, these methods overlook the fundamental latency bottleneck of autoregressive
2
generation. In a split framework, a device cannot process a new token until the server generates and transmits the preceding one. This sequential dependency requires numerous communication rounds to generate a full sequence; each token still necessitates a computationally inefficient full forward pass through the global model. In contrast, Multi-SPIN achieves higher communication efficiency by having the device upload an entire drafted sequence to the edge server in a single shot for LLM verification. Furthermore, by verifying multiuser drafts in parallel within a single forward pass, server computation becomes inherently more scalable and efficient than the sequential processing mandated by split inference. The advantages of SPIN have motivated recent studies on its edge deployment; however, the early work remains largely confined to simple point-to-point, single-user systems. Existing research primarily focuses on mitigating uplink overhead via customized compression strategies, such as importanceaware logit truncation [19], draft-token selection [20], and adaptive quantization [21]. Other studies attempt to boost the LLM token-acceptance rate by uploading multiple drafts, albeit at the expense of higher local C2 overhead [22]. In view of prior work, the single-user approaches fail to address the complexities inherent to typical multiuser edge systems [23]. The primary challenge lies in the severe heterogeneity across devices (i.e., varying C2 capabilities and prompt characteristics), which naturally leads to diverse token-acceptance rates [24]. This heterogeneity necessitates the joint optimization of individual draft lengths to maximize the sum token goodput, defined as the aggregate expected number of accepted tokens per second across all devices. Such optimization must balance a fundamental tradeoff: while longer drafts benefit devices with high acceptance rates, they inherently prolong local computation and transmission latency, and vice versa. Furthermore, although traditional radio resource management can mitigate multi-access latency, it yields sub-optimal endto-end (E2E) performance because it is decoupled from draft control, treating communication overhead independently of computational delay. The draft length is a unique control variable in Multi-SPIN that directly influences the sum token goodput. Specifically, it determines the number of tokens each user generates and transmits, thereby driving the local computation loads, the uplink communication latency, and the server’s verification overhead. For a local SLM with a given token acceptance rate, an excessively long draft incurs unnecessary C2 latency, as the server will reject many of the drafted tokens. On the other hand, an overly short draft diminishes server efficiency, as the fixed computational overhead of a batched forward pass is amortized over too few verified tokens. Because token acceptance rates and wireless channel conditions vary across devices, the optimal control of multiuser draft lengths becomes important for maximizing the sum token (generation) goodput. Addressing the issue for a multiuser system gives rise to a novel problem, termed multi-access draft control, referring to joint control of draft lengths and bandwidth allocation to maximize the sum token goodput. Formulating and solving this problem forms the core contribution of this work. The work is arguably the first study of distributed SPIN
deployment in a multiuser system by proposing the MultiSPIN framework. As summarized below, it comprises a set of algorithms to solve the optimal multi-access draft control problem under different system settings. The key contributions and findings of this work are summarized as follows. Multi-access Draft Control: To facilitate batched processing at the server, we assume uniform draft lengths across all users under centralized server control. The sum token goodput maximization problem is then optimally decoupled into two sub-problems. The first sub-problem focuses on optimal draft-length control under a given bandwidth allocation. Here, we identify a fundamental content–latency tradeoff: as the draft length increases, the number of accepted tokens gradually saturates, whereas the E2E latency grows linearly. This renders the sum token goodput a unimodal function of the draft length, allowing the unique optimum to be derived in closed form. The optimal draft length is shown to increase monotonically with both the acceptance rate and the verification latency, thereby amortizing the heavy verification overhead over more accepted tokens. The second subproblem addresses bandwidth allocation to minimize the multi-access C2 latency, where the derived optimal strategy allocates larger bandwidth to devices with weaker C2 capabilities due to the synchronization requirement inherent to server-side batching. • Multi-access Draft Control with Heterogeneous Lengths: Next, we permit heterogeneous draft lengths across users, which opens a new dimension for goodput enhancement. To maintain compatibility with server-side batched processing, draft-length variations are accommodated via zero-padding. Although this flexibility deepens the coupling between draft control and bandwidth allocation, the problem can still be optimally decomposed for a low-complexity solution. This is achieved by using the multi-access latency as an intermediate variable, which yields two nested sub-problems: draft-length control for a given multi-access latency, and bandwidth allocation to minimize this latency. Solving both in closed form reduces the original high-dimensional optimization to a highly efficient two-dimensional search. The resulting optimal strategy assigns longer drafts and greater bandwidth to devices with higher acceptance rates—in contrast to the bandwidth allocation derived for the homogeneous case. • Experiment Results: Experiments on Llama-2 and Qwen3.5 model pairs confirm the derived trade-off and the substantial goodput gain of Multi-SPIN over heterogeneity-agnostic baselines. The heterogeneouslength scheme consistently surpasses its homogeneous counterpart, with the gap enlarging as the network scales to more devices. •
The remainder of this paper is organized as follows. Sec. II presents the system model and performance metrics. Sec. III introduces the Multi-SPIN protocol and problem formulation. Sec. IV studies the multi-access draft control problem under uniform draft lengths. Sec. V extends the analysis to the general regime with heterogeneous draft lengths. Sec. VI
3
provides the experimental results, and Sec. VII concludes the paper.
∼
⋯
Resample
⋯
or
II. M ODELS AND M ETRICS LLM
The proposed Multi-SPIN framework considers a single-cell edge network in which an edge server, which hosts an LLM, provides generative AI services to K distributed devices. Each device executes a local SLM to generate and upload a draft (i.e., token sequence) to the server, where the server-side LLM collects and verifies the multiuser drafts in a batch. The associated models and performance metrics are described in the sub-sections.
in parallel
∼
∼
⋯
Sample
⋯ SLM
SLM
SLM
A. Speculative-inference Operations and Models Each epoch of the token generation process in SPIN spans multiple rounds, each comprising the following two sequential operations: local drafting and server-side verification. 1) Local Drafting: Consider an arbitrary SPIN round for device k ∈ {1, . . . , K}. In the first step, the device generates a token draft using its on-device SLM, parameterized by ΦS . This step follows the classical autoregressive inference paradigm, where the draft is generated token by token, as illustrated in the lower part of Fig. 1. The associated notation is introduced as follows. Let Mk and Lk denote the numbers of tokens in the prefix sequence and the draft of device k, respectively. Let X̂k ≜ (x̂k,1 , . . . , x̂k,Lk ) denote the draft generated by device k conditioned on the prefix sequence Xk = (Xkpt , xk,1 , . . . , xk,Mk ), where Xkpt denotes the token sequence corresponding to the input prompt of device k. Each token takes a value from the vocabulary V, whose cardinality is V = |V|. For the ℓ-th token, denoted by x̂k,ℓ , the SLM computes, based on the current prefix sequence, a probability vector that characterizes the conditional probability mass function (PMF) of x̂k,ℓ over the vocabulary: S T S S pSk,ℓ ≜ Pk,ℓ (1), . . . , Pk,ℓ (v), . . . , Pk,ℓ (V ) (1) = fΦS (Xk , x̂k,1 , . . . , x̂k,ℓ−1 ) . S Here, Pk,ℓ (v) = Pr(x̂k,ℓ = v | Xk , x̂k,1 , . . . , x̂k,ℓ−1 ) denotes the probability assigned by the SLM to vocabulary token v ∈ V, and fΦS denotes the inference mapping of the SLM. Each token realization is obtained by sampling x̂k,ℓ ∼ pSk,ℓ , and the draft X̂k is formed by recursively computing in (1) and sampling across positions ℓ = 1, . . . , Lk . Since this sequential process requires one SLM forward pass per token, the total local drafting latency is given by
Tkdr = Lk TkS ,
(2)
where TkS is the per-token inference latency of device k. The generated draft and associated probability vectors are then forwarded to the edge server for verification. 2) Server-side Verification: The edge server uses the LLM, parameterized by ΦL , to verify the draft X̂k generated by device k, as illustrated in the upper part of Fig. 1. Unlike autoregressive decoding, which requires one sequential forward pass per token, the server-side LLM verifies the entire
Fig. 1: Illustration of speculative-inference operations and models.
draft in a single parallel forward pass. Specifically, for each drafted position ℓ, the LLM computes a probability vector that characterizes the conditional PMF over the vocabulary, given by L T L L pLk,ℓ ≜ Pk,ℓ (1), . . . , Pk,ℓ (v), . . . , Pk,ℓ (V ) (3) = fΦL (Xk , x̂k,1 , . . . , x̂k,ℓ−1 ) , L where Pk,ℓ (v) = Pr(x̂k,ℓ = v | Xk , x̂k,1 , . . . , x̂k,ℓ−1 ) denotes the probability assigned by the LLM to token v ∈ V, and fΦL denotes the inference mapping of the LLM. Accordingly, the LLM-assigned probability of the SLM-sampled token x̂k,ℓ is L denoted by Pk,ℓ (x̂k,ℓ ). Based on the probability vectors of the SLM and LLM, the server performs token verification as follows. For the draft token x̂k,ℓ , the verification outcome is modeled as a Bernoulli random variable Ak,ℓ ∼ Bernoulli(βk,ℓ ), where Ak,ℓ = 1 and Ak,ℓ = 0 indicate acceptance and rejection, respectively, and βk,ℓ denotes the acceptance probability, specified as ) ( L Pk,ℓ (x̂k,ℓ ) βk,ℓ ≜ min 1, S . (4) Pk,ℓ (x̂k,ℓ ) S L Here, Pk,ℓ (x̂k,ℓ ) and Pk,ℓ (x̂k,ℓ ) are the probabilities assigned in (1) and (3), respectively, to the drafted token x̂k,ℓ . The server examines the outcomes sequentially and identifies the first rejected position ℓrej = min{ℓ : Ak,ℓ = k 0, ℓ = 1, . . . , Lk }. All tokens with ℓ < ℓrej k are accepted, and the rejected position is replaced by a calibrated token sampled from a calibrated distribution [5]. The resulting xcal k,ℓrej k
output token sequence after LLM verification, denoted by X̃k , is provided by ( (x̂k,1 , . . . , x̂k,ℓrej −1 , xcal ), ℓrej k ≤ Lk , k,ℓrej k k X̃k = (5) (x̂k,1 , . . . , x̂k,Lk , x̃k,Lk +1 ), otherwise, where x̃k,Lk +1 ∼ pLk,Lk +1 is one additional token sampled from the LLM distribution when all Lk drafted tokens are accepted. Appending X̃k to the current prefix sequence yields the prefix for the next round: Xk ← (Xk , X̃k ).
(6)
4
On one hand, tokens generated by the above draftingverification cooperation match the accuracy of direct LLM inference [10]. On the other hand, SPIN enables the LLM to evaluate all drafted tokens in parallel, approximately reducing the generation complexity from O(L · ΦL ) to O( ΦL ) [25], [26]. To reduce the verification overhead of serving K devices, we consider adopting batching at the server, packing the received drafts into a single batch tensor and processing them in one forward pass of the LLM [27]. Note that in the case of heterogeneous draft lengths, zero-padding is needed to enable batching, where shorter drafts are padded to a uniform length before being stacked into the batch tensor. Based on empirical observations on batched LLM inference [28], the batched verification latency is modeled as T ver (K) = T fix + KT lin ,
(7)
where T fix captures fixed overhead such as GPU kernel launches and T lin is the incremental latency per additional draft in the batch.
B. Multiple Access Model In the proposed Multi-SPIN system, orthogonal multiple access is adopted for uploading drafts from K distributed devices to the edge server. Specifically, the system employs orthogonal frequency-division multiple access (OFDMA) over a broadband uplink channel. For simplicity, the number of subcarriers is assumed sufficiently large such that bandwidth allocation can be approximated as continuous [29]. For device k, let pk and Bk denote the transmit power and allocated bandwidth, respectively, and let Hk denote the channel power gain over the assigned uplink resource. The channel is assumed to remain constant within one round and to be known at the server [30]. The resulting uplink transmission rate is given by pk Hk ≜ Bk rk , (8) Rk = Bk log2 1 + N0 where N0 denotes the Gaussian noise power and rk is defined to be the uplink spectrum efficiency. Consider uploading the token draft of device k with draft length Lk . To reduce the uplink overhead, each device uploads, for every drafted token, only the |V̂| retained probability values and their corresponding vocabulary indices, rather than the full-dimensional probability vector. Then, each probability value is quantized using QB bits, where QB is chosen sufficiently large such that the resulting quantization error is negligible (e.g., QB = 16). Since each vocabulary index requires ⌈log2 V ⌉ bits, the uplink transmission latency is given by Qtok Lk , (9) Tktx = Bk rk where Qtok ≜ |V̂| (QB + ⌈log2 V ⌉) is the per-token communication overhead and V is the vocabulary cardinality. The downlink transmission latency is neglected due to the ample transmit power of the edge server.
C. Definition of Sum Token Goodput To evaluate the performance of the multi-access SPIN system, we consider the sum token goodput, defined as the aggregate expected number of accepted tokens generated per second. The exact mathematical definition is given below. 1) Expected Number of Accepted Tokens: Consider an arbitrary SPIN round for device k, where the SLM generates a token draft X̂k of length Lk . Owing to the randomness of server-side verification, the number of accepted tokens, denoted by Nk , is a random variable. To enable tractable analysis, the token acceptance events are widely considered as i.i.d. across drafted positions [5]. Accordingly, the acceptance probability can be obtained as the mean verification outcome αk ≜ EX pt ,x̂k,ℓ [Ak,ℓ ] = EX pt ,x̂k,ℓ [βk,ℓ ] , k
k
(10)
where the expectation is taken over prompts Xkpt ∈ Dk and the drafted tokens generated by the SLM. The quantity αk , termed the acceptance rate, can be statistically estimated for a given SLM-LLM pair and prompt categories [21], [31]. Under this approximation, the PMF of Nk is given by ( αkℓ−1 (1 − αk ), ℓ = 1, 2, . . . , Lk , Pr(Nk = ℓ) = (11) αkLk , ℓ = Lk + 1. Accordingly, for a fixed draft length Lk of device k, the expected number of accepted tokens becomes [31] E[Nk | Lk ] =
Lk X ℓ=0
αkℓ =
1 − αkLk +1 . 1 − αk
(12)
2) Sum Token Goodput: In Multi-SPIN, the sum token goodput is defined as the expected number of accepted tokens aggregated over all K devices, divided by the E2E latency of one Multi-SPIN round. Denote T e2e (B, L) as the E2E execution latency of completing one SPIN round for all K devices, which will be quantified in the next section. Then, the sum goodput of Multi-SPIN is given by PK E[Nk | Lk ] τ (B, L) = k=1e2e , (13) T (B, L) K where B = {Bk }K k=1 and L = {Lk }k=1 denote the sets of bandwidth allocations and draft lengths of the devices.
III. P ROTOCOL AND P ROBLEM F ORMULATION In this section, we present the Multi-SPIN protocol for coordinating parallel SPIN tasks among distributed devices. Then, the problem of optimal multi-access draft control is formulated. A. Multi-SPIN Protocol The steps of the Multi-SPIN protocol are illustrated in Fig. 2 and elaborated as follows. 1) System Configuration: At the beginning of each round, each device reports its task profile (including the acceptance rate) and computation speed to the server. The server then measures the uplink channel conditions and determines the draft lengths and bandwidth allocations
5
Edge Devices
1
Edge Server
3 System Configuration
1
×
✓
2
Distributed Drafting
×
3
Multiuser Uploading
4 4
2 5
Multiuser Verification
Result Feedback
5
(a) Multi-SPIN system.
(b) Multi-SPIN operations and protocol.
Fig. 2: Overview of the Multi-SPIN framework.
by solving the multi-access draft control problem in Sec. III-B. The resulting configurations are delivered to all devices. 2) Distributed Drafting: Each device independently generates a draft of Lk tokens using its local SLM, incurring a local computation latency in (2). 3) Multiuser Draft Uploading: Each device transmits the predicted PMF vectors in (1) together with the drafted token indices to the edge server via its assigned OFDMA channel, resulting in an uplink transmission latency in (9). 4) Multiuser Draft Verification: Upon receiving all K drafts, the edge server performs batched verification via a single LLM forward pass. For each draft, tokens are accepted or rejected via (4), and the rejected one is replaced with a calibrated token. 5) Result Feedback: The server returns the verified token sequence to each device, which updates its prefix via (6). If the current request is completed, the device proceeds to its next request. The system then loops back to Step 1 for the next round.
B. Goodput Maximization Problem This subsection first characterizes the explicit expression of the Multi-SPIN goodput in (13) and then formulates the corresponding optimization problem to maximize it. We consider a uniform-length regime where all devices adopt a uniform draft length, i.e., Lk = L for all k, and a common acceptance rate, i.e., αk = α for all k. This setting is practically reasonable when devices have comparable C2 capabilities and serve prompts from the same task family, as it reduces control complexity and enables synchronized batch verification at the server [5]. This problem is extended into heterogeneous scenarios for device-specific acceptance rate and draft length control in Sec. V. Consider an arbitrary round. Under the uniform-length regime, the expected number of accepted tokens in (12) becomes identical across devices, so the aggregate over all
K devices is given by K X
E[Nk | Lk = L] = K ·
k=1
1 − αL+1 . 1−α
(14)
Local drafting and uploading are performed in parallel across devices. Combining (2) and (9), we define the multi-access latency T ma (B, L) as the time required for all K devices to generate and upload their drafts over the shared uplink, which is dominated by the slowest device and given by Qtok T ma (B, L) = L max TkS + . (15) k Bk rk The maximum arises because the server must wait for the slowest device to complete draft generation and token uploading. As the multi-access and server-side phases execute sequentially within each round, the E2E round latency is T e2e (B, L) = T ma (B, L) + T ver ,
(16)
where T ver denotes the batched verification latency in (7). Substituting (14) and (16) into (13) yields the sum goodput of the Multi-SPIN system, given by K 1 − αL+1 i , (17) τ (B, L) = h L T̂ ma (B) + T ver (1 − α) where T̂ ma (B) ≜ maxk TkS + Qtok /(Bk rk ) denotes the pertoken multi-access latency. Maximizing this goodput requires jointly controlling the draft length and the bandwidth allocation, giving rise to the multi-access draft control problem, formulated as (P1)
max τ (B, L) B, L
s.t.
L ∈ Z+ , Bk > 0, ∀ k, P k Bk ≤ B,
(18)
where B = {Bk }K k=1 collects the per-device bandwidth allocations and B denotes the total system bandwidth budget. In problem (P1), the integer variable L and the continuous vari-
6
In this section, we assume uniform draft lengths across all users to facilitate batched processing at the server. Under this assumption, we design an optimal multi-access draft control algorithm by solving the goodput maximization problem (P1). To design an efficient algorithm, we adopt a decomposition approach that optimally separates the original problem into two distinct sub-problems, both of which are subsequently solved in closed form. A. Optimal Problem Decomposition This subsection decouples problem (P1) by exploiting the structure of the goodput expression in (17). A key observation is that the per-token multi-access latency T̂ ma (B) is the only term through which B affects the goodput. Specifically, the numerator of (17) depends only on L, whereas the denominator depends on B only through T̂ ma (B). For any fixed L > 0, τ (B, L) is monotonically decreasing in T̂ ma . Therefore, maximizing the goodput with respect to B is equivalent to minimizing T̂ ma . This observation leads to the following decomposition of problem (P1). 1) Bandwidth Allocation: The bandwidth allocation problem seeks to minimize the per-token multi-access latency. Since the constant factor L in the multi-access latency T ma (B, L) = L T̂ ma (B) does not affect the minimizer over B, the subproblem is formulated as a min-max optimization problem, given by (P1.1)
min T̂ ma (B) B X s.t. Bk ≤ B, k
Bk > 0, ∀ k. Let B ⋆ denote the optimal solution of (P1.1) and ϑ⋆ ≜ T̂ ma (B ⋆ ) the corresponding minimum per-token multi-access latency. 2) Draft-Length Control: With ϑ⋆ determined, all dependence on B is eliminated, and the goodput in (17) reduces to a univariate function of L: K 1 − αL+1 . (19) τ (L) = (L ϑ⋆ + T ver )(1 − α) The multi-access draft control problem reduces to the draftlength control problem, given by (P1.2)
max
L∈Z+
τ (L).
The two decoupled Problems (P1.1) and (P1.2) are solved in closed form in Sections IV-B and IV-C, respectively. B. Optimal Bandwidth Allocation This subsection solves the bandwidth allocation problem (P1.1) obtained from the decoupling in Sec. IV-A.
Empirical Goodput Theoretical Goodput Derived Optimum
130
Token Goodput (#Tokens/s)
IV. M ULTI - ACCESS D RAFT C ONTROL
140
Token Goodput (#Tokens/s)
ables {Bk } are coupled through the straggler-limited latency in (15). The next section exploits this structure to decouple bandwidth allocation from draft-length control and derive a closed-form optimal solution.
120 110 100 90 80
200 180 160 140 120 100
Empirical Goodput Theoretical Goodput Derived Optimum
80
70 0
3
6
9
12
15
Draft Length
18
21
24
0
(a) Llama-2 Pair
3
6
9
12
15
Draft Length
18
21
24
(b) Qwen3.5 Pair
Fig. 3: Empirical and theoretical sum token goodput versus draft length for Llama-2 and Qwen3.5 models on the GSM8K dataset.
Problem (P1.1) minimizes the maximum of K terms TkS + Qtok /(Bk rk ), each depending on a single bandwidth variable Bk . Owing to this separable structure, the unique optimum is attained when all K terms are equalized [32], as stated in Lemma 1. Lemma 1 (Optimal Bandwidth Allocation). Problem (P1.1) admits a unique optimal bandwidth allocation, given by Bk⋆ =
Qtok , rk ϑ⋆ − TkS
(20)
where ϑ⋆ is determined as the unique root of K X
Qtok = B, r ϑ⋆ − TkS k=1 k
ϑ⋆ > max TkS . k
(21)
The allocation (20) satisfies TkS + Qtok /(Bk⋆ rk ) = ϑ⋆ for all k, meaning that the optimal bandwidth allocation equalizes the per-token multi-access latency across all devices, regardless of their heterogeneous computation times {TkS } and channel conditions {rk }. Moreover, ϑ⋆ decreases monotonically budget B, as verified Pwith the total bandwidth −1 K Qtok dϑ⋆ by dB = < 0. It confirms that a k=1 rk (ϑ⋆ −TkS )2 larger bandwidth budget reduces the per-token communication latency. C. Optimal Draft Length Building on Lemma 1, this subsection determines the optimal draft length that maximizes the sum goodput. We solve the draft-length control problem (P1.2) by first establishing the unimodality of the goodput and then deriving a closed-form optimum via continuous relaxation. Since problem (P1.2) is an integer program, we relax L ∈ Z+ to a continuous variable L̃ > 0 to enable closed-form analysis. The relaxed goodput function is given by K 1 − αL̃+1 , L̃ > 0. (22) τ̃ (L̃) = (L̃ ϑ⋆ + T ver )(1 − α) Then, the maximization of τ̃ (L̃) in (22) is formulated as (P1.3)
max τ̃ (L̃). L̃>0
As illustrated in Fig. 3, the sum token goodput exhibits the mentioned content–latency tradeoff with respect to the draft
5 4 3 2 100
200
300
6 5 4 3 2 20
400
Verification Overhead (ms)
(a) Effect of verification overhead T ver .
7
Optimal Draft Length
6
Optimal Draft Length
Optimal Draft Length
7
40
60
Per-token Latency (ms)
80
14 12 10 8 6 4 0.6
(b) Effect of per-token multi-access latency ϑ⋆ .
0.7
0.8
Acceptance Rate
0.9
(c) Effect of acceptance rate α.
Fig. 4: Optimal uniform draft length under varying system parameters for the Llama-2 Pair. In each subfigure, one parameter is varied while the other parameters are fixed.
length L. On the one hand, a larger L allows more drafted tokens to be verified in each round and thus increases the expected output. However, the marginal gain from drafting more tokens gradually diminishes, since a later drafted token is accepted only if all earlier drafted tokens are also accepted. On the other hand, each additional drafted token incurs one more unit of per-token multi-access latency ϑ⋆ , so the per-round E2E latency grows linearly with L under limited C2 resources. As a result, the numerator of the goodput increases with diminishing returns, whereas the denominator grows linearly in L, yielding a unimodal goodput function with a unique optimal draft length, as formalized in Theorem 1. Theorem 1 (Unimodality and Optimal Draft Length). The continuous goodput τ̃ (L̃) in (22) is strictly unimodal for L̃ > ver 0, admitting a unique global maximum. If Tϑ⋆ > α|1−α ln α| , the ⋆ optimal integer draft length L is obtained by L⋆ = arg
max
τ (L),
improving the acceptance rate via better SLM–LLM alignment in Multi-SPIN.
V. M ULTI - ACCESS D RAFT C ONTROL WITH H ETEROGENEOUS L ENGTHS In this section, we relax the assumption of uniform draft length made in the preceding section and explore the case permitting heterogeneous draft lengths across users. This introduces a new dimension for goodput enhancement but complicates the multi-access draft length control. To design the optimal control algorithm, the difficulty is overcome by adopting an alternative optimal decomposition approach, which reduces the high-dimensional joint optimization to a low-complexity two-dimensional search.
(23)
L∈{⌊L̃⋆ ⌋, ⌈L̃⋆ ⌉}
where the continuous optimum L̃⋆ is given in closed form by T ver ln −W−1 −α ϑ⋆ −1/e L̃⋆ = − − 1, (24) ln α with W−1 (·) denoting the lower branch of the Lambert W function and ϑ⋆ obtained from (21). Proof. (See Appendix A) To gain more insights into the optimal solution, we analyze the monotonicity of the optimal draft length L⋆ with respect to key system parameters: the verification overhead T ver , the pertoken multi-access latency ϑ⋆ , and the acceptance rate α. The derivation is provided in Appendix B. Fig. 4 illustrates how the optimal draft length varies with the key system parameters and motivates the design insights summarized in Remark 1. Remark 1 (Effects of System Parameters). The optimal draft length increases with the verification overhead T ver and decreases with the per-token multi-access latency ϑ⋆ , reflecting the need to amortize a costly verification step over more draft tokens when verifying is expensive relative to drafting. Moreover, the optimal draft length with α, and √ increases as α → 1, L̃⋆ grows as O 1/ − ln α , dominating the dependence on T ver and ϑ⋆ . This emphasizes the importance of
A. Problem Formulation and Decomposition This subsection formulates the multi-access draft control problem with heterogeneous draft lengths and explains why the previous decoupling strategy is no longer applicable. Under heterogeneous draft lengths L = {Lk }K k=1 and device-specific acceptance rates {αk }, the expected number of accepted tokens varies across devices. The aggregate expected number of accepted tokens over one round is given by K X k=1
E[Nk | Lk ] =
K X 1 − αLk +1 k
k=1
1 − αk
.
(25)
Since the draft length differs across devices, the draft lengths cannot be extracted from the multi-access latency. Accordingly, the multi-access latency in (15) generalizes to Qtok ma S T (B, L) = max Lk Tk + . (26) k Bk rk The corresponding E2E latency of one Multi-SPIN round is then given by T e2e (B, L) = T ma (B, L) + T ver . Substituting the above expressions into (13) yields the sum goodput in the heterogeneous draft-length setting, given by P E[Nk | Lk ] τ (B, L) = mak . (27) T (B, L) + T ver
8
Extending the multi-access draft control problem to heterogeneous draft lengths yields its general form, formulated as (P2)
max τ (B, L) B, L
s.t.
Bk (L) =
Lk ∈ Z+ , ∀ k, Bk > 0, ∀ k, P k Bk ≤ B.
(28)
Problem (P2) is a mixed-integer nonlinear program (MINLP) for which a brute-force solution approach is NPhard. The decomposition method for problem (P1) is no longer optimal since the heterogeneous draft lengths in (26) couple draft lengths and bandwidths inside the max operator, such that the bandwidth allocation can no longer be optimized independently. Nevertheless, an alternative method for optimal decomposition is possible as shown in Lemma 2. It is based on the observation that, in the objective in (26), only the multiaccess latency term, T ma (B, L), depends on the bandwidth allocation, B. Lemma 2 (Optimality of Decomposition). Since the constraints of problem (P2) on B and L are decoupled, the joint optimization decomposes without loss of optimality into the outer problem over the draft lengths, (P2.1) max τ B ⋆ (L), L L
s.t.
Lk ∈ Z+ , ∀ k,
(29)
and the inner problem over the bandwidth allocation, (P2.2)
Lemma 3 (Structure of Optimal Bandwidth Allocation). For any set of draft lengths L, the optimal bandwidth allocation that solves problem (P2.2) is given by
min T ma (B, L) B
s.t.
Bk > 0, ∀ k, P k Bk ≤ B,
(30)
where B ⋆ (L) denotes the optimal bandwidth allocation of the inner problem (P2.2). The two sub-problems are solved in turn: Section V-B solves the inner problem (P2.2) to obtain B ⋆ (L), and Section V-C substitutes it into the outer problem (P2.1) to determine the optimal draft lengths.
This subsection characterizes the structure of optimal bandwidth allocation for given draft lengths and uses it to eliminate the dependence on the multi-dimensional variable B in problem (P2). For a given set of draft lengths, the aggregate expected number of accepted tokens in (25) is fixed, so maximizing the goodput τ (B, L) over B reduces to minimizing the multi-access latency T ma (B, L) in (26). Solving the inner problem (P2.2), the optimal bandwidth allocation exhibits a latency-equalization structure: at optimality, all devices have an equalized multi-access latency, i.e., the minimum latency of problem (P2.2). Consequently, the optimal bandwidth allocated to each device is fully determined by this equalized latency and the draft lengths, as provided in Lemma 3.
∀k,
(31)
where φ is the equalized multi-access latency, determined as the unique root of K X
Qtok Lk = B, r φ − Lk TkS k=1 k
φ > max Lk TkS . k
(32)
Lemma 3 provides two structural insights for the subsequent draft length control. First, both φ and Bk (L) are monotonically increasing with respect to each individual draft length Lk , as proved in Appendix C. Hence, assigning a longer draft length to any device increases the equalized latency φ and requires more bandwidth for that device to prevent it from becoming the straggler. Second, this lemma shows that the original optimization over the 2K variables B and L can be equivalently reformulated as an optimization over only K + 1 variables: the equalized latency φ and the draft lengths L. This elimination of the bandwidth variables reduces problem (P2) from 2K to K + 1 dimensions, laying the foundation for the further reduction developed in the next subsection. C. Optimal Draft Control Building on Lemma 3, this subsection solves for the structure of optimal draft lengths under the reduced problem formulation. Substituting the bandwidth–draft-length relationship in (31) into the goodput eliminates the explicit dependence on B. Since Lemma 3 equalizes all multi-access latencies to φ, the E2E latency reduces to φ+T ver , and the goodput becomes P E[Nk | Lk ] , (33) τ (L) = k φ + T ver P where k E[Nk | Lk ] is the aggregate expected output in (25), and the equalized latency φ is implicitly determined by L through the bandwidth constraint (32). Consequently, the sum goodput τ depends only on L, and the outer problem (P2.1) reduces to (P2.1a) below, (P2.1a)
B. Optimal Bandwidth Allocation
Qtok Lk , rk φ − Lk TkS
max τ (L) L
s.t. Lk ∈ Z+ , ∀k, K X
Qtok Lk = B, r (φ − Lk TkS ) k=1 k φ 0 < Lk < S , ∀k, Tk
(34a) (34b) (34c)
where (34b) encodes the equalized-latency condition from Lemma 3, and (34c) ensures positive draft lengths and bandwidths. We next analyze the problem (P2.1a) and relax it to enable a tractable solution. Problem (P2.1a) is a MINLP whose intractability stems from two sources: the integer constraint on Lk and the nonlinear equality constraint (34b), which implicitly couples all draft lengths through the shared variable φ. To obtain a tractable reformulation, we apply two relaxations:
9
(i) relaxing the integer draft lengths to continuous variables L̃k > 0, and (ii) relaxing the equality in (34b) to the inequality K X
Qtok L̃k ≤ B. r (φ − L̃k TkS ) k=1 k
(35)
The relaxations do not alter the optimal solution, as proved in Appendix D. The resulting relaxed problem is given by (P2.1b)
max τ (L̃) L̃
s.t. L̃k > 0, ∀k, (34c) & (35). We now derive the structure of optimal draft lengths under the relaxed problem. For any fixed φ, problem (P2.1b) maximizes a strictly concave objective over a convex feasible set, so the Karush–Kuhn–Tucker (KKT) conditions are necessary and sufficient for global optimality [33]. Solving these conditions yields a closed-form structure of the optimal draft lengths, as stated in Proposition 1. Proposition 1 (Optimal Draft Length). For a given equalized latency φ and KKT multiplier λ > 0, the optimal integer draft length for device k is obtained by (36) Lk (φ, λ) = round L̃k (φ, λ) , where the continuous optimum is given by − φ s 2T S k α φ 2 λ Qtok φ | ln αk | L̃k (φ, λ) = S + . W0 k S ln αk rk αk (1 − αk )−1 Tk 2Tk (37) Here, W0 (·) denotes the principal branch of the Lambert W function. The two scalar variables φ and λ are uniquely determined by the KKT stationarity condition and the active bandwidth constraint in (32). Proof. (See Appendix E.) Proposition 1 characterizes the heterogeneous draft lengths through the device-specific parameters (αk , TkS , rk ) and the two shared scalars (φ, λ). More importantly, when combined with the structure of optimal bandwidth in Lemma 3, it reveals a qualitative shift in the bandwidth allocation principle, as summarized in Remark 2. Remark 2 (Bandwidth Allocation under Heterogeneous Draft Lengths). As proved in Appendix F, the optimal bandwidth Bk⋆ is strictly increasing in αk . In contrast to the uniform-length regime, where bandwidth compensates devices with weaker C2 capabilities (See Lemma 1), the optimal bandwidth under the heterogeneous-length regime favors devices with higher acceptance rates.
D. Joint Optimization Algorithm We now assemble the closed-form sub-problem solutions into a joint algorithm for problem (P2), summarized in Algorithm 1. In the input, we initialize the system parameters, device-specific parameters, and the search grids Φ and Λ
Algorithm 1 Multi-access Draft Control Algorithm Inputs: System parameters B, Qtok , T ver ; device parameters {TkS , rk , αk }K k=1 ; bounded search grids Φ ⊆ [φ, φ] and Λ ⊆ [λ, λ]. 1: Initialize τ ⋆ ← 0. 2: for each (φ, λ) ∈ Φ × Λ do 3: Compute draft lengths L = {Lk }K k=1 via (37) and (36). 4: Find the root φ̂ of (32); if none exists, continue. 5: Compute τ ← τ (L) via (33). 6: if τ > τ ⋆ then 7: Update (τ ⋆ , φ⋆ , L⋆ ) ← (τ, φ̂, L). 8: end if 9: end for 10: Compute B ⋆ via (31) using (φ⋆ , L⋆ ). Output: L⋆ and B ⋆ .
specified in Appendix G. For each candidate pair (φ, λ), the algorithm computes the draft lengths using Proposition 1 (Step 3), checks feasibility by evaluating the bandwidth constraint (32) under the equalized-latency structure (Step 4), and updates the current best solution whenever a higher goodput is attained (Steps 5–8). The optimal heterogeneous draft lengths and bandwidth allocations for problem (P2) are accordingly given by L⋆k = round L̃k (φ⋆ , λ⋆ ) , ∀ k, (38) where L̃k (·) is defined in (37), and Bk⋆ =
Qtok L⋆k , rk φ⋆ − L⋆k TkS
∀ k,
(39)
which is obtained from Lemma 3 (Step 10). Solving problem (P2) directly over its 2K variables incurs a complexity of O(LK max K), as the integer draft lengths span LK max combinations and each requires an O(K) bandwidth optimization. Nevertheless, the decomposition removes this exponential dependence. Specifically, combining Lemma 3 and Proposition 1 reduces the 2K variables to the two scalars (φ, λ), first from 2K to K + 1 by Lemma 3 and then to (φ, λ) by Proposition 1, with each feasible (φ, λ) mapping to a unique solution (L, B). A two-dimensional grid search over (φ, λ) therefore recovers the global optimum at a complexity of O(|Φ| |Λ| K). Our efficient algorithm replaces the exponential factor LK max with the grid size |Φ| |Λ|, scaling linearly with the number of devices. VI. E XPERIMENTAL R ESULTS A. Experiment Settings 1) Models and Tasks: We implement SPIN with two model pairs: (i) TinyLlama-1.1B as the on-device SLM paired with Llama-2-7B as the server-side LLM, and (ii) Qwen3.5-0.8B as the on-device SLM paired with Qwen3.5-27B as the serverside LLM [34], [35]. All models are initialized from pretrained checkpoints available on Hugging Face.1 The prompts 1 The adopted checkpoints are publicly available in the Hugging Face repositories TinyLlama/TinyLlama-1.1B, meta-llama/Llama-2-7b, Qwen/Qwen3.5-0.8B, and Qwen/Qwen3.5-27B.
Dataset MBPP+ GSM8K MT-Bench SQuAD
Llama2 1.1B & 7B 0.8582 ± 0.2472 0.7390 ± 0.3133 0.7393 ± 0.3127 0.7126 ± 0.3333
Qwen3.5 0.8B & 27B 0.8100 ± 0.3413 0.9340 ± 0.2089 0.9318 ± 0.2232 0.9650 ± 0.1538
200
Affine Fit Measured Latency
150 100 50 0
T = 22.68 ms T = 12.72 ms
1 2 3 4 5 6 7 8 9 10111213141516
Batch Size K
(a) Llama2-7B
400 300
Affine Fit Measured Latency
200 100 0
T = 75.07 ms T = 19.78 ms
1 2 3 4 5 6 7 8 9 10111213141516
Batch Size K
(b) Qwen3.5-27B
Fig. 5: Batched verification latency T ver as a function of the batch size K. The solid points represent the measured latency on the NVIDIA A100 GPU, and the dashed lines denote the fitted affine models for (a) Llama2-7B and (b) Qwen3.5-27B. 250
Goodput (#Tokens/s)
for each device are sampled i.i.d. from a mixture of datasets covering diverse task categories to capture heterogeneous workloads across devices. • Task-type 1 (Code Generation): We adopt the MBPP+ task [36], which augments the MBPP benchmark with extensive automatically generated test cases over 378 sanitized Python programming problems, enabling more rigorous evaluation of functional correctness. • Task-type 2 (Mathematical Reasoning): We adopt the GSM8K task [37], which consists of more than 8,500 grade-school-level math word problems and is designed to evaluate multi-step mathematical reasoning ability. • Task-type 3 (Multi-turn Dialogue): We adopt the MTBench task [38], which consists of 80 high-quality twoturn dialogue questions spanning various categories, including writing, role-play, extraction, reasoning, technology, engineering, and humanities and social sciences. • Task-type 4 (Reading Comprehension): We adopt the SQuAD task [39], a reading-comprehension benchmark of over 100,000 questions posed on Wikipedia passages, where each answer is a text span extracted from the corresponding passage. For each dataset d, an empirical acceptance rate α(d) is estimated by running SPIN on a set of sampled prompts and averaging the realized token-level acceptance probabilities in (4). The estimates are summarized in Table I. Device k is then assigned αk = α(dk ) based on its sampled dataset label dk , resulting in heterogeneous {αk } as defined in (10). 2) Computation Settings: For mobile devices, the per-token SLM inference latency T̄ S is measured on an Apple M4 Pro GPU. The inference latency of each device is independently drawn from [0.85, 1.15] × T̄ S , capturing device-side compute heterogeneity. The edge server runs on an NVIDIA A100 GPU. We formulate the batched verification latency as a function of batch size K and fit the affine model in (7). The fitted curve and the measured points are reported in Fig. 5. 3) Communication Settings: Unless otherwise specified, we consider a single-cell OFDMA uplink with K = 20 devices and a total uplink bandwidth of B = 10 MHz. The retained vocabulary size is fixed at |V̂| = 1024 for all experiments. The uplink channels follow independent block fading across MultiSPIN rounds, with device-specific average channel power gains sampled as H̄k ∼ U(0.2, 0.5) and per-round channel gains given by Hk = |hk |2 , where hk ∼ CN (0, H̄k ). The Gaussian noise power is set to N0 = 10−10 W. 4) Benchmarking Schemes: We compare Multi-SPIN against two protocol baselines and three optimization baselines, so as to isolate the architectural gain of the Multi-SPIN
Verification Latency (ms)
TABLE I: Empirical acceptance rate (mean ± std) across prompts for each dataset and SLM–LLM pair, computed by averaging realized acceptance probabilities in (4) over drafted tokens per prompt.
Verification Latency (ms)
10
200
P2P-SPIN Cen-SPIN Multi-SPIN
150 100 50 0
Llama-2 Pair
Qwen3.5 Pair
Fig. 6: Optimal sum token goodput comparison among P2P-SPIN, Cen-SPIN, and the proposed Multi-SPIN framework for the Llama-2 and Qwen3.5 model pairs.
protocol from the algorithmic gain of its joint bandwidthand-draft-length optimization. In the optimization comparison, the complete proposed scheme is denoted Hete-Multi-SPIN to emphasize that it assigns heterogeneous draft lengths across devices. For all reported results, the sum goodput is averaged over 50 prompts per device and 10 independent realizations of channel and computational conditions. In all exhaustive searches for draft-length optimization, the search space is restricted to L ∈ {1, . . . , Lmax } with Lmax = 25. We consider the following protocols as alternatives to the Multi-SPIN architecture. • P2P-SPIN: A point-to-point SPIN baseline with a single device, where the device drafts using its local SLM and uploads logits for server verification. The entire bandwidth budget B is allocated to the single uplink, and the draft length is selected by exhaustive search. • Cen-SPIN: A centralized SPIN baseline in which both drafting and verification are executed at the server for each prompt from end devices. The draft length is selected by exhaustive search. The following optimization baselines are constructed within the Multi-SPIN framework by imposing constraints on selected variables. • Fixed Draft Length and Bandwidth (Fixed BW&L): This baseline fixes the draft length at L̄ = 8 for all devices and allocates bandwidth uniformly, i.e., Bk = B/K for all k. • Multi-SPIN with Uniform Bandwidth Allocation (UniBW Multi-SPIN): This baseline obtains the per-device
11
240
200 180 160 140
Fixed BW&L Uni-BW Multi-SPIN Homo-Multi-SPIN Hete-Multi-SPIN
120 100 5.0
7.5
10.0
12.5
15.0
17.5
Bandwidth Budget (MHz)
Goodput (#Tokens/s)
Goodput (#Tokens/s)
220
220 200 180 160 120 100 5.0
20.0
Fixed BW&L Uni-BW Multi-SPIN Homo-Multi-SPIN Hete-Multi-SPIN
140
(a) Llama-2 Pair
7.5
10.0
12.5
15.0
17.5
Bandwidth Budget (MHz)
20.0
(b) Qwen3.5 Pair
Fig. 7: Comparison of sum token goodput across different control schemes under varying bandwidth budgets for both model pairs.
200 180
Fixed BW&L Uni-BW Multi-SPIN Homo-Multi-SPIN Hete-Multi-SPIN
240
Goodput (#Tokens/s)
Goodput (#Tokens/s)
220
160 140 120 100
220 200
Fixed BW&L Uni-BW Multi-SPIN Homo-Multi-SPIN Hete-Multi-SPIN
180 160 140 120
10
15
20
Number of devices
(a) Llama-2 Pair
10
15
20
Number of devices
(b) Qwen3.5 Pair
Fig. 8: Comparison of sum token goodput across different control schemes under varying numbers of devices for both model pairs.
heterogeneous draft lengths by solving problem (P2.1a) with the same relaxation-and-rounding procedure as in Multi-SPIN, while the bandwidth is uniformly allocated, i.e., Bk = B/K for all k. • Multi-SPIN with Homogeneous Draft Length (HomoMulti-SPIN): This baseline lets all devices adopt a homogeneous draft length L selected by exhaustive search to maximize the sum goodput, while the bandwidth is optimized by solving the bandwidth-allocation problem (P1.1). B. Central versus Distributed SPIN Deployment Fig. 6 compares the maximum sum goodput achieved by P2P-SPIN, Cen-SPIN, and Multi-SPIN (i.e., Hete-Multi-SPIN) under the Llama-2 and Qwen3.5 model pairs. Multi-SPIN consistently delivers the highest goodput in both cases. For the Llama-2 pair, it reaches approximately 145 tokens/s, corresponding to about 2.5× and 4.6× the goodput of Cen-SPIN and P2P-SPIN, respectively. A similar trend is observed for the Qwen3.5 pair, where Multi-SPIN attains about 153 tokens/s and outperforms Cen-SPIN by roughly 3× despite the substantially higher inference overhead of the larger models. These results demonstrate the protocol-level gain of Multi-SPIN. It combines parallel device-side drafting with batched server-side verification, thereby mitigating the server-side drafting bottleneck in Cen-SPIN while avoiding the inefficient sequential verification pattern of P2P-SPIN. C. Performance of Multi-SPIN Framework Fig. 7 compares the sum goodput of Hete-Multi-SPIN and the optimization baselines under varying total bandwidth budgets for the Llama-2 and Qwen3.5 model pairs. As the budget
increases, the goodput of all schemes improves but gradually saturates, marking a transition from a communication-limited regime to a computation-limited one. Hete-Multi-SPIN attains the highest goodput across both pairs, with its advantage most pronounced in the bandwidth-limited regime, where uniform allocation and fixed draft lengths inflate the multiaccess latency under device heterogeneity. At the smallest bandwidth budget, it improves the goodput over Fixed BW&L by about 88% for both model pairs. This gain narrows as the bandwidth grows and uplink transmission ceases to dominate the E2E latency. Among the partially optimized baselines, UniBW Multi-SPIN consistently outperforms Homo-Multi-SPIN, indicating that adapting draft lengths to heterogeneous device conditions is more beneficial than bandwidth adaptation alone, while jointly optimizing both dimensions yields the largest gain. Fig. 8 compares the scalability of the schemes as the number of participating devices grows. Hete-Multi-SPIN scales favorably with more devices, whereas Fixed BW&L saturates and eventually degrades, producing a widening performance gap. For the Llama-2 pair, the gain of Hete-Multi-SPIN over Fixed BW&L grows from about 21% at small scale to about 67% at K = 24, and for the Qwen3.5 pair from about 29% to over 80%. This widening gap arises because uniform resource allocation forces the server to wait for the slowest draft under batching synchronization, an effect that intensifies as device heterogeneity accumulates over a larger pool. By jointly adapting bandwidth allocation and draft lengths to perdevice conditions, Hete-Multi-SPIN suppresses these straggler effects and preserves efficient multi-access SPIN at larger system scales. VII. C ONCLUDING R EMARKS This work advocates for the distributed deployment of SPIN to enable cooperative token generation at the network edge. This approach has the advantage of effectively balancing computational loads between edge devices and servers. To materialize this vision, we proposed the Multi-SPIN framework that jointly optimizes multiuser draft lengths and multi-access bandwidth allocation to maximize the sum token goodput. Despite the complexity of these high-dimensional optimization problems, we developed efficient solution strategies via decomposition techniques to successfully derive closed-form solutions for the underlying sub-problems. Thereby, this study establishes draft control as an important mechanism for efficient Multi-SPIN systems by demonstrating its fundamental role in balancing computation loads and communication overhead over devices and servers. This work opens a new frontier for SPIN-based cooperative token generation at the edge. Among numerous avenues for future investigation, we view the following directions as particularly promising: • Fairness-Aware Multi-SPIN Control: Shifting the optimization objective from maximizing the aggregate token goodput to ensuring satisfactory, guaranteed goodput levels for individual devices. • Adaptive SLM Placement: Developing mechanisms to dynamically download SLMs from a cloud-based AI
12
library onto edge devices. They can be tailored to users’ heterogeneous C2 capabilities and specific task preferences. • Alternative Multi-Access and Cross-Layer Designs: Extending the proposed framework beyond OFDMA to incorporate diverse multi-access schemes, such as TDMA, CDMA, and SDMA. This naturally extends to addressing physical-layer challenges (e.g., interference management and adaptive transmission) alongside network-layer issues (e.g., token flow control). • Advanced Cooperation Strategies: Investigating more sophisticated cooperation mechanisms, such as task-aware user clustering, to further enhance overarching system efficiency and scalability. A PPENDIX A. Proof of Theorem 1 Under the uniform-length policy, K(1 − α ) . ⋆ (1 − α)(Lϑ + T ver )
Its derivative can be written as τ̃ ′ (L) =
K g(L) , (1 − α)(Lϑ⋆ + T ver )2
where g(L) = −αL+1 ln α (Lϑ⋆ + T ver ) − ϑ⋆ (1 − αL+1 ). Since g ′ (L) = −(ln α)2 αL+1 (Lϑ⋆ + T ver ) < 0, g(L) is strictly decreasing. Moreover, g(0) = α| ln α|T ver − (1 − α)ϑ⋆ ,
C. Proof of Monotonicity of φ⋆ and Bk⋆ (L) Fixing j and differentiating the equalized-latency relation (32) implicitly yields Qtok φ⋆
rj (φ⋆ −Lj TjS )2 ∂φ⋆ > 0. = − PK Qtok Lk ∂Lj − k=1 S 2 ⋆ rk (φ −Lk Tk )
Thus, φ⋆ increases with Lj . Differentiating Bj⋆ (L) in (31) with respect to Lj then gives ⋆ Qtok φ⋆ − Lj ∂φ ∂Bj⋆ ∂Lj > 0, = ∂Lj rj (φ⋆ − Lj TjS )2 ⋆
L+1
τ̃ (L) =
Finally, as α → 1, a first-order expansion of the stationarity equation yields r 2(t − 1) ⋆ L̃ + 1 ∼ . − ln α
lim g(L) = −ϑ⋆ < 0.
L→∞
Since g is strictly decreasing with limL→∞ g(L) < 0, it changes sign at most once, so τ̃ (L) is unimodal. The maximizer is interior (i.e., L̃⋆ > 0) if and only if g(0) > 0, that is, T ver 1−α > . ⋆ ϑ α| ln α| Solving g(L) = 0 gives T ver ln −W−1 −α ϑ⋆ −1 /e L̃⋆ = − − 1. ln α The optimal integer draft length is obtained by checking the two nearest integers. B. Proof of Monotonicity of L̃⋆ Let t = T ver /ϑ⋆ and β = − ln α > 0. The stationarity condition in Appendix A is ⋆
β(L̃⋆ + t) + 1 = eβ(L̃ +1) . ⋆ Implicit differentiation gives ∂ L̃⋆ /∂t = 1/ eβ(L̃ +1) −1 > 0, hence ∂ L̃⋆ /∂T ver > 0 and ∂ L̃⋆ /∂ϑ⋆ < 0. Differentiating the same condition with respect to α likewise gives dL̃⋆ /dα > 0, so the optimal draft length increases with the acceptance rate.
⋆ where the inequality follows from Lj ∂φ ∂Lj < φ for K ≥ 2. ⋆ Hence, Bj (L) also increases with Lj .
D. Proof of Tightness of Bandwidth Constraint For fixed L, the numerator of the relaxed objective is constant, and thus PK E[Nk | Lk ] τ (L) = k=1 φ + T ver is strictly decreasing in φ. Meanwhile, over the feasible region φ > maxk Lk TkS , the left-hand side of K X
Qtok Lk ≤B r (φ − Lk TkS ) k=1 k is also strictly decreasing in φ. Therefore, if the constraint were slack at an optimum, one could slightly decrease φ while preserving feasibility, which would strictly increase the objective. This contradicts optimality. Hence, the relaxed bandwidth constraint must be active at the optimum. E. Proof of Proposition 1 Consider the relaxed problem (P2.1b) for a fixed φ. As proved in Appendix D, the bandwidth constraint is active at the optimum, so the KKT stationarity condition gives −αkL̃k +1 ln αk Qtok φ =λ . 1 − αk rk (φ − L̃k TkS )2 Let βk = − ln αk and yk = φ − L̃k TkS . Then βk y λQtok φ(1 − αk ) φ 2 TkS k = exp βk yk e +1 . rk βk TkS Applying the Lambert W function yields the continuous solution − φ s 2T S k α 2 λ Q φ | ln α | φ tok k . L̃⋆k (φ, λ) = S + W0 k S ln αk rk αk (1 − αk )−1 Tk 2Tk The integer solution is then obtained by the rounding rule in Proposition 1.
13
F. Proof of Remark 2
R EFERENCES
[1] K. B. Letaief, W. Chen, Y. Shi, J. Zhang, and Y.-J. A. Zhang, “The roadmap to 6G: AI empowered wireless networks,” IEEE Commun. Mag., vol. 57, no. 8, pp. 84–90, 2019. [2] G. Zhu, D. Liu, Y. Du, C. You, J. Zhang, and K. Huang, “Toward an intelligent edge: Wireless communication meets machine learning,” IEEE Commun. Mag., vol. 58, no. 1, pp. 19–25, 2020. [3] Z. Liu, Q. Lan, and K. Huang, “Resource allocation for multiuser edge inference with batching and early exiting,” IEEE J. Sel. Areas Commun., and define vol. 41, no. 4, pp. 1186–1200, 2023. s [4] J. Shao and X. Li, “AI flow at the network edge,” IEEE Netw., vol. 40, ck β no. 1, pp. 330–336, 2026. e λQtok φ βk (e k − 1) wk ≜ W0 S . [5] Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformr 2Tk k ers via speculative decoding,” in Proc. Int. Conf. Mach. Learn. (ICML), 2023, pp. 19 274–19 286. Then (37) becomes [6] M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “EdgeShard: Efficient LLM inference via collaborative edge computing,” IEEE Internet Things 2wk 2TkS φ J., vol. 12, no. 10, pp. 13 119–13 131, 2025. S , φ − L̃k Tk = wk . L̃k = S − [7] Z. Hao, H. Jiang, S. Jiang, J. Ren, and T. Cao, “Hybrid SLM and LLM βk βk Tk for edge-cloud collaborative inference,” in Proc. Workshop Edge Mobile Found. Models (EdgeFM), 2024, pp. 36–41. Substituting the above into (31) yields [8] D. Xu, W. Yin, H. Zhang, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu, “EdgeLLM: Fast on-device LLM inference with speculative decoding,” bk = Qtok ck − 1 . B IEEE Trans. Mobile Comput., vol. 24, no. 4, pp. 3256–3273, 2025. rk TkS wk [9] X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi et al., “SpecInfer: Accelerating large Moreover, since L̃k > 0, we have 0 < wk < ck . language model serving with tree-based speculative inference and verification,” in Proc. 29th ACM Int. Conf. Archit. Support Program. Lang. Then Oper. Syst. (ASPLOS), 2024, pp. 932–949. s [10] C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, ck β k λQtok φ βk (e − 1) e “Accelerating large language model decoding with speculative samln S pling,” arXiv preprint arXiv:2302.01318, 2023. rk 2Tk [11] E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge AI: On-demand acceler ating deep neural network inference via edge computing,” IEEE Trans. 1 1 (40) = ck − ln(2TkS ) + ln βk + ln eβk − 1 Wireless Commun., vol. 19, no. 1, pp. 447–457, 2020. 2 2 [12] J. Shao and J. Zhang, “Communication-computation trade-off in 1 λQtok φ resource-constrained edge inference,” IEEE Commun. Mag., vol. 58, + ln . no. 12, pp. 20–26, 2020. 2 rk [13] Z. Wang, Q. Zeng, H. Zheng, and K. Huang, “Revisiting outage for W0 (x) inference systems,” arXiv preprint arXiv:2504.03686, 2025. for x > 0, direct differentia- [14] edge and using W0′ (x) = x(1+W 0 (x)) Z. Wang, M. Cui, H. Yang, Q. Zeng, M. Sheng, and K. Huang, tion gives “Airbreath sensing: Protecting over-the-air distributed sensing against interference,” arXiv preprint arXiv:2508.11267, 2025. d ck 1 βk eβk 1 [15] Q. Zeng, Z. Wang, Y. Zhou, H. Wu, L. Yang, and K. Huang, ln wk − ck + − = . “Knowledge-based ultra-low-latency semantic communications for dβk wk βk (1 + wk ) 2 2 (eβk − 1) robotic edge intelligence,” IEEE Trans. Commun., vol. 73, no. 7, pp. 4925–4940, 2025. Since 0 < wk < ck and eβk − 1 < βk eβk for all βk > 0, the [16] K. Zhang, H. He, S. Song, J. Zhang, and K. B. Letaief, “Communicationright-hand side is strictly negative. Therefore, efficient distributed on-device LLM inference over wireless networks,” IEEE J. Sel. Topics Signal Process., vol. 19, no. 7, pp. 1301–1317, 2025. b d ck dBk [17] N. Xue, Y. Sun, Z. Chen, M. Tao, X. Xu, L. Qian, S. Cui, W. Zhang, < 0 =⇒ < 0. and P. Zhang, “WDMoE: Wireless distributed mixture of experts for dβk wk dβk large language models,” IEEE Trans. Wireless Commun., vol. 25, pp. bk is strictly 559–572, 2026. Finally, because αk = e−βk , it follows that B [18] Z. Wang, H. Yang, M. Sheng, K. B. Letaief, and K. Huang, “Space increasing in αk . network of experts: Architecture and expert placement,” arXiv preprint arXiv:2605.00515, 2026. [19] C. Zheng and T. Yang, “Communication-efficient collaborative LLM inference via distributed speculative decoding,” arXiv preprint arXiv:2509.04576, 2025. G. Practical Search Ranges for φ and λ [20] S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote For Algorithm 1, we adopt the following bounded search large language models,” arXiv preprint arXiv:2412.12687, 2024. [21] G. Zhang, Y. Cai, G. Yu, P. Popovski, and O. Simeone, “Quantizeranges for φ and λ: sample-and-verify: LLM acceleration via adaptive edge-cloud specula tive decoding,” IEEE Commun. Lett., vol. 30, pp. 852–856, 2026. Qtok KQtok S S , φ = max Lmax Tk + , [22] C. Zheng, K. Zhang, S. Chen, W. Zhang, Q. Liu, and A. A. Tesfay, φ = max Tk + k k Brk Brk “Fast collaborative inference via distributed speculative decoding,” arXiv preprint arXiv:2512.16273, 2025. where Lmax is the prescribed maximum draft length used in [23] D. Wen et al., “Task-oriented sensing, computation, and communication the experiments, and integration for multi-device edge AI,” IEEE Trans. Wireless Commun., vol. 23, no. 3, pp. 2486–2502, 2023. S 2 rk (φ − Tk ) − ln αk 2 [24] F. Chen, P. Li, T. H. Luan, Z. Su, and J. Deng, “Spin: Accelerating large λ = ϵλ , λ = max · αk , language model inference with heterogeneous speculative models,” in k Qtok φ 1 − αk IEEE INFOCOM 2025-IEEE Conference on Computer Communications. where ϵλ is a small positive constant. IEEE, 2025, pp. 1–10.
It suffices to analyze the continuous optimizer in (37), since the rounding in (36) only affects isolated threshold points. Let βk φ , βk ≜ − ln αk > 0, ck ≜ 2TkS
14
[25] R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, “Efficiently scaling transformer inference,” Proc. Mach. Learn. Syst. (MLSys), vol. 5, pp. 606–624, 2023. [26] NVIDIA Corporation, NVIDIA A100 Tensor Core GPU: Data Sheet, NVIDIA Corporation, n.d., accessed: May 8, 2026. [Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/ a100/pdf/nvidia-a100-datasheet-nvidia-us-2188504-web.pdf [27] H. Qian, S. K. Gonugondla, S. Ha, M. Shang, S. K. Gouda, R. Nallapati, S. Sengupta, X. Ma, and A. Deoras, “BASS: Batched attention-optimized speculative sampling,” in Proc. Findings Assoc. Comput. Linguistics (ACL), 2024, pp. 8214–8224. [28] Q. Su, C. Giannoula, and G. Pekhimenko, “The synergy of speculative decoding and batching in serving large language models,” arXiv preprint arXiv:2310.18813, 2023. [29] H. Wu, X. Chen, and K. Huang, “Resource management for low-latency cooperative fine-tuning of foundation models at the network edge,” IEEE Trans. Wireless Commun., vol. 24, no. 6, pp. 4839–4852, 2025. [30] C. You, K. Huang, H. Chae, and B.-H. Kim, “Energy-efficient resource allocation for mobile-edge computation offloading,” IEEE Trans. Wireless Commun., vol. 16, no. 3, pp. 1397–1411, 2017. [31] T. Liu, Y. Li, Q. Lv, K. Liu, J. Zhu, W. Hu, and X. Sun, “PEARL: Parallel speculative decoding with adaptive draft length,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025. [32] H. Yang, Z. Wang, and K. Huang, “Optimal batch-size control for low-latency federated learning with device heterogeneity,” IEEE Trans. Commun., vol. 74, pp. 5232–5247, 2026. [33] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004. [34] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023. [35] Qwen Team, “Qwen3.5: Towards native multimodal agents,” February 2026. [Online]. Available: https://qwen.ai/blog?id=qwen3.5 [36] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al., “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732, 2021. [37] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al., “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021. [38] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing et al., “Judging LLM-as-a-judge with MT-Bench and chatbot arena,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, pp. 46 595–46 623, 2023. [39] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” in Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 2383–2392.