GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference
arXiv:2605.10124v1 [cs.NI] 11 May 2026
Zengzipeng Tang, Yuxuan Sun, Wei Chen, Jianwen Ding, and Bo Ai School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China Email: {zzptang, yxsun, weich, jwding, boai}@bjtu.edu.cn
Abstract—The recent growth of on-device Large Language Model (LLM) inference has driven significant interest in deviceedge collaborative LLM inference. As a promising architecture, Speculative Decoding (SD) is increasingly adopted where a lightweight draft model rapidly generates candidate tokens to be verified by a powerful target model. However, a fundamental challenge lies in achieving per-token resource scheduling to effectively adapt SD paradigm to resource-constrained edge environment. This paper proposes a Generative Entropy- and Lyapunov-based Adaptive Token Offloading framework, named GELATO, to maximize decoding throughput under energy constraints in a device-edge collaborative SD system. Specifically, an outer drift-plus-penalty loop makes online decisions to establish a reference drafting budget, managing long-term energythroughput trade-off. Further, a nested entropy-driven generation mechanism executes early exiting to adapt to per-token dynamic generative uncertainty. Theoretical analysis establishes a rigorous performance bound on long-term throughput for GELATO. Extensive evaluations demonstrate that GELATO achieves a globally optimal tradeoff, outperforming state-of-the-art distributed SD architectures by 64.98% in token throughput and reducing energy consumption by 47.47% under resource-constrained environments, while preserving LLM decoding quality. Index Terms—Speculative decoding, large language models, device-edge collaborative inference, resource allocation
I. I NTRODUCTION While Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse complex tasks, their inherent auto-regressive generation mechanism imposes significant computational demands. To alleviate this burden and reduce reliance on centralized resources, deploying LLM at the network edge has become a strategic evolution in system architecture [1]. Nevertheless, deployment on end devices remains highly impractical. Severe constraints in storage and memory I/O bandwidth strictly limit devices to hosting models with sub-10B parameters [2], [3]. Furthermore, restricted computational power and limited battery capacity cause devices to struggle in meeting stringent low-latency inference constraints independently [4]. Consequently, device-edge collaborative LLM inference becomes imperative. However, in resource-constrained edge environments, traditional device-edge split inference [5], [6] is fundamentally incompatible with the autoregressive mechanism, as it requires per-token transmission of large KV cache tensors, resulting
in prohibitive communication overhead and latency. To address this communication bottleneck, device-edge collaborative speculative decoding (SD) [7], [8] has emerged as a transformative paradigm. By deploying lightweight small language models (SLMs) on end devices for serial drafting and a powerful LLM on the edge server for parallel verification, this architecture successfully shifts the communication payload from dense neural activations to lightweight token sequences, theoretically unlocking the potential of edge LLM [1]. Despite the potential of device-edge collaborative speculative decoding, the limited and fluctuating edge resource availability remains a critical system bottleneck [9]. To enable efficient device-edge collaborative speculative decoding, it is necessary to jointly consider system resource management and autoregressive generation efficiency. Existing studies have explored resource-aware scheduling and optimization to reduce latency and energy consumption. Specifically, work [10] proposed a resource-aware parallel speculative decoding framework, while papers [11] and [12] developed optimization models for joint latency and energy minimization. Other prior work [13] developed a framework to decouple the drafting and verification phases across different computational nodes, while work [14] and [15] utilized uncertainty-aware inference and early exits mechanisms to improve the efficiency of the speculative process. However, these approaches typically lack the visibility required for token-level adapted resource management, leading to inefficient resource usage and performance degradation in edge environments. To enable efficient device-edge collaborative speculative decoding under coupled resource constraints and generation dynamics, we propose a novel Generative Entropy and Lyapunov-based Adaptive Token Offloading framework, termed GELATO, where a resource-constrained device collaborates with an edge server. The main contributions are summarized as follows: •
We propose a novel hierarchical optimization framework GELATO to jointly optimize long-term generation throughput and energy consumption in resourceconstrained device-edge speculative decoding, which coordinates resource scheduling with per-token adaptation by employing a nested Lyapunov-uncertainty method.
Server: parallel draft verification
We consider a device-edge collaborative LLM system, as illustrated in Fig.1 where a resource-limited device cooperates with an edge server. To facilitate efficient inference, we deploy a lightweight SLM on device and a computation-intensive LLM on server. We assume that both models share a common vocabulary V, which defines the set of all possible tokens. The system operates in a step-wise manner, where each step k ∈ {1, 2, · · · , K} corresponds to a complete round of speculative decoding. At each step k, the SLM autoregressively generates a sequence of candidate tokens, which are then transmitted over a wireless channel to the edge server for parallel verification by the LLM. A. Speculative Decoding Model At each decoding step k, the SLM generates a sequence of draft tokens Ỹk = [ỹ1 , ỹ2 , . . . , ỹγk ] with length γk . The computation complexity of the SLM depends on its architectural parameters. Specifically, let N denote the number of transformer layers, dm the hidden dimension, and df the dimension of the feed-forward network (FFN). Then, following [12], the total Floating Point Operations (FLOPs) required to generate γk tokens can be modeled as: i h γk dm + 2d2m + 4dm df , (1) Fk = N γk 6d2m + 4 Lk + 2 where Lk denotes the accumulated context length by step k and the term Lk +γk /2 accounts for the average context length during the auto-regressive drafting process. Accordingly, the inference latency of the SLM is denoted as TkD = fFDk , where f D denotes the computation ability of device in FLOPS. Following the methodology in the ML.ENERGY benchmark [16], we characterize the energy consumption of the autoregressive decoding phase based on its steady-state power. The steady state refers to the period where the system operates under a stable and saturated workload, reflecting long-term energy utilization. Therefore, for each step k, the computational
Uplink transmission
Output tokens
Target LLM
Bonus token Draft tokens
Downlink transmission
II. S YSTEM MODEL
Accepted tokens
Edge Server
Verification results
Device: draft generation and top-p compression p threshold
Battery
Device
Draft SLM
Draft tokens and entropy
Full vocabulary
To resolve the challenge of stochastic and non-explicit system token throughput optimization objective, we introduce an expectation-based surrogate that enables stepwise Lyapunov-based online draft budget scheduling, while providing theoretical performance guarantees for system stability and resource efficiency. • We design a nested real-time early-exit mechanism for the SLM drafting stage to capture per-token generative dynamics. We establish a theoretical relationship between token acceptance rate with generative entropy, and leverage it for an entropy-driven adaptation mechanism that tracks the step-wise budget to ensure consistency. • Extensive evaluations are conducted across varying wireless bandwidths and LLM verification latencies. Experimental results demonstrate that our proposed approach outperforms state-of-the-art distributed split architectures, achieving a superior trade-off between throughput and energy consumption under constrained conditions.
•
Top-p compression
Fig. 1: Illustration of device-edge collaborative speculative decoding system.
energy cost at the mobile device is defined as EkD = pD TkD , where pD represents the steady-state power of SLM decoding. Upon receiving the draft sequence Ỹk , the edge server utilizes LLM to evaluate the proposals. Unlike the auto-regressive drafting phase, LLM leverages the causal masking mechanism of Transformers to verify all γk draft tokens simultaneously in a single forward pass. Let ŷi = arg max P (y|X<i ) denote the target prediction of LLM given context X<i , then following the standard paradigm of speculative decoding, a draft token ỹi is accepted if it matches target ŷi . Verification ends at the first mismatch. Edge server then appends one additional “bonus” token, which is either a corrected token upon mismatch or the next predicted token if the entire draft is accepted. We define Nk as the total number of tokens hit and appended to the sequence in the k-th step. This value is a stochastic variable determined by draft length γk and per-token acceptance outcomes of target LLM. To quantify the stochasticity in the drafting process, we employ generative entropy as a measurable indicator of uncertainty. Specifically, for the i-th draft token, let qi (x) represent the predicted probability of token x ∈ V. Then, the generative uncertainty P quantified by the generative entropy is defined as: Hi = − x∈V qi (x) log qi (x). The wall-clock latency for this verification step is considered a stable value TkLLM given the consistent computational capability of the server [17]. B. Probabilistic Payload-Compressed Communication Model In conventional distributed speculative decoding [10], device transmits full vocabulary distribution for each draft token, resulting in a communication cost proportional to |V|. This incurs prohibitive uplink overhead for modern LLMs with |V| > 105 [13]. Consequently, limited uplink bandwidth and energy budgets in device-edge systems make it infeasible to transmit a large, fixed number of draft tokens at each step. To alleviate this, we employ a probabilistic payload compression scheme based on top-p truncation. Specifically, for the i-th drafted token, the SLM sorts the vocabulary in descending order of generation probability qi (x),Pand selects the smallest subset of tokens Si ⊂ V such that x∈Si qi (x) ≥ p, where p ∈ (0, 1) is a predefined probability threshold. By transmitting only the elements in Si , the uplink data volume for a draft sequence of length γk can be reduced to: γk X Dk = |Si | (bprob + bindex ) , (2) i=1
where bprob and bindex denote the bit-width of a probability value and a vocabulary index, respectively. The instantaneous uplink transmission rate at the decision step k is modeled by the Shannon capacity formula: rkU = pU h k Bk log2 1 + N , where Bk is the channel bandwidth, pU 0 Bk is the uplink transmission power, hk represents the channel gain, and N0 is the Gaussian noise power spectral density. Based on the compressed payload, the uplink communik cation delay is calculated as TkU = D U . Accordingly, the rk communication energy consumption is EkU = pU TkU . Since the edge server is generally grid-powered, its computation energy is not considered. Therefore, the total energy consumption on the mobile device at decision step k is : Ek = EkD + EkU .
(3)
Meanwhile, the total latency to complete decoding step k comprises the on-device drafting, uplink transmission, edge verification, and downlink feedback delays. Given that the downlink payload is minimal, its delay can be neglected. Thus, the total latency for step k is given by: Tk = TkD + TkU + TkLLM .
(4)
C. Problem Formulation k For step k, the decoding throughput is defined as Γk = N Tk . Our objective is to maximize the long-term time-averaged decoding throughput while ensuring that long-term average local energy consumption of the device remains below a prescribed threshold Ē:
K
P1:
max γk
s.t.
1 X Γk K
(5a)
1 K
(5b)
k=1 K X
Ek ≤ Ē,
k=1
γk ∈ {1, 2, . . . , γ0 },
∀k,
(5c)
where (5a) is the optimization goal of maximizing the longterm decoding throughput, (5b) is the long-term energy constraint ensuring that energy of device remains within a stable range, and (5c) is used to limit the range of draft length γk . Problem P1 is essentially a stochastic optimization problem. Its intractability arises from several coupled challenges: (1) Unknown future dynamics: the system targets long-term throughput and energy stability, but only instantaneous states are observable, rendering offline optimization intractable. (2) Non-explicit formulation of Nk : the objective depends on the hit token number Nk , which has no closed-form expression in terms of γk , making direct throughput optimization intractable. (3) Limited adaptation to token generation dynamics: stepwise scheduling operates at the granularity of a speculation step, failing to capture fine-grained, token-level dynamics, leading to inefficient resource utilization. III. P ROPOSED A LGORITHM In this section, we propose the GELATO framework to solve P1. To resolve the intractability arising from Nk , we utilize
an expectation-based surrogate and Lyapunov optimization to enable P1 into tractable per-step online scheduling for token drafting budgets. Further, a nested tracking mechanism is incorporated to capture dynamic token-level uncertainty via generative entropy, enabling adaptive early exiting to ensure system consistency and improve resource efficiency. A. Problem Conversion and Step-Wise Scheduling To address the intractability of the stochastic hit token number Nk , we utilize an expectation-based surrogate model to approximate the relationship between the draft length γk and the average acceptance rate ρ [7], expressed as: γk X 1 − ργk +1 . (6) E[Nk ] = 1 + ρi = 1−ρ i=1 Building upon this explicit surrogate, we develop a steplevel online dynamic framework based on Lyapunov optimization to address the stochastic optimization problem P1. Under the drift-plus-penalty framework [18], [19], long-term stochastic optimization is reduced to an online decision process that balances virtual queue stability and penalty minimization. To handle the long-term energy constraint in (5b), we construct a virtual queue Qk ≥ 0 for the device, where the perframe energy consumption Ek is treated as the arrival process and the energy budget Ē serves as a constant service rate. The dynamics of queue backlog are given by: + Qk+1 = Qk + Ek − Ē, 0 . (7) where [x]+ is defined as max{x, 0}, and the queue is initialized with Q1 = 0. Then, we define the Lyapunov function for Qk as L(Qk ) = 21 Q2k . Based on this, the conditional Lyapunov drift is denoted as ∆(Qk ) = E [L(Qk+1 ) − L(Qk )|Qk ]. Lemma 1. Using Lyapunov drift-plus-penalty framework, problem P1 can be transformed into an online, per-step optimizaiton problem with weight factor V : P1.1:
max U (γk ) = V Γk − Qk Ek γk
s.t.
(5c).
(8)
where U (·) is a step-wise utility function. Proof. Following the classic Lyapunov optimization theorem [18], we obtain the upper bound of the Lyapunov drift: ∆(Qk ) ≤ θ0 + Qk Ek − Ē . (9) 2 where θ0 = 12 maxk {|Ek − Ē|} . Then, by incorporating the system objective as a penalty term, problem P1 can be equivalently transformed into problem (P1.1). Fundamentally, P1.1 is a non-linear integer programming problem. The objective couples a fractional structure with an exponential geometric progression. Conventional optimization techniques would lead to a highly complex transcendental equation, rendering a closed-form solution intractable. Since γk is discrete and bounded by γ0 , the optimization is one-dimensional with a limited search space. We thus obtain the optimal budget γ̃k∗ by exhaustively evaluating U (γk ) over
all feasible values. As shown in Algorithm 1, this guarantees optimality with a time complexity of O(γ0 ). B. Dynamic Drafting via Entropy-Guided Early Exits Although III-A allocates the expected budget γ̃k∗ to ensure long-term stability, this step-level decision remains blind to the real-time quality of the drafting process. To address this, we seek a fine-grained mechanism to monitor the generative state at a per-token granularity. We identify a measurable indicator that reflects the likelihood of token acceptance based on the stochasticity of the drafting process and utilize it to adaptively refine the allocated budget. Specifically, we establish a consistent negative correlation between the draft token entropy H and the actual target acceptance rate ρ through a general function:
Algorithm 1 GELATO Algorithm 1: Initialize the virtual energy queue Q1 = 0. 2: for each decision step k = 1, 2, . . . , K do 3: Observe current hk and Qk ; 4: Initialize utility U ∗ = −∞ and γ̃k∗ = 1; 5: for each candidate i = 1 to γ0 do 6: 7: 8: 9: 10: 11: 12:
Evaluate utility: U (i); if U (i) > U ∗ then Update optimal U ∗ = U (i) and optimal γ̃k∗ = i. Track the uncertainty backlog Θ using (12); Determine adapted draft length γk∗ using (13); Transmit the compressed payload corresponding to (2); Update Qk+1 using (7).
(10)
wireless channels without specific distributional assumptions, the performance bound is given in the following theorem.
where φ(·) is monotonically decreasing. In practical deployments, each specific draft-target model pair exhibits a nominal average acceptance rate ρ0 , which can be obtained through offline experiments. To establish a theoretically grounded early exiting criterion, we map this statistical baseline in (10) to an allowable uncertainty threshold:
Theorem 1. Relative to the offline optimal solution, the cumulative throughput of GELATO is bounded by K K X X θ2 K 2 +K(K −1)δ0 θ0 Γ⋆k − 0 Γ‡k ≥ , (14) 2V
ρ = φ(H),
Hth = φ−1 (ρ0 ).
(11)
At step k, SLM tracks the cumulative entropy of draft sequence and terminates drafting by maintaining an uncertainty queue to continuously monitor the cumulative generative risk. Let Θi denote the uncertainty backlog after the i-th draft token. Inspired by the leaky bucket mechanism, the queue evolves as: Θi = max{0, Θi−1 + Hi − Hth },
(12)
where Hi is the contextual entropy of the i-th token. The autoregressive drafting immediately terminates once the backlog exceeds a predefined safety threshold Θth . Thus, the actual transmitted draft length operating within γ̃k∗ is given by: n o γk∗ = max j ≤ γ̃k∗ Θi ≤ Θth , ∀i ∈ 1, . . . , j . (13) The complete execution procedure of the proposed dualloop framework is summarized in Algorithm 1. Note that the step-level resource queues of the outer loop are continuously updated at the end of each step using the actual execution results and real resource consumption from the inner loop.
k=1
k=1
and the cumulative energy violation is bounded by K X p Ek ≤ K Ē + 2θ0 K 2 + 2K(K − 1)δ0 θ0 .
(15)
k=1
Proof. Define the K-step drift as ∆K ≜ L(K + 1) − L(1) and δk = Ẽk − Ek . Then the K-step drift-pluspenalty function is bounded by ∆K − V Γk ≤ θ0 K + PK k=1 Qk Ẽk − Ē − V Γk − Qk δk . We use ⋆ , † , and ‡ to denote the optimal offline solution of P1, the classical drift-plus-penalty solution to P1.1, and the result of our proposed GELATO, respectively. The drift-pluspenalty over K steps satisfies the following bound: K K ‡ X X † † ‡ ‡ ∆K −V Γk ≤ θ 0 K Qk Ẽk − Ē − V Γk − Qk δk (a)
k=1 K X
≤ θ0 K + (b)
≤ θ0 K +
k=1 K X
k=1
†
Qk Ek − Ēn
⋆
Qk Ẽk − Ē
−V Γ†k − Qk δk‡
−V Γ⋆k +2δ0 Qk ,
(16)
k=1
C. Performance Analysis We now characterize the theoretical performance of the proposed GELATO algorithm by establishing a comparison with an ideal offline optimum of solving problem P1. Let γk⋆ represent the optimal offline decision, and let Γ⋆k denote the corresponding maximum throughput of step k. In our setting, decisions rely on the step-wise utility U (γk ) and the corresponding energy consumption derived from the PKempirical target acceptance rate, denoted as Ẽk . Define k=1 Γ‡k as the cumulative throughput of the proposed algorithm, which is obtained by calculating (5a) corresponding to (13) in each step. Assuming temporally independent
n o where δ0 ≜ maxk Ẽk − Ek . Inequality (a) follows since the optimal solution to P1 minimizes the objective at each step k. Inequality (b) holds because the drift-plus-penalty method minimizes Qk Ek − V Γk ; thus, replacing it with the offline optimal policy can only yield a larger or equal value. Now we bound the right-hand-side of (16). Note that Qk+1 − Qk ≤ ⋆ θ0 , ∀k, and thus Qk = Q⋆k − Q1 ≤ (k − 1)θ0 , Qk Ek − Ē = (Qk − Q1 ) Ek − Ē ≤ (k − 1)θ02 . Substituting them into (16) yields: K K X X 1 ∆‡K −V Γ‡k ≤ −V Γ⋆k + θ02 K 2 +K(K −1)δ0 θ0 . (17) 2 k=1
k=1
* ( / $ 7 2 3 U R S R V H G ' 6 6 ' = 7 6 W D W L F 6 ' = 7, p = 0.99995 6 W D W L F 6 ' = 7, p = 0.99998 6 W D W L F 6 ' = 5, p = 0.99995 6 W D W L F 6 ' = 9, p = 0.99995
* 9
* 9
4 X H X H 9 4 X H X H 9
' H F L V L R Q 6 W H S k
Fig. 4: Draft token budget and queue backlog under different V .
% D Q G Z L G W K 0 + ]
* ( / $ 7 2 3 U R S R V H G ' 6 6 ' = 7 6 W D W L F 6 ' = 7, p = 0.99995 6 W D W L F 6 ' = 7, p = 0.99998 6 W D W L F 6 ' = 5, p = 0.99995 6 W D W L F 6 ' = 9, p = 0.99995
7 O O P P V
(c) System throughput under different verification latencies.
* ( / $ 7 2 3 U R S R V H G ' 6 6 ' = 7 6 W D W L F 6 ' = 7, p = 0.99995 6 W D W L F 6 ' = 7, p = 0.99998 6 W D W L F 6 ' = 5, p = 0.99995 6 W D W L F 6 ' = 9, p = 0.99995
(a) System throughput under different bandwidth.
7 K U R X J K S X W W R N H Q V
4 X H X H % D F N O R J Q
6 F K H G X O H G % X G J H W *
6 / 0 7 R S p & R Y H U D J H
( Q W U R S \ H
Fig. 2: The experimen- Fig. 3: LLM accuracy and tal and fitted curves of vocabulary scaling under difH and ρ. ferent SLM top-p coverage.
( Q H U J \ & R Q V X P S W L R Q - W R N H Q
% D Q G Z L G W K 0 + ]
(b) Energy consumption under different bandwidth.
( Q H U J \ & R Q V X P S W L R Q - W R N H Q
/ / 0 $ F F X U D F \ / R V V
$ F F H S W 5 D W H
7 K U R X J K S X W W R N H Q V
( [ S H U L P H Q W ) L W W H G & X U Y H
9 R F D E X O D U \ 6 L ] H
* ( / $ 7 2 3 U R S R V H G ' 6 6 ' 6 W D W L F 6 ' = 7, p = 0.99995 6 W D W L F 6 ' = 7, p = 0.99998 6 W D W L F 6 ' = 5, p = 0.99995 6 W D W L F 6 ' = 9, p = 0.99995
7 O O P P V
(d) Energy consumption under different verification latencies.
Fig. 5: Performance of the proposed algorithm and benchmarks on GSM8K.
Since ∆‡K ≥ 0, results in (14) follows from (17) by dividing both sides by V . As Γk ≥ 0, and 12 Q2K+1 ≤ ∆K , we get: K K X p X (Qk+1 −Qk ) = QK+1 ≤ 2∆K (18) Ek − Ē ≤ k=1
k=1 p ≤ 2θ0 K 2 + 2K(K − 1)δ0 θ0 . Thus, (15) in Theorem 1 is proved.
(19)
IV. S IMULATION R ESULTS In this section, we conduct simulations to evaluate the performance of our proposed framework. We simulate an edge speculative decoding system over a channel modeling both large-scale path loss and small-scale Rayleigh fading. The inference task deploy Qwen2.5-7B-Instruct [20] as the target model on the edge server and Qwen2.5-0.5B-Instruct [20] as the local draft model. We utilize the GSM8K dataset [21], a math benchmark designed to challenge multi-step reasoning. Through sufficient profiling on this model pair, the average acceptance rate is measured at ρ0 = 0.9. All reported results are averaged over 1000 simulation rounds. Unless otherwise specified, the main simulation parameters are set according to the configurations in [12], [22], and summarized in Table I. To validate the proposed uncertainty-acceptance rate model, we perform teacher-forced inference using the Qwen2.5 model pair. Fig. 2 shows the mapping in (10) follows an exponential decay, well approximated by ρ = exp(−0.35H), with the uncertainties discretized into uniform bins of step size 0.1. To demonstrate that our proposed payload compression communication method in II-B effectively preserves decoding quality, Fig.3 evaluates the impact of SLM vocabulary
TABLE I: Simulation Parameters Parameter Noise power spectral density N0 Device GPU frequency (f D ) Device Computation Power (pD ) Uplink Transmit Power (pU ) Long-term Energy Budget (Ē) Draft Length Capacity (γ0 ) Uncertainty Backlog Threshold (Θth )
Value -174 dBm/Hz 40.0 GFLOPS 12.0 W 23 dBm 1.2 J 15 1.2Hth
coverage p on LLM accuracy, which is normalized to the official Qwen baseline of 51.7% [20] on GSM8K. The inset plot highlights that under p = 0.99995, LLM achieves nearly lossless decoding quality compared to full-payload. To evaluate the step-wise scheduler, we consider two penalty weights with V = 10 and V = 100, which balances the expected generative yield and the virtual energy deficit. A larger V prioritizes throughput maximization, resulting in more aggressive drafting budget allocation, whereas a smaller one emphasizes energy stability by limiting the budget and queue backlog, as illustrated in Fig.4. These results validate that the drift-plus-penalty framework enables tunable adaptation to resource and performance requirements. To better demonstrate the superiority of the proposed hierarchical framework, we consider the following benchmark schemes: 1) Static Speculative Decoding (Static SD) [7], [8] where the system generates a fixed number of draft tokens γ ∈ {5, 7, 9} per step. To ensure a rigorous baseline comparison, this scheme is specifically enhanced with a top-p payload compression of p = 0.99995 and p = 0.99998; 2) Distributed Split
Speculative Decoding (DSSD) [13], representing the stateof-the-art communication-efficient architecture. This scheme mitigates the uplink bottleneck by replacing multiple uplink distribution transmissions with a single conditional downlink transmission, with the device receive power set to 19 dBm and the maximum draft token length set to 7. We then evaluate the system performance under varying channel bandwidth with a fixed TkLLM of 100ms. As shown in Fig.5(a) and Fig.5(b), our proposed framework achieves better throughput and energy consumption trade-off across all conditions. The advantage is most pronounced in the severely communication-constrained region. Specifically at 1 MHz, our method realizes up to 64.98% throughput gain. Meanwhile, our approach reduces energy consumption by 47.47% at 1 MHz and maintains the lowest usage across all bandwidths. This robustness stems from the joint effect of the outer-loop dynamic budget allocation and the inner-loop entropy-driven adaptation, instead of a fixed generation and transmission scheme. DSSD also shows robustness under limited bandwidth, whereas Static SD degrades in throughput and energy efficiency due to the absence of dynamic sequence adaptation. We also evaluate the performance under varying target verification latencies with a fixed bandwidth of 5MHz. As shown in Fig.5(c) and Fig.5(d), our framework achieves a throughput gain up to 20.04% over benchmarks at the stringent 140ms verification latency, and smoothly rising to 25.95% at 60ms. Concurrently, it reduces energy consumption by up to 23.39%. This is also attributed to the dynamic adaptation algorithm with entropy reference, resulting in stable energy consumption while achieving robust throughput against varying LLM delays. In comparison, DSSD framework also considers payload compression and shows some robustness to changes in latency. Static SD schemes suffer from throughput and energy inefficiency under high latency due to fixed-length drafting without dynamic early exiting. V. C ONCLUSION This paper proposed a tightly coupled dual-loop speculative decoding framework, named GELATO, for efficient deviceedge collaborative speculative decoding. By nesting microscopic entropy-driven dynamic halting within macroscopic Lyapunov scheduling, the system successfully resolves endogenous acceptance dynamics and time-coupled stochasticity. Evaluations demonstrate a globally optimal tradeoff between generation throughput and energy consumption, significantly outperforming state-of-the-art distributed architectures under severe communication bottlenecks. Future work will extend this collaborative optimization to the context prefill phase via distributed prompt processing and cache synchronization. R EFERENCES [1] G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, “Mobile edge intelligence for large language models: A contemporary survey,” IEEE Commun. Surveys Tuts., vol. 27, no. 6, pp. 3820–3860, 2025. [2] S. Laskaridis, K. Katevas, L. Minto, and H. Haddadi, “Mobile and edge evaluation of large language models,” in Workshop on Efficient Systems for Foundation Models, ICML, 2024.
[3] D. Xu, W. Yin, H. Zhang, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu, “EdgeLLM: Fast on-device LLM inference with speculative decoding,” IEEE Trans. Mobile Comput., vol. 24, no. 4, pp. 3256–3273, 2024. [4] M. Xu, H. Du, D. Niyato, J. Kang, Z. Xiong, S. Mao, Z. Han, A. Jamalipour, D. I. Kim, X. Shen et al., “Unleashing the power of edgecloud generative AI in mobile networks: A survey of AIGC services,” IEEE Commun. Surveys Tuts., vol. 26, no. 2, pp. 1127–1170, 2024. [5] D. Wen, P. Liu, G. Zhu, Y. Shi, J. Xu, Y. C. Eldar, and S. Cui, “Task-oriented sensing, computation, and communication integration for multi-device edge AI,” IEEE Transactions on Wireless Communications, vol. 23, no. 3, pp. 2486–2502, 2024. [6] Z. Liu, Q. Lan, and K. Huang, “Resource allocation for multiuser edge inference with batching and early exiting,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 4, pp. 1186–1200, 2023. [7] Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in Proc. Int. Conf. Mach. Learn. (ICML), 2023. [8] C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318, 2023. [9] Y. He, J. Fang, F. R. Yu, and V. C. Leung, “Large language models (LLMs) inference offloading and resource allocation in cloud-edge computing: An active inference approach,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 11 253–11 264, 2024. [10] W. Zhao, W. Jing, Z. Lu, and X. Wen, “Edge and terminal cooperation enabled LLM deployment optimization in wireless network,” in Proc. IEEE/CIC Int. Conf. Commun. China (ICCC Workshops). IEEE, 2024. [11] J. Koh and H. J. Yang, “Collaborative large language model inference via resource-aware parallel speculative decoding,” arXiv preprint arXiv:2511.01695, 2025. [12] X. Zhang, J. Liu, Z. Xiong, Y. Huang, G. Xie, and R. Zhang, “Edge intelligence optimization for large language model inference with batching and quantization,” in 2024 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2024, pp. 1–6. [13] J. Ning, C. Zheng, and T. Yang, “DSSD: Efficient edge-device deployment and collaborative inference via distributed split speculative decoding,” in ICML Workshop on Machine Learning for Wireless Communication and Networks (ML4Wireless), 2025. [14] S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. S. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote large language models,” in Proc. IEEE Int. Conf. on Mach. Learn. Commun. Netw. (ICMLCN), 2025. [15] Y. Venkatesha, S. Kundu, and P. Panda, “Fast and cost-effective speculative edge-cloud decoding with early exits,” arXiv preprint arXiv:2505.21594, 2025. [16] J.-W. Chung, J. J. Ma, R. Wu, J. Liu, O. J. Kweon, Y. Xia, Z. Wu, and M. Chowdhury, “The ML. ENERGY benchmark: Toward automated inference energy measurement and optimization,” arXiv preprint arXiv:2505.06371, 2025. [17] S. Oh, J. Kim, J. Park, S.-W. Ko, J. Choi, T. Q. Quek, and S.-L. Kim, “Communication-efficient hybrid language model via uncertaintyaware opportunistic and compressed transmission,” arXiv preprint arXiv:2505.11788, 2025. [18] M. Neely, Stochastic network optimization with application to communication and queueing systems. Morgan & Claypool Publishers, 2010. [19] M. J. Neely, “Stochastic network optimization with non-convex utilities and costs,” in 2010 Information Theory and Applications Workshop (ITA). IEEE, 2010, pp. 1–10. [20] Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/ [21] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021. [22] M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “Edgeshard: Efficient LLM inference via collaborative edge computing,” IEEE Internet of Things Journal, vol. 12, no. 10, pp. 13 119–13 131, 2024.