SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission Ce Zheng∗ , Xinghan Wang∗ , Jiahong Ning† , Yuxuan Shi∗ , Ning Huang∗ , Tingting Yang∗ ∗ Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518055, China † Dalian Maritime University, Dalian 116026, China
arXiv:2604.25777v1 [eess.SP] 28 Apr 2026
Email: {zhengc, wangxh03}@pcl.ac.cn, [email protected], {shiyx01, huangn01, yangtt}@pcl.ac.cn
Abstract—Federated inference enhances LLM performance in edge computing through weighted averaging of distributed model predictions. However, autoregressive LLM inference requires frequent full-model forward passes across workers, severely limiting decoding throughput. Distributed deployment further aggravates this due to a communication bottleneck: each worker must transmit full token probability distributions per draft token, dominating end-to-end latency. To address these challenges, we introduce speculative decoding to enable parallel LLM processing and propose a top-K compressed transmission scheme with two server-side reconstruction strategies. We theoretically analyze the robustness of our method in terms of local reconstruction error, aggregation bias, and acceptance-rate bias, and derive corresponding bounds. Experiments demonstrate that our scheme achieves high generation fidelity while significantly reducing communication overhead. Index Terms—federated inference, speculative decoding, LLM, top-K.
I. I NTRODUCTION Federated inference — also known as ensemble inference — improves accuracy and robustness by aggregating predictions from models hosted on different devices via weighted averaging [1]–[5]. This paradigm has gained significant traction in distributed AI systems, particularly for edge computing scenarios where models reside on heterogeneous devices with varying computational capabilities. In the context of large language models (LLMs), federated inference frameworks have also been explored to leverage the collective power of distributed models for enhanced generation quality [6]–[9]. However, the autoregressive nature of LLM inference introduces a critical challenge in federated settings: each token generation step requires all workers to execute a full forward pass, leading to frequent synchronization and significantly reduced decoding throughput [10]. To mitigate the inefficiency of autoregressive token generation, we consider speculative decoding [11], [12], where a small “draft” model autoregressively generates a sequence of γ candidate tokens, which are then sent to the distributed LLMs This work is supported by the National Science and Technology Major Project of China under Grant No. 2025ZD1304900, and in part by the National Key R&D Program of China (Grant No.2024YFE0200801, No.2024YFE0200804). The authors would also like to thank Dr. Ke Zhang from Waseda University for his valuable assistance with the implementation of the code.
for parallel decoding and verification. By shifting part of the sequential workload to parallel processing, speculative decoding reduces the number of full LLM forward passes, thereby alleviating the latency inherent in purely autoregressive decoding while preserving generation quality. Despite its effectiveness in reducing LLM invocation frequency, speculative decoding introduces a new challenge in federated settings: a significant communication bottleneck. In each round, every worker must transmit its full-token probability distribution for all draft tokens to the server for aggregation and verification. Given large vocabulary sizes (e.g., 32K or more) and high-precision representations (e.g., FP16), this results in substantial uplink traffic per worker—on the order of hundreds of kilobits per token. The total communication cost scales linearly with both the number of workers and the draft length, which can easily dominate end-to-end latency and undermine the throughput gains from parallel decoding [13], [14]. Recent work has explored communication-efficient techniques for distributed speculative decoding, including quantization [15] and split verification [16]. However, these approaches do not specifically address the unique challenges of federated LLM inference, where multiple heterogeneous models must collaboratively generate tokens through a weighted aggregation mechanism that requires precise distribution alignment. In this paper, we study federated LLM inference with speculative decoding. Building on prior top-K compression techniques [17]–[19], we propose a communication-efficient transmission scheme that sends only the top-K token probabilities per worker, along with two server-side reconstruction strategies. Our approach preserves generation fidelity while drastically cutting bandwidth usage. We theoretically analyze the robustness of our method in terms of local reconstruction error, aggregation bias, and acceptance-rate bias, and derive corresponding bounds, which are further validated through experiments. The remainder of this paper is organized as follows: Section II introduces our system model. Section III details our compressed transmission scheme with top-K truncation and reconstruction. Section IV presents our theoretical analysis of token distribution distortion and acceptance rate variation. Finally, Section VI concludes the paper.
Fig. 1: An illustration of federated LLM inference with Speculative Decoding.
II. S YSTEM M ODEL A. Federated LLM Inference with Speculative Decoding We consider a federated LLM inference framework consisting of a coordinating server and a set of distributed workers M = {1, 2, . . . , M }, where each worker hosts a local LLM. All LLMs share the same vocabulary V, though the model sizes and architectures may be heterogeneous. The inference task follows an autoregressive decoding paradigm. At each step, the server sends the current prefix or newly generated token to all workers. Each worker performs a forward pass to compute a probability distribution over the vocabulary, and the server aggregates the distributions to generate the next token. Due to this sequential dependency, every generated token requires all workers to execute a forward pass and upload a full token distribution, causing frequent synchronization and significant communication overhead, especially for large vocabularies, which limits decoding throughput. To mitigate this problem, we further deploy a small language model (SLM) at the server—also using the same vocabulary—and adopt the speculative decoding mechanism as illustrated in Fig. 1, which proceeds as follows. Step 1: SLM’s Draft Token Generation. Given the prefix, the SLM autoregressively generates a draft token sequence of length γ, denoted by x = [x1 , x2 , . . . , xγ ]. At each step t = 1, . . . , γ, the draft token is sampled as xt ∼ qt , t = 1, · · · , γ, where qt = qt (x) x∈V denotes the output distribution of the SLM at step t. Step 2: Draft Token Broadcast. The draft sequence x is broadcast to all workers i ∈ M Step 3: Parellel Decoding. Given the draft sequence, each worker computes the conditional probability distribution in parallel pt,i = pt,i (x) x∈V ∈ R|V| , t = 1, · · · , γ + 1,
where pt,i (x) denotes the probability of generating token x at step t by worker i. Step 4: Token Distribution Upload. Each worker uploads its output token distribution pt,i to the server. Step 5: Server Aggregation The server aggregates the received distribution via a weighted average: X p̄t = wi pt,i = p̄t (x) x∈V , (1) i∈M
P
where p̄t (x) = i∈M wi pt,i (x). Step 6: Draft token Verification. The server verifies the draft token in two sub-steps: a). Accept/Reject: the draft token is accepted if qt (xt ) < p̄t (xt ), and otherwise rejected with probability p̄t (xt )/qt (xt ). Hence, we have the expected acceptance rate as [11]: X p̄t (x) αt = Ex∼qt min ,1 = min (p̄t (x), qt (x)) . (2) qt (x) x∈V
b). Resample: If xt is rejected, a new token x′t is resampled from the residual distribution max(0, p̄t (x) − qt (x)) rt (x) = P . (3) v∈V max(0, p̄t (v) − qt (v)) If all draft tokens are accepted, the server samples an additional token x′γ+1 ∼ p̄γ+1 ; otherwise, at the first rejected step t ≤ γ, it samples x′t from the residual distribution in (3) and discards {xt+1 , . . . , xγ }. The verified sequence [x1 , . . . , x′t ], with t = γ + 1 in the full-acceptance case, is appended to the prefix for the next decoding iteration. B. Communication Bottleneck While speculative decoding can reduce the number of full LLM forward passes required at each worker, each worker still needs to upload a probability distribution over all tokens for every draft token to the server. This incurs significant communication overhead, which grows with both the number of workers and the vocabulary size. To alleviate this issue, it
is necessary to compress the transmitted distributions without severely degrading reconstruction quality. III. C OMPRESSED T RANSMISSION S CHEME To address the communication bottleneck described above, we adopt top-K truncation for probability transmission. Specifically, at decoding step t, worker i produces a probability distribution pt,i = [pt,i (x)]x∈V . Let V (ki ) ⊂ V denote the set of top-ki tokens with the highest probabilities under pt,i . Compressed Transmission Scheme: Worker i transmits V (ki ) along with {pt,i (x)}x∈V (ki ) , i.e., the identities of the top-ki tokens and their associated probabilities. With a slight abuse of notation, we denote the reconstructed distribution under different reconstruction schemes (k ) by pt,ii (x). To simplify notation, we further define X X ρt,i = pt,i (v), ϵt,i = pt,i (x) = 1 − ρt,i , x∈V (ki )
P where wi ≥ 0 and i∈M wi = 1. The resulting distribution (k) p̄t is then used for token sampling or draft-token verification. IV. T HEORETICAL A NALYSIS In this section, we analyze the robustness of the proposed compressed transmission scheme in federated speculative decoding. Specifically, we characterize how top-K compression and reconstruction affect (i) the distortion of local and aggregated token distributions, and (ii) the variation of the acceptance rate in speculative decoding.1 To this end, we first define error measures at both the local and aggregated levels, as well as the acceptance rate variation, and then derive corresponding bounds. Definition 1 (Local Reconstruction Error). The local reconstruction error for worker i at decoding step t is (k )
(4) where ρt,i denotes the retained probability mass and ϵt,i is the residual mass after top-ki truncation. As top-K truncation removes some probability mass, the server cannot fully determine the original distribution. We therefore consider two reconstruction schemes to approximate the original distribution.
(8)
Definition 2 (Aggregation Bias). The aggregation bias induced by top-K reconstruction is (k)
∆t
(k)
≜ p̄t
− p̄t 1 .
(9)
Definition 3 (Acceptance Rate Variation). The acceptance rate variation at decoding step t is defined as
A. Renormalized Top-K Reconstruction In the first scheme, the server renormalizes the received probabilities over V (ki ) , while assigning zero probability to all remaining tokens. The reconstructed distribution is given by pt,i (x) , x ∈ V (ki ) , (ki ) ρt,i (5) pt,i (x) = 0, (ki ) x∈ /V .
(k )
∆t,ii ≜ pt,ii − pt,i 1 .
x∈V / (ki )
(k)
∆αt
(k)
≜ αt
− αt .
(10)
A. Local Reconstruction Error We first characterize the error introduced by reconstructing each worker’s top-K compressed distribution and quantify the resulting ℓ1 reconstruction error in the following lemma.
This scheme preserves the relative probabilities among the top-ki tokens, but discards all probability mass outside V (ki ) .
Lemma 1 (Local Top-K Reconstruction Error Bound). For (k ) worker i at decoding step t, the ℓ1 reconstruction error ∆t,ii satisfies (k ) (k ) ∆t,ii = 2ϵt,ii (11)
B. Residual-Mass Redistribution
under the reconstructed distribution defined in (5), and
In the second scheme, the server preserves the original probabilities of the transmitted top-ki tokens and redistributes the remaining probability mass uniformly over the rest of the vocabulary. The reconstructed distribution is defined as x ∈ V (ki ) , pt,i (x), (ki ) (6) pt,i (x) = ϵt,i , x∈ / V (ki ) . |V| − ki This scheme preserves the original probabilities of the topki tokens while maintaining a valid probability distribution over the entire vocabulary. After reconstructing the distributions from all workers, the server aggregates them to form a global token distribution. Specifically, let k = [k1 , . . . , kM ] denote the top-K profile across workers. At decoding step t, the aggregated distribution is computed as X (k) (k ) p̄t = wi pt,ii , (7) i∈M
(k )
(k )
∆t,ii ≤ 2ϵt,ii
(12)
under the reconstructed distribution defined in (6). Proof. By definition, (k )
∆t,ii =
X
(k )
pt,ii (x) − pt,i (x) .
(13)
x∈V
Splitting the vocabulary into V (ki ) and its complement, we obtain X (k ) X (k ) (k ) ∆t,ii = pt,ii (x) − pt,i (x) + pt,ii (x) − pt,i (x) . x∈V (ki )
x∈V / (ki )
(14) 1 Extending the analysis to end-to-end communication cost, decoding throughput, and final generation quality is left for future work. These metrics are fundamentally coupled with the token distribution and acceptance rate, and our results already provide indicative insights into their behavior.
1). Renormalized Top-K Reconstruction. For this scheme, we have X X pt,i (x) (k ) (a) pt,i (x) ∆t,ii = − pt,i (x) + ρt,i (k ) (k ) i x∈V / x∈V i X 1 − ρt,i X pt,i (x) + pt,i (x) = ρt,i (k ) (k ) x∈V
(b)
x∈V /
i
i
(k )
As shown in Lemma 1, the local reconstruction error introduced by top-K compression is bounded, indicating that the distortion of individual worker distributions is controlled. B. Aggregation Bias We now analyze the overall bias introduced when aggregating the reconstructed distributions from all workers. The following theorem provides an upper bound on the ℓ1 aggregation bias. Theorem 2 (Aggregation Bias Upper Bound). The aggre(k) gation bias ∆t induced by top-K compression is upper bounded as X (k) (k ) ∆t ≤ 2 wi ϵt,ii . (17) i∈M
Proof. Using the triangle inequality, we have =
X i∈M
X
X (k) min qt (x), p̄t (x) − min(qt (x), p̄t (x))
x∈V
= TV
x∈V
(k) qt , p̄t
− T V (qt , p̄t )
(a)
1 (b) X (k) (k) (k ) ≤ T V p̄t , p̄t = ∆t ≤ wi ϵt,ii . 2
(20)
i∈M
Here (a) follows from the triangle inequality of the total variation distance (Proposition 1 in [20]), and (b) follows from Theorem 3
i
where (c) follows from (6). This completes the proof.
(k)
=
(15)
where (a) follows from (5) and (b) follows from (4). 2). Residual-Mass Redistribution. For this scheme, we have X ϵt,i (k ) (c) ∆t,ii = − pt,i (x) |V| − ki (k ) i x∈V / X ϵt,i (k ) ≤ + pt,i (x) = 2ϵt,ii . (16) |V| − k i (k )
∆t
(k)
∆αt
=2ϵt,ii .
x∈V /
Proof. According to Lemma P 3.3 in [11], we have T V (q , p̄ ) = 1 − and t (x), p̄t (x)), x∈V min(q t t P (k) (k) T V qt , p̄t = 1 − x∈V min qt (x), p̄t (x) , where T V (·, ·) denotes the total variation distance. Combining the above results with (2), we obtain
(k )
wi pt,ii − pt,i
≤ 1
X
(k )
wi pt,ii − pt,i 1 .
i∈M
(18) Applying Lemma 1 completes the proof. Together with the local reconstruction bound, these results guarantee that top-K truncation introduces a bounded distortion both at the individual worker level and at the aggregated distribution level. C. Acceptance Rate The acceptance rate plays a crucial role: a higher acceptance rate implies that more tokens are accepted, leading to fewer interactions between the SLM and LLM, and consequently reducing the amount of data transmitted. Theorem 3 (Bound on the acceptance rate variation). Let (k) αt denote the acceptance rate via top-k compressed transmission. ∆α is bounded by X (k) (k) (k ) ∆αt ≤ ∆t ≤ wi ϵt,ii (19) i∈M
Theorem 3 shows that the perturbation of the acceptance behavior caused by compressed transmission is also bounded, ensuring stable decoding efficiency. V. E XPERIMENT In this section, we validate the compressed transsmission scheme and assess the effects of top-K truncation on federated LLM inference. Existing evaluations, which compare final task accuracy or output similarity under identical prompts suffer from two limitations: First, token sampling introduces randomness, and the autoregressive nature of LLMs causes such randomness to propagate across generation steps, resulting in output variability even for identical inputs. Second, prior evaluations based on final task accuracy do not capture lossy inference bias at the token or distribution level, since generated outputs may be correct but not aligned with the LLM’s output [21]. Accordingly, we redesign the experiments to explicitly evaluate distribution-level distortions. A. Experiment Setup We first perform federated LLM inference without compressed transmission to obtain the full token sequence together with the corresponding LLM vocabulary distributions at each generation step. Then we obtain the reconstructed distribution from (5) and (6). The bias is then computed according to (9) and averaged across all steps and samples to quantify the overall deviation. Our federated LLM inference scenario involves two workers hosting heterogeneous LLMs: LLaMA-7B and LLaMA-13B, respectively. The coordinating server deploys the SLM, LLaMA-68M, for speculative decoding. To emulate the server–worker architecture, the SLM and LLMs are executed on NVIDIA A800 GPUs. All experiments are conducted on the wmt14_ende_de dataset [22], which consists of instruction-following examples widely used for LLM evaluation. We further set the weights w1 = w2 = 1/2 and k1 = k2 for the ease of analysis.
0.16 0.14
2
0.12
(Renormalize) (Residual-Mass )
0.175 2
0.150
(Renormalize) (Residual-Mass )
0.175 0.150
0.125
0.08
0.100
0.06
0.075
0.075
0.04
0.050
0.050
0.02
0.025
0.025
0.00
0.000
1.0%
k/| | (%)
10.0%
100.0%
(Renormalize) (Residual-Mass )
0.125
0.10
0.1%
2
0.100
0.000 1%
(a) T = 0.8
10% k/| | (%)
100%
1%
(b) T = 1
10% k/| | (%)
100%
(c) T = 1.2
¯ under varying communication payloads (K/|V| in %, with |V| = 32, 000). Fig. 2: Average aggregation bias ∆ 0.08
(Renormalize) (Residual-Mass )
0.07 0.06 0.05
(Renormalize) (Residual-Mass )
0.08
0.06
0.06
0.04
0.04
0.04
0.03 0.02
0.02
0.02
0.01 0.00
0.00
0.00
0.1%
1.0%
k/| | (%)
10.0%
100.0%
(a) T = 0.8
(Renormalize) (Residual-Mass )
0.08
1%
10% k/| | (%)
(b) T = 1
100%
1%
10% k/| | (%)
100%
(c) T = 1.2
Fig. 3: Average acceptance rate bias ∆ᾱ under varying communication payloads (K/|V| in %, with |V| = 32, 000).
B. Simulation results Temperature controls the sharpness of the output vocabulary distribution by rescaling logits before softmax, redistributing probability mass while preserving token ranking [23], [24]. Since vocabulary truncation operates on ranked token probabilities, temperature directly affects the residual mass beyond the top-k tokens; thus, we vary the temperature to evaluate performance under different distribution concentration levels. We compute the average aggregation bias under the two reconstruction strategies as ¯ = E[∆(k) ∆ t ].
(21)
Similarly, for the residual mass and acceptance rate bias, we have " # X (ki ) ϵ̄ = E wi ϵt,i . (22) i∈M
h i (k) ∆ᾱ = E ∆αt .
In Fig. 3, we further see that ∆ᾱ is also bounded by ϵ̄. Note that this bound is not tight, and ϵ̄ under the Residual-Mass Redistribution strategy is not necessarily smaller than that under the Renormalized Top-K Reconstruction strategy (e.g., for K = 320, corresponding to only 0.1% of the vocabulary). Nevertheless, these observations confirm the practicality of compressed transmission. Furthermore, across different temperatures T , even under aggressive compression (e.g., K = 320, i.e., only 1% of the ¯ and ∆ᾱ remain small, demonstrating vocabulary), both ∆ that the proposed scheme achieves favorable communication– quality trade-offs.
(23)
The results are shown in Figs. 2 and 3. In Fig. 2, we observe that for both reconstruction strategies, the average aggregation ¯ is bounded by 2ϵ̄, confirming the validity of Theobias ∆ rem 2. Moreover, the Residual-Mass Redistribution strategy consistently exhibits a smaller bias than the Renormalized Top-K Reconstruction strategy, which aligns with the intuition from Lemma 1: ∆(ki ) t, i = 2ϵt, i(ki ) for the ResidualMass Redistribution strategy, whereas ∆(ki ) t, i ≤ 2ϵt, i(ki ) for the Renormalized Top-K Reconstruction strategy.
VI. C ONCLUSION In this paper, we studied federated LLM inference with speculative decoding, addressing the communication bottleneck inherent in distributed autoregressive generation. By leveraging a top-K compressed transmission scheme combined with two server-side reconstruction strategies, our approach significantly reduces communication overhead while preserving generation fidelity. We theoretically analyzed the robustness of the method, deriving bounds on local reconstruction error, aggregation bias, and acceptance-rate bias, and validated these results through extensive experiments. The findings demonstrate that federated speculative decoding with compressed transmission is both effective and efficient, providing a practical solution for scalable edge deployment of LLMs.
R EFERENCES [1] N. Shlezinger, E. Farhan, H. Morgenstern, and Y. C. Eldar, “Collaborative inference via ensembles on the edge,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 8478–8482. [2] M. Malka, E. Farhan, H. Morgenstern, and N. Shlezinger, “Decentralized low-latency collaborative inference via ensembles on the edge,” IEEE Transactions on Wireless Communications, 2024. [3] Z. Zhou, J. Xie, M. Huang, T. Ouyang, F. Liu, and X. Chen, “Towards federated inference: An online model ensemble framework for cooperative edge ai,” in IEEE INFOCOM 2025-IEEE Conference on Computer Communications. IEEE, 2025, pp. 1–10. [4] S. F. Yilmaz, B. Hasırcıoğlu, and D. Gündüz, “Over-the-air ensemble inference with model privacy,” in 2022 IEEE International Symposium on Information Theory (ISIT). IEEE, 2022, pp. 1265–1270. [5] S. Kumazawa, J. Yu, K. Kawamura, T. Van Chu, and M. Motomura, “Toward improving ensemble-based collaborative inference at the edge,” IEEE Access, vol. 12, pp. 6926–6940, 2024. [6] Y.-C. Yu, C. C. Kuo, Y. Ziqi, C. Yucheng, and Y.-S. Li, “Breaking the ceiling of the llm community by treating token generation as a classification for ensembling,” in Findings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 1826–1839. [7] Z. Chen, J. Li, P. Chen, Z. Li, K. Sun, Y. Luo, Q. Mao, M. Li, L. Xiao, D. Yang et al., “Harnessing multiple large language models: A survey on llm ensemble,” arXiv preprint arXiv:2502.18036, 2025. [8] H. Yun, K. Ki, J. Lee, and E. Yang, “When to ensemble: Identifying token-level points for stable and fast llm ensembling,” arXiv preprint arXiv:2510.15346, 2025. [9] Y. Yao, H. Wu, S. Luo, X. Han, J. Liu, Z. Guo, L. Song et al., “Determine-then-ensemble: Necessity of top-k union for large language model ensembling,” in The Thirteenth International Conference on Learning Representations. [10] J. Fu, Y. Jiang, J. Chen, J. Fan, X. Geng, and X. Yang, “Fast large language model collaborative decoding via speculation,” arXiv preprint arXiv:2502.01662, 2025. [11] Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in International Conference on Machine Learning. PMLR, 2023, pp. 19 274–19 286. [12] C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318, 2023. [13] W. Zhao, W. Jing, Z. Lu, and X. Wen, “Edge and terminal cooperation enabled llm deployment optimization in wireless network,” in 2024 IEEE/CIC International Conference on Communications in China (ICCC Workshops). IEEE, 2024, pp. 220–225. [14] B. Zhu, Z. Chen, L. Zhao, H. Shin, and A. Nallanathan, “Efficient llm inference over heterogeneous edge networks with speculative decoding,” arXiv preprint arXiv:2510.11331, 2025. [15] J. NING, C. ZHENG, and T. Yang, “Dssd: Efficient edge-device deployment and collaborative inference via distributed split speculative decoding,” in ICML 2025 Workshop on Machine Learning for Wireless Communication and Networks (ML4Wireless). [16] G. Zhang, Y. Cai, G. Yu, P. Popovski, and O. Simeone, “Quantizesample-and-verify: Llm acceleration via adaptive edge-cloud speculative decoding,” arXiv preprint arXiv:2507.00605, 2025. [17] S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote large language models,” in 2025 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN). IEEE, 2025, pp. 1–7. [18] C. Zheng and T. Yang, “Communication-efficient collaborative llm inference via distributed speculative decoding,” arXiv preprint arXiv:2509.04576, 2025. [19] C. Zheng, K. Zhang, W. ZHANG, Q. LIU, A. A. Tesfay et al., “Fast collaborative inference via distributed speculative decoding,” Journal of Information and Intelligence, 2026. [20] M. Markatou and Y. Chen, “Non-quadratic distances in model assessment,” Entropy, vol. 20, no. 6, p. 464, 2018. [21] G. Bachmann, S. Anagnostidis, A. Pumarola, M. Georgopoulos, A. Sanakoyeu, Y. Du, E. Schönfeld, A. Thabet, and J. Kohler, “Judge decoding: Faster speculative sampling requires going beyond model alignment,” arXiv preprint arXiv:2501.19309, 2025. [22] O. Bojar, C. Buck, C. Federmann, B. Haddow, P. Koehn, J. Leveling, C. Monz, P. Pecina, M. Post, H. Saint-Amand, R. Soricut, L. Specia, and A. s. Tamchyna, “Findings of the 2014 workshop on statistical machine
translation,” in Proceedings of the Ninth Workshop on Statistical Machine Translation. Baltimore, Maryland, USA: Association for Computational Linguistics, June 2014, pp. 12–58. [Online]. Available: http://www.aclweb.org/anthology/W/W14/W14-3302 [23] M. Renze, “The effect of sampling temperature on problem solving in large language models,” in Findings of the association for computational linguistics: EMNLP 2024, 2024, pp. 7346–7356. [24] L. Li, L. Sleem, G. Nichil, R. State et al., “Exploring the impact of temperature on large language models: Hot or cold?” Procedia Computer Science, vol. 264, pp. 242–251, 2025.