1
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
arXiv:2604.25421v1 [cs.LG] 28 Apr 2026
Changyu Li1 , Shuanghong Huang2 , Jiashen Liu3 , Ming Lei1 , Jidu Xing4 , Kaishun Wu5 , Fellow, IEEE, Lu Wang6 , Senior Member, IEEE, and Fei Luo1
Abstract—Federated fine-tuning provides a practical route to adapt large language models (LLMs) on edge devices without centralizing private data, yet in mobile deployments the training wall-clock is often bottlenecked by straggler-limited uplink communication under heterogeneous bandwidth and intermittent participation. Although parameter-efficient fine-tuning (PEFT) reduces trainable parameters, per-round payloads remain prohibitive in non-IID regimes, where uniform compression can discard rare but task-critical signals. We propose F ED FSTQ, a Fisher-guided token quantization system primitive for communication-efficient federated LLM fine-tuning. F ED -FSTQ employs a lightweight Fisher proxy to estimate token sensitivity, coupling importance-aware token selection with non-uniform mixed-precision quantization to allocate higher fidelity to informative evidence while suppressing redundant transmission. The method is model-agnostic, serves as a drop-in module for standard federated PEFT pipelines (e.g., LoRA) without modifying the server aggregation rule, and supports bandwidth-heterogeneous clients via compact sparse message packing. Experiments on multilingual QA and medical QA under non-IID partitions show that F ED -FSTQ reduces cumulative uplink traffic required to reach a fixed quality threshold by 46× relative to a standard LoRA baseline, and improves end-to-end wall-clock time-to-accuracy by 52%. Furthermore, enabling Fisher-guided token reduction at inference yields up to a 1.55× end-to-end speedup on NVIDIA Jetson-class edge devices, demonstrating deployability under tight resource constraints. Index Terms—Federated learning, large language models, parameter-efficient fine-tuning, communication efficiency, heterogeneous wireless networks, straggler mitigation, token selection, mixed-precision quantization, Fisher information, mobile edge computing.
real-time on-device coding copilots [1]–[5]. Federated Learning (FL) enables such adaptation without centralizing sensitive user data [6]–[8]. In many deployments, secure aggregation or differential privacy further tightens the effective communication budget [9], [10]. However, scaling LLM adaptation to crossdevice settings collides with a fundamental systems constraint: edge accelerators keep improving local compute, while the uplink remains scarce, time-varying, and highly heterogeneous, effectively dictating end-to-end training latency under synchronous coordination [11], [12]. The uplink wall and the straggler problem. In synchronous federated fine-tuning, uplink heterogeneity turns each round into a straggler-limited process where the slowest participating clients dominate wall-clock time [12]. Parameter-Efficient FineTuning (PEFT) reduces the number of trainable parameters and local memory, but it does not eliminate repeated uplink payloads across rounds [13], [14]. Under realistic Non-IID client distributions, distinct and non-uniform updates further amplify synchronization delays and client drift, motivating robust federated optimization corrections [15]–[19]. This picture is consistent with cross-device FL benchmarks and stacks (LEAF, FedScale, Flower), where communication frequently dominates system cost in mobile deployments [20]–[22].
Why blind compression is brittle for LLM adaptation. A standard remedy is gradient compression via quantization or sparsification [23]–[25], often paired with bias correction such as error-feedback [26]. Yet these parameter-centric schemes are largely uniform or magnitude-driven and effectively perform blind compression that ignores token-level semantic sensitivity. I. I NTRODUCTION In language, rare tokens such as negations in clinical text HE deployment of Large Language Models (LLMs) or delimiters in code can be decisive for correctness, but on mobile and edge devices is reshaping personalized uniform policies do not treat them as special. Heuristic token computing, enabling privacy-preserving clinical assistants and pruning developed for Vision Transformers (e.g., DynamicViT, ToMe) [27], [28] is also risky to transplant to NLP, since 1 Changyu Li, Ming Lei, and Fei Luo are with the School of Computing and Information Technology, Great Bay University, Dongguan, low-attention tokens are not necessarily redundant. InferenceChina (e-mail: [email protected]; [email protected]; lu- time LLM quantization methods reduce storage and execution [email protected]). cost [29]–[33], but they do not address the repeated uplink 2 Shuanghong Huang is with Beijing Institute of Technology, Beijing, China payloads that dominate federated fine-tuning. (e-mail: [email protected]).
T
3 Jiashen Liu is with the University of Warwick, Coventry, U.K. (e-mail: [email protected]). 4 Jidu Xing is with City University of Hong Kong (Dongguan), Dongguan 523808, China (e-mail: [email protected]). 5 Kaishun Wu is with the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China (e-mail: [email protected]). 6 Lu Wang is with the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China (e-mail: [email protected]).
Our approach: Fisher-guided communication control. To dismantle the uplink wall, we propose F ED -FSTQ (FisherSpectrum-aware Token Quantization). To the best of our knowledge, F ED -FSTQ is the first framework to leverage Fisher information as a dynamic communication control primitive for federated LLM fine-tuning, rather than treating curvature
2
only as an optimization aid. Curvature-aware signals have To mitigate uplink costs, a large body of gradient compreslong motivated stable optimization (e.g., natural gradient and sion methods has been proposed. Quantization approaches K-FAC) [34], [35] and sensitivity-based pruning under loss- (e.g., QSGD, TernGrad, 1-bit SGD, signSGD) reduce update preserving approximations [36]–[40]. Recent systems increas- precision, often with convergence guarantees under standard ingly pursue lightweight curvature proxies that are compatible stochastic assumptions [23], [45]–[47]. Recent work also with modern workloads and constrained devices. Following this revisits binarization as a learnable compression primitive. direction, we use a token-level Fisher proxy to decide where FedBAT performs binarization-aware local training to reduce limited uplink resources should be spent during local fine- approximation error relative to post-hoc binarization, improving tuning. F ED -FSTQ estimates semantic sensitivity online and communication efficiency while maintaining accuracy [48]. couples (i) importance-aware token selection with (ii) mixed- Error-feedback corrects compression bias and stabilizes precision quantization, allocating high-fidelity bandwidth to convergence in practice [26]. Sparsification communicates load-bearing evidence while suppressing redundant updates. only a subset of significant coordinates (e.g., top-k), exploiting The resulting update improves the signal-to-noise ratio of empirical gradient sparsity [24], [49], [50]. Structured comaggregated adapter updates under Non-IID heterogeneity. pression further leverages low-rank approximations to reduce Drop-in deployability. F ED -FSTQ is a drop-in module for payloads [51]. In FL, FedPAQ combines periodic averaging standard federated PEFT pipelines (e.g., FedAvg+LoRA) [6], with quantization to reduce both communication frequency and [13]. It requires no change to server-side aggregation and payload size [25]. However, scaling parameter-centric schemes to federated remains compatible with practical deployment constraints, including secure aggregation and differential privacy [9], [10]. LLM fine-tuning exposes practical limitations. First, even Results and contributions. We evaluate F ED -FSTQ on mul- aggressively compressed updates can accumulate to prohibitive tilingual QA (Fed-Aya) and medical QA (Fed-Med) [41], traffic over many rounds under straggler-limited synchroniza[42] using a virtual edge testbed that emulates heterogeneous tion [12]. Second, many schemes remain largely uniform 4G/LTE uplinks. F ED -FSTQ reduces cumulative uplink traffic or magnitude-driven and do not explicitly model semantic to reach a fixed quality threshold by 46× relative to Fed-LoRA sensitivity at the token level; in non-IID regimes, this can and improves straggler-limited time-to-accuracy by 52%. On disproportionately attenuate rare but task-critical signals [15]. NVIDIA Jetson edge devices, Fisher-derived masks can also F ED -FSTQ complements this literature by introducing a datacentric control primitive that prioritizes semantically consequenbe reused to accelerate inference, yielding a 1.55× speedup. tial evidence before allocating communication fidelity, thereby The main contributions are: improving straggler-limited time-to-accuracy. • Fisher-guided federated communication control. We formulate token-level Fisher sensitivity as a communication control primitive for federated LLM fine-tuning, enabling B. Federated Fine-Tuning of Large Language Models mixed-precision transmission that preserves semantically Parameter-efficient fine-tuning (PEFT) is the dominant decisive evidence under severe uplink constraints. paradigm for adapting LLMs on resource-constrained devices. • System-level gains under heterogeneity. We demonstrate Techniques such as adapters [52], prefix-tuning [53], prompt substantial reductions in cumulative uplink traffic (46×) tuning [54], and LoRA [13] freeze the backbone and train only and straggler-limited time-to-accuracy (52%) under realis- a small set of parameters, substantially reducing local compute tic Non-IID client distributions and heterogeneous mobile and memory. Adaptive variants such as AdaLoRA allocate uplinks. low-rank capacity under a constrained budget [55]. Quantized • Edge deployability and dual-use benefits. We validate fine-tuning methods such as QLoRA reduce memory pressure feasibility on real edge hardware (NVIDIA Jetson), further by quantizing the frozen backbone while training lowshowing that Fisher estimation overhead is amortized rank adapters [14]. by communication savings and that learned masks benefit While PEFT makes on-device training feasible, it does not both training communication and inference efficiency. remove the uplink bottleneck. Systems evidence indicates that communication often dominates end-to-end latency in mobile II. R ELATED W ORK deployments due to bandwidth variability and intermittent A. Communication-Efficient Federated Learning connectivity [5], [12]. Moreover, non-IID distributions exacerCommunication is a first-order systems bottleneck in fed- bate client drift and instability, motivating robust optimization erated learning (FL) [6]–[8], particularly in mobile and edge corrections such as FedProx, SCAFFOLD, FedNova, and deployments where uplinks are heterogeneous, participation adaptive federated optimization [15]–[19]. Unlike methods is intermittent, and synchronous rounds become straggler- that primarily reduce the number of trainable parameters, limited [5], [11], [12]. This view is reinforced by early system F ED -FSTQ targets the information density of the transmitted and optimization analyses emphasizing constraints beyond the update by prioritizing token-level evidence and allocating mixed datacenter [43], [44], and is operationalized in benchmarks precision so that scarce uplink capacity is spent on updates and stacks such as LEAF, FedScale, Flower, and TensorFlow most consequential for downstream correctness. Federated [20]–[22]. When privacy requirements are stringent, Relatedly, LLM quantization has been extensively studied pairing FL with secure aggregation or differential privacy for efficient deployment, including integer-matrix kernels and further tightens the effective communication budget [9], [10]. post-training quantization methods (e.g., LLM.int8(), GPTQ,
3
SmoothQuant, AWQ, and ZeroQuant) [29]–[33]. These works primarily target inference-time acceleration and memory reduction, whereas F ED -FSTQ focuses on federated uplink efficiency under repeated update exchange.
to federated LLM adaptation by treating communication control as a sensitivity-aware primitive for robust learning in heterogeneous networks. III. P ROBLEM F ORMULATION AND S YSTEM M ODEL
C. Curvature-Aware Optimization and Compression
A. Federated Low-Rank Adaptation (Fed-LoRA)
Fisher information and other curvature-aware signals provide We consider a cross-device federated system with a central a principled alternative to magnitude heuristics for modeling server and K heterogeneous edge clients (e.g., mobile SoCs saliency. In optimization, natural gradient methods and scalable and Jetson-class modules), indexed by C = {1, . . . , K} [5]–[7], k curvature approximations such as K-FAC exploit local curva- [12]. Client k holds a private dataset Dk = {(xi , yi )}N i=1 drawn ture structure to improve convergence [34], [35]. In model from a client-specific distribution Pk , capturing the statistical compression, classic second-order analyses (e.g., Optimal heterogeneity (non-IID) endemic to practical deployments [7], Brain Damage and Optimal Brain Surgeon) use Hessian-based [8], [15]. At communication round t, the server samples a approximations to identify prunable parameters with minimal subset St ⊆ C (partial participation), and each selected client loss increase [36], [37]. Fisher information has also served as performs local PEFT updates; such sampling and heterogeneity a sensitivity proxy in continual learning (e.g., EWC) to protect are widely studied and benchmarked in real FL stacks (e.g., critical weights under nonstationarity [38]. More recent pruning LEAF, FedScale, Flower, TFF) [20]–[22]. work emphasizes preserving gradient flow or sensitivity at We adopt Low-Rank Adaptation (LoRA) [13]. For Transinitialization (e.g., SNIP and GraSP) [39], [40]. Complementary former layer l ∈ {1, . . . , L} with frozen weights W0(l) ∈ lines study end-to-end compression pipelines that combine Rdout ×din , LoRA injects trainable low-rank matrices B(l) ∈ pruning and quantization [56], [57], sparsity induction via Rdout ×r and A(l) ∈ Rr×din with r ≪ min(dout , din ). For input stochastic regularization [58], and structural perspectives such activation h(l) , the layer output is in as the lottery ticket hypothesis [59]. αlora (l) (l) (l) (l) (l) (l) (1) hout = W0 hin + B A hin , F ED -FSTQ draws inspiration from this tradition but pivots r the application domain in two key ways: it targets federated (l) (l) L uplink efficiency rather than inference-time compression, and it where αlora is the LoRA scaling. Let Θ ≜ {A , B }l=1 lifts sensitivity modeling from the parameter level to the token denote all trainable adapter parameters (while the backbone is level. In this context, F ED -FSTQ uses Fisher-guided sensitivity frozen, optionally quantized as in QLoRA [14]). The global not merely as a static pruning criterion, but as a dynamic rate– objective is the weighted empirical risk K distortion control signal that modulates communication fidelity X Nk Lk (Θ; Dk ), (2) min F(Θ) ≜ under client and network heterogeneity. Θ N k=1 P D. Token Pruning and Adaptive Computation where N = k Nk and Lk is the local causal LM loss (e.g., Adaptive computation has been widely explored in Trans- next-token cross-entropy) instantiated on Transformer language former architectures [4], especially in Vision Transformers modeling objectives [1]–[4]. Local update and aggregation (FedAvg-compatible).: At where patch tokens are naturally discrete [60]. Methods such (0) round t, each participating client initializes Θk,t ← Θt and as DynamicViT, TokenLearner, ATS, A-ViT, EViT, and token merging (ToMe) improve inference speed by dropping or runs E local steps: merging redundant patches using attention- or confidence(τ +1) (τ ) b k Θ(τ ) ; ξ (τ ) , Θk,t = Θk,t − η ∇L τ = 0, . . . , E − 1, k,t k,t driven heuristics [27], [28], [61]–[64]. (3) However, directly transplanting these heuristics to federated (τ ) b is a stochastic where ξ denotes a local minibatch and ∇ k,t LLM fine-tuning is non-trivial. Unlike image patches, text tokens exhibit high structural density; tokens that appear low- gradient estimator. The (uncompressed) model delta is (E) attention (e.g., negations in clinical text or delimiters in code) ∆Θk,t ≜ Θk,t − Θt . (4) can be pivotal for correctness. Under non-IID heterogeneity, inconsistent token dropping or coarse treatment of such The server updates via a FedAvg-style weighted aggregatokens can degrade semantic reliability and compound client tion [6]: X drift [15]. F ED -FSTQ departs from activation-driven heuristics g k,t , Θt+1 = Θt + wk,t ∆Θ by adopting a Fisher-guided sensitivity signal. Using Fisher k∈St :ak,t =1 (5) information as a proxy for how token evidence shapes the loss Nk wk,t ≜ P . landscape, it prioritizes and allocates precision to tokens that j∈St :aj,t =1 Nj are most consequential, preserving structural integrity under high compression ratios. g k,t is the decompressed client update and ak,t where ∆Θ Broader Context on Reliable Edge Systems. Our focus on is an availability indicator (defined below). This formulation preserving rare but safety-critical evidence under constraints remains compatible with FedOpt-style server optimizers and resonates with broader trends in reliability-aware edge analytics heterogeneity-aware corrections (e.g., FedProx, SCAFFOLD, under nonstationarity [65]. F ED -FSTQ extends this philosophy FedNova, FedDyn) [15]–[19].
4
B. Stochastic Uplink Channel and Straggler-Limited Latency Mobile and edge FL is often constrained by uplink communication under bandwidth heterogeneity and intermittent participation [5], [11], [12]. We study synchronous rounds, where completion time is governed by the slowest responding client, a dominant effect in cross-device FL systems [12], [21]. At round t, each participating client k ∈ St transmits a compressed message mk,t ≜ Enc(Qk,t (∆Θk,t )) ,
(6)
where Qk,t denotes (possibly sparse, mixed-precision) quantization/compression, and Enc(·) packs values together with required metadata (e.g., masks/indices, quantization scales, bitwidth tags) [23]–[25], [44]. Let bits(mk,t ) denote the payload length. A convenient decomposition that separates structure and value costs is X bits(mk,t ) = bits(Ik,t ) + bk,t (j) + bits(side info), {z } | | {z } j∈Ik,t scales/tags indices/mask | {z } quantized values
(7) where Ik,t is the set of transmitted coordinates and bk,t (j) ∈ Z≥0 is the bit-width assigned to coordinate j (with bk,t (j) = 0 implying pruning). Eq. (7) covers uniform quantization (fixed b), top-k sparsification, and mixed-precision signaling, and it is compatible with bias-correction mechanisms such as errorfeedback and memory [26], [49]. Throughput and intermittent participation.: Let Rk,t be the effective uplink throughput available to client k at round t. We treat {Rk,t }t as a stochastic process capturing time-varying cellular/WiFi conditions and cross-traffic: bits(mk,t ) (k) (k,t) Tcomm = , Rk,t ∼ Pbw . (8) Rk,t We model intermittent participation with an availability indicator ak,t ∈ {0, 1} (temporary offline or dropped round), consistent with cross-device deployments [11], [12]. Network profiles used in evaluation (disambiguating “4G LTE”).: To avoid conflating distinct uplink settings under a single label, we use two explicit LTE profiles throughout the paper and refer to them by name in all table and figure captions . Profile A fixes a controlled rate to isolate payload effects in per-round breakdown analysis. Profile B draws Rk,t from a client-specific distribution with a slow tail to model stragglers in end-to-end time-to-accuracy experiments. Under (k,t) Profile A, Tcomm follows directly from Eq. (8) with a fixed rate, while under Profile B the synchronous latency becomes straggler-dominated through Eq. (9). Straggler-limited round time.: In synchronous FL, the per-round wall-clock latency is bits(mk,t ) (t) (t) (k,t) Tround = Tsrv + max Tcomp + , (9) k∈St : ak,t =1 Rk,t (k,t)
(t)
where Tcomp is local computation time and Tsrv accounts for server-side aggregation overhead [12], [21]. Eq. (9) makes the key systems lever explicit: under heterogeneous uplinks with a slow tail, reducing bits(mk,t ) improves both average and tail round latency, and therefore time-to-accuracy. This remains true
when local fine-tuning is made feasible by PEFT/QLoRA [13], [14], and when privacy mechanisms further tighten effective communication budgets [9], [10]. C. Problem Statement: Fisher-Weighted Rate–Distortion Optimization Current communication-efficient FL schemes compress model updates in a largely parameter-centric manner, treating coordinates as interchangeable under a fixed bit budget and relying on uniform or magnitude-driven rules [23]–[25], [44]. Such content-agnostic compression is poorly matched to structure-sensitive language tasks: semantic correctness can hinge on rare but decisive cues (e.g., negations, delimiters, or domain entities), for which simple statistics such as attention or coordinate magnitude are unreliable proxies of importance. Under non-IID client distributions, indiscriminate compression can disproportionately attenuate long-tail evidence and amplify client drift, thereby reducing the effective signal-to-noise ratio of the aggregated update [7], [15]. In federated LLM fine-tuning, where round latency is straggler-limited (Eq. (9)) and uplink throughput is heterogeneous [11], [12], this misalignment becomes a systems bottleneck: reducing payload size alone does not guarantee improved time-to-accuracy if the remaining bits fail to preserve optimization-critical directions. We therefore formulate communication control as a semantic-aware rate–distortion optimization problem: derive a compression policy that minimizes uplink bits while constraining the distortion measured in a curvature- and sensitivity-aware metric induced by Fisher information [34], [35]. This ensures that scarce bandwidth is allocated to update components that are most consequential for the loss landscape and downstream behavior, rather than being spread uniformly across parameters. Fisher-weighted distortion.: Formally, we quantify sensitivity using a local curvature-aware metric induced by the Fisher information matrix (FIM). The FIM defines the canonical quadratic form associated with the local geometry of probabilistic models and underpins curvature-aware optimization methods such as natural gradient and tractable approximations (e.g., KFAC) [34], [35]. Let F̂t denote an empirical approximation of the FIM at round t. Given the high dimensionality of LLMs, we adopt a diagonal surrogate for tractability, which is standard in scalable curvature approximation and has been widely used as a sensitivity proxy in continual learning [35], [38]. For a client update ∆Θk,t and its compressed counterg k,t , we define the Fisher-weighted distortion as the part ∆Θ quadratic form g k,t ≜ (ek,t )⊤ F̂t (ek,t ) , DF̂t ∆Θk,t , ∆Θ (10) g k,t . ek,t ≜ ∆Θk,t − ∆Θ Under the diagonal assumption, Eq. (10) simplifies to a separable sum: X f k,t (j) 2 . DF̂t = F̂t (j) ∆θk,t (j) − ∆θ (11) j
This formulation explicitly weights compression error by the estimated sensitivity F̂t (j), penalizing distortions more
5
strongly along high-curvature directions. It mirrors classic Specifically, increasing the precision of coordinate j from second-order saliency criteria in pruning and compression b to b + ∆b is beneficial only when the distortion drop per (OBD/OBS) [36], [37] and aligns with modern sensitivity-based additional bit exceeds the bit price 1/λ, pruning objectives (e.g., SNIP and GraSP) that emphasize E ek,t (j; b)2 − E ek,t (j; b + ∆b)2 preserving gradient flow and training dynamics [39], [40]. λ F̂t (j) ≥ 1, (15) ∆b These connections motivate using Fisher-weighted distortion as the fidelity constraint in our semantic-aware rate–distortion (where the inequality becomes approximate when b is restricted formulation, where bits should be preferentially spent on to a discrete set of bit-widths). This condition naturally induces mixed-precision allocation: Fisher-critical coordinates sensitivity-critical directions. Rate–Distortion Objective.: With the Fisher-weighted receive higher fidelity, while low-sensitivity ones are coarsely distortion in Eq. (10) as our fidelity criterion, we cast communi- quantized or pruned. Through Eq. (7)–(9), this bit-allocation cation control as a semantic-aware rate–distortion optimization principle directly targets the straggler bottleneck by reducing problem: minimize expected uplink payload while constraining bits(mk,t ) without disproportionately distorting sensitivityexpected distortion in sensitivity-critical directions. Taking critical directions that drive convergence and semantic quality. Fed-FSTQ approximation (token-level sensitivity).: F ED expectation over the randomness of local minibatches, client FSTQ provides an efficient on-device approximation to the subsampling/availability, and stochastic channel states, we rate–distortion objective in Eq. (13). Directly forming F̂t formulate h i in full parameter space is prohibitive for LLMs, and even min E bits(mk,t ) diagonal curvature estimates can be expensive to maintain π h i at scale. Instead, F ED -FSTQ leverages a lightweight tokeng k,t s.t. E DF̂t ∆Θk,t , ∆Θ ≤ ϵ, (12) level sensitivity proxy: for each token, it measures the squared gradient with respect to the corresponding input embedding, where π is a (possibly data-dependent) compression policy which serves as an inexpensive estimate of how strongly mapping the local update and system state to a sparse support that token contributes to the local loss geometry. We then and a mixed-precision allocation, π : ∆Θk,t , statek,t 7→ couple (i) sensitivity-aware token selection (inducing structured Ik,t , {bk,t (j)}j∈Ik,t . This abstraction subsumes standard sparsity in the transmitted update) with (ii) mixed-precision quantization/sparsification protocols and their federated variquantization, allocating higher fidelity to tokens that are ants [23]–[25], [44]. Equivalently, using a Lagrangian relaxmore influential while aggressively suppressing redundant ation, we consider evidence. Operationally, this implements the marginal condition h i g k,t , min E bits(mk,t ) + λ · DF̂t ∆Θk,t , ∆Θ (13) in Eq. (15) at the granularity of token evidence, aiming to π maximize Fisher-critical sensitivity preserved per transmitted where λ > 0 controls the rate–distortion trade-off (i.e., the bit. distortion penalty per transmitted bit). By converting semantic sensitivity into a systems control Separable Bit Allocation (System Feasibility).: Under the signal, F ED -FSTQ reduces bits(mk,t ) in Eq. (7) and thus diagonal FIM approximation, the Fisher-weighted distortion in directly improves straggler-limited round time in Eq. (9), Eq. (11) decomposes additively across coordinates. Moreover, while avoiding indiscriminate removal of rare but task-critical with standard sparse message packing (Eq. (7)), the uplink structure. payload can be expressed as a structure/metadata overhead Roadmap. In the next section, we present the concrete plus the sum of per-coordinate value bit-widths. Consequently, algorithm and protocol that instantiate this principle with the Lagrangian objective in Eq. (13) admits the separable form negligible overhead on edge clients. In Sec. VI, we show that this design yields a 6.8× reduction in per-round endbits(Ik,t ) + bits(side info) min to-end latency, a 46× reduction in cumulative uplink, and a {Ik,t , bk,t (j)} | {z } structure/metadata 52% improvement in straggler-limited time-to-accuracy under X 2 , realistic heterogeneous networks. + bk,t (j) +λ F̂t (j) E ek,t (j; bk,t (j)) | {z } | {z } j∈Ik,t value bits Fisher-weighted distortion IV. F ED -FSTQ: M ETHODOLOGY AND S YSTEM D ESIGN (14) We now present F ED -FSTQ, a system primitive that opwhere bk,t (j) ∈ Z≥0 denotes the chosen bit-width (with erationalizes the Fisher-weighted rate–distortion objective in bk,t (j) = 0 representing pruning) and ek,t (j; b) ≜ ∆θk,t (j) − Sec. III. As shown in Fig. 2, F ED -FSTQ sits between the f k,t (j; b) is the induced coordinate-wise compression error local PEFT training loop and the bandwidth-limited uplink, ∆θ under bit-width b. Eq. (14) is pivotal for edge deployment: it acting as a semantic reliability gate that (i) estimates which reduces a high-dimensional coupled design into O(d) scalar token-level evidence is load-bearing for the loss geometry, decisions (plus a structured indexing choice), enabling efficient and (ii) allocates transmission bits accordingly. In contrast to on-device approximation under tight compute and memory activation- or attention-only pruning heuristics [28], [64], F ED budgets. FSTQ grounds token prioritization in information geometry Eq. (14) also yields a rigorous systems interpretation: a via a lightweight Fisher proxy, and couples it with mixedbit should be transmitted only if its marginal reduction in precision quantization and sparse message packing that remains Fisher-weighted distortion justifies its communication cost. compatible with standard FedAvg aggregation [6]. The pipeline
6
Fig. 1. The Uplink Bottleneck in Federated LLM Fine-Tuning. Under stochastic channel conditions (Rk,t ), standard Fed-LoRA (red arrows, dense blocks) suffers from straggler delays per Eq. (9). F ED -FSTQ (green arrows, sparse blocks) reduces bits(mk,t ) via Fisher-guided semantic compression, enabling efficient transmission even under constrained and heterogeneous uplinks. The straggler client (highlighted with clock icon) dominates the round completion time in the baseline scenario, while F ED -FSTQ mitigates this bottleneck through aggressive yet semantically-aware compression.
consists of three stages: (1) Riemannian sensitivity estimation, (2) Fisher-weighted rate–distortion bit allocation, and (3) sparse uplink and aggregation.
increase in Token Recall (Sec. VI-E), consistent with Fisherguided pruning acting as a semantic denoiser rather than a purely lossy compressor.
A. Stage 1: Riemannian Sensitivity from a Token-Level Fisher Proxy
B. Token-to-Parameter Fisher Coupling
A critical design choice in F ED -FSTQ is that token sensiIn non-IID federated fine-tuning, a client update intermixes tivities are used during training to shape the parameter-wise (i) transferable signal and (ii) locally idiosyncratic variation. Fisher statistics that determine uplink sparsification and mixed F ED -FSTQ extracts a sensitivity signal directly from the loss precision. Therefore, the token mask is not an inference-only geometry using Fisher information, but in a token-centric and mechanism; it directly affects which adapter coordinates are transmitted and at what precision. edge-feasible manner. a) Gradient-mediated coupling via a token-filtered objeca) Token-level sensitivity.: Consider a training sequence de tive.: Let {zi }Ti=1 ∈ {0, 1}T denote the token mask derived of length L with tokens {ti }L and embeddings e ∈ R . At i i=1 client k and local step s, we define the instantaneous token from Stage 1 (Top-K by the EMA score in Eq. (17)); a soft weighting variant replaces zi with nonnegative weights wi . We sensitivity as do not construct a heuristic token-to-parameter lookup table. 2 gk,s (i) ≜ ∥∇ei Lk,s ∥2 , (16) Instead, coupling is realized by gradient reweighting inside where Lk,s is the local loss for the minibatch at step s. This the local PEFT optimization. Concretely, for a minibatch ξ T quantity is an empirical Fisher proxy in embedding space: it with token-wise losses {ℓi }i=1 on target positions, we form a measures how strongly the loss reacts to perturbations of token token-filtered objective ti at the current iterate, and therefore highlights structurally T T X X e ξ) = decisive evidence (e.g., clinical negations and code delimiters) L(θ; zi ℓi (or Le = wi ℓi ). (18) that attention magnitude can underweight. i=1 i=1 b) Temporal smoothing (variance control).: Because The adapter gradient used for the local update is computed gk,s (i) is noisy under minibatch SGD, we apply an exponential e so the gradient statistics, and hence the Fisher estimate, from L, moving average (EMA), are explicitly token-coupled. b) Token-coupled Fisher that drives support and mixed Sk,s (i) = ρ Sk,s−1 (i) + (1 − ρ) gk,s (i), ρ ∈ (0, 1), (17) precision.: Using the same diagonal approximation as Eq. (20), with ρ = 0.9 by default. The EMA serves two roles: it we accumulate Fisher from token-filtered gradients: stabilizes token ranking under stochastic gradients and provides !2 e ∂ L a predictable control signal for subsequent bit allocation. Fb(j) ← ρ Fb(j) + (1 − ρ) , (19) Empirically, this stabilizing effect aligns with the observed ∂θj
7
to reflect which adapter coordinates matter under the selected token evidence. This yields an implicit, model-consistent coupling that controls uplink support and bit-width allocation without manual grouping. C. Stage 2: Fisher-Weighted Rate–Distortion Bit Allocation We next instantiate the rate–distortion objective in Sec. III with a hardware-aligned mixed-precision allocation rule. Let ∆θ denote the client-side LoRA update vector to be transmitted at the end of local training (for notational simplicity, we omit b denote a diagonal empirical Fisher approximation (k, t)). Let F in parameter space, believed sufficient for importance ranking in large models [35], [38]: X b≜ F (∇θ Lk,s ⊙ ∇θ Lk,s ) , (20) s
where ⊙ is elementwise product. We define a coordinate-wise Fisher-weighted importance score uj ≜ Fb(j) · (∆θj )2 .
Fig. 2. System Architecture of F ED -FSTQ. F ED -FSTQ decouples transmission fidelity from parameter magnitude by allocating bits according to Fisherguided sensitivity. (1) Sensitivity estimation: During standard backpropagation, each client computes squared gradients w.r.t. input embeddings as a tokenlevel Fisher proxy [35]. (2) Mixed-precision allocation: A Fisher-weighted rate–distortion policy assigns discrete bit-widths (e.g., 0/2/4/16-bit) to update coordinates, preserving load-bearing evidence while pruning low-impact components. (3) Sparse uplink & aggregation: Clients transmit a packed sparse message (indices/masks + bit-tags + values); the server dequantizes and aggregates via FedAvg-compatible weighted averaging.
(21)
Intuitively, uj is large when a coordinate both changes substantially and lies in a high-sensitivity direction. a) From continuous optimum to discrete hardware precisions.: In high-rate quantization, the optimal bit allocation under a quadratic distortion metric scales logarithmically with importance (a classical rate–distortion principle) [23]: u 1 j b⋆j ∝ log2 , (22) 2 λ for a Lagrange multiplier λ controlling the distortion budget. Edge hardware, however, supports a small set of efficient precisions. We therefore discretize to B = {0, 2, 4, 16} using percentile thresholds over the batch distribution of {uj }: 16 (FP16) if uj ≥ Phigh (u), 4 (INT4) if P (u) ≤ u < P mid j high (u), bj = (23) 2 (INT2) if P (u) ≤ u < P (u), low j mid 0 (pruned) otherwise. This policy concentrates the uplink budget on coordinates that contribute the most Fisher-weighted information per bit, yielding a practical approximation to the constrained RDO problem while maintaining deployment-friendly kernels. D. Stage 3: Sparse Uplink, Aggregation Compatibility, and Edge Feasibility a) Message format and server compatibility.: Client k transmits a packed sparse message f k,t (j)}j∈I mk,t = Ik,t , {bk,t (j)}j∈Ik,t , {∆θ , (24) k,t
where θj indexes adapter coordinates. Through Eq. (19), tokens e reducing the accumuwith zi = 0 contribute no gradient to L, b lated F (j) for coordinates mainly activated by low-sensitivity tokens. The resulting importance score uj = Fb(j) (∆θj )2 (Eq. (21)) therefore decreases, pushing such coordinates below containing indices (or a compressed mask), per-coordinate bitthe allocation thresholds in Eq. (23) and assigning them either tags, and quantized values. The server applies dequantization low precision or bj = 0 (pruned). Conversely, coordinates and performs FedAvg-compatible weighted aggregation: required to fit load-bearing tokens retain large token-filtered X Nk θt+1 = θt + η Dequant(mk,t ), (25) gradients, high Fb (j), and receive higher precision. N k∈S t c) Why explicit token parameter mapping is unnecessary.: In Transformers, each token contributes gradients to shared which preserves the modularity of existing FL stacks and can projections and MLP blocks across layers. Token filtering be composed with standard system components (e.g., secure modifies these gradients directly, allowing the Fisher estimate aggregation) [9].
8
Input Sequence
Attention-Driven Pruning(Fed-ToMe)
Fisheries-Guided Pruning(Fed-FSTQ)
After Pruning (Removed Tokens)
Syntax Error
Semantically Valic
Removes critical structural tokens
Remove critical structural tokens (if, :, indentation) Logic Broken
Preserves load-bearing tokens
Structural tokens Preserved (if, :, return) Syntax valic + Compressed
Fig. 3. Fisher vs. Attention Heatmap. Attention may emphasize high-frequency connectors, whereas the Fisher proxy highlights structurally decisive tokens whose removal breaks logical validity, motivating high-fidelity retention.
b) Why the overhead is dominated by uplink savings.: F ED -FSTQ is designed to be backprop-aligned: both the token Fisher proxy in Eq. (16) and the diagonal Fisher accumulation in Eq. (20) reuse gradients already computed for SGD. The additional work is primarily (i) a per-token ℓ2 reduction and (ii) thresholding over {uj }. In practice, this adds a modest compute increment (e.g., +0.85s per round in our Jetson testbed), while reducing communication time by orders of magnitude when Tcomm ≫ Tcomp , which is the typical mobile regime (Sec. VI-B).
E. Client Algorithm Algorithm 1 summarizes the client-side procedure. Importantly, sensitivity estimation is performed on-the-fly during local SGD and does not require extra forward/backward passes.
V. E XPERIMENTAL S ETUP
Algorithm 1 F ED -FSTQ Client Protocol (LoRA/QLoRACompatible) Require: Global adapter Θt , data Dk , local steps S, EMA decay ρ, mask interval H, bit-set B = {0, 2, 4, 16}, uplink budget Bmax , ratio rtok Ensure: Compressed uplink message mk,t 1: Phase 1: Token-Guided Local PEFT & Fisher Tracking 2: Θk ← Θt ; F̂ ← 0; ĝ ← 0; zi ← 1 (∀i) 3: for s = 1, . . . , S do 4: Sample minibatch ξs = (x, y) ∼ Dk , T ← |x| 5: if s mod H = 0 then Refresh Step: Use full gradients for stable scoring & update. 6: 7: 8:
gΘ , {∇ei L} ← Backward(Lfull (Θk ; ξs )) Update token scores: gi ← ∥∇ei L∥22 ; ĝ ← ρĝ+(1−ρ)g
Update mask for next interval: z ← TopK(ĝ, ⌈rtok T ⌉) else Masked Step: Use token-pruned gradients for efficiency. Compute per-token losses P {ℓi } excluding padding Form masked loss L̃ ← Ti=1 zi ℓi {Token-to-Param Coupling} 14: Backward to obtain token-coupled grads gΘ ← ∇Θ L̃ 15: end if 16: Update adapters: Θk ← Optimizer(Θk , gΘ ) 17: Accumulate Fisher: F̂ ← ρF̂ + (1 − ρ) gΘ ⊙ gΘ 18: end for 19: Phase 2: Fisher-Weighted Bit Allocation 20: ∆Θk,t ← Θk − Θt 2 21: Calculate importance: uj ← F̂(j) ∆Θk,t (j) ∀j 22: Sort {uj } descending 23: Greedy allocation with metadata accounting: P 24: Choose bj ∈ {16, 4, 2, 0} such that j cost(j, bj ) ≤ Bmax 25: Phase 3: Sparse Uplink g k,t (j) ← Qb (∆Θk,t (j)) 26: Ik,t ← {j | bj > 0}; ∆Θ j g 27: mk,t ← Ik,t , b[Ik,t ], ∆Θk,t [Ik,t ] 28: return mk,t 9: 10: 11: 12: 13:
We evaluate F ED -FSTQ as a mobile/edge systems primitive for federated PEFT of LLMs, targeting the practical regime where end-to-end wall-clock time is dominated by stragglerlimited synchronous rounds under heterogeneous uplink bandwidth and intermittent participation [5], [11], [12], [43]. In this regime, communication control is not merely a bandwidthsaving knob: it directly determines tail round latency and thus time-to-accuracy. Accordingly, our evaluation focuses on three system-critical dimensions. First, robustness under statistical heterogeneity (non-IID partitions) and client drift, which are known to stress both convergence and stability in cross-device FL [7], [15]–[19]. Second, uplink efficiency and straggler-limited time-to-accuracy under bandwidth heterogeneity and partial A. Workloads, Datasets, and Non-IID Partitioning participation, reflecting real mobile deployments [11], [12]. Third, semantic reliability on structure-sensitive workloads, We follow a federated LLM evaluation discipline grounded where rare but decisive tokens (e.g., negations, delimiters, and in widely used cross-device FL benchmarks and frameworks, domain entities) can be load-bearing and thus disproportionately which consistently emphasize that statistical heterogeneity, harmed by indiscriminate compression. system heterogeneity, and partial participation are the default
9
operating regime rather than corner cases [20]–[22]. We instantiate three workloads that stress complementary aspects of federated LLM adaptation under edge constraints: multilingual generalization, domain robustness under privacy-sensitive data, and structure-sensitive generation. Unless stated otherwise, all methods use the backbone’s native tokenizer with identical sequence-length truncation to ensure tokenization-level comparability.
B. Federated Protocol and Baselines
We compare F ED -FSTQ against baselines representing three complementary paradigms for reducing end-to-end time-toaccuracy: (i) Uncompressed Federated PEFT (reference), (ii) Parameter-Centric Compression (quantization and sparsification), and (iii) Heuristic Data-Centric Reduction (adaptive computation). a) (I) Uncompressed Federated PEFT Reference.: • FedAvg-LoRA: Standard FedAvg aggregation [6] applied a) Multilingual instruction/QA (Fed-Aya).: We conto LoRA adapter updates without compression. This serves struct a multilingual FL workload from the Aya instructionas our primary uncompressed reference, contextualizing tuning corpus, using a controlled subset of languages gains over widely used parameter-efficient adaptation {ar, en, es, fr, pt, ru, te, zh} to match our evaluation mechanisms [13], [52]. suite. To emulate extreme client heterogeneity, we generate b) (II) Parameter-Centric Communication Compression.: per-client language-mixture proportions from a Dirichlet (LDAThese methods reduce uplink payloads by compressing adapter style) prior with concentration αdir = 0.1, and sample each updates in parameter space, typically using coordinate- or client’s local data accordingly, yielding linguistic silos where magnitude-based criteria without explicitly modeling tokeneach client is dominated by 1–2 languages. Such low-α level semantic sensitivity. Dirichlet splits are a standard stress-test for non-IID instability • QSGD [23]: Stochastic quantization applied to LoRA and client drift in federated optimization [7], [8], [15], [16]. updates. We tune the effective bit-width to match the This workload evaluates whether federated aggregation can target uplink budget. preserve diverse tokenization patterns and grammatical structure • FedPAQ [25]: Periodic averaging combined with quantizaunder tight communication budgets. tion to reduce both communication frequency and payload b) Medical question answering (Fed-Med).: We derive a size. domain-specialized workload from PubMedQA [41], represen• Top-k Sparsification: A standard magnitude-based basetative of privacy-sensitive clinical scenarios where centralizing line that retains only the largest-magnitude coordidata is often infeasible. In such deployments, privacy-enhancing nates [24], [49]. We include this as the representative mechanisms (e.g., secure aggregation or differential privacy) coordinate-selection method and account for its indexing may be required, further tightening the effective communication overhead (bitmap/CSR-style side information) in the full budget [9], [10]. Clients are partitioned by medical subtopics payload. when metadata is available; otherwise, we apply text-derived • Context (Representative selection): We do not separately topic clustering to construct pseudo-categories, and then impose evaluate additional gradient-compression families such a Dirichlet split with αdir = 0.1 to induce long-tail entity as TernGrad, signSGD, or low-rank compressors (e.g., skew. This setting directly tests whether compression preserves PowerSGD) [45], [47], [51], as QSGD (quantization) and rare medical entities and logically decisive operators (e.g., Top-k (sparsification) already cover the two dominant negations) that are pivotal for diagnostic correctness. For parameter-centric design axes under matched uplink completeness, Appendix reports additional domain-robustness budgets. Related variants (including error-feedback and results on MedQA [42]. other encoding schemes) are discussed in §II [26], [44]. c) (III) Data-Centric Token Reduction (Heuristic Comc) Code-style generation (Fed-Code).: We fine-tune on petitor).: CodeAlpaca-20k and evaluate functional correctness using • Fed-ToMe [28]: An attention-driven token mergPass@1 under deterministic decoding. This workload is ing/pruning baseline adapted to federated PEFT. This intentionally adversarial for communication-efficient FL: lowis our direct heuristic competitor, testing whether Fisherfrequency syntactic tokens (e.g., delimiters, indentation cues, guided sensitivity is superior to attention-only scoring and type constraints) are decisive for correctness yet are easily under non-IID heterogeneity (e.g., DynamicViT and corrupted by indiscriminate compression, making it a stringent EViT) [27], [60], [64]. test of semantic reliability. d) Controls for Optimizer and Robustness.: To disd) Clients and participation.: Unless stated otherwise, entangle communication control from optimization correcwe simulate K = 100 clients and sample |St | = 10 clients tions, we additionally report: (i) Adaptive Optimizers per round (10% participation), consistent with cross-device (FedAdam/FedYogi) [18] to verify robustness to server update synchronous FL protocols and system analyses [6], [7], [12]. rules; and (ii) Drift Corrections (FedProx/SCAFFOLD) on Each selected client performs E local update steps with batch severe non-IID splits [15], [17]. These are treated as orthogonal size B and maximum sequence length Lmax , held identical controls rather than primary communication baselines. across methods to isolate the impact of communication control. e) Fair Comparison Protocol.: For all compressed methTo eliminate confounding from client sampling, all methods ods, we tune hyperparameters to match comparable uplink share the same sampled client sequence {St } under fixed budgets (bits/client/round). Reported communication volume random seeds. includes the full payload (values + indices/masks + metadata),
10
since indexing overhead can dominate at extreme sparsity [24], [44].
d) Hardware measurements (deployability).: We measure on-device inference latency (end-to-end) with warm-up and repeated trials, and report energy-per-request when applicable. These measurements complement inference-oriented quantiC. Virtual Edge Testbed and System Metrics zation work [29], [30], while our primary focus is uplink To assess deployability, we build a virtual edge testbed efficiency. that captures mobile constraints, including heterogeneous e) Task and semantic reliability metrics.: Beyond standard uplinks, stragglers, and intermittent participation [5], [11], task quality metrics (ROUGE-L, METEOR, Pass@1), we report [12]. We use synchronous rounds consistent with production- Token Recall, defined as the fraction of top-p Fisher-sensitive inspired FL systems, where round completion is governed by tokens (identified from an uncompressed reference run) that stragglers [12]. remain retained after compression. This metric tests whether a method preserves load-bearing evidence rather than only a) Dual uplink profiles (disambiguating “4G LTE”).: We use two LTE uplink profiles and refer to them by name in reducing bits. all tables and figures. Controlled LTE-20Mbps fixes Rk,t ≡ 20 Mbps for all clients and rounds, and is used only for perround breakdown analysis to isolate the impact of payload D. Implementation Details size. Heterogeneous LTE (straggler-tail) draws Rk,t from a) Backbones and training.: We use decoder-only Transa client-specific distribution with a slow tail (e.g., 0.5–2 former backbones from the LLaMA family (Llama-2-7B Mbps stragglers) and is used for wall-clock time-to-accuracy and Llama-3-8B) [2], following standard pretraining and experiments, where synchronous latency is dominated by adaptation practices [1]. To isolate the impact of communication stragglers. control, all methods fine-tune the same base model with Unless stated otherwise, clients independently drop out with identical tokenizers, maximum sequence length, client sampling schedule, and client-side optimization protocol. We apply LoRA probability pdrop = 0.1 per round. b) Round time and time-to-accuracy.: For synchronous with rank r = 16 and scaling factor αlora = 32 [13]. Local optimization uses AdamW with learning rate 2 × 10−4 and FL, the round time is straggler-limited: matched hyperparameters (batch size, number of local steps, bits(mk,t ) (t) (k,t) Tround = max Tcomp + , (26) weight decay, and gradient clipping) across all baselines. For k∈St :ak,t =1 Rk,t memory-constrained edge settings, we enable QLoRA-style 4-bit backbones while training low-rank adapters in mixed where ak,t is the availability indicator. We report (i) wallP (t) precision [14]. clock time-to-accuracy, the cumulative t Tround until a target b) F ED -FSTQ configuration.: We track token sensitivity validation metric is reached, and (ii) cumulative uplink volume, using an exponential moving average (EMA) with decay ρ = summed over participating clients and rounds. 0.9, and refresh the token mask and mixed-precision allocation c) Latency and energy accounting (hybrid measurement every H = 10 local steps to amortize overhead. F ED -FSTQ and emulation).: Our virtual edge protocol measures compuemploys a discrete mixed-precision bit-set B = {0, 2, 4, 16}, tation on real hardware and emulates communication under (k,t) where b = 0 denotes structural pruning. We include b = the specified uplink profile. Compute time Tcomp is measured 16 to align with native FP16/BF16 kernels on modern edge during local PEFT on an NVIDIA Jetson Orin Nano (8GB). GPUs (e.g., Jetson-class accelerators), avoiding overhead from (k,t) Compute energy Ecomp is obtained by integrating on-device non-standard arithmetic. During training, token-coupled Fisher power telemetry (INA3221 via tegrastats). Communicastatistics guide the uplink allocation; during inference, the (k,t) tion time is computed as Tcomm = bits(mk,t )/Rk,t under same scoring rule (without gradients) enables dynamic token either uplink profile. Communication energy is estimated using reduction. a standard radio model c) Quantization and dequantization (group-wise scaling).: (k,t) (k,t) Ecomm = Ptx Tcomm , (27) F ED -FSTQ quantizes only the transmitted adapter update ∆Θk,t . To balance metadata overhead and quantization reswhere we use Ptx = 2.0 W by default and include a sensitivity olution, we use group-wise scaling aligned with the LoRA sweep in Sec. VI-C. structure. Adapter parameters are partitioned into groups G, To match the synchronous completion model, we report two where each group corresponds to one LoRA low-rank matrix complementary per-round energy summaries: (per layer and per projection). For a group g ∈ G and bit-width b ∈ {2, 4, 16}, we define X 1 (t) (k,t) (k,t) Ēround = Ecomp + Ecomm , (28) |St | maxj∈g |∆θj | k∈St qmax (b) = 2b−1 − 1, sg (b) = , (30) qmax (b) + ϵ (t) (k,t) (k,t) (29) Estr = max Ecomp + Ecomm , k∈St :ak,t =1 and quantize each retained coordinate j ∈ g with its assigned bit-width bj as (t) (t) where Ēround reflects average fleet battery drain and Estr upper-bounds the worst-case client drain. We specify in each ∆θj ṽ = clip round , −q (b ), q (b ) . (31) j max j max j table caption which summary is used. sg (bj )
11
We use deterministic round-to-nearest for reproducibility, as Metrics. We report: (i) cumulative uplink traffic (total transstochastic rounding yielded negligible gains in our experiments. mitted bytes until reaching the target threshold); (ii) timec j = sg (bj ) · ṽj . to-accuracy (wall-clock time under heterogeneous uplinks On the server, dequantization is ∆θ d) Payload accounting (tags and scales).: The packed with straggler-limited synchronization); (iii) round latency and uplink message includes indices (or a compressed mask), 2- energy on a virtual edge testbed (4G LTE + Jetson); and bit precision tags for bj ∈ {2, 4, 16}, quantized values, and (iv) semantic reliability via Token Recall / ROUGE-L and the group scales required by the active bit-widths in each downstream QA metrics. Unless otherwise specified, the target transmitted LoRA group. All reported payload sizes and uplink threshold in Pareto and time-to-accuracy plots corresponds to a validation accuracy of 60% for the respective setting; volumes account for this side information. e) Server-side stability.: The server aggregates dequan- task-specific semantic metrics (e.g., ROUGE/LLM-Judge for tized updates using standard FedAvg weighted summation [6]. medical QA) are reported in Sec. VI-E. F ED -FSTQ requires no change to the server aggregation rule. Scope of Comparison (Baselines). We compare against Under non-IID distributions, stability is maintained because baselines spanning the federated learning stack: Fisher-guided allocation assigns higher precision to high• System & Optimization: Canonical aggregation [6], [12] importance coordinates, while quantization noise is confined and robust optimization under objective inconsistency [15]– to low-sensitivity parameters. [19]. f) Reproducibility.: All experiments are implemented in • Communication Efficiency: Representative quantization, PyTorch using the Flower framework [22]. We fix random sparsification, and structured compression baselines [23]– seeds for client sampling, initialization, and data shuffling, [26], [44]–[47], [49]–[51]. and report means over at least three independent runs with • Model & Architecture Context: PEFT/LoRA family [13], standard deviations. Code and configuration files are aligned [14], [52]–[55] on LLaMA-based Transformer backbones [1]– with common cross-device FL tooling to facilitate reproduction [4]. and future comparisons [21]. • Token Reduction: Attention-driven pruning/merging originally developed for vision transformers [27], [28], [60]–[64]. VI. E VALUATION Note: Privacy mechanisms (secure aggregation / differential This section evaluates F ED -FSTQ as a deployable systems privacy) tighten budgets but are treated as deployment conprimitive for federated LLM fine-tuning. Our focus is the text [9], [10]. Our heterogeneity protocol aligns with established practical cross-device regime where synchronous rounds are difficulty taxonomies [7], [8]. straggler-limited by heterogeneous and time-varying uplinks beyond the datacenter [5], [11], [12], [43]. In this regime, reducing uplink payload is valuable not only in bytes, but A. Communication Efficiency Pareto frontier (uplink vs. quality). Fig. 4 plots model because it directly shortens the wall-clock path to a target performance against cumulative uplink traffic. F ED -FSTQ imquality threshold under synchronization delays. Methodology. We follow production-inspired FL benchmarking proves the communication–quality Pareto frontier and reaches methodology (e.g., LEAF/FedScale) and implement all methods a target validation accuracy of 60% with 46× less cumulative on standard federated stacks to ensure portability [20]–[22]. uplink transmission than the standard Fed-LoRA baseline Unless stated otherwise, we evaluate under partial participation, (LoRA [13] atop FedAvg [6]). This indicates that a substantial bandwidth heterogeneity, and intermittent connectivity—the portion of token-level updates can be compressed without default operating conditions in cross-device FL systems [11], sacrificing target quality when compression is guided by a [12]. To ensure fairness, all methods use the same backbone, principled sensitivity signal (Fisher/natural-gradient motivaPEFT configuration, client sampling policy, and local train- tions [34], [35]). Multilingual cost robustness (Fed-Aya). Table I reports ing budget; only the communication/compression mechanism normalized cumulative uplink cost across languages. F ED differs. FSTQ achieves the lowest average cost (2.85), improving over Evaluation Questions. We answer four system questions FedAvg [6], QSGD [23], and periodic-averaging quantization central to mobile learning systems: (FedPAQ [25]). Notably, for Chinese (zh), F ED -FSTQ reduces 1) Communication efficiency: Does Fisher-guided token the cost from 4.35 to 2.08 (a 52% reduction). This suggests quantization reduce uplink traffic under non-IID clients Fisher-guided allocation better preserves information-dense, to reach a fixed target quality? (Sec. VI-A) semantically decisive tokens (high sensitivity) while aggres2) End-to-end latency & energy: Is the additional compute sively compressing low-sensitivity components, avoiding the for Fisher estimation amortized by uplink savings under heuristic failure modes reported for token pruning/merging mobile links? (Sec. VI-B) under distribution shift [27], [28], [64]. 3) Robustness at scale: Does the method remain stable under severe heterogeneity, lossy channels, and client dropout— B. System Efficiency: Latency and Energy common failure modes in real deployments? (Sec. VI-C) 4) Resource feasibility: Can the method run within realistic A common concern for curvature-inspired methods is extra edge budgets, and which components drive the gains? computation on constrained devices. We therefore profile end(Sec. VI-D) to-end performance on a virtual edge testbed (LTE uplink
12
TABLE I F ED -AYA ( MULTILINGUAL QA): NORMALIZED CUMULATIVE UPLINK COST ( LOWER IS BETTER ). F ED -FSTQ ACHIEVES THE BEST AVERAGE COST (2.85) AND REDUCES Z H COST BY 52% (4.35→2.08).
Algorithm
ar
en
es
fr
pt
ru
te
zh
Avg
FedAvg FedToMe FedPAQ QSGD FedBAT
2.20 2.25 2.05 1.50 2.80
6.05 6.50 6.45 6.40 6.90
4.65 5.25 5.45 4.85 5.20
5.15 4.75 4.15 3.55 4.30
3.90 3.45 4.40 4.05 3.85
4.25 3.80 4.75 4.55 4.15
1.45 1.70 1.50 1.30 1.40
4.35 3.55 4.25 3.90 3.10
4.00 3.91 4.13 3.76 3.96
Fed-FSTQ
1.15
4.85
3.20
3.10
2.90
3.15
0.95
2.08
2.85
& R P P X Q L F D W L R Q $ F F X U D F \ 3 D U H W R ) U R Q W L H U
6 \ V W H P / D W H Q F \ % U H D N G R Z Q 9 L U W X D O ( G J H 7 H V W E H G
/ D W H Q F \ S H U 5 R X Q G V
5 2 8 * ( / $ F F X U D F \
& R P P X Q L F D W L R Q & R P S X W D W L R Q
) H G $ Y J ) H G 3 $ 4 4 6 * ' E L W ) H G % $ 7
) H G 7 R 0 H ) H G ) 6 7 4 2 X U V 3 D U H W R ) U R Q W L H U
1 R U P D O L ] H G & R P P X Q L F D W L R Q & R V W
[ 6 S H H G X S
Fig. 4. Communication–accuracy Pareto frontier. F ED -FSTQ reaches target accuracy with 46× less cumulative uplink traffic than Fed-LoRA (FedAvg [6] + LoRA [13]).
Y J ) H G $
4
) H G 3 $
R 0 H ) H G 7
4 6 * '
$ 7
) H G %
6 7 4
) H G )
2 X U V
Fig. 5. Latency breakdown (LTE). F ED -FSTQ slightly increases computation but substantially reduces communication time.
gains are not tied to a single radio-power assumption, we + NVIDIA Jetson), following edge FL system evaluation vary the uplink transmit power Ptx in the radio accounting (Ecomm = Ptx Tcomm ) while keeping all measured compute practices [5], [12], [21]. Round latency breakdown. Fig. 5 and Table II decompose costs unchanged. Table III reports the resulting per-round total per-round time into local computation and uplink commu- energy. Across a wide range spanning low-power IoT uplinks nication. F ED -FSTQ increases client computation modestly (0.1W) to high-power cellular transmission (5.0W), F ED -FSTQ by 0.85s (5.00s→5.85s) due to Fisher estimation (curvature consistently consumes substantially less energy than FedAvg. proxies [34], [35]), while reducing communication time from This indicates that the energy advantage is driven primarily 409.60s to 55.20s. Overall, the total round time drops from by reduced communication time, and remains robust across 414.60s to 61.05s, yielding a 6.8× end-to-end speedup ver- physical-layer regimes typical in edge deployments [5], [11]. Time-to-accuracy. Fig. 6a shows test accuracy versus sus FedAvg [6]. This supports a practical takeaway: when wall-clock time under heterogeneous uplinks. By mitigating Tcomm ≫ Tcomp on mobile uplinks [11], spending small straggler-limited aggregation through reduced uplink payload compute to reduce payload is cost-effective. (systems scaling effects [12], [21]), F ED -FSTQ reaches the Energy. Despite additional on-device compute, F ED -FSTQ target accuracy 52% faster than Fed-LoRA. reduces total per-round energy from 634.40J (FedAvg [6]) to 98.50J, and is the only method in our comparison that drops below 100J while maintaining strong quality. This is consistent C. Robustness and Scalability with the common observation that radio transmission often We stress-test F ED -FSTQ under heterogeneity and network dominates energy in mobile learning pipelines [5], [11]. unreliability commonly observed in mobile/edge FL [7], [11], Transmit-power sensitivity (Ptx ). To verify that the energy [12].
13
TABLE II S YSTEM EFFICIENCY (LTE). F ED -FSTQ ACHIEVES 61.05 S ROUND TIME AND 98.50J ENERGY.
Payload (MB)↓
Comm. (s)↓
Comp. (s)↓
Total (s)↓
Energy (J)↓
FedAvg QSGD (4-bit) FedPAQ FedBAT Fed-ToMe
1024.00 256.00 819.20 128.00 512.00
409.60 102.40 327.68 51.20 204.80
5.00 5.50 5.25 7.50 6.50
414.60 107.90 332.93 58.70 211.30
634.40 175.60 512.52 106.80 333.20
Fed-FSTQ
153.60
55.20
5.85
61.05
98.50
5 2 8 * ( / 6 F R U H
Method
, P S D F W R I ' D W D + H W H U R J H Q H L W \
TABLE III T RANSMIT- POWER SENSITIVITY ( RADIO ACCOUNTING ). T OTAL PER - ROUND ENERGY (J) UNDER DIFFERENT UPLINK TRANSMIT POWERS PTX . F ED -FSTQ REMAINS SIGNIFICANTLY MORE ENERGY- EFFICIENT ACROSS LOW- POWER TO HIGH - POWER UPLINKS . Ptx (W)
0.1
0.5
1.0
2.0
5.0
FedAvg Energy (J) Fed-FSTQ Energy (J)
60.96 28.92
224.80 51.00
429.60 78.60
839.20 133.80
2068.00 299.40
) H G $ Y J 4 6 * ' ) H G ) 6 7 4 2 X U V & R O O D S V H = R Q H
% D V H O L Q H & R O O D S V H
' L U L F K O H W ' D W D + H W H U R J H Q H L W \
Fig. 7. Impact of data heterogeneity (Non-IID). Accuracy under Dirichlet client partitions. F ED -FSTQ remains stable under extreme heterogeneity (robust FL under heterogeneity [15], [17]).
& R Q Y H U J H Q F H 6 S H H G + H W H U R J H Q H R X V 1 H W Z R U N ) D V W H U + L J K H U $ F F X U D F \
9 D O L G D W L R Q $ F F X U D F \
) H G $ Y J ) H G ) 6 7 4 2 X U V
& R P P X Q L F D W L R Q 5 R X Q G V
(a) Time-to-accuracy: 52% faster
, Q I H U H Q F H 3 H U I R U P D Q F H R Q 1 9 , ' , $ - H W V R Q 2 U L Q 1 D Q R * %
[ 6 S H H G X S
, Q I H U H Q F H 6 S H H G W R N H Q V V H F
Non-IID robustness. Table IV reports accuracy under Dirichlet client partitions, a standard stress protocol for objective inconsistency and client drift in FL [15], [16]. At extreme heterogeneity (α = 0.1), FedAvg [6] and QSGD [23] degrade sharply, whereas F ED -FSTQ remains stable (0.5120), even exceeding FedAvg at much milder heterogeneity (α = 0.5). Fig. 7 visualizes the same trend and highlights the relative stability of F ED -FSTQ as heterogeneity increases. Scalability. We report convergence time (hours) under increasing client population, following at-scale benchmarking practice [12], [21]. Compared with FedAvg, F ED -FSTQ consistently reduces time-to-convergence across 10/50/100 clients (11.74→3.38 hours at 10 clients; 28.77→8.45 hours at 100 clients). Fig. 8 provides the corresponding scalability curve. Network unreliability: packet loss and dropout. Under packet loss up to 20%, FedAvg accuracy drops from 0.65 to 0.342, while F ED -FSTQ remains at 0.579. Under client dropout up to 70%, FedAvg incurs a 0.40 accuracy drop, while F ED -FSTQ limits the drop to 0.10. Fig. 9 and Fig. 10 visualize resilience under lossy channels and partial participation, respectively.
D. Resource Feasibility and Ablations
) H G / R 5 $ % D V H O L Q H
) H G 7 R 0 H
4 6 * ' E L W
) H G ) 6 7 4 2 X U V
(b) Inference: 1.55× on Jetson Fig. 6. End-to-end speedups. (a) Faster convergence due to reduced straggler delay. (b) Faster on-device inference enabled by Fisher-guided token reduction (efficient transformer foundations [4]).
Peak memory footprint. Table V and Fig. 12 report peak memory usage. F ED -FSTQ requires 1450MB peak memory, fitting within a 2GB edge/IoT budget, while full FedAvg requires server-grade memory (4500MB). This complements LLM deployment evidence that quantization and compression are essential for edge feasibility [14], [29]–[33]. On-device energy trend (battery drain visualization). Fig. 11 visualizes the energy/battery drain trend under sustained on-device training. Consistent with Table II, reducing radio transmission time dominates the energy savings, aligning with edge-computing constraints in mobile systems [5], [11].
14
0 D V V L Y H 6 F D O D E L O L W \ / L Q H D U Y V ( [ S R Q H Q W L D O * U R Z W K
& O L H Q W & K X U Q 5 H V L O L H Q F H ) H G $ Y J ) H G ) 6 7 4 2 X U V
9 D O L G D W L R Q $ F F X U D F \
& R Q Y H U J H Q F H 7 L P H + R X U V
) H G $ Y J ) H G ) 6 7 4 2 X U V
1 X P E H U R I & O L H Q W V
Fig. 8. Scalability with client population. Convergence time (hours) versus the number of clients (at-scale FL systems [12], [21]).
9 D O L G D W L R Q $ F F X U D F \
TABLE IV ROBUSTNESS AND SCALABILITY.
) H G $ Y J ) H G ) 6 7 4 2 X U V 3 H U I R U P D Q F H * D S
& O L H Q W ' U R S R X W 5 D W H
Fig. 10. Client dropout resilience. Accuracy under partial participation and client dropout, a defining feature of real FL deployments [7], [12].
1 H W Z R U N 5 H V L O L H Q F H 5 R E X V W Q H V V W R 3 D F N H W / R V V
Non-IID Acc.
α = 0.1
α = 0.5
α = 1.0
FedAvg QSGD Fed-FSTQ
0.1781 0.0396 0.5120
0.4372 0.3633 0.5735
0.5032 0.4338 0.6019
Convergence Time (hours)
FedAvg QSGD Fed-FSTQ
Packet Loss
3 D F N H W / R V V 5 D W H
Fig. 9. Packet loss resilience. Accuracy under packet loss rates up to 20% in mobile uplinks [11].
10 clients
50 clients
100 clients
11.74 4.93 3.38
19.14 8.87 5.85
28.77 14.01 8.45
0%
5%
10%
15%
20%
0.6500 0.6600
0.5820 0.6450
0.4950 0.6180
0.4100 0.5950
0.3420 0.5790
Client Dropout
0%
30%
50%
70%
FedAvg Acc. drop Fed-FSTQ Acc. drop
0.00 0.00
-0.05 -0.01
-0.15 -0.04
-0.40 -0.10
FedAvg Acc. Fed-FSTQ Acc.
Multilingual cost radar. Fig. 13 provides an at-a-glance view of normalized multilingual communication costs. F ED - additionally note that the gains are complementary to sparsityFSTQ maintains a compact, balanced profile across languages, inducing regularization and sparse-subnetwork perspectives whereas heuristic token reduction exhibits larger variance on commonly used for efficient training [58], [59]. Finally, the information-dense inputs, consistent with known sensitivity of overarching system-design motivation, preserving rare but token heuristics [28], [64]. safety-critical evidence under nonstationarity and constraints, Ablation: what drives the gains. Table VI isolates key connects to reliability-aware edge analytics more broadly [65]. components. Removing Fisher guidance collapses quality Matched-budget ablation: impact of token–parameter (0.6610→0.4215) at the same payload, indicating that Fisher- coupling. To directly validate that token-level guidance imbased prioritization is essential (connection sensitivity and proves training-time communication (beyond optional inference pruning foundations [36], [37], [39], [40]). Disabling token pruning), we conduct a matched-budget ablation on Fedpruning preserves quality but substantially increases payload Med under the same per-round uplink budget (Bmax = (153.6MB→512MB), while disabling quantization increases 150MB/round) and identical message accounting (indices + bitpayload further (614.4MB). Together, these results show that the tags + values). We compare F ED -FSTQ against (i) U NIFORM best efficiency–quality trade-off comes from both Fisher-guided F ISHER (token coupling removed; zi = 1 for all tokens) and prioritization and mixed-precision quantization [56], [57]. We (ii) R ANDOM S UPPORT. To avoid confounding, inference-
15
2 Q ' H Y L F H % D W W H U \ ' U D L Q * U H H Q $ , % H Q H I L W V ) H G $ Y J % D W W H U \ ' U D L Q ) H G ) 6 7 4 * D L Q ) H G $ Y J ) H G ) 6 7 4 2 X U V
' H Y L F H % D W W H U \ 5 H P D L Q L Q J
0 X O W L O L Q J X D O & R P P X Q L F D W L R Q & R V W 6 S D Q L V K
) H G ) 6 7 4 6 W L O O $ O L Y H
( Q J O L V K
) U H Q F K
) H G $ Y J ) H G 7 R 0 H ) H G ) 6 7 4 2 X U V
& U L W L F D O = R Q H
) H G $ Y J ' H D G
3 R U W X J X H V H
7 U D L Q L Q J 5 R X Q G
$ U D E L F
Fig. 11. On-device energy/battery drain visualization. Reduced communication time yields substantially improved energy sustainability under continuous training [5], [11].
5 X V V L D Q
& K L Q H V H 7 H O X J X
2 Q ' H Y L F H 0 H P R U \ ) R R W S U L Q W - H W V R Q 1 D Q R / L P L W * %
3 H D N 0 H P R U \ 0 %
Fig. 13. Multilingual cost radar. F ED -FSTQ maintains low and balanced communication cost across languages.
TABLE V P EAK MEMORY (MB). F ED -FSTQ FITS WITHIN 2GB EDGE DEVICES .
Method
Peak Memory (MB) ↓
J
) H G $ Y
4
) H G 3 $
H
0 ) H G 7 R
4 6 * ' L W E
7 ) H G % $
7 4 ) H G ) 2 6 X U V
FedAvg (Server GPU) FedPAQ (Server GPU) Fed-ToMe (High-End Edge) QSGD (High-End Edge) FedBAT (Mid-Range Edge) Fed-FSTQ (IoT/Mobile 2GB)
4500 4500 3800 2100 1800 1450
Fig. 12. On-device memory footprint. F ED -FSTQ is the only method that stays below the 2GB Jetson limit, while uncompressed and heavier baselines exceed the edge budget.
where low-information tokens are preferentially removed and updates focus on decisive evidence. Downstream quality (medical QA). Table IX shows that time token pruning is disabled for all methods. As shown token preservation translates into downstream QA quality on in Table VII, F ED -FSTQ converges faster and reaches a medical QA benchmarks [41], [42]. F ED -FSTQ achieves the substantially higher ROUGE-L under the same budget (36.15 vs. best overall Fed-Med quality, with Rouge-L 36.15, Meteor 31.50 at round 50), confirming that coupling Fisher estimation 4.06, and LLM-as-a-judge 2.78. with token sensitivity improves which adapter coordinates are prioritized for mixed-precision transmission and sparsification. VII. C ONCLUSION E. Semantic Reliability and Integrity We next evaluate whether F ED -FSTQ preserves semantically decisive evidence under non-IID compression. Token-level preservation. Table VIII compares F ED -FSTQ with attention-driven token reduction inspired by tokenmerging/pruning lines [28], [64]. Fed-ToMe reduces average length the most, but suffers a substantial Token Recall drop (0.6540). In contrast, F ED -FSTQ achieves 0.8320 Token Recall, exceeding FedAvg (0.8214), consistent with a denoising effect
This paper introduced F ED -FSTQ, a Fisher-guided token quantization primitive for communication-efficient federated fine-tuning of LLMs on bandwidth-constrained mobile and edge clients. The central systems takeaway is that, in cross-device federated adaptation, heterogeneous uplinks make synchronous training straggler-limited, so deployment-relevant progress is measured by wall-clock time-to-accuracy rather than by model quality alone. In this regime, compression cannot be treated as a uniform, content-agnostic operation. For language workloads under non-IID data, preserving rare but decision-critical token
16
TABLE VI A BLATION S TUDY: PARETO E FFICIENCY. F ED -FSTQ ACHIEVES 4× COMPRESSION RELATIVE TO UNQUANTIZED BASELINES WITH NEGLIGIBLE QUALITY LOSS (< 1.7%), WHILE OUTPERFORMING RANDOM COMPRESSION BY +56% IN ROUGE-L.
ROUGE-L ↑
Payload (MB) ↓
Full F ED -FSTQ (Ours)
0.6610
153.6
w/o Fisher (Random Policy) w/o Token Pruning w/o Quantization
0.4215 0.6650 0.6720
153.6 512.0 614.4
Variant
TABLE VII M ATCHED - BUDGET ABLATION (F ED -M ED , Bmax = 150MB/ ROUND ). I NFERENCE - TIME TOKEN PRUNING IS DISABLED FOR ALL METHODS . F ED -FSTQ OUTPERFORMS U NIFORM F ISHER AND R ANDOM S UPPORT UNDER THE SAME UPLINK BUDGET, VALIDATING TOKEN – PARAMETER COUPLING IN TRAINING - TIME COMMUNICATION . Method
Round 1
Round 10
Round 20
Round 50
F ED -FSTQ (Ours) U NIFORM F ISHER (zi = 1) R ANDOM S UPPORT
12.00 10.00 5.00
32.50 25.00 10.50
35.10 28.50 14.20
36.15 31.50 18.05
Gain vs. Uniform Fisher
+2.00
+7.50
+6.60
+4.65
( I I L F L H Q F \ Y V 5 H O L D E L O L W \ 7 U D G H R I I
7 R N H Q 5 H F D O O 5 H O L D E L O L W \
6 2 7 $ 3 H U I R U P D Q F H
) H G $ Y J ) H G 3 $ 4 ) H G 7 R 0 H 4 6 * ' ) H G % $ 7 ) H G ) 6 7 4 2 X U V
$ Y J 2 X W S X W / H Q J W K ( I I L F L H Q F \
Fig. 14. Efficiency–reliability trade-off. F ED -FSTQ occupies the highefficiency, high-reliability region.
evidence is often the difference between stable improvement and brittle degradation. F ED -FSTQ operationalizes this view with a lightweight Fisher proxy that couples token-level sensitivity to mixedprecision allocation and sparse uplink packing. The resulting messages remain compatible with standard FedAvg-style aggregation and integrate as a drop-in module in federated PEFT pipelines (e.g., FedAvg+LoRA) without server-side changes. Across multilingual QA and medical QA workloads under heterogeneous uplinks and intermittent participation, F ED FSTQ reduces cumulative uplink traffic by up to 46× relative to FedAvg-LoRA while maintaining target quality, translating into a 52% reduction in straggler-limited wall-clock time-toaccuracy. On a 4G-LTE plus Jetson virtual edge setup, F ED -
TABLE VIII T OKEN - LEVEL INFORMATION RETENTION . Method
Token Recall (↑)
ROUGE-L (↑)
Avg Len. (↓)
FedAvg QSGD (4-bit) FedPAQ FedBAT Fed-ToMe
0.8214 0.7845 0.7620 0.7105 0.6540
0.6532 0.6102 0.5980 0.5540 0.5210
452.1 448.3 440.5 412.0 375.5
Fed-FSTQ
0.8320
0.6610
372.8
TABLE IX Q UALITY ON F ED -M ED . F ED -FSTQ ACHIEVES THE BEST OVERALL QUALITY.
Algorithm
Rouge-L
Meteor
LLM-Judge (1–5)
FedAvg FedToMe FedPAQ QSGD
33.47 17.15 34.16 35.03
4.00 2.35 3.98 4.01
2.71 2.00 2.72 2.56
Fed-FSTQ
36.15
4.06
2.78
FSTQ achieves a near-minute round time with sub-100J energy per round, showing that modest on-device computation for sensitivity tracking is amortized when uplink dominates endto-end cost. Beyond training, the same Fisher-derived token saliency can be optionally reused at inference to shorten the effective token stream, yielding up to a 1.55× end-to-end speedup on Jetsonclass devices. This connects communication-aware federated adaptation to tangible on-device efficiency gains. Future work should pursue two deployment-relevant directions. First, extending F ED -FSTQ to asynchronous or partially synchronous protocols, where straggler mitigation must be balanced against update staleness and delayed aggregation. Second, strengthening robustness and privacy under secure aggregation and differential privacy, where tighter effective budgets increase the value of sensitivity-aware allocation while imposing additional constraints on metadata, packing, and estimation. More broadly, treating Fisher-guided token sensitivity as a systems control signal offers a practical path toward federated LLM adaptation that remains efficient, reliable, and deployable under real mobile networks. R EFERENCES [1] T. Brown, B. Mann, N. Ryder et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020. [2] H. Touvron, T. Lavril, G. Izacard et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023. [3] J. Devlin, M.-W. Chang, K. Lee et al., “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186. [4] A. Vaswani, N. Shazeer, N. Parmar et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [5] W. Shi, J. Cao, Q. Zhang et al., “Edge computing: Vision and challenges,” IEEE internet of things journal, vol. 3, no. 5, pp. 637–646, 2016.
17
[6] B. McMahan, E. Moore, D. Ramage et al., “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–1282. [7] P. Kairouz, H. B. McMahan, B. Avent et al., “Advances and open problems in federated learning,” Foundations and trends® in machine learning, vol. 14, no. 1–2, pp. 1–210, 2021. [8] T. Li, A. K. Sahu, A. Talwalkar et al., “Federated learning: Challenges, methods, and future directions,” IEEE signal processing magazine, vol. 37, no. 3, pp. 50–60, 2020. [9] K. Bonawitz, V. Ivanov, B. Kreuter et al., “Practical secure aggregation for privacy-preserving machine learning,” in proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1175–1191. [10] M. Abadi, A. Chu, I. Goodfellow et al., “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318. [11] W. Y. B. Lim, N. C. Luong, D. T. Hoang et al., “Federated learning in mobile edge networks: A comprehensive survey,” IEEE communications surveys & tutorials, vol. 22, no. 3, pp. 2031–2063, 2020. [12] K. Bonawitz, H. Eichner, W. Grieskamp et al., “Towards federated learning at scale: System design,” Proceedings of machine learning and systems, vol. 1, pp. 374–388, 2019. [13] E. J. Hu, yelong shen, P. Wallis et al., “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https: //openreview.net/forum?id=nZeVKeeFYf9 [14] T. Dettmers, A. Pagnoni, A. Holtzman et al., “Qlora: Efficient finetuning of quantized llms,” Advances in neural information processing systems, vol. 36, pp. 10 088–10 115, 2023. [15] T. Li, A. K. Sahu, M. Zaheer et al., “Federated optimization in heterogeneous networks,” Proceedings of Machine learning and systems, vol. 2, pp. 429–450, 2020. [16] J. Wang, Q. Liu, H. Liang et al., “Tackling the objective inconsistency problem in heterogeneous federated optimization,” Advances in neural information processing systems, vol. 33, pp. 7611–7623, 2020. [17] S. P. Karimireddy, S. Kale, M. Mohri et al., “Scaffold: Stochastic controlled averaging for federated learning,” in International conference on machine learning. PMLR, 2020, pp. 5132–5143. [18] S. J. Reddi, Z. Charles, M. Zaheer et al., “Adaptive federated optimization,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=LkFG3lB1 3U5 [19] D. A. E. Acar, Y. Zhao, R. Matas et al., “Federated learning based on dynamic regularization,” in International Conference on Learning Representations, 2021. [Online]. Available: https: //openreview.net/forum?id=B7v4QMR6Z9w [20] S. Caldas, S. M. K. Duddu, P. Wu et al., “Leaf: A benchmark for federated settings,” arXiv preprint arXiv:1812.01097, 2018. [21] F. Lai, Y. Dai, S. Singapuram et al., “Fedscale: Benchmarking model and system performance of federated learning at scale,” in International conference on machine learning. PMLR, 2022, pp. 11 814–11 827. [22] D. J. Beutel, T. Topal, A. Mathur et al., “Flower: A friendly federated learning research framework,” arXiv preprint arXiv:2007.14390, 2020. [23] D. Alistarh, D. Grubic, J. Li et al., “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” Advances in neural information processing systems, vol. 30, 2017. [24] Y. Lin, S. Han, H. Mao et al., “Deep gradient compression: Reducing the communication bandwidth for distributed training,” in International Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=SkhQHMW0W [25] A. Reisizadeh, A. Mokhtari, H. Hassani et al., “Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization,” in International conference on artificial intelligence and statistics. PMLR, 2020, pp. 2021–2031. [26] S. P. Karimireddy, Q. Rebjock, S. Stich et al., “Error feedback fixes signsgd and other gradient compression schemes,” in International conference on machine learning. PMLR, 2019, pp. 3252–3261. [27] Y. Rao, W. Zhao, B. Liu et al., “Dynamicvit: Efficient vision transformers with dynamic token sparsification,” Advances in neural information processing systems, vol. 34, pp. 13 937–13 949, 2021. [28] D. Bolya, C.-Y. Fu, X. Dai et al., “Token merging: Your vit but faster,” arXiv preprint arXiv:2210.09461, 2022. [29] T. Dettmers, M. Lewis, Y. Belkada et al., “Llm. int8 () 8-bit matrix multiplication for transformers at scale,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, 2022, pp. 30 318–30 332.
[30] E. Frantar, S. Ashkboos, T. Hoefler et al., “Gptq: Accurate post-training quantization for generative pre-trained transformers,” arXiv preprint arXiv:2210.17323, 2022. [31] G. Xiao, J. Lin, M. Seznec et al., “Smoothquant: Accurate and efficient post-training quantization for large language models,” in International conference on machine learning. PMLR, 2023, pp. 38 087–38 099. [32] J. Lin, J. Tang, H. Tang et al., “Awq: Activation-aware weight quantization for on-device llm compression and acceleration,” Proceedings of machine learning and systems, vol. 6, pp. 87–100, 2024. [33] Z. Yao, R. Yazdani Aminabadi, M. Zhang et al., “Zeroquant: Efficient and affordable post-training quantization for large-scale transformers,” Advances in neural information processing systems, vol. 35, pp. 27 168– 27 183, 2022. [34] S.-I. Amari, “Natural gradient works efficiently in learning,” Neural computation, vol. 10, no. 2, pp. 251–276, 1998. [35] J. Martens and R. Grosse, “Optimizing neural networks with kroneckerfactored approximate curvature,” in International conference on machine learning. PMLR, 2015, pp. 2408–2417. [36] Y. LeCun, J. Denker, and S. Solla, “Optimal brain damage,” Advances in neural information processing systems, vol. 2, 1989. [37] B. Hassibi and D. Stork, “Second order derivatives for network pruning: Optimal brain surgeon,” Advances in neural information processing systems, vol. 5, 1992. [38] J. Kirkpatrick, R. Pascanu, N. Rabinowitz et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017. [39] N. Lee, T. Ajanthan, and P. Torr, “SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=B1VZqjAcYX [40] C. Wang, G. Zhang, and R. Grosse, “Picking winning tickets before training by preserving gradient flow,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=SkgsACVKPH [41] Q. Jin, B. Dhingra, Z. Liu et al., “Pubmedqa: A dataset for biomedical research question answering,” in Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLPIJCNLP), 2019, pp. 2567–2577. [42] D. Jin, E. Pan, N. Oufattole et al., “What disease does this patient have? a large-scale open domain question answering dataset from medical exams,” Applied Sciences, vol. 11, no. 14, p. 6421, 2021. [43] J. Konečnỳ, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” arXiv preprint arXiv:1511.03575, 2015. [44] J. Konečnỳ, H. B. McMahan, F. X. Yu et al., “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016. [45] W. Wen, C. Xu, F. Yan et al., “Terngrad: Ternary gradients to reduce communication in distributed deep learning,” Advances in neural information processing systems, vol. 30, 2017. [46] F. Seide, H. Fu, J. Droppo et al., “1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns.” in Interspeech, vol. 2014. Singapore, 2014, pp. 1058–1062. [47] J. Bernstein, Y.-X. Wang, K. Azizzadenesheli et al., “signsgd: Compressed optimisation for non-convex problems,” in International conference on machine learning. PMLR, 2018, pp. 560–569. [48] S. Li, W. Xu, H. Wang et al., “Fedbat: Communication-efficient federated learning via learnable binarization,” arXiv preprint arXiv:2408.03215, 2024, accepted by ICML 2024 (as stated on arXiv). [49] S. U. Stich, J.-B. Cordonnier, and M. Jaggi, “Sparsified sgd with memory,” Advances in neural information processing systems, vol. 31, 2018. [50] F. Sattler, S. Wiedemann, K.-R. Müller et al., “Sparse binary compression: Towards distributed deep learning with minimal communication,” in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–8. [51] T. Vogels, S. P. Karimireddy, and M. Jaggi, “Powersgd: Practical lowrank gradient compression for distributed optimization,” Advances in Neural Information Processing Systems, vol. 32, 2019. [52] N. Houlsby, A. Giurgiu, S. Jastrzebski et al., “Parameter-efficient transfer learning for nlp,” in International conference on machine learning. PMLR, 2019, pp. 2790–2799. [53] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 4582–4597.
18
[54] B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameterefficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045– 3059. [55] Q. Zhang, M. Chen, A. Bukharin et al., “Adaptive budget allocation for parameter-efficient fine-tuning,” in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=lq62uWRJjiY [56] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” in ICLR, 2016. [Online]. Available: http://arxiv.org/abs/1510.00149 [57] B. Jacob, S. Kligys, B. Chen et al., “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704–2713. [58] D. Molchanov, A. Ashukha, and D. Vetrov, “Variational dropout sparsifies deep neural networks,” in International conference on machine learning. PMLR, 2017, pp. 2498–2507. [59] J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” in International Conference on Learning Representations, 2019. [Online]. Available: https: //openreview.net/forum?id=rJl-b3RcF7 [60] A. Dosovitskiy, L. Beyer, A. Kolesnikov et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy [61] M. S. Ryoo, A. Piergiovanni, A. Arnab et al., “Tokenlearner: Adaptive space-time tokenization for videos,” in Advances in Neural Information Processing Systems (NeurIPS), 2021. [62] M. Fayyaz, S. A. Koohpayegani, F. R. Jafari et al., “Adaptive token sampling for efficient vision transformers,” in European conference on computer vision. Springer, 2022, pp. 396–414. [63] H. Yin, A. Vahdat, J. M. Alvarez et al., “A-vit: Adaptive tokens for efficient vision transformer,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 809–10 818. [64] Y. Liang, C. GE, Z. Tong et al., “EVit: Expediting vision transformers via token reorganizations,” in International Conference on Learning Representations, 2022. [Online]. Available: https: //openreview.net/forum?id=BjyvwnXXVn_ [65] C. Li, D. Huang, K. Yao et al., “Physics-guided tiny-mamba transformer for reliability-aware early fault warning,” arXiv preprint arXiv:2601.21293, 2026.