arXiv CSopen access

Mixed-Timescale Differential Coding for Downlink Model Broadcast in Wireless Federated Learning

1

Mixed-Timescale Differential Coding for Downlink Model Broadcast in Wireless Federated Learning

arXiv:2607.13119v1 [cs.IT] 14 Jul 2026

Chung-Hsuan Hu, Zheng Chen, and Erik G. Larsson

Abstract—In standard federated learning systems, the parameter server broadcasts the global model to the participating devices in every iteration. Motivated by the temporal correlation between consecutive global models, differential coding can be applied to global model dissemination to reduce the information magnitude, thereby enabling communication with fewer quantization bits. However, due to wireless link failures, devices may occasionally miss differential updates and consequently fail to reconstruct the global model. As a result, they either continue local training based on an outdated model or remain idle until the next full-model broadcast becomes available. To address this challenge, we propose a mixed-timescale differential coding (MTDC) scheme that performs differential coding at two different levels by adjusting the reference model. With MTDC, a device can reconstruct the latest global model between two full-model broadcasts even if it misses a differential update. We provide a convergence analysis that motivates the design of an age-aware variant of MTDC, along with a device scheduling policy to further improve communication efficiency. Simulation results demonstrate that the proposed MTDC schemes achieve superior learning performance compared to baseline methods under similar communication resource budgets in the presence of downlink transmission failures. Index Terms—Federated learning, differential coding, global model broadcast, downlink failure, scheduling

I. I NTRODUCTION Federated learning (FL) is a popular distributed machine learning framework in which a set of devices collaboratively train a model through iterative local training and centralized aggregation at a server [2]. The iterative exchange of model information between the server and devices incurs substantial communication overhead, making communication efficiency a key challenge in FL systems. For FL over wireless networks, a wide range of communication-efficient transmission strategies have been investigated in the literature [3], [4]. In particular, device scheduling and resource allocation can be optimized using data-importance or learning-aware metrics [5]–[8], as well as by accounting for communication uncertainties such as time-varying channel conditions and interference [9]–[14]. In addition, data compression techniques, including sparsification [15], [16] and quantization [17], [18], have been widely adopted to further improve communication efficiency. Most prior work on communication-efficient FL has focused on uplink (UL) transmission from devices to the server, while The authors are with the Department of Electrical Engineering (ISY), Linköping University, Linköping, SE-58183 Sweden. E-mail:{chunghsuan.hu, zheng.chen, erik.g.larsson}@liu.se. This work was supported in part by Zenith, ELLIIT, the Swedish Research Council (VR), and the Knut and Alice Wallenberg (KAW) Foundation. A preliminary version of this paper was presented at the 2024 Asilomar conference on Signals, Systems, and Computers [1].

the downlink (DL) transmission from the server to the devices has received less attention. On the DL, the server can, in principle, broadcast the global model so that all devices receive it simultaneously using the same timefrequency resources. In contrast, on the UL, communication resources need to be divided among transmitting devices such that the server can receive multiple local model updates without interference. DL communication efficiency has been studied in [19], which focuses on the comparison between analog (based on overthe-air computation [20]) and digital transmission designs. Joint UL and DL communication-efficient designs have been investigated in terms of data compression [21], [22], resource allocation [23], and analog transmission designs [24]. This paper focuses on the DL transmission of global models in wireless FL with unreliable communication links, and specifically with the use of differential coding (DiC) for the global model compression. DiC is widely used for image and video coding as a lossy source coding scheme that exploits temporal and spatial redundancy [25]. The key idea is to encode the differences between adjacent pixels, consecutive frames, or prediction residuals rather than the absolute values. By doing so, the entropy of the signal can be greatly reduced, enabling more efficient compression and lower bit rates. In FL, existing studies have illustrated the evidence of temporal correlation between learning models in consecutive iterations and proposed various DiC-based techniques for gradient or model update compression [26]–[30]. Nevertheless, all these studies focus on the uplink transmission of model updates from local devices to the server. The potential advantages of using DiC for DL communication cost reduction have not been explored. One drawback of the standard DiC framework is that it is prone to decoding failures. The reconstruction of the original frame critically depends on whether the differential update (or residual) is correctly received. In the context of FL, missing one (differentially encoded) model broadcast hinders the reconstruction of the current and subsequent global models. Existing works mentioned above have not addressed this issue, which is the main motivation behind our work. A. Contributions We propose a novel mixed-timescale differential coding (MTDC) scheme to robustify DiC-based compression for DL global model transmission in wireless FL. MTDC is a hierarchical differential coding scheme operating at three timescales (with different quantization levels): (1) the full model is broadcast on a slow timescale (i.e., infrequently), without DiC and at high resolution; (2) a first-level DiC-coded

2

II. S YSTEM M ODEL We consider a wireless FL system with a parameter server and a set of devices K ≜ {1, ..., 𝐾 } participating in the training of a shared learning model parameterized by 𝜽 ∈ R𝑑 . The Í goal is to minimize a global loss function 𝐹 (𝜽) ≜ 𝑘 ∈ K 𝑤 𝑘 𝐹𝑘 (𝜽), which is a weighted average of local loss functions 𝐹𝑘 (𝜽), ∀𝑘 ∈ K, evaluated over the local datasets S𝑘 , 𝑘 ∈ K. A widely used algorithm for this is Federated Averaging (FedAvg) [2]. It operates by iteratively combining local training on devices with centralized model aggregation. A. Federated Averaging (FedAvg) At the 𝑡-th global iteration (𝑡 = 1, ...): 1) The server broadcasts the global model 𝜽 (𝑡) to K and schedules a device subset Π(𝑡) ⊆ K for model training. 2) Each device 𝑘 ∈ Π(𝑡) updates the model with 𝐸-step mini-batch stochastic gradient descent: 𝜽 𝑘 (𝑡, 𝜄 + 1) = 𝜽 𝑘 (𝑡, 𝜄) − 𝜂∇𝐹𝑘 (𝜽 𝑘 (𝑡, 𝜄); B𝑘 (𝑡, 𝜄)),

(1)

𝜄 = 0, ..., 𝐸 −1, where 𝜽 𝑘 (𝑡, 0) = 𝜽 (𝑡), B𝑘 (𝑡, 𝜄) ⊆ S𝑘 , and 𝜂 is the learning rate. The corresponding model update △𝜽 𝑘 (𝑡) = 𝜽 𝑘 (𝑡, 𝐸) − 𝜽 𝑘 (𝑡, 0)

(2)

is transmitted to the server. 3) The server aggregates the received gradient updates, and updates the global model according to Õ 𝜽 (𝑡 + 1) = 𝜽 (𝑡) + 𝑤 𝑘 △𝜽 𝑘 (𝑡), (3) 𝑘 ∈Π (𝑡 )

Í whereÍ 𝑘 ∈Π (𝑡 ) 𝑤 𝑘 = 1. A common choice is 𝑤 𝑘 = |S𝑘 |/ 𝑗 ∈Π (𝑡 ) |S 𝑗 |. In FedAvg, the server needs to broadcast the global model at each iteration. The model parameters 𝜽 (𝑡) progress in the

1 sim𝜽 (𝑡, 𝑡 + 𝛿)

model update is broadcast with fewer quantization bits, on an intermediate timescale; and (3) a second-level DiC-coded model update is broadcast with even fewer quantization bits, on a fast timescale. The difference between the two levels of differential updates lies in their choice of reference frame: the first-level update uses an older version of the global model – which is more likely to have been correctly reconstructed at the local devices – as the reference for computing the residual (differential update). In this way, the first-level differential update (at the intermediate timescale) serves as a fallback mechanism for the second-level differential update (at the fast timescale), and the full-model broadcast serves as a fallback mechanism for the first-level differential update, in case of transmission failures. In addition to the MTDC scheme, we propose an age-aware extension, termed A-MTDC, which adaptively selects the type of DL broadcast (full model or differential update) based on the model staleness level at local devices and the statistics of decoding failures. In the presence of DL decoding failures, we establish the convergence of Federated Averaging under the proposed MTDC scheme (see Section IV for specifics). This theoretical result further motivates the design of A-MTDC and the associated age-aware scheduling policy.

0.95 𝛿=1 𝛿=5 𝛿 = 10 𝛿 = 15 𝛿 = 20

0.9 0.85 0.8

0

20

40

60

𝑡 Fig. 1: Illustration of the temporal correlation of the iterates 𝜽 (𝑡) quantified by the cosine similarity between 𝜽 (𝑡) and 𝜽 (𝑡 + 𝛿), denoted by sim𝜽 (𝑡, 𝑡 + 𝛿), where 𝛿 = 1, 5, 10, 15, 20. gradient descent directions. This naturally introduces temporal correlation between consecutive global models. In Figure 1, we illustrate this temporal correlation in the global model evolution when using FedAvg with a convolutional neural network for an image classification problem.1 In this example, high correlation is observed even when two iterates are 20 iterations apart. In general, the correlation between two iterates weakens when the time window increases, and it increases with a smaller learning rate. This observation motivates the usage of DiC techniques to improve DL communication efficiency. In the literature, DiC has been considered for UL local update transmission [26], [27], but to our knowledge it has not been investigated for DL global model transmission. B. Differential Coding Differential coding (DiC) is a common technique for multimedia data compression [25]. For instance, let 𝒙 𝑡 be the current image frame and 𝒙 ref be the reference frame. The sender transmits the compressed residual △𝒙 𝑡 = 𝑄(𝒙 𝑡 − 𝒙 ref ), where 𝑄(·) is a compression operator (e.g., quantizer). We will call △𝒙 𝑡 the differential update throughout the paper. Let 𝒙˜ 𝑡 be the reconstructed frame at the receiver side, which can be computed by 𝒙˜ 𝑡 = 𝒙 ref + △𝒙 𝑡 at the receiver. To avoid error propagation in 𝒙˜ 𝑡 , the residual calculation will use the reference frame 𝒙 ref based on { 𝒙˜ 𝑖 }𝑖<𝑡 rather than the original frames {𝒙 𝑖 }𝑖<𝑡 . A special case is 𝒙 ref = 𝒙˜ 𝑡 −1 , the latest reconstructed frame. DiC offers the advantage of reducing the signal magnitude and dynamic range, thereby requiring fewer quantization bits to achieve a similar mean-squared error (MSE) as the full model transmission. However, DiC schemes rely on perfect reception of the differential update △𝒙 𝑡 and the reconstructed reference frame. Missing a differential update △𝒙 𝑡 makes the reconstruction of the current and all the subsequent frames (i.e., { 𝒙˜ 𝑖 }𝑖 ≥𝑡 ) impossible. Wireless communication is generally susceptible to random decoding failures, and in the context of FL, missing a model update will cause a device to work on an outdated global model when performing local training. 1 We consider the same learning setting and use the same parameters as in the other simulation results presented in Section VI.

3

𝜽ˆ (1) and 𝜽ˆ (51) are the compressed full models corresponding to 𝜽 (1) and 𝜽 (51). Thus, 𝜽ˆ (1) = 𝑄 0 (𝜽 (1)) and 𝜽ˆ (51) = 𝑄 0 (𝜽 (51)). ˆ (11) carries the differ• A first-level differential update 𝜽 ence between 𝜽 (11) and 𝜽˜ (1) after compression, i.e., 𝑄 1 (𝜽 (11) − 𝜽˜ (1)). Similarly, 𝜽ˆ (21) = 𝑄 1 (𝜽 (21) − 𝜽˜ (11)). • A second-level differential update carries the difference between 𝜽 (𝑡) and 𝜽˜ (𝑡 − 1) after compression, e.g., 𝜽ˆ (2) = 𝑄 2 (𝜽 (2) − 𝜽˜ (1)) and 𝜽ˆ (6) = 𝑄 2 (𝜽 (6) − 𝜽˜ (5)). Wireless transmissions are susceptible to decoding failures. We define the decoding failure probability of device 𝑘 at iteration (𝑡 ) 𝑡 as 𝑃 𝑘,𝑖 , with 𝑡 ∈ T𝑖 , 𝑖 ∈ {0, 1, 2}. To ensure successful reconstruction, the transmissions of 𝜽ˆ (𝑡), 𝑡 ∈ T𝑖 , 𝑖 ∈ {0, 1, 2} (𝑡 ) (𝑡 ) (𝑡 ) 4 are coded such that 𝑃 𝑘,0 ≪ 𝑃 𝑘,1 ≤ 𝑃 𝑘,2 . When a device fails to decode 𝜽ˆ (𝑡), the local training is based on an outdated model 𝜽˜ (𝜏), for some 𝜏 < 𝑡. The proposed scheme enables re-synchronization with the server, improving model staleness in case of decoding failures. Fig. 3b exemplifies: ˆ (𝑡) is decoded successfully by device 𝑘 and • When 𝜽 the reference model is available (e.g., at 𝑡 = 2, 11, the reference 𝜽˜ (1) is available), the adopted model for local training is up-to-date, i.e., 𝜽 𝑘 (𝑡, 0) = 𝜽˜ (𝑡). ˆ (6), it has to use 𝜽˜ (5), the model • When device 𝑘 misses 𝜽 reconstructed at 𝑡 = 5 as the adopted model, until 𝑡 = 11, at which point it can rely on the first-level differential 𝜽ˆ (11) and the memory of 𝜽˜ (1) to compute 𝜽˜ (11). ˆ (21), • When a device misses the transmitted information 𝜽 ˜ it has to adopt the reconstructed model 𝜽 (20) until the next full-model broadcast at 𝑡 = 51. Compared to the baseline DiC where 𝜽 𝑘 (50, 0) = 𝜽˜ (5) (due to decoding failure of 𝜽ˆ (6)), the proposed MTDC improves the staleness with 𝜽 𝑘 (50, 0) = 𝜽˜ (20). •

෡ Broadcast vector 𝜽(𝑡)

෡ 𝜽(1)

෡ 𝜽(11)

෡ ෡ ෡ 𝜽(10) 𝜽(2) 𝜽(3)

Full model

…

…

Differential update 𝑡

…

෡ 𝜽(12)

𝑡 1

2

3

෩ ෩ ෩ 𝜽𝑘 (𝑡, 0) 𝜽(1) 𝜽(2) 𝜽(2)

… …

… 10 11 12 ෩ ෩ ෩ … 𝜽(2) 𝜽(11) 𝜽(12)

Fig. 2: An example of FL with DiC-enabled model broadcast: the server broadcasts the full model every 10 iterations and in between differential updates are broadcast instead.

One way to tackle this issue is to occasionally schedule fullmodel broadcasts along the process. An example is illustrated in Figure 2, where at any iteration 𝑡, 𝜽ˆ (𝑡) is the transmitted vector (either a full model or a differential update) and 𝜽˜ (𝑡) is the reconstructed model with respect to 𝜽 (𝑡). Depending on whether 𝜽ˆ (𝑡) is successfully received at device 𝑘, the local model before training, 𝜽 𝑘 (𝑡, 0), can be either 𝜽˜ (𝑡) or an outdated model 𝜽 𝑘 (𝑡 − 1, 0). The infrequent full-model broadcasts offer the possibility to re-synchronize to the latest global model, but with an increased communication cost compared to only transmitting differential updates. Bearing this in mind, we propose the following mixed-timescale differential coding (MTDC) scheme. C. Mixed-Timescale Differential Coding We consider DiC-enabled FL model broadcast with one reference model.2 Other than assigning the latest reconstructed model as the reference (e.g., DiC in Sec. II-B), MTDC offers flexibility in choosing references from different past instances in the training process. In this work, we focus on MTDC with two-level differential updates. We first introduce the iteration index sets {T𝑖 }2𝑖=0 . When 𝑡 ∈ T0 , the server broadcasts a full model, and when 𝑡 ∈ T1 and 𝑡 ∈ T2 , a first-level and a second-level differential updates are broadcast, respectively. All broadcast vectors from the server are compressed before being sent. We denote the compression functions by {𝑄 𝑖 (𝜽)}2𝑖=0 when 𝑡 ∈ T𝑖 , 𝑖 = 0, 1, 2. Let 𝜽ˆ (𝑡) be the transmitted vector and 𝜽˜ (𝑡) be the reconstructed model with respect to 𝜽 (𝑡). When the full model is broadcast, 𝜽ˆ (𝑡) = 𝑄 0 (𝜽 (𝑡)). At other times, 𝜽ˆ (𝑡) is the corresponding differential update. Then, in lieu of (3), the updating rule becomes3 Õ 𝜽 (𝑡 + 1) = 𝜽˜ (𝑡) + 𝑤 𝑘 △𝜽 𝑘 (𝑡), (4) 𝑘 ∈Π (𝑡 )

with △𝜽 𝑘 (𝑡) defined in (2) and 𝜽 𝑘 (𝑡, 0) = 𝜽˜ (𝑡), to keep the models at the server and the devices synchronized. An illustrative example with T0 = {1, 51, ...}, T1 = {11, 21, ...}, and T2 = {2, 3, ...} is in Fig. 3a: 2 It can be extended to more advanced prediction schemes using multiple reference models reconstructed at different time instances. 3 We assume perfect reception of △𝜽 (𝑡 ) by the server. 𝑘

Remark 1. Note that with transmitting either full models or differentially coded updates, the communication frequency is the same; the difference lies in what the communicated message contains. When successive global models are highly correlated, differential updates exhibit a much smaller magnitude and dynamic range than full models, thus requiring fewer quantization bits. In the following sections, we explain in detail how MTDC is implemented (Sec. III). Then, we provide a convergence analysis (Sec. IV), which motivates the age-aware MTDC and scheduling design proposed in Sec. V. Table I summarizes the notation. III. F EDERATED L EARNING WITH M IXED -T IMESCALE D IFFERENTIAL C ODING For an FL system with MTDC, at iteration 𝑡, 𝜽˜ (𝜏), 𝜏 ≤ 𝑡, is the global model that a device has worked on in order to compute its local update. This model also serves as the base model for the server to compute the broadcast for some of the future iterations (differential updates). To simplify notation, we denote the locally adopted model before local training at device 𝑘 by 𝜽 𝑘 (𝑡) (i.e., 𝜽 𝑘 (𝑡) = 𝜽 𝑘 (𝑡, 0)). Fig. 4 explains: 4 Making full-model broadcasts available to all facilitates device participation in model evolution under differential coding schemes.

4

෡ Broadcast vector 𝜽(𝑡)

෡ 𝜽(1)

෡ 𝜽(51) ෡ 𝜽(11)

Full model (a)

෡ 𝜽(21) ෡ 𝜽(20)

෡ ෡ ෡ ෡ ෡ 𝜽(5) 𝜽(2) 𝜽(10) 𝜽(12) 𝜽(6)

First-level differential Second-level differential

…

…

𝑡

…

…

…

𝑡 …

(b)

…

1

… …

2

෩ ෩ … 𝜽𝑘 (𝑡, 0) 𝜽(1) 𝜽(2)

5

6

…

෩ 𝜽(5)

෩ 𝜽(5)

…

10 ෩ 𝜽(5)

11

…

20

21

…

෩ 𝜽(11)

…

෩ 𝜽(20)

෩ 𝜽(20)

…

50 ෩ 𝜽(20)

51 ෩ 𝜽(51)

Fig. 3: (a) The server broadcasts a full model at times 1, 51, 101, ..., a first-level differential update at times 11, 21, ..., and a second-level differential update at times 2, ..., 10, 12, ..., 20, .... (b) Reception of 𝜽ˆ (𝑡) and possession of the corresponding reference model determine the adopted model at device 𝑘, 𝜽 𝑘 (𝑡, 0), to be either 𝜽˜ (𝑡) or 𝜽 𝑘 (𝑡 − 1, 0).

Device 𝑘

Server 𝜽(𝑡)

Compress: 𝑄𝑖 (∙)

encode

෡ 𝜽(𝑡)

𝑡 ∉ 𝒯0

෩ 𝑡) 𝜽(𝑟 memory

෩ 𝜽(𝑡)

…

decode

𝜽𝑘 (𝑡)

𝑡 ∉ 𝒯0 𝜽𝑘 (𝑡 − 1)

෩ 𝑡) 𝜽(𝑟

memory

Fig. 4: Block diagram of an FL system with MTDC. TABLE I: Parameter definitions at iteration 𝑡 Parameter 𝜽 (𝑡)/𝜽˜ (𝑡) 𝜽ˆ (𝑡) 𝑄 𝑖 (𝜽) △𝜽 𝑘 (𝑡) (𝑡 ) 𝑃 𝑘,𝑖

𝜽 𝑘 (𝑡) 𝑟𝑡 𝑎 𝑘(𝑡 )

Definition true/reconstructed model transmitted vector compression function for 𝑡 ∈ T𝑖 , 𝑖 = 0, 1, 2 model update from device 𝑘 decoding failure probability of device 𝑘 for 𝑡 ∈ T𝑖 , 𝑖 = 0, 1, 2 adopted model before local training at device 𝑘 (i.e., 𝜽 𝑘 (𝑡) = 𝜽 𝑘 (𝑡, 0)) timestamp of reference model. See (6) age of 𝜽 𝑘 (𝑡)

how the server computes the broadcast vector 𝜽ˆ (𝑡) and the reconstructed model 𝜽˜ (𝑡), and; • how device 𝑘 computes the adopted model 𝜽 𝑘 (𝑡), based on the received 𝜽ˆ (𝑡) and on knowledge of previous reconstructed models (e.g., 𝜽˜ (𝜏) for some 𝜏 < 𝑡), at any iteration 𝑡. More details are elaborated as follows. •

A. Server-Side Operations As illustrated in Fig. 4, the broadcast vector at iteration 𝑡 is ( 𝑄 0 (𝜽 (𝑡)), 𝑡 ∈ T0 𝜽ˆ (𝑡) = , (5) 𝑄 𝑖 (𝜽 (𝑡) − 𝜽˜ (𝑟 𝑡 )), 𝑡 ∈ T𝑖 , 𝑖 = 1, 2 where 𝑟 𝑡 is the timestamp of the reference model ( Ð max{𝜏|𝜏 ∈ T0 T1 , 𝜏 < 𝑡}, 𝑡 ∈ T1 𝑟𝑡 = , 𝑡 − 1, 𝑡 ∈ T2

(6)

being either the last full-model broadcast or first-level differential broadcast (for 𝑡 ∈ T1 ), or the model at the previous iteration (for 𝑡 ∈ T2 ). We consider 𝜈𝑖 -level random quantizers,5 for the different levels of differential coding, 𝑖 = 0, 1, 2 [31]. That is, for 𝑖 ∈ {1, 2, 3}, 𝜽 ≜ [𝑥1 , ..., 𝑥 𝑑 ], the 𝑗-th element of 𝑄 𝑖 (𝜽) is ∥𝜽 ∥ 2 · sign(𝑥 𝑗 ) · Z 𝑗 (𝜽, 𝜈𝑖 ),

(7)

5 We adopt the random quantizer for its unbiasedness property. The quantization noise has zero mean, conditioned on the iterate, which facilitates a convergence analysis of the proposed algorithm.

5

where

(

with prob. 𝜈𝑖 |𝑋 𝑗 |/∥𝜽 ∥ 2 − 𝑋 𝑗 otherwise (8) is a random variable and 𝑋 𝑗 = ⌊𝜈𝑖 |𝑥 𝑗 |/∥𝜽 ∥ 2 ⌋, 𝑗 = 1, ..., 𝑑. Clearly, 𝑄 𝑖 (0) = 0, and each pair in the following cases has the same probability distribution: (1) 𝑄 𝑖 (𝜂𝜽) and 𝜂𝑄 𝑖 (𝜽) for any 𝜂 > 0; (2) 𝑄 𝑖 (𝜽) and −𝑄 𝑖 (−𝜽). For every transmission of 𝜽, the quantization step is dynamically adjusted as ∥𝜽 ∥/𝜈𝑖 . Z 𝑗 (·, ·), ∀ 𝑗, are real numbers taken from a finite set of at most 𝜈𝑖 + 1 rational numbers. This set of rational numbers in turn can be mapped one-to-one onto a finite set of integers. Z 𝑗 (𝜽, 𝜈𝑖 ) =

𝑋 𝑗 + 1 /𝜈𝑖 , 𝑋 𝑗 /𝜈𝑖 ,

Remark 2. Note that each 𝜈𝑖 -level random quantizer 𝑄 𝑖 (𝜽), 𝑖 = 0, 1, 2, has the properties   E [𝑄 𝑖 (𝜽)|𝜽] = 𝜽, E ∥𝜽 − 𝑄 𝑖 (𝜽) ∥ 22 |𝜽 ≤ 𝜎𝑖 ∥𝜽 ∥ 22 , (9) where a higher 𝜈𝑖 gives a smaller precision constant 𝜎𝑖 . Since a full model, a first- and a second-level differential updates tend to have the highest to the lowest signal ranges respectively, the required 𝜈𝑖 for a sufficient signal quality decreases over 𝑖. This leads to 𝜎2 ≥ 𝜎1 ≥ 𝜎0 . Remark 3. For 𝑡 ∈ T𝑖 , ∀𝑖, the transmission
of the 𝜈𝑖 -levelquantized 𝜽ˆ (𝑡) requires 𝑑 ⌈log2 (𝜈𝑖 + 1)⌉ + 1 + 32 bits. The server computes the reconstructed global model by ( 𝜽ˆ (𝑡), 𝑡 ∈ T0 (10) 𝜽˜ (𝑡) = 𝜽ˆ (𝑡) + 𝜽˜ (𝑟 𝑡 ), otherwise, which is then saved in the memory of the server. This reconstructed model, 𝜽˜ (𝑡), serves as a reference model for future differential broadcasts.

compared to the maximum payload size of a data packet. Also, the differential-coding-based schemes effectively improve the communication efficiency, but require additional memory for storing the reference model. Remark 4. In principle, the MTDC mechanism is applicable also to decentralized FL frameworks [32], [33]. However, the modeling would be more involved as the links between different devices may have different quality and fail independently, and each node needs to keep track of historical model information for all its neighbors. Note that MTDC may not be directly applicable to decentralized FL with over-the-air computation, as in this case the aggregation of analog signals makes it impossible to distinguish and track individual local models separately. IV. C ONVERGENCE A NALYSIS In our MTDC FL system, some devices may conduct the local training (gradient computation) based on outdated models. The question is then, whether convergence of the learning algorithm can be guaranteed. In the following analysis, we answer this question affirmatively under standard assumptions on the objectives and some idealized additional assumptions on the model: no device scheduling (Π(𝑡) = K), a single local gradient step per iteration (𝐸 = 1), no sampling noise in the gradient computation (B𝑘 (𝑡, 0) = S𝑘 , ∀𝑘), and no quantization or communication noise in the uplink gradient transmissions. In more detail, these assumptions are as follows. Assumption 1. (Smoothness): Each local loss function 𝐹𝑘 (𝜽), ∀𝑘 is 𝐿-smooth, i.e., ∀𝜽 1 , 𝜽 2 ∈ R𝑑 ,6 ∥∇𝐹𝑘 (𝜽 1 ) − ∇𝐹𝑘 (𝜽 2 ) ∥ ≤ 𝐿 ∥𝜽 1 − 𝜽 2 ∥, or equivalently,

𝐿 B. Device-Side Operations 𝐹𝑘 (𝜽 1 ) − 𝐹𝑘 (𝜽 2 ) ≤ ∇𝐹𝑘 (𝜽 2 ) 𝑇 (𝜽 1 − 𝜽 2 ) + ∥𝜽 1 − 𝜽 2 ∥ 2 . 2 Device 𝑘 relies on 𝜽ˆ (𝑡) broadcast by the server, together with the reference model reconstructed at an earlier iteration, to compute the adopted model 𝜽 𝑘 (𝑡) before local training. That Assumption 2. (Strong convexity): Each local loss function is, 𝐹𝑘 (𝜽), ∀𝑘 is 𝜇-strongly convex, i.e., ∀𝜽 1 , 𝜽 2 ∈ R𝑑 ,  𝜇  𝜽˜ (𝑡), 𝜽ˆ (𝑡) received, 𝑡 ∈ T0 ; or  𝐹𝑘 (𝜽 1 ) − 𝐹𝑘 (𝜽 2 ) ≥ ∇𝐹𝑘 (𝜽 2 ) 𝑇 (𝜽 1 − 𝜽 2 ) + ∥𝜽 1 − 𝜽 2 ∥ 2 .   2 𝜽 𝑘 (𝑡) = 𝜽ˆ (𝑡) received, 𝑡 ∉ T0 , and 𝜽˜ (𝑟 𝑡 ) available    𝜽 𝑘 (𝑡 − 1), otherwise  (11) Let 𝑎 𝑘(𝑡 ) be the age of 𝜽 𝑘 (𝑡) relative to 𝜽˜ (𝑡), which measures ˜ where 𝑟 𝑡 and 𝜽 (𝑡) are defined in (6) and (10), respectively. A the outdatedness of the adopted model before local training. device has 𝜽˜ (𝑟 𝑡 ) in its memory only when it has successfully Then, 𝑎 𝑘(𝑡 ) = 𝑐 when 𝜽 𝑘 (𝑡) = 𝜽˜ (𝑡 − 𝑐), for 𝑐 ≥ 0. We define Õ reconstructed it in iteration 𝑟 𝑡 . If 𝜽 𝑘 (𝑡) = 𝜽˜ (𝑡), corresponding 𝜁 = 𝑤 𝑘 ∥𝜽 ∗ − 𝜽 ∗𝑘 ∥ 2 (12) to a successful reconstruction of the latest model, device 𝑘 𝑘 ∈ K ˜ saves 𝜽 (𝑡) in its memory for future computation. 𝜽 𝑘 (𝑡) will be saved as well in case of a future model reconstruction failure. to quantify the device heterogeneity, where 𝜽 ∗ = arg min 𝐹 (𝜽) and 𝜽 ∗𝑘 = arg min 𝐹𝑘 (𝜽). Then, the device model update △𝜽 𝑘 (𝑡) becomes C. Discussion on Communication and Memory Overhead Compared to the full model broadcast, the two-level MTDC requires additional communication of ⌈log2 (𝑡 − 𝑟 𝑡 )⌉ and 2 bits, for transmitting the timestamp of the reference model, and to convey the broadcast model type information in each transmission block. This extra signaling overhead is negligible

△𝜽 𝑘 (𝑡) = −𝜂∇𝐹𝑘 ( 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) )).

(13)

Assumption 3. A full model is encoded and broadcast in a way that every device can receive it. Consequently, there exists 6 ∥ · ∥ denotes the Euclidean norm.

6

𝑎 lim > 0 such that 𝑎 𝑘(𝑡 ) ≤ 𝑎 lim , ∀𝑘, ∀𝑡. This age limit 𝑎 lim is no larger than the time difference between any two adjacent fullmodel broadcasts. Furthermore, we assume no quantization error for a full-model broadcast, that is, 𝑄 0 (𝜽) = 𝜽, ∀𝜽. The following is our main theoretical result. Theorem 1. Under Assumptions 1-3, with a stepsize satisfying 𝜇  𝜂< (14)
 , √ 2 2𝐿 2 + 𝜎𝑎 ˆ lim + 4𝜎𝑎 ˆ 2lim 𝜁 + 2 where

√ √ 𝜎 ˆ = 𝜎2 + (𝜎2 + 1) max(𝜎1 , 𝜎1 , 2),

the following result holds:   E ∥ 𝜽˜ (𝑡 + 1) − 𝜽 ∗ ∥ 2 −1    2𝜂𝜖 𝜂𝜇  ⌊ 3𝑎𝑡lim +1 ⌋+1 , ≤ 1− E ∥ 𝜽˜ (1) − 𝜽 ∗ ∥ 2 + 2 𝜇

(15)

(16)

where

o p np 𝜁 [ 𝜎𝑎 ˆ lim (2𝑎 lim + 1) + 2] + 4𝜎𝑎 ˆ 2lim . 𝜖 = 𝐿2 𝜁

(17)

The expectation is taken w.r.t. the randomness in the device decoding failures and the quantization of the downlink broadcast of 𝜽ˆ (𝑡). See Appendix A for the proof. Remark 5. A larger 𝑎 lim or a larger 𝜎 ˆ requires a smaller learning rate, as indicated in (14). This increases 1 − 𝜂𝜇/2, which slows down the per-iteration contraction in (16). Fur thermore, E ∥ 𝜽˜ (𝑡 + 1) − 𝜽 ∗ ∥ 2 → 2𝜂𝜖/𝜇 when 𝑡 → ∞. The asymptotic error, 𝜂𝜖/𝜇, increases with 𝜖, indicating that smaller 𝑎 lim and 𝜎 ˆ can lead to better learning performance. Remark 6. With the MTDC scheme, some devices will use outdated models in the local training, which will cause an error in the gradient updates transmitted to the server. In the analysis, this error term, ∇𝐹𝑘 ( 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) )) − ∇𝐹𝑘 ( 𝜽˜ (𝑡)), is treated as extra additive noise on the gradients. The magnitude of this additive noise scales with ∥ 𝜽˜ (𝜏) − 𝜽 ∗ ∥, 𝜏 ≤ 𝑡. The consequence is a slowdown in the contraction speed. More explicitly, we have the following observation. Proposition 1. Because of the outdated models, the linear convergence factor changes from (1 − 𝜂𝜇) 𝑡 nominally to (1 − ⌊ 𝑡 −1 ⌋+1 −1 𝜂𝜇/2) 3𝑎lim +1 . Since 1−𝜂𝜇 < 1−𝜂𝜇/2 < 1 and ⌊ 3𝑎𝑡lim +1 ⌋+1 ≤ ⌊

𝑡 −1

⌋+1

𝑡, we have (1 − 𝜂𝜇/2) 3𝑎lim +1 > (1 − 𝜂𝜇) 𝑡 . Therefore, this represents a slow-down of the convergence. Note that this situation is somewhat akin to stochastic gradient descent with noise whose conditional second-order moment scales with the iterate, in which case the contraction speed also reduces (see, for example, [34], [35, Th. 4.6], and the NC3 T in [36]). However, we stress that Proposition 1 is valid for the specific convergence bound that we derived in (16), which in turn holds under Assumptions 1-3. The convergence analysis of FedAvg with MTDC-based DL transmission is complex. Our convergence bound is rigorous and captures all phenomena. It makes no assumptions on the statistical distributions of the difference between consecutive global models, and it explains analytically how the model

staleness caused by decoding failures impacts convergence. This in turn inspires the age-aware design introduced next. V. F EDERATED L EARNING WITH AGE -AWARE D IFFERENTIAL C ODING AND D EVICE S CHEDULING The discussion so far has assumed that every device is scheduled for uplink transmission in every iteration. To make more efficient use of uplink resources, we next introduce uplink device scheduling. As shown in Theorem 1, the convergence bound depends on 𝑎 lim , and the gradient updates obtained from stale models due to decoding failures generally lead to slower convergence. Motivated by this result, we propose a dynamic, age-aware version of MTDC and an associated age-aware scheduling policy. A. Age-Aware Mixed-Timescale Differential Coding Recall that MTDC alleviates the model staleness in the system, as exemplified in Fig. 3b. However, are there alternative ways of deciding T0 , T1 , and T2 other than the fixed periodic patterns considered in the given example? Intuitively, if decoding failures regularly happen, we should broadcast higher-level models more frequently. Otherwise, broadcasting lower-level models is more beneficial since it consumes less communication resources. As mentioned in Section II-C, with appropriate channel (𝑡 ) coding, the decoding failure probabilities follow 𝑃 𝑘,0 ≪ (𝑡 ) (𝑡 ) 𝑃 𝑘,1 ≤ 𝑃 𝑘,2 . We define i 1 Õ (𝑡 ) h (𝑡 ) (18) 𝐴0 = 𝑃 𝑘,0 𝑎 𝑘 + 1 𝐾 𝑘∈K ( (𝑡 ) 𝑎 𝑘(𝑟𝑡+1 ) > 0 1 Õ 𝑎 𝑘 +h 1, i (19) 𝐴1 = (𝑡 ) 𝐾 𝑎 𝑘(𝑡 ) + 1 , otherwise 𝑃 𝑘,1 𝑘∈K ( (𝑡 ) (𝑡 ) 1 Õ 𝑎 𝑘 + 1, 𝑎 𝑘 > 0 𝐴2 = (20) (𝑡 ) 𝐾 𝑃 𝑘,2 , otherwise 𝑘∈K Then, at any iteration 𝑡, we can predict the average device age at the next iteration 𝑡 + 1 by computing 𝐴𝑖 for 𝑡 + 1 ∈ T𝑖 , 𝑖 = 0, 1, 2. Recall that the required resources for transmitting 𝜽ˆ (𝑡), 𝑡 ∈ T𝑖 , is decreasing with 𝑖, as stronger coding protection and more-level data quantization are applied for smaller 𝑖. To balance between improving model staleness and saving communication resources, we decide 𝑡 + 1 ∈ T𝑖 , where ¯ 𝑗 = 0, 1, 2} and 𝐴¯ is a predetermined 𝑖 = max{ 𝑗 | 𝐴 𝑗 ≤ 𝐴, age limit. This way, the average device age is expected to be ¯ while for 𝜽ˆ (𝑡) with 𝑡 ∈ T𝑖 , a larger 𝑖 is preferred below 𝐴, to save communication resources since lower-level differential updates are quantized with fewer bits. We call this scheme age-aware MTDC (A-MTDC). B. Age-Aware Device Scheduling In the UL transmission phase of FL, device scheduling is typically implemented to reduce the number of communication links. Several works propose age-based scheduling designs 1) to guarantee fairness of device participation, which effectively tackles the issue of data heterogeneity [37]–[41]; or 2) to

7

TABLE II: The notation in the algorithms. Lvl𝑡 DlModelLvl(·) 𝐴¯ BroadcastMdl(·) Rct(·) MemUpdateSrv(·) MemUpdateUsr(·) 𝑝 𝑘 (𝑡), 𝑘 ∈ K

Definition 0: 𝑡 ∈ T0 ; 1: 𝑡 ∈ T1 ; 2: 𝑡 ∈ T2 the function to decide 𝑡 ∈ T0 , T1 , or T2 age limit of A-MTDC the function to compute 𝜽ˆ (𝑡) the function to compute 𝜽 𝑘 (𝑡) memory update at the server memory update at a device device scheduling probability

Algorithm 1 FL with A-MTDC and Age-aware Device Scheduling (0)

1: Initialize: Lvl1 = 0, 𝑎 𝑘 = 0, 𝜽 𝑘 (0) = 0, ∀𝑘. 2: for 𝑡 = 1, ..., 𝑇 do 3: if 𝑡 > 1 then 4: Lvl𝑡 ← (𝑡 ) (𝑡 ) (𝑡 ) ¯ 5: DlModelLvl({𝑎 𝑘(𝑡 ) , 𝑎 𝑘(𝑟𝑡+1 ) } ∀𝑘 , {𝑃 𝑘,0 , 𝑃 𝑘,1 , 𝑃 𝑘,2 } ∀𝑘 , 𝐴) 6: end if 7: 8: 9: 10:

down-weight the contributions of stale updates and therefore improve the learning performance [42]. In our system, since the model update △𝜽 𝑘 (𝑡) based on an outdated model (i.e., 𝜽 𝑘 (𝑡) = 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) ) with 𝑎 𝑘(𝑡 ) > 0) may negatively affect the FL performance, we prioritize devices with fresher models, by deciding Π(𝑡) at random in every iteration, based on a set of age-aware scheduling probabilities {𝑝 𝑘 (𝑡)} 𝐾 𝑘=1 . We select these probabilities as (𝑡 )

𝑝 𝑘 (𝑡) = Í

𝑒 −𝑎𝑘 /𝑎max (𝑡 )

−𝑎 /𝑎max 𝑖∈K 𝑒 𝑖

, ∀𝑘,

(21)

where 𝑎 max = max 𝑘 ∈ K 𝑎 𝑘(𝑡 ) . Consequently, devices with outdated models will be assigned smaller scheduling probabilities and thus are less likely to participate in the model aggregation. Remark 7. Our proposed device scheduling policy prioritizes devices whose local model updates are computed based on fresher global models. As shown in [40], [41], maintaining fairness in device participation is another important aspect for the learning performance in non-IID (independently and identically distributed) data scenarios. Note that in [40] and [41], the local updates are computed based on the same global model, while in our framework we need to deal with stale global models at the participating devices. Finding the optimal balance between participation fairness and information freshness in the scheduling design could be worth exploring in future work. We summarize the operation of the proposed FL system in Algorithm 1 (notation in Table II). Detailed steps at the server and devices are in Algorithms 2, 3, respectively.

11:

𝜽ˆ (𝑡) ←BroadcastMdl 𝜽 (𝑡), 𝜽˜ (𝑟 𝑡 ), Lvl𝑡 MemUpdateSrv( 𝜽ˆ (𝑡), 𝜽˜ (𝑟 𝑡 ), Lvl𝑡 ) for all device 𝑘 ∈ K do in parallel (𝑡 ) 𝜽 𝑘 (𝑡),  𝑎𝑘 ←  Rct 𝜽ˆ (𝑡), 𝜽 𝑘 (𝑡 − 1), 𝜽˜ (𝑟 𝑡 ), 𝑎 (𝑡 −1) , Lvl𝑡 𝑘

MemUpdateUsr(𝜽 𝑘 (𝑡), 𝑎 𝑘(𝑡 ) ) end for The server computes (21) to determine Π(𝑡). for all device 𝑘 ∈ Π(𝑡) do in parallel Local training with 𝜽 𝑘 (𝑡), obtain △𝜽 𝑘 (𝑡) and transmit it to the server. 17: end for Í 18: The server computes 𝑘 ∈Π (𝑡 ) 𝑤 𝑘 (𝑡)△𝜽 𝑘 (𝑡) and renews the model by (4). 19: end for 12: 13: 14: 15: 16:

Algorithm 2 Server Operations   (𝑡 ) (𝑟 ) (𝑡 ) (𝑡 ) (𝑡 ) 1: Lvl𝑡 =DlModelLvl {𝑎 𝑘 , 𝑎 𝑘 𝑡+1 } ∀𝑘 , {𝑃 𝑘,0 , 𝑃 𝑘,1 , 𝑃 𝑘,2 } ∀𝑘 , 𝐴¯ : 2: Compute { 𝐴𝑖 } 2𝑖=0 in (18)-(20).

¯ 𝑗 = 0, 1, 2}. 3: Lvl𝑡 = max{ 𝑗 | 𝐴 𝑗 ≤ 𝐴, 4:
5: 𝜽ˆ (𝑡) = BroadcastMdl 𝜽 (𝑡), 𝜽˜ (𝑟 𝑡 ), Lvl𝑡 : 6: if Lvl𝑡 = 0 then 7: 𝜽ˆ (𝑡) ← 𝑄 0 (𝜽 (𝑡)). 8: else
9: 𝑖 ← Lvl𝑡 , 𝜽ˆ (𝑡) ← 𝑄 𝑖 𝜽 (𝑡) − 𝜽˜ (𝑟 𝑡 ) . 10: end if 11: 12: MemUpdateSrv( 𝜽ˆ (𝑡), 𝜽˜ (𝑟 𝑡 ), Lvl𝑡 ): 13: if Lvl𝑡 = 0 then

14: Save 𝜽˜ (𝑡) = 𝜽ˆ (𝑡) to the memory. 15: else 16: Save 𝜽˜ (𝑡) = 𝜽ˆ (𝑡) + 𝜽˜ (𝑟 𝑡 ) to the memory. 17: end if

VI. S IMULATIONS We train two convolutional neural networks parameterized by 𝜽 ∈ R21840 and 𝜽 ∈ R62006 , with MNIST [43] and CIFAR10 [44] datasets, respectively. There are 𝐾 = 20 devices in the system. The training data are allocated to each device in a non-IID fashion. Each device contains training data of up to 6 different classes/labels. To evaluate the learning performance, at every iteration, the global model is tested on the testing datasets. The test accuracy (defined as the percentage of correct classification instances) is then measured. A. Gain of Mixed-Timescale Differential Coding We evaluate the performance of the following schemes:

•

’AllFull’: baseline method, always broadcasting a full model in every iteration, i.e., 𝑡 ∈ T0 , ∀𝑡. • ’DiC-𝜌’: state-of-the-art method [19], broadcasting a full model every 𝜌 iterations and a differential update at all other iterations, i.e., ( T0 , 𝑡 = 1 + 𝜌𝑛, 𝑛 = 0, 1, ... 𝑡∈ T2 , otherwise To ensure a fair performance comparison, we periodically allocate the full model to DiC [19]. • ’MTDC-(𝜌1 , 𝜌2 )’: proposed method, broadcasting either a full model, with fixed period 𝜌1 ; or a first-level differen-

8

Algorithm 3 Device Operations   (𝑡 ) (𝑡 −1) 1: 𝜽 𝑘 (𝑡), 𝑎 𝑘 = Rct 𝜽ˆ (𝑡), 𝜽 𝑘 (𝑡 − 1), 𝜽˜ (𝑟 𝑡 ), 𝑎 𝑘 , Lvl𝑡 : 2: if fail to decode 𝜽ˆ (𝑡) then 3: 𝜽 𝑘 (𝑡) = 𝜽 𝑘 (𝑡 − 1), 𝑎 𝑘(𝑡 ) = 𝑎 𝑘(𝑡 −1) + 1. 4: else if Lvl𝑡 = 0 then 5: 𝜽 𝑘 (𝑡) = 𝜽ˆ (𝑡), 𝑎 𝑘(𝑡 ) = 0. 6: else if 𝜽˜ (𝑟 𝑡 ) available then 7: 𝜽 𝑘 (𝑡) = 𝜽ˆ (𝑡) + 𝜽˜ (𝑟 𝑡 ), 𝑎 𝑘(𝑡 ) = 0. 8: else 9: 𝜽 𝑘 (𝑡) = 𝜽 𝑘 (𝑡 − 1), 𝑎 𝑘(𝑡 ) = 𝑎 𝑘(𝑡 −1) + 1. 10: end if 11:

(a)

100

(𝑡 )

(𝑡 )

14: if 𝑎 𝑘

= 0 then

15: Save 𝜽˜ (𝑡) = 𝜽 𝑘 (𝑡) to the memory. 16: end if

tial update, with period 𝜌2 ; or a second-level differential update, in all other iterations:   T , 𝑡 = 1 + 𝜌1 𝑛, 𝑛 = 0, 1, ...    0 𝑡 ∈ T1 , 𝑡 = 1 + 𝜌2 𝑛, 𝑡 ≠ 1 + 𝜌1 𝑛, 𝑛 = 0, 1, ...    T2 , otherwise.  The decoding failure probabilities are fixed over time and (𝑡 ) 2 across devices, i.e., {𝑃 𝑘,𝑖 }𝑖=0 = [0.001, 0.2, 0.25], ∀𝑘, ∀𝑡. We schedule the devices uniformly at random, with scheduling ratios |Π(𝑡)|/𝐾 = 0.1, and 0.25, respectively for the datasets MNIST and CIFAR-10. For fair comparison between the different methods (full model broadcast, DiC, and MTDC), we keep the time-average of the DL transmission bit rate approximately the same (up to rounding effects). The exact number of bits in a given iteration may vary between different methods. Following this guideline, the quantization levels (𝜈𝑖 ) are chosen as: • ’AllFull’: 𝜈0 = 31 • ’DiC-𝜌’: 𝜈0 = 255 and 𝜈2 = 15 • ’MTDC-(𝜌1 , 𝜌2 )’: 𝜈0 = 255, 𝜈1 = 127, and 𝜈2 = 7 The assignment of {𝜈𝑖 } is consistent with the ordering discussed in Remark 2. For the case of MNIST, the average bit rate of each method is illustrated in Fig. 5a. With this normalization, all methods consume approximately the same amount of communication resources over time. Minor discrepancies arise due to rounding effects in the selection of quantization levels. In general, the average bit rate increases as the frequency of higher-resolution broadcasts increases. As discussed earlier, our MTDC scheme requires extra memory usage. As an example, the 62006-parameter convolutional neural network model (for the CIFAR-10 dataset), with 32-bit precision, requires less than 1 MB memory. This is relatively little compared to the storage capacity of modern edge devices. Figs. 5b and 6 show the test accuracy comparison between different schemes, for the datasets MNIST and CIFAR-10, respectively. The AllFull method performs the worst, since it transmits the full model at every iteration and therefore

Test accuracy (%)

12: MemUpdateUsr(𝜽 𝑘 (𝑡), 𝑎 𝑘 ): 13: Save 𝜽 𝑘 (𝑡) to the memory.

80 AllFull DiC-5 DiC-7 DiC-10 MTDC-(8,4) MTDC-(10,5)

60

40

20

0

10

20

30

40

50

60

iteration (t) (b)

Fig. 5: Comparison of average communication resource consumption and test accuracy for different schemes (MNIST). relies on low-resolution quantization. In contrast, the differential coding schemes – both the state-of-the-art DiC and the proposed MTDC – mitigate this limitation by transmitting updates with a smaller dynamic range that can be more aggressively compressed. Moreover, for the differential coding schemes, more frequent higher-level broadcasts generally lead to improved test accuracy (e.g., in Fig. 5b, DiC-5 outperforms DiC-10, and MTDC-(8,4) outperforms MTDC-(10,5)), albeit at the cost of slightly higher average bit rates. Finally, and most importantly, MTDC consistently achieves strong learning performance with lower communication overhead than the state-of-the-art methods, owing to its increased resilience to decoding failures.7 B. A-MTDC versus MTDC To demonstrate the effectiveness of A-MTDC, we simulate the following two vanilla MTDC methods for comparison: MTDC-(10, 5) and MTDC-(15, 5). Two cases of device (𝑡 ) 2 decoding failure probabilities are considered: {𝑃 𝑘,𝑖 }𝑖=0 = (𝑡 ) 2 [0.0005, 0.05, 0, 1] and {𝑃 𝑘,𝑖 }𝑖=0 = [0.0005, 0.1, 0, 3]. The device scheduling ratios |Π(𝑡)|/𝐾 are 0.5 or 0.1. We set the precision of the random quantizer to {𝜈𝑖 }2𝑖=0 = [127, 63, 15]. 7 Since the conclusions from the experiments with CIFAR-10 are consistent with those from MNIST, we only show the results for MNIST for the remaining experiments.

9

100

40 AllFull DiC-10 MTDC-(10,5) 20 0

200

400

600 iteration (𝑡)

800

Test accuracy (%)

Test accuracy (%)

60

0.5: (10,5) 0.5: (15,5) 0.5: 𝐴¯ = 2 0.1: (10,5) 0.1: (15,5) 0.1: 𝐴¯ = 2

60 40 20

1,000

0

10

20

30

iteration (𝑡)

Fig. 6: Comparison of test accuracy for different schemes (CIFAR-10).

(𝑡 ) 2 (a) {𝑃 𝑘,𝑖 }𝑖=0 = [0.0005, 0.05, 0, 1].

100 Test accuracy (%)

When decoding failures are relatively rare, as in Fig. 7a, AMTDC (with 𝐴¯ = 2) and the vanilla MTDC methods perform similarly in test accuracy for both scenarios |Π(𝑡)|/𝐾 = 0.5 (curves with legend ’0.5:’) and |Π(𝑡)|/𝐾 = 0.1 (legend ’0.1:’). As A-MTDC keeps track of the device age along the learning process and dynamically chooses an appropriate model type to broadcast, the test accuracy improves more smoothly over time compared to the vanilla MTDC methods. All methods consume a similar amount of communication resources, according to the calculation from Remark 3. With more frequent decoding failures, as in Fig. 7b, the vanilla MTDC methods face regular test accuracy drops while A-MTDC does not, thanks to its quick reaction to model staleness. The price to pay for this is additional communication resources (5% more than vanilla MTDC in this case).

80

80 0.5: (10,5) 0.5: (15,5) 0.5: 𝐴¯ = 2 0.1: (10,5) 0.1: (15,5) 0.1: 𝐴¯ = 2

60 40 20

0

10

20

30

iteration (𝑡) (𝑡 ) 2 (b) {𝑃 𝑘,𝑖 }𝑖=0 = [0.0005, 0.1, 0, 3].

Fig. 7: Test accuracy comparison of A-MTDC ( 𝐴¯ = 2) and vanilla MTDC, with patterns (10, 5) and (15, 5), for device scheduling ratios 0.5 and 0.1.

C. Advantages of Age-Aware Device Scheduling We demonstrate the performance gain of our proposed ageaware scheduling over the baseline random scheduling, and over a state-of-the-art version-age-based scheduling policy [38]8 , in the presence of decoding failures. We consider the same setting in Section VI-A. As shown in Figure 8, our age-aware scheduling method generally outperforms the others for FL frameworks with differential model broadcasts. Our method prioritizes devices with fresher model updates and achieves better test accuracy than the version-age-based policy. This suggests that, for scheduling design, excluding stale updates is more important than ensuring fairness of device participation when the system is subject to decoding failures. On the other hand, when the overall situation of model outdatedness is mild, e.g., when AllFull is adopted, all three methods will have similar device scheduling probabilities, which is reflected by their similar learning performance. VII. C ONCLUSIONS AND F UTURE W ORK This paper proposed a mixed-timescale differential coding (MTDC) framework for DL transmission of global models in 8 This method prioritizes devices with low participation frequency, and therefore minimizes the overall device staleness in the system.

FL systems, leveraging the temporal correlation among model iterates. Compared to always broadcasting the full model and to conventional DiC schemes, MTDC achieves competitive learning performance while significantly improving communication efficiency and robustness to DL decoding failures. We established convergence guarantees for FedAvg under the proposed MTDC framework, revealing how model staleness induced by decoding failures can slow the per-iteration contraction toward the optimum. These insights motivated the design of an age-aware extension, A-MTDC, along with an age-aware device scheduling policy. The effectiveness of the proposed schemes was validated through simulations, which demonstrate consistent learning performance gains over existing methods under comparable communication budgets. The MTDC framework inherently involves trade-offs between the frequency of full-model broadcasts and that of firstlevel differential updates. Optimally balancing these transmissions is highly scenario-dependent (e.g., on the DL decoding failure statistics) and a detailed quantitative optimization is therefore left for future work. On the other hand, while this work adopts scalar quantization, further performance improvement may be achievable by incorporating more advanced

10

AllFull, rdm AllFull, age AllFull, v-age DiC-10, rdm DiC-10, age

DiC-10, v-age A-MTDC: 𝐴¯ = 2, rdm A-MTDC: 𝐴¯ = 2, age A-MTDC: 𝐴¯ = 2, v-age

Test accuracy (%)

100 80

we have (hereafter, 1 = [1, ..., 1] 𝑇 ) h  i 2 2 𝛿𝑡+1 = 𝜽˜ (𝑡) − 𝑄 2 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 − 𝜽 ∗
𝑇 = 𝛿𝑡2 − 2𝜂 𝜽˜ (𝑡) − 𝜽 ∗ 𝑮 (𝑡1) (22a) h  i 2 + 𝑄 2 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 (22b) ( ) h  i   𝑇 (𝑡 ) . + 2 𝜽˜ (𝑡) − 𝜽 ∗ 𝜂𝑮 (𝑡1) − 𝑄 2 𝜂𝑮 {𝑡 − 𝑎 } ∀𝑘 𝑘

(22c) 60

First, based on Assumption 2 and using that 𝐹 ( 𝜽˜ (𝑡)) ≥ 𝐹 (𝜽 ∗ ), Õ  
𝑤 𝑘 𝐹𝑘 𝜽˜ (𝑡) − 𝐹𝑘 (𝜽 ∗ ) + 𝜇𝛿𝑡2 /2 (22a) ≤ 𝛿𝑡2 − 2𝜂

40

𝑘∈K

≤ (1 − 𝜇𝜂) 𝛿𝑡2 , 20

0

20

40

60

iteration (t) Fig. 8: Learning performance with random scheduling (’rdm’), version-age-based scheduling (’v-age’), and the proposed ageaware scheduling scheme (’age’) for different schemes.

quantization techniques, such as vector quantization [45], [46]. Finally, the MTDC mechanism is also applicable to decentralized FL frameworks, which requires more complex modeling and design considerations. We leave this part to our future studies. A PPENDIX We introduce the following auxiliary variables: • the aggregated gradient updates Õ 𝑤 𝑘 ∇𝐹𝑘 ( 𝜽˜ (𝑡 𝑘 )), 𝑮 (𝒕) = 𝑘∈K

]𝑇

where 𝒕 = [𝑡 1 , ..., 𝑡 𝐾 records the timestamps of the global model at each device, and ˜ (𝑡) to the optimum • the distance of the 𝑡-th iterate 𝜽 𝛿𝑡 = ∥ 𝜽˜ (𝑡) − 𝜽 ∗ ∥. Useful lemmas and inequalities can be found in Appendix B. A. Proof of Theorem 1 2 We investigate how ∥ 𝜽˜ (𝑡 + 1) − 𝜽 ∗ ∥ 2 = 𝛿𝑡+1 evolves over time in different scenarios, i.e., when 𝑡 + 1 ∈ T0 , T1 , and T2 . 1) If 𝑡 + 1 ∈ T2 , a Second-Level Differential Update: Based on (10), (6), and (5),

𝜽˜ (𝑡 + 1) = 𝜽˜ (𝑡) + 𝜽ˆ (𝑡 + 1) = 𝜽˜ (𝑡) + 𝑄 2 [𝜽 (𝑡 + 1) − 𝜽˜ (𝑡)].

and therefore after taking total expectation,   E [(22a)] ≤ (1 − 𝜇𝜂) E 𝛿𝑡2 .

(24)

For (22b), we first evaluate the conditional expectation based on a realization up to iteration 𝑡, and then take expectation:   n o𝐾   (𝑡 ) ˜ E [(22b)] = E E (22b) 𝜽 (𝑡 − 𝑎 𝑘 ) 𝑘=1    2 ≤ (𝜎2 + 1)𝜂2 E 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 (25) # "  Õ  Õ ∗ ∗ 2 2 2 2 𝑤 𝑘 ∥𝜽 𝑘 − 𝜽 ∥ ≤ 2𝐿 𝜂 (𝜎2 + 1) 𝑤 𝑘 E 𝛿 (𝑡 ) + 𝑡 −𝑎 𝑘

𝑘∈K

( ≤ 𝐿 2 𝜂2 𝜎 ˆ



Õ 𝑘∈K

(𝑡 )

𝑡 −𝑎 𝑘

𝑘∈K

)

(26)

+𝜁 ,

(27)



𝑤 𝑘 E 𝛿2

where (25) follows from (9) as     E ∥𝑄 2 (𝜽)∥ 2 |𝜽 = E ∥𝑄 2 (𝜽) − 𝜽 ∥ 2 |𝜽 + ∥𝜽 ∥ 2 = (𝜎2 + 1) ∥𝜽 ∥ 2 ,

(28)

and 𝑄 2 (𝜂𝒙) has the same probability distribution as 𝜂𝑄 2 (𝒙); (26) follows from (48), Assumption 1, and (49); (27) is based on (12) and (15).9 We handle (22c) with a similar approach (evaluating conditional expectation and using E [𝑄 2 (𝜽)] = 𝜽): o ii n h h E [(22c)] = E E (22c) 𝜽˜ (𝑡), 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) ) ∀𝑘 ( )  h i
𝑇 (𝑡 ) ∗ 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 } ∀𝑘 = 2𝜂E 𝜽˜ (𝑡) − 𝜽 𝑘

(

  ≤ 2𝜂E 𝛿𝑡 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘

) ,

(29)

"

 p    p ≤ 2𝐿 𝜎𝑎 ˆ lim 𝜂 2𝑎 lim 𝜁 + 𝑎 lim 2 𝜁 + 3/2 E 𝛿𝑡2 2

2

Furthermore with (4), (13), and (𝑡 ) 𝑇 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 = [𝑡 − 𝑎 1(𝑡 ) , ..., 𝑡 − 𝑎 𝐾 ] ,

(23)

+

Õmin(𝑡 −1,3𝑎lim ) 𝑖=1

  E 𝛿𝑡2−𝑖 /2

# (30)

where (29) follows from CauchySchwarz inequality and (30) is based on Lemma 1 (given in Appendix B). 9 Introducing the constant 𝜎 ˆ yields a simpler bound, though it may not be the tightest.

11

Combining (24), (27), and (30),  p i    2  h E 𝛿𝑡+1 ≤ 1 − 𝜇𝜂 + 2𝐿 2 𝜂2 𝜎𝑎 ˆ 2lim 2 𝜁 + 3/2 E 𝛿𝑡2 (   Õ p p 2 2 +𝐿 𝜂 𝜎 ˆ 𝜁 𝜁 + 4𝑎 2lim + 𝑤 𝑘 E 𝛿2 𝑘∈K

+ 𝑎 lim

Õmin(𝑡 −1,3𝑎lim ) 𝑖=1

  E 𝛿𝑡2−𝑖

)

For (35b), we first deal with the randomness of 𝑄 1 (·),  (𝑡 )

𝑡 −𝑎 𝑘

(31)

2) If 𝑡 +1 ∈ T1 , a First-Level Differential Update: Note that the server broadcasts second-level differential updates between iteration 𝑟 𝑡+1 and 𝑡 + 1, i.e., {𝑟 𝑡+1 + 1, ..., 𝑡} ⊂ T2 . Hence, the reference model for reconstructing 𝜽˜ (𝑡 + 1) satisfies 𝜽˜ (𝑟 𝑡+1 ) +

𝑡 −1 Õ 𝜏=𝑟𝑡+1

h

𝑄 2 −𝜂𝑮



{𝜏 − 𝑎 𝑘( 𝜏 ) } ∀𝑘

i

= 𝜽˜ (𝑡).

E [(35b)] " # h n i o (𝜏) ˜ = E E (35b) 𝜽 (𝜏 − 𝑎 𝑘 ) 𝑟 𝑡+1 ≤ 𝜏 ≤ 𝑡, ∀𝑘 , 𝑄 2 (·)   ≤ 𝜎1 E ∥ 𝒚 𝑡+1 ∥ 2 (

𝑡 −1 Õ

+

𝜏=𝑟𝑡+1

h

Õ𝑡 −1 (

≤ 𝑎 lim 𝜎1 𝜂2 𝐿 2 (32)

h  i   𝑄 2 −𝜂𝑮 {𝜏 − 𝑎 𝑘( 𝜏 ) } ∀𝑘 − 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 .



≤ 𝜎1 E (𝑡 + 1 − 𝑟 𝑡+1 ) 𝜂2 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘

+ (𝜎2 + 1) E

Define 𝒚 𝑡+1 =

(37) h

𝜏=𝑟𝑡+1

Õ 𝑘∈K

≤ 2𝑎 lim 𝜎1 𝜂 𝐿

{𝜏 − 𝑎 𝑘( 𝜏 ) } ∀𝑘

i 2 i

) (38)

𝑤 𝑘 E ∥ 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) ) − 𝜽 ∗𝑘 ∥ 2 #)

" 𝑡 −1 Õ Õ (

2 2

𝑄 2 −𝜂𝑮



 2

𝜏=𝑟𝑡+1 𝑘 ∈ K

𝑤 𝑘 ∥ 𝜽˜ (𝜏 − 𝑎 𝑘( 𝜏 ) ) − 𝜽 ∗𝑘 ∥ 2

𝜁 [(𝜎2 + 1) 𝑎 lim + 1] +

Õ 𝑘∈K

(39)

 𝑤 𝑘 E 𝛿2

 (𝑡 )

𝑡 −𝑎 𝑘

) Õ
(33) Õ 𝑡 −1 The broadcast model 𝜽ˆ (𝑡 + 1) = 𝑄 1 𝜽 (𝑡 + 1) − 𝜽˜ (𝑟 𝑡+1 ) can + (𝜎2 + 1) 𝑤𝑘E 𝛿2 , (40) 𝑘∈K 𝜏=𝑟𝑡+1 𝜏−𝑎 𝑘( 𝜏) then be expressed as h   i 𝜽ˆ (𝑡 + 1) = 𝑄 1 𝜽˜ (𝑡) − 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 − 𝜽˜ (𝑟 𝑡+1 ) = 𝑄 1 ( 𝒚 𝑡+1 ), where (37) evaluates the expectation over the random quantizer (34) 𝑄 1 (·) according to (9), conditioned on the previously reconstructed models and 𝑄 2 -quantized components in 𝒚 𝑡+1 ; (38)   by using 𝜽 (𝑡 + 1) = 𝜽˜ (𝑡) − 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 , (32), and (33). is based on (33) and (49); (39) follows from applying (48), Assumption 1, and (28); and (40) is obtained by applying (49) Since 𝜽˜ (𝑡 + 1) = 𝜽˜ (𝑟 𝑡+1 ) + 𝜽ˆ (𝑡 + 1), by using (32) and (34), and (12). For (35c), we have 2 h   i 𝛿𝑡+1 (𝑡 ) [(35c)] E ≤ 2𝜂E 𝛿 · 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 } 𝑡 ∀𝑘 𝑘 𝑡 −1 h  i Õ 2
" 𝑄 2 −𝜂𝑮 {𝜏 − 𝑎 𝑘( 𝜏 ) } ∀𝑘 + 𝑄 1 𝒚 𝑡+1 − 𝜽 ∗ = 𝜽˜ (𝑡) −     p p 𝜏=𝑟𝑡+1 ≤ 2𝐿 2 𝜎𝑎 ˆ lim 𝜂2 2𝑎 lim 𝜁 + 𝑎 lim 2 𝜁 + 3/2 E 𝛿𝑡2   2
# = 𝜽˜ (𝑡) − 𝜽 ∗ + 𝑄 1 𝒚 𝑡+1 − 𝒚 𝑡+1 − 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 Õmin(𝑡 −1,3𝑎lim )   2   E 𝛿𝑡 −𝑖 /2 , + (41) 2
𝑇 𝑖=1 = 𝛿𝑡2 − 2𝜂 𝜽˜ (𝑡) − 𝜽 ∗ 𝑮 (𝑡1) + 𝜂2 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 (35a) based on CauchySchwarz inequality and Lemma 1. For (35d),
10 2 + ∥𝑄 1 𝒚 𝑡+1 − 𝒚 𝑡+1 ∥ (35b) since 𝑄 1 (·) is unbiased, as given in (9), we have  i h i
𝑇 h + 2𝜂 𝜽˜ (𝑡) − 𝜽 ∗ 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 (35c) E (35d) 𝜽˜ (𝑡), { 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) )|1 ≤ 𝑘 ≤ 𝐾 }, 𝑄 2 (·) = 0.  o  𝑇 n
Together with (36), (40), (41), 𝜎 ˆ ≥ 2𝜎2 ≥ 2𝜎1 , and 𝜎 ˆ ≥ + 2 𝑄 1 𝒚 𝑡+1 − 𝒚 𝑡+1 𝜽˜ (𝑡) − 𝜽 ∗ − 𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 . 𝜎 (𝜎 + 1), we have 1 2 (35d)     p  2   E 𝛿𝑡+1 ≤ 1 − 𝜇𝜂 + 2𝐿 2 𝜎𝑎 ˆ 2lim 𝜂2 2 𝜁 + 3/2 E 𝛿𝑡2 By applying (23), (48), Assumption 1, (49), and (12), Õ i p hp 2 2 2 (35a) ≤ (1 − 𝜇𝜂) 𝛿𝑡2 + 2𝐿 2 𝜂2 𝑤 𝑘 𝛿2 (𝑡 ) + 2𝐿 2 𝜂2 𝜁 . (2 (2𝑎 + 𝐿 𝜂 𝜁 𝜁 + 𝜎𝑎 ˆ + 1)) + 4 𝜎𝑎 ˆ lim lim lim 𝑘∈K 𝑡 −𝑎 𝑘   Õ 2 2 2 By taking total expectation of (35a), + 𝐿 𝜂 (2 + 𝜎𝑎 ˆ lim ) 𝑤 𝑘 E 𝛿 (𝑡 ) ) ( 𝑘∈K 𝑡 −𝑎 𝑘   Õ  Õ  2 Õ 2 2 2 𝑡 −1 E [(35a)] ≤ (1 − 𝜇𝜂) E 𝛿𝑡 + 2𝐿 𝜂 𝑤 𝑘 E 𝛿 (𝑡 ) + 𝜁 . + 2𝑎 lim 𝜎𝐿 ˆ 2 𝜂2 𝑤𝑘E 𝛿2 ( 𝜏) 𝑡 −𝑎 𝑘 𝑘∈K 𝜏=𝑟𝑡+1 𝜏−𝑎 𝑘 𝑘∈K Õmin(𝑡 −1,3𝑎lim )  (36)  + 𝐿 2 𝜎𝑎 ˆ lim 𝜂2 E 𝛿𝑡2−𝑖 . (42) 𝑖=1

10 The expectation is conditioned on those 𝑄 -quantized components in 2

𝒚 𝑡+1 .

12

2 3) If 𝑡 + 1 ∈ T0 : The upper bound of 𝛿𝑡+1 can be similarly derived as in the case of 𝑡 +h 1 ∈T2 . Specifically, i

{𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘

(22) holds with 𝑄 2 𝜂𝑮 replaced by   𝜂𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 ; • (27) holds with 𝜎 ˆ replaced by a scaling 2; • (24) and (30) hold without any change. This gives i    p  2  h E 𝛿𝑡+1 ≤ 1 − 𝜇𝜂 + 2𝐿 2 𝜂2 𝜎𝑎 ˆ 2lim 2 𝜁 + 3/2 E 𝛿𝑡2 (    Õ p p 2 2 + 𝐿 𝜂 2 𝜁 𝜁 + 2𝑎 2lim 𝜎 ˆ +2 𝑤 𝑘 E 𝛿2 (𝑡 ) •

𝑘∈K

+ 𝑎 lim 𝜎 ˆ

Õmin(𝑡 −1,3𝑎lim ) 𝑖=1

𝑡 −𝑎 𝑘

)

  E 𝛿𝑡2−𝑖 .

Then, (42) can be rearranged as11    h i  2  2 E 𝛿𝑡+1 ≤ 𝐶 max E 𝛿𝑡2 , ..., E 𝛿max(1,𝑡 + 𝜂2 𝜖, (44) −3𝑎lim ) where 𝜖 is defined in (17). We will prove by induction that Õ𝑡 −1  2  ⌊ 𝑡 −1 ⌋+1   𝐶𝑖 . (45) E 𝛿𝑡+1 ≤ 𝐶 3𝑎lim +1 E 𝛿12 + 𝜂2 𝜖 𝑖=0  2  2 When 𝑡 = 1, (44) gives E 𝛿2  ≤ 𝐶E 𝛿1 + 𝜂2 𝜖. Assuming  2 that (45) holds, we evaluate E 𝛿𝑡+2 based on (44): i  h  2   2  2 + 𝜂2 𝜖 E 𝛿𝑡+2 ≤ 𝐶 max E 𝛿𝑡+1 , ..., E 𝛿max(1,𝑡+1−3𝑎 lim )   max ( 1,𝑡+1−3𝑎 ) −2 Õ𝑡 −1 lim ⌋+1  2  ⌊ 3𝑎lim +1 E 𝛿1 + 𝜂 2 𝜖 𝐶 𝑖 + 𝜂2 𝜖 (46) ≤𝐶 𝐶 =𝐶

  E 𝛿12 + 𝜂2 𝜖

𝑖=0

𝑘∈K

𝑘∈K

As a special case, Õ𝑁 𝑖=1

𝒂𝑖

2

≤𝑁

Õ𝑁 𝑖=1

∥ 𝒂𝑖 ∥ 2 .

(48)

(49)

Lemma 1. The following result holds, h   i h p E 𝛿𝑡 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 ≤ 𝐿 2 𝜎𝑎 ˆ lim 𝜂 2𝑎 lim 𝜁  p    Õmin(𝑡 −1,3𝑎lim )   i E 𝛿𝑡2−𝑖 /2 . + 𝑎 lim 2 𝜁 + 3/2 E 𝛿𝑡2 +

(43)

+ 1 − 𝜇𝜂 < 1 − 𝜂𝜇/2 ≜ 𝐶.

Õ𝑡

Let 𝒂 𝑘 ∈ R𝑑 , ∀𝑘 ∈ K. Based on Jensen’s inequality, Õ Õ 𝑝 𝑤𝑘 𝒂𝑘 ≤ 𝑤 𝑘 ∥ 𝒂 𝑘 ∥ 𝑝 , 𝑝 = 1, 2.

𝑖=1

Comparing (31), (42), and (43), we conclude that (42) holds for all 𝑡, as it has the highest upper bound. 4) Overall Convergence Bound: Based on (14),  p h  i ˆ 2lim 2 𝜁 + 3/2 + 2 + 𝜎𝑎 𝐿 2 𝜂2 2𝜎𝑎 ˆ lim + 2𝜎𝑎 ˆ 2lim + 3𝜎𝑎 ˆ 2lim

max ( 3𝑎 ,𝑡 ) ⌊ 3𝑎 lim ⌋+1 lim +1

B. Useful Inequalities and Lemmas

𝑖=0

𝐶𝑖

(47)

where (46) holds because the first and second terms in (45) decreases and increases with 𝑡, respectively. Note that   ( 3𝑎lim ⌊ ⌋ = ⌊ 3𝑎lim𝑡 +1 ⌋, 𝑡 < 3𝑎 lim max (3𝑎 lim , 𝑡) . = 3𝑎lim𝑡 +1 3𝑎 lim + 1 ⌊ 3𝑎lim +1 ⌋, 𝑡 ≥ 3𝑎 lim Then (47) can be rewritten as Õ𝑡  2  ⌊ 𝑡 ⌋+1   E 𝛿𝑡+2 ≤ 𝐶 3𝑎lim +1 E 𝛿12 + 𝜂2 𝜖

𝑖=0

𝐶𝑖 ,

which the proof. Finally, (16) follows from (45) and Í𝑡 −1 completes 𝑖 < 1/(1 − 𝐶). 𝐶 𝑖=0 11 In (42), the 𝜏-summation has no more than 𝑎 lim terms and the 𝑖summation has at most 3𝑎lim terms, respectively.

Proof. By applying (48), Assumption 1, and the triangle inequality, " #   (𝑡 ) E 𝛿𝑡 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘 } ∀𝑘 n Õ o ≤ 𝐿E 𝛿𝑡 𝑤 𝑘 ∥ 𝜽˜ (𝑡) − 𝜽˜ (𝑡 − 𝑎 𝑘(𝑡 ) )∥ 𝑘∈K ( Õ (𝑡 ) ) Õ 𝑎𝑘 ≤ 𝐿E 𝛿𝑡 𝑤𝑘 ∥ 𝜽˜ (𝑡 − 𝑖 + 1) − 𝜽˜ (𝑡 − 𝑖)∥ . (50) 𝑘∈K

𝑖=1

We first handle the partial terms 𝛿𝑡 ∥ 𝜽˜ (𝑡 − 𝑖 + 1) − 𝜽˜ (𝑡 − 𝑖)∥,

𝑖 ∈ {1, ..., 𝑎 𝑘(𝑡 ) }

(51)

in (50) as follows. We define 𝜏𝑖 = 𝑡 −𝑖, ∀𝑖. Then, (51) becomes 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) ∥,

𝑖 ∈ {1, ..., 𝑎 𝑘(𝑡 ) }.

1) If a First-Level Differential Update Is Broadcast at Iter
ation 𝜏𝑖 + 1: 𝜽ˆ (𝜏𝑖 + 1) = 𝑄 1 𝜽 (𝜏𝑖 + 1) − 𝜽˜ (𝑟 𝜏𝑖 +1 ) = 𝑄 1 (𝒛𝑖 ), where   𝒛𝑖 = 𝜽˜ (𝜏𝑖 ) − 𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 − 𝜽˜ (𝑟 𝜏𝑖 +1 ). Then, 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) = 𝜽ˆ (𝜏𝑖 + 1) + 𝜽˜ (𝑟 𝜏𝑖 +1 ) − 𝜽˜ (𝜏𝑖 ) gives   𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) = 𝑄 1 (𝒛𝑖 ) − 𝒛 𝑖 − 𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘   ≤ ∥𝑄 1 (𝒛𝑖 ) − 𝒛𝑖 ∥ + ∥𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥, (52) by applying the triangle inequality. Then, from (52) and evaluating the effect of 𝑄 1 (·), h i E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 )∥ ) ( n o𝐾 i h ( 𝜏 ) ≤ E E 𝛿𝑡 ∥𝑄 1 (𝒛 𝑖 ) − 𝒛𝑖 ∥ 𝜽˜ (𝜏𝑖 ), 𝜽˜ (𝑟 𝜏𝑖 +1 ), 𝜽˜ (𝜏𝑖 − 𝑎 𝑖 ) 𝑘

h



 i + E 𝛿𝑡 ∥𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥ h   i √ ≤ 𝜎1 E [𝛿𝑡 ∥𝒛𝑖 ∥] + E 𝛿𝑡 ∥𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥   √ ≤ 𝜎1 E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 ) − 𝜽˜ (𝑟 𝜏𝑖 +1 ) ∥ h   i √
+ 1 + 𝜎1 𝜂E 𝛿𝑡 ∥𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥ ,

𝑘=1

(53)

(54)

13

( where (53) follows from applying Jensen’s inequality to (9) and (54) follows from using triangle inequality on ∥𝒛𝑖 ∥. Note that 𝑟 𝜏𝑖 +1 + 1, ..., 𝜏𝑖 ∈ T2 , by applying triangle inequality,   Õ 𝜏𝑖 −1   E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 ) − 𝜽˜ (𝑟 𝜏𝑖 +1 ) ∥ ≤ E 𝛿𝑡 ∥ 𝜽˜ ( 𝑗 + 1) − 𝜽˜ ( 𝑗) ∥ 𝑗=𝑟𝜏𝑖 +1 n h  i o Õ 𝜏𝑖 −1 ( 𝑗) = E 𝛿𝑡 𝑄 2 −𝜂𝑮 { 𝑗 − 𝑎 𝑘 } ∀𝑘 𝑗=𝑟𝜏𝑖 +1 ( Õ 𝜏𝑖 −1 = E 𝑗=𝑟𝜏𝑖 +1

n

h

E 𝛿𝑡 𝑄 2 −𝜂𝑮 p

≤ 𝜂 𝜎2 + 1



( 𝑗) { 𝑗 − 𝑎 𝑘 } ∀𝑘

h

Õ 𝜏𝑖 −1 𝑗=𝑟𝜏𝑖 +1

i

n

( 𝑗) 𝜽˜ ( 𝑗 − 𝑎 𝑘 )



 i ( 𝑗) E 𝛿𝑡 ∥𝑮 { 𝑗 − 𝑎 𝑘 } ∀𝑘 ∥ .

o𝐾 o

)

𝑘=1

(55)

Inserting (55) into (54), h i E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) ∥ h   i Õ 𝜏𝑖 −1 p ( 𝑗) ≤ 𝜂 𝜎1 (𝜎2 + 1) E 𝛿𝑡 ∥𝑮 { 𝑗 − 𝑎 𝑘 } ∀𝑘 ∥ 𝑗=𝑟𝜏𝑖 +1   i √
h + 𝜂 1 + 𝜎1 E 𝛿𝑡 ∥𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥ . (56) 2) If a Full Model Is Broadcast at Iteration 𝜏𝑖 + 1: Based on Assumption 3,

≤ 𝐿 𝜎𝑎 ˆ lim 𝜂E 𝛿𝑡 2

"

𝑡 −1 Õ

Õ

𝑗=𝑟𝑡 −𝑖+1 𝑙 ∈ K

p

𝑤 𝑙 𝛿 𝑗 −𝑎 ( 𝑗) + 𝜁 𝑙

!# ) ,

(61)

where (60) is based on 𝑎 𝑘(𝑡 ) ≤ 𝑎 lim in Assumption 3; (61) is by applying the triangle inequality and rÕ Õ p ∗ ∗ 𝑤 𝑘 ∥𝜽 − 𝜽 𝑘 ∥ ≤ 𝑤 𝑘 ∥𝜽 ∗ − 𝜽 ∗𝑘 ∥ 2 = 𝜁 . 𝑘∈K

𝑘∈K

Since (𝑡 − 𝑎 lim + 1) − 𝑟 𝑡 −𝑎lim +1 ≤ 𝑎 lim , Õ𝑡 −1 1 ≤ 𝑡 − 𝑟 𝑡 −𝑎lim +1 ≤ 2𝑎 lim and 𝑟 𝑡 −𝑎lim +1 ≥ 𝑡 − 2𝑎 lim . 𝑗=𝑟𝑡 −𝑖+1

(61) can then be rearranged as " #   (𝑡 ) E 𝛿𝑡 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘 } ∀𝑘 h Õmin(𝑡 −1,3𝑎lim ) ii h p 𝛿𝑡 −𝑖 , ≤ 𝐿 2 𝜎𝑎 ˆ lim 𝜂 2𝑎 lim 𝜁E [𝛿𝑡 ] + E 𝛿𝑡 𝑖=1

The proof is complete by relaxing the bound above with: • 𝛿 𝑡 < 𝛿 𝑡2 + 1, since 𝑥 < 𝑥 2 + 1, ∀𝑥 • the arithmetic-geometric mean inequality, leading to  Õmin(𝑡 −1,3𝑎lim )  Õmin(𝑡 −1,3𝑎lim ) 𝛿𝑡 𝛿𝑡 −𝑖 ≤ 𝛿𝑡2 + 𝛿𝑡2−𝑖 /2 𝑖=1 𝑖=1 Õmin(𝑡 −1,3𝑎lim ) 2 𝛿𝑡2−𝑖 /2. ≤ 3𝑎 lim 𝛿𝑡 /2 + 𝑖=1

∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 )∥  n o = 𝑄 0 𝜽˜ (𝜏𝑖 ) − 𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 − 𝜽˜ (𝜏𝑖 )

R EFERENCES

[1] C.-H. Hu, Z. Chen, and E. G. Larsson, “Mixed-timescale differential coding for downlink model broadcast in wireless federated learning,” in   58th Asilomar Conference on Signals, Systems, and Computers, 2024, ( 𝜏𝑖 ) = 𝜂∥𝑮 {𝜏𝑖 − 𝑎 𝑘 } ∀𝑘 ∥. pp. 1837–1842. h   i [2] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,   “Communication-efficient learning of deep networks from decentralized ⇒ E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 )∥ = 𝜂E 𝛿𝑡 ∥𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 ∥ . data,” in Artificial Intelligence and Statistics, 2017, pp. 1273–1282. (57) [3] T. Gafni, N. Shlezinger, K. Cohen, Y. C. Eldar, and H. V. Poor, “Federated learning: A signal processing perspective,” IEEE Signal 3) If a Second-Level Differential Update Is Broadcast at Processing Magazine, vol. 39, no. 3, pp. 14–41, 2022. Iteration 𝜏𝑖 + 1: [4] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: h h  i i Challenges, methods, and future directions,” IEEE Signal Processing   Magazine, vol. 37, no. 3, pp. 50–60, 2020. E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) ∥ = E 𝛿𝑡 𝑄 2 −𝜂𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 [5] B. Luo, W. Xiao, S. Wang, J. Huang, and L. Tassiulas, “Tackling system h   i p and statistical heterogeneity for federated learning with adaptive client . (58) ≤ 𝜂 𝜎2 + 1E 𝛿𝑡 𝑮 {𝜏𝑖 − 𝑎 𝑘( 𝜏𝑖 ) } ∀𝑘 sampling,” in IEEE Conference on Computer Communications, 2022, pp. 1739–1748. Based on (56), (57), and (58), we conclude that [6] M. M. Amiri, D. Gündüz, S. R. Kulkarni, and H. V. Poor, “Convergence   of update aware device scheduling for federated learning at the wireless E 𝛿𝑡 ∥ 𝜽˜ (𝜏𝑖 + 1) − 𝜽˜ (𝜏𝑖 ) ∥ edge,” IEEE Transactions on Wireless Communications, vol. 20, no. 6, h   i √ pp. 3643–3658, 2021. ( 𝜏𝑖 ) ≤ 𝜂(1 + 𝜎2 )E 𝛿𝑡 ∥𝑮 {𝜏𝑖 − 𝑎 𝑘 } ∀𝑘 ∥ [7] S. Dogan-Tusha, F. E. Bouanani, and M. Qaraqe, “Optimal stopping h   i theory-based online node selection in IoT networks for multi-parameter Õ 𝜏𝑖 −1 p ( 𝑗) federated learning,” IEEE Transactions on Machine Learning in ComE 𝛿𝑡 ∥𝑮 { 𝑗 − 𝑎 𝑘 } ∀𝑘 ∥ + 𝜂 𝜎1 (𝜎2 + 1) 𝑗=𝑟𝜏𝑖 +1 munications and Networking, vol. 3, pp. 659–676, 2025. h i Õ 𝜏𝑖 Õ [8] D. Yang, W. Zhang, Q. Ye, C. Zhang, N. Zhang, C. Huang, H. Zhang, ( 𝑗) ∗ ˜ ≤ 𝜂 𝜎𝐿 ˆ 𝑤 𝑘 E 𝛿𝑡 ∥ 𝜽 ( 𝑗 − 𝑎 𝑘 ) − 𝜽 𝑘 ∥ , (59) and X. Shen, “DetFed: Dynamic resource scheduling for deterministic 𝑗=𝑟𝜏𝑖 +1 𝑘∈K federated learning over time-sensitive networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 5, pp. 5162–5178, 2024. 𝜏𝑖 +1 ∈ {T0 , T1 , T2 }, by applying (15), Assumption 1, and (48). [9] M. Salehi and E. Hossain, “Federated learning in unreliable and By inserting (59) in (50) and recovering 𝜏𝑖 by 𝑡 − 𝑖, resource-constrained cellular wireless networks,” IEEE Transactions on  h  i Communications, vol. 69, no. 8, pp. 5136–5151, 2021. ≤ 𝐿 2 𝜎𝜂 ˆ E 𝛿𝑡 𝑮 (𝑡1) − 𝑮 {𝑡 − 𝑎 𝑘(𝑡 ) } ∀𝑘 [10] D. Liu, G. Zhu, J. Zhang, and K. Huang, “Data-importance aware user scheduling for communication-efficient edge machine learning,” IEEE (𝑡 ) ( " 𝑡 −𝑖 #) 𝑎𝑘 Transactions on Cognitive Communications and Networking, vol. 7, Õ Õ Õ Õ ( 𝑗) no. 1, pp. 265–278, 2021. ·E 𝑤𝑘 𝛿𝑡 𝑤 𝑙 ∥ 𝜽˜ ( 𝑗 − 𝑎 𝑙 ) − 𝜽 𝑙∗ ∥ [11] M. E. Ozfatura, J. Zhao, and D. Gündüz, “Fast federated edge learning 𝑗=𝑟𝑡 −𝑖+1 𝑙 ∈ K 𝑖=1 𝑘∈K with overlapped communication and computation and channel-aware fair ( " 𝑡 −1 #) client scheduling,” in IEEE International Workshop on Signal Processing Õ Õ ( 𝑗) Advances in Wireless Communications (SPAWC), 2021, pp. 311–315. ≤ 𝐿 2 𝜎𝑎 ˆ lim 𝜂E 𝛿𝑡 𝑤 𝑙 ∥ 𝜽˜ ( 𝑗 − 𝑎 ) − 𝜽 ∗ ∥ (60) 𝑗=𝑟𝑡 −𝑖+1 𝑙 ∈ K

𝑙

𝑙

14

[12] H. Chen, S. Huang, D. Zhang, M. Xiao, M. Skoglund, and H. V. Poor, “Federated learning over wireless IoT networks with optimized communication and resources,” IEEE Internet of Things Journal, vol. 9, no. 17, pp. 16 592–16 605, 2022. [13] W. Shi, S. Zhou, Z. Niu, M. Jiang, and L. Geng, “Joint device scheduling and resource allocation for latency constrained wireless federated learning,” IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 453–467, 2021. [14] M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 269–283, 2021. [15] Y. Oh, N. Lee, Y.-S. Jeon, and H. V. Poor, “Communication-efficient federated learning via quantized compressed sensing,” IEEE Transactions on Wireless Communications, vol. 22, no. 2, pp. 1087–1100, 2023. [16] D. Alistarh, T. Hoefler, M. Johansson, N. Konstantinov, S. Khirirat, and C. Renggli, “The convergence of sparsified gradient methods,” Advances in Neural Information Processing Systems, vol. 31, 2018. [17] D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, “QSGD: Communication-efficient SGD via gradient quantization and encoding,” Advances in neural information processing systems, vol. 30, 2017. [18] D. Jhunjhunwala, A. Gadhikar, G. Joshi, and Y. C. Eldar, “Adaptive quantization of model updates for communication-efficient federated learning,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 3110–3114. [19] M. M. Amiri, D. Gündüz, S. R. Kulkarni, and H. V. Poor, “Convergence of federated learning over a noisy downlink,” IEEE Transactions on Wireless Communications, vol. 21, no. 3, pp. 1422–1437, 2022. [20] Z. Chen, E. G. Larsson, C. Fischione, M. Johansson, and Y. Malitsky, “Over-the-air computation for distributed systems: Something old and something new,” IEEE Network, vol. 37, no. 5, pp. 240–246, 2023. [21] L. Cui, X. Su, Y. Zhou, and Y. Pan, “Slashing communication traffic in federated learning by transmitting clustered model updates,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 8, pp. 2572– 2589, 2021. [22] Y. Hu, T. Liu, C. Yang, Y. Huang, and S. Suo, “A flexible model compression and resource allocation scheme for federated learning,” IEEE Transactions on Machine Learning in Communications and Networking, vol. 1, pp. 168–184, 2023. [23] Y. Mu, N. Garg, and T. Ratnarajah, “Communication-efficient federated learning for massive MIMO systems,” in IEEE Wireless Communications and Networking Conference (WCNC), 2022, pp. 578–583. [24] D. Zhang, M. Xiao, and M. Skoglund, “Over-the-air computation empowered federated learning: A joint uplink-downlink design,” in IEEE 98th Vehicular Technology Conference, 2023. [25] K. Sayood, Introduction to Data Compression. Morgan Kaufmann Publishers, 2000. [26] K. Yue, R. Jin, C.-W. Wong, and H. Dai, “Communication-efficient federated learning via predictive coding,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 3, pp. 369–380, 2022. [27] A. Edin, Z. Chen, M. Kieffer, and M. Johansson, “Temporal predictive coding for gradient compression in distributed learning,” in Allerton Conference on Communication, Control, and Computing, 2024. [28] T. B. Adikari and S. C. Draper, “Compressing gradients by exploiting temporal correlation in Momentum-SGD,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 3, pp. 970–986, 2021. [29] R. Song, L. Zhou, L. Lyu, A. Festag, and A. Knoll, “ResFed: Communication-efficient federated learning with deep compressed residuals,” IEEE Internet of Things Journal, vol. 11, no. 6, pp. 9458–9472, 2024. [30] S. Zheng, C. Shen, and X. Chen, “Design and analysis of uplink and downlink communications for federated learning,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 7, pp. 2150–2167, 2021. [31] D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, “QSGD: Communication-efficient SGD via gradient quantization and encoding,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 1707–1718. [32] H. Xing, O. Simeone, and S. Bi, “Federated learning over wireless device-to-device networks: Algorithms and convergence analysis,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 12, pp. 3723– 3741, 2021. [33] Z. Yan and D. Li, “Performance analysis for resource constrained decentralized federated learning over wireless networks,” IEEE Transactions on Communications, vol. 72, no. 7, pp. 4084–4100, 2024. [34] A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani, “An exact quantized decentralized gradient descent algorithm,” IEEE Transactions on Signal Processing, vol. 67, no. 19, pp. 4934–4947, 2019.

[35] L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018. [Online]. Available: https://doi.org/10.1137/16M1080173 [36] E. G. Larsson and N. Michelusi, “Unified analysis of decentralized gradient descent: A contraction mapping framework,” IEEE Open Journal of Signal Processing, vol. 6, pp. 507–529, 2025. [37] H. H. Yang, A. Arafa, T. Q. Quek, and H. V. Poor, “Age-based scheduling policy for federated learning in mobile edge networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 8743–8747. [38] X. Hu, N. Pappas, and H. H. Yang, “Version age-based client scheduling policy for federated learning,” in IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops, 2024, pp. 695– 699. [39] J. Zheng, X. Liu, Z. Ling, and F. Hu, “AoU-based local update and user scheduling for semi-asynchronous online federated learning in wireless networks,” IEEE Internet of Things Journal, vol. 11, no. 18, pp. 29 673– 29 688, 2024. [40] J. Liu, Z. Chang, and Y.-C. Liang, “Age-based device selection and transmit power optimization in over-the-air federated learning,” IEEE Transactions on Communications, vol. 74, pp. 4320–4335, 2026. [41] K. Wang, Y. Ma, M. B. Mashhadi, C. H. Foh, R. Tafazolli, and Z. Ding, “Convergence acceleration in wireless federated learning: A stackelberg game approach,” IEEE Transactions on Vehicular Technology, vol. 74, no. 1, pp. 714–729, 2025. [42] H. Zeng, X. Li, G. Xu, J. Hao, X. Wang, and V. C. M. Leung, “Cluster-based device scheduling design for semi-asynchronous federated learning in mobile edge computing networks,” in ICC 2025 - IEEE International Conference on Communications, 2025, pp. 1590–1595. [43] Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010. [Online]. Available: http://yann.lecun.com/exdb/mnist/ [44] A. Krizhevsky, “Learning multiple layers of features from tiny images,” Tech. Rep., 2009. [45] Y. Oh, Y.-S. Jeon, M. Chen, and W. Saad, “FedVQCS: Federated learning via vector quantized compressed sensing,” IEEE Transactions on Wireless Communications, vol. 23, no. 3, pp. 1755–1770, 2024. [46] N. Shlezinger, M. Chen, Y. C. Eldar, H. V. Poor, and S. Cui, “UVeQFed: Universal vector quantization for federated learning,” IEEE Transactions on Signal Processing, vol. 69, pp. 500–514, 2021.