FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

arXiv:2608.15310v1 [cs.CR] 15 Aug 2026

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting Zhenyan Liu

Hua Zhang✉∗

Haoran Gao

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China lzhy@bupt.edu.cn

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China zhanghua_288@bupt.edu.cn

JIUTIAN Research, China Mobile Beijing, China gaohaoranit@chinamobile.com

Qi Li

Hongliang Zhu

Huiyu Zhou

School of Cyberspace Security, Beijing University of Posts and Telecommunications Beijing, China liqi2001@bupt.edu.cn

School of Cyberspace Security, Beijing University of Posts and Telecommunications Beijing, China zhuhongliang@bupt.edu.cn

University of Leicester Leicester, United Kingdom hz143@leicester.ac.uk

Zongliang Shen

Yanxin Xu

Jiahui Wang

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China shenzongliang@bupt.edu.cn

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China yanxinxu@bupt.edu.cn

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications Beijing, China jiahuiwang@bupt.edu.cn

Abstract Multimodal data collected by heterogeneous devices are used for collaborative training, where federated learning (FL) serves as a key paradigm for effective distributed modeling with data privacy preservation. However, local training suffers from the forgetting of previously learned global knowledge under cross-client data heterogeneity, which leads to significant declines in both performance and convergence speed. Most previous studies rely on global alignment strategies to retain global knowledge, which hinder local optimization and lead to inadequate supervision of missing classes. Some studies introduce proxy datasets to supplement supervision for missing classes. However, it remains a challenge to balance class-wise global consistency and local optimization objectives without proxy datasets. In this work, we propose FedADB, a Class Anchor–Driven Dual-Branch FL framework. Specifically, the server generates class anchors optimized in a differentiable input space, which are shared across clients. These class anchors serve as global references that provide supervision for missing classes ∗ Also with National Engineering Research Center of Disaster Backup and Recovery,

Beijing University of Posts and Telecommunications.

This work is licensed under a Creative Commons Attribution 4.0 International License. MM ’26, Rio de Janeiro, Brazil © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2213-4/2026/11 https://doi.org/10.1145/3767308.3835615

during local training. A dual-branch collaborative training mechanism is designed for clients. In this mechanism, the anchor-based global branch focuses on learning with global consistency, achieving global knowledge alignment by class-anchor balanced sampling. The local calibration branch focuses on learning discriminative local features, mitigating the degradation of local representations caused by excessive global alignment. Extensive experiments across multiple medical and natural datasets demonstrate that FedADB achieves significant improvements in both accuracy and convergence speed.

CCS Concepts • Computing methodologies → Distributed algorithms.

Keywords Federated Learning, Forgetting Mitigation ACM Reference Format: Zhenyan Liu, Hua Zhang, Haoran Gao, Qi Li, Hongliang Zhu, Huiyu Zhou, Zongliang Shen, Yanxin Xu, and Jiahui Wang. 2026. FedADB: Class AnchorDriven Dual-Branch Federated Learning for Mitigating Forgetting. In Proceedings of the 34th ACM International Conference on Multimedia (MM ’26), November 10–14, 2026, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3767308.3835615

1

Introduction

The widespread deployment of digital terminals and sensing devices has driven an explosive growth of multimedia data generated by distributed sources such as medical institutions, mobile terminals, and surveillance systems[22, 45, 55]. These data have substantially

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

advanced multimedia intelligence, enabling applications such as autonomous driving, fraud detection, and clinical decision support. However, due to legal, ethical, and institutional constraints, these data cannot be used for centralized training. Federated learning (FL) enables clients to collaboratively train models without sharing raw data[5, 18, 37, 44]. This framework significantly improves the utility of isolated data and facilitates the collaborative optimization of neural network models [4, 42, 53, 54]. However, in real-world applications, data across devices are typically Non-Independent and Identically Distributed (non-IID)[17, 46, 58, 59]. Recent studies reveal that under such statistical heterogeneous data distributions, local training on client data leads to a divergence between local and global optimization objectives [35, 47]. This divergence leads to overfitting to local data distributions while forgetting knowledge acquired from previous training rounds. This phenomenon is known as Catastrophic Forgetting (CF) [40]. Consequently, the global model suffers from significant performance degradation and a notable decline in convergence speed [16, 19, 28, 49]. To alleviate issues of CF, existing approaches can roughly be categorized into two main types. The first category typically guides local optimization through gradient or objective function adjustments derived from the global model [1, 24, 39, 49]. This technique ensures that local updates maintain consistency with the global optimization direction. The second category usually employs knowledge distillation (KD) to preserve global knowledge [3, 19, 29, 38]. This paradigm leverages the global model’s outputs to guide local training, thereby preserving knowledge acquired from previous tasks. Inherently, both categories employ alignment strategies that reference the global model to calibrate local updates. Although these methods can mitigate CF through knowledge transfer and variance control, they are prone to local-global optimization conflicts and may suppress local learning due to excessive alignment [28, 50]. More importantly, these methods struggle to provide effective supervision for classes missing in clients’ local data. Specifically, the feature representations and decision boundaries for missing classes remain unconstrained during local optimization. Consequently, the global model exhibits severe performance degradation on these classes. To address this issue, recent studies provide proxy datasets to clients for knowledge compensation [7, 16]. These studies introduce explicit supervision for clients’ missing classes, which helps maintain accuracy for those classes. However, proxy datasets are often unavailable in privacy-sensitive domains [57, 64, 65], and may be limited by cross-domain distribution shifts. To address these challenges, this paper proposes Class AnchorDriven Dual-Branch Federated Learning (FedADB), a framework that includes two main components: a Class Anchor Generation (CAG) method and a Dual-Branch Cooperative Training (DBCT) mechanism. Specifically, the CAG method generates shared class anchors for each class via differentiable optimization in the input space. These anchors are incorporated into local training as global references to supply supervision for missing classes. During local update, the DBCT mechanism balances global and local optimization objectives via two complementary branches: The Anchor-based Global Branch (AGB) maintains global consistency by leveraging class anchors and balanced sampling, thereby preserving and propagating global knowledge. The Local Calibration Branch (LCB) captures each client’s distinctive data distribution by optimizing

Zhanyan Liu et al.

directly on the client-specific local dataset. This design avoids the degradation of local representation capability due to excessive alignment with the global objective. Under the coordinated guidance of both branches, a shared feature extractor learns representations that are simultaneously globally generalizable and locally adaptive. Experiments were conducted on multiple natural image classification datasets and medical image classification datasets, including extensive evaluation includes performance comparisons, ablation studies, analysis by case studies. The results validate that FedADB effectively mitigates catastrophic forgetting under non-IID data settings. Our main contributions are as follows: • We propose a class anchor generation method CAG. Unlike prior approaches that rely on external data or auxiliary priors, CAG constructs anchors solely from the global model, thereby avoiding additional privacy-leakage risks. Moreover, each class requires only a single anchor sample to achieve lightweight yet effective global knowledge preservation. • We propose a dual-branch cooperative training mechanism DBCT. Unlike existing FL training schemes, DBCT employs differentiated training strategies with branch-specific optimization objectives. One branch preserves global knowledge through class anchor constraints and class-balanced sampling, while the other focuses on local data adaptation. This design enables the model to retain global knowledge while effectively learning local feature representations. • Extensive experiments demonstrate that FedADB effectively mitigates catastrophic forgetting while improving the smoothness of the global model. FedADB achieves superior or competitive performance against state-of-the-art (SOTA) methods in both accuracy and convergence speed.

2 Related Work 2.1 Heterogeneity in Federated Learning FL enables clients to collaboratively train models without sharing their raw data. However, the mainstream algorithm FedAvg [41] suffers substantial performance degradation under severe data heterogeneity, which has motivated numerous refinement strategies [33, 63]. The first category mitigates client drift by constraining or correcting local updates. Representative methods include FedProx [35], SCAFFOLD [24], MOON [34], and FedDyn [1]. Another category focuses on server-side aggregation, such as FedAvgM [17], which incorporates momentum into global model updates. Additionally, personalized FL instead learns client-specific models adapted to individual data distributions [51, 52]. Representative strategies include fine-tuning the global model [13], introducing personalized projection heads [11], and designing adaptive aggregation schemes [20, 61]. Although these methods alleviate data heterogeneity, their generalizability and robustness remain limited. In particular, some methods do not consistently outperform FedAvg across heterogeneous settings, as shown in our experiments. This underscores the necessity for more robust FL frameworks capable of fundamentally resolving data heterogeneity

2.2

Forgetting in Local Learning

CF primarily results from distribution shifts during local training in FL, which cause clients to forget out-of-distribution knowledge and

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

degrade global performance[16, 19, 30]. Unlike the sequential data problem in continual learning, this paper focuses on mitigating CF across spatially distributed data. FedCurv [49] was among the earliest studies to identify CF in FL. Subsequent research has predominantly followed two directions. The first employs optimizationbased guidance to mitigate knowledge forgetting during local training [28, 39]. The second category employs KD. For instance, FedNTD [29] preserves knowledge of non-target classes by masking target-class logits, whereas FedLMD [38] transfers knowledge for minority-class by masking majority-label predictions. However, local updates constrained by global knowledge tend to suppress local learning and offer inadequate supervision for the missing classes. To address this, several methods introduce proxy data to preserve missing-class knowledge. Flashback [3] leverages public datasets to fuse knowledge across rounds during aggregation. FedBR [16] and FedKA [7] reduce local learning bias with globally shared proxy datasets. Since proxy datasets are often unavailable, we generate class anchors by optimizing noise via the global model. We further develop a dual-branch collaborative training framework that jointly preserves global knowledge and facilitates local learning.

3

Preliminaries

Consider a FL system with a central server and 𝐾 clients. Client |D | 𝑘 ∈ [𝐾] holds a local dataset D𝑘 = {(x𝑖 , 𝑦𝑖 )}𝑖=1𝑘 with a label space C𝑘 ⊆ C, where C denotes the global label space. FL aims to learn a global model w𝐺 that generalizes across the global data distribution. The global optimization objective is formulated as

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

coupled through an adaptive loss weight, yielding a more effective trade-off between global alignment and local adaptation.

4.2

Class Anchor Generation

To mitigate the performance degradation caused by insufficient supervision of missing classes, we supplement the supervisory signal by introducing class anchors. After obtaining the global model, the server initializes e x𝑖(0) ∼ N (0, 𝜎 2 I) for target class 𝑐𝑖 ∈ C, then optimizes them via the feature extractor 𝜙 (·; 𝜽 𝑡𝜙 ) and AGB classifier 𝜓𝐴 (·; 𝜽 𝑡𝐴 ). The optimization objective is defined as h  i Lach (e x𝑖 ; 𝑐𝑖 ) = − log 𝜓𝐴 𝜙 (e x𝑖 ; 𝜽 𝑡𝜙 ); 𝜽 𝑡𝐴 + 𝜆∥e x𝑖 ∥ 22, (3) 𝑐𝑖

where the first term is the negative log-likelihood (NLL) loss, which explicitly drives e x𝑖 toward class 𝑐𝑖 under the current global model, while the second term is an 𝐿2 regularizer with coefficient 𝜆 that penalizes extreme values during optimization. The class-anchor update at iteration 𝑗 is given by   e x𝑖( 𝑗+1) = Π X e x𝑖( 𝑗 ) − 𝜂∇ex ( 𝑗 ) Lach (e x𝑖( 𝑗 ) ; 𝑐𝑖 ) , (4) 𝑖

where 𝜂 is the learning rate. Starting from the Gaussian initialization, we iteratively update e x𝑖 to progressively increase the target class posterior. Π X projects each update onto the valid input domain X. This improves numerical stability and keeps anchor generation consistent with the data space. After 𝐼 optimization iterations, we obtain a total of 𝑁 class anchors. For anchor 𝑖, we define its targetclass confidence as 𝑦e𝑖 and quantify the overall anchor quality by 𝑁

min L (w𝐺 ) =

𝐾 ∑︁

w𝐺

𝑘=1

|D𝑘 | Í L𝑘 (w𝐺 ), 𝑘 ∈ [𝐾 ] |D𝑘 |

𝜏𝑡 =

(1)

where L𝑘 is the local objective of client 𝑘. At communication round 𝑡 to the participating client set K 𝑡 ⊆ [𝐾]. 𝑡, the server broadcasts w𝐺 For client 𝑘, the local objective is defined as 
 𝑡 𝑡 L𝑘 (w𝐺 ) = E (x,𝑦)∼D𝑘 ℓ𝑘 𝑓 (x; w𝐺 ), 𝑦 , (2) where 𝑓 (x; w) denotes the model prediction, and ℓ is the samplewise loss. After local training, the server aggregates the updated local models with weights 𝜎𝑘 to obtain the updated global model 𝑡 +1 = Í 𝑡 +1 w𝐺 𝑘 ∈ K 𝑡 𝜎𝑘 w𝑘 . This procedure is repeated for 𝑇 rounds.

4 Methodology 4.1 Overview of FedADB FL training often suffers from inadequate supervision for missing classes and an inherent conflict between global consistency and local adaptation, both of which jointly aggravate catastrophic forgetting. FedADB addresses both issues by supplying class-level global supervision without forcing local optimization to over-align with the global objective. The overall framework of FedADB is shown in Fig. 1. Specifically, 1 the server first generates class anchors from the aggregated model and broadcasts them as shared supervisory signals. 2 Each client performs dual-branch collaborative training, comprising an Anchor-guided Global Branch (AGB), which preserves globally consistent decision boundaries through anchor-guided optimization, whereas a Local Calibration Branch (LCB) leverages local data to maintain client-specific discriminative capability. The two branches share a feature extractor and are

1 ∑︁ [𝜓𝐴 (𝜙 (e x𝑖 ; 𝜽 𝑡𝜙 ); 𝜽 𝑡𝐴 )]𝑐𝑖 , 𝑁 𝑖=1

(5)

e𝑡 = {(e The server broadcasts the anchor set A x, 𝑦e, 𝜏𝑡 )} to all participating clients, providing a lightweight yet consistent global supervisory signal. Because the global model changes after every aggregation, the server regenerates the anchors at each round to keep the supervisory signal synchronized with the current model.

4.3

Dual-Branch Collaborative Training

To mitigate the conflict between global consistency and local adaptability, we propose dual-branch collaborative training (DBCT). The model comprises a shared feature extractor 𝜙 (·; 𝜽 𝜙 ) and two complementary classification branches: the branch AGB for global alignment and the branch LCB for local representation learning. Gradients from both branches jointly optimize the shared extractor 𝜽 𝜙 , while each branch follows a distinct optimization strategy. Anchor-guided Global Branch (AGB). For the input x, the AGB employs a classifier 𝜓𝐴 (·; 𝜽 𝐴 ) to produce predictions 𝑓𝐴 (x) = 𝜓𝐴 (𝜙 (x; 𝜽 𝜙 ); 𝜽 𝐴 ). For client 𝑘, we construct a class-balanced mixed e𝑡 , ensuring that every batch B𝑘𝐴 from local samples and anchors in A class can contribute supervisory signals even when it is absent from D𝑘 . Let H (·, ·) denotes the cross-entropy loss. The AGB loss is L𝐴𝐺𝐵 = E (x,𝑦)∼B𝐴 H (𝑦, 𝑓𝐴 (x)),

(6)

𝑘

By minimizing L𝐴𝐺𝐵 , class anchors provide stable gradient constraints across the global label space, thereby preserving global knowledge and mitigating forgetting during local updates.

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

Anchor Data

Feature AGB Exactor

Step 1：Model Aggregation

LCB

Logit Output

Backpropagate

Label info. Communicate

Finish Aggregate Pull Feature Space

Server

…

Sample num

Client Data

Zhanyan Liu et al.

Client k

Step 2：Class Anchor generate

Class

softmax

Sample num

Class Balanced Sampling

Class

Random Sampling

Update By

Class Anchors Optimization

AGB: Global Consistency

LCB: Local Feature Learning

Figure 1: Overview of FedADB. 1 After aggregation, the server generates class anchors and broadcasts them to the clients. 2 Each client performs dual-branch collaborative training: the AGB constructs class-balanced mixed batches from class anchors and local data, while the LCB is optimized using only local data. The shared feature extractor is jointly optimized by both branches. The updated parameters are then uploaded to the server. Local Calibration Branch (LCB). For the input x, the LCB employs a classifier 𝜓𝐿 (·; 𝜽 𝐿 ) and outputs 𝑓𝐿 (x) = 𝜓𝐿 (𝜙 (x; 𝜽 𝜙 ); 𝜽 𝐿 ). Unlike the AGB, this branch is optimized solely on the local data D𝑘 with a batch size aligned to B𝑘𝐴 , so that it can captures clientdependent class priors and feature statistics. As a result, the LCB preserves local discriminative structure and compensates for representation bias introduced by strict global alignment. Its loss is L𝐿𝐶𝐵 = E (x,𝑦)∼D𝑘 H (𝑦, 𝑓𝐿 (x)),

(7)

Let 𝑟 = 𝑡/𝑇 denote the FL progress. We define the LCB weight for client 𝑘 as 𝛽𝑘 (𝑟 ) = 𝑟 𝛾𝑘 ·𝜏𝑡 , where 𝛾𝑘 = 1 − |C𝑘 |/|C| represents the class-missing rate for client 𝑘, and 𝜏𝑡 measures the quality of the anchors. When 𝛾𝑘 or 𝜏𝑡 is high, 𝛽𝑘𝑡 (𝑟 ) is attenuated to emphasize global consistency. Otherwise, 𝛽𝑘𝑡 (𝑟 ) increases more rapidly to strengthen local learning. The total loss for client 𝑘 is given by (𝑘,𝑡 ) L𝑡𝑜𝑡𝑎𝑙 = L𝐴𝐺𝐵 + 𝛽𝑘𝑡 (𝑟 ) · L𝐿𝐶𝐵 ,

(8)

𝑘,𝑡 Client 𝑘 minimizes Ltotal with respect to {𝜽 𝜙 , 𝜽 𝐴 , 𝜽 𝐿 } and returns

the updated parameters to the server for aggregation. Through the decoupling of knowledge preservation and local learning, DBCT enables effective adaptation to local data while maintaining global knowledge, thereby alleviating catastrophic forgetting.

5 Experiment 5.1 Experimental Setup Datasets and Models: We evaluated all methods on four natural image datasets (SVHN [43], CINIC-10 [12], CIFAR-10 [27], and CIFAR-100 [27]) and three medical image datasets (OrganAMNIST [60], PathMNIST [60], TissueMNIST [60]). We utilized LeNet [2] for the single-channel datasets (OrganAMNIST and TissueMNIST),

whereas ResNet-18 [29, 38] was adopted for the remaining datasets. Following prior work [28, 29, 38], clients trained 5 local epochs per round using SGD with a momentum coefficient of 0.9 and weight decay of 1 × 10−5 . The batch size was set to 128. After each round, the learning rate was decayed exponentially with a factor of 0.99. To ensure sufficient convergence, we set the total training rounds to 300 for OrganAMNIST and 1000 for all remaining datasets. By default, one anchor per class was generated in each round. Baseline Methods: We compared FedADB with a broad set of representative baselines. Specifically, the benchmark suite includes the canonical FedAvg, representative methods for mitigating data heterogeneity, including FedDyn [1], FedASAM [6], FedGELA [14], FedExp [23], FedDecorr [48], and FedACG [25], as well as SOTA methods for mitigating forgetting in FL, including FedNTD [29], FedLMD [38], and FedSOL [28]. All methods were implemented in PyTorch (v2.2.1, CUDA 12.1) and evaluated on an Intel Xeon Platinum 8375C CPU (2.90 GHz) with an NVIDIA RTX 3090 GPU. Scenario. We evaluated all methods under two standard FL scenarios: cross-silo and cross-device[8, 32, 36]. We fixed the total number of clients to 𝐾 = 20 with a participation rate of 𝜌 = 0.2 for the cross-silo setting, and 𝐾 = 100 with 𝜌 = 0.05 for the cross-device setting. We adopted the Latent Dirichlet Allocation (LDA) scheme to simulate non-IID data distributions[29, 38], where 𝛼 controls the degree of cross-client data heterogeneity.

5.2

Performance under Data Heterogeneity

We evaluated FedADB against all baselines under both cross-silo and cross-device settings, with the results summarized in Tab.1. We report the mean ± standard deviation of the top-5 round accuracies across three independent trials with different random seeds. While baselines manifest sensitivity to specific configurations and

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

Table 1: Comparison of test accuracy@1 (%) among the baselines and FedADB on different datasets under cross-device and cross-silo scenarios. We set 𝛼 ∈ {0.05, 0.1, 0.2, 0.5} for CIFAR-10 and 𝛼 = 0.1 for the other datasets. Bold indicates the best result. Method

SVHN

CIFAR-10

CINIC-10 𝛼 = 0.05

𝛼 = 0.1

𝛼 = 0.2

𝛼 = 0.5

CIFAR-100 OrganAMNIST

PathMNIST

TissueMNIST

Average

69.01(±3.19) 74.73(±1.59) 71.85(±1.30) 77.31(±1.48) 73.57(±1.99) 73.55(±0.81) 74.75(±1.40) 67.18(±1.15) 66.97(±2.10) 72.23(±2.74)

21.84(±5.07) 32.11(±0.68) 28.23(±3.51) 15.65(±2.93) 19.26(±3.43) 30.29(±0.80) 14.54(±0.68) 35.15(±0.61) 23.70(±2.80) 33.03(±0.81)

49.87(±1.97) 55.13(±1.72) 53.57(±1.69) 55.88(±1.20) 53.77(±1.65) 54.52(±0.75) 55.45(±0.90) 57.03(±0.97) 53.71(±1.50) 55.47(±0.88)

Cross-Device Scenario (𝐾 = 100, 𝜌 = 0.05) FedAvg[41] FedDyn[1] FedASAM[6] FedNTD[29] FedGELA[14] FedEXP[23] FedLMD[38] FedDecorr[48] FedACG[25] FedSOL[28]

84.56(±1.57) 43.55(±1.96) 32.32(±1.40) 83.49(±4.28) 44.69(±1.00) 28.38(±0.10) 73.86(±3.31) 46.38(±1.44) 30.80(±2.04) 89.23(±0.44) 41.78(±0.75) 39.64(±2.17) 89.25(±0.46) 48.01(±0.84) 41.86(±1.53) 69.10(±0.81) 41.05(±0.67) 40.64(±0.81) 89.80(±0.14) 42.99(±0.87) 42.47(±0.48) 88.99(±1.16) 41.51(±1.15) 37.77(±1.92) 83.64(±0.65) 43.98(±0.68) 30.61(±1.32) 87.14(±0.33) 41.60(±0.18) 42.54(±0.96)

48.76(±1.14) 57.66(±1.43) 66.76(±0.81) 42.62(±2.66) 66.91(±1.91) 69.38(±0.52) 47.19(±2.33) 62.63(±0.39) 69.64(±0.28) 54.03(±1.00) 59.53(±0.52) 70.86(±0.40) 56.84(±0.72) 61.27(±4.05) 66.42(±0.47) 49.73(±0.16) 55.56(±1.06) 65.74(±0.05) 55.87(±0.21) 59.68(±0.92) 70.80(±0.35) 55.15(±0.41) 58.11(±0.52) 66.88(±0.37) 51.18(±1.55) 56.76(±1.66) 66.83(±0.41) 50.38(±0.23) 56.32(±1.17) 65.61(±0.15)

47.54(±0.31) 42.87(±0.89) 46.79(±0.85) 46.19(±1.23) 35.45(±1.28) 46.04(±0.52) 38.50(±1.88) 48.15(±0.90) 47.49(±0.49) 38.33(±0.18)

26.68(±2.78) 66.10(±3.61) 58.31(±1.47) 64.61(±1.03) 45.75(±1.72) 73.49(±1.79) 65.13(±2.05) 71.39(±1.49) 65.90(±3.37) 67.54(±2.02)

FedADB (Ours) 90.06(±0.40) 47.67(±0.69) 50.06(±0.61) 59.21(±0.47) 64.55(±0.56) 70.97(±0.29) 57.70(±0.31) 78.04(±1.66) 79.08(±0.53) 40.75(±1.11) 63.81(±0.66) Cross-Silo Scenario (𝐾 = 20, 𝜌 = 0.2) FedAvg[41] 91.90(±0.15) 35.02(±0.06) FedDyn[1] 89.75(±0.13) 37.22(±0.33) FedASAM[6] 93.00(±0.36) 38.79(±2.46) FedNTD[29] 91.32(±0.12) 36.69(±0.35) FedGELA[14] 92.30(±0.24) 33.50(±0.18) FedEXP[23] 92.06(±0.20) 36.35(±0.18) FedLMD[38] 91.78(±0.45) 37.19(±0.22) FedDecorr[48] 91.96(±0.10) 36.06(±0.61) FedACG[25] 91.97(±0.12) 35.61(±1.22) FedSOL[28] 90.98(±0.26) 39.35(±0.41)

31.67(±1.17) 31.95(±0.88) 35.29(±1.75) 34.14(±1.37) 32.17(±1.21) 33.28(±0.37) 34.25(±0.60) 32.49(±1.31) 31.30(±1.38) 40.13(±0.34)

55.96(±1.49) 58.59(±0.95) 59.74(±0.70) 60.02(±1.54) 59.44(±1.56) 58.38(±1.44) 61.89(±0.06) 58.42(±0.53) 56.01(±0.50) 58.08(±1.55)

66.17(±0.43) 72.96(±0.49) 67.53(±0.44) 72.67(±0.22) 69.81(±0.50) 75.45(±0.32) 71.32(±0.56) 76.51(±0.48) 64.96(±0.78) 70.27(±1.00) 66.49(±0.10) 71.79(±0.09) 71.67(±0.46) 75.89(±0.23) 65.44(±1.11) 72.35(±0.23) 65.78(±0.66) 72.88(±0.43) 66.49(±0.55) 72.75(±0.13)

63.78(±0.16) 60.40(±0.22) 65.15(±0.77) 63.35(±0.08) 60.43(±0.30) 63.69(±0.31) 53.95(±0.87) 62.85(±0.46) 63.94(±0.14) 59.67(±0.35)

67.44(±0.17) 67.36(±0.37) 68.90(±0.31) 67.90(±1.54) 59.22(±0.43) 70.36(±1.17) 68.73(±0.18) 70.99(±0.70) 67.50(±1.54) 69.41(±2.49)

75.26(±1.38) 76.11(±1.44) 78.27(±0.69) 73.98(±1.77) 70.06(±1.65) 75.19(±0.36) 74.93(±1.51) 74.83(±1.21) 75.93(±0.72) 73.08(±1.20)

39.81(±0.50) 38.67(±0.92) 39.80(±1.53) 36.00(±0.51) 33.95(±4.05) 41.24(±0.82) 40.05(±1.19) 39.56(±0.55) 37.06(±3.58) 35.86(±0.49)

60.00(±0.60) 60.03(±0.59) 62.42(±0.94) 61.12(±0.83) 57.63(±1.14) 60.88(±0.50) 61.03(±0.58) 60.50(±0.68) 59.80(±1.03) 60.58(±0.78)

FedADB (Ours) 92.36(±0.08) 44.62(±0.44) 54.02(±0.60) 67.65(±0.25) 71.84(±0.33) 75.07(±0.48) 65.57(±0.51) 78.21(±0.34) 80.48(±1.27) 41.45(±0.50) 67.13(±0.48)

occasionally fall below FedAvg, FedADB consistently outperforms SOTA methods in nearly all settings and achieves the best results in most cases. Across all datasets and non-IID settings, FedADB improved the average accuracy over the second-best method by 6.78% and 4.71% in cross-device and cross-silo scenarios, respectively. Compared to canonical FedAvg, the performance gains reached 13.94% and 7.13%, respectively. Notably, under the most severe Non-IID condition (𝛼 = 0.05) where catastrophic forgetting is most pronounced, FedADB surpassed the second-best method by 7.52% and 13.89%. These results demonstrate that FedADB remains effective under extreme data heterogeneity, stabilizing global model performance, and effectively mitigating catastrophic forgetting.

5.3

Convergence Speed Analysis

Following previous works[15, 39, 56, 62], we measured convergence speed by the number of communication rounds required to reach a target accuracy 𝑇Acc . To suppress spurious spikes, we smoothed each accuracy trajectory using an Exponentially Weighted Moving Average (EWMA)[21] with a factor of 0.2. We set the maximum accuracy as a reference and recorded the minimum rounds needed to reach 45%, 65%, and 85% of this threshold. The cross-device results are summarized in Tab. 2. FedADB exhibited markedly faster convergence than all baselines: on SVHN, it reached 80% accuracy in 50 rounds, whereas FedAvg required 478 rounds (9.56× longer). Under more challenging heterogeneous conditions (CIFAR-10 with 𝛼 = 0.05), FedADB reached 35% accuracy in 63 rounds, while other methods required 323 rounds on average (5.13× longer). Across all datasets, FedADB achieved the target accuracy with 74.4% and 86.5%

Figure 2: Performance of FedADB and FedAvg on CIFAR-10 (𝛼 = 0.1) across varying FL hyperparameter settings. fewer rounds than the second-best ( 3.9× longer) and FedAvg ( 7.4× longer), respectively. Notably, methods failing to reach the𝑇Acc were excluded from the average, suggesting that FedADB’s advantages are even more pronounced. As shown in Fig. 4, FedADB exhibits more stable optimization and faster convergence, benefiting from effective global knowledge retention and local adaptation.

5.4

Hyperparameter Sensitivity

We evaluated the sensitivity of FedADB to key FL hyperparameters, including the number of anchors per class, local epochs, and learning rate. Figure 2 shows that FedADB consistently outperforms FedAvg across all evaluated settings. Moreover, FedADB is insensitive to the number of anchors per class, exhibiting only minor variations in accuracy. Although increasing local epochs or changing learning rate could improve the performance, FedADB maintains a distinct advantage throughout the evaluated range. Overall, these results confirm the robustness and tunability of FedADB. We further evaluated the sensitivity of FedADB to the key CAG hyperparameters in Fig. 3. The default values in our experiments,

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

Zhanyan Liu et al.

Table 2: Communication rounds required to reach 𝑇Acc and the corresponding relative speedup over FedADB. “–” indicates that the method fails to reach 𝑇Acc and is therefore excluded from the average calculation. Unless otherwise specified, 𝛼 = 0.1. Dataset

𝜶

SVHN

42% 274 (13.7 × ) 96 (4.8 × ) 0.1 61% 330 (12.2 × ) 157 (5.8 × ) 80% 478 (9.6 × ) 272 (5.4 × )

426 (21.3 × ) 585 (21.7 × ) –

83 (4.2 × ) 118 (4.4 × ) 191 (3.8 × )

80 (4.0 × ) 106 (3.9 × ) 149 (3.0 × )

60 (3.0 × ) 65 (3.3 × ) 106 (3.9 × ) 95 (3.5 × ) – 147 (2.9 × )

159 (8.0 × ) 184 (6.8 × ) 245 (4.9 × )

321 (16.1 × ) 380 (14.1 × ) 571 (11.4 × )

CINIC-10

20% 74 (10.6 × ) 38 (5.4 × ) 0.1 29% 192 (6.9 × ) 112 (4.0 × ) 38% 563 (6.1 × ) 215 (2.3 × )

62 (8.9 × ) 192 (6.9 × ) 401 (4.3 × )

29 (4.1 × ) 99 (3.5 × ) –

93 (13.3 × ) 241 (8.6 × ) 401 (4.3 × )

37 (5.3 × ) 133 (4.8 × ) –

28 (4.0 × ) 63 (2.3 × ) –

38 (5.4 × ) 191 (6.8 × ) –

74 (10.6 × ) 37 (5.3 × ) 7 (1.0 × ) 207 (7.4 × ) 88 (3.1 × ) 28 (1.0 × ) 562 (6.0 × ) 254 (2.7 × ) 93 (1.0 × )

25% 271 (10.0 × ) 356 (13.2 × ) 443 (16.4 × ) – – – 0.05 35% 46% – – –

166 (6.2 × ) 445 (7.1 × ) –

217 (8.0 × ) 356 (5.7 × ) –

73 (2.7 × ) 159 (5.9 × ) 221 (3.5 × ) 361 (5.7 × ) – –

199 (7.4 × ) 445 (7.1 × ) –

361 (13.4 × ) 62 (2.3 × ) 27 (1.0 × ) – 114 (1.8 × ) 63 (1.0 × ) – – 148 (1.0 × )

31% 209 (13.1 × ) 156 (9.8 × ) 0.1 44% 383 (6.6 × ) 384 (6.6 × ) 58% – –

225 (14.1 × ) 388 (6.7 × ) –

137 (8.6 × ) 253 (4.4 × ) 619 (3.9 × )

103 (6.4 × ) 196 (3.4 × ) 310 (2.0 × )

79 (4.9 × ) 111 (6.9 × ) 204 (3.5 × ) 227 (3.9 × ) – 519 (3.3 × )

111 (6.9 × ) 252 (4.3 × ) 486 (3.1 × )

208 (13.0 × ) 381 (6.6 × ) –

49 (3.1 × ) 16 (1.0 × ) 86 (1.5 × ) 58 (1.0 × ) – 157 (1.0 × )

33% 89 (7.4 × ) 65 (5.4 × ) 0.2 47% 196 (7.0 × ) 125 (4.5 × ) 62% – 192 (1.9 × )

89 (7.4 × ) 195 (7.0 × ) 415 (4.2 × )

33 (2.8 × ) 132 (4.7 × ) 311 (3.1 × )

64 (5.3 × ) 149 (5.3 × ) 405 (4.1 × )

47 (3.9 × ) 33 (2.8 × ) 118 (4.2 × ) 125 (4.5 × ) – 310 (3.1 × )

68 (5.7 × ) 175 (6.3 × ) 463 (4.7 × )

89 (7.4 × ) 196 (7.0 × ) –

27 (2.3 × ) 87 (3.1 × ) –

12 (1.0 × ) 28 (1.0 × ) 99 (1.0 × )

35% 24 (3.4 × ) 0.5 50% 69 (3.5 × ) 65% 169 (3.1 × )

20 (2.9 × ) 46 (2.3 × ) 94 (1.7 × )

24 (3.4 × ) 68 (3.4 × ) 150 (2.7 × )

8 (1.1 × ) 25 (1.3 × ) 80 (1.5 × )

20 (2.9 × ) 55 (2.8 × ) 168 (3.1 × )

19 (2.7 × ) 53 (2.6 × ) 132 (2.4 × )

7 (1.0 × ) 22 (1.1 × ) 68 (1.2 × )

20 (2.9 × ) 65 (3.3 × ) 147 (2.7 × )

24 (3.4 × ) 69 (3.5 × ) 168 (3.1 × )

14 (2.0 × ) 37 (1.9 × ) 95 (1.7 × )

7 (1.0 × ) 20 (1.0 × ) 55 (1.0 × )

30% 141 (2.4 × ) 110 (1.9 × ) 0.1 43% 273 (2.6 × ) 234 (2.3 × ) 56% – –

157 (2.7 × ) 283 (2.7 × ) –

130 (2.2 × ) 274 (2.6 × ) –

244 (4.1 × ) – –

123 (2.1 × ) 245 (4.1 × ) 237 (2.3 × ) 747 (7.2 × ) – –

129 (2.2 × ) 244 (2.4 × ) –

142 (2.4 × ) 85 (1.4 × ) 59 (1.0 × ) 272 (2.6 × ) 283 (2.7 × ) 104 (1.0 × ) – – 209 (1.0 × )

65 (10.8 × ) 108 (5.4 × ) –

112 (18.7 × ) – –

56 (9.3 × ) 134 (22.3 × ) 35 (5.8 × ) 54 (9.0 × ) 115 (5.8 × ) – 90 (4.5 × ) 115 (5.8 × ) – – 208 (4.1 × ) –

38 (6.3 × ) 90 (4.5 × ) –

88 (14.7 × ) 193 (9.7 × ) –

PathMNIST

36% 96 (8.7 × ) 33 (3.0 × ) 0.1 52% 155 (6.0 × ) 78 (3.0 × ) 68% – 156 (2.1 × )

83 (7.6 × ) 155 (6.0 × ) 400 (5.3 × )

31 (2.8 × ) 123 (4.7 × ) 264 (3.5 × )

37 (3.4 × ) 137 (5.3 × ) 358 (4.7 × )

34 (3.1 × ) 20 (1.8 × ) 82 (3.2 × ) 123 (4.7 × ) 202 (2.7 × ) 268 (3.5 × )

82 (7.4 × ) 236 (9.1 × ) –

96 (8.7 × ) 20 (1.8 × ) 11 (1.0 × ) 235 (9.0 × ) 78 (3.0 × ) 26 (1.0 × ) – 157 (2.1 × ) 76 (1.0 × )

TissueMNIST

20% 0.1 29% 37%

– – –

67 (67.0 × ) – –

262 (262.0 × ) – –

– – –

– – –

83 (83.0 × ) – –

– – –

33 (33.0 × ) 411 (11.4 × ) –

411 (411.0 × ) 33 (33.0 × ) 1 (1.0 × ) – 117 (3.3 × ) 36 (1.0 × ) – – 222 (1.0 × )

Average

–

7.4×

7.5×

20.6×

4.2×

5.7×

7.2×

4.0×

6.8×

CIFAR-10

CIFAR-100

𝑻Acc FedAvg FedDyn FedASAM FedNTD FedGELA FedEXP FedLMD FedDecorr FedACG FedSOL FedADB

36% OrganAMNIST 0.1 51% 67%

– – –

indicated by red dots, consistently lie within high-performing regions. Overall, FedADB exhibits minimal performance fluctuations across diverse CAG configurations, further confirming its stability. Noise Scale

L2 Wight

Learning Rate

Iteration

Figure 3: Performance of FedADB under different hyperparameters settings of CAG

5.5

Ablation Study

To assess the contribution of each component, we conducted an ablation study on CIFAR-10 under three non-IID settings 𝛼 ∈ {0.05, 0.1, 0.2}, as summarized in Tab.3. First, combining AGB with LCB (w/o CAG) yielded absolute gains of +8.20%, +2.64%, and +1.41%, respectively. This demonstrates that the dual-branch architecture effectively balances local learning and global knowledge preservation, whose critical role is highlighted by the more pronounced gains under severe heterogeneity (𝛼 = 0.05). Next, combining AGB with CAG (w/o LCB) produced consistent improvements

31 (1.6 × ) 56 (2.1 × ) 81 (1.6 × )

38 (6.3 × ) 87 (4.4 × ) –

27.7×

3.9×

20 (1.0 × ) 27 (1.0 × ) 50 (1.0 × )

6 (1.0 × ) 20 (1.0 × ) 51 (1.0 × )

1.0×

Table 3: Ablation study for different components of FedADB. DBCT AGB

LCB

× ✓ ✓ ✓

× ✓ × ✓

CAG × × ✓ ✓

Test accuracy (%) 𝜶 = 0.05

𝜶 = 0.1

𝜶 = 0.2

33.87 42.07(+8.20) 40.40(+6.53) 50.06(+16.19)

47.68 50.32(+2.64) 50.22(+2.54) 59.21(+11.53)

56.03 57.44(+1.41) 58.71(+2.68) 64.55(+8.52)

of +6.53%, +2.54%, and +2.68%. These gains suggest that class anchors supply explicit supervision for missing or underrepresented classes, which is increasingly valuable under severe heterogeneity. Furthermore, integrating all components yielded the largest performance boosts of +16.19%, +11.53%, and +8.52%, highlighting the importance of jointly leveraging global knowledge retention, local adaptation, and anchor-guided supervision. Crucially, the overall gains surpass the sum of individual contributions, further validating the structural complementarity of FedADB. We further conducted an ablation study in Tab. 4 to verify the dynamic balancing coefficient 𝛽. Compared to fixed branch weights ranging from 0.1 to 0.9, the dynamic 𝛽 achieved superior performance on both CIFAR-10 and CIFAR-100. Although fixed weights

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

CINIC-10

CIFAR-10

CIFAR-100

Rounds

Rounds

Rounds

Accuracy (%)

SVHN

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

Rounds

PathMNIST

TissueMNIST

Accuracy (%)

OrganAMNIST

Rounds

Rounds

Rounds

Figure 4: Test accuracy trajectories across seven datasets (𝛼 = 0.1). Curves are smoothed using an EWMA with factor of 0.2, and the shaded areas represent the 95% confidence intervals based on three random seeds. Table 4: Ablation study of fixed branch weights and the proposed dynamic balancing coefficient 𝛽. Dataset

0.1

0.3

0.5

0.7

0.9

𝛽

CIFAR-10 CIFAR-100

56.95 56.03

55.32 54.40

55.09 56.03

52.08 56.64

52.65 56.61

59.21 57.70

Loss Landscape. Prior studies indicate that models converging to flatter minima are more robust in retaining previously learned knowledge when adapting to new data distributions [9, 10]. We analyze the global models using Hessian eigen-decomposition [26], computing the top-5 eigenvalues via power iteration at each converged solution. Fig. 6 visualizes the loss landscapes along the two principal eigendirections, while Tab. 5 quantifies 𝜆1 (sharpness), 𝜆1 /𝜆5 (anisotropy), and training accuracy ( knowledge retention).

also bring certain gains, 𝛽 provided a more optimal balance by dynamically adjusting branch contributions based on data heterogeneity and anchor quality, ultimately leading to better performance.

Table 5: Hessian spectrum characteristics and training accuracy of the global models on CIFAR-10 with 𝛼 = 0.1.

6

Metric

FedAvg

FedNTD

FedLMD

FedSOL

FedADB

𝜆1 𝜆1 /𝜆5 Accuracy

311 2.41 52.27

169 1.95 57.48

298 1.47 66.23

126 1.46 62.66

99 1.23 74.78

Class 0-9

ACC 0.8

300

Rounds

600

900

0

300

600 Rounds

900

0.5

FedADB Server

FedSOL Server

0.6

0.3 0.2

0 -0.5 1.0

FedLMD Server

FedAvg Server

Direction

0

Notably, FedADB converges to the flattest and most isotropic region (lowest 𝜆1 and 𝜆1 /𝜆5 ), yielding the highest accuracy (74.78%). Conversely, FedAvg suffers from severe directional non-uniformity (highest 𝜆1 = 311 and 𝜆1 /𝜆5 = 2.41), and the lowest accuracy (52.27%). This quantitative analysis validates the intrinsic relationship between landscape flatness and knowledge retention, further demonstrating FedADB’s substantial advantage in constructing smooth optimization landscapes to preserve global knowledge. FedAvg

FedNTD

FedLMD

FedSOL

FedADB 100 80

60 40

Erro (%)

Class 0-9

Knowledge Preservation. We systematically evaluated the ability of FedADB to preserve global knowledge throughout federated training. Fig. 5 shows the evolution of class-wise accuracy on CIFAR10 with 𝛼 = 0.1, with lighter colors indicating higher accuracy. FedAvg exhibits periodic collapse and inter-round instability on minority classes: after each round of local updates, the performance on several classes drops sharply and fluctuates dramatically. Although fluctuations persist, FedLMD and FedSOL improve knowledge retention relative to FedAvg. In contrast, FedADB maintains higher and more balanced class-wise accuracy, effectively suppressing inter-round oscillations. These results demonstrate that FedADB consistently preserves and propagates global knowledge, resulting in more stable convergence and better performance.

-1.0 -0.5

Analysis

20

-1.0 -0.5 0 -0.5 1.0-1.0 -0.5 0 -0.5 1.0 -1.0 -0.5 0 -0.5 1.0 -1.0 -0.5 0 -0.5 1.0 -1.0 -0.5 0 -0.5 1.0

Direction

Direction

Direction

Direction

Direction

0.0

0

300

Rounds

600

900

0

300

Rounds

600

900

Figure 5: Class-wise accuracy of the global model over rounds.

Figure 6: Loss landscapes of converged global models projected onto the two dominant Hessian eigendirections.

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

Zhanyan Liu et al.

Class Anchor Prototype Alignment. We investigate whether class anchors align with the class-semantic prototypes represented by the evolving global model. Fig. 7 presents the t-SNE embeddings of class anchors and real-data on SVHN with 𝛼 = 0.1. From early (T1) to late (T2) stages, anchors progressively converge toward their corresponding real-data cluster centers. Simultaneously, inter-class margins expand, yielding more pronounced cluster separation. By T2, anchor embeddings substantially overlap with the real-data distribution. These observations suggest that class anchors serve as consistent global references across clients, preserving inter-class separability and decision boundaries during local training.

Cross-Domain Dataset. To further assess the robustness of FedADB against severe feature shifts and label skew, we conduct experiments on the PACS cross-domain benchmark [31] under a cross-silo setting with label-space mismatch. As reported in Tab. 6, FedADB yields a substantial performance margin of 7.59% over the strongest baseline. This demonstrates FedADB’s superior capability in generalizing across diverse visual domains. Table 6: Accuracy comparison on PACS with InceptionV1 Method

FedAvg

FedNTD FedLMD

FedSOL

FedADB

Accuracy 40.35(±0.13) 44.29(±1.54) 48.71(±1.03) 43.08(±0.44) 56.30(±0.58)

T1

T2

T1

T2

Figure 7: t-SNE visualization of class anchors and real-data representations at different training stages on SVHN. We further quantify this alignment statistically. Specifically, we project features onto the first three principal components via PCA and compute the Pearson correlation coefficients between class anchors and real-data class centers. As shown in Fig. 8, anchor representations increasingly align with real-data semantics: diagonal correlations are progressively strengthened while off-diagonal correlations are suppressed. This trend demonstrates improved classwise correspondence and reduced inter-class confusion, confirming that the synthesized anchors effectively capture the semantic prototypes, corroborating the t-SNE visualizations. T1

T2

Figure 8: PCA-based feature distributions and correlation analysis between class anchors and real-data class prototypes at different training stages on SVHN.

Privacy Analysis of Class Anchors. We visualize the class anchors generated on CIFAR-10, SVHN, and PathMNIST in Fig. 9. CAG only makes explicit the public prior knowledge in the global model. Therefore, transmitting class anchors does not introduce additional privacy leakage beyond sharing the global model itself. This privacy-preserving property is further supported by the low PSNR values and the visually unrecognizable details shown in Fig. 9. Notably, the effectiveness of FedADB does not depend on photorealistic anchor generation. Although the anchors appear as high-confidence noise in the pixel space, they remain aligned with the cluster centers of real data in the feature space while preserving inter-class separability, as illustrated by the t-SNE visualization in Fig. 7. Consequently, these anchors can provide stable supervisory signals and gradient constraints for locally missing classes. CIFAR-10 Class

SVHN

PathMNIST

Airplane

Cat

Num. 0 Num. 1

Adipose Mucus

18.22

17.41

16.40

13.42

Anchor Images PSNR

16.31

14.05

Figure 9: Visualization and PSNR of the class anchors.

7

Conclusion

Federated learning is an important technical paradigm for joint analysis of multimodal data across multiple data sources. Existing FL methods for mitigating catastrophic forgetting suffer from conflicting objectives between global knowledge preservation and local adaptation, while lacking effective supervision for missing classes under non-IID settings. To address these challenges, we propose FedADB, which incorporates two key innovations. First, we construct class anchors directly from the global model through differentiable optimization, thereby eliminating the dependence on auxiliary priors and avoiding additional privacy risks. Notably, only one anchor per class is required, achieving lightweight yet effective global knowledge preservation. Second, we introduce a dual-branch collaborative training mechanism that simultaneously preserves global knowledge and learns locally discriminative features, alleviating the global-local conflict inherent in existing methods. Extensive experiments on both medical image and natural image datasets validate the effectiveness and advantages of FedADB.

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

Acknowledgements This work is supported by the National Natural Science Foundation of China (Grant No. 62472047, No. 62572068, No. 62502042, No. 62572073), and the 111 Project (Grant No. B21049).

References [1] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. 2021. Federated Learning Based on Dynamic Regularization. In International Conference on Learning Representations. https://openreview.net/forum?id=B7v4QMR6Z9w [2] Meenakshi Aggarwal, Vikas Khullar, Nitin Goyal, Bhavani Sankar Panda, Hardik Doshi, Nafeesh Ahmed, Vivek Bhardwaj, and Gaurav Sharma. 2025. FL-QNNs: Memory Efficient and Privacy Preserving Framework for Peripheral Blood Cell Classification. IEEE Access 13 (2025), 138197–138214. [3] Mohammed Aljahdali, Ahmed M Abdelmoniem, Marco Canini, and Samuel Horváth. 2025. Flashback: Understanding and mitigating forgetting in federated learning. In 2025 3rd International Conference on Federated Learning Technologies and Applications (FLTA). IEEE, IEEE, New York, NY, USA, 408–415. [4] Jinyu Cai, Yunhe Zhang, Jicong Fan, and See-Kiong Ng. 2024. Lg-fgad: An effective federated graph anomaly detection framework. In Proceedings of the International Joint Conference on Artificial Intelligence. 3760–3769. [5] Jinyu Cai, Yunhe Zhang, Zhoumin Lu, Wenzhong Guo, and See-Kiong Ng. 2024. Towards effective federated graph anomaly detection via self-boosted knowledge distillation. In Proceedings of the 32nd ACM International Conference on Multimedia. 5537–5546. [6] Debora Caldarola, Barbara Caputo, and Marco Ciccone. 2022. Improving Generalization in Federated Learning by Seeking Flat Minima. In European Conference on Computer Vision. 654–672. [7] Jinqian Chen, Jihua Zhu, and Qinghai Zheng. 2023. Towards fast and stable federated learning: Confronting heterogeneity via knowledge anchor. In Proceedings of the 31st ACM International Conference on Multimedia. 8697–8706. [8] Mengmeng Chen, Xiaohu Wu, Xiaoli Tang, Tiantian He, Yew Soon Ong, Qiqi Liu, Qicheng Lao, and Han Yu. 2024. Free-rider and conflict aware collaboration formation for cross-silo federated learning. Advances in Neural Information Processing Systems 37 (2024), 54974–55004. [9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PmLR, 1597–1607. [10] Xinlei Chen, Saining Xie, and Kaiming He. 2021. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision. 9640–9649. [11] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. 2021. Exploiting shared representations for personalized federated learning. In International conference on machine learning. PMLR, 2089–2099. [12] Luke N. Darlow, Elliot J. Crowley, Antreas Antoniou, and Amos J. Storkey. 2018. CINIC-10 is not ImageNet or CIFAR-10. arXiv:1810.03505 [cs.CV] https://arxiv. org/abs/1810.03505 [13] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. 2020. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in neural information processing systems 33 (2020), 3557–3568. [14] Ziqing Fan, Jiangchao Yao, Bo Han, Ya Zhang, Yanfeng Wang, et al. 2023. Federated learning with bilateral curation for partially class-disjoint data. Advances in Neural Information Processing Systems 36 (2023), 32006–32019. [15] Dehong Gao, Duanxiao Song, Guangyuan Shen, Xiaoyan Cai, Libin Yang, Gongshen Liu, Xiaoyong Li, and Zhen Wang. 2024. Fedsts: A stratified client selection framework for consistently fast federated learning. IEEE Transactions on Neural Networks and Learning Systems 36, 5 (2024), 9180–9194. [16] Yongxin Guo, Xiaoying Tang, and Tao Lin. 2023. Fedbr: Improving federated learning on heterogeneous data via local learning bias reduction. In International conference on machine learning. PMLR, 12034–12054. [17] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. 2019. Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification. arXiv:1909.06335 [cs.LG] https://arxiv.org/abs/1909.06335 [18] Ming Hu, Zhihao Yue, Xiaofei Xie, Cheng Chen, Yihao Huang, Xian Wei, Xiang Lian, Yang Liu, and Mingsong Chen. 2024. Is aggregation the only choice? federated learning via layer-wise model recombination. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1096–1107. [19] Wenke Huang, Mang Ye, and Bo Du. 2022. Learn from others and be yourself in heterogeneous federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10143–10153. [20] Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang. 2021. Personalized cross-silo federated learning on non-iid data. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35(9). 7865–7873.

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

[21] J Stuart Hunter. 1986. The exponentially weighted moving average. Journal of quality technology 18, 4 (1986), 203–210. [22] Yash Jani1 , Arth Jani, Kashyap Prajapati, and CND Windsor. 2023. Leveraging multimodal ai in edge computing for real-time decision-making. computing 1 (2023), 2. [23] Divyansh Jhunjhunwala, Shiqiang Wang, and Gauri Joshi. 2023. FedExP: Speeding Up Federated Averaging via Extrapolation. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=IPrzNbddXV [24] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning. PMLR, 5132–5143. [25] Geeho Kim, Jinkyu Kim, and Bohyung Han. 2024. Communication-efficient federated learning with accelerated client gradient. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12385–12394. [26] Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2017. Federated Learning: Strategies for Improving Communication Efficiency. arXiv:1610.05492 [cs.LG] https: //arxiv.org/abs/1610.05492 [27] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. 2009. Cifar-10 and cifar-100 datasets. URl: https://www. cs. toronto. edu/kriz/cifar. html 6, 1 (2009), 1. [28] Gihun Lee, Minchan Jeong, Sangmook Kim, Jaehoon Oh, and Se-Young Yun. 2024. Fedsol: Stabilized orthogonal learning with proximal restrictions in federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12512–12522. [29] Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, and Se-Young Yun. 2022. Preservation of the global knowledge by not-true distillation in federated learning. Advances in Neural Information Processing Systems 35 (2022), 38461–38474. [30] Gwen Legate, Lucas Caccia, and Eugene Belilovsky. 2023. Re-weighted softmax cross-entropy to control forgetting in federated learning. In Conference on Lifelong Learning Agents. PMLR, 764–780. [31] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017. Deeper, Broader and Artier Domain Generalization. In Proceedings of the IEEE International Conference on Computer Vision. 5542–5550. [32] Gang Li, Jun Cai, Jianfeng Lu, and Hongming Chen. 2024. Incentive mechanism design for cross-device federated learning: A reinforcement auction approach. IEEE Transactions on Mobile Computing 24, 4 (2024), 3059–3075. [33] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. 2022. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE). IEEE, 965–978. [34] Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10713–10722. [35] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems 2 (2020), 429–450. [36] Junxu Liu, Jian Lou, Li Xiong, Jinfei Liu, and Xiaofeng Meng. 2024. Cross-silo federated learning with record-level personalized differential privacy. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 303–317. [37] Tianhan Liu, Zhuang Qi, Zitan Chen, Xiangxu Meng, and Lei Meng. 2023. Crosstraining with prototypical distillation for improving the generalization of federated learning. In 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 648–653. [38] Jianghu Lu, Shikun Li, Kexin Bao, Pengju Wang, Zhenxing Qian, and Shiming Ge. 2023. Federated learning with label-masking distillation. In Proceedings of the 31st ACM international conference on multimedia. 222–232. [39] Kangyang Luo, Xiang Li, Yunshi Lan, and Ming Gao. 2023. Gradma: A gradientmemory-based accelerated federated learning with alleviated catastrophic forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3708–3717. [40] Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. Vol. 24. Elsevier, 109–165. [41] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273–1282. [42] Lei Meng, Zhuang Qi, Lei Wu, Xiaoyu Du, Zhaochuan Li, Lizhen Cui, and Xiangxu Meng. 2024. Improving global generalization and local personalization for federated learning. IEEE Transactions on Neural Networks and Learning Systems 36, 1 (2024), 76–87. [43] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, Vol. 2011(5). Granada, 7.

MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil

[44] Zhuang Qi, Yuqing Wang, Zitan Chen, Ran Wang, Xiangxu Meng, and Lei Meng. 2022. Clustering-based curriculum construction for sample-balanced federated learning. In CAAI international conference on artificial intelligence. Springer, 155– 166. [45] Vijay Janapa Reddi. 2025. Generative AI at the edge: challenges and opportunities: the next phase in AI deployment. Queue 23, 2 (2025), 79–137. [46] Amirhossein Reisizadeh, Farzan Farnia, Ramtin Pedarsani, and Ali Jadbabaie. 2020. Robust federated learning: The case of affine distribution shifts. Advances in neural information processing systems 33 (2020), 21554–21565. [47] Changlong Shi, He Zhao, Bingjie Zhang, Mingyuan Zhou, Dandan Guo, and Yi Chang. 2025. FedAWA: adaptive optimization of aggregation weights in federated learning using client vectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 30651–30660. [48] Yujun Shi, Jian Liang, Wenqing Zhang, Vincent Tan, and Song Bai. 2023. Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=EXnIyMVTL8s [49] Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, and Itai Zeitak. 2019. Overcoming Forgetting in Federated Learning on Non-IID Data. arXiv:1910.07796 [cs.LG] https://arxiv.org/abs/1910.07796 [50] Changlin Song, Divya Saxena, Jiannong Cao, and Yuqing Zhao. 2024. FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning. arXiv:2404.09210 [cs.LG] https://arxiv.org/abs/2404.09210 [51] Canh T Dinh, Nguyen Tran, and Josh Nguyen. 2020. Personalized federated learning with moreau envelopes. Advances in neural information processing systems 33 (2020), 21394–21405. [52] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. 2022. Towards personalized federated learning. IEEE transactions on neural networks and learning systems 34, 12 (2022), 9587–9603. [53] Haozhao Wang, Yichen Li, Wenchao Xu, Ruixuan Li, Yufeng Zhan, and Zhigang Zeng. 2023. Dafkd: Domain-aware federated knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20412–20421. [54] Haozhao Wang, Peirong Zheng, Xingshuo Han, Wenchao Xu, Ruixuan Li, and Tianwei Zhang. 2024. Fednlr: Federated learning with neuron-wise learning rates. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3069–3080.

Zhanyan Liu et al.

[55] Xubin Wang, Zhiqing Tang, Jianxiong Guo, Tianhui Meng, Chenhao Wang, Tian Wang, and Weijia Jia. 2025. Empowering edge intelligence: A comprehensive survey on on-device ai models. Comput. Surveys 57, 9 (2025), 1–39. [56] Meihan Wu, Li Li, Tao Chang, Peng Qiao, Cui Miao, Jie Zhou, Jingnan Wang, and Xiaodong Wang. 2024. FedEKT: Ensemble Knowledge Transfer for ModelHeterogeneous Federated Learning. In 2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS). IEEE, 1–10. [57] Zhiyuan Wu, Sheng Sun, Yuwei Wang, Min Liu, Quyang Pan, Junbo Zhang, Zeju Li, and Qingxiang Liu. 2024. Exploring the distributed knowledge congruence in proxy-data-free federated distillation. ACM Transactions on Intelligent Systems and Technology 15, 2 (2024), 1–34. [58] Yunlu Yan, Huazhu Fu, Yuexiang Li, Jinheng Xie, Jun Ma, Guang Yang, and Lei Zhu. 2025. A simple data augmentation for feature distribution skewed federated learning. In Proceedings of the Computer Vision and Pattern Recognition Conference. 25749–25758. [59] Haibo Yang, Minghong Fang, and Jia Liu. 2021. Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=jDdzh5uld [60] Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. 2023. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data 10, 1 (2023), 41. [61] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. 2023. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37(9). 11237–11244. [62] Rongyu Zhang, Yun Chen, Chenrui Wu, Fangxin Wang, and Bo Li. 2024. Multilevel personalized federated learning on heterogeneous and long-tailed data. IEEE Transactions on Mobile Computing 23, 12 (2024), 12396–12409. [63] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. 2021. Federated learning on non-IID data: A survey. Neurocomputing 465 (2021), 371–390. [64] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. 2021. Data-free knowledge distillation for heterogeneous federated learning. In International conference on machine learning. PMLR, 12878–12889. [65] Zhaowei Zhu, Yuanshun Yao, Jiankai Sun, Hang Li, and Yang Liu. 2023. Weak proxies are sufficient and preferable for fairness with missing sensitive attributes. In International Conference on Machine Learning. PMLR, 43258–43288.

Related documents

• FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting#615282
• FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting#614315
• ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning#1143434
• FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning#173320
• C2FL: Clustered Continual Federated Learning under Spatial and Temporal Drift#282802