arXiv CSopen access

FedIDM: Achieving Fast and Stable Convergence in Byzantine Federated Learning through Iterative Distribution Matching

FedIDM: Achieving Fast and Stable Convergence in Byzantine Federated Learning through Iterative Distribution Matching

arXiv:2604.15115v1 [cs.LG] 16 Apr 2026

He Yang , Dongyi Lv , Wei Xi , Song Ma , Hanlin Gu , Jizhong Zhao

Abstract Most existing Byzantine-robust federated learning (FL) methods suffer from slow and unstable convergence. Moreover, when handling a substantial proportion of colluded malicious clients, achieving robustness typically entails compromising model utility. To address these issues, this work introduces FedIDM, which employs distribution matching to construct trustworthy condensed data for identifying and filtering abnormal clients. FedIDM consists of two main components: (1) attacktolerant condensed data generation, and (2) robust aggregation with negative contribution-based rejection. These components exclude local updates that (1) deviate from the update direction derived from condensed data, or (2) cause a significant loss on the condensed dataset. Comprehensive evaluations on three benchmark datasets demonstrate that FedIDM achieves fast and stable convergence while maintaining acceptable model utility, under multiple state-of-the-art Byzantine attacks involving a large number of malicious clients.

1

Introduction

Federated learning (FL) [Kairouz et al., 2021; Huang et al., 2024; Yang et al., 2019; Lyu et al., 2022] facilitates learning from decentralized data sources while preserving privacy, enabling a wide range of promising, privacy-enhancing applications. Despite its advantages, the non-transparent nature of local training data and processes renders FL vulnerable to various Byzantine attacks [Wan et al., 2023; Li et al., 2023; Dong et al., 2023; Zhao et al., 2024; Liu et al., 2024], where the attacker-controlled malicious clients corrupt the model utility by submitting manipulated model updates. Thus far, considerable efforts have been dedicated to developing Byzantine-robust methods, which can be broadly classified into two categories. Firstly, one prevalent approach focuses on eliminating geometrical outliers using statistical techniques such as [Blanchard et al., 2017; Guerraoui et al., 2018; Yin et al., 2018]. Secondly, another line of research explores anomaly detection techniques, including consistency checks [Zhang et al., 2022], update inversion [Zhao et al., 2022], and trust bootstrapping [Cao et al., 2021], to

Existing Defenses

…

… Σ

…

Σ

…

client 1 client 2 client N convergence curve client 1 client 2 malicious clients

Figure 1: Problem illustration. When handling a substantial proportion of colluded malicious clients, most existing Byzantine-robust methods may inadvertently discard valuable updates while preserving poisoned ones, hindering fast and stable convergence.

mitigate the impact of poisoned updates. Although these defenses demonstrate effectiveness against Byzantine attacks in certain scenarios, they face an important limitation when a substantial proportion of malicious clients engage in collusion. In such a case, these methods may inadvertently discard valuable updates from honest participants while retaining poisoned ones, hindering fast and stable convergence (see Figure 1), which is crucial for the effectiveness of Byzantinerobust FL methods [Huang et al., 2024]. To address this issue, we introduce FedIDM, an efficient Byzantine-robust federated learning framework based on iterative distribution matching(in Figure 2). FedIDM divides the federated training process into two stages. In the former stage, FedIDM employs iterative distribution matching to construct compact synthetic datasets (i.e., condensed data) for global model training, which encapsulates rich and representative information, thereby accelerating convergence. In the latter stage, the server adjusts each client’s local update using historical information and evaluates them on the condensed data. These updates are then aggregated using robust aggregation (RA) with a negative contribution-based rejection strategy to mitigate the impact of poisoned updates, preserving model utility and ensuring stable convergence. However, a significant challenge in FedIDM arises when constructing trustworthy condensed data, as malicious clients may launch label-flipping attacks [Jiang et al., 2023; Jebreel et al., 2024; Jebreel and Domingo-Ferrer, 2023] during this process. Defending against such attacks is challenging for two reasons: Firstly, the compression process in distribu-

tion matching necessitates a delicate balance between efficiency and information granularity, potentially causing the loss of critical details needed to detect subtle alterations. Furthermore, data heterogeneity introduces diversity in condensed data, complicating the establishment of a standardized anomaly detection mechanism. Therefore, we introduce an attack-tolerant condensed data generation (ACDG) scheme via contrastive label rectification into FedIDM. Concretely, contrastive learning combined with a Gaussian Mixture Model (GMM) is used to update a rectification network, which mitigates label pollution caused by label-flipping attacks in the condensed data. The rectification network processes each data point, generating model predictions as pseudo-labels. The pseudo-labeled condensed data are then utilized to update the global model. Overall, our contributions can be summarized as follows: • We introduce FedIDM, which leverages iterative distribution matching to ensure robustness while facilitating fast and stable convergence. • FedIDM has a minimal impact on model utility, even in scenarios involving a significant proportion of colluded malicious clients. • We empirically evaluate FedIDM against multiple stateof-the-art attacks on three benchmark datasets, demonstrating a substantial improvement compared to existing defenses.

2

Related Work

Byzantine Attacks against FL. In FL, Byzantine attacks are designed to diminish the global model’s performance by corrupting a portion of the training data or manipulating local model updates during aggregation. In this study, we investigate three state-of-the-art Byzantine attacks: little is enough (LIE) [Baruch et al., 2019], static optimization (STAT-OPT) [Shejwalkar and Houmansadr, 2021; Shejwalkar et al., 2022], and Dynamic Optimization (DYNOPT) [Shejwalkar and Houmansadr, 2021; Shejwalkar et al., 2022]. LIE: The Little Is Enough (LIE) attack subtly corrupts the aggregation process by injecting minimal noise into each dimension of the averaged benign updates. Initially, the adversary calculates the average (∇b ) and standard deviation (σ) of the benign updates. Subsequently, a scaling coefficient (z) is derived from the ratio of compromised to benign clients. The adversarial update is formulated as ∇′ = ∇b + z · σ. STAT-OPT: The Static Optimization (STAT-OPT) attack introduces a general framework for poisoning in FL and customizes it for specific aggregation rules. The attack initiates by computing the average benign update, denoted as ∇b . It then determines a static malicious direction, denoted as ω = −sign(∇b ). The final poisoned update, ∇′ , is computed as −γω, where γ is a suboptimal value chosen to bypass the target aggregation rule. DYN-OPT: The Dynamic Optimization (DYN-OPT) begins by calculating the average of the benign updates, denoted as ∇b . It then perturbs this average in a dynamic, datadependent malicious direction, ω, to compute the final poi-

Table 1: Notations and Definitions.

Notation

Definition

f (·) h(·) h(si )k γik

Feature extractor of the rectification network. Classifier of the rectification network. Probability of si belonging to the k-th class. Probability of the i-th data point belonging to the k-th class. The transformed data of si . Local update of the i-th client. Base update on the server-side. Global model The condensed data during the t-th round.

(1)

(2)

si , si gi gs w St

soned update, ∇′ = ∇b + γω. The parameter γ is chosen to be the largest value that effectively circumvents the target aggregation rule. Byzantine-Robust FL Methods. Considerable research efforts have been dedicated to developing robust FL methods to counter Byzantine attacks.These approaches generally fall into four categories: statistical defenses, anomaly detection defenses, trust-bootstrapping defenses, and client-side defenses. However, several challenges persist: (1) The large number of participants and the non-IID nature of client data complicate the detection of poisoned updates, as local updates inherently exhibit high variability. (2) colluded malicious clients can blend poisoned updates with legitimate ones. (3) The adversary can leverage its potential knowledge to craft stealthy attacks, further complicating the detection of poisoning attempts. Consequently, existing defense strategies often suffer from slow and unstable convergence. Furthermore, in scenarios with a significant proportion of compromised and colluded clients, achieving robustness typically comes at the expense of model utility. These limitations underscore the need for advanced and more effective defense mechanisms.

3

Methodology

This section focuses on the detailed design of FedIDM. For clarity, the symbols used in this work and their corresponding explanations are provided in Table 1.

3.1

Threat Model

Adversary’s Objective. The training objective of FedIDM is defined as the following optimization problem: P|S| 1 w∗ = arg minw∈Rd 1[t ≤ T ] |S| i=1 ℓ(si , yi ; w) + 1[t > PN 1 d T ] N i=1 fi (w), where w ∈ R represents the parameter to be optimized, t denotes the communication round between clients and the server, and T is a constant. S represents the condensed data, si denotes a single data point in S, yi corresponds to the annotated label, which could potentially be a dirty label from an adversary. N is the total number of clients, each of whom conducts local training with the local objective fi : Rp → R based on its private dataset Di . In Byzantine FL, the adversary seeks to disrupt convergence by injecting malicious updates, thereby hindering global model optimization and diminishing its utility in real-world applications.

ACDG …

1 0 0

encoder

0 0 1

Eq.(1)

f(·) h(·)

0.1 0.2 0.2 … 0.2 correction 0.7 0.6 label

local

RA

training server training

𝒈𝑖 𝑁 𝑖=1

𝒈1′

𝒈1

𝒈′2 𝒈2 𝒈′3

𝑔1′

1.32

𝛼𝑖 = 𝑐𝑜𝑠 𝒈′𝑖 , 𝒈𝑠

𝒈3

𝒈′4

0.97

𝒈′5

1.04

σ𝑖 𝛼𝑖 𝒈′𝑖 𝒈= σ𝑖 𝛼𝑖

calculate loss

aggregation

𝒈𝑠 𝒈𝑠

𝒈′5

training

pseudo labels

polluted labels condensed data rectification network

local data

global

𝒈5 𝒈′4

𝒈4 Eq.(8)

global updating

Figure 2: Framework of FedIDM. f (·) and h(·) denote the feature extractor and classifier, respectively.

Adversary’s Capabilities. Following the attack setting in previous works [Fang et al., 2020; Shejwalkar and Houmansadr, 2021; Shejwalkar et al., 2022], FedIDM adopts and extends certain assumptions regarding the capabilities of the adversary: (1) The adversary cannot access the training data of honest clients but has visibility into the local updates of honest clients. (2) The adversary can control multiple clients, referred to as malicious clients, under the assumption that their number may constitute up to 50% of the total clients. (3) The adversary repetitively and dynamically executes attacks throughout the training process across multiple rounds. In this work, we randomly designate 50% of the total rounds as adversarial rounds. (4) The adversary cannot influence the server (such as altering aggregation rules) or interfere with the training process of other honest clients.

3.2

Attack-tolerant Condensed Data Generation

During the attack-tolerant condensed data generation (ACDG) stage, clients locally generate a series of condensed data and then communicate condensed data to the central server. Subsequently, the server aggregates condensed data from all clients to facilitate the training of a global model. Inspired by recent advances in data distillation [Zhao and Bilen, 2023; Zhao et al., 2023; Wang et al., 2022; Xiong et al., 2023], we formulate the following minimization problem to generate condensed data: |D| |S| 1 X 1 X S = arg min Eθ∼Pθ ∥ ϕθ (xi ) − ϕθ (sj )∥2 S |D| i=1 |S| j=1 ∗

+ υLCE (S) (1) where D and S denote the local training data and condensed data, respectively. Pθ represents the distribution of the randomly initialized network parameters, and ϕθ refers to the model ϕ with parameters θ. LCE indicates the crossentropy loss, and υ represents the regularization term. For simplicity, we omit the client subscript, as all clients follow the same optimization process.

However, this process is vulnerable to label-flipping attacks by the adversary, potentially disrupting model training. Therefore, we propose a contrastive label rectification method that incorporates a rectification network, consisting of a shared encoder, a feature extractor f (·), and a classifier h(·). ACDG involves two main stages: (1) modeling the distribution of condensed data. (2) generating pseudo-labels as the rectified labels and updating the global model. Modeling the Distribution of Condensed Data. Firstly, ACDG utilizes a sliding window mechanism combined with semantic transformations for data augmentation. Specifically, at round t, we consolidate condensed data from t−(δ −1) to t and feed them into the rectification network to cleanse potentially polluted labels. Given the aggregated condensed data S̄ (defined as {S t−(δ−1) , · · · , S t }), comprising m data points {s1 , · · · , sm }, we apply semantic transformations through data augmentation techniques (such as flipping and cropping) to each data point twice. This yields a new set Se = (1) (2) (1) (2) (1) (2) {s1 , s1 , · · · , sm , sm }, where si and si denote transformed data points. Subsequently, a GMM is utilized to model the distribue Upon tion of Se over its encoded representation r = f (S). introducing discrete latent variables z ∈ {1, 2, · · · , K}, which correspond to the distinct categories within the original dataset, the GMM is specified as follows: p(r) =

K X k=1

p(r, z = k) =

K X

p(z = k)N (r; µk , σk I)

k=1

(2) where µk and σk represent the mean vector and standard deviation scalar of the k-th component in the GMM. I denotes the identity matrix. For the sake of convenience in description, we denote Se as {s′1 , · · · , s′2m } here. During the parameter update phase of the GMM, we employ the model predictions h(s′i ) for i = 1, · · · , 2m to guide the parameter adjustment process. Specifically, we integrate the model predictions into a standard Expectation-Maximization (EM) algorithm [Dempster et al., 1977], replacing the traditional posterior probabilities.

Consequently, the updated GMM parameters are determined as follows: P2m ′ i=1 h(si )k ri µk = P ′ i h(si )k (3) P2m ′ T k (ri − µk )(ri − µk ) i=1 h(si )P σk = ′ i h(si )k where h(s′i )k is the probability that the rectification network assigns data point s′i to the k-th category. Additionally, ri denotes the embedded representation of s′i . Generating Pseudo-labels After constructing the GMM, we proceed to calculate posterior probabilities γik , which quantify the probability that the i-th data point is assigned the k-th category:
exp −(ri − µk )T (ri − µk )/2σk γik = PK (4) T k=1 exp (−(ri − µk ) (ri − µk )/2σk ) For each original sample si with its claimed label yi , we determine the probability of the sample belonging to category k = yi based on γik . After calculating the posterior probabilities for the condensed data, we construct a two-component GMM, defined as follows: 1 X p(γiyi ) = p(γiyi , b) (5) b=0

where b is a binary latent variable representing the probability that γiyi belongs to a benign or malicious component. Subsequently, FedIDM relabels the transformed data points e within S: ( (1) (1) ỹi = βi yi + (1 − βi )h(si ) (6) (2) (2) ỹi = βi yi + (1 − βi )h(si ) where βi ∈ [0, 1] represents the posterior probability that si is free from pollution, as estimated by the two-component GMM, and yi denotes the claimed label. After relabeling the condensed knowledge, we calculate the loss for updating the rectification network: L = Lce + Lctr + Lmixup (7) where Lce denotes the cross-entropy loss, which   is Pm   (1) (2)  (2) (1) given by i=1 ℓ h(si ), ỹi + ℓ h(si ), ỹi , ensuring that the rectification network’s predictions for a pair of transformed data points are consistent. Lctr ] defined as is the InfoNCE loss [Oord et al., 2018   (1) (2) exp f (si )T f (si )/τ P   , which pro− log (1) (2) P (1) T s ,s ∈S̃ i

i

sj ∈S̄\si exp

f (si ) f (sj )/τ

motes the learning of meaningful representations. Furthermore, Lmixup is the cross-entropy loss for a mixed sample using the Mixup method [Zhang et al., 2018], calculated as (m) (m) (m) ℓ(h(si ), ỹim ), where si = ρsi + (1 − ρ)sj , ỹi = (1) (2) (1) (2) ρ((ỹi + ỹi )/2) + (1 − ρ)((ỹj + ỹj )/2), and si , sj ∈ S̄. When the rectification network is updated, FedIDM utilizes it to generate pseudo-labels for the aggregated condensed knowledge S̄. For each si ∈ S̄, the pseudo-label ỹi is computed using h(si ). Then these pseudo-labeled samples are integrated into the global model to facilitate its update.

3.3

Robust Aggregation with Negative Contribution-based Rejection

During the Robust Aggregation(RA) stage, each client independently refines a replica of the local model using its local data and then transmits the local update to the central server. During this process, the adversary may potentially execute attacks, which can compromise the integrity of the global model. In parallel, the server trains a separate replica of the global model using condensed knowledge, generating a base update. Subsequently, the server evaluates the contribution of each local update by calculating the cosine distance to the base update. This metric quantifies the alignment between the local and global updates, enabling the identification of updates with negative contributions. Local updates with negative contributions are discarded, while those with positive contributions are aggregated through a weighted sum based on their contributions. Contribution Evaluation. In the t-th round of the interaction between clients and the server, each client i computes a local update gti based on its private data. Simultaneously, the server maintains a memory pool, denoted as Gt = {gt−∆ , · · · , gt−1 }, storing historical global updates from the recent ∆ rounds, where gt represents the global update at round t. Additionally, the server calculates a base update gts using the aggregated knowledge S̄ with pseudolabels. Firstly, a local update correction is conducted to exclude those updates that significantly differ from the server’s base update, gts . To achieve this, we recalibrate each local update using a moving average between historical global updates and the local update. This correction aims to ensure the stationarity of the parameter updating process. Then a contribution αi of gti is evaluated based on the cosine similarity between this average and gts , determining whether to discard gti . If αi > 0, it indicates that gti will be retained; otherwise, it will be discarded. g′t i =

∆ X

λj+1 · norm(gt−j ) + λj · norm gti

j=1

αi = cos

t g′t i , gs

(8)

where λ is a constant, αi denotes the contribution of client i, and norm(·) is ℓ2 -normalization such that ∥gti ∥ is equal to 1. We primarily focus on the directional deviation of the gti from the base update gts , which necessitates the exclusion of the magnitude dimension to enhance the reliability of the contribution evaluation. Consequently, FedIDM applies normalization to each local update and historical global updates. Anomaly Detection and Aggregation Assuming that the recalibrated local updates having positive contribution constitute a plausible honest set χ′t = {g′t i |αi > 0}. Additionally, the corresponding set of original gradients is denoted as χt = {gti |αi > 0}. Although they exhibit positive contributions, stealthy poisoned updates may still be present. Therefore, additional anomaly detection is required. Firstly, we apply the DBSCAN algorithm to cluster updates in χ′t according to their contributions. The minimum number of samples required to form a cluster is set to 1, allowing even

/,(

67$7237

'<1237

 7(5

     

  5RXQGV



)HG$9*



 

%XO\DQ

0XOWLNUXP

  5RXQGV



7ULPPHGPHDQ



 

)/7UXVW

)HG'HI

Figure 3: The convergence results on the CIFAR-10 dataset.

/,(

67$7237

  5RXQGV









)HG,'0

'<1237

 7(5

     

  5RXQGV



)HG$9*



 

%XO\DQ

0XOWLNUXP

  5RXQGV



7ULPPHGPHDQ



 

)/7UXVW

)HG'HI

  5RXQGV

)HG,'0

Figure 4: The convergence results on the CIFAR-100 dataset.

a single update to be considered a separate cluster. Within each cluster, one update is randomly selected to represent the entire cluster, as updates with similar contributions may have comparable effects. This process aims to reduce redundancy and mitigate the undue influence of anomalous updates on the global model. Secondly, to prevent extreme deviations in the magnitude of local updates, we adjust the magnitude of each update in χ′t to the median of the original local updates in χt . This adjustment is formulated as follows: ′t g′t i = g · norm(gi ),

′t g′t i ∈χ |χt |

g = median(∥{gti }i=1 ∥),

gti ∈ χt

(9)

This step aims to address optimization issues caused by extreme updates from anomalous clients, thereby improving the stability of model training. Furthermore, we filter effective updates by assessing their impact on global model optimization. Specifically, each update is applied to the global model, and the loss on the distilled data is computed. The top K updates with the highest losses are then eliminated, and the set χ′t changes to χ′t = {g′t i |αi > 0, ℓi < ℓo }, where ℓo is a fixed lower bound loss value. This process removes updates that are ineffective or detrimental to global model optimization, ensuring a more reliable optimization process. Finally, the global update is computed by averaging the remained recalibrated local updates in χ′t , with each update

weighted according to its contribution αi . Formally, the computation is expressed as follows: X 1 αi g′t (10) gt = P i g′t ∈χ′t αi ′t ′t i

gi ∈χ

Subsequently, the server proceeds to update the global model using the resulting global update, as described below, where η is the global learning rate. wt+1 ← − wt − ηgt

4

(11)

Experiment

Datasets: We present experimental evaluations on three benchmark datasets for image classification: CIFAR-10, CIFAR-100 [Krizhevsky et al., 2009], and Tiny-20 [Le and Yang, 2015]. The CIFAR-10 and CIFAR-100 datasets consist of 50,000 32 × 32 training images, distributed across 10 and 100 categories, respectively. For CIFAR-100, classification is performed using superclasses. The Tiny-20 dataset, derived from Tiny-ImageNet, is created by randomly selecting 20 classes and includes 10,000 64 × 64 training images. Federated Learning Setup and Attack Assumption: In our experiments, the default number of clients is set to N = 250. During each communication round, 50 clients are randomly selected to contribute updates to the global model, which is constructed using a ResNet-18 or ResNet-34 architecture [He et al., 2016]. Consistent with prior studies [Wang

/,(

67$7237

'<1237

 7(5

     

  5RXQGV



)HG$9*



%XO\DQ

  0XOWLNUXP

  5RXQGV



7ULPPHGPHDQ



 

)/7UXVW

)HG'HI

  5RXQGV





)HG,'0

Figure 5: The convergence results on the Tiny-20 dataset. Table 2: TERs of multiple FL defenses against Byzantine attacks. All values are reported as percentages.

Dataset

Bulyan

Multi-Krum

Trimmed-mean

FLTrust

FedDef

FedIDM

LIE 82.01±7.40 CIFAR-10 STAT 90.00±0.00 DYN 90.00±0.00

87.83±3.65 90.00±0.00 90.00±0.00

88.04±1.96 90.00±0.00 90.00±0.00

88.71±2.49 74.38±8.14 90.00±0.00

36.88±5.02 34.01±13.16 76.18±0.67

84.60±6.06 89.93±0.26 89.97±0.10

14.08±0.29 13.87±0.29 13.75±0.19

LIE 87.94±3.31 CIFAR-100 STAT 95.00±0.00 DYN 95.00±0.00

92.99±2.26 95.00±0.00 95.00±0.00

92.98±1.62 95.00±0.00 95.00±0.00

93.44±1.56 85.95±8.75 95.00±0.00

35.74±0.81 39.82±1.08 39.50±0.90

92.31±6.61 91.52±3.48 94.51±0.96

16.68±0.23 16.32±0.23 16.40±0.23

LIE 89.05±1.45 STAT 95.00±0.00 DYN 95.00±0.00

94.53±1.23 95.00±0.00 95.00±0.00

94.34±0.94 95.00±0.00 95.00±0.00

94.53±1.03 82.96±6.36 95.00±0.00

46.80±2.50 49.44±1.24 49.37±0.83

94.26±3.06 95.00±0.00 94.51±0.96

27.43±0.87 27.98±0.68 27.49±1.01

Tiny-20

Attack FedAVG

et al., 2020; Li et al., 2021], we assume a Non-IID data distribution and utilize a Dirichlet distribution with a concentration parameter of 0.5 to partition data among clients. For experiments on CIFAR-100 and Tiny-20, the number of clients is set to N = 50, with all clients participating in federated training during each round. The federated learning process spans 1,000 communication rounds. We randomly designate 50% of the 1,000 rounds as adversarial rounds. During each adversarial round, we assume that the adversary compromises 50% of the participating clients. During the ACDG stage of FedIDM, the adversary performs label-flipping attacks, including SLF [Fang et al., 2020] and DLF [Shejwalkar et al., 2022], on the condensed data. During the RA stage in FedIDM, the adversary launches attacks including LIE, STATOPT, and DYN-OPT. Evaluation Metrics: We employ the standard testing error rate (TER) of the global model as the primary metric to assess the effectiveness of defense methods. A defense method is considered more robust if it yields lower TERs under adversarial conditions. Convergence. Figure 3, Figure 4, and Figure 5 depict the convergence results of FedAVG [McMahan et al., 2017], Bulyan [Guerraoui et al., 2018], Multi-Krum [Blanchard et al., 2017], Trimmed-mean [Yin et al., 2018], FLTrust [Cao et al., 2021], FedDef [Park et al., 2023], and FedIDM with respect to communication rounds. Statistical defenses are ineffective against potent attacks involving a significant pro-

portion of compromised clients. FedIDM demonstrates superior robustness against various attacks. Furthermore, FedIDM achieves the fastest and most stable convergence compared to other methods. During the ACDG phase, clients efficiently extract representative condensed data, which accelerates model training compared to other FL approaches. In the subsequent RA phase, each local update is precisely evaluated and corrected, ensuring stable convergence throughout the training process. Defense Efficacy. As shown in Table 2, FedIDM consistently achieves the lowest TERs across various datasets when defending against multiple attacks. Statistical defenses, such as Bulyan, Multi-Krum, and Trimmed-mean, fail to effectively filter anomalous updates due to the substantial proportion of compromised clients, leading to significant performance degradation. FLTrust, which assigns trust scores to local updates based on a root dataset, ensures robust aggregation. However, its performance can degrade if the root dataset deviates from the global data distribution, as shown in our experiments. Additionally, FedDefender, which introduces noise into the model, adversely impacts global model performance and fails to prevent Byzantine attacks. In contrast, FedIDM accurately evaluates the contribution of each local update and ensures a stable update process through anomaly detection, effectively mitigating the impact of poisoned updates and preserving model performance.

7(5

     



   

  

  5RXQGV

 

  

  7(5

   ,,'

7(5



 

  5RXQGV

(a)

 

)XOO ZR$&'* ZR5$

  

  5RXQGV

(b)

 

(c)

Figure 6: (a) Defense efficacy of FedIDM under varying degrees of Non-IID data distribution. (b) Defense efficacy of FedIDM on varying attacker ratios. (c) Effectiveness of different components within FedIDM, where “full” refers to the complete implementation of FedIDM.

Defense against Non-IID extent. The extent of Non-IID data distribution significantly affects FedIDM, influencing both label rectification during the ACDG stage and the evaluation of local updates during the RA stage. As shown in Figure 6(a), we assess the robustness of FedIDM under varying levels of Non-IID data distribution by adjusting the Dirichlet parameter. Specifically, we use Dirichlet distributions with parameters 0.1, 0.3, and 0.5 to simulate different degrees of Non-IID conditions. The results demonstrate FedIDM’s effectiveness in mitigating Byzantine attacks across various Non-IID scenarios. However, as the degree of Non-IID data increases, a decline in model performance is observed.

Table 3: RSRs, FPRs, and FNRs of FedIDM against SLF and DLF.

Defense Efficacy on Varying Attacker Ratio. We conducted additional experiments to examine the impact of varying proportions of compromised clients on the robustness of FedIDM. This variation provides deeper insights into the dynamics and resilience of FedIDM across different scenarios. Specifically, Figure 6(b) presents results from experiments on the CIFAR-10 dataset, evaluating FedIDM’s effectiveness against SLF and DYN-OPT attacks while varying the percentage of compromised clients. The findings demonstrate that FedIDM maintains resilience to these variations, underscoring its inherent robustness.

Furthermore, Figure 6(c) provides an intuitive assessment of the effectiveness of each FedIDM component. Firstly, we deactivate the ACDG module, which causes FedIDM to fail against SLF and DYN-OPT attacks. Without ACDG, FedIDM cannot effectively mitigate label corruption, making it vulnerable to SLF attacks. Moreover, the absence of ACDG causes inaccurate evaluation of local update due to polluted condensed data, making FedIDM susceptible to DYN-OPT attacks. Subsequently, we deactivate the RA module and compute the average of the local updates, which again leads to insufficient defense against the DYN-OPT attack.

Effectiveness of Different Component. Finally, we conduct a comprehensive analysis of the individual components of FedIDM to evaluate their respective contributions. The ACDG stage plays a critical role in the effectiveness of FedIDM, as it is responsible for generating trustworthy condensed data, which is essential for subsequent robust aggregation. We introduce three auxiliary metrics: rectification success rate (RSR), false positive rate (FPR), and false negative rate (FNR). Specifically, RSR quantifies the proportion of corrupted labels that are accurately rectified. FPR reflects the fraction of legitimate labels that are incorrectly rectified, and FNR captures the percentage of polluted labels that are not properly rectified. Table 3 presents the RSRs, FPRs, and FNRs against SLF and DLF across different datasets. The results demonstrate the effectiveness of the ACDG stage in FedIDM, particularly in its ability to identify and rectify polluted labels while maintaining the integrity of legitimate ones.

5

Dataset

Attack

RSR

FPR

FNR

CIFAR-10

SLF DLF

0.712 0.841

0.003 0.007

0.288 0.159

CIFAR-100

SLF DLF

0.763 0.971

0.037 0.004

0.237 0.029

Tiny-20

SLF DLF

0.757 0.927

0.037 0.016

0.243 0.072

Conclusion

In this study, we present FedIDM, a novel Byzantine-robust federated learning (FL) approach that ensures fast and stable convergence while addressing the challenges posed by colluded malicious clients. Specifically, FedIDM employs iterative distribution matching to generate reliable condensed data, facilitating the identification and filtering of abnormal clients. The method involves a two-stage process: (1) attack-tolerant condensed data generation, and (2) robust aggregation with negative contribution-based rejection. FedIDM effectively maintains model utility while ensuring robustness against a range of state-of-the-art Byzantine attacks. Extensive empirical evaluations on three benchmark datasets demonstrate that FedIDM outperforms existing defense mechanisms, effectively mitigating the impact of malicious clients without compromising model utility.

References [Baruch et al., 2019] Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. Advances in Neural Information Processing Systems, 32, 2019. [Blanchard et al., 2017] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems, 30, 2017. [Cao et al., 2021] Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. Fltrust: Byzantine-robust federated learning via trust bootstrapping. In ISOC Network and Distributed System Security Symposium (NDSS), 2021. [Dempster et al., 1977] Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1–22, 1977. [Dong et al., 2023] Caiqin Dong, Jian Weng, Ming Li, JiaNan Liu, Zhiquan Liu, Yudan Cheng, and Shui Yu. Privacy-preserving and byzantine-robust federated learning. IEEE Transactions on Dependable and Secure Computing, 21(2):889–904, 2023. [Fang et al., 2020] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. Local model poisoning attacks to {Byzantine-Robust} federated learning. In 29th USENIX security symposium (USENIX Security 20), pages 1605– 1622, 2020. [Guerraoui et al., 2018] Rachid Guerraoui, Sébastien Rouault, et al. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, pages 3521–3530. PMLR, 2018. [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. [Huang et al., 2024] Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang. Federated learning for generalization, robustness, fairness: A survey and benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. [Jebreel and Domingo-Ferrer, 2023] Najeeb Moharram Jebreel and Josep Domingo-Ferrer. Fl-defender: Combating targeted attacks in federated learning. Knowledge-Based Systems, 260:110178, 2023. [Jebreel et al., 2024] Najeeb Moharram Jebreel, Josep Domingo-Ferrer, David Sánchez, and Alberto BlancoJusticia. Lfighter: Defending against the label-flipping attack in federated learning. Neural Networks, 170:111– 126, 2024. [Jiang et al., 2023] Yifeng Jiang, Weiwen Zhang, and Yanxi Chen. Data quality detection mechanism against label flipping attacks in federated learning. IEEE Transactions on Information Forensics and Security, 18:1625–1637, 2023.

[Kairouz et al., 2021] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and trends® in machine learning, 14(1–2):1–210, 2021. [Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. [Le and Yang, 2015] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. [Li et al., 2021] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713–10722, 2021. [Li et al., 2023] Shenghui Li, Edith C-H Ngai, and Thiemo Voigt. An experimental study of byzantine-robust aggregation schemes in federated learning. IEEE Transactions on Big Data, 2023. [Liu et al., 2024] Yi Liu, Cong Wang, and Xingliang Yuan. Badsampler: Harnessing the power of catastrophic forgetting to poison byzantine-robust federated learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1944–1955, 2024. [Lyu et al., 2022] Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, and S Yu Philip. Privacy and robustness in federated learning: Attacks and defenses. IEEE transactions on neural networks and learning systems, 2022. [McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017. [Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. [Park et al., 2023] Sungwon Park, Sungwon Han, Fangzhao Wu, Sundong Kim, Bin Zhu, Xing Xie, and Meeyoung Cha. Feddefender: Client-side attack-tolerant federated learning. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 1850–1861, 2023. [Shejwalkar and Houmansadr, 2021] Virat Shejwalkar and Amir Houmansadr. Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning. In NDSS, 2021. [Shejwalkar et al., 2022] Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, and Daniel Ramage. Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1354–1371. IEEE, 2022. [Wan et al., 2023] Wei Wan, Shengshan Hu, Minghui Li, Jianrong Lu, Longling Zhang, Leo Yu Zhang, and Hai Jin.

A four-pronged defense against byzantine attacks in federated learning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7394–7402, 2023. [Wang et al., 2020] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. In International Conference on Learning Representations, 2020. [Wang et al., 2022] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196–12205, 2022. [Xiong et al., 2023] Yuanhao Xiong, Ruochen Wang, Minhao Cheng, Felix Yu, and Cho-Jui Hsieh. Feddm: Iterative distribution matching for communication-efficient federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16323–16332, 2023. [Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019. [Yin et al., 2018] Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pages 5650– 5659. Pmlr, 2018. [Zhang et al., 2018] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018. [Zhang et al., 2022] Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2545–2555, 2022. [Zhao and Bilen, 2023] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514–6523, 2023. [Zhao et al., 2022] Bo Zhao, Peng Sun, Tao Wang, and Keyu Jiang. Fedinv: Byzantine-robust federated learning by inversing local model updates. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9171–9179, 2022. [Zhao et al., 2023] Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Improved distribution matching for dataset condensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7856–7865, 2023. [Zhao et al., 2024] Puning Zhao, Fei Yu, and Zhiguo Wan. A huber loss minimization approach to byzantine robust

federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21806–21814, 2024.